Data Ingest: PAN (Product Arrival Notices) Messages

Product arrival notices are sent at the completion of a product to a specified PAN receiving program.  The PAN receiver will use this message to trigger an action based on the arrival of that product.  For example, a PAN receiver might be interested in the arrival of severe thunderstorm warning messages so it can warn the user.  The PAN message is broadcast over a socket using a UDP transmission.  This is a connectionless process where the PAN is sent to a specific address and port and it is up to the PAN receiver to be active and waiting for the message using a receive from call.

The PAN message is sent as a single line of information for each product received by the ingestor.  The information in the PAN message is broken up into fields delimited by a bar "|":

ID|Server|###|YYYYMMDDhhmmss|WMO/Extra|Filename|Offset|Size

Fields:

• ID -- Message Type ID (901 for NOAAPORT)
• Server --  NOAAPORT server number which uniquely identifies server (0-99)
• ### -- Sequence number from server (0-999). Increments by one for each PAN sent from that server. It cycles through numbers 0 to 999 and back to 0.
• YYYYMMDDhhmmss -- Timestamp of when product is sent from WXP ingestor.
• WMO/Extra -- WMO Header plus additional information. For text products, this is the first 20 bytes of the product (newlines and unprintables changed to spaces). This will often contain the AWIPS header. For GRIB products, this is the decoded header information from the GRIB Product Definition Block. Included is the model number (PDS Octet 6), grid type (PDS Octet 7), valid/forecast time (derived from PDS Octet 19,20 and 21), vertical level type (PDS Octet 10), level value (PDS Octet 11-12) and parameter (PDS Octet 9).
• Filename -- Filename including full path.  This is the filename that the WXP ingestor saved the product to.  NOTE: This filename and path may be different from the filename and path you need to access the data.  If the data is mounted on an NFS drive, the appropriate NFS path will need to be substituted for the path listed here.
• Offset -- Byte offset of product header in file.  This is the exact location (first byte in file is 0) of the start of the product header.  An fseek using this number is all that is needed to locate the product.
• Size -- Size in bytes of product from header to end of product including any leading or trailing blank lines

Examples:

901|45|909|19980428152512|SDXX99 KWBC 281522 / RCMFWS|/home/wxp/data/98042815_rad.wmo|75975|2410| 

1. 901 - identifies NOAAPORT PAN message
2. 45 - identifies local NOAAPORT server
3. 909 - is the sequence number
4. 19980428152512 - Date product arrived on server and PAN message sent (depends on server time). It arrived at 15:25:12Z on 24 APR 1998
5. SDXX99 KWBC 281522 - WMO header
   RCMFWS - AWIPS header
6. /home/unisys/wxp/data/98042815_rad.wmo - server filename where product is located. Each file can contain more than one product
7. 75975 - byte offset in file
8. 2410 - product size in bytes

901|45|907|19980428152510|YSRG98 KWBD 281200 PAA / 89 212 4030036 1 0 66|/home/wxp/model/98042812_eta2.grb|3412593|32943|

1. 901
2. 45 - identifies local NOAAPORT server
3. 907 - is the sequence number
4. 19980428152510 - Date product arrived on server and PAN message sent
5. YSRG98 KWBD 281200 PAA - WMO header
   89 212 4030036 1 0 66 - Extra GRIB info, model 89 is Eta model, grid 212 is AWIPS grid 212, time 4030036 is a 6 hour accumulation from forecast hour 30 to 36, level type 1 is surface, level 0 is ignored for surface and parameter 66 is snow depth.
6. /home/unisys/wxp/model/98042812_eta2.grb
7. 3412593 - byte offset in file
8. 32943 - product size in bytes

PAN Message Setup

To set up the WXP ingestor for PAN messages the following pieces of information must be added to the "ingest.bul" file. At some point in the file, a PAN configuration line must be added.

# PAN Setup
@PAN id=45 sock:steve:5566 sock:dev5:5000 pan.log

The "@PAN" is a keyword in the bulletin file for the PAN configuration line. The "id=45" specifies the NOAAPORT unique server ID which is broadcast as field 2 in the PAN message. The rest of the line lists destinations. The "sock" keyword specifies the PAN go over a UDP socket. The string "steve:5566" is the network name of the destination computer and the TCP/IP port number. If the sock keyword is omitted, the PAN is save to the listed filename such as "pan.log". Up to 10 destinations can be listed. Each destination is addressed starting with 0 and going to 9 in the order listed on the PAN line.

By default, no PAN messages are sent even if the PAN line is added to the bulletin file. To enable PAN messages, the "P" flag must be added to the action for each product being saved on the server. For example a product line would look like:

# Pattern Action Filename             Header Filename
FT          >>   %D/%y%m%d%h_term.wmo %D/%y%m%d%h_term.hdr

To enable this product type for PAN messages, add the "P" flag to the action.

FT         P>>   %D/%y%m%d%h_term.wmo %D/%y%m%d%h_term.hdr

This will send a PAN message to all listed destinations whenever this products is received. If you don't want to send a PAN to all destinations, the destination IDs can be listed:

FT      P035>>   %D/%y%m%d%h_term.wmo %D/%y%m%d%h_term.hdr

In this case, PAN messages will only be sent to the 0, 3 and 5th destinations.

For further information about WXP, email devo@ks.unisys.com
Last updated by Dan Vietor on May 30, 1998