Setting up WXP Data Sources

WXP can run from either sample data or a live data stream. Sample data can be downloaded from the WXP web site:

http://wxp.vietorweather.com/sample

and download the data from the specific data directories (see local system setup for directory information).

Setting up the Ingestor

If live data is needed, WXP can access data from data feeds such as NOAAPort. WXP is ideally set up to access data from the LDM (Local Data Manager from Unidata). WXP will read LDM files directly but file naming is critical for easy access of data.

File Naming Conventions

WXP would like to see data distributed into directories based on date and data type. Typically, this means encoded data in a data directory, human readable advisories, warnings, forecasts in a text directory, model data in a model directory, and so on. In those directories, the files should be named as such:

{year}{month}{day}_{hour}{min}_{subtype}.{format}

The data should be organized by date, put in a subdirectory such as "20260921".

Examples of this would be:

Encoded raw METAR data	/data/noaaport/data/sao/20260921/20260921_21_sao.wmo
Severe weather warnings	/data/noaaport/text/sev/20260921/20260921_21_sev.wmo
NIDS data	nids/DIX/0612201432_n0r.nid
NAM model data	model/06122012_nam.grb
GOES east satellite images	sat/0612201415_gncir.sat

WXP has a name convention file that can be modified to use nearly any naming convention. The default setup is to use this type of naming convention.

Decoding Data

WXP does need to decode certain types of data prior to use. This includes:

METAR/SYNOP data - This reformats the data into a common format so that a single program can access both data types.
Upper air data - This collapses the TTAA, TTBB, PPAA, etc products down into a single report for each upper air site. This is done for convenience.
RAD(MDR)/RCM data. This creates a mosaic grid of data for all reporting sites as well as extracting pertinent data from each encoded product.

There are scripts available to do this decoding available in the scripts directory on the web site. These include saoconvert, synconvert, uaconvert, radconvert and rcmconvert.

Cron setup for automatic decoding

The cron program in Unix can be used to schedule decode scripts to run on a routine basis.

Data type	Script	Times
METAR hourlies	saoconvert	5,10,15,20,30,45 after each hour
SYNOP data	synconvert	1:30,2:30,7:30,8:30,13:30,14:30,19:30,20:30 UTC
Upper air data	upaconvert	1:30,3:30,13:30,15:30 UTC
RAD/MDR data	radconvert	55 after each hour
RCM data	rcmconvert	29,59 after each hour

Here is a sample cron setup:

> crontab -l

0,5,10,15,20,30,45   * * * * /home/wxp/scripts/saoconvert

30 1,2,7,8,13,14,19,20 * * * /home/wxp/scripts/synconvert

30
1,3,13,15           *
* * /home/wxp/scripts/upaconvert

55
*                  
* * * /home/wxp/scripts/radconvert

29,59
*               
* * * /home/wxp/scripts/rcmconvert

Its important that these be set up to run relative to Universal time (UTC or Z time). If the computer clock is on local time rather than UTC, then appropriate changes need to be made to these times relative to offset to UTC time and daylight savings time. The easiest solution to this is to run the computer on UTC time rather than local time.

Go to next step: Setting Initial WXP Resource

For further information about WXP, email devo@ks.unisys.com
Last updated by Dan Vietor, March 2007