The Weather Processor can be logically divided into three data processing sections: ingest, decoding and analysis. Each of these logical data processing divisions is treated in some detail in subsequent sections.
Most data analysis packages don't work well with the raw data. They require conversion of that data into a format the programs can manage. WXP is different in that it will work with the raw data whenever possible. WXP can handle a wide variety of data formats:
- Surface Observations - METAR, SAO, SYNOP, SHIP, DRIBU, CMAN
- Upper Air Data - TEMP (from balloons)
- Radar - MDR, RCM, NIDS (single site) Level 2 and 3, NSSL, Kavouras, WSI and Unisys radar mosaics
- Satellite - McIDAS AREA, GINI (from NOAAPort), Unisys Satellite
- Lightning - USPLN, NLDN, Albany, Unisys
- Grid - GRIB ver 1 and 2
- Miscellaneous - Redbook, Fronts (analysis and forecast), Watches
WXP 6 has a set of programs that processes each type of data. Unlike WXP version 5, each program only works with a single type of data. The sources of these data can be from almost anywhere as long as they match the format specifications. WXP is fairly flexible in using data ingested from programs like the Unidata LDM (Local Data Manager) or directly from sources like the NWS FTP servers. The decoders account for headers and other pieces of data that might also be in a file
WXP typically uses data in real-time. To determine the date and time of the data, WXP uses files which have a time-stamp in the filename. Typically, the data type and format are also part of the file name, This is referred to as the name convention of the file.
To set up the files with the proper data and name convention, a data ingestor is used. This selects data from a data stream by a type selector, which is normally a WMO header and saves files based on the time embedded in the data or by the time the data were processed by the ingestor.
WXP comes with it's own data ingest program which can be used with the NOAAPort data feed from the NWS. It can ingest any data that uses the WMO data format. WXP can also use data from the Unidata LDM. A sample pqact.conf file is available that works with the default WXP naming convention.
Data Parsing and Decoding
Once the data has been saved to disk by the ingest process, it is ready for processing. In some cases, the data are in a format where parsing it is appropriate. By parsing, we refer to selecting a specific product from all the ingested data files. This could be a local forecast, a weather advisory or climate report.
In other cases, the data are encoded and require some degree of preprocessing before they can be used effectively by other programs. There are several decoders available to handle various types of data. In some cases, the encoded data is too complicated to decode every time the data is needed. Often data of similar types like surface and synoptic data are decoded and put into a common format. Then there are cases where the data for one site is split up into several products and it is desirable to collate the data into a single data structure. Also, decoders do some limited quality control and error checking on the data. The new decoded formats also provide a way to get non-standard data processed by converting data into these formats.
- sfcdec - The surface data decoder (SAO, METAR, SPECI, SYNOP, BUOY, DRIBU, CMAN)
- upadec - The upper air data decoder
- mdrdec - The MDR radar data decoder
- rcmdec - The RCM radar data decoder
- hurdec - Hurricane advisories decoder
- wmoparse - The general purpose parsing program
- sfctext - The surface data parsing program
- upatext - The upper air data parsing program
- mostext - The MOS data parsing program
- gribfile - The GRIB data viewing program
Data Analysis and Plotting
The application programs available within WXP can produce a variety of output products. These programs analyze many of the standard meteorological data types including surface, upper air, radar, satellite, model output statistics, and model gridpoint data. The analysis programs can plot derived variables such as relative humidity, lifted index and vorticity. The output of these programs can either be plotted data such as a temperature plot or a skew T or it can be contoured plots of gridded fields. The output devices range from X windows to Postscript printing and metafiles.
- sfcplot - Plots surface data
- upaplot - Plots upper air data
- soundplot - Plots soundings (Skew Ts) from upper air and GRIB data
- upxplot - Plots upper air cross sections
- radplot - Plots radar data from MDR, RCM, NIDS data as well as NOWRad, Kavouras, Unisys radar mosaic data
- mosplot - Plots MOS data
- gridplot - Contours model gridpoint data (GRIB files)
- satplot - Displays satellite images
- lightplot - Plots lightning data
- profplot - Plots profiler data
- meteoplot - Plots time series of surface and MOS data
- frontplot - Plots fontal data
- watchplot - Plots severe weather watch boxes
- redbookplot - Plots redbook graphics files
- gridcont - Contours existing grids
- vectplot - Plots vectors and streamlines from existing grids
- rawplot - Plots raw data
- mapplot - Plots maps and station locations
Overlay and Looping Programs
These applications provide overlay, annotation and animation capabilities.
There are two programs: plot and figure. The plot program is a basic data
overlay program. This can be used to create simple overlay plots for web
graphics. The figure program links all the different plot types together into
a full overlay and looping program. Both programs work with the various
plotting objects to provide a wide range of overlays and loops.
These programs are provided in order to calculate some simple meteorological variables and indices.
- rawmath - Performs math operations on raw files
- gridmath - Performs math operations on grid files
- suncalc - Sunrise/set computations
- domain- Domain/projection computation program
General Purpose Programs
These programs are used to interface with the other WXP programs.
Updated January 2021