WXP Overview

The Weather Processor can be logically divided into three data processing sections: ingest, decoding and analysis. Each of these logical data processing divisions is treated in some detail in subsequent sections.

Data

Most data analysis packages don't work well with the raw data. They require conversion of that data into a format the programs can manage. WXP is different in that it will work with the raw data whenever possible. WXP can handle a wide variety of data formats:

• Surface Observations - METAR, SAO, SYNOP, SHIP, DRIBU, CMAN
• Upper Air Data - TEMP (from balloons)
• Radar - MDR, RCM, NIDS (single site) Level 2 and 3, NSSL, Kavouras, WSI and Unisys radar mosaics
• Satellite - McIDAS AREA, GINI (from NOAAPort), Unisys Satellite
• Lightning - USPLN, NLDN, Albany, Unisys
• Grid - GRIB ver 1 and 2
• Miscellaneous - Redbook, Fronts (analysis and forecast), Watches

WXP 6 has a set of programs that processes each type of data. Unlike WXP version 5, each program only works with a single type of data. The sources of these data can be from almost anywhere as long as they match the format specifications. WXP is fairly flexible in using data ingested from programs like the Unidata LDM (Local Data Manager) or directly from sources like the NWS FTP servers. The decoders account for headers and other pieces of data that might also be in a file

Data Ingest

WXP typically uses data in real-time. To determine the date and time of the data, WXP uses files which have a time-stamp in the filename. Typically, the data type and format are also part of the file name, This is referred to as the name convention of the file.

To set up the files with the proper data and name convention, a data ingestor is used. This selects data from a data stream by a type selector, which is normally a WMO header and saves files based on the time embedded in the data or by the time the data were processed by the ingestor.

WXP comes with it's own data ingest program which can be used with the NOAAPort data feed from the NWS. It can ingest any data that uses the WMO data format. WXP can also use data from the Unidata LDM. A sample pqact.conf file is available that works with the default WXP naming convention.

Programs

• wmoingest - The WMO data ingestor
• Array - A Program to remove older ingested data files

Data Parsing and Decoding

Once the data has been saved to disk by the ingest process, it is ready for processing. In some cases, the data are in a format where parsing it is appropriate. By parsing, we refer to selecting a specific product from all the ingested data files. This could be a local forecast, a weather advisory or climate report.

In other cases, the data are encoded and require some degree of preprocessing before they can be used effectively by other programs. There are several decoders available to handle various types of data. In some cases, the encoded data is too complicated to decode every time the data is needed. Often data of similar types like surface and synoptic data are decoded and put into a common format. Then there are cases where the data for one site is split up into several products and it is desirable to collate the data into a single data structure. Also, decoders do some limited quality control and error checking on the data. The new decoded formats also provide a way to get non-standard data processed by converting data into these formats.

Decoders Programs

• sfcdec - The surface data decoder (SAO, METAR, SPECI, SYNOP, BUOY, DRIBU, CMAN)
• upadec - The upper air data decoder
• mdrdec - The MDR radar data decoder
• rcmdec - The RCM radar data decoder
• hurdec - Hurricane advisories decoder

Parsing Programs

• wmoparse - The general purpose parsing program
• sfctext - The surface data parsing program
• upatext - The upper air data parsing program
• mostext - The MOS data parsing program
• gribfile - The GRIB data viewing program

Data Analysis and Plotting

The application programs available within WXP can produce a variety of output products. These programs analyze many of the standard meteorological data types including surface, upper air, radar, satellite, model output statistics, and model gridpoint data. The analysis programs can plot derived variables such as relative humidity, lifted index and vorticity. The output of these programs can either be plotted data such as a temperature plot or a skew T or it can be contoured plots of gridded fields. The output devices range from X windows to Postscript printing and metafiles.

Plotting Programs

• sfcplot - Plots surface data
• upaplot - Plots upper air data
• soundplot - Plots soundings (Skew Ts) from upper air and GRIB data
• upxplot - Plots upper air cross sections
• radplot - Plots radar data from MDR, RCM, NIDS data as well as NOWRad, Kavouras, Unisys radar mosaic data
• mosplot - Plots MOS data
• gridplot - Contours model gridpoint data (GRIB files)
• satplot - Displays satellite images
• lightplot - Plots lightning data
• profplot - Plots profiler data
• meteoplot - Plots time series of surface and MOS data
• frontplot - Plots fontal data
• watchplot - Plots severe weather watch boxes
• redbookplot - Plots redbook graphics files
• gridcont - Contours existing grids
• vectplot - Plots vectors and streamlines from existing grids
• rawplot - Plots raw data
• mapplot - Plots maps and station locations

Overlay and Looping Programs

These applications provide overlay, annotation and animation capabilities. There are two programs: plot and figure. The plot program is a basic data overlay program. This can be used to create simple overlay plots for web graphics. The figure program links all the different plot types together into a full overlay and looping program. Both programs work with the various plotting objects to provide a wide range of overlays and loops.

Programs

• plot - The basic overlay program
• figure - The extended ovelay and looping program

Meteorological Calculations

These programs are provided in order to calculate some simple meteorological variables and indices.

Programs

• rawmath - Performs math operations on raw files
• gridmath - Performs math operations on grid files
• suncalc - Sunrise/set computations
• domain- Domain/projection computation program

General Purpose Programs

These programs are used to interface with the other WXP programs.

Programs

• nameconv - Generates file names
• wxpres - Queries WXP resources

Updated January 2021