System Administration Overview
In earlier versions of WXP, there was little ability to tailor the input and output of WXP. For example the file naming conventions were hard coded into WXP. If the files did not fit the naming convention, it often didn't work well with WXP. Another example was the use of variables where the units were hard coded and there was no way to derive new variables. This lack of tailorability made WXP easy to use but limited its capabilities. With WXP 5, most of limitations have been removed. The result is a set of configuration files to tailor everything from input to output. This makes WXP somewhat harder to use. WXP comes with a default configuration that may not need to be changed. If there is a need to change the configuration, this section will describe each of the configuration files and what may need to be changed. To set WXP up, there are a few configuration files that need to be checked.
Files that likely need changing
WXP Resource File
The first is the WXP resource file. This is critical since some base resources drive all WXP programs. There are a couple of critical resources to set in the resource file:
- file_path -- This tells every WXP program where the database and configuration files are located. This usually points to a directory such as "/home/wxp/etc" or "/usr/local/etc". It is important to install all the configuration and database files in this directory.
- data_path -- This tells WXP where the ingested raw data files are located. This is usually a directory like "/home/wxp/data".
- con_path -- This tells WXP where all the converted data files are located. This is usually a directory like "/home/wxp/convert".
- raw_path -- This tells WXP where all the raw data files produced by WXP are saved. This is usually a directory like "/home/wxp/raw".
- grid_path -- This tells WXP where all the grid files produced by WXP are saved. This is usually a directory like "/home/wxp/grid".
- image_path -- This tells WXP where all the images produced by WXP are saved. This is usually a directory like "/home/wxp/image".
- name_conv -- This tells WXP which file name convention to use. This is the name of a name convention file saved in the file_path directory. There are 3 base name convention files to choose from: ymdh, hmdy, and the default ymdh_t or name_conv. The first two are provided for those sites that require 8.3 file names (old DOS standard).
The resource file location can be anywhere but is usually in the file_path directory. The location is specified by the wxpdefault environment variable.
setenv wxpdefault /home/wxp/etc
This is normally added to either the .cshrc file or to the systems environment file for system wide use. For more on the setting of resources, see the section of the Users Guide on the resource file.
Ingest Bulletin File
The selection of products and the output filenames are controlled by the ingest bulletin file. WXP uses a standard selection scheme and output filename convention. The default bulletin file can be used but in many cases, the output filenames and grouping of products may need to be changed. The bulletin file syntax is:
header [action] [command/filename...] [header file] header [action] [command/filename...] [header file] ...
The header is a regular expression pattern to make WMO headers. The action is how the product matching the header is processed. Actions include file, append, pipe to a program and run a command when the product completes. The command/filename specifies which command to run in the case of the pipe and run actions or the filename to save the product to. This filename can contain wildcard characters which define the date and time of arrival as well as components of the header. This way, files can be named based on when the product arrives. Generally, headers are sorted by type and appended to files with the name based on the date the product arrived and the type of data. Here is a sample of a bulletin file:
# Pattern Action Filename Header Filename # S[AP] >>-15 %D/%y%m%d%h_sao.wmo S[IMNS] >>-05 %D/%y%m%d%h_syn.wmo SD >>+07 %D/%y%m%d%h_rad.wmo U[^AB] >>-65 %D/%y%m%d%12h_upa.wmo ASUS1_ >> %D/%y%m%d%3h_frt.wmo WWUS40 >> %D/%y%m%d%6h_wws.wmo FO >> %D/%y%m%d%12h_mod.wmo %D/%y%m%d%12h_mod.hdr A >> %D/%y%m%d%6h_sum.wmo %D/%y%m%d%6h_sum.hdr C >> %D/%y%m%d%6h_cli.wmo %D/%y%m%d%6h_cli.hdr W >> %D/%y%m%d%6h_sev.wmo %D/%y%m%d%6h_sev.hdr F[^O] >> %D/%y%m%d%6h_for.wmo %D/%y%m%d%6h_for.hdr
For example, all SA bulletins go into an hourly file named year, month, day, hour with an extension of "_sao.wmo". There is a 15 minute offset to get early products. All products starting with "W" go into a 6 hourly file with the extension "_sev.wmo". All of these files go into the "%D" or data_path directory. In addition, header files are generated for the severe weather files to aid in parsing.
The default bulletin file is "ingest.bul" and is located in the file_path directory. For more information on the use of the bulletin file, see the ingest program section of the Users Guide.
NOTE: For those sites using the LDM ingestor, the pqact.conf file serves as the bulletin file. Refer to the users guide for the LDM for more information. For those sites using a different ingestor, refer to the users guide for the setup of those ingestors. For WXP to work optimally, the files should be laid out similarly.
Name Convention Files
Once the data has been saved to disk for local use, the rest of WXP can be used to decode, parse, manipulate or display the data. The name convention file is set up to emulate the files generated by the WXP ingestor. If this is the case, the default name convention file "name_conv" need not be changed. In some cases, there will be data ingested by other programs and the files saved into a different naming convention. The name convention file allows the WXP programs to handle these files. A good example of this is NLDN, NIDS and NOWRad data that WXP cannot ingest at this time. There is also the case of files, FTPed from other sources that may have strange naming conventions. Each WXP program uses a file name tag that is searched for in the name convention file for a match. This is then converted into a name convention often based on the date and type of file. For example surface data has the following tags:
sfc_dat -> sacvt
-> sfc_cvt, sfc_cvt_wxp (WXP format), sfc_cvt_cdf (netCDF
format).
sfc_cvt -> sfcwx -> sfc_raw
(raw output), sfc_grd (grid output), sfc_grd_wxp (WXP
grid output), sfc_grd_cdf (netCDF grid output)
These name conventions are then set in the name convention file:
sfc_dat %D/%y%m%d%h_sao.wmosfc_cvt_cdf %C/%y%m%d%h_sao.ncsfc_cvt %C/%y%m%d%h_sao.wxp sfc_raw %R/%y%m%d%h-%v_sao.rawsfc_grd_cdf %G/%y%m%d%h-%v_sao.nc sfc_grd %G/%y%m%d%h-%v_sao.grd
Not all of these need to be set for each type. In general only the "sfc_dat" and "sfc_cvt" need to be set. These tags can be changed using the in_file and out_file resources. To get more information on use of the name convention file, see the name conventions section of the Users Guide.
Generating Header Files
The WXP ingestor automatically generates header files for those files listed in the bulletin file. If another ingestor generates the files, header files may need to be generated for these files. To do this, use the hdrparse and griblook programs. For more information, see the parsing data and header files section of the Users Guide.
Automatic Conversion, File Generation and Scouring
As a final note for setup, the computer will need to be set up to automatically convert data, generate files and scour ingested data. This is done using the cron task in Unix. The following tasks need to be setup:
- convert surface data once an hour
- convert radar data once an hour
- convert upper air data once every 12 hours
- generate header files for non-WXP ingested data on a regular basis
- scour ingested data
- scour converted data
For more information on these tasks, see the automatic data conversion and scouring section of the Users Guide.
Files that likely don't need modification
Parse Lookup File
For most WXP program, they can find data files by use of the name convention tag. For parsing, there is an additional method that simplifies the process. When the parse program parses for a WMO header, it searches the parse lookup file "parse.lup" to cross reference headers to name convention tags. This eliminates the need to specify the in_file file name tag. The syntax of the file is:
pattern tag
The pattern can be a regular expression. It returns the tag of the first match in the file. That name convention is used to setup the filenames to search for the desired product. For more information on the parse lookup file, see the parsing data and header files section of the Users Guide.
Model Lookup File
For gridded data from the forecast models, information is broadcast on a set of grids. Often these grids are saved into output files based on the model and grid type. In some cases, several types of grids are broadcast from a single model. For example, the Eta model has grids that cover the US (211), Alaska (207). In other cases, the grid is broken into subgrids, as is the case with the aviation model. The global grids are broken up into 8 subgrids which must be pieced together to produce the final global grid. In addition, similar to the parse loopkup table, there needs to be a mapping between model type (model resource) and the file name tags. The "model.lup" file serves these purposes. The syntax of the model lookup file is:
model geometry tags[#grid]...
The model is the name of the model used in the model resource (-mo=eta). The piece section defines how grids are to be pieced together. The tags[#grid]... is a list of file name tags to use for the filenames. If the grid is specified, only those grids matching this grid number are parsed from the file. Here is a sample:
eta 1 grib_eta#211 eta_ak 1 grib_eta#207 ngm 1 grib_ngm#211 avn 4x2 grib_avn_n1w,grib_avn_n0w,grib_avn_n0e,grib_avn_n1e,grib_avn_s1w,grib_avn_s0w,grib_avn_s0e,grib_avn_s1e mrf 1x2 grib_mrf_n,grib_mrf_s
For more information, see the forecast model files section of the Users Guide.
Variable Files
Variable files define how variables will be derived and plotted by the WXP analysis programs. There is a variable file set up for each WXP analysis program. This allows each program to be setup differently, offering a different user interface. The variable file syntax is as follows:
abbrev name toggle definition
- abbreviation - This define an abbreviation which is used to define the variable for use in the variable resource.
- name - this is the text to be used in the menu listing. No spaces are allowed but underscores "_" can be used in their place.
- toggle - this is a menu toggle which defines when the menu item will appear in the variable menu of the program.
- definition -- This defines which variable will be plotted and how it is
to be plotted. The syntax of the definition is:
time:level:variable:units:attributes...The time, level and variable are used to search for the variable in the input data file. In some cases, time and level can be ignored. The units is used to define the output units and attributes define how the variable is to be plotted. The definition can be a series of these definitions for overlays. The variable can be a function of other variables in order to derive new variables.
For more information on variable files, see the variable files section of the Users Guide.
Units File
WXP uses a units lookup file "units.lup" to define how to convert from one set of units to another. This file has most of the standard units conversions but new ones can be added it needed. The syntax is:
in out conversion
where in is the input units, out is the output units and conversion is the conversion between the two units. For example, the conversion from C to F is "*1.8+32". For more information on units, see the units conversion section of the Users Guide.
Time and Level Menu Files
These files define which times and levels will appear in the forecast time and vertical level menus in appropriate WXP programs. These files are similar to the variable files except there is no need for the definition section. The file contains only the abbreviation, name and toggle sections. For more information on these files, see the time and level menu files section of the Users Guide.
Region Files
WXP uses a default region file called "wxp.reg". In this file, there is a list of all regions that can be used in WXP and their appropriate plot_domain specifications. Often, the default set of regions does not satisfy the users needs so new regions can be added to this file if needed. The syntax of this file is similar to the variable and menu files:
abbr name toggle plot_domain
This file also controls which regions will show up in the region menu for appropriate WXP programs. For more information on region files, see the database section of the Users Guide.
GRIB Lookup Files
There are two GRIB lookup files that may need to be changed. GRIB parameters often change to reflect new models and variables. The model lookup table is "mod_name.lup". This is a cross reference between model number in the GRIB file and a label to be used on the output. For example model number 89 is the Eta model. The second lookup file pertains to GRIB variables or parameters. This file is "variable.lup". This cross references the variable number, its abbreviation used in the variable files (see above) and a label to use on the plots. An example of a line from this file is:
11 temp Temperature
GRIB variable 11 maps to temperature. Any variable used in WXP that has the abbreviation "temp" will be given the label "Temperature".
Color Table Files
Color table files give a cross reference between a color name and its RGB values for display. This allows colors to be specified by name for easier usability. Colors from these color tables can be used with any color resource or as part of the color fill listing of contour colors. Several color table files are provided to use in different situations but new color tables can be created to serve a particular purpose. The syntax of the file is:
[index] name red green blue
The name is a string (no spaces) and the RGB values are floating point numbers between 0 and 1. To use a color table file, just specify it with the color_table resource. For more information on color table files, see the graphics colors section of the Users Guide.
Color Fill and Enhancement Files
These files list a set of colors to be used in color fill contours. Not only can the file contain a list of colors, but it also can specify attributes for files and line contours as well as contour levels for non-uniform contour intervals. The user may wish to define their own color fill or enhancement schemes and then specify the file with the color_fill resource. For more information on color table files, see the graphics colors section of the Users Guide.
Symbol Files
These files are used to plot various meteorological symbols. The file contains a mapping between a meteorological code such as "RW-" and a set of line and fill commands to draw the appropriate symbol. In general, these files are not modified but at times there will be the need to define new symbols.
Font List Files
This files define a list of fonts, either WXP, X windows or Windows fonts to use in the programs. The file contains a mapping between a font alias and a full font name and size. If the fonts are WXP fonts, they are defined in WXP font files.
Font Files
These files define a set of WXP fonts to use in the display of text on plots. The file contains a mapping between a printable ASCII character and a set of line commands to draw the character.
WXP Shell Menu File
This file defines the menus to show in the WXP shell. These can be changed to fit individual needs. For more information on WXP shell menu files, see the WXP shell menu files section of the Users Guide.
Case Study Setup Files
This file defines the case studies available to the WXP shell. These can be changed to fit individual needs. For more information on case studies, see the case study setup section of the Users Guide.
Database files that may need to be updated
City Database Files
City database files are critical for plotting since they define where stations reports are located. For the most part, these files are up to date but there are times that these file need to be updated. There are two reasons to change a city database file:
- add or delete a station from the file
- change a stations priority to affects the priority filter for plotting
City Raw Files
These are raw files that are used with the wxpcity program. They include: cities.raw, counties.raw, zones.raw, states.raw. These will need to updated to reflect changes to zones and forecast office responsibility.
Map Files
Generally, these files do not need to be modified. The DLG map files at times will need to updated to reflect changes to roads.
For further information about WXP, email devo@ks.unisys.com
Last updated by Dan Vietor on June 8, 1998