Forecast Model Files

GRIB Data

GRIB is a compressed grid file format which is used heavily for the transmission of gridded data. The HRS data feed uses this format almost exclusively. The grid decoder module in WXP decodes these grids on the fly so there is no need for an intermediate format.

The standard method for storing GRIB data is to put all grids from a single model into one file. For example, the ETA model has nearly 1500 grids which are available on HRS. WXP uses a standard search procedure to find grids in these files. Since all the information passed in GRIB files are numeric, there needs to be a lookup table for actual parameters such as 500 mb and temperature.  Most of the GRIB parameters for forecast time, level and parameter/variable are defined in WXP (see the time, level and variable resources).

Since GRIB data can be corrupted, it is necessary to check for bad data when decoding a GRIB product. Since data is compressed from a normal 4 bytes per gridpoint to 6-12 bits per gridpoint, when a byte is missed, it can cause very bad values from one gridpoint to the next. If the difference between one gridpoint to the next is more than half the possible range (6 bits=64 range, bad if >32), it is considered a bad datapoint. If this is a consistent feature, the rest of the grid is flagged as bad. The default is 5 bad gridpoints in 20.  This can be changed with the max_bad=num plot parameter resource.

Model Lookup File

The model lookup file is a way for WXP to determine which grids go with which model. In some cases, more than one model file is associated with the model or more than one grid type within a model. Each line of the file represents another model type that is specified .  The syntax is:

   alias  geometry  tag[#type]

The alias is a keyword that is specified with the model resource. The geometry is the way the grids fit together when pieced together. A value of 1 means there is only one grid type with the mode. A value of XxY represents a segmented model. For example, 4x2 means there are 8 grid types to be pieced together. The list of grids follow with the first 4 going into the top row from left to right and the second 4 in the bottom row. The list of grids is generally a name convention tag that is referenced in the name convention file for the exact filename. An additional grid type can be specified if a specific grid type needs to be extracted from the file.

Here is a sample:

eta       1   grib_eta#211
eta_ak    1   grib_eta#207
ngm       1   grib_ngm#211
avn_us    1   grib_avn_us
avn       4x2 grib_avn_n1w,grib_avn_n0w,grib_avn_n0e,grib_avn_n1e,grib_avn_s1w,g
rib_avn_s0w,grib_avn_s0e,grib_avn_s1e
avn_nhem  4x1 grib_avn_n1w,grib_avn_n0w,grib_avn_n0e,grib_avn_n1e
avn_shem  4x1 grib_avn_s1w,grib_avn_s0w,grib_avn_s0e,grib_avn_s1e
avn_whem  2x2 grib_avn_n1w,grib_avn_n0w,grib_avn_s1w,grib_avn_s0w
avn_ehem  2x2 grib_avn_n0e,grib_avn_n1e,grib_avn_s0e,grib_avn_s1e
avn_na    2x1 grib_avn_n1w,grib_avn_n0w
avn_nae   3x1 grib_avn_n1w,grib_avn_n0w,grib_avn_n0e
avn_asia  2x1 grib_avn_n0e,grib_avn_n1e
avn_eur   3x1 grib_avn_n0w,grib_avn_n0e,grib_avn_n1e
avn_sam   2x1 grib_avn_s0w,grib_avn_s0e
mrf       1x2 grib_mrf_nh,grib_mrf_sh
mrf_nhem  1   grib_mrf_nh
mrf_shem  1   grib_mrf_sh                                         

The ETA model has two grid types which are in the same file and must be separated. The AVN model has 8 subgrids which need to be pieced together. There are several aliases for the AVN model each specifying a different combination of the grids to be pieced together. In other words, only those grids that need to be pieced together ought to be specified. Any grids above and beyond this represents wasted CPU time trying to find the grids.