HDRPARSE
Sections
NAME
hdrparse - Generates header files from raw ingested data
SYNOPSIS
hdrparse [parameters...] filename
PARAMETERS
| Command Line | Resource | Default | Description |
| -h | help | No | Lists basic help information. |
| -df=filename | default | .wxpdef | Sets the name of the resource file. |
| -na=name | name | hdrparse | Specifies the name used in resource file parsing. |
| -ba | batch | No | Run program in batch mode |
| -me=level | message | out2 | Specifies level of messages to be displayed
|
| -fp=filepath | file_path | current directory | Specifies location of database files. |
| -dp=datapath | data_path | current directory | Specifies the location (path) of the input raw data files. This is the location where the ingest program has saved the data files. This may be modified in the name convention file. |
| -nc=name_conv | name_conv | name_conv | The name convention file specifies how files are named in WXP. This sets which name convention file to use. |
| -if=in_file | in_file | raw | Specifies the input file name tag. The default is raw, but it can be modified to any value in the filename convention file to reflect the type of file to parse. A full name convention can be specified as well. |
| -cu=[hour|la] | current | None | This specifies to use current data files. The current filename is based on the name convention. An optional hour can be specified for older data. If la is specified, the program will search back to find the most recent available file. |
| -ho=hour | hour | None | This resource specifies the exact hour that a data file is valid for. This locks in the start hour for a multi-file sequence. |
| -nh=num_hour | num_hour | 0 | This specifies the number of hours that will be searched for hourly data. |
| -pa=param[,param...] | parameter | None | Extra parameters (none currently) |
| filename[#seq] | filename | None User Pompt Batch: current=la |
The name of the raw data file to be converted. An optional sequence number can be added to designate the time for non-WXP files. |
DESCRIPTION
When parsing data, most of the data is processed a line at a time. This means that searching for a particular product in a file can consume tremendous amounts of time. To simplify the process, there are header files which list the product headers in a separate file along with byte offsets into the actual ingested data file. This file then becomes a lookup table for specific products. The use of a header file speeds up searching for products by nearly an order of magnitude and is recommended for LDM ingested data. Since the LDM does not create header files directly, the hdrparse program can be used to post process the LDM data and create the header files.
To run hdrparse, the file name convention file must be set up with a header file name syntax:
for_dat %D/%h%m%d%y.for %D/hd%h%m%d.for sev_dat %D/%h%m%d%y.sev %D/hd%h%m%d.sev sum_dat %D/%h%m%d%y.sum %D/hd%h%m%d.sum cli_dat %D/%h%m%d%y.cli %D/hd%h%m%d.cli
For each listed file type, there is the name convention for the ingested file plus a second listing which is the name convention of the header file. If the second name convention is omitted, WXP assumes there are no header files for this particular type of file.
Once the name convention file is set up to use header files, run header parse on the type of file:
hdrparse -cu=la -if=for_dat
and this will generate the header file. The header output will appear on the screen. The num_hour resource can be used to create header files over several hours.
Once done, programs like parse, forecast and fouswx will run faster since the lookup table has been created and quick lookup can occur. This also will minimize network access over an NFS drive since the small header files are used for the bulk of the searching rather than a line by line search through the ingested data file.
NOTE: This program does not work for GRIB files. Use griblook to generate header files for GRIB data.
FILES
SEE ALSO
Last updated Jun 12, 1997