Filenames
Filenames used in WXP can be specified in several ways. Often, it is left to a prompt in the program but can be specified on the command line or by specifying the resource in the resource file or via environment variables. The syntax of the filename is defined by the file name convention that affects how the file name is specified. There are several resources that can be used to specify the filename:
- filename - this resource specifies the actual filename. This is either prompted for in the program or a positional parameter specified at the end of the command line.
- in_file - this resource specifies the input file name convention. This can be done by specifying a tag that is found in the file name convention file or as an actual file name with wildcard characters.
- out_file - this resource is similar to the in_file resource except it is for output files.
- current - this resource specifies the offset from the current time to use in determining the filename. This has a current and a latest component. The latest component will search a set number of hours for an existing file.
- hour - this resource specifies a set time during the day such as 12Z for a particular filename.
- num_hour - this resource is used to specify a range of files to use. This can be used for generating loops and animations or for decoding data that is spread over multiple files.
Direct Filename
When a direct filename is specified either on the command line, it takes precedence over any of the other filename specifications. The filename can be specified in a number of ways:
- filename - this is the filename. The path as specified in the name convention is automatically prepended to the filename. Make sure to specify the name convention if it is not the default for that program.
- /path/filename - this is the fully pathed filename. Name convention is applied to determine date from filename.
- @/path/filename - this is a full filename but the name convention is overridden.
- #sequence_number - this is a number in the format "yyyymmddhhnn" which exactly specifies the time to use and the filename convention is then used to produce the appropriate filename for that time. In some cases, the shell will treat "#" as the start of a comment so a "\" may need to preceed the "#".
In each case, the program attempts to extract a date/time from the filename using the file name convention. This often is used to label data and plots. This can be used along with the num_hour resource to produce loops. If the filename does not match the naming convention, incorrect date information will be extracted. This could result in a file not being found or a loop incorrectly generating. Using the "@" character disables name convention parsing.
Current Filenames
Current data files use the current date/time to construct a filename. The name convention must have a date element in order to take advantage of the current feature. Current data files can be specified either by using the current resource (-cu) or by specifying it with the filename resource. When using the filename resource, "cu" or "la" can be used to specify the current file . Everything following these characters is passed to the current file algorithm.
The value of the current resource is actually an offset from the current time. The simplest ways to specify the offset are as follows:
- -cu - uses the current data file no matter what. If the current data file does not exist, the program will exit.
- -cu=1 - uses the data file that is one hour old.
- -cu=3d04 - uses the data file that is 3 days, 4 hours old.
- -cu=0:15 - uses the data file that is 15 minutes old.
Latest Filenames
If the current file is not available, the user may want to just use the most current file or latest file. If "-cu=la" is specified, the program will search back through the most recent 500 times, create a filename and check to see if it exists. If the file name convention specifies a file every 12 hours as with upper air data, it will first check the current time, if the file does not exist, it check for the file that would be 12 hours old, and so on. If the file name convention specifies a file every 5 minutes as with NIDS and NOWRad data, it will search back in 5 minute increments.
Now the absolute offset and the latest can be combined. If you wanted the first available data file that is at least 5 hours old, you would specify "-cu=5-la". If you want the file that is 5 hours previous to the latest, you would specify "-cu=la-5". You can have forward searches as in "-cu=3d+la" or "-cu=la+2". This would find the latest and search forward 2 hours. This will be discussed more in the next section.
Here is the complete syntax of the current resource. Any of the parts can be omitted.
[#yyyymmddhhnn][+-##d##:##][+-la][+-##d##:##]
#yyyymmddhhnn - sequence number
##d##:## - number of days, hours, minutes (all are optional)
la - find latest in each direction
Specifying Specific Hours
The hour resource can be used in combination with the current resource to request specific hours. For example, if the latest 12Z file is needed, use "-cu=la -ho=12". The hour resource can be used to specify specific days with the syntax "ddhh". For example, to use the 20Z file on the 14th, specify "-cu=la -ho=1420". To get the latest file on a 12 hour boundary, specify a negative hour. For example, "-cu=la -ho=-12" would yield 12 at 18Z and 00 at 4Z. This can be taken one step farther with an offset. If the first hour after a 12 hour boundary is needed (01 or 13), use the offset "-oohh". The offset "oo" is added to the hour after it is rounded to the nearest boundary. For example, "-cu=la -ho=-106" would yield 13 at 15Z, 19 at 20Z, etc.
Specifying a Range of Hours and Looping
This is useful in a couple of cases. First, in decoding data, the data could be located in several files over a range of hours. To decode all data for a specific time, several files must be parsed. To do this, use the num_hour resource. For example, to decode the previous hours data but also using the current file to get all the reports decoded, use:
sfcdec -cu=1 -nh=1
The number of hours specified represents the requested file plus one hour. Thus two hours of data files are decoded.
This can be used with plotting programs to set up loops.
sfcplot -cu=la -nh=-5 -va=all
In this case, a loop of 6 hours, current plus the 5 previous hours, will be generated. The number can represent hours and minutes such as "-nh=-3:20" in which the program will loop through all files within the previous 3 hours and 20 minutes. This is useful for radar data which comes in every 5 minutes. The programs will delete any frames where the data file does not exist.
In addition to specifying the number of hours, a skip factor can be specified. This is useful if not every hour is needed. For example, a plot of 21Z temperatures is needed over 4 days. This can be done as follows:
sfcplot -cu=la -ho=21 -nh=-96,24 -va=temp ...
Again, the skip factor can be minutes "-nh=-2,0:30" which will display every file over the previous 2 hours with a 30 minute increment.
Multiple Filenames and Looping
In many programs, there is the ability to list more than one filename on the command line. Decode programs use the list of files to process data. The decode time is the time of the first file listed. The other files are decoded in the order they appear in order to extract data for that decode time.
For plotting programs, the list of files is used to generate a loop. The files are processed in the order listed with the resulting plot inserted into the loop. Any nonexistent data files are not used in the loop.
Decode a Range of Hours
The decode programs need to know the hour to decode to eliminate data which is too far from the requested time. By default, the decoder uses the specified time from either the filename or current resources. Sometimes this is not the time requested for deocding. A good example of this is upper air data. In some cases, upper air data for a particular hour is located in 3 to 4 hourly files (11,12,13 and 14Z). To deocde this range, the following can be used "upadec -cu=la -ho=-12" but this will use 12Z as the decode hour and of course this will decode only a handful of data. The num_hour resource solves this problem, by specifying a range to of hour files to decode. The syntax is "-nh=min-max".The correct way to decode this data is:
upadec -cu=la -ho=-12 -nh=-1-2
Again, the "-12" on the decode hour specifies to use the nearest 12 hourly file. If the number is positive, it would only decode 12Z data and not 00Z data. the num_hour specified as "-1-2" specifies a range from one hour before the decode time to two hours after.
Also with upper air and synoptic data, there needs to be a way of including data from surrounding hours. By default, if 12Z synoptic data is decoded, syndec will only decode 12Z reports. The problem is that some places like Australia report at off hours such as 11 and 23Z and thus these reports would not be included. The "-pa=range=pre:post" specifies the time range to search for and decode.
Sequence Numbers and wxpcurtime Environment Variable
WXP offers the concept of a sequence number that is an absolute time specification that is irrelevant of the file naming convention. If you wanted the data file from 12Z 22 Dec 2020, you would specify:
-cu=#202012221200
The format of the sequence number is 4 digit year, 2 digit month, 2 digit day, 2 digit hour and 2 digit hour. Any value omitted is assumed to be 0 so you could have left off the last two zeros.
Now we can offset from this time:
-cu=#2020122112-la
which will start at this time and search backward for the first available file.
If you want this sequence number to be the default starting point and don't want to continue to specify it in the current resource, you can set the wxpcurtime environment variable. Since this can only be a sequence number, the number sign "#" can be omitted:
setenv wxpcurtime 1996122112
This is a nice feature when dealing with case studies and being able to use the current resource with old data. This environment variable would be set to be the most important time of the case, for example, when the tornadoes are occurring. Then the students can just use -cu to reference that time without worrying about the file naming convention. Also, forward and backwards time offsets from the reference time can be by specifying the current resource.
Updated January 2021