MET Online Tutorial for METv3.0

Ensemble-Stat Tool: Output

The output of Ensemble-Stat is a NetCDF file containing the derived ensemble fields, one or more ASCII files containing statistics summarizing the verification performed, and a NetCDF file containing the gridded matched pairs. In this example, the output is written to the tutorial/out/ensemble_stat directory as we requested on the command line.

All of the ASCII statistics output is written to the file ending in .stat. While other MET statistics tools write many output line types, Ensemble-Stat currently only writes out two, RHIST and ORANK. The RHIST line type contains counts for a ranked histogram. The ORANK line type is similar to the matched pair (MPR) output of Point-Stat. For each point observation value, one ORANK line is written out containing the observation value, it's rank, and the corresponding ensemble values for that point.

The STAT file contains all the ASCII output while the _rhist.txt and _orank.txt files contain the same data but sorted by line type. Since so much data can be written for the ORANK line type, we recommend disabling the output of the optional text file using the output_flag parameter in the configuration file.

arrow Since the lines of data in these ASCII file are so long, we strongly recommend configuring your text editor to NOT use dynamic word wrapping. The files will be much easier to read that way.

Open up the tutorial/out/ensemble_stat/ensemble_stat_20100101_120000V_rhist.txt RHIST file using the text editor of your choice and note the following:

  • There are 6 lines in this output file resulting from using 3 verification regions (FULL, NWC, and SWC) and two observations datasets (ADPSFC point observations and gridded observations).
  • Each line contains columns for the observations ranks (RANK_#), the continuous ranked probability score (CRPS), and the ignorance score (IGN).
  • There is output for 7 ranks - since we verified a 6-member ensemble, there are 7 possible ranks the observation values could attain.

Close this file, and open up the tutorial/out/ensemble_stat/ensemble_stat_20100101_120000V_orank.txt ORANK file, and note the following:

  • This file contains close to 2000 lines, 1 for each observation value falling inside each verification region (vx_mask).
  • Each line contains columns for the observation location and value, it's rank, and the 6 values for the ensemble members at that point.
  • When there are ties, Ensemble-Stat randomly assigns a rank from all the possible choices. This can be seen in the SWC masking region where all of the observed values are 0 and the ensemble forecasts are 0 as well. Ensemble-Stat randomly assigns a rank between 1 and 7.

Close this file, and use the ncview utility to view the NetCDF ensemble fields file:

    ncview tutorial/out/ensemble_stat/ensemble_stat_20100101_120000V_ens.nc

This file contains variables for the following:

  1. Ensemble Mean
  2. Ensemble Standard Deviation
  3. Ensemble Mean minus 1 Standard Deviation
  4. Ensemble Mean plus 1 Standard Deviation
  5. Ensemble Minimum
  6. Ensemble Maximum
  7. Ensemble Range
  8. Ensemble Valid Data Count
  9. Ensemble Relative Frequency (for 3 thresholds)

The output of any of these summary fields may be disabled using the output_flag parameter in the configuration file.

Use the ncview utility to view the NetCDF gridded observation rank file:

    ncview tutorial/out/ensemble_stat/ensemble_stat_20100101_120000V_orank.nc

This file is only created when you've verified using gridded observations and have requested its output using the output_flag parameter in the configuration file. Click through the variables in this file. Note that for each of the three verification areas (FULL, NWC, and SWC) this file contains 4 variables:

  1. The gridded observation value
  2. The observation rank
  3. The probability integral transform
  4. The ensemble valid data count

Close this file.