Verifying Wind Direction

In this section, we'll verify wind direction by running the Point-Stat and Stat-Analysis tools. MET does not verify wind direction directly. Instead, it does so using the vector partial sums (VL1L2) output of Point-Stat and Grid-Stat. When running Point-Stat and Grid-Stat, you may select one or more wind speed thresholds to determine which U/V pairs are included in the VL1L2 lines. You then use Stat-Analysis to compute errors in wind direction for your VL1L2 data aggregated over your verification region.

First, we'll configure and run Point-Stat:

cp scripts/config/PointStatConfig tutorial/config/PointStatConfig_tutorial_run3

Open up the tutorial/config/PointStatConfig_tutorial_run3 Point-Stat configuration file and edit is as follows:

• Set fcst_wind_thresh[] = [ "ge0.0", "ge2.5", "ge5.0" ];
  To select wind speed thresholds of >=0.0, >=2.5, and >=5.0 m/s.
• Set output_prefix = "run3";
  To customize the output file names for this run.

Next, run Point-Stat by executing the following:

bin/point_stat \
data/sample_fcst/2007033000/nam.t00z.awip1236.tm00.20070330.grb \
tutorial/out/pb2nc/tutorial_pb_run1.nc \
tutorial/config/PointStatConfig_tutorial_run3 \
-outdir tutorial/out/point_stat \
-v 2

Open up the Point-Stat VL1L2 output file with the text editor of your choice:

tutorial/out/point_stat/point_stat_run3_360000L_20070331_120000V_vl1l2.txt

Make note of the following:

• Look in the FCST_THRESH and OBS_THRESH columns to see the wind speed thresholds applied.
• Look in the TOTAL column for the number of U/V pairs included in the partial sums. Does the count increase or decrease for higher thresholds?

Close that file, and we'll proceed to run some Stat-Analysis jobs on the command line to look at the errors in wind direction. Execute the following commands:

bin/stat_analysis \
-lookin tutorial/out/point_stat/point_stat_run3_360000L_20070331_120000V.stat -v 2 \
-job aggregate_stat -line_type VL1L2 -out_line_type WDIR \
-vx_mask DTC165 -interp_mthd UW_MEAN -interp_pnts 1 -fcst_thresh ">=0.000"

bin/stat_analysis \
-lookin tutorial/out/point_stat/point_stat_run3_360000L_20070331_120000V.stat -v 2 \
-job aggregate_stat -line_type VL1L2 -out_line_type WDIR \
-vx_mask DTC166 -interp_mthd UW_MEAN -interp_pnts 1 -fcst_thresh ">=0.000"

Each of the two jobs above run on one line each. They both look at the wind direction error for the nearest neighbor method (UW_MEAN/1) using the wind speed threshold of >=0.000. The first job is run for DTC165, while the second is run for DTC166. Look at the ME, for Mean Error, column in the output. Do they differ for the two regions?

Next, run the same jobs again, but this time using the wind speed threshold of 2.5 m/s. Execute the following commands:

bin/stat_analysis \
-lookin tutorial/out/point_stat/point_stat_run3_360000L_20070331_120000V.stat -v 2 \
-job aggregate_stat -line_type VL1L2 -out_line_type WDIR \
-vx_mask DTC165 -interp_mthd UW_MEAN -interp_pnts 1 -fcst_thresh ">=2.500"

bin/stat_analysis \
-lookin tutorial/out/point_stat/point_stat_run3_360000L_20070331_120000V.stat -v 2 \
-job aggregate_stat -line_type VL1L2 -out_line_type WDIR \
-vx_mask DTC166 -interp_mthd UW_MEAN -interp_pnts 1 -fcst_thresh ">=2.500"

Have the wind direction errors improved or gotten worse for the higher wind speed threshold? Lastly, let's aggregate together the DTC165 and DTC166 regions and see how the results change:

bin/stat_analysis \
-lookin tutorial/out/point_stat/point_stat_run3_360000L_20070331_120000V.stat -v 2 \
-job aggregate_stat -line_type VL1L2 -out_line_type WDIR \
-vx_mask DTC165 -vx_mask DTC166 -interp_mthd UW_MEAN -interp_pnts 1 -fcst_thresh ">=2.500"

The output of each of these jobs contains 3 lines, a header line followed by two computations of wind direction error. In all the previous jobs, these two computations yielded the same results since they were operating on a single VL1L2 line. However, this last job is run over 2 VL1L2 lines and the wind direction error results differ. Here is a short description of how these wind direction errors are computed:

• For the ROW_MEAN_WDIR line, the wind direction error is computed for each of the VL1L2 lines the job reads. Then, the overall wind direction error is defined as the mean of those individual errors. Therefore, each of the VL1L2 lines is given equal weight in the total error. The TOTAL column contains a value of 2, since this mean was computed over 2 lines.
• For the AGGR_WDIR line, the VL1L2 partial sums are aggregated together into a total VL1L2 count. The process of aggregation weights each VL1L2 line by the number of points in its partial sum. For the aggregated VL1L2 partial, the mean forecast and observed wind directions are computed, and the overall wind direction error is defined as their difference.

Lastly, try rerunning some of the jobs above, but do so using the other interpolation methods in the VL1L2 output. For example, -interp_mthd DW_MEAN -interp_pnts 9 or -interp_mthd MEDIAN -interp_pnts 9. Do the interpolation methods affect the results much?