The DTC was established in 2003 as a multi-agency effort with funding from National Oceanic and Atmospheric Administration (NOAA), the US Air Force, and the National Center For Atmospheric Research (NCAR) and has made its mark as the “clearing-house” for testing and evaluation (T&E) activities within the meteorological and associated Earth science community. As such, it provides a fundamental bridge between research and operations where cutting-edge ideas can be explored and vetted to enhance understanding of NWP physical processes and improve forecast verification with various techniques and metrics. The DTC's mission priorities overlap with its fellow NOAA testbeds, as they are unique collaborative spaces where researchers and forecasters work together to improve weather products and services (https://wpo.noaa.gov/testbeds/). Thus, the DTC’s role and involvement in partner NOAA testbeds have broadened for more than a decade.
The DTC’s participation in NOAA’s Hazardous Weather Testbed (HWT) Spring Forecasting Experiments (SFE) happened to coincide with my role as a facilitator of the SFE in 2010 and 2011 while working at the NOAA/NWS Storm Prediction Center (SPC). The DTC’s role was to conduct objective verification of the experimental model forecasts and provide results as feedback to the participants in the evaluation process (Clark et al. 2012; https://doi.org/10.1175/BAMS-D-11-00040.1). For this evaluation, DTC applied its locally developed Model Evaluation Tools (MET). Traditionally, the SFE examines experimental high-resolution (convection-allowing) models and ensembles that can explicitly simulate convective mode, and provide details about potential hazards (tornadoes, severe hail, or strong-straight line winds). While the storm-attribute fields often appear realistic, conventional grid-point verification methods routinely penalize high-resolution forecasts when small offsets in time and space exist between observed and forecast event objects. As a result, more emphasis has been placed on spatial techniques that avoid the inherent “double penalty problem” (i.e., the standard verification metrics produce both a miss and false alarm for what subjectively appears a reasonable forecast). Under this framework, a more reliable practice of object-oriented verification was applied by the DTC by applying MET’s Method for Object-based Diagnostic Evaluation (MODE). Some results from the 2010 SFE using MODE are available in Clark et al. (2012). Alternatively, starting with the 2012 SFE, neighborhood-type evaluations also tended to be a popular choice with the participants, as the statistics (e.g., Fractions Skill Score) were made available in near real-time the next day, along with spatial plots on dedicated webpages (Melick et al. 2012; https://www.spc.noaa.gov/publications/melick/hwt2012.pdf).
MET was developed over several years, continually maintained, and expanded by DTC, and has served the academic, government, and private sectors, as well as the international community. DTC provides a consistent code repository for MET with robust documentation, training, and support, which are always improving. These capabilities and resources are essential for sustaining both research and operational purposes. In 2017, I made the transition to becoming a meteorologist for the 16th Weather Squadron (within the 557th Weather Wing) at Offutt Air Force Base, Nebraska. An extensive collaborative history exists between the Air Force and DTC as they’ve funded the DTC on specific T&E projects related to improving understanding weather phenomena and ultimately, environmental intelligence on a global level. Some of the noteworthy byproducts from these investigations have been incorporated into the MET software upgrades. In the past, MODE had been used by the 16th Weather Squadron for case-study evaluations, although not on a routine basis. The adoption of the remainder of the MET tools into an operational context for objective verification has been gradual over the last five years. Our experience with DTC has been very encouraging as there is often a quick turnaround when troubleshooting problems with the software suite and providing solutions (as well as identifying shortcomings that are often addressed in subsequent releases).
The proper infusion of science into T&E activities at DTC has been guided on an annual basis through review, evaluation, and recommendations from the DTC Science Advisory Board (SAB). The SAB convenes experts from all fields in the society that can help to shape the strategic direction and objectives for DTC. For my second year on the DTC SAB, I was honored to serve as one of the chairs during the Fall of 2022. During the three days of meetings in September and October, my responsibilities included guiding SAB discussions and giving a presentation on my career insights on the DTC, which included offering suggestions on limitations in the field that could be addressed in the future. Finally, with feedback from other members on the board, I oversaw the development of our final report delivering constructive critique and advice to DTC. I am extremely grateful on both a personal and professional level to have this experience and I view it as rewarding to the US Air Force to serve on their behalf and promote their interests and goals, with respect to weather challenges.
