WRF Executive Oversight Board (ExOB)
Meeting 5 (2006-1)
NCAR, Boulder, CO
Attendees: Tim Killeen (NCAR), John Lanicci (AFWA),
Robert Winokur (N7C), Louis Uccellini (NCEP), Steve Koch (ESRL, formerly FSL),
Gloria Kulesa (FAA), Pamela Clark (ARL), Simon Chang (NRL), Nelson Seaman,
Robert Gall, Mark Surmeier, David Knapp, Mike Clancy, Richard Wagoner, Steve
Payne, Greg Holland, Joe Klemp, Rich Hodur.
1. Briefing 1: Welcoming Remarks and Introduction:
As host, Dr. Tim Killeen welcomed meeting attendees to NCAR. In his opening remarks, Dr. Killeen gave a brief overview of the WRF program and its importance to the nation. He emphasized NCAR’s continuing commitment to WRF.
2. Briefing 2: Meeting Objectives
Dr. Seaman (PC) introduced the primary meeting objectives, emphasizing transformation of WRF into a formal program. Specific objectives included developing a consensus on what steps are necessary to carry out that transformation and what the scope of a fully functional DTC should look like. Consensus in these areas lays a foundation for establishing shared resourcing and a management plan to sustain the WRF program. These will largely define criteria for participation in the proposed program.
Dr. Bob Winokur said that the fundamental question facing WRF is, “Do we want to transform WRF into a program?” That needs to be answered before we delve into what is necessary to bring about that transformation.
Dr. Killeen asked what differentiates a WRF program from a WRF project?
Dr. Seaman replied that before deciding whether we want to transform WRF into a program, it is necessary to define what is meant by a program and what existing problems a program can address. To facilitate discussion the term “project” is used to describe the current situation in which resources for common WRF projects are insecure and based on whatever certain WRF partners are able to make available on a year-to-year basis. Participating agencies currently have no sustained budget line items identified as “WRF”. Consequently, DTC cannot perform multi-year planning or adequately address many requests for WRF support made by the operations and research communities. Thus, the structural shortfalls of the WRF project are inadequate provision for resourcing and management of common activities. These shortfalls often result in costly duplications of effort, gaps in much-needed support capabilities, and increased risk for WRF development at individual agencies to become incompatible with one another. Later briefings from this meeting further explain these difficulties and approaches proposed to address them.
The term “program” is used to describe a vision for WRF in which a community-guided center (the DTC) establishes and executes community-wide WRF support activities and is sustained through a shared resource and management plan. The DTC thereby complements the separate in-house WRF modeling activities of individual WRF partners, while overall coordination and direction of WRF remains the responsibility of the WRF ExOB.
3. Briefing 3: WRF Program Status
Dr. Seaman began the WRF program status briefing by summarizing key accomplishments of the WRF program over the past 5 months. These include:
The WRF program status summary for Phase 1 showed that no additional metrics of progress (tracked tasks) have been completed during the past 5 months. Two metrics have been downgraded from “on track” to “delayed or hold.” These are:
· Goal 1b threshold metric downgraded to “hold”: “TOR for the distributed DTC written and signed.” A draft DTC TOR was approved by ExOB at its Aug. 2005 meeting, but subsequent consideration of WRF program reorganization makes it necessary to revisit the TOR once a reorganization plan is established.
· Goal 2 threshold metric downgraded to “hold”: “One core supported with grid nudging.” Most coding necessary to establish this capability is finished, but funds provided by EPA have been exhausted. Absence of funding places task completion at risk, hence the downgrade of metric status. Dr. Klemp indicated that release of this capability is now expected in June 2006. Readiness of obs-nudging for release is uncertain.
In addition to the above changes, the following significant modifications are noted:
· Goal 1a objective metric delayed: “Full core interoperability established with release of 3 cores to the community, all operating under WRF-ESMF as a single reference code.” NRL has delayed its estimate for readiness of a WRF-ESMF system supporting 3 cores (including COAMPS) to FY10 (previously listed as FY07).
·
Goal 1b threshold metric redefined: “Write proposal for a DTC node at
NCEP/EMC.” NCEP and DTC have entered discussions to develop a plan for
establishing a new node of DTC at NCEP’s R&D center,
· Goal 1b objective metric redefined: “Establish functional DTC node at NCEP/EMC.” This metric replaces the original plan for a functioning “stand-alone” NCEP OTC.
Meanwhile, although not an official metric, NCEP announced that implementation of WRF-NMM in its North American Mesoscale (NAM) domain will be delayed by one quarter to Q3/FY06. AFWA remains on track for its planned implementation of WRF-ARW in its classified windows in Q2/FY06.
· Discussion on Grid Nesting, Physics Interoperability and the WRF CMI:
Dr. Louis Uccellini began by expressing concern over difficulties in getting the one-way nesting capability for WRF-NMM released to the community. This opened a discussion from the floor on whether the WRF CMI can support moving nested grids for different dynamic cores, as intended.
Dr. Bob Gall indicated the nesting capability is available, but has not been installed into the WRF code repository, which is a prerequisite for release as supported (reference) code. Conflicting priorities and deadlines leading up to the next WRF-NMM tutorial and the pending implementation of WRF-NMM in the NAM domain at NCEP have made it difficult to complete all steps necessary to release the one-way nesting version of WRF-NMM.
Dr. Killeen asked whether there was any particular difficulty getting the nested version of WRF-NMM working inside the WRF CMI, since no problems are encountered with the two-way nested version of WRF-ARW? Dr. Seaman replied that the CMI did not appear to create any extraordinary problems peculiar to WRF-NMM. NCEP is coordinating with NCAR to install the movable nested-grid version of WRF-NMM following the NMM tutorial in February and two-way nesting is under development.
Dr. Uccellini indicated that there remain problems that prevent WRF from running fully “plug and play” physics options for both cores. Dr. Killeen said that the ExOB needs to understand why running plug-and-play physics isn’t already established.
Dr. Greg Holland replied that it’s important not to be confused whether the obstacle to physics interoperability is an internal issue in the physics modules themselves or an external issue related to the CMI software. The CMI does not prevent interoperability. However, installation of new physics parameterizations often require modification of those schemes to operate with other physics or with a different core because variables need to be passed back and forth in a consistent way (units, scaling, etc.).
·
Discussion
on WRF and ESMF CMI Issues
Dr. Seaman summarized the work carried out by NCAR/MMM to extend the WRF CMI so that it can be run as an ESMF component. This task has been conducted as part of Goal 1 under Phase 2 of the WRF project established by the ExOB to “modify the existing WRF Software Infrastructure to enhance its functionality, usability and extensibility in sync with an ESMF end state.” This initial task has been completed by MMM, to the extent currently possible. Final testing by actually running under ESMF awaits further development of ESMF to allow it to support ARW’s map projections, scheduled for Spring 2006.
Dr. Seaman went on to explain that enabling WRF to be run as
a component of ESMF fulfills only the first phase of Goal 1. The next step involves conducting a technical
workshop of software developers and expert users to evaluate the feasibility of
further convergence of the WRF and ESMF codes.
The workshop has been scheduled for
Dr. Uccellini asked what is meant by “streamlining” the CMI”? Does this refer to making it easier to use, or combining WRF with ESMF?
Dr. Seaman replied that “streamlining” refers to the ExOB’s goal of modifying the CMI to make it easier to use and support, and to make it more efficient computationally (especially for codes other than ARW), while maintaining the range of functionalities most important for research and operations. The ExOB also has directed that the CMI be made compatible with an ESMF end state.
Mike Clancy stated that there are significant issues to be addressed concerning the ease of use of the WRF CMI in operations, particularly its efficiency and maintainability in an operational environment. Dr. Seaman added that addressing these issues should not be interpreted as “dumbing down” the CMI, but finding more effective and efficient approaches to accomplish its objectives, whether by modifying the existing CMI or by adopting useful components that are expected to emerge in ESMF. Dr. Holland agreed these are issues that need to be addressed, but within the limits of the technology. Many researcher users appreciate the ease and reliability with which they can configure model experiments using the existing CMI without being concerned about many of the details of hardware architectures.
Dr. Hodur said that model developers and operational staff may make up only a small fraction of all WRF users, but they should have at least equal representation in changing the WRF CMI.
Dr. Uccellini added that NCEP model developers at EMC have made progress in becoming familiar with the WRF CMI, but it is the engineers at NCEP Central Operations who have to maintain the WRF code in operations around the clock. If problems arise during night or on weekends, NCO cannot wait until the developers are available to ask for help.
On a related topic, Dr. Killeen expressed concern that according to the status briefing COAMPS won’t become interoperable within the CMI until FY10. Is the Navy concerned about this delay?
Dr. Chang replied that NRL is working to establish interoperability between the physics and I/O of COAMPS and the WRF CMI. These steps offer distinct advantages, but there is no identifiable advantage to putting COAMPS dynamical core in the CMI that warrants the considerable investment it would require. Thus, NRL plans to defer core interoperability until ESMF develops further.
Given that, Dr. Killeen asked, what is the degree of Navy’s actual commitment to WRF?
Dr. Hodur said that Navy is interested in supporting community modeling. However, Dr. Hodur asked if there is any advantage to putting COAMPS dynamical core into the infrastructure, especially if interoperability is established with the COAMPS physics and I/O? Mike Clancy added that Navy has to deal with many other types of models, in addition to atmospheric prediction models, so ESMF seems a naturally attractive framework.
Dr. Klemp interjected that it should be in our interest to move toward a more common system, rather than establishing separate systems that reduce opportunities to leverage one another’s developments and increase maintenance burdens. Dr. Seaman added there appears to be increasing pressure at high levels in NOAA, Air Force and Navy to consolidate model systems, rather then continue to support multiple versions, unless specific advantages can be proven.
Dr. Uccellini replied that multi-model ensembles are the way operational centers will provide greater accuracy, address forecast uncertainty and provide probabilistic guidance. Having multiple cores in WRF facilitates this goal for limited area models.
Dr. Chang stated that Navy remains concerned about the impact of the WRF CMI on model runtime. He returned to the issue of the moving nested grids, saying that COAMPS became operational five years ago with nested grids. Had Navy adopted the WRF CMI, it would have resulted in a degradation of model capability and Navy would be using its resources to recode the nested grids for WRF. Dr. Uccellini also expressed disappointment that NCEP had had to invest effort to develop and implement a nested grid capability for WRF-NMM.
Dr. Chang asked if it is possible to extend the Technical
Workshop on WRF-ESMF Convergence beyond the single day currently proposed (
4. Briefing
4: WRF at NCAR
Representing the hosting institution, Dr. Holland gave a presentation on WRF activities at NCAR, an update on WRF-ARW research and plans, and the development of a nested regional climate model based on ARW. WRF development at NCAR is centered in the Mesoscale and Microscale Modeling (MMM) Division, which resides in the Earth-Sun Systems Laboratory (ESSL). ESSL is also home to the Climate and Global Dynamics Division, which is collaborating with MMM on the Nested Regional Climate Model. Other parts of NCAR heavily involved with WRF are the Research Applications Lab and the Institute for the Study of Society and the Environment. Along with supporting the WRF-ARW and the WRF CMI to the atmospheric modeling community, MMM Division also is the developer of community data assimilation techniques and hosts the annual WRF user’s workshop.
Special emphasis was given to recent work in which the WRF-ARW has been applied for the prediction of tropical storms. The model had been applied during the 2005 Atlantic hurricane season using the two-way nested version of ARW, on 12-km and 4-km grids. Initial conditions were based on the NCEP GFS and GFDL models. Evaluation statistics were shown for predicted average track and intensity errors on the 12-km and 4-km domains of WRF-ARW versus several other numerical and statistical prediction systems. The sample ranged from 34 cases to 2 cases, depending on resolution and forecast lead time, out to +120 h. Since the cases were selected to focus on storms threatening landfall within ~3 days, a large majority of cases covered the 0-48 h period, with the totals decreasing thereafter. At both resolutions, the WRF-ARW results in the sample were shown generally to be comparable to or more accurate than those found with other predictive approaches. Dr. Holland characterized these results as very promising and indicated that further model evaluations are necessary. It was urged that DTC conduct evaluations as soon as possible to validate the results against a larger set of cases and to accelerate the work along the path leading to operations.
·
Discussion
on ARW for Hurricanes
Dr. Uccellini cautioned that results in the current round of testing, however promising, is insufficient to adequately answer a number of questions necessary before operational implementation could be considered. In particular, the sample contains too few cases and they were all drawn from one season, whereas multiple seasons must be evaluated to account for very significant year-to-year variability in tropical storm developments. NASA’s finite volume model gives an example of why such thorough testing is necessary. For that model, preliminary results based on a limited number of cases also appeared very promising and its developers anticipated it would perform very well relative to operational prediction systems in more complete multi-season evaluations. Unfortunately, NCEP spent considerable effort conducting those tests, only to learn that the model performed more poorly than other models.
Dr. Uccellini also pointed out that running a model in real time falls far short of meeting the degree of adaptation necessary to support the same model in full operational mode. NCEP is constrained to a very tight time schedule for each model due to the large number of models that must be run in every 6-h cycle.
Dr. Seaman pointed out that this is a prime example of why the WRF process was established, so that promising developments in the research community would have a protocol under which they could be rigorously evaluated and, if successful, accelerated along the path toward operations. The DTC was established (among other reasons) so that at least half of the steps in this process could be conducted outside the operations centers. Only the final steps need to be transferred to the OPCs for site-specific adaptations and final testing, thereby leveraging community resources. This approach saves operational centers from each conducting similar costly early-stage tests that can greatly sap resources needed to advance current operational systems. Dr. Killeen agreed that we need to take advantage of DTC for further evaluating WRF-ARW for hurricanes.
Dr. Payne added that coupling the WRF-ARW hurricane model to an ocean model, plus providing fully cycled data assimilation, will be critical for evaluating the modeling system against operational systems.
Greg Holland and Rich Wagoner concluded with an example showing how the WRF-ARW hurricane predictions have been coupled experimentally with wind-damage models to predict impacts on electrical grids. This sort of impact modeling can be done for a host of applications and increases the value of weather forecasts by relating them to customer-specific risk management.
5. Briefing 5: Briefing on Status of
Dr. Gall began the briefing by presenting notable recent accomplishments of the DTC, giving special attention to results emerging from the DTC Winter Forecast Experiment (DWFE). It was also reported that the 2nd DTC tutorial on the WRF-NMM will be held in February, 2006. The attendees at both tutorials have come from the university community, mission agencies and the private sector, indicating that there is broad interest in exploring this model for research and other applications outside of government.
It was reported that the 2005 DTC visitors program has been quite successful. All but one of the visitors have been from universities, which makes this an important part of WRF’s outreach to the academic community. About half of the visitors selected under the 2005 program will actually visit DTC this year, using funds carried over from last year. However, at this time it appears that there will be no funds available to support a new announcement of opportunity for a 2006 visitors program. NSF has reported that its FY06 funding situation is very tight and it does not expect to provide any support to DTC.
Dr. Gall explained the status of the DTC TOR, which had been negotiated in 2005 between NCAR, NRL and NOAA. The ExOB had endorsed the TOR at its April 2005 meeting and the final step was to have it signed by representatives of the three agencies. However, at that the August meeting the ExOB had initiated a request to develop options through which WRF might be established as a program. Since such a reorganization of WRF would inevitably raise issues about the scope, management and resourcing of DTC, the effort to secure final signatures for the TOR has been suspended. It is anticipated that provisions in the TOR will undergo some modification following WRF’s reorganization. In addition DTC and NCEP/EMC have been holding discussions aimed at establishing a fourth node of the distributed DTC at EMC. Examination of the existing TOR revealed that it would not be difficult to insert modifications to accommodate the EMC node, should those discussions reach maturity.
Dr. Gall proceeded to review the DTC Annual Operating Plan
for FY06. This is the first year that
the newly established DTC Advisory Board participated in prioritizing proposed
DTC tasks. The planning process resulted
in selection of seven projects, based on (1) available human and financial
resources, (2) priorities established by the Advisory Board, (3) priorities
established by the organizations providing DTC resources. One the FY06 tasks will be DTC’s
participation in TREX, the Terrain-induced Rotor Experiment. TREX is notable because both the
The DTC FY06 budget was introduced, showing a projected shortfall of ~$22.5K. Dr. Uccellini reported that the NOAA contribution from USWRP had recently been reduced by $50K due to severe budget reductions at NWS, leading to a revised funding gap at DTC of ~$72.5K. This represents a very serious threat to the viability of the DTC, since the budgeted expenditures are almost entirely directed toward staff salaries.
Dr. Gall mentioned that if additional funds were to be made available for expansion, DTC’s highest priority would be to hire a software engineer who would provide user support for the WRF software infrastructure. At present, DTC does not have staff qualified to provide this service. Dr. Holland said that another priority should be to obtain the necessary support to begin evaluation of ARW for hurricane predictions as soon as possible. Dr. Gall agreed and said the full scope of services that DTC needs to provide to the WRF modeling community will be presented later in Briefing 8 (see below).
6. Briefing 6: WRF Cores and Rapid Refresh WRF Testing
Dr. Steve Koch presented a briefing on DTC’s FY06 task to perform testing and evaluations of the WRF dynamical cores. The immediate objective of these tests will be to support development of the Rapid Refresh WRF (RR-WRF), most of which is being conducted under separate funding by FAA. The DTC’s role through this task will be to isolate the influence of core dynamics on model solutions, particularly for aviation-sensitive weather parameters and other forecast parameters that are a high priority for RR-WRF customers. The DTC cores tests are designed to be as “clean” as possible. This means using model configurations for WRF-ARW and WRF-NMM that are as nearly identical as is feasible, given the inherent differences in the models’ design. Tests will be made using similar horizontal and vertical resolutions as those projected for the initial implementation of RR-WRF (13 km and 50 layers). Experiments will be performed in two phases. In the first phase both cores will use the NNM physics; in the second phase both will use RUC-like physics. Results from the DTC evaluations of WRF cores will be used to select the dynamical core of RR-WRF. By summer of FY06, NCEP’s operational data assimilation system, the Gridpoint Statistical Interpolation (GSI), will be integrated with the prototype RR-WRF as part of the continuing model development at GSD under FAA support.
·
Discussion
on WRF Core Testing
The ExOB discussed the merits and limitations of using
traditional statistical measures of model accuracy (e.g., RMS errors, equitable
threat scores, bias scores) to evaluate mesoscale models. Dr. Killeen said it is well recognized that
these measures do not give a complete assessment of the precipitation skill
produced in mesoscale models, especially as resolutions become finer. Dr. Uccellini agreed, but said that does not
justify ignoring these statistics since they do reveal some important characteristics
of the precipitation forecasts and are officially tracked operational NWS GPRA
(Government Performance and Results Act of 1993) goals. In addition, traditional statistics remain
very valuable for assessing model skill for non-bifurcated (continuous)
variables like temperature and wind.
Col. Lanicci added that AFWA model evaluations include a combination
score, based on a design originated in
Dr. Koch continued by saying the DTC experiments to be conducted under the RR-WRF test protocols will be much better controlled to isolate the impact of the dynamical cores than in any of the major WRF evaluations conducted to date (WRF Test Plan, SPC-NSSL Spring Experiment, and DWFE). Dr. Joe Klemp questioned whether the Phase 1 tests would be relevant since use of NMM’s physics with both cores could give an advantage to the NMM dynamical core and the NMM physics is not expected to be used in RR-WRF in any case. Drs. Gall and Koch explained that in addition to providing extra insights valuable for the development of RR-WRF, the Phase 1 tests are an important step needed to establish the physics interoperability that remains an important WRF goal. Future work can build on Phases 1 and 2 by performing similar tests in which the NCAR physics is applied in the same model configurations for both WRF-ARW and WRF-NMM.
Dr. Seaman pointed out that NCEP is installing GFDL’s tropical physics package in the WRF-NMM as part of its Hurricane-WRF development. If that additional physics also is made compatible for application in WRF-ARW, WRF will have full interoperability among four mature physics suites for use with either dynamical core, all operating under the WRF CMI. NRL’s work to make its COAMPS physics suite interoperable with WRF will add a fifth option. These developments, along with emerging DA capabilities, will make WRF undoubtedly the most versatile modeling system in the world for conducting carefully controllable NWP research and development. Rather than considering whether to limit the RR-WRF testing with different physics, it is important to view these tests in the context of this larger strategic goal. Moreover, we need to have similar controlled experiments performed in the future at finer resolutions where explicit microphysics schemes can be used to predict both convective and non-convective precipitation.
Dr. Holland said that the equitable threat score is meaningless at these finer resolutions. Others replied that while a number of position-oriented statistical scores (“object-oriented” scores) have been developed and tested in recent years, the NWP community has not yet developed a consensus on their value or how to interpret them. Thus, we have few practical alternatives at this time.
7. Briefing 7: Transforming WRF from a Project to a Program
Dr. Seaman introduced a draft plan for transforming WRF from
a project to a program. (See Section 2
above for definitions of “project” and “program.”) The draft plan was presented in response to
the ExOB’s Action Item 4.1-2005, stating “options are needed to establish a more firm management plan to sustain the
WRF Program, including resourcing for the WRF DTC.”
The briefing
indicated that the principal deficiencies of the current WRF project are (1)
DTC is insufficiently staffed to provide the range of services demanded by the
WRF community, (2) DTC is insufficiently resourced to provide these services,
and (3) WRF management does not provide sufficient authority to avoid costly
duplications of effort and erratic implementations of its decisions, thus increasing
risks and threatening investments made by the WRF partners.
A set of goals and recommended ExOB actions were presented to mitigate these deficiencies. The most important recommendations were that ExOB should adopt (1) the proposed scope for a DTC that would support the range of services required by users (see Section 8 below) and (2) a management plan that would place greater obligation on the WRF partners to uniformly implement program decisions made by the ExOB. Modifications to various specifics of the draft plan are anticipated before the ExOB can reach final consensus and approval on the proposed WRF program plan.
·
Discussion
on WRF Program Plan
Dr. Winokur expressed concern that the proposed management plan gives the WRF ExOB some control over how each partner’s base funds could be applied.
Dr. Uccellini replied that NSF, NOAA and NASA are already directed through a congressional appropriations committee to work together in developing their spending plans. This is a recognition that some degree of joint control on spending will be required in the future. We are going to have to make changes in the way we do business.
Dr. Winokur expressed appreciation for development of a draft management and resource plan in response to the ExOB’s instructions to the Program Coordinator. However, it was believed that the central issue facing the ExOB is how to fix the resource problem faced by DTC. It was recommended that ExOB defer discussion of individual parts of the proposed plan until all parts have been laid out, including Briefing 8.
Dr. Uccellini said ExOB must keep in mind that the issues we face are broader than DTC resourcing. They include how we approach community model development, transition and implementation in operations. Our involvement in WRF is based on establishing an efficient process that reduces the costs associated with these activities. We should not be making decisions, individually or collectively, that end up hindering development and increasing costs.
8. Briefing 8: What is the Scope of the DTC?
Dr. Gall began the briefing by reviewing the vision, goals, purpose and desired outcomes originally established when DTC was created to serve the WRF community. The role of DTC includes providing a path by which promising new model codes can be rigorously evaluated and transitioned for use by research and operations communities. To demonstrate this code-transition path, a set of eight transition steps was presented that were originally established for development codes entering NCEP. Very similar steps are routinely applied at the other OPCs. That is, the specific tasks are nearly universal, even though they may be described using somewhat different language at individual sites. Thus, perhaps with some rewording, they can be adopted at DTC as a standard model transition plan. As indicated on two subsequent slides provided by NCEP, DTC (and perhaps other testbeds external to operational centers) can perform 5 of the 8 steps leading toward operational code implementation. These steps include (1) selection of promising code, based on community needs and preliminary demonstration of value by the developer, (2) establishing the code’s benefits for forecasting and/or analysis through rigorous testing and evaluation, (3) ensuring efficiency requirements are met, (4) establishing IT compatibility (portability) for target platforms, and (5) ensuring codes are understandable and sustainable for users. Appropriate criteria for satisfying each of these requirements need to be established “up front” for each code entering the transition path. The five steps appropriate for execution at DTC are sufficient to support such codes to the research community as WRF reference code. However, the final three steps (see briefing 8) are specific to a given operational center and must be performed internally by that center. The commonality of the first five represent a major way in which DTC acts as a shared community resource that leverages the investments of the WRF partnership.
To reap the benefits of DTC, it must be staffed and resourced at a level enabling it to provide these services for the range of areas needed by the research and operations communities. Dr. Gall presented a recommendation for staffing needs in specific areas: testing and evaluation, verification development, code transition, WRF documentation and user support, support of the WRF science software and common modeling infrastructure, visiting scientist program, and administration. The plan estimated the total required staff at 24.25 at a cost of ~5.6M/yr. This cost includes $5.1M/yr for DTC staff, plus $0.5M/yr to support temporary visiting scientists.
·
Discussion
on Scope of the DTC
Dr. Uccellini said that, based on experience in developing the JCSDA, the staff and resource estimates proposed for DTC appear reasonable. Dr. Killeen agreed that ~$5M/yr is about the right level and that ExOB should work to make that happen.
Dr. Uccellini added that senior levels at NOAA and NSF have been pressing for this sort of shared center that leverages increasingly scarce resources. At NOAA, Fred Toepfer, Jack Hayes and he have argued to the Deputy Secretary (Adm. Lautenbacher) that WRF will fail unless a fully functional DTC is established.
Dr. Killeen said that it appears the DTC proposal makes a shift of emphasis to include some functions originally expected to be assigned to the OTCs. Mike Clancy also suggested that some functions now proposed for DTC (in the first 5 of the eight steps of code transition to operations) are specific to operational centers and thus are OTC functions.
Dr. Koch said it’s necessary for DTC to have people assigned to handle interface duties with operational centers, not simply those assigned to conduct basic code testing. Dr. Uccellini pointed out that the functional relationship between ESRL/GSD and NCEP/EMC for development and transition of the RUC model and more recently the RR-WRF closely approximates the functions proposed for DTC and demonstrates this can work.
Dr. Chang asked whether the DTC should prioritize all activities and conduct only the highest-priority activities, if full funding was not possible?
Dr. Gall replied that DTC uses that approach now through the DTC Advisory Board. Some of the expanded functions listed, such as user support for the WRF-ARW, are provided currently by NCAR/MMM. These current support points could continue and would not require new funds. In addition he noted that the current budget for the DTC given in his earlier briefing represented the absolute minimum budget for the DTC to pay its present employees.
Dr. Klemp expressed concern that, given the scarcity of resources, merging the DTC and OTC could mean that the firm requirements levied by operations would result in giving highest priority to OTC functions at the expense of DTC’s ability to support the user community.
9. Briefing 9: Executive Session
The executive session focused on how to move ahead with expanding the DTC consistent with the proposed scope and what additional steps are needed to more firmly establish the WRF program.
Col. Lanicci opened the executive session by reminding the ExOB that the WRF program plan must address not only resourcing and management issues, but also should ensure that scientific advancement receives equal emphasis.
Dr. Killeen said that the new DTC plan needs to more clearly define and justify the particular services it will provide. This will help delineate whether each function supports wide portions of the community or must be done separately at a particular operational center and thus would be more suitable for an OTC. The DTC plan should also identify what “lost-opportunity costs” will be incurred by not moving ahead with DTC expansion. So far, the case for the expanded DTC is incomplete.
Dr. Killeen suggested that many elements of the proposed WRF program plan are being done right now. For example, the DTC is overseen by its Advisory Board, which recommends tasks and helps set priorities. This is an important function of WRF program management. Another example is the impressive visitor program, in which the Advisory Board also plays an important role. Perhaps it is only necessary to review what we already have and then make fairly modest adjustments or additions.
Dr. Uccellini said that from a management perspective it is desirable to have EMC participation in DTC management to ensure adequate accountability for establishing and following through on task priorities. It is not acceptable to simply re-label people as performing DTC work, while their actual work is unchanged. This creates the impression of DTC expansion and activity, but there’s actually no added benefit.
Rich Wagoner said that establishing the DTC Advisory Board is very good step. However, a missing first step is to have a body that establishes the requirements for the DTC. Once requirements are defined, prioritization of proposed projects and tasks becomes much easier. Projects are selected and prioritized to satisfy those requirements. In addition there needs to be a periodic review of completed DTC projects to evaluate their success/failure in meeting the requirements
Dr. Winokur said there are be a number of different approaches that can be used to create a WRF program. The MOA route has been suggested by the PC and DTC Director. An alternative would be to recognize that most provisions of the existing AIP are satisfactory, but addition of some new language might be needed to bring “rigor” to program management. This route may be easier to accomplish than an MOA, while recognizing that the issues facing WRF go beyond merely resourcing for the DTC. ExOB needs to reach consensus on the following:
Do we have a resourcing problem?
Do we have a management problem?
Do we have a requirements problem?
Dr. Winokur continued by noting that NOAA, Navy and Air Force will conduct a Tri-Agency Workshop in February to address modeling resources. The workshop scope is broader than WRF, but it is expected to begin by establishing resource baselines that should help address DTC resource issues. Dr. Winokur proposed that the WRF ExOB call another meeting following the Tri-Agency Workshop to specifically address DTC scope, management and resource issues. The ExOB agreed with this suggestion.
Pam Clark said that the upcoming meeting also must identify the benefits to result from an expanded DTC. Dr. Killeen agreed and mentioned ARW hurricane evaluations as an example of how the expanded DTC can be used to accelerate model evaluations and code transition. If DTC results in a new modeling capability being implemented into operations even one year earlier than would be possible otherwise, that represents a tremendous value to the nation. Dr. Uccellini confirmed this, saying that the SPC-NSSL Spring Program is a definite example where an experimental program, based on a cooperative approach involving research and operations, has paid off. (DWFE is another example.)
Ms. Clark also indicated that the WRF partners must define what requirements they have that can be assigned to DTC. Col. Lanicci agreed that, once the requirements are defined, it may be much easier to define the approach needed to transform the WRF program. Dr. Uccellini said that NOAA is much in need of a stronger WRF program, including the expanded DTC.
Glossary of Common WRF Acronyms
AFWA Air Force Weather Agency
AIP Agreement In Principle
AO Announcement of
ARL Army Research Laboratory
AWRP FAA’s Aviation Weather Research Program
BAMEX Bow Echo And MCV Experiment
COAMPS Coupled
Ocean-Atmosphere Mesoscale Prediction SystemTM
COMET Cooperative Program for Operational Meteorology, Education and Training
CONUS CONtinental
COPC Committee for Operational Processing Centers
DTC
DWFE DTC Winter Forecast Experiment
ESMF Earth System Modeling Framework
ESRL Earth System Research Laboratory
ExOB Executive Oversight Board
FSL Forecast Systems Laboratory
FTE Full-Time Equivalent
FNMOC Fleet
Numerical Meteorology and
GFS NCEP Global Forecast System
GPRA Government Performance and Results Act of 1993
GSD Global Systems Division of ESRL, formerly FSL
GSI Gridpoint Statistical Interpolation
JAG/OCM Joint Action Group for Operational Community Modeling
JCSDA
MCV Mesoscale Convective Vortex
MOA Memorandum of Agreement
MMM NCAR Mesoscale and Microscale Meteorology division
NCAR
NCEP
NCOM Navy Coupled Ocean Model
NOGAPS Navy Operational Global Atmospheric Prediction System
NOAA National Oceanic and Atmospheric Administration
NRL Naval Research Laboratory
NSSL National Severe Storms Laboratory
NWS National Weather Service
OAR NOAA/Office of Oceanic and Atmospheric Research
OST NWS/Office of Science and Technology
OTC Operational
PC Program Coordinator
POP Princeton Ocean Prediction model
QPF Quantitative Precipitation Forecast
RAMS
RR Rapid Refresh version of WRF
RUC Rapid Update Cycle
RTVS Real Time Verification System
SPC
SREF
TOR Terms of Reference
USWRP
WRF Weather Research and Forecast modeling system and program
WRF-ARW WRF Advanced Research WRF dynamical core
WJIP WRF Joint Implementation Plan of COPC
WRF-NMM WRF Nonhydrostatic Mesoscale Model dynamical core
WRF-SI WRF Standard Initialization