Skip to page content

Confidence Metrics

Background

For operational use, it is not sufficient for the NextGen Weather Processor (NWP) to only provide deterministic "radar-forward" storm predictions out to eight hours; the associated confidence in the predictions must also be conveyed so strategic planners know whether it is advisable to take action or not.

The NWP Confidence metrics are specifically designed to convey uncertainty in the predicted airspace constraints due to weather (not uncertainty in the weather forecast itself, which is communicated adequately via the display of baseline planning guidance overlaid upon the NextGen Weather predictive products). Strategic planners are often concerned with the impact of weather on busy regions of airspace containing heavily used, closely spaced jet routes. To control the enroute flow through such regions, an airspace flow program for a pre-defined Flow Constrained Area (FCA) can be implemented, where flow rates are constrained hour by hour through the FCA before and during the weather impact. Predicting weather constraints on busy terminal airspace is also of great concern to planners; both enroute and terminal Confidence metrics are required for strategic traffic flow management.

Machine Learning

Refer to the following caption

Machine Learning used in computing the NWP Confidence metrics.

The Machine Learning process used in computing the NWP Confidence metrics is illustrated in the figure. The box labeled "Machine Learning" in the center of the diagram has two main input sources for each forecast time. The Model Inputs, shown in the upper left, include various forecast features from the NWP Predictive Products and three NOAA models: the HRRR (High Resolution Rapid Refresh model), with multiple hourly runs combined to form a time-lagged ensemble; the SREF (Short Range Ensemble Forecast); and the LAMP (Localized Aviation MOS Program, where MOS stands for Model Output Statistics). The Historical Forecast Performance statistics, shown as a database in the upper right, provide historical information for all the Model Inputs and are used to train the algorithm that estimates the probability of weather-related airspace constraint at each forecast time.

The output from Machine Learning takes the form of a continuous probability distribution function at each forecast time; an example of the distribution for 18:00 UTC is shown at the bottom of the figure. The mean of the distribution indicates the most likely airspace constraint, quantified in terms of the permeability of the airspace within the FCA. (Airspace is permeable if aircraft can flow through it; unconstrained airspace has 100% permeability and severely restricted airspace has 0% permeability.) The spread of the distribution indicates the probable range of airspace constraints. A wide distribution implies broad uncertainty in the estimate, while a narrow, sharply peaked distribution implies a high degree of certainty.

Confidence Metrics

The resulting NWP Confidence metrics are shown in the 8-hour timeline figure. The display indicates predicted FCA weather impacts via airspace permeability, ranging from 0 to 100% (left axis). The blue line is a plot of the distribution mean at each forecast time, and represents the most likely values of FCA permeability at each hour, out to 8 hours in the future. The purple shading above and below the blue line delineates the range of probable impacts, derived from the distribution spread at each forecast time. The red box around the data at 18:00 UTC shows the portion of the Confidence timeline chart that corresponds with the continuous probability distribution function example shown in the Machine Learning figure. The confidence metrics are mapped against three levels of weather impact (low, medium, high) shown as green, yellow, red background shading.

Refer to the following caption

NWP Confidence metrics shown in an 8-hour timeline format.

Display Concept

A display concept for NWP Confidence metrics is illustrated in the figure below, where the display home is pre-set to New York and the NWP Precipitation at 8 hours is shown in the top panel. A notional FCA named ZNY001, designed to capture the major east-west flows through Pennsylvania into New York, is shown overlaid as a light blue rectangle oriented roughly east-west. The bottom panel shows the estimated weather impact at each hour out to 8 hours in the future, for a set of five pre-defined FCAs that control major flows into New York. A green-yellow-red color scheme is used to convey low-medium-high impacts. A link in the first column of each row brings up the full 8-hour Confidence timeline plot described in the previous section, and shown again here in the upper left portion of the figure.

Refer to the following caption

Display concept for NWP confidence metrics.

Summary

NWP Confidence metrics provide a translation of "radar-forward" weather predictions, which show the future storm pattern, storm intensity and vertical extent, into predicted operational weather constraints on the National Airspace System. They allow strategic traffic flow managers to quickly glean the severity and duration of weather constraints over the next 8 hours. With a display concept including Confidence metrics, evidence of a predicted weather constraint – perhaps the motivation for a restrictive airspace flow program – can be viewed by all stakeholders, thus enhancing shared situational awareness. Every stakeholder can make informed decisions based on the most probable outcomes, without suffering from weather information overload or from limited single-source guidance. The NWP Confidence Metrics are undergoing testing, and user feedback is solicited.

Back to Top

Page last modified:

This page was originally published at: https://www.faa.gov/nextgen/programs/weather/tfm_support/confidence/