The Department of Defense, specifically the Navy, is seeking innovative solutions through the SBIR program for the development of a machine learning weather prediction (MLWP) system aimed at generating complete model analyses and skillful forecasts from sparse and irregular observational data. The objective is to create a technique that transcends traditional data assimilation methods, allowing for the integration of various data types and qualities, ultimately enhancing the operational capabilities of MLWP models. This opportunity is critical for advancing weather prediction technologies, with the potential for dual-use applications in collaboration with agencies like NOAA and NASA. The solicitation is currently in pre-release, with an open date of January 8, 2025, and a close date of February 5, 2025, by which interested parties must submit their applications.

## Description

In the last few years, there has been a substantial increase in the number of skillful MLWP models that have demonstrated competitive results with state-of-the-art traditional physics-based numerical weather prediction (NWP) systems. However, these efforts not only require multiple decades of high-quality full physics reanalysis information to train the model, but similar quality high-resolution gridded fields from which to initialize the model. This latter point limits the utility of running MLWP systems operationally since full scale traditional data assimilation and reanalysis methods are needed to inform the MLWP model initial conditions.While MWLP model development requires sufficiently large and balanced datasets to train appropriate physical relationships, it is not clear that a full data assimilation system is needed as with traditional NWP. MWLP time integration is not achieved via discretized partial differential equations, and thus does not share some NWP limitations such as requirements for Courant-Friedrichs-Lewy stability conditions or dynamically balanced states. The need to map real-world data onto a numerically regular grid necessitates smoothing, observation thinning, and methods to spread observational influence in space and time, all of which may lead to a loss of information. This topic solicits innovative machine learning development to build off of recent references (see selection below) that inform methodologies capable of initializing a MWLP model without the need of a full end-to-end NWP-type of data assimilation capability.Given operational constraints of real-time data quality and quantity, this SBIR topic seeks to scope, prototype, and demonstrate a technique to create a MLWP analysis/initialization capability that: 1) is informed by and transcends state-of-the-science data assimilation methods and practices; 2) accepts a variety of observations and data sources, types, qualities, and characteristics; and 3) can be processed with varying and irregular amounts of data over consecutive model cycles.

## Objective

Develop a machine learning weather prediction (MLWP) system to generate a complete model analysis, initialization, and produce skillful forecasts from an incomplete, non-gridded set of observations that may include sparse and irregular spatial and temporal sampling of in-situ, remote sensing, satellite, and qualitative forecaster-based information.

## Phase I

Focus on understanding and documenting the technical limitations of initializing MLWP forecast models with sparse observations and formulating innovative concepts to overcome those challenges. Perform a background study of both data assimilation and state-of-the-art forecast analysis methods that will be required to motivate and inform how the proposed effort will address gaps in current processes. Develop a theory and/or simple method to demonstrate the feasibility of the initialization methods to produce robust analyses and stable, skillful forecast fields based on AI/ML techniques. It will be important to properly scope the breadth of analyzed and model environmental variables as well as the appropriate downstream applications for their use.

## Phase II

Using results from the Phase I, develop, demonstrate, and validate an end-to-end prototype MLWP software suite focused on the novel data ingest, initialization, and analysis scheme. Ensure that the toolset must be able to accept widely varying modalities, qualities, and types of observational data, including in-situ state variables, remotely sensed raw and retrieved quantities, gridded background fields of varying age and accuracy, and qualitative assessments of the environment including forecaster notations of important features and locations of environmental phenomena. Ensure that the prototype software must also allow for discontinuities in data streams (temporally or spatially) and have the ability to run or restart with old/degraded information. Intermediate processing outputs of data impacts and sensitivity to the analysis and/or forecast fields is highly desired. The developed workflow should also include robust methods of validation and verification as well as identify strengths and weaknesses of the product compared to traditional NWP modeling. Perform multiple demonstrations in coordination with field testing (may be required). Submit required Phase II deliverables to include regular reporting, participation in program reviews, technical documentation, and the end-to-end prototype software at the conclusion of the effort.

## Phase III: Dual Use Applications

Operational hardening and establishing utility and trust for real-time application forms the main effort for transition and commercialization. Dynamic analysis software tools that quickly and accurately convey software system health, error logging and debugging, and processing metadata will need to be created and demonstrated. Develop additional metrics and diagnostics to facilitate expert forecaster guidance on using the product (and comparing to current state-of-the-art weather forecast information). Ensure that the system has a formalized methodology and data/compute needs for model training and a separate, leaner set of requirements for operational runs. Techniques should be generalizable to apply to a variety of environmental modeling use cases such that follow-on work and commercial applications can be addressed. Dual-use applications will include partnering with other intergovernmental meteorological agencies such as USAF, NOAA, and NASA as well as commercialization for multiple potential markets with decision making requirements based on forecast skill.

Sparse Data Initialization for Machine Learning Weather Prediction Models

SBIR Opportunity Analysis

SBIR Documents

Related SBIR/STTR Opportunities

Opportunity Snapshot

Key Dates