DOD STTR 24.B Annual

Active
No
Status
Open
Release Date
April 17th, 2024
Open Date
May 15th, 2024
Due Date(s)
June 12th, 2024
Close Date
June 12th, 2024
Topic No.
AF24B-T002

Topic

Neural Collapse for Responsible Artificial Intelligence in Directed Energy

Agency

Department of DefenseN/A

Program

Type: STTRPhase: Phase IYear: 2024

Summary

The Department of Defense (DOD) is seeking proposals for the Small Business Innovation Research (SBIR) program, specifically for Phase I of the STTR program. The topic of the solicitation is "Neural Collapse for Responsible Artificial Intelligence in Directed Energy." The research aims to develop theoretical foundations to support and sustain Artificial Intelligence (AI) technologies within Directed Energy. The motivation behind this research is the need for trustworthy AI systems that are compatible with Defense strategies and directives. The potential applications of this technology include target acquisition and tracking problems in Directed Energy, such as acquiring small unmanned aircraft, cruise missiles, and anti-aircraft missiles. The main goal is to understand neural network technology beyond black box models and develop explainable and mathematically understood models. The research will be conducted in two subgoals: basic research to study the fundamentals of neural networks and achieve explainability, and applied research to understand the applicability of neural networks to Directed Energy problems. The deliverables of the project include reports, a software package, and material necessary to understand and reproduce the results. The project will be conducted in three phases: Phase I involves downselecting imaging datasets and conducting preliminary research, Phase II focuses on completing basic and applied research efforts, and Phase III involves conducting statistical analysis and demonstrating the implications of Neural Collapse in Directed Energy systems. The ultimate outcome of the project is the integration of Neural Collapse concepts into Directed Energy AI system development, enabling compliance with orders, strategies, and regulations mandating trustworthy AI. The project is open for proposals until June 12, 2024. For more information and to submit a proposal, visit the solicitation agency's website at [solicitation_agency_url].

Description

OUSD (R&E) CRITICAL TECHNOLOGY AREA(S): Trusted AI and Autonomy; Advanced Computing and Software; Directed Energy (DE)

 

The technology within this topic is restricted under the International Traffic in Arms Regulation (ITAR), 22 CFR Parts 120-130, which controls the export and import of defense-related material and services, including export of sensitive technical data, or the Export Administration Regulation (EAR), 15 CFR Parts 730-774, which controls dual use items. Offerors must disclose any proposed use of foreign nationals (FNs), their country(ies) of origin, the type of visa or work permit possessed, and the statement of work (SOW) tasks intended for accomplishment by the FN(s) in accordance with the Announcement. Offerors are advised foreign nationals proposed to perform on this topic may be restricted due to the technical data under US Export Control Laws.

 

OBJECTIVE: Purpose:

To develop theoretical foundations necessary to support and sustain Artificial Intelligence technologies within Directed Energy.

 

Motivation:

Modern neural network-based machine learning techniques achieve high performance, at the cost of limited explainability and understandability through their black box structure. As described in a recent Defense strategy [1] and directive [2] Defense Artificial Intelligence systems in general require a level of trust incompatible with state-of-the art machine learning systems. A recent executive order [3] further demonstrates the need for trustworthy Artificial Intelligence systems.

 

Benefit:

AFRL/RD is interested in investigating solutions that address the described black box incompatibility for Directed Energy-specific target acquisition and tracking problems. Specific problems might include acquisition of small, unmanned aircraft, cruise missiles, small ground-based targets, and anti-aircraft missiles. If successful, compatibility will have impact to both AFRL/RD and other Directed Energy organizations, with strong potential for generalization to other Department of Defense weapons and commercial applications such as medical diagnostic systems (e.g. the Food and Drug Administration's oncology diagnostics program [4].) The compatibility will enable trust and proliferation of predictive artificial intelligence-based decision-making systems, enabling the Warfighter to automate tasks and achieve well-understood behaviors.

 

Main Goal:

To understand neural network technology, beyond traditional black box models towards fully explainable, mathematically understood models, enabling Defense users to implement trusted artificial intelligence systems.

Subgoals:

(Basic Research) Study fundamentals of neural networks to enable deeper understanding in terms of neural network performance, performance bounds, and limitations, and achieve neural network explainability.
(Applied Research) Understand neural network fundamentals and their applicability to Directed Energy problems. Achieve compatibility with Defense strategies and directives [1] [2]. 

 

Deliverables:

Reports in Powerpoint and text-based manuscript formats.

 

A software package including material necessary to understand and reproduce SBIR results.

 

DESCRIPTION: Recent research has demonstrated a neural network training phenomenon, Double Descent [5]. Here, sufficiently large predictive classification networks can train beyond the overfitting regime and achieve lower local minimum test loss, approaching the global minimum. Later, it was shown that double descent-achieving networks can collapse, or converge, to a structure with well-defined mathematical properties [6]. This is called Neural Collapse, and its study has recently been an active research field regarding its implications on performance, optimality, robustness, and other traits.

 

AFRL/RD has interest in the Neural Collapse phenomenon because conclusions derived from its study may generalize across all neural networks achieving Neural Collapse. More specifically, recent research has investigated Neural Collapse for traits such as optimality [7], training success [8], transfer learning [9], and robustness [10]. While the state-of-the-art is not fully peer reviewed, success in these lines of research could lead to explainability and trust in a technology that has traditionally been considered a black box, achieving the objective of the STTR.

 

To meet the objective, potential tasks in several research directions are described below. These tasks are open ended, and comprehensive results are not expected before Phase III. Further related tasks in support of the goals may be proposed due to the fast-paced nature of the research area.

 

Applied Research (Export Controlled):

Demonstrate that networks trained on Directed Energy datasets such as CLIPS, in applications such target acquisition of small, unmanned aircraft, can achieve Double Descent and Neural Collapse.
Validation of recent research (References 7-10) with regards to correctness and applicability of Neural Collapse to the Directed Energy dataset.
Quantify generalizability of Neural Collapse across different Directed Energy application datasets and sensors. Understand Neural Collapse robustness to specific sensor, noise, and scenario types.

 

Information Theory/Optimality Basic Research (Not Export Controlled):

Using techniques such as Task Specific Information [11] to calculate quantitative optimal detector/classifier task performance, show that a Neural Collapsed network quantitatively approaches this performance value.
If such optimal performance is demonstrated, investigate implications derived from optimality including bounded network outputs and network robustness.
Quantify the degree of neural collapse in a network and relate this value as a function to quantified task performance.

 

Neural Collapse Behavior/Limitations Basic Research (Not Export Controlled):

Neural Collapse represents an end state; a neural network converges to the state but may never fully reach collapse. In addition, multiple layers in the network may reach an intermediate collapsed state [8, 12]. Define and justify what is quantitatively meant when a network should be declared to be in a collapsed state, and the implications in this declaration.
A network needs to be sufficiently large and have sufficient training to reach Neural Collapse. Quantify requirements that define when a network is capable of/qualified to reach a double descent/neural collapse state.
Quantify the resource costs and performance benefits associated with Neural Collapse. Are there performance benefits (Quantified with ROC/Precision and Recall/Confusion Matrix) associated with reaching Neural Collapse, and are there additional training/testing/inference/other costs associated with operating in a Neural Collapse state?

 

Results are expected through a software package, as well as in written form including reports and presentation, with likely submission to the DTIC database. Publication through conferences, articles, and press release is encouraged.

 

PHASE I: TECHNICAL OBJECTIVES:

Downselect imaging datasets for training a neural network-based detection system, including Defense/Directed Energy datasets and public, popular datasets. Successfully demonstrate Neural Collapse behavior on Directed Energy target datasets to demonstrate application, as well as popular public datasets, through network training and transfer learning.
Identify promising Neural Collapse research tasks for Phase II and conduct preliminary research demonstrating potential value in this tasking.

 

TECHNICAL OUTCOMES:

Neural collapse is confirmed as a phenomenon affecting neural networks and confirmed as relevant to Directed Energy applications.
Delivery of a software package demonstrating existence of the Neural Collapse phenomenon. The package should contain source code, build scripts and instructions, a software dependency list including acquisition and installation instructions, dataset acquisition instructions, and software documentation including structural and functional descriptions.

 

PROGRAM OUTCOMES:

Establish plans including statement of work, work breakdown, and software development documents for specific tasks to complete in Phase II, either from the Project Description, or self-developed.
Establish relationships and collaborations with interested Directed Energy and Defense partners 

 

PHASE II: TECHNICAL OBJECTIVES:

Complete, demonstrate, and report on basic and applied research efforts to further project goals.

 

TECHNICAL OUTCOMES:

Basic Research: Inform the Applied Research effort and report on and publish results from task completion.



Applied Research: Demonstrate that Neural Collapse
Delivery of software package demonstrating the specifically researched Neural Collapse phenomenon. The package should contain source code, build scripts and instructions, a software dependency list including acquisition and installation instructions, dataset acquisition instructions, and software documentation including structural and functional descriptions.

 

PROGRAM OUTCOMES:

Demonstrate that Neural Collapse may have a significant impact on Directed Energy applications as well as broader detection/classification applications.

 

PHASE III DUAL USE APPLICATIONS:

 

ENTRY CRITERIA: TRL-3 or TRL-4

 

TECHNICAL OBJECTIVES:

Conduct statistical analysis using effort-specific data recordings from either a Directed Energy testbed such as BC TRAIL or a laboratory setup, demonstrating discovered Neural Collapse implications as applied to Directed Energy systems. These implications might include robustness, optimality or transferability, as discussed in the Project Description.

 

TECHNICAL OUTCOMES:

1)Demonstration of Neural Collapse and its implications as a TRL-5 capability. If successful, this demonstration will enable integration with DE imaging systems and AI subsystems.

 

PROGRAM OUTCOMES:

Successful Integration of Neural Collapse concepts into DE AI system development, enabling compliance with orders, strategies, and regulations mandating trustworthy AI.

 

REFERENCES:

DoD Responsible AI Working Council, “DoD Responsible Artificial Intelligence Strategy and Implementation Pathway,” June 21, 2022
DoD Directive 3000.09, “Autonomy in Weapons Systems”, January 25, 2023
Biden, Joseph R. "Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence." (2023)
https://www.fda.gov/about-fda/oncology-center-excellence/oncology-therapeutic-and-diagnostic-devices;

 

KEYWORDS: Machine Learning; Artificial Intelligence; Applied Mathematics; System Performance; Directed Energy; Computer Vision; Image Processing

Similar Opportunities

DOD SBIR 24.4 Annual - Proliferated Warfighter Space Architecture (PWSA) Advanced Capability Development Open Topic
Department of Defense
The Department of Defense (DOD) is seeking proposals for the Proliferated Warfighter Space Architecture (PWSA) Advanced Capability Development Open Topic. The Space Development Agency (SDA) is looking for novel architecture concepts, systems, technologies, and capabilities that enable leap-ahead improvements for future tranches of currently planned PWSA capability layers or address other emerging warfighter needs. The research areas include trusted AI and autonomy, advanced computing and software, integrated sensing and cyber, hypersonics, microelectronics, integrated network systems-of-systems, space technology, renewable energy generation and storage, advanced infrastructure, and advanced manufacturing. The solicitation is open for Phase II proposals only, and proposers must demonstrate the scientific and technical merit and feasibility of their projects. The research will be conducted in multiple themes, including integrating commercial sensing to the transport layer, developing optical inter-satellite link (OISL) technology and industrial base, cybersecurity, networking, in-space processing, increasing power for spacecraft bus, generic BMC3 hardware and middleware solutions, seamless multi-level security (MLS), and high-performance clocks for space. The Phase III applications of this research include providing low earth orbit communication systems and space-based processing for the distribution of overhead sensor data. The proposal submission deadline is March 31, 2025. For more information, visit the DOD SBIR 24.4 Annual solicitation notice on grants.gov or the SDA website.
DOD SBIR 24.4 Annual - AI-Enhanced TPS Development and Sustainment
Department of Defense
The Department of Defense (DOD) is seeking proposals for the topic of "AI-Enhanced TPS Development and Sustainment" as part of their SBIR program. The objective of this project is to develop an organic capability for field-level maintenance and repair of weapon system electronics, reducing supply chain latency and screening electronic components for No Evidence of Failure (NEOF) at the source. The goal is to achieve faster weapon system repairs, shorter component Turn-Around-Times (TAT), high equipment Operational Availability (Ao), and high unit readiness at lower life-cycle costs. The project will apply Artificial Intelligence (AI) and Model-Based Systems Engineering (MBSE) to improve the development, operation, and sustainment of Test Program Sets (TPS) for electronic components of weapon systems. Phase I proposals are accepted with a budget of up to $250,000 for a 6-month period of performance. Phase II will involve TPS hardware and software prototype development, critical design, test, integration, verification, validation, and acceptance. Phase III will focus on dual-use applications such as sensor integration in mobile platforms with AI-assisted guided diagnostics. The project has potential applications across all Army commodities, ground, air, missile, and C5ISR. The solicitation is open until March 31, 2025. For more information, visit the [SBIR topic link](https://www.sbir.gov/node/2651293) or the [solicitation agency website](https://www.defensesbirsttr.mil/SBIR-STTR/Opportunities/).
DOD SBIR 24.4 Annual - xTechScalable AI
Department of Defense
The Department of Defense (DOD) is seeking proposals for the topic of "xTechScalable AI" as part of the SBIR program. The Army branch is specifically interested in novel and disruptive concepts and technology solutions that can address the vulnerabilities of current machine learning pipelines and models. The goal is to develop comprehensive security models that can defend against universal AI threat vectors and are scalable to rapidly evolving threats. The Army is particularly interested in proposals that focus on systematic testing and evaluation methods, trusted and secure validation and verification strategies, continuous monitoring capabilities, improved transparency and assurance of code and data, and improved telemetry capabilities. The Army will prioritize submissions from winners of the xTechScalable AI prize competition. The project will involve a Direct to Phase II submission, followed by the production of prototype solutions that are easy to operate by soldiers. Phase III will focus on maturing the technology to TRL 6/7 and producing prototypes for further development and commercialization. The deadline for proposal submission is March 31, 2025. For more information and to submit a proposal, visit the solicitation agency's website at [solicitation_agency_url].
DOD SBIR 24.4 Annual - Proliferated Warfighter Space Architecture (PWSA) Advanced Capability Development Open Topic
Department of Defense
The Department of Defense (DOD) is seeking proposals for the Proliferated Warfighter Space Architecture (PWSA) Advanced Capability Development Open Topic. The Space Development Agency (SDA) is looking for novel architecture concepts, systems, technologies, and capabilities that enable leap-ahead improvements for future tranches of currently planned PWSA capability layers or address emerging warfighter needs. The research areas include trusted AI and autonomy, advanced computing and software, integrated sensing and cyber, hypersonics, microelectronics, integrated network systems-of-systems, space technology, renewable energy generation and storage, advanced infrastructure, and advanced manufacturing. The solicitation is open for Phase II proposals only, and proposers must demonstrate the scientific and technical merit and feasibility of their projects. The research will support the development of the PWSA, a resilient military sensing and data transport capability in Low Earth Orbit (LEO). The solicitation provides specific themes and focus areas for potential deliverables, such as integrating commercial sensing to the transport layer, developing optical inter-satellite link (OISL) technology, advancing cyber and networking capabilities, and increasing power for spacecraft bus. The Phase III applications of the research include improving low Earth orbit communication systems and space-based processing for effective distribution of sensor data. The proposal submission deadline is March 31, 2025. For more information and to access the proposal template, visit the DOD SBIR website.