DOD STTR 24.B Annual

Active
No
Status
Open
Release Date
April 17th, 2024
Open Date
May 15th, 2024
Due Date(s)
June 12th, 2024
Close Date
June 12th, 2024
Topic No.
OSD24B-001

Topic

Image Quality and Task Complexity for Machine Learning

Agency

Department of DefenseN/A

Program

Type: STTRPhase: Phase IYear: 2024

Summary

The Department of Defense (DOD) is seeking proposals for the topic of "Image Quality and Task Complexity for Machine Learning" as part of their Small Business Technology Transfer (STTR) Phase I program. The specific agency involved is the National Geospatial-Intelligence Agency. The objective of this solicitation is to explore the relationship between task complexity, image quality, and machine learned model capacity using information theory. The goal is to develop a predictive analytic that can assess machine learning performance for previously unseen image sources. The focus is on down-looking image detection and classification tasks, such as counting civilian vehicles in a factory parking lot. The research will be used for triaging and updating machine learned models' decision confidence based on current image streams. The project duration is divided into two phases: Phase I involves critically assessing the results and identifying suitable operational domains, while Phase II requires justifying the proposal, setting clear milestones, and developing a strong test plan. The ultimate aim is to expand these methods to characterize ML models, other image modalities, and analysis tasks, leading to increased efficiencies in automated image workflow. The solicitation is currently open, with a closing date of June 12, 2024. For more information, interested parties can visit the SBIR topic link at https://www.sbir.gov/node/2606021 or the DOD SBIR/STTR opportunities page at https://www.defensesbirsttr.mil/SBIR-STTR/Opportunities/.

Description

OUSD (R&E) CRITICAL TECHNOLOGY AREA(S): Information Systems

 

OBJECTIVE: This announcement seeks proposals that relate task complexity to image quality and machine learned model capacity, where capacity is the ability to perform discrimination, using information theory.  The outcome from this work generates a predictive analytic for machine learning performance from previously unseen image sources for any machine learned model.  The focus here is on down-looking image detection and classification tasks, which is defined here collectively as discrimination.  An example discrimination task is counting civilian vehicles in a factory parking lot.  This work will be used to:

 

Triage: Compare the task-relevant image information to the machine learned model’s capacity for answering the specific intelligence question.  Future efforts would then use the same strategy to perform tasking.

Analytics: Update the model’s decision confidence based upon the current image stream that might be different from the training data.

 

An individual machine learned model’s ability to perform triage and analytics can be solved analytically.  In practice, the sheer volume of machine learned models, their frequent update training, and variety of image sources make the problem intractable.  We therefore seek a generalized description for behaviors that hold across a wide range of machine learning model, where its domain of applicability can be characterized.  We expect this research to increase the opportunities to answer intelligence questions and the intelligence yield of the system.

 

 

DESCRIPTION: The current image quality equations (IQE) and National Imagery Interpretation Rating Scales (NIIRS) are based upon subjective human cognition, and neither effectively supports automation or machine learning (ML).  This opportunity is accepting proposals to identify information-theoretic approaches that support automation within machine learning workflows.  During Task 1, the performer reviews the state-of-the-art information-theoretic measures that predict ML discrimination performance with a compact set of descriptors given image quality and task complexity.  Though Task 1 images are single band electro-optical, the performer should describe how their metric and process could be extended to two or more spectral bands.  Task 1 deliverables are the literature review showing the application of information theory to task complexity; the information-theoretic metric algorithmic description with implementation strategy; and results using a mutually agreed upon down-looking image dataset.  Task 2 applies the workflow laid out in Task 1 to a variety of image discrimination tasks, image quality, and model architectures.  Task 2 extends the information-theoretic utility to unlabeled conditions.  Finally, the performer will predict performance for additional task complexity and image quality conditions from known image and machine learned model relationships.  The performer will critically assess the results across task complexity, image quality, and model capacity.  From the assessment, the performer will identify suitable operational domains for their information-theoretic metric and quantify the limitations.  The effort should justify the Phase II proposal, identify clear milestones, and include a strong test plan for assessing the performer’s selected metric(s) against image quality and machine learned model capacity.  Proposers are expected to demonstrate their expertise through a relevant publication record.

 

 

PHASE I: The performer will critically assess the results across task complexity, image quality, and model capacity.  From the assessment, the performer will identify suitable operational domains for their information-theoretic metric and quantify the limitations.

 

PHASE II: The effort should justify the Phase II proposal, identify clear milestones, and include a strong test plan for assessing the performer’s selected metric(s) against image quality and machine learned model capacity.  Proposers are expected to demonstrate their expertise through a relevant publication record.

 

PHASE III DUAL USE APPLICATIONS: Follow-on activities are expected to expand these methods to characterize ML models, other image modalities, and analysis tasks.  Ultimately, these technologies will support additional efficiencies in automated image workflow.

 

REFERENCES:

J. Irvine, "National imagery interpretability rating scales (NIIRS): overview and methodology," in SPIE Proceedings Volume 3128, Airborne Reconnaissance XXI, 1997.
I. Israel, S. Israel and J. Irvine, "Factors Influencing CNN Performance," in IEEE Applied Imagery Pattern Recognition Workshop (AIPR), 2021.
Z. Wang, A. Bovik, H. Sheikh and E. Simoncelli, "Image quality assessment from error visibility to structural similarity," IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600-612, 2014.

 

KEYWORDS: AI/ML, information content, quality metrics, task complexity

Similar Opportunities

DOD SBIR 24.4 Annual - Lightweight AI-enabled image processing for Soldier-borne thermal imagers
Department of Defense
The Department of Defense (DOD) is seeking proposals for lightweight AI-enabled image processing for Soldier-borne thermal imagers. The objective of this solicitation is to leverage advances in artificial intelligence and other image processing algorithms to generate higher quality longwave thermal and fused thermal and near-infrared imagery suitable for use on embedded hardware systems for Soldier-borne use. The technology should reduce cognitive burden during long duration missions and improve user acceptance of systems that employ LWIR and NIR sensors. The algorithms should be capable of generating high-quality imagery under various illumination and ambient conditions and provide feedback to the system to adjust camera settings. The proposed processing schema should be capable of running on low size, weight, power, and cost (SWAP-C) embedded hardware. The project will be conducted in three phases: Phase I involves generating a detailed description of the proposed solution, Phase II focuses on completing the image processing pipeline, and Phase III involves instantiating the image pipeline on relevant low SWAP-C embedded hardware. The project duration is from the release date (October 3, 2023) to the close date (March 31, 2025). For more information, visit the [solicitation link](https://www.sbir.gov/node/2651327).
DOD SBIR 24.4 Annual - Artificial Intelligence/ Machine Learning (AI/ML) Focused Open Topic
Department of Defense
The Department of Defense (DOD) is seeking proposals for an Artificial Intelligence/Machine Learning (AI/ML) Focused Open Topic. The purpose of this open topic is to bring potentially valuable small business innovations to the Army and expand the relevance of the Army SBIR program to firms who do not normally compete for SBIR awards. The topic accepts both Phase I and Direct to Phase II submissions. Phase I proposals are accepted for a cost up to $250,000 for a 6-month period of performance, while Direct to Phase II proposals are accepted for a cost up to $2,000,000 for a 24-month period of performance. The research areas of focus include synthetic data generation, data validation and verification, AI risk mitigation, large language models, retrieval augmented generation, and collaborative AI technologies. Phase I submissions require a 5-page technical volume, an 8-slide commercialization plan, and a statement of work outlining deliverables. Phase II submissions require a 10-page technical volume, a 5-page feasibility documentation, an 8-slide commercialization plan, and a statement of work. During Phase II, firms must produce prototype solutions that are practical and feasible to operate in edge and austere environments. Phase III focuses on the maturation of the technology to TRL 6/7 and the production of prototypes to support further development and commercialization. The Army will evaluate each product in a realistic field environment and provide solutions to stakeholders for further evaluation. The submission deadline is March 31, 2025. For more information, visit the [solicitation link](https://www.sbir.gov/node/2651299).