DOD SBIR 24.2 Annual

Active
No
Status
Open
Release Date
April 17th, 2024
Open Date
May 15th, 2024
Due Date(s)
June 12th, 2024
Close Date
June 12th, 2024
Topic No.
AF242-D011

Topic

Segment Anything For Extended Reality

Agency

Department of DefenseN/A

Program

Type: SBIRPhase: BOTHYear: 2024

Summary

The Department of Defense (DOD) is seeking proposals for the topic "Segment Anything For Extended Reality" as part of their SBIR 24.2 Annual solicitation. The Air Force branch is specifically interested in researching, evaluating, and implementing Artificial Intelligence (AI) based 3D spatial mapping techniques using Commercial Off The Shelf Extended Reality (XR) devices. The goal is to develop fast and precise detection of multiple known and unknown objects without the need for additional AI training. This technology will enable other automated systems to understand their environment quickly and easily, while allowing humans to provide oversight. The use of AI-based 2D image segmentation methods, such as the Segment Anything Model (SAM), will be extended to 3D images captured by XR devices. The resulting 3D spatial mapping and segmentation information can be used for precise path planning for mobile robots in dynamic operational environments. The project will progress through Phase II, where a working prototype will be developed, and potentially Phase III, where the technology will be refined for production and marketing to the Air Force and other federal agencies. The project duration is not specified, but the application due date is June 12, 2024. More information can be found on the grants.gov website.

Description

OUSD (R&E) CRITICAL TECHNOLOGY AREA(S): Trusted AI and Autonomy; Integrated Sensing and Cyber; Human-Machine Interfaces

 

OBJECTIVE: there is a clear AFSC and DoD need to automate sustainment processes in order to improve safety, quality, capacity, and readiness. Our near peers are automating at an alarming rate, and we should be developing technologies that keep us as many steps ahead as possible. The purpose of this topic is to research, evaluate, and implement Artificial Intelligence (AI) based 3D spatial mapping techniques using COTS XR (Commercial Off The Shelf Extended Reality) devices for fast and precise detection of multiple known and unknown objects without the additional need of AI training. This data will enable other automated systems to understand their environment quickly and easily while empowering the human to add value and oversight.

 

DESCRIPTION: recent advances in AI-based 2D image segmentation methods such as Segment Anything Model (SAM) has enabled automatic detection and precise segmentation of any object in a 2D image without additional AI training. Additionally, this image segmentation can be customized based on the environment by points, areas, and size of objects in the 2D image. The segmentation AI models transfer over a range of different data sets and environments with a high accuracy, making it robust to changes in environments and objects in that space.

The use of mobile industrial robots in sustainment and depot environments has grown significantly over the years and provides great improvements in safety, quality, agility, and throughput metrics. A key challenge for mobile robots is to have an accurate 3D spatial map of dynamically changing environments in order to reach the target workpieces without accidentally colliding into other 3D objects or humans in the environment.

Using COTS XR devices to capture 3D images and applying and extending the 2D SAM models toward captured 3D images can enable real time 3D spatial mapping and segmentation of all objects in a dynamic operational environment. This segmentation information and precise localization of various objects in 3D space can be automatically transferred into robotic controls for precise path planning without collision. The desired process will be seamless for the operator, who can confirm on XR devices the accuracy of the segmentation in real time, practically eliminating error and making mobile robotic systems faster and more agile.

 

PHASE I: As this is a Direct-to-Phase-II (D2P2) topic, no Phase I awards will be made as a result of this topic. To qualify for this D2P2 topic, the Government expects the applicant(s) to demonstrate feasibility by means of a prior “Phase I-type” effort that does not constitute work undertaken as part of a prior or ongoing SBIR/STTR funding agreement. Applicant(s) will demonstrate feasibility by describing the ability to accurately and precisely detect various objects of interest in an aircraft sustainment depot in an operational environment.

 

PHASE II: Awardee(s) will develop a working prototype to detect multiple known and unknown 3D objects using an XR device and provide precise 3D segmentation of the objects in real time in a dynamic operational environment. The prototype will interface with a robotic controller software to automatically transfer the segmentation masks with 3D localization data in the world coordinate space.

 

PHASE III DUAL USE APPLICATIONS: If the Phase II is successful in developing the needed technology, WR-ALC will purchase additional systems using organization (working capital) funds. The procurement will include the refinement the AI and XR systems to increase accuracy and reliability. Achieve production-ready state for marketing to the Air Force, other related federal agencies, and private industry.

 

REFERENCES:

Alexander Kirillov, et. al., "Segment Anything", IEEE/CVF International Conference on Computer Vision (ICCV), 2023.
Chao-Yuan Wu, et. al., "Multiview Compressive Coding for 3D Reconstruction", Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
A. Mirzaei, et. al., "SPIn-NeRF: Multiview Segmentation and Perceptual Inpainting With Neural Radiance Fields", Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.;

 

KEYWORDS: Extended Reality 3D Segmentation

Similar Opportunities

DOD SBIR 24.4 Annual - xTechScalable AI
Department of Defense
The Department of Defense (DOD) is seeking proposals for the topic of "xTechScalable AI" as part of the SBIR program. The Army branch is specifically interested in novel and disruptive concepts and technology solutions that can address the vulnerabilities of current machine learning pipelines and models. The goal is to develop comprehensive security models that can defend against universal AI threat vectors and are scalable to rapidly evolving threats. The Army is particularly interested in proposals that focus on systematic testing and evaluation methods, trusted and secure validation and verification strategies, continuous monitoring capabilities, improved transparency and assurance of code and data, and improved telemetry capabilities. The Army will prioritize submissions from winners of the xTechScalable AI prize competition. The project will involve a Direct to Phase II submission, followed by the production of prototype solutions that are easy to operate by soldiers. Phase III will focus on maturing the technology to TRL 6/7 and producing prototypes for further development and commercialization. The deadline for proposal submission is March 31, 2025. For more information and to submit a proposal, visit the solicitation agency's website at [solicitation_agency_url].
DOD SBIR 24.4 Annual - xTech Search 8 SBIR Finalist Open Topic Competition
Department of Defense
The Department of Defense (DOD) is seeking proposals for the xTech Search 8 SBIR Finalist Open Topic Competition. The objective of this solicitation is to find novel and disruptive concepts and technology solutions with dual-use capabilities that can address the Army's current needs and apply to current Army concepts. The technology areas of interest include Electronics, Human Systems, and Sensors. The Army is particularly interested in technologies related to Artificial Intelligence/Machine Learning, Advanced Materials, Advanced Manufacturing, Autonomy, Cyber, Human Performance, Immersive, Network Technologies, Position, Navigation and Timing (PNT), Power, Software Modernization, and Sensors. The Phase I of the project requires a feasibility study and concept plans, while Phase II involves producing prototype solutions that can be easily operated by soldiers. Phase III focuses on the maturation of the technology and its transition to TRL 6/7, as well as further development and commercialization. The solicitation is open until March 31, 2025. For more information, visit the [solicitation agency website](https://www.defensesbirsttr.mil/SBIR-STTR/Opportunities/).
DOD SBIR 24.4 Annual - xTech Search 8 SBIR Finalist Open Topic Competition
Department of Defense
The Department of Defense (DOD) is seeking proposals for the xTech Search 8 SBIR Finalist Open Topic Competition. The objective of this solicitation is to find novel and disruptive concepts and technology solutions with dual-use capabilities that can address the Army's current needs and apply to current Army concepts. The technology areas of interest include Electronics, Human Systems, and Sensors. The Army is particularly interested in technologies related to Artificial Intelligence/Machine Learning, Advanced Materials, Advanced Manufacturing, Autonomy, Cyber, Human Performance, Immersive, Network Technologies, Position, Navigation and Timing (PNT), Power, Software Modernization, and Sensors. The Phase I of the project requires a feasibility study and concept plans, while Phase II involves producing prototype solutions for evaluation by soldiers. Phase III focuses on the maturation of the technology and commercialization. The solicitation is open until March 31, 2025. For more information, visit the [solicitation agency website](https://www.defensesbirsttr.mil/SBIR-STTR/Opportunities/).
DOD SBIR 24.4 Annual - Robust Computer Vision for Better Object Detection with Limited Training Data
Department of Defense
The Department of Defense (DOD) is seeking proposals for the topic "Robust Computer Vision for Better Object Detection with Limited Training Data" as part of their SBIR 24.4 Annual solicitation. The goal of this topic is to experiment with innovative AI/ML approaches to object identification and imagery scene analysis. The increasing availability of digital imagery requires automated methods to process and analyze vast amounts of multi-modal data efficiently. One critical application is the identification of objects of interest (OoI) within imagery data or the scene generated by the imagery, which can provide valuable insights and facilitate decision-making processes in various fields such as military intelligence, environmental monitoring, transportation management, and security surveillance. The solicitation is open for Direct to Phase II (DP2) proposals with a cost of up to $2,000,000 for an 18-month period of performance. Proposers interested in submitting a DP2 proposal must provide documentation to substantiate that the scientific and technical merit and feasibility equivalent to a Phase I project has been met. The focus of this SBIR topic is robust AI/ML object detection techniques for computer vision that do not rely on extensive availability of labeled training data. The use of foundational knowledge and methods, such as handcrafted features, evolutionary algorithms, and newer techniques based on transformers, can be leveraged for this topic without requiring a feasibility study. During DP2, firms should develop and implement novel or hybrid AI/ML models for object detection that do not rely on extensive training data and train models in Project Linchpin's AI Unclassified Operations Environment using Linchpin data for DOD use cases. The Phase III dual-use applications include autonomy, retail, public safety, traffic management, enhanced security, and agriculture. Computer vision solutions in the private sector encompass a wide range of applications, and companies like Amazon, Google, and Microsoft offer cloud-based object detection and recognition services. The solicitation is open until March 31, 2025. For more information and to submit a proposal, visit the DOD SBIR website: [link](https://www.defensesbirsttr.mil/SBIR-STTR/Opportunities/).