DoD SBIR 23.3 BAA

Active
No
Status
Closed
Release Date
August 23rd, 2023
Open Date
September 20th, 2023
Due Date(s)
October 18th, 2023
Close Date
October 18th, 2023
Topic No.
OSD233-001

Topic

Deep Rational 3D Geospatial Analytics for Generative AI

Agency

Department of DefenseN/A

Program

Type: SBIRPhase: BOTHYear: 2023

Summary

The Department of Defense (DoD) is seeking proposals for the topic of "Deep Rational 3D Geospatial Analytics for Generative AI" as part of the SBIR 23.3 BAA. The specific agency involved is the National Geospatial-Intelligence Agency (NGA). The objective of this solicitation is to develop advanced AI/ML algorithms that combine generative AI with discriminative AI to enhance 3D geospatial analytics. The current AI/ML techniques for geospatial analysis lack the ability to exploit the rich contextual relational information in the scene. The proposed approaches should leverage generative AI in the form of large language models capable of generating relationships between objects while capturing the relational diversity present in the real world. The proposed approaches must combine generative AI with discriminative AI such as deep convolutional neural network models for segmentation and classification. Phase I of the project requires demonstrating a proof-of-concept approach using open source 3D datasets. Phase II involves optimizing and implementing the selected generative AI and discriminative AI into an end-to-end model. Phase III focuses on the dual-use applications of the developed software, including DoD use cases, civil engineering missions, and commercial robotics applications. The project duration, funding specifics, and performance goals are not provided in the document. For more information, refer to the solicitation agency URL: https://www.defensesbirsttr.mil/.

Description

OUSD (R&E) CRITICAL TECHNOLOGY AREA(S): Trusted AI and Autonomy

OBJECTIVE: Develop advanced AI/ML algorithms that combine generative AI with discriminative AI to enhance 3D geospatial analytics.

DESCRIPTION: DESCRIPTION: Current artificial intelligence and machine learning (AI/ML) techniques for geospatial analysis use pixel or voxel information for semantic segmentation, detection, and classification tasks, but do not exploit the rich contextual relational information in the scene (e.g., cars drive on roads, ships/boats sail on water, etc.). Scenes are fundamentally compositional, adhering to relational rules, which can be exploited to improve geospatial analytics. We seek approaches that combine generative AI, which can describe relations between objects, with discriminative AI to improve multi-task geospatial analysis. Specifically, this topic seeks approaches that leverage generative AI in the form of large language models capable of generating relationships between objects while capturing the relational diversity present in the real world (e.g., cars drive on roads, cars park in drive ways, drive ways connect houses to roads). The proposed approaches must combine generative AI with discriminative AI such as deep convolutional neural network models for segmentation and classification in an end-to-end system. The relations from the generative AI can be considered as constraints and regularization that aid the discriminative AI in solving highly under-constrained problems in 3D geospatial analysis. NGA anticipates that Phase II work will involve input data that may be Controlled Unclassified Information (CUI) or classified.

PHASE I: Demonstrate proof-of-concept approach capable of combining generative AI with discriminative AI using open source 3D datasets derived from commercial satellite (COMSAT) imagery and full motion video (FMV). For a given set of classes (buildings, houses, cars, roads, trees), demonstrate that approach is able to improve multi-task performance in semantic segmentation and object detection beyond baseline approaches that utilize only discriminative AI. Develop a Phase II plan that includes integration, test, and validation of the end-to-end system.

PHASE II: Realize the optimization and implementation of the selected generative AI and discriminative AI into an end-to-end model. Demonstrate the proposed model is trainable with an expanded class/target set, and is able to perform inference on 3D datasets derived from COMSAT and FMV. Develop a technology transition plan and business case assessment.

PHASE III DUAL USE APPLICATIONS: 3D geospatial analytics software leveraging generative AI capable of supporting DoD use cases including scene segmentation, classification, and target detection; civil engineering missions such as surveying, urban mapping and city planning; commercial robotics applications such as route planning.

REFERENCES:

  1. Zhao, Wayne Xin, et al. "A survey of large language models." arXiv preprint arXiv:2303.18223 (2023);
  2. Xiao, Aoran, et al. "Unsupervised Point Cloud Representation Learning with Deep Neural Networks: A Survey." IEEE Transactions on Pattern Analysis and Machine Intelligence (2023).

KEYWORDS: 3D geospatial analytics; generative AI; AI/ML