The Defense Threat Reduction Agency (DTRA) is seeking innovative solutions through the Small Business Innovation Research (SBIR) program to develop AI/ML models and pipelines for the automated extraction and processing of elements from scientific documents. The primary objective is to enhance the fidelity of digitized information from non-digital technical documents related to national nuclear projects, ensuring compliance with DoD and NIST guidelines. This technology aims to benefit the broader U.S. defense sector by automating the extraction of critical data elements such as tables, plots, and photos, ultimately improving the efficiency of document processing. Interested parties should note that the solicitation is currently closed, with the application due date having been set for October 23, 2024. For more information, visit the official SBIR website at https://www.dodsbirsttr.mil/topics-app/.

## Description

The Defense Threat Reduction Information Analysis Center (DTRIAC) houses a wealth of technical documents and multimedia related to national nuclear projects. Much of this information is still non-digital, and the digitized documents often suffer from structural inconsistencies. This project proposes the development of AI/ML models and pipelines to automate the extraction and processing of document elements, enhancing the fidelity of digitized information and increasing the throughput of scanning processes. The solution will comply with DoD and NIST guidelines, ensuring secure and efficient operations.Key Features:<ul><li>&emsp;Identification and Extraction: Automatically identify and extract elements such as tables, plots, and photos from scientific documents.</li><li>&emsp;Additional Processing: Perform additional processing on extracted elements, such as transposing tables and identifying explicit and implicit axis values in plots.</li><li>&emsp;Automation: Ensure every step &ndash; identification, extraction, and processing &ndash; is fully automated and system agnostic.</li><li>&emsp;Deployment: Develop models and pipelines that are containerized, capable of being hardened, and ready for immediate deployment in a production environment.</li></ul>ATTRIBUTES DESIRED:<ul><li>&emsp;Compatibility: The solution must ingest documents in common formats (e.g., PDF) and output data in standard formats (e.g., JSON, XML).</li><li>&emsp;Robustness: The models must handle digitized documents with readability artifacts and be stable for long-term use.</li><li>&emsp;Modularity: The solution should be modular, pipeline, and system agnostic, and interoperable with other modern software.</li><li>&emsp;Security: All computations must be performed locally with no off-premises resource access, ensuring data security and compliance with DoD and NIST standards.</li></ul>POTENTIAL MARKET:The primary customer for this technology is the Defense Threat Reduction Agency (DTRA). Additionally, the broader U.S. defense sector, including various military branches and defense contractors, can benefit from this technology by automating the extraction and processing of scientific and technical document elements. EXPECTED OUTCOMES:<ul><li>&emsp;Standalone Pipeline: A fully operational pipeline capable of running in an air-gapped environment without internet connectivity.</li><li>&emsp;Comprehensive Data Extraction Tool: A tool that processes scientific documents, extracting and processing elements such as tables and plots, and providing the data in standard, machine-readable formats.</li></ul>

## Objective

The Defense Threat Reduction Agency (DTRA) seeks to develop AI/ML models and pipelines capable of identifying, extracting, and processing elements from scanned technical and scientific documents. This project aims to automate the extraction of tables, plots, photos, and other elements embedded within structured and unstructured text, ensuring high fidelity and accuracy in a production environment.

## Phase I

PHASE I – Proof of Concept:Objective:Define, develop, and determine the feasibility of AI/ML models for identifying, extracting, and processing elements from scientific documents.Required Efforts:<ol><li> Literature Review and Requirements Analysis:<ol><li>Conduct a comprehensive review of existing technologies and methodologies related to document element and layout extraction.</li><li>Identify specific requirements and constraints of DTRIAC's documents.</li></ol></li><li> Development of Initial Models:<ol><li>Element Identification and Extraction: Develop AI/ML models to identify and extract elements like tables, plots, and photos.</li><li>Additional Processing: Implement techniques to process extracted elements (e.g., transposing tables, identifying axis values).</li></ol></li><li> Integration and Testing:<ol><li>Integrate the initial models into a cohesive proof-of-concept pipeline.</li><li>Test the pipeline using a representative sample set of documents.</li><li>Evaluate the performance of the models in terms of accuracy, efficiency, and usability.</li></ol></li><li> Technical Report:<ol><li>Prepare a detailed technical report summarizing the proof of concept.</li><li>Include results from initial testing, highlighting the performance of the models and identifying any areas for improvement.</li><li>Propose a detailed plan for Phase II prototype development, outlining the steps required to refine and expand the models and pipeline.</li></ol></li></ol>Deliverables:<ul><li> Technical Report: A comprehensive report detailing the literature review, development process, testing results, and Phase II plan.</li><li> Proof-of-Concept Pipeline: An initial version of the pipeline demonstrating the feasibility of the proposed solution.</li></ul>&nbsp;

## Phase II

PHASE II – Prototype Development: Objective: Develop, demonstrate, and validate a fully functional prototype of the AI/ML data extraction and processing system, ready for deployment and evaluation. Required Efforts: 1.&nbsp;&nbsp;&nbsp;&nbsp;Refinement and Optimization: &nbsp;&nbsp;&nbsp;&nbsp;o&nbsp;&nbsp;&nbsp;&nbsp;Refine the AI/ML models based on feedback and results from Phase I. &nbsp;&nbsp;&nbsp;&nbsp;o&nbsp;&nbsp;&nbsp;&nbsp;Optimize the models for improved accuracy, efficiency, and scalability. 2.&nbsp;&nbsp;&nbsp;&nbsp;Prototype Development: &nbsp;&nbsp;&nbsp;&nbsp;o&nbsp;&nbsp;&nbsp;&nbsp;Develop a robust and scalable pipeline integrating refined models. &nbsp;&nbsp;&nbsp;&nbsp;o&nbsp;&nbsp;&nbsp;&nbsp;Ensure the pipeline can operate in a standalone, air-gapped environment without internet connectivity. &nbsp;&nbsp;&nbsp;&nbsp;o&nbsp;&nbsp;&nbsp;&nbsp;Implement server-side functionality to support user-side applications, ensuring compatibility with both traditional application-centric and on-premises cloud services environments. 3.&nbsp;&nbsp;&nbsp;&nbsp;Functional and Security Testing: &nbsp;&nbsp;&nbsp;&nbsp;o&nbsp;&nbsp;&nbsp;&nbsp;Conduct comprehensive functional testing to ensure the pipeline accurately identifies, extracts, and processes document elements. &nbsp;&nbsp;&nbsp;&nbsp;o&nbsp;&nbsp;&nbsp;&nbsp;Perform security testing to verify the pipeline's resilience against potential security threats and vulnerabilities. 4.&nbsp;&nbsp;&nbsp;&nbsp;User Interface and Usability Testing: &nbsp;&nbsp;&nbsp;&nbsp;o&nbsp;&nbsp;&nbsp;&nbsp;Develop a user-friendly interface to present the extracted information to end-users. &nbsp;&nbsp;&nbsp;&nbsp;o&nbsp;&nbsp;&nbsp;&nbsp;Conduct usability testing with potential end-users to gather feedback and make necessary improvements. 5.&nbsp;&nbsp;&nbsp;&nbsp;Documentation and Training: &nbsp;&nbsp;&nbsp;&nbsp;o&nbsp;&nbsp;&nbsp;&nbsp;Prepare detailed documentation covering the installation, operation, and maintenance of the prototype. &nbsp;&nbsp;&nbsp;&nbsp;o&nbsp;&nbsp;&nbsp;&nbsp;Provide training materials and sessions for end-users to ensure effective use of the system. Deliverables: •&nbsp;&nbsp;&nbsp;&nbsp;Functional Prototype: A fully operational prototype ready for deployment and evaluation. •&nbsp;&nbsp;&nbsp;&nbsp;Testing Reports: Comprehensive reports detailing the results of functional, security, and usability testing. •&nbsp;&nbsp;&nbsp;&nbsp;Documentation: Complete user manuals, operational guides, and maintenance documentation. •&nbsp;&nbsp;&nbsp;&nbsp;Training Materials: Training resources, including guides and sessions, to support end-users in utilizing the prototype effectively. &nbsp;

## Phase III: Dual Use Applications

PHASE III &ndash; Prototype Refinement:Objective:Refine the AI/ML data extraction and processing system developed in Phase II. Apply the prototype to DTRA-specific scenarios.

Topic

Agency

Program

Topic Link

Additional Information