The Department of Defense, through the Defense Health Agency, is seeking proposals for the development of an outdoor capable perception system for autonomous medical applications, specifically targeting tactical combat casualty care (TC3). The objective is to create a commercially viable hardware and software solution that enhances human-machine teaming by autonomously interpreting sensor data in various operational environments, including challenging lighting and weather conditions. This initiative is crucial for augmenting medical capabilities in military settings, particularly when medical personnel are overwhelmed, allowing for improved casualty care through advanced robotics and automation. Interested parties must submit their proposals by April 23, 2025, with the opportunity to access government-sponsored data sets for algorithm development and testing, as well as potential FDA regulatory strategies for commercialization.

## Description

Novel tools and strategies are required to enable the limited number of trained medical providers in the field to meet the medical capability and capacity demanded by future military engagements. Following the guidance of the Defense Health Agency’s (DHA) Initial Capabilities Documents 1,2 and the Army 2028 Medical Modernization Strategy, 3 investigations into the utilization of artificial intelligence (AI), robotics, automation, and human-technology teaming to augment medical capability and capacity in the pre-hospital setting 4. The rapid evolution of robotics and unmanned systems for military purposes offers a potential new technology that can augment combat casualty care (C3). In situations where medical personnel are overburdened, robotic and autonomous systems can offload physical and cognitive tasks resulting in a force multiplier effect on the battlefield when attending to multiple casualties. This is especially apparent when perception systems provide the autonomous capability to document casualty care, thus freeing attending medics to focus on performing life-saving interventions. Human-Machine Teaming applications provide the possibilities for robotics and autonomous systems (RAS) to assist soldiers and medics in search and rescue in hazardous areas leading autonomous extraction missions. With further robotics development RAS will eventually be able to assist medics in diagnostics and intervention procedures leading to human-machine medical teams capable of care for larger amounts of casualties at once.This topic calls for the development of a novel commercially viable and militarily relevant combined hardware and software solution that enables effective human-machine teaming through passive data collection and scene interpretation to provide a shared context between human provider and assistive technologies including RAS. The solution should be a perception system capable of working in tactical settings, including a variety of lighting conditions (from direct sunlight to low-light operations), and inclement weather environments. This perception system shall autonomously interpret sensor data to establish a baseline understanding of the care context with limited manual input from the care provider to allow human providers to quickly offload tasks to assistive technologies. Inferring care context autonomously presents significant technologies challenges. For example, a significant challenge is the ability to identify and uniquely track individual patients during care of multiple patients. Advances in the state of the art algorithms are expected, including human detection, registration, and tracking of a subject when they come in and out of view; body part segmentation even when part of the body is partially occluded; skeletal tracking for advanced pose estimation; and medical object detection of items found in a medic’s aid bag or in field hospital settings. The solution can utilize a multi-sensor setup and commercial-off-the-shelf (COTS) sensors are allowed but the design is required to be a low SWaP-C (size, weight, power and cost) able to be outfitted on Soldier’s helmets or small unmanned ground vehicles.

## Objective

This open topic is intended for technology proven ready to move directly into Phase II and is accepting Direct to Phase II proposals only. The objective of this topic is to develop a camera/perception hardware and software solution/system capable of enabling human-machine teaming for tactical combat casualty care (TC3), which includes solutions for depth sensing, skeletal tracking, and human three-dimensional (3D) body reconstruction that performs in applicable operational environments including, but not limited to, outdoor use in direct sunlight and low light conditions.

## Phase I

The topic is intended for technology proven ready to move directly into Phase II. Therefore, in lieu of a phase I proof of feasibility proposal, the offeror shall provide foundational documentation of existing early-stage concept technologies in the Phase II submission. The phase II submission should demonstrate the accomplishment of “Phase I like” effort, including a feasibility study in their Phase II submission. This includes the scientific and technical merit of an existing prototype that will demonstrate initial development of software algorithms for depth sensing, human registration, body-part segmentation, skeletal tracking, and medical object detection, should be developed to a point of proof-of-concept. Hardware solutions should be considered and narrowed down to prove initial capability in both bright outdoor and low-light settings. Feasibility documentation of particular interest includes any safety validation data and proof -of-concept that the product is cable of detecting. Additional documentation about the existing resources to be provided for consideration of a Phase II award include:<ul><li>Design of an existing proof-of-concept:<ul><li>Achievable low SWaP-C design approach</li><li>Depth sensing capabilities in indoor &amp; outdoor environments</li><li>Ruggedized, small form factor</li><li>Capable to run on edge computing devices</li><li>Interoperable architecture to integrate with other systems</li></ul></li><li>Proof-of-Concept software demonstrating:<ul><li>Human detection, registration and tracking</li><li>Body part segmentation</li><li>Skeletal tracking</li><li>Medically relevant tools and object detection</li></ul></li></ul>

## Phase II

The goal of Phase II is to develop novel technology approaches that demonstrate unified and integrated component validation in relevant environments focusing on advancing software technologies for human registration and tracking for multi-patient scenarios, body part segmentation, skeletal tracking, and medical object detection.To support this effort, the government sponsor, can provide existing data sets that include both ego-centric and exocentric video of simulated TC3 scenarios and incorporates simulation data that is designed for algorithm development or algorithm testing. To obtain the simulated data, performers will need to complete a government sponsored onboarding process for access determination levels on the data repository. Performers are also encouraged to train algorithms on other relevant datasets, as available. The software should be integrated into an edge compute device that can run without cloud/internet connection at the tactical edge. All software systems should be integrated with the chosen hardware in a hardware + software combined system. This includes creating a ruggedized housing for the sensors and computer hardware in a weather-proof case. Phase II allows for the creation of multiple configurations of the proposed solution. Different configurations can include a separate configuration for UGV mounted sensors and helmet mounted sensors as tradeoffs can be made for helmet mounted configurations to be lighter and more ergonomical for the Soldier. Final demonstrations of the Phase II prototype are requested to be carried out at the Telemedicine and Advance Technology Research Center’s (TATRC) laboratory at Fort Detrick, Maryland.&nbsp;The final solution demonstration will include a government evaluation, using 10 discrete performance criteria.(1) The solution’s ability to successfully detect simulated casualties in various operational environment settings.(2) The systems capability to uniquely identify and track multiple patients during TC3 care scenarios.(3) The means of the system to perform 3D body mapping, body segmentation and skeletal pose tracking reliably.(4) When performed, the capability of the system to identify common TC3 interventions.(5) The system’s ability to determined which patient received an identified medical (TC3) procedure.(6) The system reliability to identify medical objects (tools, consumables, et al) that used when providing patient care(7) The ability of the solution form factor to be either Solider-worn or as a payload on a small UGV.(8) The solutions ability to function without connectivity to the cloud/internal connection(9) The systems capability to withstand operationally relevant weather conditions.(10) The reliability of the system to operate for a minimum of 2 hours without recharging.&nbsp;Parameters such as brightness of environment, distance from subject, and occlusions in the scene will be varied and tested. Operationally relevant elements will also be included such as subjects in camouflaged clothing, body armor, and face paint. With a successful Phase II prototype, the solution should be on the path towards acceptance and inclusion in future DoD operational exercises and experimentation, including, but not limited to Project Convergence efforts. An applicable FDA regulatory strategy outlining requirements for testing from safety and effectiveness should be included.

## Phase III: Dual Use Applications

If the solution is intended to be integrated into medical devices or systems, FDA approval is a goal. Phase III allows for the advancement of the developed solution with developments made to pursue commercialization goals for both civilian and government use. A refined Phase III end state would focus on interoperability to allow the perception system to integrate with existing equipment and allow for further research and development. Phase III provides an opportunity to apply the Phase II development work to specific needs identified by laboratories and program offices at the U.S. Army Medical Research and Development Command (USAMRDC). For example, TATRC as a USAMRDC science and technology laboratory is currently engaged in a variety of research initiatives related to medical applications of RAS and software-based Soldier solutions. Solutions developed under this topic may also result in technology transition to Acquisition Program managers within the DoD such as US Army, PEO Aviation, and Air Force. Phase III provides an opportunity, as well, for additional improvements to the system that enable commercialization for civilian use. This includes adapting the system for first responders in search-and-rescue equipment, modifying it to benefit in-hospital care, and other situations where safe human-robot interaction is desired. By addressing current limitations and incorporating modern AI and sensor technologies, a system can be created that is a versatile and robust machine perception platform, capable of supporting real-world, high-demand applications. This development can also benefit training applications inside and out of the DoD by utilizing the human registration and tracking technology to develop digital twin models of patient encounters to be recorded and analyzed further. The next-generation human body perception system can have the potential to become a powerful foundational technology for a wide range of use cases.

Open Topic for Outdoor Capable Perception System for Autonomous Medical Applications (Direct to Phase II)

SBIR Opportunity Analysis

SBIR Documents

Related SBIR/STTR Opportunities

Opportunity Snapshot

Key Dates