The Department of Defense, specifically the United States Air Force, is seeking proposals for the development of a cockpit-compatible eye tracking system aimed at enhancing pilot performance during military aviation operations. The objective is to create a system that can monitor eye gaze dynamics, saccades, and pupil physiology in real-time, integrating seamlessly with existing Air Force airframes and pilot headgear, while addressing unique flight-specific environmental challenges. This initiative is critical for improving cognitive state assessments, such as fatigue and workload, which are essential for operational effectiveness in military contexts. Proposals are due by June 25, 2025, with the opportunity being part of the SBIR program, and further details can be found at the provided source link.

## Description

Recent efforts in the Air Force have emphasized measuring and modeling of the human operator&rsquo;s functional and cognitive status during military aviation, in order to drive human performance enhancement and gain a decisive advantage in the future fight.&nbsp; Psychophysiological methods &ndash; including technology to monitor changes in human physiological variables that correlate with psychological states &ndash; are widely used to monitor an individual&rsquo;s functional status and to infer internal cognition.&nbsp; Extensive work in academia, medicine, and military research labs has demonstrated the promise of monitoring the human operator via robust field-deployable biosensors (e.g., using neural activity, heart rate, pupil dilation)&nbsp; to infer an individual&rsquo;s cognitive state in real-time/semi-real time (Chakladar &amp; Roy, 2024; Charles &amp; Nixon, 2019; Wilbanks et al., 2021). However, state-of-the-art sensor technologies in several biosensing modalities are unsuitable for sustained operational use during military aviation (e.g., electroencephalogram, EEG). Further, physiological sensing modalities achievable during flight (e.g., heart rate, respiration, skin conductance) often do not covary with cognition at the &ldquo;speed of thought,&rdquo; lacking the temporal variability required for high frequency, real-time decoding of internal cognitive states. &nbsp;&nbsp;&nbsp;Measuring and decoding cognitive states during military operations in real-time and in the cockpit will require technology for inferring large-scale patterns of central nervous system (CNS) activity, motivating development of high-quality systems for field-deployable eye tracking. As the adage goes, the eyes are a window to the mind. Ample evidence suggests that eye and pupil parameters vary at the &ldquo;speed of thought&rdquo; and are highly correlated with various cognitive states: ranging from arousal, attention, working-memory, cognitive control, and relational long-term memory (Mahanama et al., 2022). State-of-the-art eye tracking systems (both wearable and off-body) have begun to transition this technology to automotive (Said, et al., 2018), gaming (Zhang et al., 2024), healthcare (Liu et al., 2021), and educational settings (Strohmaier et al., 2020). Current software and hardware limitations for such systems can include the need for operator setup and (re)calibration, the impact of head and body motion on signal quality, the impact of changes in ambient light on image quality and pupil diameter, near-infrared light interference, reliability and transparency of underlying processing algorithms, and on-line generation of gaze and pupil physiological data. These challenges are amplified in a cockpit as additional equipment is donned by the human operator and environmental variables are introduced (such as altitude, cabin pressure, vibrations, or G-forces). Further, workable solutions for the Department of Defense (DoD) must address these challenges by making design choices from a human-factors engineering perspective that attends to the realities and ergonomics of military aviation, including robustness, reliability, user comfort, and cockpit ergonomics. Specifically, the eye tracking system must be designed to be operational with helmets required by most Air Force aircraft both with and without an additional visor.&nbsp; If principles of universal design are invoked, the eye tracking system may also be operable outside of the cockpit on additional aircrew not required to wear helmets (e.g., medical flight crew). This SBIR therefore seeks to develop an eye-tracking system that retains the functionality of similar lab-grade eye-tracking systems and can deploy these capabilities during military aviation. The system must be able to output real-time measures of gaze/saccade and pupil physiology, be capable of recording these data for post-hoc processing and analysis, and sample at sufficient frequency to guide real-time inferences about pilot&rsquo;s internal cognitive states (hardware/software requirements).&nbsp; Additionally, the developed system should use an open-architecture framework that can easily interface with third-party software and hardware (open-architecture requirement).&nbsp; The hardware/software and open-architecture requirements are the focus of this SBIR, however implementations of algorithms to transparently and accurately assess an individual&rsquo;s cognitive state, such as fatigue, workload, stress, etc.,are of potential interest and may be evaluated for relevance to the Air Force. References:&nbsp; Debashis Das Chakladar, &amp; Partha Pratim Roy. (2023). Cognitive workload estimation using physiological measures: A review. Cognitive Neurodynamics. https://doi.org/10.1007/s11571-023-10051-3 Charles, R. L., &amp; Nixon, J. (2019). Measuring mental workload using physiological measures: A systematic review. Applied Ergonomics, 74, 221&ndash;232. https://doi.org/10.1016/j.apergo.2018.08.028 &nbsp;Mahanama, B., Jayawardana, Y., Rengarajan, S., Jayawardena, G., Chukoskie, L., Snider, J., &amp; Jayarathna, S. (2022). Eye Movement and Pupil Mea

## Objective

Develop an eye-tracking system for pilots capable of integrating with Air Force airframes during training and military operations. Such a system should be capable of monitoring pilot eye gaze dynamics, saccades, and pupil physiology in real-time/semi-real time via hardware that meets all airworthiness standards, per MIL-HDBK-516. for integration into military aircraft.

## Phase I

Phase 1 should focus on concept development and technical feasibility. The resulting proposal should completely document:<ol><li>The proposed approach to implementing a cockpit compatible eye tracking system, including assessment of and solution for flight-specific environmental challenges to eye tracking.</li><li>The design of the proposed system, including a description of any proposed modifications to existing eye tracking systems as well as the technical specifications for each camera/optical sensing component in the eye tracking system.</li><li>The interoperability of the proposed eye tracking system with various required piloting headgear, including the HGU-55/P helmet by Gentex worn by most Air Force pilots and associated, modular components like visors.</li><li>If available, examination of preexisting data collected in dynamic, naturalistic, outdoor environments using eye tracking.</li><li>Plans for a relevant hardware and open software API that allows users to interface with raw or processed eye tracking data.</li><li>Plans for a small form-factor (i.e., pilot wearable) processing computer and data storage device to process real-time eye tracking data and store for later use or stream for online integration with other sensors.</li><li>Plans for testing the functionality of the system in a systematic fashion, stepping up from simulation, phantom testing, to human subjects testing. Include demonstrated capability to gain human subjects research protocol approval, conduct a study, and analyze data in 24 months.&nbsp;</li></ol>

## Phase II

Performers will develop and demonstrate a prototype eye tracking system. This prototype shall demonstrate 1) ability to output eye tracking data of sufficient fidelity under flight-like conditions, and 2) a flexible ergonomic form factor compatible with various aviation equipment (helmets, corrective lenses, respirators, night vision googles/devices, etc.)The ability to output eye tracking data of sufficient fidelity under flight-like conditions will depend on a high sampling rate of eye and gaze cameras (&gt;200 Hz), the ease of use for the operator to setup and (re)calibrate the device quickly (&lt; 2 minutes), the ability of the device to mitigate the impact of head and body motion on signal quality, and the ability of the device to adjust to various ambient lighting conditions while maintaining high image quality. Online processing of data must be conducted in real-time generating online gaze and eye physiology data), to either record to the device or stream over a native API.&nbsp;&nbsp; Recording high-fidelity data will enable development of methods to transparently and accurately assess fatigue, workload, stress etc.&nbsp;Specifically, the eye tracking system must be operational while pilots are wearing a helmet and also wearing a visor. Additional form factors regarding helmets without visors and operability on non-helmet wearing aircrew will also be evaluated, though not required.&nbsp;A flexible and ergonomic form factor will depend on user comfort level from wearing the device, its suitability for use in a military cockpit, and interoperability with aircrew flight equipment.&nbsp;Schedule/Milestones/Deliverables:Month 1: Report on product development project plan that adapts existing technology as much as possible or develops a new platform if necessary.&nbsp; IRB and HRPO approvals or data collection effort enrollment when approvals obtained.Month 3: Report on: Progress toward month 6 goals; IRB and HRPO approvals or data collection effort enrollment when approvals obtained.Month 6: Report on: Month 6 demonstration; IRB and HRPO approvals or data collection effort enrollment when approvals obtained.Month 9: Report on: Progress toward month 12 goals; IRB and HRPO approvals or data collection effort enrollment when approvals obtained.Month 12: Report on: Month 12 demonstration; IRB and HRPO approvals or data collection effort enrollment when approvals obtained.Month 15: Report on: Progress toward month 18 goals; IRB and HRPO approvals or data collection effort enrollment when approvals obtained.Month 18: Report on: Month 18 demonstration; Performers must show performance enhancement benefit using prototype neural interface; IRB and HRPO approvals or data collection effort enrollment when approvals obtained.Month 21: Report on: Progress towards month 24 goals enrollment; IRB and HRPO approvals or data collection effort enrollment when approvals obtained.Month 24: Report on: Month 24 demonstration: Performers must show usability and performance benefit in finalized form factor.&nbsp;

## Phase III: Dual Use Applications

The performer will pursue further generalization and pursue airworthiness approvals of the technology developed, with the aim of transition to a working commercial or warfighter solution that can be operated by non-experts. Special attention should be paid to MIL-STD-810G, &ldquo;Department of Defense Test Method Standard: Environmental Engineering Considerations and Laboratory Tests&rdquo; and the latest version of MIL-HDBK-516C, &ldquo;Department of Defense Handbook: Airworthiness Certification Criteria.&rdquo;&nbsp;

Topic

Agency

Program

Topic Link

Additional Information