The Department of Defense, through DARPA, is seeking proposals for the Small Business Innovation Research (SBIR) program focused on "Analyzing the Trade-off Between Bias and Accuracy" in artificial intelligence (AI) systems. The objective is to develop novel methods for measuring and validating the trade-offs between accuracy and fairness in AI models, particularly in DoD-relevant use cases. This initiative is critical as it addresses the pervasive issue of bias in AI, which impacts both government and industry applications, including the de-biasing of generative models. Proposals for Phase I are due by February 26, 2025, with the opportunity opening on January 29, 2025, and further details can be found at the official solicitation link: https://www.dodsbirsttr.mil/topics-app/.

## Description

There are well-studied trade-offs between accuracy and fairness of AI models. For example, some degree of fairness may be sacrificed in terms of misclassifying an irrelevant attribute of a patient to optimize for the accuracy of predicting whether that patient has a terminal illness. Performers should propose novel methods for measuring the bias-accuracy trade-offs and demonstrate the utility of these methods in DoD relevant use cases.&nbsp;While the measurement of accuracy is intuitive, measuring fairness is a more nebulous endeavor. Some examples in the literature include penalizing attribute disparities as a regularization term in the loss function of the model or training LLMs using RLHF with the human rankings of the LLM outputs being based on their subjective valuation of fairness. Under this topic, performers must specify what their notions of fairness will be and how those will be made manifest in the training or inference of the models while analyzing the tradeoff incurred on accuracy.&nbsp;To address the foregoing technical challenges performers in this effort will:•&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Develop novel methods of constructing AI systems that enable flexible trade off in “fairness” as appropriate for a given use case.•&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Develop methods to measure and validate this trade-off in a relevant use case given by the proposer.

## Objective

To develop new methods to analyze the trade-off between accuracy and bias in Artificial Intelligence (AI).

## Phase I

The goal of Phase I proposals is to present a new technology to address AI bias as described previously. The technology need not be mature by the end of the phase, but a convincing proof-of-concept for its utility must be demonstrated. This proof-of-concept may come in the form of a live demo, publications in peer-reviewed venues, and open-source software, among others. Phase I deliverables and milestones for this STTR should include: <ul><li>&emsp;Month 3 : report detailing technical progress made to date and tasks accomplished. </li><li>&emsp;Month 6: finalize the technical report, including remaining challenges directions to be addressed, a tentative plan for future work, and lessons learned. </li></ul>

## Phase II

Develop, install, integrate and demonstrate a prototype system determined to be the most feasible solution during the Phase I feasibility study. This demonstration should focus specifically on: <ol><li>&emsp;Validating the product-market fit between the proposed solution and the proposed topic and define a clear and immediately actionable plan for running a trial with the proposed solution and the proposed customer.</li><li>&emsp;Evaluating the proposed solution against the objectives and measurable key results as defined in the Phase I feasibility study.</li><li>&emsp;Describing in detail how the solution can be scaled to be adopted widely (e.g., how can it be modified for scale).</li><li>&emsp;A clear transition path for the proposed solution that takes into account input from all affected stakeholders including, but not limited to: end users, engineering, sustainment, contracting, finance, legal, and cyber security.</li><li>&emsp;Specific details about how the solution can integrate with other current and potential future solutions.</li><li>&emsp;How the solution can be sustainable (i.e. supportability).</li><li>&emsp;Clearly identifying other specific DoD or governmental customers who want to use the solution.</li></ol>

## Phase III: Dual Use Applications

The contractor will pursue commercialization of the various technologies developed in Phase II for transitioning expanded mission capability to a broad range of potential government and civilian users and alternate mission applications. Interested government end users may include the Air Force, the DoD Chief Digital and AI Office (CDAO), DARPA, White House Office of Science and Tech Policy (OSTP), Dept of Education, Dept of Commerce, and NIST, all of whom have been looking at the problem of detecting and mitigating bias in AI as part of an inter-agency working group. For example, mitigating bias in one of the DoD&rsquo;s responsible AI principles and it is widely recognized that bias remains a hurdle for responsible AI adoption. Bias also remains a hurdle for operational AI adoption, to ensure robustness of AI to rare and unlikely events. Of course, these problems also affect and are pervasive in industry, thereby motivating the dual use of the proposed technologies. Example industrial applications include the de-biasing of generative models, which have been shown to both reflect inherent racial biases, but also to create new biases as a result of current de-biasing techniques.Direct access with end users and government customers will be provided with opportunities to receive Phase III awards for providing the government additional research and development, or direct procurement of products and services developed in coordination with the program.

PDF.html

Please provide the text you would like me to summarize.

Analyzing the Trade-off Between Bias and Accuracy

STTR Opportunity Analysis

SBIR Documents

Related SBIR/STTR Opportunities

Opportunity Snapshot

Key Dates