Release Date
August 23rd, 2023
Open Date
September 20th, 2023
Due Date(s)
October 18th, 2023
Close Date
October 18th, 2023
Topic No.


Improved Road Network Extraction Through Reinforcement Learning


Department of DefenseN/A


Type: STTRPhase: Phase IYear: 2023


The Department of Defense (DoD) is seeking proposals for the topic of "Improved Road Network Extraction Through Reinforcement Learning" as part of their Small Business Innovation Research (SBIR) Phase I program. The specific agency involved in this solicitation is the National Geospatial-Intelligence Agency (NGA). The objective of this research is to develop an automated geographic feature extraction system that replicates human performance using reinforcement learning. Currently, the production of foundation feature data vector (FFD) features such as roads and building outlines is a labor-intensive and time-consuming manual process. The proposed technology aims to enhance or replace existing computer vision methods by augmenting them with reinforcement learning to improve completeness and accuracy. The Phase I proposal should focus on demonstrating the feasibility of extracting roads in a small geographic area using reinforcement learning. Phase II will involve developing prototypes for road and building footprint extraction in areas of varying complexity and testing the accuracy improvements over existing methods. The ultimate goal is to fully develop and transition the technology for Department of Defense (DoD) and other commercial feature extraction applications. The project duration and funding specifics can be found on the solicitation agency's website.



OBJECTIVE: Develop the next generation automated geographic feature extraction by replicating human performance through reinforcement learning.

DESCRIPTION: The National Geospatial-Intelligence Agency (NGA) has a requirement to generate foundation feature data vector (FFD) features like roads and building outlines. Today’s FFD production process remains a labor intensive, time consuming manual process, and slow to deliver products that rapidly become outdated. NGA’s traditional production, evaluation, and dissemination cycle takes too long to allow for regularly updated feature data.

Methods for automated feature extraction (AFE) have advanced significantly (Chen, et al., 2022). Most of these computer vision AFE methods rely on convolutional neural network (CNN) interpretation and segmentation of satellite imagery. The reliance on visual interpretation leaves these methods susceptible to incomplete, inaccurate, and non-routable networks because of obstructions (e.g. tree-lined roads) visible in the source imagery CNN-based AFE cannot properly interpret.

This topic will develop new hybrid artificial intelligence (AI) methods to augment computer vision results with reinforcement learning to replicate human decision making. Reinforcement Learning (RL) is a technique in AI for teaching machines to make decisions (Sutton & Barto, 2018) that has been applied to classic video games like Pac Man (DeNero, Klein, & Abbeel, 2023). RL’s central advantage is that it learns sequences of actions similar to the way humans map, such as generating points along a roadway or the side of a structure, to facilitate validation. New techniques for feature tracing similar to how humans would approach digitizing should significantly increase vector data production and accuracy from imagery.

RL has been further applied to automated and semi-automated feature extraction to refine pixel-wise segmented vector data directly (Botteghi, Sirmacek, & Unsalan, 2021). Proposals for this topic should address a similar hybrid approach for new methods to enhance or replace deep learning/computer vison only approaches, augmented with decision making to improve completeness. The method should scale to city or regional scales across diverse geographic context and environments. NGA expects that software prototyping/development under Phase II will be subject to export controls as described under ECCN 0D521 in the 0Y521 series table found in Supplement No. 5 to part 774 of the Export Administration Regulations (EAR). Phase II work most likely be conducted on NIST SP 800-171 compliant information systems.

PHASE I: The Phase I proposal should focus on demonstrating the feasibility of extracting roads in at least one small geographic area using an RL approach. Phase I will also provide a concept of operations to build and deploy at larger scales with associated costs.

PHASE II: Phase II will focus on developing a mature capability creating two (2) prototypes, one for roads and a second expanding to building footprint extraction, both in geographic areas of varying complexity (e.g. urban and rural, canopy vs. desert). Phase II will also test the accuracy improvements over existing computer vision-only methods.

PHASE III DUAL USE APPLICATIONS: Fully develop and transition the technology and methodology based on the research and development results developed during Phase II for DOD and other commercial feature extraction applications.


  1. Botteghi, N., Sirmacek, B., & Unsalan, C. (2021). RLSNAKE: A Hybrid Reinforcement Learning Approch for Road Detection. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B3-2021. doi:;
  2. Chen, Z., Deng, L., Luo, Y., Li, D., Marcato Junior, J., Nunes Goncalves, W., . . . Li, D. (2022). Road extraction in remote sensing data: A survey. International Journal of Applied Earth Observations and Geoinformation, 112. doi:
  3. DeNero, J., Klein, D., & Abbeel, P. (2023, May 22). UC Berkeley CS188 Intro to AI. Retrieved from;

KEYWORDS: Reinforcement Learning; Artificial Intelligence (AI); Satellite Imagery; Automated Feature Extraction (AFE)