Satellite-based remote sensing imagery is an effective means for detecting objects and structures in support of many applications. However, detecting the spatial and temporal bounds of a specific activity in satellite imagery is inherently more complex and research in this area is nascent. One reason for this is that describing an activity implies defining both spatial and temporal bounds and while activity is inherently continuous in nature, the geospatial (imagery) time series for any particular swath of ground provided by satellite imagery is relatively sparse and discrete in comparison. The IARPA Space-Based Machine Automated Recognition Technique (SMART)1 program is the first large-scale research program to target advancing the state of the art for automatically detecting, characterizing, and monitoring large-scale anthropogenic activity in global, multispectral satellite imagery. The program has two primary research objectives: 1) the “harmonization” of multiple imagery sources and 2) automated reasoning at scale to detect, characterize, and monitor activities of interest. This paper provides details on the goals, dataset, metrics, and lessons learned of the IARPA SMART program. By releasing the annotated dataset, the program aims to foster additional research in this area by the community at large.
Naval intelligence plays a critical role in multi-domain operations by identifying and tracking vessels of interest, especially suspected “dark ships” operating in an emissions-controlled (EMCON) state. While applying machine learning (ML) to maritime satellite imagery could enable an automated open-ocean search capability for dark ships, ensuring the robustness of ML models to environmental variations in the maritime domain remains a challenge because training sets do not encapsulate all possible environmental conditions. To address the challenge of unsupervised domain adaptation (UDA) in ship classification, i.e. transferring a ML model from a labeled source domain to an unlabeled target domain, we propose employing combinations of semi-supervised learning (SSL) techniques with standalone UDA approaches. Specifically, we incorporate combinations of FixMatch, minimum class confusion, gradient reversal, and mixup augmentation into the standard cross-entropy supervised loss function. These interventions were compared in two domain shift settings, one in which the source and target domains are both comprised of simulated data, and another in which the source domain consists of only simulated data, and the target domain consists of only real data. Experimental results comparing the combinations of interventions to a regularized fine-tuning baseline demonstrate that the greatest improvements in model robustness were achieved when combinations of our SSL strategy (FixMatch) and UDA algorithms were incorporated into training.
Recent breakthroughs and rapid progress in AI will impact, if not transform, every mission. JHU/APL developed an AI Technology Roadmap to guide the Laboratory’s contributions to the critical challenges the nation will face developing and implementing intelligent systems for these missions over the coming decades. We began this exercise by describing a series of envisioned futures for intelligent systems across sea, land, air, space and information, and examined them to identify the common AI technology vectors needed to achieve each vision: (1) Autonomous Perception, describing the path to intelligent systems that perceive in the context of the extreme uncertainty and complexity of the real world; (2) Superhuman Decision-Making and Autonomous Action, to realize the potential for intelligent systems to reason over more information than any team of analysts or operators and act in ways systems under manned control cannot; (3) Human-Machine Teaming at the Speed of Thought, to ensure humans can stay involved at speed and scale; and of particular importance for national security applications, (4) Safe and Assured Operation, so these systems can be trusted to stay true to commander’s intent, in adversarial and sensitive contexts. Each technology vector is aligned with a targeted goal, and with each goal we provide a roadmap in the form of near-, mid-, and long-term AI advances critical to reaching the goal. This paper describes the JHU/APL AI Technology Roadmap and presents key examples of recent progress and forward-looking research and exploratory development along each vector.
The ability to perform remote forensics in situ is an important application of autonomous undersea vehicles (AUVs). Forensics objectives may include remediation of mines and/or unexploded ordnance, as well as monitoring of seafloor infrastructure. At JHU/APL, digital holography is being explored for the potential application to underwater imaging and integration with an AUV. In previous work, a feature-based approach was developed for processing the holographic imagery and performing object recognition. In this work, the results of the image processing method were incorporated into a Bayesian framework for autonomous path planning referred to as information surfing. The framework was derived assuming that the location of the object of interest is known a priori, but the type of object and its pose are unknown. The path-planning algorithm adaptively modifies the trajectory of the sensing platform based on historical performance of object and pose classification. The algorithm is called information surfing because the direction of motion is governed by the local information gradient. Simulation experiments were carried out using holographic imagery collected from submerged objects. The autonomous sensing algorithm was compared to a deterministic sensing CONOPS, and demonstrated improved accuracy and faster convergence in several cases.
The ability to autonomously sense and characterize underwater objects in situ is desirable in applications of unmanned underwater vehicles (UUVs). In this work, underwater object recognition was explored using a digital holographic system. Two experiments were performed in which several objects of varying size, shape, and material were submerged in a 43,000 gallon test tank. Holograms were collected from each object at multiple distances and orientations, with the imager located either outside the tank (looking through a porthole) or submerged (looking downward). The resultant imagery from these holograms was preprocessed to improve dynamic range, mitigate speckle, and segment out the image of the object. A collection of feature descriptors were then extracted from the imagery to characterize various object properties (e.g., shape, reflectivity, texture). The features extracted from images of multiple objects, collected at different imaging geometries, were then used to train statistical models for object recognition tasks. The resulting classification models were used to perform object classification as well as estimation of various parameters of the imaging geometry. This information can then be used to inform the design of autonomous sensing algorithms for UUVs employing holographic imagers.
KEYWORDS: General packet radio service, LIDAR, Land mines, Detection and tracking algorithms, Antennas, Metals, Target detection, Sensors, Prototyping, Global Positioning System
Vehicle-mounted ground-penetrating radar (GPR) has proved to be a valuable technology for buried threat
detection, especially in the area of military route clearance. However, detection performance may be degraded in
very rough terrain or o-road conditions. This is because the signal processing approaches for target detection
in GPR rst identify the ground re
ection in the data, and then align the data in order to remove the ground
re
ection. Under extremely rough terrain, antenna bounce and multipath eects render nding the ground
re
ection a dicult task, and errors in ground localization can lead to data alignment that distorts potential
target signatures and/or creates false alarms. In this work, commercial-o-the-shelf light detection and ranging
(LIDAR), global positioning system (GPS), and inertial measurement unit (IMU) were integrated with a GPR
into a prototype route clearance system. The LIDAR provided high-resolution measurements of the ground
surface prole, and the GPS/IMU recorded the vehicle's position and orientation. Experiments investigated
the applicability of the integrated system for nding the ground re
ection in GPR data and decoupling vehicle
motion from the rough surface response. Assessment of ground-tracking performance was based on an experiment
involving three prepared test lanes, each with dierent congurations of buried targets and terrain obstacles.
Several algorithms for target detection in GPR were applied to the data, both with traditional preprocessing and
incorporating the LIDAR and IMU. Experimental results suggest that the LIDAR and IMU may be valuable
components for ground tracking in next-generation GPR systems.
Roadside explosive threats continue to pose a significant risk to soldiers and civilians in conflict areas around the world.
These objects are easy to manufacture and procure, but due to their ad hoc nature, they are difficult to reliably detect
using standard sensing technologies. Although large roadside explosive hazards may be difficult to conceal in rural
environments, urban settings provide a much more complicated background where seemingly innocuous objects (e.g.,
piles of trash, roadside debris) may be used to obscure threats. Since direct detection of all innocuous objects would flag
too many objects to be of use, techniques must be employed to reduce the number of alarms generated and highlight only
a limited subset of possibly threatening regions for the user. In this work, change detection techniques are used to
reduce false alarm rates and increase detection capabilities for possible threat identification in urban environments. The
proposed model leverages data from multiple video streams collected over the same regions by first applying video
aligning and then using various distance metrics to detect changes based on image keypoints in the video streams. Data
collected at an urban warfare simulation range at an Eastern US test site was used to evaluate the proposed approach, and
significant reductions in false alarm rates compared to simpler techniques are illustrated.
KEYWORDS: General packet radio service, Land mines, Detection and tracking algorithms, Data fusion, Feature extraction, Metals, Performance modeling, Ground penetrating radar, Sensors, Soil science
Ground-penetrating radar (GPR) is a very useful technology for buried threat detection applications which is
capable of identifying both metallic and non-metallic objects with moderate false alarm rates. Several pattern
classication algorithms have been proposed and evaluated which enable GPR systems to achieve robust per-
formance. However, comparisons of these algorithms have shown that their relative performance varies with
respect to the environmental context under which the GPR is operating. Context-dependent fusion has been
proposed as a technique for algorithm fusion and has been shown to improve performance by exploiting the
dierences in algorithm performance under dierent environmental and operating conditions. Early approaches
to context-dependent fusion clustered observations in the joint condence space of all algorithms and applied
fusion rules within each cluster (i.e., discriminative learning). Later approaches exploited physics-based fea-
tures extracted from the background data to leverage more environmental information, but decoupled context
learning from algorithm fusion (i.e., generative learning). In this work, a Bayesian inference technique which
combines the generative and discriminative approaches is proposed for physics-based context-dependent fusion
of detection algorithms for GPR. The method uses a Dirichlet process (DP) mixture as a model for context, and
relevance vector machines (RVMs) as models for algorithm fusion. Variational Bayes is used as an approximate
inference technique for joint learning of the context and fusion models. Experimental results compare the pro-
posed Bayesian discriminative technique to generative techniques developed in past work by investigating the
similarities and dierences in the contexts learned as well as overall detection performance.
KEYWORDS: General packet radio service, Data modeling, Land mines, Detection and tracking algorithms, Process modeling, Performance modeling, Ground penetrating radar, Feature selection, 3D modeling, Data processing
In landmine detection applications, fluctuation of environmental and operating conditions can limit the performance
of sensors based on ground-penetrating radar (GPR) technology. As these conditions vary, the classification
and fusion rules necessary for achieving high detection and low false alarm rates may change. Therefore,
context-dependent learning algorithms that exploit contextual variations of GPR data to alter decision rules have
been considered for improving the performance of landmine detection systems. Past approaches to contextual
learning have used both generative and discriminative methods to learn a probabilistic mixture of contexts, such
as a Gaussian mixture, fuzzy c-means clustering, or a mixture of random sets. However, in these approaches the
number of mixture components is pre-defined, which could be problematic if the number of contexts in a data
collection is unknown a priori. In this work, a generative context model is proposed which requires no a priori
knowledge in the number of mixture components. This was achieved through modeling the contextual distribution
in a physics-based feature space with a Gaussian mixture, while also incorporating a Dirichlet process prior
to model uncertainty in the number of mixture components. This Dirichlet process Gaussian mixture model
(DPGMM) was then incorporated in the previously-developed Context-Dependent Feature Selection (CDFS)
framework for fusion of multiple landmine detection algorithms. Experimental results suggest that when the
DPGMM was incorporated into CDFS, the degree of performance improvement over conventional fusion was
greater than when a conventional fixed-order context model was used.
KEYWORDS: Video, General packet radio service, Roads, Land mines, Cameras, Video processing, Explosives, Feature extraction, Ground penetrating radar, Sensors
Forward looking video can provide a large amount of tactically-relevant information to vehicle operators regarding
roadside explosive threats. However it is difficult for vehicle operators to keep track of what roadside objects have
changed since their last excursion, or what new objects have appeared on a road and might therefore contain an
explosive threat. Furthermore, the large amount of data generated by forward looking video can overwhelm users. It
would be of benefit to vehicle operators if only objects that had significantly changed since a recent excursion were
flagged and presented to the user. In this work we develop techniques for video and ground-penetrating radar (GPR)
tracking and aligning, and novel-object identification for application in route clearance patrols. We focus on aligning
video data collected using vehicle mounted forward-looking video cameras and downward-looking GPR using locallyinvariant
feature transforms and set-based distance metrics. Based on these aligned image streams, we then apply
pattern classification approaches to discriminate new explosive threats from stationary and persistent objects. The
techniques described in this work are widely applicable to other forward and downward-looking sensor systems, and are
computationally tractable. The results indicate the potential to robustly identify recently changed roadside threats, and to
present a significantly reduced amount of information to end-users for further operational analysis.
It has been established throughout the ground-penetrating radar (GPR) literature that environmental factors
can severely impact the performance of GPR sensors in landmine detection applications. Over the years, electromagnetic
inversion techniques have been proposed for determining these factors with the goal of mitigating
performance losses. However, these techniques are often computationally expensive and require models and responses
from canonical targets, and therefore may not be appropriate for real-time route-clearance applications.
An alternative technique for mitigating performance changes due to environmental factors is context-dependent
classification, in which decision rules are adjusted based on contextual shifts identified from the GPR data. However,
analysis of the performance of context-dependent learning has been limited to qualitative comparisons of
contextually-similar GPR signatures and quantitative improvement to the ROC curve, while the actual information
extracted regarding soils has not been investigated thoroughly. In this work, physics-based features of GPR
data used in previous context-dependent approaches were extracted from simulated GPR data generated through
Finite-Difference Time-Domain (FDTD) modeling. Statistical techniques where then used to predict several potential
contextual factors, including soil dielectric constant, surface roughness, amount of subsurface clutter,
and the existence of subsurface layering, based on the features. Results suggest that physics-based features of
the GPR background may contain informatin regarding physical properties of the environment, and contextdependent
classification based on these features can exploit information regarding these potentially-important
environmental factors.
KEYWORDS: General packet radio service, Land mines, Feature extraction, Data modeling, Autoregressive models, Feature selection, Fourier transforms, Detection and tracking algorithms, Soil science, Reflection
Context-dependent classification techniques applied to landmine detection with ground-penetrating radar (GPR)
have demonstrated substantial performance improvements over conventional classification algorithms. Context-dependent
algorithms compute a decision statistic by integrating over uncertainty in the unknown, but probabilistically
inferable, context of the observation. When applied to GPR, contexts may be defined by differences
in electromagnetic properties of the subsurface environment, which are due to discrepancies in soil composition,
moisture levels, and surface texture. Context-dependent Feature Selection (CDFS) is a technique developed for
selecting a unique subset of features for classifying landmines from clutter in different environmental contexts.
In past work, context definitions were assumed to be soil moisture conditions which were known during training.
However, knowledge of environmental conditions could be difficult to obtain in the field. In this paper, we utilize
an unsupervised learning algorithm for defining contexts which are unknown a priori. Our method performs unsupervised
context identification based on similarities in physics-based and statistical features that characterize
the subsurface environment of the raw GPR data. Results indicate that utilizing this contextual information
improves classification performance, and provides performance improvements over non-context-dependent approaches.
Implications for on-line context identification will be suggested as a possible avenue for future work.
KEYWORDS: General packet radio service, Autoregressive models, Land mines, Feature extraction, Feature selection, Detection and tracking algorithms, Environmental sensing, Data modeling, Antennas, Ground penetrating radar
We present a novel method for improving landmine detection with ground-penetrating radar (GPR) by utilizing
a priori knowledge of environmental conditions to facilitate algorithm training. The goal of Context-Dependent
Feature Selection (CDFS) is to mitigate performance degradation caused by environmental factors. CDFS
operates on GPR data by first identifying its environmental context, and then fuses the decisions of several
classifiers trained on context-dependent subsets of features. CDFS was evaluated on GPR data collected at several
distinct sites under a variety of weather conditions. Results show that using prior environmental knowledge in
this fashion has the potential to improve landmine detection.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.