We report on processing techniques to effectively control the data bandwidth in larger format Focal Plane Array (FPA) sensors. We have developed an image processing architecture for variable acuity FPAs that give a controlled reduction in the data rate via simple circuits that estimate activity on the FPA image plane. Integrated on-FPA signal processing goals are to perform pre-processing that is usually performed downstream in a dedicated processing module. Techniques for image pre-processing described in this paper allow transmitting "active" pixel data while skipping unchanging pixels. These techniques for image pre-processing adjacent to the FPA allows significant reductions in the data rate, size, weight and power for small and low cost systems that cannot work with a large image processing.
We report on processing techniques to effectively control the data bandwidth in larger format Focal Plane Array (FPA)
sensors. Bandwidth reduction techniques are possible using image processing functions near or on the FPA. We have
developed image processing techniques that give a controlled reduction in the data rate via simple circuit that estimate
Activity on the FPA image plane. By filtering on the FPA and sensing pixel signal changes, this allows only transmitting
pixel data that are have less interest, and more efficiently manages data flow from the FPA. We describe and
demonstrate results of "Activity Sensing" processing near the FPA. The associated computational efficiency with this
type of on FPA processing allows data rate reduction from > 20 Mbytes/sec to under 10Kbytes/sec . We report on the
continuing development and performance benefits expected from an Activity Sensing algorithm using recorded infrared
(IR) data from a large format 1024 x 1024 variable acuity1 FPA.
In our previous papers, the FPGA-based processing package and the co-processor board have been introduced for numerous commercial and military applications including motion detection, optical flow, background velocimetry, and target tracking. The processing package is being continually upgraded by new point- and area-applied algorithms for a variety of real-time digital video camera systems including foveal sensors based on Nova's Variable Acuity Superpixel Imager (VASITM) and Large Format VASITM (LVASITM) technologies. This paper demonstrates the FPGA-based processor for high frame-rate target detection in a cluttered background using variable acuity sensors. For the 1024 x 1024 pixel LVASITM Focal Plane Array (FPA), the proposed target-detection algorithm increases the frame rate from 4 Hz for the full resolution mode up to 450 Hz for the foveal mode while maintaining full field of view and target-detection performances on cluttered backgrounds that are comparable with detection performances at the full resolution mode.
We report on recently developed algorithms and architectures capable of point source target detection near or on the FPA. The goals of this work are to demonstrate image processing functions near or on the FPA in a manner efficient enough to allow hardwired algorithms for Camera Systems on a Chip (SOC) implementation. These SOCs have the potential to improve the size and power requirements for existing IR sensor systems which require larger board sets and hardware enclosures. We report on the algorithm development for hardwired target detection algorithms using recorded IR Data.
Cyan Systems has developed algorithms and architectures capable of performing temporal and spatial filtering near or on the FPA. Cyan is performing research into more advanced techniques to allow functional target detection near/on the FPA. The goals of this work is to perform image processing near or on the FPA to improve the size and power requirements for existing IR sensor systems which require larger board sets and hardware enclosures. We use representations of the biological vision system as models for the algorithm development. We report measured data on the near/on FPA target detection performance.
Nova Sensors produces miniature electronics for a variety of real-time digital video camera systems, including foveal sensors based on Nova's Variable Acuity Superpixel Imager (VASITM) technology. An advanced image-processing package has been designed at Nova Sensors to re-configure the FPGA-based co-processor board for numerous applications including motion detection, optical, background velocimetry and target tracking. Currently, the processing package consists of 14 processing operations that cover a broad range of point- and area-applied algorithms. Flexible FPGA designs of these operations and re-programmability of the processing board allows for easy updates of the VASITM sensors, and for low-cost customization of VASITM sensors taking into account specific customer requirements.
This paper describes the image processing algorithms implemented and verified in Xilinx FPGAs and provides the major technical performances with figures illustrating practical applications of the processing package.
KEYWORDS: Image processing, Digital signal processing, Video, Cameras, Imaging systems, Video processing, Staring arrays, Electronics, Sensors, Field programmable gate arrays
Programs at Nova Biomimetics have led to the design and development of a set of miniature electronics to be used for the application of a wide variety of point- and area-type mathematical operations to be applied in real time to the digital data produced by a variety of real-time digital video camera systems. Nova is planning to market these electronics in partial satisfaction of Small Business Innovation Research (SBIR) Program dual-use commercialization requirements.
Nova's development of the "Modular Infrared Imaging Applications Development System" (MIRIADS) produced a longwave infrared (LWIR) camera system that operated the "Adaptive Infrared Sensor" (AIRS) focal plane device produced by the Raytheon Infrared Operations (RIO) organization. A novel system architecture permitted the integration of an infrared fisheye lens system produced by Optics 1, Inc., which permitted a complete hemispherical field of view to be imaged onto the AIRS FPA. This paper will describe applications for this system as an extremely wide field-of-view IR sensor (early warning detection, fire detection, etc.), and will present test imagery collected with the system.
This technology advancement has been the result of the coordinated effort of a variety of companies and government agencies. This presentation will highlight significant contributions of individuals and will indicate the effectiveness of the Small Business Innovative Research (SBIR) program in helping to advance this nation's technology base.
Nova Research, Inc. has developed a novel two-dimensional imaging chip whose design is based on properties exhibited by biological retinas. The 'Variable Acuity' imager permits the user to program a unique spatial arrangement of 'superpixels' that may be updated in real time. Any spatial configuration of pixels in the imager may be realized by programming the device in a way that permits pixels to share their individually-collected photocharge with any or all neighbors. Single and multiple 'foveal' configurations are possible, and these high spatial resolution regions may be 'flown' around the FPA at the will of the controlling processor. This device was developed through the combined requirements of (a) covering a wide total field of view while (b) retaining the highest possible spatial resolution on the targets of interest while at the same time (c) operating at the highest possible frame rate. Many thousands of frames per second are possible with the prototype imager while maintaining high spatial resolution. The prototype device operates as a visible imager, and Nova is pursuing the transition of this technology into the infrared domain. This paper will concentrate on applications of the technology and will show some imagery collected with the prototype system.
A cooperative effort between the U.S. Air Force Research Laboratory, Nova Research, Inc., the Raytheon Infrared Operations (RIO) and Optics 1, Inc. has successfully produced a miniature infrared camera system that offers significant real-time signal and image processing capabilities by virtue of its modular design. This paper will present an operational overview of the system as well as results from initial testing of the 'Modular Infrared Imaging Applications Development System' (MIRIADS) configured as a missile early-warning detection system. The MIRIADS device can operate virtually any infrared focal plane array (FPA) that currently exists. Programmable on-board logic applies user-defined processing functions to the real-time digital image data for a variety of functions. Daughterboards may be plugged onto the system to expand the digital and analog processing capabilities of the system. A unique full hemispherical infrared fisheye optical system designed and produced by Optics 1, Inc. is utilized by the MIRIADS in a missile warning application to demonstrate the flexibility of the overall system to be applied to a variety of current and future AFRL missions.
Raytheon's Infrared Operations (RIO) has invented and developed a new type of focal plane array with 'Image Processing on-the-chip' named the Adaptive IR Sensor (AIRS). The AIRS FPA is based upon the human retina in that it performs signal processing near the photoreceptors. The AIRS FPA has been reduced to practice and adaptively removes detector and optic temperature drift and 1/f induced fixed pattern noise. This 3rd-generation multi-mode IRFPA, also called a Smart FPA, is a 256 X 256-array format capable of operation in four modes: (1) Direct Injection (DI), (2) Adaptive Non-uniformity Correction (NUC), (3) Motion/Edge Detection, and (4) Subframe Averaging. Nova Research has developed a Modular IR Application Development System (MIRIADS) which is a compact single board camera system that is highly integrated with the dewar assembly. The MIRIADS system coupled with the AIRS Smart FPA results in a very high performance wide field of view IR Sensor and processing system with integrated in one of the smallest packages to date.
Third generation FPAs are being designed to incorporate numerous sophisticated 'smart' functions, useful in the preprocessing and filtering of real-time image data. Designers at Nova Research, Inc. have developed ROIC designs which have increased the capabilities of these devices in the are of signal and image processing. These new FPAs are more versatile than their predecessors through the implementation of a variety of programmable modes of operation. This paper will discuss a variety of such processing functions and modes. Design configurations for such FPAs sill be discussed with regard to the incorporation of general and specific signal processing functions. Such functions will include but not be limited to: edge enhancement and edge extraction, accommodating high and low signal flux environments, high speed windowing and foveated pixel arrangements.
Several technologies have been developed in recent years to advance the state of the art of IR sensor systems including dual color affordable focal planes, on-focal plane array biologically inspired image and signal processing techniques and spectral sensing techniques. Pacific Advanced Technology (PAT) and the Air Force Research Lab Munitions Directorate have developed a system which incorporates the best of these capabilities into a single device. The 'NeuroSeek' device integrates these technologies into an IR focal plane array (FPA) which combines multicolor Midwave IR/Longwave IR radiometric response with on-focal plane 'smart' neuromorphic analog image processing. The readout and processing integrated circuit very large scale integration chip which was developed under this effort will be hybridized to a dual color detector array to produce the NeuroSeek FPA, which will have the capability to fuse multiple pixel-based sensor inputs directly on the focal plane. Great advantages are afforded by application of massively parallel processing algorithms to image data in the analog domain; the high speed and low power consumption of this device mimic operations performed in the human retina.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.