|
1.IntroductionLong-wave infrared (LWIR) cameras are used in a tremendously wide range of applications, ranging from environmental monitoring and industrial process control to military surveillance. The range of applications is expanding rapidly because of recent advances in uncooled microbolometer detector arrays, which enable much smaller, lighter, and lower-cost infrared imagers.1–3 A growing number of these applications require the camera to be calibrated radiometrically, sometimes in terms of temperature and other times in terms of radiance or a similar quantity involving optical power. For example, microbolometers provide resource-efficient radiometric remote sensing of Earth from space, but space-borne systems typically include on-board calibration sources or temperature stabilization.4,5 However, many other applications become viable if they can be deployed without on-board calibration sources or temperature stabilization, such as sensing in agriculture and food processing,6 airborne remote sensing of streams and rivers,7 face recognition in thermal imagery,8 noninvasive monitoring of beehive populations,9 vegetation imaging to detect leaking gas,10 and many others.11 Yet other applications may be feasible with on-board calibration sources, but still benefit from the smaller size and lower cost associated with the use of microbolometer imagers without external calibration sources or camera temperature stabilization, such as infrared imaging polarimetry,12 cloud imaging in climate studies,13 and characterizations of Earth–space optical communication paths.14 Radiometric calibration typically requires quantitatively relating the camera output to source radiance or temperature. This is most commonly done by measuring the camera’s output while it views one or more blackbody sources. Such a calibration assumes the camera’s response is time invariant, and thus a measurement of a scene taken at a later time can be calibrated to give a quantitative value. However, this assumption does not hold for a microbolometer imager with no thermo-electric cooler, whose response depends on both the focal plane array (FPA) temperature and the scene temperature. Such a camera will exhibit changes in its output that arise solely from changes in the camera temperature. Without stabilization, such cameras cannot maintain a stable radiometric calibration. For example, in a study of face recognition using thermal imagery,8 the authors noted that their capabilities were limited by calibration changes caused by moving the camera outdoors. This was likely a result of the calibration coefficients changing as a result of temperature fluctuations of the optics and the FPA. Honeywell was one of the first companies to produce a thermally nonstabilized microbolometer camera, which achieved a stable response through a per-pixel calibration for every FPA temperature expected to be experienced during operation.1 This method was mathematically intense and required the camera to be tested over all expected operating temperatures. Alternatively, there are multiple proposed methods for calibrating thermally nonstabilized infrared cameras in patent literature. For example, fourth-order polynomial curve fits have been used to relate FPA temperature to output digital number, thereby compensating for changes in FPA temperature and even temperature-induced germanium lens and window transmittance variations.15 Other methods alter the camera integration time,16 the readout bias,17 or other camera operating parameters18 to compensate for sensor response variations that result from a changing FPA temperature. Other methods determine a correction based on measurements of the camera response as a function of temperature over a wide range of conditions.19,20 At least one method relies on heat transfer models to estimate and remove the nonscene energy to determine a calibration.21 Most of the patent descriptions provide only the basics of the idea—with very little of the supporting mathematics—and rarely present data showing the proposed method applied to a real camera. This paper presents a technique that allows stabilization to take place through software rather than actual physical means. We present the underlying mathematics and describe a simple method of determining the coefficients required to stabilize the output of a microbolometer in the presence of camera temperature variations (we assume the FPA temperature is representative of the system). In this solution, the digital response of a camera is stabilized to the value the camera would experience at a reference FPA temperature, selected to be 25°C for this paper. This method was applied to a Photon 320 LWIR microbolometer imager (FLIR Systems), and a calibration accuracy of was obtained for FPA temperature deviations within changing by , limited primarily by the uncertainty of the blackbody source emissivity and temperature. 2.Proposed Temperature Correction MethodThe need to account for variations in FPA temperature is illustrated in this section using thermally nonstabilized microbolometer measurements of a temperature-controlled blackbody source. We placed a Photon 320 microbolometer camera and a large-area blackbody inside an environmental chamber and stepped the blackbody source temperature while cycling the chamber temperature. Specifically, the chamber air temperature was cycled with a repeating triangle ramp between 10°C and 30°C, which resulted in an oscillating FPA temperature inside the camera (Fig. 1). The blackbody source was held stable for approximately 3 h at 10°C, increased to 20°C and held constant, and so forth up to a maximum temperature of 50°C (Fig. 2). The camera viewed the blackbody source whose temperature followed the pattern in Fig. 2. The camera output (Fig. 3) had a mixed response to the blackbody source and the time-varying FPA temperature. For this particular camera, the response to the blackbody scene decreased with increasing FPA temperature. The method described in detail in the next section calculates the corrected digital number from the error-containing raw response , the difference between the current FPA temperature and the reference temperature, , and correction coefficients and , This essentially locks the camera response to the response at the reference temperature. In practice, the reference temperature should be selected to lie at the center of the expected operating FPA-temperature range. Once the coefficients and have been determined (following procedures outlined in the next sections), the camera can be radiometrically calibrated using a standard method of viewing blackbodies at various temperatures. The benefit of using Eq. (1) for a correction is that this equation uses only the digital values and FPA temperature from the camera, without requiring knowledge of parameters such as the camera’s spectral response, and so forth. Thus the correction for the FPA-temperature dependence can be accomplished without a full calibration of the camera. This produces a stabilized camera that subsequently can be either used in applications that require only a stable output or calibrated for applications that require radiometric data (which then requires use of the camera’s spectral response). 3.Mathematical BasisThe FPA-temperature dependence of microbolometer cameras can be modeled as a temperature-dependent response that is a linear function of a temperature-dependent responsivity or gain and a temperature-dependent dark signal or offset , which determine the digital number output by the camera in response to scene radiance : The terms and can be written as and where is an FPA temperature–dependent camera gain, is a temperature-independent camera gain, is a temperature-dependent dark signal, and is a temperature-independent dark signal. These temperature dependences can arise from a variety of sources, including temperature dependence of the bolometer resistance, emission from the optics, emission and self-detection of the FPA, changes in the biasing resistances of the microbolometer readout circuit, and potentially other sources.15This method assumes that all sources of temperature dependence can be related to the FPA temperature and that over a modest range the temperature dependence can be described adequately by a linear function. These assumptions hold true when the FPA and optics remain approximately equally coupled, but begin to break down when these temperatures begin to vary at different rates (e.g., with rapid or large ambient temperature changes). But with these assumptions, the terms in Eqs. (2)(3)–(4) can be combined to express the FPA temperature–dependent camera response as If the camera views a blackbody source held at a constant temperature while the FPA temperature is changed from the reference temperature to a different value , the response digital values are represented by andBecause the scene radiance does not change for these two blackbody views, we can solve these equations for radiance and equate the results to obtain We now solve for and expand the terms over a common denominator to obtain Next we group terms with and without : The denominator of this equation has one term that changes with the FPA temperature and one that does not. To remove this complication, we first multiply each side by to give which simplifies to which we further simplify to obtainThe numerator of this equation has two variables, and ; therefore, it will simplify calculations to put the bottom in terms of . This can be done by adding a term of to find and then rearranging to obtainThe two terms multiplied by in Eq. (15) are composed of only camera parameters and other known values that are invariant with FPA temperature changes, such as the reference temperature to which the camera response is stabilized. Therefore, we define the following two parameters: andThe first term, , is the temperature-dependent camera gain divided by the gain at the reference temperature. However, the value for expressed by Eq. (17) is more complicated. This value is an interaction term containing the temperature-independent and temperature-dependent dark signals and gains in the numerator and the reference-temperature gain in the denominator. If we also define then Eq. (15) becomes the simple form of Eq. (1).4.Determining the Correction CoefficientsRewriting the reference-temperature response function from Eq. (1) as allows us to write an equation for determining the coefficients and . These correction coefficients can be determined by viewing two constant-temperature blackbody scenes with radiances and , each with the camera at a minimum of two different temperatures, and . A third camera temperature can be experienced while viewing the second scene, but there must be at least one common FPA temperature between the two blackbody scenes. Thus, we must consider the following responses: with the camera at and the blackbody at radiance ; with the camera at and the blackbody at radiance ; with the camera at and the blackbody at radiance , and with the camera at and the blackbody at radiance . The camera responses and are at the same FPA temperature and will be used as the references. Further, note that the responses and can be at the same FPA temperature, but this is not required ( could equal ). Using as the reference camera temperature leads to the following differences.These differences can be used in Eq. (19) to write the following matrix equation: which can be inverted to obtain and .This method is the minimal approach to deriving these coefficients, but in practice we use a large range of blackbody temperatures (for example 10°C to 60°C in steps of 10°C) and a large range of camera FPA temperatures (for example from 10°C to 30°C). This can be accomplished by placing the camera in an environmental chamber and changing the ambient temperature to drive the temperature of the camera while the blackbody remains constant, then changing the blackbody temperature and repeating the ambient temperature cycle. Doing this generates multiple reference responses, one for each combination of blackbody temperature and FPA temperature, leading to an over-determined matrix as shown in Eq. (24): In such a case, a pseudo-inversion is required, and in practice the Moore–Penrose pseudo-inversion is performed. This leads to a least-squares approach that reduces noise in the estimation of m and b through Eq. (25): It is important to note that this correction is determined with a unique value for every pixel. Therefore, and actually represent matrices whose dimensions match the pixel count of the FPA. In Eq. (25), and are both coefficients of first order in , but sometimes it is useful to determine these coefficients as polynomials up to fourth order in . However, typically only requires a higher-order description (especially in situations with widely ranging FPA temperature), whereas can remain a first-order coefficient. Also note that the absolute temperature or radiance of the blackbody source is not needed in this temperature-stabilization method, since the blackbody is only used as a stable reference. The only requirement is for the source to remain stable during the measurements at multiple FPA temperatures. The correction uses only the measured responses and the camera FPA temperature, with no need for additional information about the camera’s spectral response or the blackbody source temperature (the camera spectral response is needed for the final radiometric calibration, but not for compensating for the FPA-temperature dependence). Nevertheless, performing this technique at two or more blackbody temperatures improves the accuracy of the coefficients in the correction. This case generates multiple reference responses, one for each blackbody temperature, leading to an overdetermined matrix, thus reducing noise in the estimation of and . 5.Application of the MethodThe procedure to use this method to determine a temperature-stabilized radiometric calibration for a LWIR camera is presented in Fig. 4. First, the scene image and the FPA temperature are measured concurrently. The temperature-dependent responsivity and offset coefficients and are then used to stabilize the camera to its response at . Finally, the stabilized response is calibrated by applying conventional gain and offset terms that were measured at or referenced to . These gain and offset values convert the FPA temperature–stabilized data into values of integrated radiance seen by the detector. If needed, the radiance data can be converted to temperature through a lookup table created by integrating the product of the blackbody function and the camera’s spectral response function. Calibrating the imager in units of integrated radiance rather than temperature has advantages, most notably the nominally linear response of photo detectors with radiance or other radiometric quantities proportional to photon irradiance. This in turn allows extrapolation to very low radiance values using a calibration performed with measurements of much warmer blackbody sources.13,22,23 Although our technique calibrates to radiance, techniques that calibrate directly to temperature could also use the FPA temperature–stabilized data (calibrating directly in temperature requires compensating for the nonlinear relationship between the temperature of an object and its emitted radiance24). As an example of successful implementation on a microbolometer camera without thermal stabilization, we applied this method to a FLIR Photon 320 camera with a 14.25-mm lens. The camera was placed with a large-area blackbody inside an environmental chamber. Throughout the experiment, the blackbody filled the 50 deg- by 38-deg camera field of view. The blackbody was an Electro-Optical Industries CES-100 with a 15-cm aperture and microgroove surface specified with a 0.995 emissivity. The blackbody was factory calibrated with a NIST-traceable temperature uncertainty of 0.1°C. Background radiance reflected from the blackbody was removed from blackbody measurements, but was typically less than 0.75% of the observed blackbody radiance. In this experiment, blackbody images were acquired as the source temperature was increased from 10°C to 60°C in steps of 16.7°C every 6 h. While the blackbody was held constant at each setting, the ambient temperature of the chamber was ramped in a triangle wave pattern between 15°C and 30°C in steps of 1°C every 5 min. These images were used to derive matrices of and coefficients for the stabilization Eq. (1) with terms up to order The uncertainty in the resulting calibration was determined by placing the camera back inside the chamber and acquiring calibrated images of the blackbody source at different temperatures. The chamber air temperature was varied to match the outdoor air temperature measured by our weather station on a summer day and night in Bozeman, MT (ranging between 13.9°C and 26.4°C). The blackbody temperature was stepped between 10°C and 50°C and held constant for a variable length of time. The FPA temperatures and blackbody temperatures during this experiment are shown in Fig. 5. The measured data before FPA-temperature correction (black) and after correction (red) are shown in Fig. 6. Solid lines indicate the spatial mean across the FPA, while dashed lines indicate the maximum and minimum values at each instant of time (the spatial standard deviations were indistinguishably close to the mean line). With the camera’s response stabilized as shown in Fig. 6, a conventional two-point radiometric calibration was applied using blackbody temperatures of 10°C and 60°C. Figure 7 is a time-series plot of the calibrated camera output and the blackbody source temperature for the 24-h experiment just described. The solid line indicates the spatial mean across the FPA, whereas the dashed lines indicate the maximum and minimum values across the FPA at each instant of time (the spatial standard deviations again were indistinguishably close to the mean line). Figure 8 is a time-series plot of the difference between the blackbody source temperature and the temperature reported by the camera (from the calibrated radiance). The dashed lines in this figure indicate the expected blackbody-related calibration uncertainty of . This value arises from combining the blackbody temperature uncertainty () and the blackbody emissivity uncertainty () in the two-point calibration.25 The blackbody temperature uncertainty includes blackbody spatial variation and uncertainties in readout, temperature sensor calibration, and temporal variation. The effect of the emissivity uncertainty on the calibration depends on the ambient temperature range experienced in the experiment. Figure 8 shows that the difference of the measurements from the blackbody set-point temperatures was within the expected uncertainty for nearly all readings. The spatial rms variation of the difference varied over time, with typical values of 0.08°C and a maximum value of 0.19°C. The temporal rms variation of this difference was 0.09°C, leading to a total spatial-temporal variability of 0.21°C. The maximum sustained difference was 0.75°C, which occurred between 300 and 450 min after the experiment began. The maximum instantaneous error was in a single observation near 1150 min. However, this and other single-frame errors are believed to be blinking of residual bad pixels. Figure 9 shows two histograms that provide a final illustration of the significance of this camera stabilization method, using 300 frames chosen at random times from the 24-h measurement sequence on which Figs. 5Fig. 6Fig. 7–8 are based. The histogram in Fig. 9(a) shows the distribution of measurement errors relative to the blackbody set point without camera stabilization, and Fig. 9(b) shows the distribution of errors with stabilization. The errors are spread over without stabilization and with stabilization. It is also interesting to note that stabilization converts the relatively uniform error distribution in Fig. 9(a) to a Gaussian-like distribution in Fig. 9(b). During these validation experiments, the flat field correction (FFC) of the camera was manually controlled to happen every 5 min or 0.4°C change in FPA temperature. By studying images collected before and after the FFC, it was found that the FFC had only a minor impact on the stabilized camera calibration. Just after an FFC is done, the reported temperature changed by an amount between 0.1°C and 0.25°C, decreasing when the FPA temperature was decreasing and increasing when the FPA temperature was increasing. 6.Discussion and ConclusionThis method allows for correction of the FPA temperature-dependent errors in the response of microbolometer cameras. It is unique from previously reported techniques in that it allows the stabilization to take place without a full radiometric calibration of the camera, and without requiring specific information about the camera that would be required in other techniques. In operation, the output data from the imager can be adjusted for the FPA temperature-induced errors, thus stabilizing the response of otherwise highly temperature-dependent cameras. Radiometric calibration and other data processing then can be performed on the stabilized camera response. In many thermal infrared imaging applications, use of the technique described here can eliminate costly and space-intensive blackbody sources in field-deployed instruments, provide radiometric calibration of otherwise uncalibrated cameras, and improve the stability and accuracy of data from thermal imagers. For example, the reported technique benefits any application in which a radiometrically calibrated thermal camera is required and wherein the cost, space, or weight of an external blackbody is not desired or practical. The reported technique was carefully implemented on a FLIR Photon 320 camera, producing a calibration with rms variability of 0.21°C, total calibration uncertainty of 0.38°C, and a maximum error of 0.75°C in a 24-h period. This calibration uncertainty is better than the value of specified for most commercial microbolometer imagers. The primary contributors to the calibration uncertainty are blackbody source emissivity and temperature. This process has produced similar results on a variety of FLIR cameras, including the Photon 320 with several wide and narrow lenses, PathFindIR, Photon 640, Tau 320, and Tau 640. ReferencesP. W. Kruse, Uncooled Thermal Imaging, Arrays, Systems, and Applications, 64 SPIE, Bellingham, WA
(2002). Google Scholar
F. NiklausC. VieiderH. Jakobsen,
“MEMS-based uncooled infrared bolometer arrays: a review,”
Proc. SPIE, 6836 68360D
(2007). http://dx.doi.org/10.1117/12.755128 PSISDG 0277-786X Google Scholar
R. K. Bhanet al.,
“Uncooled infrared microbolometer arrays and their characterisation techniques,”
Defence Sci. J., 59
(6), 580
–589
(2009). DSJOAA 0011-748X Google Scholar
S. Garcia-Blancoet al.,
“Radiometric packaging of uncooled microbolometer FPA arrays for space applications,”
Proc. SPIE, 7206 720604
(2009). http://dx.doi.org/10.1117/12.810025 PSISDG 0277-786X Google Scholar
H. Geoffrayet al.,
“Evaluation of a COTS microbolometers FPA to space environments,”
Proc. SPIE, 7826 78261U
(2010). http://dx.doi.org/10.1117/12.869726 PSISDG 0277-786X Google Scholar
R. VadivambalD. S. Jayas,
“Applications of thermal imaging in agriculture and food industry: a review,”
Food Bioprocess. Technol., 4
(2), 186
–199
(2011). http://dx.doi.org/10.1007/s11947-010-0333-5 FBTOAV 1935-5130 Google Scholar
S. RayneG. S. Henderson,
“Airborne thermal infrared remote sensing of stream and riparian temperatures in the Nicola River watershed, British Columbia, Canada,”
J. Environ. Hydrol., 12 1
–11
(2004). Google Scholar
D. A. SocolinskyA. SelingerJ. D. Neuheisel,
“Face recognition with visible and thermal infrared imagery,”
Comp. Vis. Image Understand., 91
(1–2), 72
–114
(2003). http://dx.doi.org/10.1016/S1077-3142(03)00075-4 CVIUF4 1077-3142 Google Scholar
J. A. Shawet al.,
“Long-wave infrared imaging for non-invasive beehive population assessment,”
Opt. Express, 19
(1), 399
–408
(2011). http://dx.doi.org/10.1364/OE.19.000399 OPEXFF 1094-4087 Google Scholar
J. E. Johnsonet al.,
“Long-wave infrared imaging of vegetation for detecting leaking gas,”
J. Appl. Rem. Sens., 6
(1), 063612
(2012). http://dx.doi.org/10.1117/1.JRS.6.063612 Google Scholar
M. VollmerK.-P. Möllmann, Infrared Thermal Imaging, Wiley-VCH, Weinheim, Germany
(2010). Google Scholar
M. W. KudenovJ. L. PezzanitiG. R. Gerhart,
“Microbolometer-infrared imaging Stokes polarimeter,”
Opt. Eng., 48
(6), 063201
(2009). http://dx.doi.org/10.1117/1.3156844 OPEGAR 0091-3286 Google Scholar
J. Shawet al.,
“Radiometric cloud imaging with an uncooled microbolometer thermal infrared camera,”
Opt. Express, 13
(15), 5807
–5817
(2005). http://dx.doi.org/10.1364/OPEX.13.005807 OPEXFF 1094-4087 Google Scholar
P. W. NugentJ. A. ShawS. Piazzolla,
“Infrared cloud imaging in support of Earth-space optical communication,”
Opt. Express, 17
(10), 7862
–7872
(2009). http://dx.doi.org/10.1364/OE.17.007862 OPEXFF 1094-4087 Google Scholar
T. HoelterB. Meyer,
“The challenges of using an uncooled microbolometer array in a thermographic application,”
(2012) http://www.dtic.mil/cgi-bin/GetTRDoc?Location=U2&doc=GetTRDoc.pdf&AD=ADA399432 September ). 2012). Google Scholar
Y. Hanet al.,
“Infrared spectral radiance measurements in the tropical Pacific atmosphere,”
J. Geophys. Res., 102
(D4), 4353
–4356
(1997). http://dx.doi.org/10.1029/96JD03717 JGREA2 0148-0227 Google Scholar
R. O. Knutesonet al.,
“Atmospheric emitted radiance interferometer. Part II: instrument performance,”
J. Atmos. Ocean. Technol., 21
(12), 1777
–1789
(2004). http://dx.doi.org/10.1175/JTECH-1663.1 JAOTES 0739-0572 Google Scholar
J. A. ShawL. S. Fedor,
“Improved calibration of infrared radiometers for cloud temperature remote sensing,”
Opt. Eng., 32
(5), 1002
–1010
(1993). http://dx.doi.org/10.1117/12.133232 OPEGAR 0091-3286 Google Scholar
C. L. WyattV. PrivalskyR. Datla,
“Recommended practice; symbols, terms, units and uncertainty analysis for radiometric sensor calibration,”
NIST Handbook 152, U.S. Government Printing Office, Washington, DC
(1998). Google Scholar
BiographyPaul W. Nugent is a research engineer with the Electrical and Computer Engineering Department at Montana State University and president of NWB Sensors Inc. in Bozeman, Montana. He received his MS and BS degrees in electrical engineering from Montana State University. His research activities include radiometric thermal imaging and optical remote sensing. Joseph A. Shaw is the director of the Optical Technology Center, professor of electrical and computer engineering, and affiliate professor of physics at Montana State University in Bozeman, Montana. He received PhD and MS degrees in optical sciences from the University of Arizona, an MS degree in electrical engineering from the University of Utah, and a BS degree in electrical engineering from the University of Alaska–Fairbanks. He conducts research on the development and application of radiometric, polarimetric, and laser-based optical remote sensing systems. He is a Fellow of the OSA and SPIE. |