An adaptive target detection algorithm for forward-looking infrared (FLIR) imagery is proposed, which is based on measuring differences between structural information within a target and its surrounding background. At each pixel in the image a dual window is opened, where the inner window (inner image vector) represents a possible target signature and the outer window (consisting of a number of outer image vectors) represents the surrounding scene. These image vectors are then preprocessed by two directional highpass filters to obtain the corresponding image gradient vectors. The target detection problem is formulated as a statistical hypotheses testing problem by mapping these image gradient vectors into two linear transformations, and via principal component analysis (PCA) and eigenspace separation transform (EST), respectively. The first transformation is only a function of the inner image gradient vector. The second transformation is a function of both the inner and outer image gradient vectors. For the hypothesis (target), the difference of the two functions is small. For the hypothesis (clutter), the difference of the two functions is large. Results of testing the proposed target detection algorithm on two large FLIR image databases are presented. © 2004 Society of Photo-Optical Instrumentation Engineers.