A novel real-time-invariant target tracking algorithm is proposed for forward-looking IR (FLIR) image sequences based on a modified synthetic discriminant function (SDF)-based fringe-adjusted joint transform correlator (JTC) algorithm. The proposed approach utilizes fringe-adjusted JTC-based tracking, which efficiently compensates camera motion and accommodates 3-D distortions. To further increase the tracking accuracy, the target model is continuously updated using a modified version of the synthetic discriminant function (SDF) algorithm. The proposed technique can track small objects comprising only a few pixels and is capable of compensating the high egomotion of the sensor. We demonstrate the robustness of the proposed technique by computer simulation performed on sequences of real-life FLIR imagery taken from an airborne moving platform. © 2004 Society of Photo-Optical Instrumentation Engineers.