The binary joint transform correlator is a good discriminator with good noise tolerance. However, this is offset by its high sensitivity to distortions. Distortion tolerance can be improved by reducing the degree of nonlinearity applied to the joint power spectrum. However, this results in large reductions in noise tolerance and light efficiency. A technique to derive joint power spectrum windows is introduced that will optimally trade-off noise tolerance with distortion tolerance for the binary joint transform correlator. The window functions do not assume knowledge of the input or reference images, can easily be implemented both optically and digitally, and can be used in conjunction with a composite filter or an optical neural network. The feasibility of applying the windowed binary joint transform correlator to the problem of real-time face recognition is demonstrated by using real faces with synthetic distortions and noise. The results show that the windowed binary joint transform correlator is less sensitive to distortions and noise than is the k’th-law nonlinear joint transform correlator. Encouraged by the insensitivity to signal-like background noise, further experiments treat the upper part of the faces as background noise. A feedback configuration of the windowed binary joint transform correlator is used to detect and segment the lower part of the faces. These experiments demonstrate the feasibility of using this configuration as a real-time facial gesture recognizer. © 1999 Society of Photo-Optical Instrumentation Engineers.