We propose an electronic speckle pattern interferometry-based measurement method in which a hardware device in the reference arm is used to track the out-of-plane displacement in the objective arm. We developed a real-time one-point out-of-plane displacement measurement system, which uses a Michelson interferometer, a lead zirconate titanate (PZT) device, a charge-coupled device camera, and a tracking control system. The system works by checking the movement of carrier fringes, and the PZT is used to track the displacement. We also developed an efficient tracking algorithm based on direction judgment and correlation computation to determine whether the PZT is activated and the distance that the PZT is ordered to move. Our experimental results demonstrate the effectiveness of the system, and finally, we discuss the detailed mechanism of the system.