A time-delayed integration charge-coupled device (TDI CCD) in an aerial panoramic camera compensates the sweep image motion correctly on the premise that the TDI charge shifting direction is coincident with that of the sweep image motion. The coordinate transformation method is used to find out how the included angle between the two directions originated. Then the precise alignment method of the TDI charge shifting direction is proposed to eliminate the included angle. TDI CCD is operated in area mode, nodding the scan mirror, the trajectory of the image point is derived to be a hyperbola, which could be equivalent to an obliquely straight line. The tilt angle of the line is exactly the included angle between the two directions. Meanwhile, the tilt angle can be calculated by the least square method. Then the scan head or the focal plane assembly is precisely rotated to eliminate the included angle between the two directions. The assembling error after precise alignment is calculated at −16.4 s, which could hardly influence the MTF of TDI CCD. The panoramic imaging experiment and the flight test show that the precise alignment method is feasible and completely satisfies the operating requirement of the aerial panoramic camera.