Computational holography becomes highly complicated and demanding when it is employed to produce real three-dimensional (3D) images. Here we present a novel algorithm for generating a full parallax 3D computer generated hologram (CGH) with occlusion effect, which is an important property of 3D perception, but has often been neglected in most CGH related works. The ray casting technique is introduced to handle the occlusion issue. Horizontally and vertically distributed rays are projected from each hologram sample to the 3D objects to obtain the complex amplitude distribution. The proposed algorithm has no restriction on—or approximation to—the 3D objects, and it can produce reconstructed images with correct shading effect and no visible artifacts. An optical experiment is performed to validate our approach, using a phase-only spatial light modulator to optically reconstruct a 3D scene. The experimental result confirmed that the CGHs produced by our algorithm can successfully reconstruct 3D images with full parallax and occlusion effect.