The current light field occlusion removal methods are generally computationally demanding and have insufficient effect on the global receptive field. To address these issues, we introduce SwinSccNet, an occlusion removal network based on the Swin-Unet encoder–decoder system. We employ Scconv to compress redundant features in the shallow convolutional neural network (CNN), and the Swin transformer is used to improve the global receptive field of the deep Swin-Unet encoder–decoder. The experimental results show that our technique not only minimizes computational costs and complexity but also achieves state-of-the-art performance on publicly accessible datasets.