Efficient and accurate extraction of water areas from remote sensing images is a popular research topic. Currently, researchers have attempted to use neural networks to extract water from remote sensing images. However, most of these studies used computer vision techniques to improve the model results without considering the multi-band information unique to remote sensing images. Thus our study proposes an improved DeepLabv3+ network to increase the water body extraction accuracy in urban remote sensing images. The DeepLabv3+ network has the characteristics of extracting image features at multiple scales. We improved the network structure to incorporate multi-band features. By comparing several multi-band input methods, the feature map calculated by the normalized difference water index (NDWI) was converted into an input suitable for the neural network by comparing several multi-band input methods. Simultaneously, we developed a parallel convolution structure to combine the NDWI feature map with a standard false color remote sensing image during feature extraction. This allows the network to focus more on image areas that may be water bodies. We used atmospherically corrected Sentinel-2A L2A-level data, divided the training set at multiple scales, and conducted several experiments. The results show that the proposed network can improve water extraction accuracy when training subregions are unified from different sizes to 512  ×  512. Finally, we used the model to extract water bodies from remote sensing images from different regions. We combined the images with visual interpretation to verify the reliability of the model results. Moreover, the model scores of four types of multi-scale neural networks in the two categories are compared, which proves the effectiveness of the method.