In an optical synthetic aperture imaging system, it is necessary to extract edge information of subapertures with complex shape and large gaps, aiming at cophasing of segmented mirrors ultimately. As a measure of image surface irregularities, fractal dimension (FD) was calculated for edge extraction and surface complexity evaluation. The modified differential box-counting (DBC) method was adopted to calculate FD, specifically the window merge replication method was presented to achieve FD estimation with a relatively small scanning window. The subaperture region and interference fringe edge were extracted by using a two-step strategy of preprocessing and postprocessing. Simulations and experiments are conducted to verify the feasibility and accuracy for edge extraction. As well, the capabilities of the proposed method are demonstrated with the results of numerical calculation.