During the operation of a laser in an unstable resonator with a large Fresnel number, as the output power of a chemical oxygen iodine laser (COIL) is increased, instability in the output laser beam will occur, behaving mainly as far-field beam spot drift and deformation. To interpret this phenomenon, a simulation of the output beam mode of this kind of laser was developed by using the method of fast Fourier transform. From the result of calculation, we propose that the presence of the nonuniform gain in the COIL produces a skewed output intensity distribution, which causes the mirror to tilt and bulge due to thermal expansion. As the output power is increased, the mirror surface, especially the back of the scraper mirror, will absorb more and more heat, causing thermal expansion of the mirror front surface to occur more seriously, thus resulting in the tilting and deformation of the far-field beam spot. The initial misalignment of the direction is also an important factor leading to far-field beam spot drift and deformation. © 2002 Society of Photo-Optical Instrumentation Engineers.