To protect the security of image information in the process of storage or transmission, we propose an efficient image encryption method using chaos and game of life (GoL). First, a plaintext-related image scrambling method based on chaos and GoL is proposed to reduce the adjacent pixel correlation of the plain image. In this method, double scrambling is performed on the original image: the first scrambling matrix is constructed by logistic map and the rules of GoL; the second scrambling matrix is generated based on two-dimensional (2D) logistic map and magic square. The original image is scrambled by the above-mentioned scrambling matrices, which can efficiently reduce the strong adjacent pixel correlation, thus enhancing the image scrambling effect. Next, the 4D hyperchaotic system and GoL are employed to diffuse the scrambled image and get the cipher image. Additionally, the average pixel value of the original image is used to generate the initial values of the logistic map and 2D logistic map in scrambling process. Therefore, our encryption scheme is very sensitive to the plain image and can effectively resist the chosen-plaintext attack and known-plaintext attack. Finally, the experimental results and performance analyses show that the proposed encryption algorithm has the advantages of large key space and good scrambling effect, and it can effectively resist various typical attacks, so it can effectively protect the security of the image information.