A high-precision extrinsic calibration is the underlying premise of the accurate perception of light detection and ranging (LiDAR) and camera systems commonly used in the autonomous driving industry. We propose a coarse-to-fine strategy to get rigid-body transformation between solid-state LiDAR with non-repetitive scanning and a RGB camera system using a chessboard as the calibration target. This method exploits the reflectance intensity characteristics of the LiDAR point cloud, which exhibit the distinct distribution in white and black blocks of chessboard. In the coarse calibration step, a reflectance intensity Gaussian mixture model was used to extract the unicolor block point cloud from the chessboard point cloud. Therefore, the initial estimate of the extrinsic parameter was obtained by aligning the corners in the point cloud and calculating the centroid of the unicolor block point cloud and corners in the image. In the refinement step, we extracted points on the border of each block as LiDAR features and designed an iterative optimization algorithm to align the intensity of LiDAR features with grayscale features in the image. This method utilizes the intensity information and compensates for corner errors in the point cloud due to reflectance intensity binarization. The results of the comparative experiment revealed that the proposed method outperformed existing methods in terms of accuracy. Experiments based on simulations and real-world conditions revealed that the proposed algorithm demonstrated a high accuracy, robustness, and consistency.