Optical metasurfaces, which consist of subwavelength scale meta-atoms, represent a novel platform to manipulate the polarization and phase of light. The optical performance of metasurfaces heavily relies on the quality of nanofabrication. Retrieving the Jones matrix of an imperfect metasurface optical element is highly desirable. We show that this can be realized by decomposing the generalized Jones matrix of a meta-atom into two parallel ones, which correspond to the ideal matrix and a phase retardation. To experimentally verify this concept, we designed and fabricated metasurface polarizers, which consist of geometric phase-controlled dielectric meta-atoms. By scanning the polarization states of the incident and transmitted light, we are able to extract the coefficients of the two parallel matrices of a metasurface polarizer. Based on the results of the Jones matrix decomposition, we also demonstrated polarization image encryption and spin-selective optical holography. The proposed Jones matrix retrieval protocol may have important applications in computational imaging, optical computing, optical communications, and so on.