Measuring the orbits of directly imaged exoplanets requires precise astrometry at the milliarcsec level over long periods of time due to their wide separation to the stars (≳10  au) and long orbital period (≳20  yr). To reach this challenging goal, a specific strategy was implemented for the instrument Spectro-Polarimetric High-contrast Exoplanet Research (SPHERE), the first dedicated exoplanet imaging instrument at the Very Large Telescope of the European Southern Observatory (ESO). A key part of this strategy relies on the astrometric stability of the instrument over time. We monitored for five years the evolution of the optical distortion, pixel scale, and orientation to the True North of SPHERE images using the near-infrared instrument IRDIS. We show that the instrument calibration achieves a positional stability of ∼1  mas over 2″ field of views. We also discuss the SPHERE astrometric strategy, issues encountered in the course of the on-sky operations, and lessons learned for the next generation of exoplanet imaging instruments on the Extremely Large Telescope being built by ESO.