The temperature of hot mix asphalt (HMA), base, and subgrade layers plays a significant role in pavement performance, because temperature influences the strength of the materials. Therefore, a model to predict temperature throughout the entire pavement structure is desirable. However, most existing models only focus on predicting the temperature of the road surface or the HMA layer, and these models usually need some information related to boundary conditions or material properties that is difficult to obtain. This research aims to demonstrate that machine learning (ML) model can be a powerful generalized approach to predict the temperature within a pavement structure at multiple depths. Data collected from sensors (thermistors and time domain reflectometers) installed in the Integrated Road Research Facility test road in Edmonton, Alberta, Canada, were used to train ML models. Sensitivity analysis was performed to analyze the influence of several input parameters on asphalt and soil temperature. ML models with three input parameters—average daily air temperature, day of the year, and depth—resulted in better performance compared to ML models based on other combinations of parameters. Three ML models were established to predict the average daily temperature, minimum daily temperature, and maximum daily temperature of the pavement structure. To validate model performance, the three ML models were compared with four existing models, and of these the ML models showed the highest accuracy with the coefficient of determination values above than 0.97 and root mean square error values below than 2.21. These results demonstrate that ML models can be used to give accurate predictions of road temperature at multiple depths with only one environmental predictive parameter, average daily air temperature.