Identifying and analyzing the sentiment information of an image is a complicated task in content understanding with nondeterministic features. To establish an effective model for multidimensional emotion recognition of film and television scene images and explore the relationship between the inputs and prediction results, we chose eight feature selection schemes based on principal component analysis, least absolute shrinkage operator (Lasso) regression, Pearson correlation, and the importance of random forest (RF) to establish the input vectors and the relevant model of machining learning. After the test on the data set with fixed combination of dimension reduction of 19 emotional descriptors, the fuse of Lasso regression and the RF importance extraction were proved to have the best performance on accuracy with the regression models. The experimental results showed that our algorithm performed well on the regression task of multidimensional affective dataset, which was convenient and time-saving. The extracted features and relevant dimension reduction methods also achieved good results on the cross-dataset classification tasks.