In this paper, we experimentally verified that deep learning based on Convolutional Neural Network (CNN) is an effective method to judge the freshness of seafood fillets from its images. Currently, freshness of seafood is generally estimated by human’s eyes (connoisseurs) in Japanese fishing industry. However, it requires many years of experience and is a difficult task for new workers due to lack of experience. Furthermore, the Japanese fishing industry is faced with a worker shortage that makes it difficult to pass on their skills to next generation due to an aging and a lack of new workers.