Dr. Wenbo Liu Profile

Dr. Wenbo Liu

at Mississippi State Univ

SPIE Involvement:

Author

Publications (2)

Proceedings Article | 13 June 2023 Poster + Paper

Automating catfish cutting process using deep learning-based semantic segmentation

Thevathayarajh Thayananthan, Xin Zhang, Wenbo Liu, Tianqi Yao, Yanbo Huang, Nuwan Wijewardane, Yuzhen Lu

Proceedings Volume 12545, 125450D (2023) https://doi.org/10.1117/12.2663370

KEYWORDS: Image segmentation, Semantics, Data modeling, Deep learning, Head, Education and training, Visual process modeling, Computer vision technology, Performance modeling, Transformers

Read Abstract +

Mississippi and Alabama are the top two states producing and processing catfish in the United States, with the annual production of $382 million in 2022. The catfish industry supplies protein-rich catfish products to the U.S. market and contributes considerably to the development of the local economy. However, the traditional catfish processing heavily relies on human labors leading to a high demand of workforce in the processing facilities. De-heading, gutting, portioning, filleting, skinning, and trimming are the main steps of the catfish processing, which normally require blade-based cutting device (e.g., metal blades) to handle. The blade-based manual catfish processing might lead to product contamination, considerable fish meat waste, and low yield of catfish fillet depending on the workers’ skill levels. Furthermore, operating the cutting devices may expose the human labors to undesired work accidents. Therefore, automated catfish cutting process appears to be an alternative and promising solution with minimal involvement of human labors. To further enable, assist, and automate the catfish cutting technique in near real-time, this study presents a novel computer vision-based sensing system for segmenting the catfish into different target parts using deep learning and semantic segmentation. In this study, 396 raw and augmented catfish images were used to train, validate, and test five state-of-the-art deep learning semantic segmentation models, including BEiTV1, SegFormer-B0, SegFormer-B5, ViT-Adapter and PSPNet. Five classes were pre-defined for the segmentation, which could effectively guide the cutting system to locate the target, including the head, body, fins, tail of the catfish, and the image background. Overall, BEiTV1 demonstrated the poorest performance with 77.3% of mIoU (mean intersection-over-union) and 86.7% of MPA (mean pixel accuracy) among all tested models using the test data set, while SegFormer-B5 outperformed all others with 89.2% of mIoU and 94.6% of MPA on the catfish images. The inference speed for SegFormer-B5 was 0.278 sec per image at the resolution of 640x640. The proposed deep learning-based sensing system is expected to be a reliable tool for automating the catfish cutting process.

Proceedings Article | 13 June 2023 Presentation + Paper

Multi-ripeness level blackberry detection using YOLOv7 for soft robotic harvesting

Xin Zhang, Thevathayarajh Thayananthan, Muhammad Usman, Wenbo Liu, Yue Chen

Proceedings Volume 12539, 1253908 (2023) https://doi.org/10.1117/12.2663367

KEYWORDS: Robotics, Object detection, Education and training, Robotic systems, RGB color model, Image resolution, Agriculture, Image processing, Data acquisition, Computing systems

Read Abstract +

Blackberry crop production is an essential sector of high-value specialty crops. Blackberries are delicate and easy to be damaged during harvest process. Besides, the blackberries in an orchard are not ripe at the same time so that multiple passes of harvesting are often needed. Therefore, the production is highly labor intensive and could be addressed using robotic solutions while maintaining the post-harvest berry quality for desired profitability. To further empower the developed tendon-driven soft robotic gripper specifically designed for berries, this study aims at investigating a state-of-the-art deep-learning YOLOv7 for accurately detecting the blackberries at multi-ripeness level in field conditions. In-field blackberry localization is a challenging task since blackberries are small objects and differ in color due to various levels of ripeness. Furthermore, the outdoor light condition varies depending on the time of day/location. Our study focused on detecting in-field blackberries at multi-ripeness levels using the state-of-the-art YOLOv7 model. In total, 642 RGB images were acquired targeting the plant canopies in several commercial orchards in Arkansas. The images were augmented to increase the diversity of data set using various methods. There are mainly three ripeness levels of blackberries that can present simultaneously in individual plants, including ripe (in black color), ripening (in red color), and unripe berries (in green color). The differentiation of ripeness levels can help the system to specifically harvest the ripe berries, and to keep track of the ripening/unripe berries in preparation for the next harvesting pass. The aggregation of total number of berries at all ripeness levels can also help estimate the crop-load for growers. The YOLOv7 model with seven configurations and six variants were trained and validated with 431 and 129 images, respectively. Overall, results of the test set (82 images) showed that YOLOv7-base was the best configuration with mean average precision (mAP) of 91.4% and F1-score of 0.86. YOLOv7-base also achieved 94% of mAP and 0.93 of True Positives (TPs) for ripe berries, 91% and 0.88 for ripening berries, and 88% and 0.86 for unripe berries under the Intersection-over-Union (IoU) of 0.5. The inference speed for YOLOv7-base was 21.5 ms on average per image with 1,024x1,024 resolution.

View contact details

UPDATE YOUR PROFILE

Is this your profile? Update it now.

Sign into your SPIE.org account

Don’t have a profile and want one?

Create an account on SPIE.org

Keywords/Phrases

Search In:

Publication Years