1.

INTRODUCTION

China’s e-commerce booms with the arrival of the information age, as well as the logistics industry, changing from the traditional ways of classifying and handling goods by manpower to ways combining artificial intelligence and information management. The huge logistics volume, brings a substantial income for logistics companies, and also poses a great challenge to them. How to improve the utilization rate of storage space and optimize the time of goods sorting, the manpower efficiency and the arrangement and deployment of truck space becomes a strong basis for logistics companies to compete with their counterparts and is problems that need to be optimized and solved constantly. The key to these problems is the rational use of the dimension information of logistics cartons.

With the coming of the automated industrial age and the maturity of OpenCV algorithms, computer vision as a subject has been studied and improved by scholars at home and abroad. Meanwhile, many scholars have also come up with their own schemes for the contactless measurement of the express box volume.

2.

RELATED WORK

According to Mi Yizhou, a binocular vision system was composed of two industrial CCD cameras, stereo matching was conducted for the measured box by semi-global dynamic programming algorithm to obtain a dense disparity map containing the three-dimensional space information of the parcels to be measured, and the volume of the measured box was obtained by three-dimensional reconstruction¹. Wang and Yi used a trained RCF network model to binarize the edges of the measured box, and preprocessed the binarization images of the edges to accurately locate the vertices and edges of the box². Liu proposed the method of self-built datasets and Viola-Jones algorithm-based optimization for automatic detection of logistics parcels and preprocessing of images. After the depth image was transformed into the three-dimensional point cloud image, point clouds were sampled and outliers were removed by Voxel Grid voxel filter and statistics filter. Point clouds on each plane were segmented and counted by RANSAC algorithm, and finally the volume of the box was obtained by statistical calculation³. Qiu designed two-object dimension measuring devices, which were respectively placed in two non-parallel vertical surfaces of the measured logistics carton, and the laser of the devices was adjusted to point to the geometric center of the object according to the position-type PID algorithm. Then, images taken by OpenMV module were processed by Hough transform to obtain the dimension of the corresponding vertical surface of the measured box. Finally, the dimension data of the two vertical surfaces were compared and integrated to obtain the dimension of the measured box⁴.

The existing measurement of the size of express packaging boxes has the following problems: first, the outline of the box cannot be well traced to obtain an approximate box model; second, the accuracy of the box size automatically measured by the existing method is not high enough, the error is relatively large. In view of these deficiencies in the current research, a method for measuring the dimensions of express packaging boxes based on binocular camera is proposed, and the outline of the box is obtained by combining the HSV color space and the Otsu method.

3.

MEASUREMENT OF EXPRESS PACKAGING BOX DIMENSION BASED ON BINOCULAR CAMERA

(1) Obtaining of box contour based on HSV color space and Otsu

To obtain an ideal contour extraction effect, the advantages of complete box segmentation by HSV color space and clear lower edge segmentation by Otsu were combined in the experiment^{5, 6}. Let’s assume that the image processed by HSV is 𝑝₁, and that processed by Otsu is 𝑝₂. First, the maximum areas of the while parts in 𝑝₁ and 𝑝₂ images are obtained by the function get_lagrest_connect_component1() to obtain parameters 𝑝_1max and 𝑝_2max. Since the advantages of both areas are needed, namely the intersection of 𝑝_1max and 𝑝_2max, the formula is:

Through the above combination based on HSV and Otsu characteristics, we can obtain a good carton contour, as shown in Figure 1.

Figure 1.

Obtaining of the measured box contour based on HSV and Otsu.

(2) Extraction of corner points based on straight line fitting

To measure the dimension of the measured box, we actually only need to extract the four corner points of the box from the extracted contour⁷, as shown in Figure 2.

Figure 2.

Schematic diagram of extracting corner points.

Therefore, the corner points were extracted based on straight line fitting in the experiment. Specific steps were as follows:

Step 1: the maximum contour was extracted from Figure 2, and the point with y_max within the contour was searched as 𝑝₁ and denoted by 𝑝₁(x,y_max). The reason for using Otsu method to extract the maximum contour is that the carton contour obtained is smooth and can be easily operated.

Step 2: the parameter k = 50 was set, 50 points were respectively taken on both sides of point 𝑝₁ along the carton edge and were denoted by 𝑝_Li and 𝑝_Ri. Another 50 points were respectively taken by straight line fitting, and were denoted by and . By calculation:

The errors of the left and right points were included in the array and denoted by E₁ and E₂.

Step 3: by calculating , we could obtain the mean error parameter delta. The refined point 𝑝₁ could be calculated by the formula 𝑝′ = 𝑝 + delta, which was denoted by .

Step 4: based on the refined point , straight line fitting was made clockwise or counterclockwise along the carton edge, errors between points on the straight line before and after straight line fitting were recorded, and the graph was drawn, as shown in Figure 3.

Figure 3.

Error curve.

According to Figure 3, when the point exceeds a certain value, the error curve significantly increases. In the theory, this is because the point falls outside the box contour and the error between the point and those on straight line fitting is large. Therefore, the point before the error increases is taken as another corner point. Step 4 was repeated to obtain points 𝑝₂ and 𝑝₃.

Step 5: Step 2 and Step 3 were repeated and points 𝑝₂ and 𝑝₃ were refined, and the refined points were denoted by and .

Step 6: Step 4 was repeated and the point was changed to , and the point 𝑝₄ could be obtained in the same way.

The above steps are the method used in the experiment to extract corner points based on straight line fitting, which is used in the experiment.

(3) Image matching based on epipolar constraint

In this experiment, image matching is the key step, and those before image matching are actually preparations for this step, and the accuracy of image matching directly affects the accuracy of numerical values. In traditional methods, violent matching of feature points was generally carried out. Let’s assume that there are m feature points in the left image and n feature points in the right image, then all feature points on both images need to be matched mⁿ times. Obviously, this method is inefficient, and cannot ensure the accurate matching of feature points⁸. Very limited results can be achieved even optimization conditions are added in the violent matching. Thus, it is found in the study of binocular vision principle that images are matched by epipolar constraint.

First, corner points are marked and lined up in the left image, as shown in Figure 4.

Figure 4.

Image after lining up the feature corner points in the left image.

Then, corner points are marked and lined up in the right image shown in Figure 5, where the green line is the mapping of the epipolar line of the point in the lower left corner of Figures 4 and 5.

Figure 5.

Image after lining up the feature corner points in the right image.

Through the epipolar line, we can determine that the point closest to the epipolar line is the matching point. In this way, we can easily find the matching point of each point in the left and right images by lining up each point.

4.

DATA ANALYSIS

After image matching, the image was triangulated to obtain the length⁹, width and height of the measured box. Then we analyze the experiment accuracy based on the experimental data.

In the experiment, 30 different types of boxes were selected as the measured samples, and the samples were measured several times (Unit: cm).

As shown in Table 1, it is the first set of measurement data of the selected box 1. It can be seen that the measurement accuracy of length, width and height can basically be maintained at an error rate of 5%-10%. Similar experiments were carried out in the following 30*5 groups, and the measurement data can be obtained. The measurement accuracy of length, width and height is basically kept within 10% of the error rate.

Table 1.

Measured data of box 1 (Unit: cm).

Through the above multiple sets of experiments, it can be obtained that the box size obtained by this method is close to the theoretical accuracy¹⁰, which is reliable and practical.

5.

CONCLUSION

In this paper, a binocular camera-based express delivery size measurement scheme is proposed for the regular express box. Through the analysis of multiple groups of experimental data, the accuracy and validity of the data obtained by this method are verified, which can be considered in the The measurement of the actual express box has practical application value.