2.1

Multi-Granularity Clustering Theory

Human perception of images has a large-scale priority characteristic. The human brain observes things starting from the big features and moves from the general to the details¹³. Although the information received is comprehensive, when analyzing these things, people always start with a macroscopic perspective and then delve into the micro level of things¹⁴. Just like when people see a selfie, they always first see the outline of the face in the photo, then look at the nose, eyes, and some small features. Based on the granularity theory, this paper proposes the granularity cognition theory¹⁵. Currently, granule theory has been applied to rough sets, sampling, classificatio, and has achieved good results. The applicability and authenticity of granule theory have been validated.

Definition 1: Given a data set D ⊆ R^N and D’ ⊆ D (D’ ≠ ∅), the granule ball GB generated is defined as having a center point C, a radius r, and a set of all points in the ball as D’. C is the center of all sampling points in the ball, r is the average distance from all sampling points to the center of the ball. For any point x_i∈D’ (i = 1, 2, …, K), where K is the number of data points in the ball.

Finally, in the resulting clustering, some points may be isolated into very small spheres between several larger ones. However, according to the granularity cognitive theory, the absence of these points will not affect the overall judgment. These points are equivalent to outliers in the overall sample, and their small number will not affect the results. It is possible to choose better-quality data to replace the entire dataset, as long as the classification boundary is well described at the granularity scale. The points within the sphere do not need to be identical, as sometimes noise needs to be tolerated. Although these points are close during clustering, their actual labels may differ significantly, indicating that they are likely noise points. Therefore, the purity of the sphere needs to be adjusted as required. If these points are too numerous, the sphere will be large, and if the sphere cannot tolerate outliers, it will be small¹⁶. If all spheres do not have outliers, the result is the same as that without clustering, and the noise is not processed at all¹⁷.

Definition 2: Given a data set D⊆R^N and D’⊆D (D’≠∅), the granule GB generated will have its center defined as C, and its point set defined as D’. C is the center of all the sampling points in the sphere, and it is not a real sample. r_i is the distance from x_i to C. For any point x_i∈D’ (i=1,2,…,K), where K is the number of data points in the sphere, |y| represents the number of points where r_i<threshold.

According to the above concepts and theories, the process of forming granules can be obtained. The first step is to randomly generate m balls that cover most of the samples through clustering, as shown in Figure 1(a), where m is the number of balls. The focus is on the second step of classifying the data set. Then, the centers of two granules are determined based on Definition 1. If the purity requirements are not satisfied between two granules, the granules are split further to generate more granules, as shown in Figure 1(b). The granules that meet the purity conditions remain unchanged and stop splitting, while the others continue to split until the purity of all cores meets the requirements, as shown in Figure 1(c). It can be seen that after all the balls are divided, all sample points are successfully clustered together.

Fig.1

granule iterative process.(a)first iteration; (b) intermediate results; (c) Granule clustering complete.

2.2

Multi-Granularity Image Clustering Algorithm

The main idea of the granular ball image clustering algorithm is shown in Figure 2(a). Figure 2(a) shows the distribution of each pixel in the feature space, which is represented in two-dimensional space, but in reality, it may be multidimensional due to the RGB channels and spatial coordinate information of the image. In Figure 2(b), all the points are clustered into spheres represented by B_i (i=1,2,…,n). These spheres can be treated as a whole, represented by their centers. After obtaining these spheres, the algorithm calculates the purity of each sphere through the similarity measure in the sphere, if the sphere purity does not meet the conditions, continue to split the sphere through 2-means, as shown in Figure 2(c). In the end, all the spheres are clustered successfully, and the data points not in the spheres are noise points, which need to be removed, as shown in Figure 2(d). As shown in Figure 2(e), there is no more noise in the final data.

Fig. 2.

The splitting process of granules in the dataset (a) The original data set;(b) granule initial disintegration; (c) Granular iterative clustering; (d) Remove noise points in the ball; (e) get final data.

2.3

Analysis of denoising in granular image clustering

The particle clustering algorithm is described as Algorithm 1 above. The multi-scale clustering denoising algorithm for image data mainly consists of three parts: image pixel sample generation, sample multi-scale clustering, and clustering result processing. In Step 2, the image is first converted into a two-dimensional data format, where the image of size (h, w, 3) is transformed into (w*h, 3), with each pixel’s feature being its RGB value. The second step is to cluster all samples. Here, the entire data set is initially treated as a large sphere, and then, in Steps 6 to 18, the algorithm iteratively clusters the data until the number of spheres no longer changes, indicating the clustering has become stable. In Steps 7 to 17, the algorithm first performs purity analysis on a sphere, comparing the data points in the sphere to its center, and identifying the number of noisy data points that are farther than the given threshold from the sphere center. If the number of noisy points is greater than the given purity value, the algorithm then performs a 2-kmeans clustering on that sphere (B_i). In this way, the size and number of sphere divisions are determined based on the threshold and purity, where the threshold is typically fixed, but the purity can be designed for different scenarios.

Algorithm 1:

Generate image pixel samples

3.1

Comparison of image results at different purities

First, a normal image is used and its pixels are converted into different sample points for clustering. The purity can be set to a fixed value, and the number of final images can be viewed by adjusting the threshold. Alternatively, the threshold can be fixed, and the purity can be adjusted to observe the effect of the image and the number of particles. The following results show the difference between the images obtained by basic k-means clustering and those obtained by multi-scale clustering. The specific effect is shown in Figure 3.

Fig. 3.

K-means algorithm image segmentation results

These three images show the results of k-means clustering with k=2, k=10, and k=50, respectively. It can be clearly seen that as the number of clusters increases, the colors in the image become more diverse. The specific effect is shown in Figure 4.

Fig. 4.

The image processing result of this algorithm

These three images show the cases where the difference between the gray value of the points inside the ball and the gray value of the center point is within 40, 20, and 10, respectively.

Compared with the results obtained by k-means clustering, the images processed by the multi-scale clustering algorithm show significant improvements in the color difference on both sides of the edges. This is because the value of k in k-means is fixed, which may cause even pixels with similar colors to be classified into different clusters, leading to a significant difference in colors on the edges or even the edges being clustered with the colors on both sides. Such differences can have a significant impact on image classification, which requires more details from the images. Therefore, k-means is not suitable for image processing before image classification.

In the multi-scale clustering algorithm, the splitting conditions for the particles involved in image classification, namely the threshold and purity, actually only need to adjust one of them during actual use. The purity is mainly based on the quality of the image. If the quality is not good, the purity can be reduced to prevent interference. The threshold adjustment scenario is based on whether the colors in the image are rich or not. If the colors in the image are monotonous, the threshold can be reduced to retain more details. However, in the case of rich colors, unnecessary features can be reduced to prevent them from interfering with the classification results.

3.2

Image performance under noise attack

This experiment uses Gaussian noise to add noise to MNIST data, and this experiment mainly uses Resnet as a classifier, and then uses this algorithm to process the image after adding noise, and compares the results with the original results. The results are shown in Table 1:

Table 1.

Effect of neutrophil processing on images under Gaussian noise.

It can be seen from the results that the prediction accuracy of the classification model continues to decline rapidly under the condition that the Gaussian noise intensity of the image without multi-granularity processing is continuously strengthened. However, after the image is processed, it can be seen that the prediction accuracy of the image disturbed by noise in the classifier decreases slower.

Gaussian noise is just a common natural noise. The threat to classifiers is adversarial example attacks. Adversarial example attacks can specifically perturb a model, causing it to be misclassified. Therefore, the following will use FGSM to attack the image, and then test whether the algorithm can resist the attack. Since the adversarial example attack is best used for color images, the data set used in this experiment is CIFAR10, and the attack results are shown in Table 2:

Table 2.

Effect of neutrophil processing on images under FGSM