An image compression method utilizing multiple sampling-rate downsampling and super-resolution upconversion is proposed. The multiple sampling-rate downsampling modes and different quantization patterns are designed for each 32  ×  32 macroblock at the encoder side. Meanwhile, the rate distortion optimization strategy is investigated to obtain the optimal downsampling and quantization mode. The chosen mode is used to downsample and code the original macroblocks. To obtain the full resolution block, the deep learning-based multiple models super-resolution upconversion is designed to reconstruct the decoded block at the decoder side. The experimental results demonstrate that our method can obtain the higher quality compressed image than JPEG and some state-of-the-art downsampling-based methods at almost all bit rates. At the same decoded image quality, our method can achieve 30% to 55% bit savings at low bit rates and 15% to 30% bit savings at medium to high bit rates. In addition, the proposed framework is applicable to other image compression standards.

Convolution filtering is one of the most important algorithms in image processing. It is data-intensive, especially when dealing with high-definition images. Most previous studies on accelerating convolution computation in parallel focus on the use of graphics processing units (GPUs), whereas the central processing units (CPUs) always play the role of host to manage the data buffer and control flow. However, recent CPU architectures have seen significant modifications to parallel data computing capabilities, and the trend of integrating the CPU and GPU on a single chip is on a rise. We propose an approach to accelerate convolution filtering on the heterogeneous architecture of integrated CPU–GPU. We exploit the parallel processing power of vector instructions on a CPU and make it collaboratively function with the on-chip GPU. Two task assignment methods, static and dynamic task partitioning, are proposed for CPU–GPU collaboration. We evaluate our approach with images and filters of different sizes. The experimental results demonstrate that we can achieve 146 GFLOP/s at best using a quad-core CPU and the performance is 2.5 to 4.8 times faster than that of the single-GPU version of the OpenCV library. We also obtain 90 times speedup over the single-threaded CPU version. The results demonstrate that the proposed algorithm is efficient.

By considering the rate distortion optimization (RDO) characteristics, a multilevel optimal λ decision algorithm for the rate control is proposed. With the process of getting the optimal λ for the group of picture (GoP)-level rate control, the frame-level rate control, and the coding tree unit (CTU)-level rate control, a special relationship of the current level λ and the next level λ can be built. First, an approximate linear bit allocation relationship is used between the bit rate and the motion compensation prediction error to assign the bit rate more reasonably for every frame. Second, the multilevel optimal λ decision scheme is proposed. To minimize the whole distortion of a sequence, an effective way is to get the optimal RDO of every GoP. Then, with the RDO process of the GoP-level rate control, an optimal frame λ ratio for each frame in current GoP can be obtained. For the optimal frame λ ratio, its value is mainly decided by the distortion effect, which is calculated by an employed source distortion propagation chain. Finally, an optimal CTU λ clip scheme is proposed to get the optimal RDO of every CTU. The experimental results show that the coding quality of the proposed rate control algorithm is improved significantly with little loss of the bit rate accuracy.

Moving object detection is one of the most promising research areas, which is required in different applications, such as video monitoring and surveillance systems, human activity recognition systems, vehicle counting, and anomaly detection. Various methods for object detection using single sensor and a few using multimodal techniques have been reported in the literature. However, such systems fail to handle adverse or challenging atmospheric conditions such as illumination variations, scale and appearance change of objects or targets, occlusions, and camouflaged conditions. We have presented an approach for the detection of moving objects using structural similarity metric (SSIM) and Gaussian mixture model (GMM). SSIM is used to compute similarity between reference mean background frame and foreground frame of visible spectrum (VIS) and thermal infrared (IR) independently. The computation of similarity measure is performed in an image spatial domain. The threshold results of SSIM are fused together using different pixel-level fusion methods such logical “OR,” discrete wavelet transform, and principal components analysis. Temporal analysis is performed to eliminate noise and false positives (unwanted background regions) using GMM on fused results. We have compared the results with recent methods for different complex scenarios and found out that approximately F-measure increases up to 80%. Hence, the proposed method proves to be a robust moving object detection technique in multimodality domain.

Long-term tracking tasks remain challenging, especially in areas of occlusion. Herein, we propose an enhanced occlusion handling and multipeak redetection method for long-term object tracking. First, our appearance model is constructed based on two complementary cues. Each model is trained independently and combined by adaptive merging, and considers the reliability of each representation to provide a preliminary estimation. Then, we present an occlusion detection scheme relying on the response variation to activate a redetection module in case of track failure. Finally, we introduce an adaptive model update strategy using the most confident tracking predictions to retain reliable memories. The redetection module is designed based on the multipeak property of the merged response and the model is updated adaptively based on the reliability of each representation and the occlusion detection result, which allows the proposed method to deal with heavy occlusions effectively. Extensive experiments are conducted on two public benchmark datasets with 100 challenging sequences. The experimental results demonstrate that the proposed method performs favorably against 17 state-of-the-art trackers while running efficiently in real time.

Image deblocking is a postprocessing method that aims to suppress the compression artifacts without changing existing JPEG coding standard. We propose an image deblocking method, which is based on deep convolutional neural networks. The proposed method takes full advantage of the characteristics of wavelet domain and pixel domain to restore the high-frequency information of compressed images and maintain low-frequency information, respectively. In addition, a fusion layer is employed to fuse the merits of two domains. Extensive experiments demonstrate that the proposed method outperforms the state-of-the-art deblocking methods in both subjective vision and objective evaluation.

Image processing methods that take place in different areas of life from identification to traffic flow control are starting to be used on mobile devices. Hence, usage of optical mark recognition systems (OMRS) on mobile devices is also not widespread as it is for image processing applications. A mobile OMRS is developed using image processing methods. In this developed system, image enhancement, edge detection, and tilt-shifting operations were performed on the optical answer paper images that are obtained using mobile devices. Within the scope of this study, it is aimed to achieve the maximum success of image processing operations on a mobile device that has limited hardware resources. The evaluation of optical answer papers that are obtained using mobile devices is performed using the image processing methods and algorithms developed within the scope of this study. The developed system is tested using different type of optical answer papers. According to the test results, the developed system provides a flexible and efficient use and partly eliminates previous work constraints. The success rate of the developed system is determined to be 93.54% using the test results obtained.

Performance of modern automated pattern recognition (PR) systems is heavily influenced by accuracy of their feature extraction algorithm. Many papers have demonstrated uses of deep learning techniques in PR, but there is little evidence on using them as feature extractors. Our goal is to contribute to this field and perform a comparative study between classical used methods in feature extraction and deep learning techniques. For that, a biometric recognition system, which is a PR application, is developed and evaluated using a proposed evaluation metric called expected risk probability. In our study, two deeply learned features, based on PCANet and DCTNet deep learning techniques, are used with two biometric modalities that are palmprint and palm-vein. Subsequently, the efficiency of these techniques is compared with various classical feature extraction methods. From the obtained results, we drew our conclusions on a very positive impact of deep learning techniques on overall recognition rate, and thus these techniques significantly outperform the classical techniques.

As a representative deep learning model, convolutional neural networks (CNNs) have accomplished great achievements in image classification and object detection. However, CNNs require the resizing of the input images to a fixed size, which may affect the representations of objects. To overcome this limitation, we replace the last pooling layer with a topic model and call it a topic network. For arbitrary sizes and ratios of input images, the outputs of the topic network are fixed-size features due to the topic model of the topic layer, and they can reflect the global or regional characteristics of images by means of different scales. Two topic models, namely latent Dirichlet allocation (LDA) and Markov topic random fields (MTRF), are applied to the topic layer, and we call them latent Dirichlet allocation topic network and Markov topic random fields topic network, respectively. Both of them perform well in image classification with original size. More importantly, as a framework, any topic model can be easily applied to the topic layer of a topic network, which makes it much more flexible and extensible.

The adaptive multiclass correlation filters (AMCF) method is proposed to exploit different kinds of features and information in a unified framework for recognition. Theoretical investigation into AMCF shows that it obtains a closed-form subsolution to constrain the optimization objective, simplifying the entire inference mechanism in the multiclass classification. The time series recognition problems, such as human action recognition and radar behavior recognition, are important yet challenging tasks. However, it is still time-consuming to acquire enough labeled training samples. AMCF is capable to exploit different kinds of features to solve the time series recognition problem. With this new correlation filters-based method, we extend the original signals and handle the insufficient training set effectively. Experiments are done on the depth image based action recognition and radar behavior recognition with a small number of training examples, including MSRAction3D, MSRGesture3D, UTD-MHAD, and radar behavior datasets. Particularly, we demonstrate that the proposed action recognition system is based on the completed local binary patterns and AMCF, and successfully achieves superior performances over the state-of-the-arts.

Although the flexible quad-tree structure in high efficiency video coding (HEVC) has achieved significant performance improvement over H.264/AVC, it remarkably increases computation complexity due to the search for the optimal coding unit (CU) size in a very large space. We propose perceptually motivated fast CU decision for HEVC intracoding based on visual regularity. Since visual regularity represents visual masking effect in the human visual system (HVS) on structure information, i.e., irregularity concealment effect, we use it for early CU termination. We predict irregular CUs by exploiting the information of neighboring CUs, which would not introduce too much distortion. Also, the distortion cannot be perceived by HVS due to the visual masking effect in irregular regions. The experimental results demonstrate that the proposed method saves 24% computational cost on average as compared to anchor with only 0.52% increase in Bjøntegaard delta bit rate and 0.02 dB loss in Bjøntegaard delta PSNR.

Human emotions are known to always have four phases in the temporal domain: neutral, onset, apex, and offset. This has been demonstrated to be of great benefit for emotion recognition. Therefore, temporal segmentation has attracted considerable research interest. Although state-of-the-art techniques use recurrent neural networks to highly increase the performance, they ignore the relevance of each frame (time step) of a video, and they do not consider the changing contribution of different features when fusing them. We propose a framework called dual-level attention-aware bidirectional grated recurrent unit, which integrates ideas from attention models to discover the most important frames and features for improving temporal segmentation. Specifically, it applies attention mechanisms at two levels: frame and feature. A significant advantage is that the two-level attention weights provide a meaningful value to depict the importance of each frame and feature. The experiments demonstrated that the proposed framework outperforms state-of-the-art methods.

Remote sensing image pansharpening involves the fusing of multispectral (MS) images with a panchromatic (PAN) image to produce an image with high-spatial as well as high-spectral resolution. We propose an improved pansharpening algorithm based on deep learning. A four-layer residual network is used as a reconstructed model to enable the accurate estimation of high-frequency details. We consider two priors to take advantage of MS information. The first prior indicates interspectral similarity, wherein the relationship between high- and low-resolution PAN images is used in the estimation of high-resolution MS images. The second prior provides the location of edges and textures according to the gradient of the PAN image. Consistency in the spectral characteristic is used as the basis in creating a pretrained model with the aim of accelerating convergence. Multiple evaluation metrics were applied to simulated and real images in order to compare the efficacy of the proposed method with that of state-of-the-art image fusion methods.

The checkerboard is a frequently used pattern in camera calibration, an essential process to get intrinsic parameters for more accurate information from images. An automatic checkerboard detection method that can detect multiple checkerboards in a single image is proposed. It contains a corner extraction approach using self-correlation and a structure recovery solution using constraints related to adjacent corners and checkerboard block edges. The method utilizes the central symmetric feature of the checkerboard crossings as well as the spatial relationship of neighboring checkerboard corners and the grayscale distribution of their neighboring pixels. Five public datasets are used in the experiments to evaluate the method. Results show high detection rates and a short average runtime of the proposed method. In addition, the camera calibration accuracy also presents the effectiveness of the proposed detection method with reprojected pixel errors smaller than 0.5 pixels.

Recent years have witnessed the rapid development of mobile video services. The diversity of mobile devices and video services can result in a difference in a user’s perceptual video quality, which presents a significant challenge for video quality evaluation. An objective assessment model is proposed that considers both video and screen characteristics, where the video characteristics include the video resolution and video coding quality (VCQ), and the screen characteristics include the screen size and screen resolution. By analyzing the relationships between the perceived video quality (PVQ) and the video and screen characteristics, an influential parameter, effective video pixels per inch (EV-PPI), is proposed that combines the two characteristics. Then, the PVQ is estimated with the VCQ and EV-PPI. Experiment results confirm that the proposed method can effectively estimate the user’s PVQ. It can be used as a guideline for both video service providers and mobile device manufacturers to improve the performance of their services and devices.

This work proposes an improved ranking model of learning local feature descriptors for matching image patches by introducing a variance shrinkage constraint. Previous ranking losses, such as triplet ranking loss and quadruplet ranking loss, have proven powerful in separating corresponding patch pairs from noncorresponding ones. However, they are unable to restrict the intraclass variation since they are only designed to keep noncorresponding pairs away from corresponding ones. Consequently, those scattered pairs get mixed up near the separating hyperplane, which are difficult to discriminate and may disrupt the performance. To resolve this problem, we introduce a variance shrinkage constraint that aims to reduce the variance of patch pairs in the same class and force them to be close to each other. The combination of ranking losses and the variance shrinkage constraint can efficiently reduce overlaps between patch pairs of different classes, which is confirmed by our experiments. Experiments also show that our model achieves a significant improvement in performance compared with original ranking models and other latest methods.

Shifted superimposition is a resolution-enhancement method that has gained popularity in the projector industry the last couple of years. This method consists of shifting every other projected frame spatially with subpixel precision, and by doing so creating a new pixel grid on the projected surface with smaller effective pixel pitch. There is still an open question of how well this technique performs in comparison with the native resolution, and how high the effective resolution gain really is. To help investigate these questions, we have developed a framework for simulating different superimposition methods over different image contents, and evaluate the result using several image quality metrics (IQMs). We have also performed a subjective experiment with observers who rate the simulated image content, and calculated the correlation between the subjective results and the IQMs. We found that the visual information fidelity metric is the most suitable to evaluate natural superimposed images when subjective match is desired. However, this metric does not detect the distortion in synthetic images. The multiscale structural similarity metric which is based on the analysis of image structure is better at detecting this distortion.

Person’s identity validation is becoming much more essential due to the increasing demand for high-security systems. A biometric system testifies the authenticity of specific physiological or behavioral characteristics-based biometric technology. This technology has been successfully applied to verification and identification systems. We analyze the multispectral palmprint biometric identification system in unimodal and multimodal modes. In an identification system, the feature extraction is a crucial step. For this reason, we propose an efficient deep learning feature extraction algorithm called discrete cosine transform network (DCTNet). The effectiveness of the proposed approach has been evaluated on two publicly available databases: CASIA and PolyU. The obtained results clearly indicate that the DCTNet deep learning-based feature extraction technique can achieve comparable performance to the best of the state-of-the-art techniques.

Video quality assessment (VQA) is becoming increasingly important as a comprehensive measure of video quality. This paper proposes a full-reference VQA (FR-VQA) algorithm based on the motion structure partition similarity of spatiotemporal slice (STS) images. To achieve this objective, a number of FR-image quality assessment algorithms were applied slice by slice to video STS images to compare their performance of detecting structure similarity of STS images. The algorithm that performed the best was selected to detect the similarity between motion-partitioning STS images. Next, as motion objects in the video sequence were found to have different influences on the prediction performance in terms of moving speed and track, the STS images were divided into simple and complex motion regions, and their contributions to the VQA task determined. Consequently, a promising effective and efficient VQA model, called STS-MSPS, is also proposed. Experimental evaluations conducted based on various annotated VQA databases indicate that the proposed STS-MSPS achieves state-of-the-art prediction performances in terms of correlations with subjective evaluation and statistical significance tests. This paper also shows that STS images by themselves provide sufficient information for VQA tasks and that the proposed complex motion region of an STS image is predominantly responsible for yielding a high-precision model.

Biprism-based monocular stereovision systems can acquire different views of the same object at a single shot, which have several advantages over conventional two-camera systems. To measure the position or recover the shape of the object, effective image pairs have to be captured first, which depends on the field of view of the system. In this paper, we propose a common method for establishing a practical biprism-based monocular stereovision system. The relationship between system parameters and object distance has been analyzed in details. The criterion for parameter optimization and procedure for system setup have been introduced. Experimental results show that our method can effectively calculate different conditions under different parameters and optimize the parameter configurations for the particular system.

Object tracking plays a vital role in many computer vision systems applications, such as video surveillance, robotics, 3-D image reconstruction, medical imaging, human computer interface, etc. In many proposed approaches, feature-based object tracking is widely used due to its accuracy. Feature extraction and feature correspondence are two main components of feature-based object tracking. In our proposed method, we have used gray level co-occurrence matrix (GLCM) features for object tracking in thermal imagery. As spatial resolution of the thermal sensor is fairly coarse, it also implies that temperature scales are close but may not exactly be the same, which indicates the presence of mutually related pixels or group of pixels. The GLCM texture analysis is based on assumption that the texture information of an image is an average spatial relationship between the gray tones in the image. Thus, this similarity in spatial resolution properties makes GLCM features suitable for object tracking in thermal infrared imagery. Initially, the target blobs to be tracked are provided by object detection stage. Then, for a given target blob in a frame, we first calculate GLCM feature points and then find corresponding features in the next successive frame. The sum of squared differences between two feature point sets is calculated to find feature correspondence between two frames for object tracking. Simultaneously, the codebook of the center of the blobs for prediction of the target candidate region in the next frame is maintained in order to have robust object tracking under occlusions. GLCM-based object tracking in thermal imagery outperforms the color or LBP-based mean-shift approach. This algorithm is also able to track objects in split and merge condition. The accuracy of this algorithm also depends on the object detection stage, such as Kalman tracking.

With the increasing size of video sequences, the dramatically increased access bandwidth and power consumption for fetching and storing reference frames with external memory during motion estimation become the bottleneck of realizing a practical video codec system. To reduce the access bandwidth and power consumption with no quality loss for reference frames, a lossless reference frame compression method based on pixelwise adaptive prediction is proposed. The algorithm allocates pixels from the first line and first column with horizontal prediction and vertical prediction separately. Meanwhile, two direction coefficients of horizontal and vertical are calculated to evaluate the similarity between two adjacent samples. For pixels in the rest locations, prediction value is assigned to every pixel adaptively from left, upper, and upper left adjacent reconstructed pixels according to the appropriate mode decision scheme based on similarity coefficients. At last, residuals are compressed by Huffman encoding process to generate bit-stream stored into external memory. By implementing the algorithm with high-efficiency video coding (HEVC) reference software HEVC test model-16.5 (HM-16.5), the experimental results show that the method can obtain 63.83% data reduction ratio averagely for luminance component.

Ghosts always appear in the multiexposure fusion for dynamic scenes. Most of the existing deghosting methods require complicated and subjective parameter settings, easily resulting in many artifacts. A robust exposure fusion method that removes ghosts while preserving the details of the image is presented. The proposed method first detects the motion area by bidirectional intensity mapping and calculates the motion weight. Then, the latent images are generated by weighted optimization in gradient domain, and ghosts are removed automatically. After removing the ghosts, fusion weights are calculated using a patch-based fusion method, and exposure fusion is performed with details retained and outlier values suppressed. Experiments show that the proposed method can remove ghosts effectively, which demonstrates strong robustness to ghosts in a variety of circumstances. It also avoids influence from complicated and subjective parameter settings.

This paper proposes a recognition system for handling Arabic literal amounts based on three proposed strategies. First, an enhanced feature extraction model based on the part of Arabic word (PAW) analysis is introduced. It combines both the statistical features extracted through the word and the structural ones extracted through its PAWs. Second, the existing lexicon used in literal amounts is first partitioned into four subgroups according to the number of PAWs of the candidate word. Each subgroup is independently learned by a corresponding set of support vector machines (SVMs). However, several candidate words may contain some touching PAWs. For this reason, a proposed preclassification step is included and the classification of each candidate word is simultaneously done with and without correction of the touching PAWs. In both cases, the candidate sample is presented to the corresponding set of SVMs, where the two classification scores are computed. Then, the final decision is given according to the better score. Experimental results based on the Arabic handwritten database dataset confirm the effectiveness and robustness of the proposed recognition system in terms of classification accuracy, which reaches 95.91%.

The quality of screen content images (SCIs) plays a crucial benchmark of displaying the system and directly affects the user experience of multidevice communication. However, accurately predicting the quality of SCIs with an objective algorithm still has many challenges. Hence, we propose a new no-reference method for evaluating the quality of SCIs effectively and efficiently. Specifically, we first propose a simple but effective strategy to partition the SCIs into the rough and smooth regions. Then, a most preferred structure (MPS) metric is derived to represent the meaningful part of each image patch according to the free-energy theory of human brain. Subsequently, the MPSs of the rough and smooth regions are pooled for constructing the rough and smooth features of SCIs, separately. Finally, to evaluate the final visual quality, the rough and smooth features are combined as the quality-aware features of SCIs, and a learning-based framework is employed to map the quality-aware features to objective quality scores. The experimental results show that the performance of the proposed method is highly consistent with the human perception and superior to the state-of-the-art quality assessment methods on the SCI-oriented databases. Furthermore, the proposed method delivers a high computational efficiency and is very suitable to be integrated into the displaying system.

A camera-based scheme is proposed for detecting vehicles at user-defined virtual loops, simulating the operation of inductive loops. False vehicular detections are minimized by a combination of efficient edge detection and color information. The experimental results suggest that the proposed scheme potentially can detect and count vehicles at user-defined virtual loops accurately (with more than 98% correct detection rate, in average), besides being more robust to cast shadows and sudden illumination changes than comparable methods that represent the state of the art.

Human action recognition is a challenging task in machine learning and pattern recognition. This paper presents an action recognition framework based on depth sequences. An effective feature descriptor named depth motion maps pyramid (DMMP) inspired by DMMs is developed. First, a series of DMMs with temporal scales are constructed to effectively capture spatial–temporal motion patterns of human actions. Then these DMMs are fused to obtain the final descriptor named DMMs pyramid. Second, we propose a discriminative collaborative representation classifier (DCRC), where an extra constraint on the collaborative coefficient is imposed to provide prior knowledge for the representation coefficient. In addition, we apply DCRC to encode the obtained features and recognize the human actions. The proposed framework is evaluated on MSR three-dimensional (3-D) action datasets, MSR hand gesture dataset, UTD-MHAD, and MSR daily Activity3D dataset, respectively. The experimental results indicate the effectiveness of our proposed method for human action recognition.

An iterative texture-oriented image denoising technique is proposed for the restoration of images corrupted with random-valued impulse noise (RVIN). The proposed technique opts a switching approach that first identifies the pixels corrupted by RVIN and then estimates their intensity values to restore the images. Textons of distinct orientations conforming to bilateral symmetry are proposed for the identification of corrupted pixels. To estimate the intensity values of the identified corrupted pixels, the textons having local similarity are used. As the textons are fundamental elements of texture perception, the proposed technique preserves the texture information of images, effectively. The performance of the proposed denoising technique is evaluated on standard benchmark test images under various intensities of RVIN by comparing it with state-of-the-art techniques. The simulation results depict the significant performance of the proposed denoising technique for low as well as higher intensities of RVIN.

To address the scale invariance and the mismatching problems of ORB (oriented FAST and rotated BRIEF), an improved algorithm based on multipurification was put forward and applied in PCB matching and positioning. ORB feature points were initially extracted from both the template image and the target image. Rough matching was then performed by the combination of k-nearest neighbor (k-NN) algorithm and best bin first search algorithm. Considering the existence of a large number of mismatches, the multipurification that consists of neighborhood ratio, bidirectional matching, and cosine similarity was used to purify the set of matching pairs. Finally, the affine matrix between the two images was solved by progressive sample consensus. The experimental results showed that the matching accuracy was significantly improved after the purification by the proposed method. The improved algorithm outperformed original ORB under the condition of image rotation and scaling. The mean rotation angle error was just 1/10th of that of original ORB. The time overhead of the improved algorithm was comparable to that of original ORB, which was about six times faster than that of speeded up robust features and 30 times faster than that of scale-invariant feature transform. The enhanced algorithm demonstrated great advantages in the improvement of accuracy, time overhead, and robustness for the PCB alignment.

Three-dimensional (3-D) holoscopic imaging is a candidate promising 3-D technology that can overcome some drawbacks of current 3-D technologies. Due to the particular optical structure, a holoscopic image consists of an array of two-dimensional microimages (MIs) that represent different perspectives of the scene. To address the data-intensive characteristics and specific structure of holoscopic images, efficient coding schemes are of utmost importance for efficient storage and transmission. We propose a 3-D holoscopic image-coding scheme using a sparse viewpoint image (VI) array and disparities. In the proposed scheme, a holoscopic image is decomposed into a VI array totally and the VI array is sampled into a sparse VI array. To reconstruct the full holoscopic image, disparities between adjoining MIs are calculated. Based on the remainder set of VIs and disparities, a full holoscopic image is reconstructed and encoded as a reference frame for the coding of the full holoscopic image. As an outcome of the representation, we propose a multiview plus depth compression scheme for 3-D holoscopic images coding. Experimental results show that the proposed coding scheme can achieve an average of 51% bit-rate reduction compared with high efficiency video coding intracoding.

Visual tracking is a challenging computer vision problem with numerous practical applications. We propose a convolutional features selection-based tracking framework to improve accuracy and robustness. First, we investigate the impact of features extracted from different convolutional neural network layers for the visual tracking problem. Second, we learn correlation filters on each layer outputs to encode the target appearance and design a fluctuation detection technique to select the appropriate convolutional layers, which can improve the target localization precision and avoid drifting caused by the challenging factors, such as occlusions and appearance variations. Third, we present an improved model update strategy to keep positive samples while removing corrupted ones. Extensive experimental results on the OTB-2013 and OTB-2015 benchmarks demonstrate that the proposed algorithm performs favorably against several state-of-the-art trackers.

Text detection in natural scenes has long been an open challenge and a lot of approaches have been presented, in which the deep learning-based methods have achieved state-of-the-art performance. However, most of them merely use coarse-level supervision information, limiting the detection effectiveness. We propose a deep method utilizing coarse-to-fine supervisions for multiorientation scene text detection. The coarse-to-fine supervisions are generated in three levels: coarse text region (TR), text central line, and fine character shape. With these multiple supervisions, the multiscale feature pyramids and deeply supervised nets are integrated in a unified architecture, and the corresponding convolutional kernels are learned jointly. An effective top-down pipeline is developed to obtain more precise text segmentation regions and their relationship from coarse TR. In addition, the proposed method can handle texts in multiple orientations and languages. Four public datasets, i.e., ICDAR2013, MSRA-TD500, USTB, and street view text dataset, are used to evaluate the performance of our proposed method. The experimental results show that our method achieves the state-of-the-art performance.

Deep metric learning is an effective method for person reidentification. In practice, impostor samples generally possess more discriminative information than other negative samples. Specifically, existing triplet-based deep-learning methods cannot effectively remove impostors, because they cannot consider congeners of impostor and it may produce new impostors when removing existing impostors. To utilize discriminative information in triplets and make impostor and its congeners more clustering, we design oversymmetric and overasymmetric relationships and apply these two constraints to triplet and impostors’ congeners to train our deep triplet-group network with original individual images rather than handcrafted features. Extensive experiments with five benchmark datasets demonstrate that our method outperforms the state-of-the-art methods with regards to the rank-N matching accuracy.

A saliency detection method based on the PageRank algorithm and local spline regression (LSR) is proposed. Unlike the principles of most existing bottom-up methods, in the proposed method, saliency detection is considered a label propagation and regression problem. The input image is represented as two-scale graphs with homogeneous superpixels. Multiple color features and spatial information are effectively captured to define the relevance of each node to its surroundings. PageRank is used to assign the saliency value to each region depending on the similarity of the image elements with boundary cues. Furthermore, to comprehensively consider both foreground and background cues, a robust classifier based on LSR is employed to highlight more complete foreground regions. The integrated and refined pixel-level saliency map provides a significant performance boost in the final result. Extensive experiments on three large public datasets demonstrate that the proposed algorithm consistently achieves superior performance compared with state-of-the-art saliency models.

Finger-knuckle-print (FKP) is considered as one of the emerging hand biometric traits due to its potentiality toward the identification of individuals. However, extracting features out of poor contrast FKP images is the most challenging problem faced in this area. We propose a method for personal recognition using FKP images based on a preprocessing step to improve the contrast of input FKP image and a processing step for features extraction. In the first part, we compared the performances of different histogram equalization-based contrast enhancement algorithms. The enhanced image with better performance is considered in a second step for feature extraction and personal identification. We experimentally compared the proposed approach to other existing approaches in literature using PolyU FKP database framework, and results show that our technique performed favorably.

Annotations of subject IDs in images are very important as ground truth for face recognition applications and news retrieval systems. Face naming is becoming a significant research topic in news image indexing applications. By exploiting the uniqueness of name, face naming is transformed to the problem of multiple instance learning (MIL) with exclusive constraint, namely the eMIL problem. First, the positive bags and the negative bags are automatically annotated by a hybrid recurrent convolutional neural network and a distributed affinity propagation cluster. Next, positive instance selection and updating are used to reduce the influence of false-positive bag and to improve the performance. Finally, max exclusive density and iterative Max-ED algorithms are proposed to solve the eMIL problem. The experimental results show that the proposed algorithms achieve a significant improvement over other algorithms.

Domain adaptation (DA) solves a learning problem in a target domain by utilizing the training data in a different but related source domain, when the two domains have the same feature space and label space but different distributions. An unsupervised DA approach based on iterative landmark selection and subspace alignment (SA) is proposed. The proposed method automatically selects source landmarks from the source domain and iteratively selects target landmarks from the target domain. These well-selected landmarks accurately reflect the similarity between the two domains and are applied to kernel projection of both source and target samples onto a common subspace, where SA is performed. In each iteration, target labels are updated by a classifier that is retrained with the source samples aligned with the target domain. Thus, the distribution of the selected target landmarks gradually approximates the distribution of the source domain. During landmark selection, the quadratic optimization functions are constrained such that the proportions of selected samples per class remain the same as in the original domain, which makes the problem easy to solve and avoids setting hyperparameters. Comprehensive experimental results show that the proposed method is effective and outperforms state-of-the-art adaptation methods.

Existing methods for touchless fingerprint recognition using a phone camera have the difficulties to quickly verify and extract fingerprint images with high recognition accuracy. It is because specialized techniques for touchless fingerprint images were not provided for them. To overcome these difficulties, three methods are proposed. First, a finger-shaped guide is displayed on the phone screen to enable the user to position their fingers correctly and to focus the camera on the position to quickly produce a good fingerprint image. Second, the profile line tests at selected points on the guide are proposed to quickly determine whether the user’s fingers are properly placed according to the guide shape and to simplify the segmentation of fingerprint image regions. Third, a existing local structure-based matching method is improved for touchless fingerprint images, and a score-level fusion of the three finger matchings is performed to ensure high recognition accuracy. Experiments were performed using our custom-built database, where touchless fingerprint images were collected using a 13-MP mobile camera in various indoor and outdoor environments. Resulting error rates were between 0.50% and 1.04% depending on whether the image was captured indoors or outdoors.

This paper presents a temporal video error concealment method specially designed for H.265/HEVC. We propose the quad-tree partitioning prediction and the coherency sensitive hashing in order for better error concealment performance in the corrupted frames via HEVC codec. First, we try to deduce the most probable partitioning of the missing coding tree unit (CTU) using the proposed quad-tree partitioning prediction, which generates several CUs that constitute the CTU. Then, a coding unit (CU) priority choosing method is applied to select the best one from these CUs for prior concealment. Last, the coherency sensitive hashing is adopted for concealing the chosen best CU for better searching quality. The experiments shows that the recovery performance of the purposed method surpasses the compared state-of-the-art methods since the quad-tree partitioning prediction, the priority choosing process, and the coherency sensitive hashing help to improve the overall performance.

In order to obtain accurate and stable image stitching results, we propose a stitching method for two images captured from different viewpoints based on correlation transformation. Aiming at resolving the limitation of the projective transformation that is commonly used in image stitching, a transformation called dual-correlation transformation is proposed in this paper. First, the estimation result of the fundamental matrix is calculated by the direct linear transformation based on the corresponding points in two images. Second, according to the presented dual-correlation transformation, a pair of correlation transformation matrices that are needed for dual-correlation warp can be obtained to realize the correspondence of each pixel in different images. Up to this stage, the method of image stitching based on transformation matrices has been accomplished. Finally, an optimization method based on factorization is especially proposed to solve the discontinuity problem that may occur in the dual-correlation warp. The experimental results and analyses show that the proposed method can achieve more accurate and natural stitching effects and has less computing time of the images in separate scenes compared with other similar methods.

Person reidentification (Re-ID) aims at establishing global identities for individuals as they move across a camera network. It is a challenging task due to the drastic appearance changes that occur between cameras as a consequence of different pose and illumination conditions. Pairwise matching models yield state-of-the-art results in most of the person Re-ID datasets by capturing nuances that are robust and discriminative for a specific pair of cameras. Nonetheless, pairwise models are not scalable with the number of surveillance cameras. Therefore, elegant solutions combining scalability with high matching rates are crucial for the person Re-ID in real-world scenarios. We tackle this problem proposing a multicamera nonlinear regression model called kernel multiblock partial least squares (kernel MBPLS), a single subspace model for the entire camera network that uses all the labeled information. In this subspace, probe and gallery individual can be successfully matched. Experimental results in three multicamera person Re-ID datasets (WARD, RAiD, and SAIVT-SoftBIO) demonstrate that the kernel MBPLS presents favorable aspects, such as the scalability and robustness with respect to the number of cameras combined with the high matching rates.

Nighttime vehicle detection is part of an intelligent transportation system for road safety, driving navigation, and surveillance at night. However, previous nighttime vehicle detection methods only deal with a single class or two classes of vehicles. This paper presents an effective system based on cascade feature selection and a coarse-to-fine mechanism for detecting preceding multiclass vehicles at night. First, we train a coarse-level classifier using contrast features. Second, we combine a cascade selection framework with feature mutual correlation, similarity and regions’ overlap to select a set of image regions, where we can extract effective features. Three features, including local binary pattern, histogram of oriented gradients, and four direction features, are extracted from the selected regions for training a fine-level multiclass vehicle classifier. During the detection stage, we utilize a coarse-to-fine mechanism. In the coarse level, a multiscale sliding window is classified by the coarse-level classifier to find regions of interest (ROIs) that are likely to be vehicles. In the fine level, these ROIs are identified by the trained multiclass vehicle classifier. Evaluations on a Hong Kong nighttime multiclass vehicle dataset show that our proposed system successfully detects a car, taxi, bus, and minibus within nighttime images under different scenes. Quantitatively, our proposed system obtains 95.48% detection rate at 0.055 false positives per image, outperforming some state-of-the-art detection approaches including two current nighttime vehicle detection methods, and two deep learning-based methods.

We propose a total variation-based variational model for nonblind binary image deblurring. The binary constraint is considered using the double-well function as the penalty term. We show the existence of a minimizer for the proposed model. By using operator splitting and alternating split Bregman, we get an effective numerical algorithm for the proposed model. Different from the existing methods in which the binary values are assumed to be known, our method can estimate the binary values automatically in the iteration process. Numerical results and comparisons demonstrate that the proposed algorithm is promising.

To speed up the segmentation procedure and improve the segmentation quality of synthetic aperture radar (SAR) image, this paper presents an adaptive unsupervised SAR image segmentation method, which is integrated in the grayscale histogram and perceptual hash algorithm. The final clusters can be acquired self-adaptively without predefining. The two stages of the algorithm are discussed in details: histogram-based preprocessing and region merging. In preprocessing, histogram-based gray-level reduction is adopted to generate the set of initial segmented regions to address the problem of over-segmentation. When merging regions, the similarity of different regions can be obtained by comparing their different fingerprints calculated in the perceptual hash algorithm, which is the kernel of the whole SAR image segmentation method. The application of the perceptual hash algorithm for image segmentation can reduce the computational complexity and improve the efficiency of region merging. Experimental simulations and theoretical analysis are used to demonstrate the effectiveness of this algorithm.

Nowadays, with the proliferation of data exchange across the Internet and the storage of sensitive data on open networks, intensive works are done to secure data. Recently, researchers made more considerable effort to utilize biometric features instead of a memorable password to authenticate the user identity. Based on the theories of chaos map, which are known to possess desirable properties of pseudorandomness, high sensitivity to initial conditions, and very large key space, we propose a biometric cryptosystem for online users’ authentications. The biggest advantage of our proposal is the secure nature of the key. In fact, our system is designed first so that the key never needs to be transmitted or revealed but it will be derived from biometric template and second even if we introduce the same biometric template twice, our system generates different key values. In order to assess the effectiveness of our approach, a biometric-based online automatic teller machine system is proposed and evaluated. Experimental results and statistical analysis, using palmprint as well as palm-vein databases of 200 users, are presented. The obtained results show that the proposed scheme is more secure, fast, and points at increased authentication accuracy. Therefore, it can be implemented in real-time applications, which will undoubtedly help to promote more widespread use of biometric cryptosystems based on chaos theory.

The image quality was studied using event-related potentials (ERPs), behavioral data, and psychological measurement. In our experiment, different quality images were obtained by changing σ values in the Gaussian kernel function. The results showed that measurable ERPs, behavioral data, and psychological parameters varied with the change in the image quality. The more impaired image elicited the higher and earlier ERPs components over all scalp areas. Meanwhile, the response time reduced and detection rate increased with an increase in the blurriness level σ. The P300 and N200 components of different scalp areas elicited by different image qualities were analyzed. The N200 amplitude and P300 latency over the anterior scalp region were stronger and shorter than the ones over central and posterior scalp regions. The P300 and N200 amplitude over the middle scalp location were stronger than the ones over left and right scalp locations. The influence of image content is not significant. The image quality could be assessed by P300 and N200 amplitude over the nine scalp areas, and the anterior and middle scalp areas have higher PLCC, SROCC, and KROCC values than other scalp areas. The influence of P300 amplitude on the estimated scores is smaller than that of N200 for most scalp areas. The result had a good correlation with subjective rating scores, which proved the feasibility of utilizing ERPs to assess image quality.