In parallel computing applications, containerization technology provides an efficient, reliable, and scalable way to manage and deploy applications. Additionally, RDMA (Remote Direct Memory Access) technology meets the low-latency, highbandwidth, and high-performance network communication requirements of parallel computing applications. This is achieved through the use of low-latency, high-bandwidth, and high-performance network communications. The objective of this research is to investigate the utilization of containerization techniques in combination with RDMA and IB (InfiniBand) networks. This is achieved by packaging applications and their dependencies in independent, portable containers and utilizing RDMA technology for fast data transfer and processing. The result is an efficient and flexible operating environment that is comparable in efficiency to physical machines. This research provides an in-depth study of how containerization techniques and the advantages of RDMA combined with IB networks can improve the performance and efficiency of massively parallel computing applications.
Cloud-native virtualization technology combines virtualization technology with cloud-native computing to provide a more efficient, flexible, and scalable cloud computing environment. In the process of analysis and research in the field of bioinformatics, it is usually necessary to deal with large-scale data sets and complex computing tasks, and the demand for computing power throughout the research and development cycle is characterized by peaks and troughs. The elastic scalability of cloud-native virtualization technology allows for the expansion of computing resources according to demand, meeting the data processing and analysis requirements throughout the entire research and development cycle. By integrating virtualized InfiniBand high-speed NICs, data transfer and the execution of computational tasks are accelerated, further reducing the research and development cycle. In summary, cloud-native virtualization technology has significant application value in the field of bioinformatics, providing an efficient computing environment while saving time and costs.
To evaluate the resource usage in the scheduler queue, assist users in selecting appropriate queues for fast calculation, and improve the throughput and utilization of the system in the high-performance computing platform, it is necessary to use the historical job data in the queue for data analysis, and then make timely and effective predictions on the number of nodes occupied by jobs. In this paper, a queue node occupancy prediction method based on improved Convolutional Neural Network (CNN) and Long Short Memory network (LSTM) is proposed. Cluster and group the historical data of high-performance clusters to obtain balanced samples. In the improved CNN-LSTM network, the weights of different channels are determined by replacing the pooling layer with the customized attention mechanism layer. Samples are selected and extracted, and L2 regularization is used to prevent over fitting training to obtain more accurate results of node resource occupation. The test results of historical operation data of a supercomputer show that the improved CNN-LSTM hybrid network model proposed in this paper is more accurate than CNN-LSTM hybrid network model, LSTM model, MLP model and random forest model.
The progress of scientific development requires the use of high-performance computers for large-scale simulations, resulting in a significant communication overhead and thereby constraining the computer's performance. To address this issue, optimizing the topological mapping of application processes to computing nodes is essential for enhancing the communication performance of high-performance computers. However, this topic has not been extensively explored in the literature. In order to reduce the communication overhead of high-performance applications, this study formulates the optimization of topological mapping from application processes to computing nodes as a quadratic allocation problem. The proposed method collects communication features to assess the communication intimacy between processes and considers the communication relationship between application processes and network topology. To overcome the limitations of traditional genetic algorithms, this study introduces elite learning and adaptive selection into the mutation operator. In this algorithm, individuals undergoing mutation learn from fragments of the best individuals in the current population. Additionally, three functions are selected to control the probability of selecting the elite learning mutation during the mutation process, thereby enhancing the algorithm's efficiency and accuracy. The results of the experiments demonstrate that the suggested methodology yields a noteworthy enhancement in communication performance compared to the widely adopted round-robin approach in NPB test suites. Furthermore, the enhanced genetic algorithm displays superior optimization efficiency in comparison to conventional genetic algorithms and other heuristic approaches.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.