A Cluster Scheduler for Real-time GOCI-II Data Processing in Heterogeneous Computing Systems

Title
A Cluster Scheduler for Real-time GOCI-II Data Processing in Heterogeneous Computing Systems
Author(s)
허재무; 한희정; 양현; 정재훈
KIOST Author(s)
Heo, Jae Moo(허재무)Yang, Hyun(양현)
Publication Year
2018-05-10
Abstract
In June 2017, we conducted a Critical Design Review (CDR) of GOCI-II Ground Segment (G2GS) system and are currently developing G2GS system in line with the launch of the GOCI-II in 2019. As the performance improvement of the GOCI-II sensor, the amount of data to be processed has increased gradually, such as 4 times the spatial resolution, 2 times the L2 product, and 1.5 times the observation band. Due to this increase, we had to utilize both parallel and distributed processing techniques for processing GOCI-II L2 products in real time. First, parallel processing is applied to utilize optimal hardware resources (i.e. Multi-core CPU, GPU, and Xeon-Phi) for each L2 product application. Then, heterogeneous computing systems have been built so that these applications can be distributed and processed across multiple computing resources. In order to efficiently operate the resources of the heterogeneous computing systems, there is need for a scheduler that can load-balance all available resources. In this study, we implemented a scheduler to allocate tasks considering the load of all computing resources and the priority of each L2 application. By using optimal hardware resources for each algorithm and using more resources at the same time, the utilization rate of the entire system can be increased. the amount of data to be processed has increased gradually, such as 4 times the spatial resolution, 2 times the L2 product, and 1.5 times the observation band. Due to this increase, we had to utilize both parallel and distributed processing techniques for processing GOCI-II L2 products in real time. First, parallel processing is applied to utilize optimal hardware resources (i.e. Multi-core CPU, GPU, and Xeon-Phi) for each L2 product application. Then, heterogeneous computing systems have been built so that these applications can be distributed and processed across multiple computing resources. In order to efficiently operate the resources of the heterogeneous computing systems, there is need for a scheduler that can load-balance all available resources. In this study, we implemented a scheduler to allocate tasks considering the load of all computing resources and the priority of each L2 application. By using optimal hardware resources for each algorithm and using more resources at the same time, the utilization rate of the entire system can be increased.
URI
https://sciwatch.kiost.ac.kr/handle/2020.kiost/23389
Bibliographic Citation
ISRS 2018, pp.1, 2018
Publisher
KSRS
Type
Conference
Language
English
Publisher
KSRS
Related Researcher
Files in This Item:
There are no files associated with this item.

qrcode

Items in ScienceWatch@KIOST are protected by copyright, with all rights reserved, unless otherwise indicated.

Browse