Vh-Dsi: Speeding Up Data Visualization Via A Heterogeneous Distributed Storage Infrastructure

Keywords

Data Intensive Scientific Computing; Distributed Computing; Heterogeneous Cluster; I/O; Parallel Computing; Scientific Visualization

Abstract

Visualizing and analyzing large-scale datasets are both critical and challenging, as they require substantial resources for data processing and storage. While the speed of supercomputers continues to set higher standard, the I/O systems have not kept in pace, resulting in a significant performance bottleneck. To alleviate the I/O bottleneck for scientific visualization applications, we propose a Visualization via a Heterogeneous Distributed Storage Infrastructure (VH-DSI) solution to improve I/O speed and accelerate overall visualization performance. VH-DSI replaces the traditional parallel file system with a distributed file system to support visualization applications. A new scheduling algorithm HeterSche is proposed in VH-DSI to assign computing tasks to data nodes with the consideration of cluster heterogeneity and data locality. VH-DSI also includes a design to support POSIX-IO for distributed file system. The performance evaluation has shown that the proposed VH-DSI solution can achieve significant performance improvement for visualization applications. Compared to the traditional visualization, the VH-DSI solution reduces the response time by at least 5 times. The HeterSche scheduling algorithm is capable to speed up visualization compared to other scheduling algorithms especially for large scale datasets.

Publication Date

1-15-2016

Publication Title

Proceedings of the International Conference on Parallel and Distributed Systems - ICPADS

Volume

2016-January

Number of Pages

658-665

Document Type

Article; Proceedings Paper

Personal Identifier

scopus

DOI Link

https://doi.org/10.1109/ICPADS.2015.88

Socpus ID

84964666740 (Scopus)

Source API URL

https://api.elsevier.com/content/abstract/scopus_id/84964666740

This document is currently not available here.

Share

COinS