Title

Visdsi: Locality Aware I/O Solution For Large Scale Data Visualization

Keywords

Data Intensive; Distributed Computing; I/O Bandwidth; Parallel Computing; Scientific Visualization

Abstract

The computational science community is approaching petascale level simulations that will produce massive amount of datasets. While the computational power of supercomputers keep increasing, the I/O systems have not kept pace, resulting in a significant performance bottle neck. We propose a solution, VisDSI, to address the problem by 1) using traditional high performance clusters with disks directly attached within each node, 2) deploying a data-intensive distributed file system on the cluster, and 3) developing a POSIX-compatible I/O layer to enable the traditional visualization applications to smoothly port to this new platform as well as to provide a new I/Osemantic of retrieving the data location with respect to the POSIX standards. In particular, VisDSI guarantees the colocated compute and data storage by introducing a scheduling of work assignments to nodes with local copies of needed data. Compared with the original visualization application, our solution runs at least 10 times faster. © 2013 IEEE.

Publication Date

1-1-2013

Publication Title

Proceedings - 2013 IEEE/ACM 6th International Conference on Utility and Cloud Computing, UCC 2013

Number of Pages

278-281

Document Type

Article; Proceedings Paper

Personal Identifier

scopus

DOI Link

https://doi.org/10.1109/UCC.2013.60

Socpus ID

84901660289 (Scopus)

Source API URL

https://api.elsevier.com/content/abstract/scopus_id/84901660289

This document is currently not available here.

Share

COinS