Title

Slam: Scalable Locality-Aware Middleware For I/O In Scientific Analysis And Visualization

Keywords

HDFS; MPI/POSIX I/O; Parallel blast; Paraview

Abstract

Whereas traditional scientific applications are computationally intensive, recent applications require more data-intensive analysis and visualization. As the computational power and size of compute clusters continue to increase, the I/O read rates and associated network cost for these data-intensive applications create a serious performance bottleneck when faced with the massive data sets of today's "big data" era. In this paper, we present "Scalable Locality-Aware Middleware" (SLAM) for scientific data analysis applications. SLAM leverages a distributed file system (DFS) to provide scalable data access for scientific applications. To reduce data movement and enforce data-process locality, a datacentric scheduler (DC-scheduler) is proposed to enable scientific applications to read data locally from a DFS. We prototype our proposed SLAM system along with the Hadoop distributed file system (HDFS) on two well-known scientific applications. We find in our experiments that SLAM can greatly reduce I/O cost and double the overall performance, as compared to existing approaches. Copyright © 2014 ACM.

Publication Date

1-1-2014

Publication Title

HPDC 2014 - Proceedings of the 23rd International Symposium on High-Performance Parallel and Distributed Computing

Number of Pages

257-260

Document Type

Article; Proceedings Paper

Personal Identifier

scopus

DOI Link

https://doi.org/10.1145/2600212.2600709

Socpus ID

84904428984 (Scopus)

Source API URL

https://api.elsevier.com/content/abstract/scopus_id/84904428984

This document is currently not available here.

Share

COinS