Title

Usfd: A Unified Storage Framework For Soar Hpc Scientific Workflows

Keywords

data-intensive computing; scientific workflows; storage systems for data-intensive computing

Abstract

Emerging scientific workflows in high performance computing (HPC) focus more on analysis rather than on simulation. Simulation output is so dense with information that copious amounts of analysis must be performed on a single output to understand the results of that simulation. We identify this repetitive analysis as a new application type, simulate once analyse repeatedly (SOAR) computing. Current scientific HPC, when extended to SOAR computing, results in excessive data migration between compute and storage resources. For a workflow bound by file I/O, a large data migration overhead is unacceptable. We propose a framework that uses a data-intensive storage cluster coupled with an interoperability layer, called unified storage framework designed (USFD). USFD is a better support SOAR HPC scientific workloads through enhanced file I/O support and co-located storage and analysis. In this work, we analyse the performance of USFD and other traditional HPC approaches for SOAR scientific workloads. Our results show that SOAR workflows using USFD complete 7.5 times faster over other approaches with quantum chromodynamic workflows and 4 times faster with FLASH workflows. © 2012 Copyright Taylor and Francis Group, LLC.

Publication Date

8-1-2012

Publication Title

International Journal of Parallel, Emergent and Distributed Systems

Volume

27

Issue

4

Number of Pages

347-367

Document Type

Article

Personal Identifier

scopus

DOI Link

https://doi.org/10.1080/17445760.2011.638294

Socpus ID

84864695482 (Scopus)

Source API URL

https://api.elsevier.com/content/abstract/scopus_id/84864695482

This document is currently not available here.

Share

COinS