A metropolitan area infrastructure for data intensive science

Abramson, David, Carroll, Jake, Jin, Chao and Mallon, Michael (2017). A metropolitan area infrastructure for data intensive science. In: Proceedings - 13th IEEE International Conference on eScience, eScience 2017. Annual IEEE International Conference on e-Science (e-Science), Auckland, New Zealand, (238-247). 24-27 October 2017. doi:10.1109/eScience.2017.37


Author Abramson, David
Carroll, Jake
Jin, Chao
Mallon, Michael
Title of paper A metropolitan area infrastructure for data intensive science
Conference name Annual IEEE International Conference on e-Science (e-Science)
Conference location Auckland, New Zealand
Conference dates 24-27 October 2017
Proceedings title Proceedings - 13th IEEE International Conference on eScience, eScience 2017   Check publisher's open access policy
Journal name 2017 IEEE 13Th International Conference On E-Science (E-Science)   Check publisher's open access policy
Series Proceedings - 13th IEEE International Conference on eScience, eScience 2017
Place of Publication Piscataway, NJ, United States
Publisher Institute of Electrical and Electronics Engineers
Publication Year 2017
Year available 2017
Sub-type Fully published paper
DOI 10.1109/eScience.2017.37
Open Access Status Not yet assessed
ISBN 9781538626863
ISSN 2325-372X
Start page 238
End page 247
Total pages 10
Language eng
Abstract/Summary The increasing amount of data being collected from simulations, instruments and sensors creates challenges for existing e-Science infrastructure. In particular, it requires new ways of storing, distributing and processing data in order to cope with both the volume and velocity of the data. The University of Queensland has recently designed and deployed MeDiCI, a data fabric that spans the metropolitan area and provides seamless access to data regardless of where it is created, manipulated and archived. MeDiCI is novel in that it exploits temporal and spatial locality to move data on demand in an automated manner. This means that data only needs to reside locally in high speed storage whilst being manipulated, and it can be archived transparently in high capacity, but slower, technologies at other times. MeDiCI is built on commercially available technologies. In this paper, we describe these innovations and present some early results.
Subjects 1101 Agricultural and Biological Sciences (miscellaneous)
1301 Biochemistry, Genetics and Molecular Biology (miscellaneous)
1705 Computer Networks and Communications
1706 Computer Science Applications
1903 Computers in Earth Sciences
3301 Social Sciences (miscellaneous)
Keyword Data Intensive Science
Distributed Storage Systems
High Performance Computing
Parallel File Systems
Q-Index Code E1
Q-Index Status Provisional Code
Institutional Status UQ

Document type: Conference Paper
Sub-type: Fully published paper
Collections: HERDC Pre-Audit
School of Information Technology and Electrical Engineering Publications
 
Versions
Version Filter Type
Citation counts: TR Web of Science Citation Count  Cited 0 times in Thomson Reuters Web of Science Article
Scopus Citation Count Cited 0 times in Scopus Article
Google Scholar Search Google Scholar
Created: Fri, 23 Mar 2018, 13:32:52 EST by Ms Diana Cassidy on behalf of Learning and Research Services (UQ Library)