Worldwide Fast File Replication on Grid Datafarm

Osamu Tatebe^1, Satoshi Sekiguchi^1, Youhei Morita^2, Satoshi Matsuoka^3, Noriyuki Soda^4

1. National Institute of Advanced Industrial Science and Technology (AIST)
2. High Energy Accelerator Research Organization (KEK)
3. Tokyo Institute of Technology
4. Software Research Associates, Inc.

Abstract

The Grid Datafarm architecture is designed for global petascale
data-intensive computing.  It provides a global parallel filesystem
with online petascale storage, scalable I/O bandwidth, and scalable
parallel processing, and it can exploit local I/O in a grid of
clusters with tens of thousands of nodes.  One of features is that it
manages file replicas in filesystem metadata for fault tolerance and
load balancing.

This paper discusses and evaluates several techniques to support
long-distance fast file replication.  The Grid Datafarm manages a
ranked group of files as a Gfarm file, each file, called a Gfarm file
fragment, being stored on a filesystem node, or replicated on several
filesystem nodes.  Each Gfarm file fragment is replicated
independently and in parallel using rate-controlled HighSpeed TCP with
network striping.  On a US-Japan testbed with 10,000 km distance, we
achieve 419 Mbps using 2 nodes on each side, and 741 Mbps using 4
nodes out of 893 Mbps with two transpacific networks.