A paper by Koyama (student), Hiraga (Researcher) and Tatebe (Professor) of our laboratory has been accepted at the international conference IEEE Cluster 2024. IEEE Cluster 2024 will be held in Kobe, Japan, from September 24 to 27.

Sohei Koyama, Kohei Hiraga, Osamu Tatebe, “FINCHFS: Design of Ad-Hoc File System for I/O Heavy HPC Workloads”, Proceedings of IEEE International Conference on Cluster Computing (CLUSTER), 2024 (to appear)

This work focuses on recent HPC workloads that are difficult to support with parallel file systems, and designs the ad-hoc file system FINCHFS as a solution. Once an ad-hoc file system is redefined as playing a role complementary to a parallel file system, its requirements become metadata access performance and small-size I/O performance. FINCHFS meets these requirements at a high level by combining a shared-nothing architecture with number-aware hashing, a newly proposed method. In a performance evaluation using 64 nodes of the Pegasus supercomputer, it achieved 32.5 MIOPS on MDTest Hard write and 796.0 GiB/s on IOR Hard write, which are 42.8x and 174.2x the performance of CHFS, a state-of-the-art ad-hoc file system.

The results of this work are significant in that they show that, in designing an ad-hoc file system with distributed metadata servers, introducing a shared-nothing architecture — which had been assumed not to scale because of skew in requests — can substantially raise the performance of small RPCs.

Keywords: FINCHFS, Ad-Hoc File System, Persistent Memory, Parallel I/O, HPC