A paper by Hiraga (Researcher) and Tatebe (Professor) of our laboratory has been accepted at the international conference Euro-Par 2024. Euro-Par 2024 will be held in Madrid, Spain, from August 26 to 30.

Kohei Hiraga and Osamu Tatebe, “PEANUTS: A Persistent Memory-Based Network Unilateral Transfer System for Enhanced MPI-IO Data Transfer,” in Euro-Par 2024: Parallel Processing - 30th International Conference on Parallel and Distributed Computing, Madrid, Spain, August 26-30, 2024, Proceedings, in Lecture Notes in Computer Science, Springer, 2024, to appear.

This work proposes a new storage system, “PEANUTS (PErsistent memory And Network Unilateral Transfer System)”, which aims to solve the performance problems of parallel I/O to a single shared file in high-performance computing. Although parallel I/O to a single shared file is widely used in scientific computing, existing storage systems have suffered from scalability and performance issues. By combining persistent memory, a new class of storage device, with one-sided RDMA communication over InfiniBand, PEANUTS draws out hardware performance close to its limit and achieves a substantial improvement of more than 2.2x over existing state-of-the-art work.

The results of this work are significant in that they suggest that the use of large-scale single shared files, which had been infeasible until now, may become a standard I/O framework in data science, and in that they present a practical solution that overcomes conventional performance limitations.

Keywords: PEANUTS, Storage System, Persistent Memory, RDMA, One-Sided Communication, MPI-IO, Parallel I/O, HPC