Metadata management has always been the focus of distributed file systems research. Metadata replication exhibits a comparative advantage in maintaining locality and balancing system load. However, according to the requirements of POSIX semantics, existing systems use strict consistency to guarantee the consistency and atomicity of metadata operations. The strict consistency impedes I/O, resulting in a performance bottleneck.
To solve the problems, a research team led by Limin Xiao of China’s Beihang University recently published a study in Frontiers of Computer Science.
The team proposed a low-cost and efficient framework with adaptive synchronization for metadata replication. This synchronization method is verified and tested in the global virtual data space file system. Compared with the existing research results, the proposed method has better metadata operation performance and better scalability.
Within the research, the team analyzed the consistency and performance requirements of metadata operations and data operations for distributed file systems in a wide-area, high-performance computing environment, and then decoupled directories from file metadata. They propose different synchronization methods for directory trees and file metadata to ensure system performance without sacrificing consistency.
ICGDT, an incremental consistency guaranteed directory tree synchronization method, enables fine-grained tracking of access conflict status for files and directories. By utilizing a decentralized conflict status tree, it allows dynamic switching of the message synchronization channel for files or directories within the directory tree.
Causal consistency guaranteed replica index synchronization (CCGRI) is a method that ensures causal consistency for data access in wide-area DFSes. It converts data update synchronization requests into data index update synchronization requests to reduce the wide-area bandwidth occupied by data replication and delay the actual synchronization of replica data until the replica data is accessed or the wide-area link is idle by lazy replication. They work together to achieve efficient and consistent metadata replication.
The experiment was carried out on a real distributed file system, GVDS. The experimental results show that compared with the existing methods, the proposed method greatly reduces the synchronization delay and has good scalability.
More information:
Chenhao Zhang et al, ICCG: low-cost and efficient consistency with adaptive synchronization for metadata replication, Frontiers of Computer Science (2024). DOI: 10.1007/s11704-023-2772-y
Provided by
Frontiers Journals
Citation:
Low-cost framework enhances metadata replication with adaptive synchronization (2025, February 18)
retrieved 18 February 2025
from
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no
part may be reproduced without the written permission. The content is provided for information purposes only.
