Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can support striped directory to increase metadata performance of very large directories? #25

Open
hmings888 opened this issue Nov 19, 2024 · 0 comments
Labels
enhancement New feature or request new Issues that haven't been triaged yet

Comments

@hmings888
Copy link

In current implementation of beegfs metadata server, all the files under one directory are stored on the same metadata server that their parent directory is .
Image a very large directory containing tens of thousands or even more files, which would be visited frequently, then the load of the metadata server storing this directory would be very very high, evenly response timeout. Could the files under one directory be hashed to multiple shards and then stored across multiple metadata servers? I think this would avoid the performance issue. As I known, Lustre supports this feature.

@hmings888 hmings888 added enhancement New feature or request new Issues that haven't been triaged yet labels Nov 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request new Issues that haven't been triaged yet
Projects
None yet
Development

No branches or pull requests

1 participant