Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support more than 23 million spots in Mastodon #353

Draft
wants to merge 1 commit into
base: dev
Choose a base branch
from

Conversation

maarzt
Copy link
Contributor

@maarzt maarzt commented Nov 14, 2024

Using the MultiArrayMemPool rather than SingleArrayMemPool eliminates the upper bound for the number of Spots in Mastodon. Without this change Mastodon is limited to approx. 23.3 million spots. Reason for that is that the SingleArrayMemPool limits the available space for spots to 2 GB. MultiArrayMemPool removes this limitation.

…odon

Using the MultiArrayMemPool rather than SingleArrayMemPool eliminates the
upper bound for the number of Spots in Mastodon. Without this change
Mastodon is limited to approx. 23.3 million spots. Reason for that is that
the SingleArrayMemPool limits the available space for spots to 2 GB.
MultiArrayMemPool removes this limitation.

The are other similar pools in the source code, that probably would need to
be changed to avoid similar problems these are at least:
* LinkPool
* DataGraph
* TrackSchemeGraph
@maarzt
Copy link
Contributor Author

maarzt commented Nov 14, 2024

The are other similar pools in the source code. They would probably need to be changed as well to avoid similar problems. These are at least: LinkPool, DataGraph, TrackSchemeGraph

@maarzt maarzt assigned maarzt and stefanhahmann and unassigned maarzt Nov 14, 2024
@tinevez
Copy link
Contributor

tinevez commented Nov 15, 2024

Can we slow down on this?
I am not sure we want to do this change yet before having a use case. And particularly before the preprint.

@stefanhahmann
Copy link
Collaborator

Can we slow down on this? I am not sure we want to do this change yet before having a use case. And particularly before the preprint.

I would not mind slowing down on this.

However, this PR was in fact motivated while writing the manuscript. Johannes wants to make a point about the max. data Mastodon can handle. The number he can write depends on this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants