Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add docs on data retrievability and pruning #1123

Merged
merged 13 commits into from
Oct 11, 2023
Merged

Conversation

musalbas
Copy link
Member

@musalbas musalbas commented Oct 9, 2023

No description provided.

@github-actions
Copy link
Contributor

github-actions bot commented Oct 9, 2023

PR Preview Action v1.4.4
Preview removed because the pull request was closed.
2023-10-11 20:13 UTC

Copy link
Member

@liamsi liamsi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do think the params and suggested practices all make sense.


Rollups may need to access historic data in order to allow new rollup nodes to reconstruct the latest state by replaying historic blocks. Once data has been published on Celestia and guaranteed to have been made available, rollups and applications are responsible for storing their historical data.

While it is possible to continue do this by using the `GetAll` API method in celestia-node on historic blocks as long as archival nodes exist on the public Celestia network, rollup developers should not rely on this as the only method to access historical data, as archival nodes serving requests for historical data for free is not guaranteed. Below are some other suggested methods to access historical data.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One idea, i’ve been discussing with Mary is the ability to plug various data sources into the node. In the same way, we have multiple options to get data from p2p, we can also allow developers to plug in CDNs, cloud providers, and web3 storage providers. This is similar to marketed Espresso’s tiramisu with the difference that we have required foundation implemented. If we enable plugging various standardized Getters, then there will be less need for Rollup devs to migrate off GetAll and implement custom interfaces and data retrievals. They would be able to just use standardized data sources in our ecosystem.

Copy link
Member

@Bidon15 Bidon15 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@musalbas musalbas marked this pull request as ready for review October 11, 2023 14:45
@jcstein jcstein self-requested a review October 11, 2023 15:23
docs/developers/retrievability.md Outdated Show resolved Hide resolved
docs/developers/retrievability.md Outdated Show resolved Hide resolved
docs/developers/retrievability.md Outdated Show resolved Hide resolved
docs/developers/retrievability.md Outdated Show resolved Hide resolved
docs/developers/retrievability.md Outdated Show resolved Hide resolved
Copy link
Member

@jcstein jcstein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - left some suggestions

docs/developers/retrievability.md Outdated Show resolved Hide resolved
docs/developers/retrievability.md Outdated Show resolved Hide resolved
docs/developers/retrievability.md Outdated Show resolved Hide resolved
docs/developers/retrievability.md Outdated Show resolved Hide resolved
@jcstein jcstein merged commit 4b8ff62 into main Oct 11, 2023
4 checks passed
@jcstein jcstein deleted the musalbas/retrievability branch October 11, 2023 20:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants