Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Iterative and Concurrent Traversal #50

Open
4 tasks
Tracked by #91
tabcat opened this issue Feb 4, 2023 · 1 comment
Open
4 tasks
Tracked by #91

Iterative and Concurrent Traversal #50

tabcat opened this issue Feb 4, 2023 · 1 comment

Comments

@tabcat
Copy link
Member

tabcat commented Feb 4, 2023

The traverser needs to be turned into an async iterator.
The reason this hasn't been done yet is mainly time and difficulty.
The traverser needs to be able to traverse efficiently for ordered and unordered traversal
so there is no duplicate code.
Fetching and caching entries ahead of when they are needed would be nice.

IIRC the difficulty is doing this with the using the yield statement.
May need to make a custom AsyncGenerator

  • Traverser Function returns Async-Iterator
  • Replica.traverse returns an Async-Iterator
  • Store API consumes Async-Iterator
  • IPFS/Pubsub Heads Exchange Replicator consumes Async-Iterator
@tabcat tabcat mentioned this issue Jul 18, 2023
14 tasks
@tabcat
Copy link
Member Author

tabcat commented Aug 15, 2023

might be good to separate this part out into a utils package or its own as it's generally useful

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant