Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make sstableloader incremental #64

Open
haaawk opened this issue Sep 3, 2018 · 3 comments
Open

Make sstableloader incremental #64

haaawk opened this issue Sep 3, 2018 · 3 comments
Labels
enhancement New feature or request

Comments

@haaawk
Copy link
Contributor

haaawk commented Sep 3, 2018

Currently when the process of loading sstables is interrupted, we have to start over. It would be great if the process could pick up where it finished and continue without doing the same work again.

@haaawk
Copy link
Contributor Author

haaawk commented Sep 3, 2018

@slivne @elcallio @avikivity

@elcallio
Copy link
Contributor

elcallio commented Sep 3, 2018

Note that this is not so easy:
We both have to assume that the source file tree is readonly (and should be always treated as such), and we also have to take into account the various options that control whether a file can really be considered "processed", such as initial and subsequent connected nodes and other parameters.

Perhaps adding a "progress" output file, and the option to restart from it would be better.

@haaawk
Copy link
Contributor Author

haaawk commented Sep 3, 2018

Yes. I didn't mean that it has to always be incremental. I should have stated it better. Progress file and option to use it is exactly what we should do IMO.

@nyh nyh added the enhancement New feature or request label Mar 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants