Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

External RPCDaemon: to open/close minimum files #11518

Closed
AskAlexSharov opened this issue Aug 8, 2024 · 1 comment
Closed

External RPCDaemon: to open/close minimum files #11518

AskAlexSharov opened this issue Aug 8, 2024 · 1 comment
Labels
erigon3 imp2 Medium importance
Milestone

Comments

@AskAlexSharov
Copy link
Collaborator

Here cmd/rpcdaemon/cli/config.go:onNewSnapshot
Now we use there agg.Open - it's not enough. Fully re-open all files - is also too brutal.
Less brutal is agg.OpenList -> closeWhatNotInList but this does "immediate close" - but we need "lazy close" (by last-alive-reader of this file).
We already have logic of "lazy remove" - DomainRoTx.Close() -> filesItem.canDelete. Maybe we also need "lazy close" - maybe even use same atomic (like change atomic to int bitmap).

reply.HistoryFiles - has list of good files (Can use it - but also can ingnore it - maybe OpenFiles -> scanDirtyFiles logic is enough)

  • Files which are not useful anymore: must be deleted from _dirtyFiles, but not closed. Need mark them as "ready to close" - and last-alive-reader will close it in AggTx.Close()

UnitTest: after handling onNewSnapshot notification:

  • last reader must close file without remove
  • old reader must be able to read from it - until it call aggTx.Close

Additional feature:

  • if reply.HistoryFiles is empty - don't call agg.Open
  • if reply.BlocksFiles is empty - don't call allSnapshots.ReopenList
@AskAlexSharov
Copy link
Collaborator Author

done by #12332

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
erigon3 imp2 Medium importance
Projects
None yet
Development

No branches or pull requests

4 participants