Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/trie values column #2288

Merged
merged 23 commits into from
Dec 2, 2024
Merged

Feature/trie values column #2288

merged 23 commits into from
Dec 2, 2024

Conversation

Harrm
Copy link
Contributor

@Harrm Harrm commented Nov 21, 2024

Referenced issues

Description of the Change

  • Introduce spaced batch to atomically write to several db spaces
  • Put trie values in a separate column from nodes
  • Switch exceptions in state sync flow to outcome::result
  • Remove batchable from generic storage

Possible Drawbacks

  • A migration is required

Checklist Before Opening a PR

Before you open a Pull Request (PR), please make sure you've completed the following steps and confirm by answering 'Yes' to each item:

  1. Code is formatted: Have you run your code through clang-format to ensure it adheres to the project's coding standards? Yes
  2. Code is documented: Have you added comments and documentation to your code according to the guidelines in the project's contributing guidelines? Yes
  3. Self-review: Have you reviewed your own code to ensure it is free of typos, syntax errors, logical errors, and unresolved TODOs or FIXME without linking to an issue? Yes
  4. Zombienet Tests: Have you ensured that the zombienet tests are passing? Zombienet is a network simulation and testing tool used in this project. It's important to ensure that these tests pass to maintain the stability and reliability of the project. No

@Harrm Harrm marked this pull request as ready for review November 22, 2024 12:43
@@ -144,6 +147,13 @@ namespace kagome::application {
"platform. Proceed at your own risk.");
#endif

if (app_config_->enableDbMigration()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have to consider separate flag for each type of migration in the future

core/network/impl/state_sync_request_flow.cpp Show resolved Hide resolved
core/network/impl/synchronizer_impl.cpp Outdated Show resolved Hide resolved
auto logger = log::createLogger("Migration", "storage");
SL_INFO(logger,
"Begin trie storate migration to separate nodes and values");
if (storage.getSpace(Space::kTrieValue)->cursor()->isValid()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we have to check getSpace result for nullptr and print critical error if it is, same about cursor()

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's the responsibility of SpacedStorage to ensure it never returns a nullptr, otherwise it would return an optional. Same for cursor.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do you know that? getSpace's descripion says "return a pointer buffer storage for a space" and that's it. So it's class user's responsibility to check a method result. Same for cursor: "return kv iterator". Check how they're handled in store_impl.cpp

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, we never use null smart pointers, which means if getSpace returns a shared pointer to a space, it's always a pointer to a space, as specified in its description. The way it's handled in store_impl.cpp should be fixed so it doesn't have an unnecessary check.

core/storage/migrations/migrations.cpp Outdated Show resolved Hide resolved
core/storage/rocksdb/rocksdb.cpp Outdated Show resolved Hide resolved
std::optional<PolkadotTrieCursor::CertainlyValueAndHash>
PolkadotTrieCursorImpl::value_and_hash() const {
if (const auto *search_state = std::get_if<SearchState>(&state_);
search_state != nullptr) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need to check for nullptr, search_state will be checked automatically

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer an explicit check.

@Harrm Harrm requested a review from ErakhtinB November 27, 2024 16:24
@Harrm Harrm merged commit 2a9083b into master Dec 2, 2024
13 checks passed
@Harrm Harrm deleted the feature/trie-values-column branch December 2, 2024 13:20
kamilsa added a commit that referenced this pull request Dec 21, 2024
@Harrm Harrm mentioned this pull request Dec 23, 2024
Harrm added a commit that referenced this pull request Dec 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants