Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Patch 5 feature rondb 753 ttl feature rebase 24.10 #605

Open
wants to merge 7 commits into
base: 24.10-main
Choose a base branch
from

Conversation

KernelMaker
Copy link
Collaborator

TTL Purging

1. add ttl_only_expired flag to only retrieve expired rows
2. ttl_purge_demo.cpp is an example of how ttl purging is going to work in the
   restapi server
3. bugfix, set op_type from ZINSERT_TTL to ZUPDATE to make sure the
   rondb internal triggers can be done correctly.

TODO:
1. integret purge program into restapi
2. handle ttl_only_expired on non-TTL table
3. double checke the ZINSERT_TTL bugfix, add test cases
1. use murmur3 hash function in GetShard()
2. define the behavior of TTL_EXPIRED_ONLY flag on non-TTL table: return
   not-found
3. add example of how to use TTL_EXPIRED_ONLY and TTL_IGNORE flags in
   ndbapi_recattr_vs_record example program
4. bugfix: options can be nullptr in NdbScanOperation::scanImpl()
1. handle NdbObjects properly by removeCachedXXX and InvalidateCacheXXX
2. adaptive scan batch size on each TTL table
3. adaptive sleeping time between each purging round
4. Handle the potential error 296 and 499 if TransactionInactiveTimeout
   is configured too small
std::mt19937 gen(rd());
std::uniform_int_distribution<int> dist(lower_bound, upper_bound);

int sleep_duration = dist(gen);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use NdbTick.h and NdbSleep.h, the RonDB portability library

// Set it to true to make purge worker load cache
cache_updated_ = true;
purge_worker_exit_ = false;
purge_worker_ = new std::thread(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use RonDB portability library to create thread

return;
}

bool TTLPurger::GetShard(int32_t* shard, int32_t* n_purge_nodes,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use RonDB data types

if (!schema_watcher_running_) {
assert(schema_watcher_ == nullptr);
assert(!purge_worker_running_);
schema_watcher_ = new std::thread(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use RonDB portability library to create thread

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants