Build custom runtime for configurable worker threads #91
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #91 +/- ##
==========================================
+ Coverage 50.02% 50.08% +0.05%
==========================================
Files 31 31
Lines 11805 11792 -13
==========================================
Hits 5906 5906
+ Misses 5899 5886 -13 ☔ View full report in Codecov by Sentry. |
| } | ||
| } | ||
| let cores = available_parallelism().unwrap().get(); | ||
| let worker_threads: &usize = args.get_one::<usize>("worker-threads").unwrap_or(&cores); |
There was a problem hiding this comment.
What if the number of threads provided is higher than the number of threads allowed?
There was a problem hiding this comment.
Whats meant by number of threads allowed? You can spawn more threads but if you do not have more cores than it will block the thread. We could limit it to the cores available but I do not think it is necessary.
There was a problem hiding this comment.
I mean like for example, the user has no idea how many cores the test device has and provides a woker-threads parameter higher than the available number of cores.
There was a problem hiding this comment.
As I understand it is nothing that prevents you from having more threads than cores, but you do not gain anything from it, but most likely you will get a small performance penalty if specifying more than number of cores. I think it makes sense to write some advice in documentation on when it makes sense to have something else than default.
Is a possible use-case that you want to run Databroker with high priority (for minimal latency), but at the same time you do not want it to steal more than a certain share of the capacity and by that reason you state that it only might use X cores.
There was a problem hiding this comment.
Would there be a value to print something like "Using X cores, Y cores available", either on debug or info level? Info-level if we consider it as relevant info to get when someone has performance problems and we do not want to get a huge debug log.
There was a problem hiding this comment.
Added info message how many threads are spawned and how many cores are available on the system.
There was a problem hiding this comment.
LGTM
Did some smoke test using kuksa-client. No problems detected
erik@debian6:~/kuksa-databroker$ cargo run --bin databroker -- --metadata data/vss-core/vss_release_4.0.json --insecure --worker-threads=2000
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.17s
Running `target/debug/databroker --metadata data/vss-core/vss_release_4.0.json --insecure --worker-threads=2000`
2024-11-05T13:39:01.519395Z INFO databroker: Init logging from RUST_LOG (environment variable not found)
2024-11-05T13:39:01.519456Z INFO databroker: Starting Kuksa Databroker 0.4.7-dev.0
2024-11-05T13:39:01.519470Z INFO databroker: Using 2000 threads with 3 cores available on the system
to make the worker threads configurable via env varibale and command line. Documentation currently missing!!!!