-
Notifications
You must be signed in to change notification settings - Fork 348
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improved logic for checking overflow workers #89
base: master
Are you sure you want to change the base?
Conversation
When workers are expensive to start and transactions are quick killing all workers that are checked in when in overflow is very expensive. This change allows delaying the termination of overflow workers when there is peak load and alleviates worker churn.
When workers are expensive to start and transactions are quick killing all workers that are checked in when in overflow is very expensive. This change allows delaying the termination of overflow workers when there is peak load and alleviates worker churn.
Conflicts: README.md
Get changes
@fishcakez I could really use another set of eyes on this here. |
When reaping in batches there is no need to read/store the monotonic time in the workers queue, only the time of the poll when it would be reaped or the time of the last reap. This need not be a clock time but could be a counter that increments once per reap, in a similar way to a timer wheel works. Using a counter (instead of monotonic time) may introduce some lag in the TTL but I don't think that will be an issue here because strict timing is not required. This would remove the OTP-18 requirement and should be faster. With OTP-18 strict timing would be possible using absolute timers. This branch needs to access both ends of the worker queue so you may want to consider a different data structure. The |
I am not sure how I feel about adding features to |
As discussed in #83 (comment) I made improvements in zugolosian`s changes.
In zugolosian#1 all commits are discussed.
Summing up: