Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New rebalance #909

Open
wants to merge 7 commits into
base: devel
Choose a base branch
from
Open

New rebalance #909

wants to merge 7 commits into from

Conversation

leonidc
Copy link
Collaborator

@leonidc leonidc commented Oct 20, 2024

Auto rebalance is done from the separate thread

@leonidc leonidc force-pushed the new-rebalance branch 2 times, most recently from ccc24da to 45aa326 Compare October 20, 2024 09:39
@leonidc leonidc force-pushed the new-rebalance branch 8 times, most recently from d13bc55 to d3c2bc5 Compare November 10, 2024 15:33
@leonidc leonidc force-pushed the new-rebalance branch 13 times, most recently from 543a9ba to 91b5b35 Compare November 14, 2024 15:38
@@ -48,6 +48,8 @@
CNTLID_RANGE_SIZE = 2040
DEFAULT_MODEL_NUMBER = "Ceph bdev Controller"

MAX_POSSIBLE_ANA_GRPS = 100
Copy link
Collaborator

@baum baum Nov 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@leonidc Please remove MAX_POSSIBLE_ANA_GRPS arbitrary constant. Let me know if you need any assistance with this.

tests/ha/auto_load_balance.sh Outdated Show resolved Hide resolved
@baum
Copy link
Collaborator

baum commented Nov 17, 2024

@leonidc great work! could you please extract the new rebalance task into a separate class and file, control/rebalance.py? This appears to be a significant piece of logic that is not directly related to serving gRPC. Rebalance-specific state could be managed within the new class, which would also help clarify which state is shared between threads. Let me know if you need any assistance with this.

To ensure thread safety:

  • Identify shared data structures and protect them with a mutex.
  • Verify that shared objects, such as cephutil, which are currently used across multiple threads, are thread-safe.

@leonidc leonidc force-pushed the new-rebalance branch 4 times, most recently from 27d0041 to e5577cb Compare November 19, 2024 08:09
Leonid Chernin added 6 commits November 19, 2024 10:48
merged with last  devel branch
works for 3 GWs and 3 subsystem. Just 1 GW in time performs rebalance
added test for github, merged again,
accelerate rebalance when several ANA are in Optimized state on some GW.
added protection for shared data.

Signed-off-by: Leonid Chernin <[email protected]>
…ema based on rpc-lock

Signed-off-by: Leonid Chernin <[email protected]>
control/cephutils.py Outdated Show resolved Hide resolved
control/grpc.py Show resolved Hide resolved
control/grpc.py Outdated Show resolved Hide resolved
control/grpc.py Outdated Show resolved Hide resolved
control/grpc.py Outdated Show resolved Hide resolved
control/rebalance.py Show resolved Hide resolved
control/rebalance.py Outdated Show resolved Hide resolved
control/rebalance.py Outdated Show resolved Hide resolved
control/rebalance.py Show resolved Hide resolved
tests/ha/auto_load_balance.sh Show resolved Hide resolved
Signed-off-by: Leonid Chernin <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Changes requested
Development

Successfully merging this pull request may close these issues.

3 participants