Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

balance-learner-schedule: balance learners among stores(tiflash). #8278

Open
AndreMouche opened this issue Jun 12, 2024 · 0 comments
Open

balance-learner-schedule: balance learners among stores(tiflash). #8278

AndreMouche opened this issue Jun 12, 2024 · 0 comments
Labels
type/feature-request Categorizes issue or PR as related to a new feature.

Comments

@AndreMouche
Copy link
Member

Feature Request

Describe your feature request related problem

Describe the feature you'd like

Currently,

  • for the distribution issue of leaders among stores, we have the balance-leader-scheduler to address the uniformity issue.
  • for the distribution issue of peers among stores, we have the balance-region-schedule to address the uniformity issue.

while balance-region-schedule only consider the balance between all stores, without consider the roles(learner and follower) and the type of stores(tikv or tiflash ).
However, for TiFlash users, if the distribution of regions(learners) among TiFlash instances becomes unbalanced, it may lead to computational hotspots that slow down performance.
From the following logic, we can see balance-region-scheduler choose the source store order by region-score , and if the number of learner region on tiflash is small, the region-score of tiflash node should be always the smallest, which makes tiflash nodes could never get the chance to run balance-region, that leads the imbalance of peers on the tiflash nodes.

sort.Slice(sourceStores, func(i, j int) bool {
iOp := solver.GetOpInfluence(sourceStores[i].GetID())
jOp := solver.GetOpInfluence(sourceStores[j].GetID())
return sourceStores[i].RegionScore(conf.GetRegionScoreFormulaVersion(), conf.GetHighSpaceRatio(), conf.GetLowSpaceRatio(), iOp) >
sourceStores[j].RegionScore(conf.GetRegionScoreFormulaVersion(), conf.GetHighSpaceRatio(), conf.GetLowSpaceRatio(), jOp)
})

In summary,I think we need a scheduler similar to balance-learner-scheduler to balance the distribution of learner nodes among the stores.

@AndreMouche AndreMouche added the type/feature-request Categorizes issue or PR as related to a new feature. label Jun 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/feature-request Categorizes issue or PR as related to a new feature.
Projects
None yet
Development

No branches or pull requests

1 participant