Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Collect histogram of process loads (modeled and raw) in LB data reduction #1382

Open
PhilMiller opened this issue Apr 14, 2021 · 3 comments · May be fixed by #1873
Open

Collect histogram of process loads (modeled and raw) in LB data reduction #1382

PhilMiller opened this issue Apr 14, 2021 · 3 comments · May be fixed by #1873
Assignees
@stmcgovern @ppebay
Labels
pri: low

Comments

@PhilMiller
Copy link
Member

PhilMiller commented Apr 14, 2021 •
edited
Loading

There's already an implementation of an approximate histogram for the diagnostics. Using that in the process stats reduction would allow quantile estimation, with potential uses in LB strategies.
@PhilMiller
Copy link
Member Author

One possible use would be to iteratively focus refinement-based load balancing on the most-overloaded processes, as judged by their place in the ranking, rather than their absolute degree of overload which is the only thing they can compute with available stats right now.

@PhilMiller
Copy link
Member Author

https://github.com/DARMA-tasking/vt/blob/develop/src/vt/utils/adt/histogram_approx.h

http://www.jmlr.org/papers/volume11/ben-haim10a/ben-haim10a.pdf

@lifflander
Copy link
Collaborator

@ppebay Are you still planning on doing this?

@PhilMiller PhilMiller mentioned this issue Mar 2, 2022
Open
@lifflander lifflander mentioned this issue May 10, 2022
Open
@lifflander lifflander changed the title Collect histogram of process loads (total and subphase) in LB stats reduction Collect histogram of process loads in LB stats reduction May 12, 2022
@lifflander lifflander changed the title Collect histogram of process loads in LB stats reduction Collect histogram of process loads (modeled and raw) in LB data reduction May 12, 2022
stmcgovern added a commit that referenced this issue Jul 19, 2022
@stmcgovern
#1382: add test skeleton for lb histogram
0e5f745
stmcgovern added a commit that referenced this issue Jul 19, 2022
@stmcgovern
#1382: add histogram and serialize member function to LoadData
810e9db
stmcgovern added a commit that referenced this issue Jul 19, 2022
@stmcgovern
#1382: add getter and update operator+
fa0926d
stmcgovern added a commit that referenced this issue Jul 19, 2022
@stmcgovern
#1382: add histogram as StatisticQuantity and to StatsHandler
7209641
stmcgovern added a commit that referenced this issue Jul 19, 2022
@stmcgovern
#1382: add union in StatisticQuantityMap
e4f1dec
@stmcgovern stmcgovern linked a pull request Jul 19, 2022 that will close this issue
Draft
@stmcgovern stmcgovern linked a pull request Jul 19, 2022 that will close this issue
#1382: collect histogram of process loads #1873
Draft
stmcgovern added a commit that referenced this issue Jul 19, 2022
@stmcgovern
#1382: fix license and remove trailing whitespace
d28fbed
stmcgovern added a commit that referenced this issue Jul 19, 2022
@stmcgovern
#1382: add lbh to JSON_SERIALIZE_ENUM
fd748ca
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@stmcgovern stmcgovern

@ppebay ppebay

Labels
pri: low
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

#1382: collect histogram of process loads DARMA-tasking/vt
4 participants
@PhilMiller @stmcgovern @lifflander @ppebay