Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

#2281: Add new allreduce algorithms for Group, Objgroup and Collection #2337

Draft
wants to merge 69 commits into
base: develop
Choose a base branch
from
Draft

#2281: Add new allreduce algorithms for Group, Objgroup and Collection #2337

wants to merge 69 commits into from

Conversation

JacobDomagala
Copy link
Contributor

@JacobDomagala JacobDomagala commented Aug 13, 2024 •
edited
Loading

Fixes #2281 Fixes #2240

@JacobDomagala JacobDomagala self-assigned this Aug 13, 2024
@JacobDomagala JacobDomagala linked an issue Aug 13, 2024 that may be closed by this pull request
Use a proper all-reduce algorithm for GroupManager #2281
Open
@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 13, 2024 •
edited
Loading

Pipelines results

PR tests (gcc-12, ubuntu, mpich, verbose, kokkos)

Build for eee5e69 (2024-11-05 15:29:52 UTC)

Compilation - successful

Testing - passed

Build log

@JacobDomagala JacobDomagala force-pushed the 2281-use-a-proper-all-reduce-algorithm-for-groupmanager branch 4 times, most recently from ee1a15e to 06165a0 Compare September 3, 2024 17:12
@JacobDomagala JacobDomagala force-pushed the 2281-use-a-proper-all-reduce-algorithm-for-groupmanager branch 3 times, most recently from 11426c1 to 1568c4a Compare September 16, 2024 23:39
@JacobDomagala JacobDomagala force-pushed the 2281-use-a-proper-all-reduce-algorithm-for-groupmanager branch 3 times, most recently from 897f570 to 455829a Compare September 21, 2024 20:46
@JacobDomagala JacobDomagala force-pushed the 2281-use-a-proper-all-reduce-algorithm-for-groupmanager branch from 08113bb to 5523e17 Compare September 26, 2024 17:49
@JacobDomagala JacobDomagala changed the title #2281: Use a proper all reduce algorithm for GroupManager #2281: Add new allreduce algorithms for Group, Objgroup and Collection Sep 27, 2024
@JacobDomagala JacobDomagala mentioned this pull request Sep 27, 2024
Closed
@JacobDomagala
#2240: Initial work for new allreduce
e7e1a2e
@JacobDomagala
#2240: Semi working Rabenseifner
f90e5c7
@JacobDomagala
#2240: Working Rabenseifner (non-commutative ops)
0715c52
@JacobDomagala
#2240: Fix non power of 2 for new allreduce
0070e71
@JacobDomagala
#2240: Initial work for adding recursive doubling allreduce algorithm
a2ad6e6
@JacobDomagala
#2240: Make sure the order of reduce operations is correct
19df2f7
@JacobDomagala
#2240: Working Recursive doubling
831168f
@JacobDomagala
#2240: Code cleanup and make Rabenseifner work with any Op type
594fce5
@JacobDomagala
#2240: Improve accuracy of timing allreduce algorithms in allreduce.cc
b5095b6
@JacobDomagala
#2240: Add unit tests for new allreduce and cleanup code
42cf608
@JacobDomagala
#2240: DataHandler for Rabenseifner allreduce that provides common AP…
0ee7848 
…I for various data types
@JacobDomagala
#2240: Fix warnings
d70050e
@JacobDomagala
#2240: Update ObjGroup test to use custom DataHandler for Rabenseifne…
1f86e55 
…r allreduce test
@JacobDomagala
#2240: Add unit test for Rabenseifner with Kokkos::View as DataType a…
699a6d8 
…nd fix compile issues realted to using Kokkos::View for allreduce
@JacobDomagala
#2281: Working perf test with collection
503d64b
@JacobDomagala
#2281: Setup perf test for groups and remove the parent proxy, as we …
5bbbb59 
…now store the final handler with callback
@JacobDomagala
#2281: Remove RabenseifnerGroup as we've now integrated groups and co…
23c0a6f 
…llections into the original struct
@JacobDomagala
#2281: Add unit tests for new callback (BcastCollective)
9f2cbaf
@JacobDomagala
#2281: Refactor Rabensifer a bit so it doesn't require that many temp…
57aa52b 
…late arguments, also let the User specify which algorithm to use
@JacobDomagala
#2281: Implement state holder and use it for Rabenseifner
028a354
@JacobDomagala
#2281: Implement clear function for StateHolder that frees dynamic me…
1aa6f98 
…mory allocated for allreduce state
@JacobDomagala
#2281: Working Rabenseifner with updated StateHolder
fde360f
@JacobDomagala
#2281: Typeless Rabensifner and small fixes to StateHolder
679344d
@JacobDomagala
#2281: Typeless RecursiveDoubling and general code refactor/cleanup
2e7b6e6
@JacobDomagala
#2281: RecursiveDoubling allreduce - add Collection support
0c7f5dd
@JacobDomagala
#2281: Add unit tests for new Collection allreduce
c86244b
@JacobDomagala
#2281: Small fixes and code cleanup
76c377c
@JacobDomagala
#2281: Update documentation for new allreduce algorithms
1bafd3b
@JacobDomagala
#2281: Initial work for AllreduceHolder
30b1882
@JacobDomagala
#2281: Fix runtime Kokkos when we run with both device/host memory sp…
0eb340b 
…aces
@JacobDomagala
#2281: Working AllreduceHolder with all (Collection/Group/ObjGroup) c…
3fb1ddd 
…omponents
@JacobDomagala
#2281: Fixed runtime issues with StateHolder generating extra ID
ae65729
@JacobDomagala
#2281: Update license and add reduce_op.h file for Kokkos reduce oper…
d118fa9 
…ators
@JacobDomagala
#2281: Always create allreducers for new collective group
c61db0c
@JacobDomagala
#2281: New allreducers no longer require Objgroup for internal commun…
7caed11 
…ication. This helps for cases when not all Nodes are part of allreduce
@JacobDomagala
#2281: Code refactor and minor bug fixes
e24cacc
@JacobDomagala
#2281: Sort out the PendingSends in new allreduce methods
46c73af
@JacobDomagala
#2281: Don't use explicit Kokkos::HostSpace for Kokkos::View in Raben…
33c5e98 
…seifnerMsg and ShouldUseView
@JacobDomagala
#2281: Cleanup StateHolder (move implementation to .cc impl.h files)
ed1f880
@JacobDomagala JacobDomagala force-pushed the 2281-use-a-proper-all-reduce-algorithm-for-groupmanager branch 2 times, most recently from 332ad94 to 164328c Compare October 11, 2024 23:37
@JacobDomagala JacobDomagala force-pushed the 2281-use-a-proper-all-reduce-algorithm-for-groupmanager branch from 164328c to 5c33e20 Compare October 14, 2024 14:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@JacobDomagala JacobDomagala

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Use a proper all-reduce algorithm for GroupManager Use a proper all-reduce algorithm for ObjGroup
2 participants
@JacobDomagala @cwschilly