276 Automatically synthesize and deduce message types for sends to arbitrary handlers #2064

lifflander · 2023-01-13T04:47:10Z

Fixes #276

cmake/load_bundled_libraries.cmake

examples/hello_world/hello_world.cc

codecov · 2023-01-24T19:07:38Z

Codecov Report

Merging #2064 (f40ee4d) into develop (ad0273d) will decrease coverage by 0.06%.
The diff coverage is 79.67%.

❗ Current head f40ee4d differs from pull request most recent head 6b69c91. Consider uploading reports for the commit 6b69c91 to get more accurate results

@@             Coverage Diff             @@
##           develop    #2064      +/-   ##
===========================================
- Coverage    84.92%   84.86%   -0.06%     
===========================================
  Files          721      721              
  Lines        25737    25667      -70     
===========================================
- Hits         21857    21783      -74     
- Misses        3880     3884       +4

Impacted Files	Coverage Δ
examples/rdma/rdma_simple_put.cc	`14.03% <0.00%> (ø)`
examples/rdma/rdma_simple_put_direct.cc	`20.00% <0.00%> (ø)`
src/vt/collective/scatter/scatter.impl.h	`94.44% <0.00%> (ø)`
src/vt/event/event.cc	`57.36% <0.00%> (ø)`
src/vt/group/group_manager.h	`100.00% <ø> (ø)`
src/vt/objgroup/manager.h	`100.00% <ø> (ø)`
src/vt/parameterization/parameterization.h	`100.00% <ø> (ø)`
src/vt/rdma/collection/rdma_collection.cc	`0.00% <0.00%> (ø)`
src/vt/registry/auto/auto_registry_common.h	`93.93% <ø> (ø)`
src/vt/registry/auto/auto_registry_general.h	`100.00% <ø> (ø)`
... and 73 more

... and 14 files with indirect coverage changes

github-actions · 2023-01-24T21:27:51Z

Pipelines results

PR tests (clang-10, ubuntu, mpich)

Build for 6b69c91 (2023-03-14 17:08:17 UTC)

Compilation - successful

Testing - passed

Build log

PR tests (nvidia cuda 11.0, ubuntu, mpich)

Build for 6b69c91 (2023-03-14 17:08:17 UTC)

/vt/src/vt/runnable/runnable.h(248): warning: missing return statement at end of non-void function "vt::runnable::RunnableNew::runLambda(Callable &&, Args &&...) [with Callable=__nv_bool (NodeObj::*)(), Args=<NodeObj *>]"
          detected during:
            instantiation of "decltype(auto) vt::runnable::RunnableNew::runLambda(Callable &&, Args &&...) [with Callable=__nv_bool (NodeObj::*)(), Args=<NodeObj *>]" 
/vt/src/vt/runnable/make_runnable.h(282): here
            instantiation of "auto vt::runnable::RunnableMaker<MsgT>::runLambda(Callable &&, Args &&...) [with MsgT=vt::BaseMsgType, Callable=__nv_bool (NodeObj::*)(), Args=<NodeObj *>]" 
/vt/src/vt/objgroup/manager.impl.h(228): here
            instantiation of "decltype(auto) vt::objgroup::ObjGroupManager::invoke<ObjT,f,Args...>(vt::objgroup::ObjGroupManager::ProxyElmType<ObjT>, Args &&...) [with ObjT=NodeObj, f=&NodeObj::isWorkFinished, Args=<>]" 
/vt/src/vt/objgroup/proxy/proxy_objgroup_elm.impl.h(114): here
            instantiation of "decltype(auto) vt::objgroup::proxy::ProxyElm<ObjT>::invoke<f,Args...>(Args &&...) const [with ObjT=NodeObj, f=&NodeObj::isWorkFinished, Args=<>]" 
/vt/examples/collection/jacobi1d_vt.cc(325): here

/vt/src/vt/runnable/runnable.h(248): warning: missing return statement at end of non-void function "vt::runnable::RunnableNew::runLambda(Callable &&, Args &&...) [with Callable=bool (NodeObj::*)(), Args=<NodeObj *>]"
          detected during:
            instantiation of "decltype(auto) vt::runnable::RunnableNew::runLambda(Callable &&, Args &&...) [with Callable=bool (NodeObj::*)(), Args=<NodeObj *>]" 
/vt/src/vt/runnable/make_runnable.h(282): here
            instantiation of "auto vt::runnable::RunnableMaker<MsgT>::runLambda(Callable &&, Args &&...) [with MsgT=vt::BaseMsgType, Callable=bool (NodeObj::*)(), Args=<NodeObj *>]" 
/vt/src/vt/objgroup/manager.impl.h(228): here
            instantiation of "decltype(auto) vt::objgroup::ObjGroupManager::invoke<ObjT,f,Args...>(vt::objgroup::ObjGroupManager::ProxyElmType<ObjT>, Args &&...) [with ObjT=NodeObj, f=&NodeObj::isWorkFinished, Args=<>]" 
/vt/src/vt/objgroup/proxy/proxy_objgroup_elm.impl.h(114): here
            instantiation of "decltype(auto) vt::objgroup::proxy::ProxyElm<ObjT>::invoke<f,Args...>(Args &&...) const [with ObjT=NodeObj, f=&NodeObj::isWorkFinished, Args=<>]" 
/vt/examples/collection/jacobi1d_vt.cc(325): here

/vt/src/vt/runnable/runnable.h(248): warning: missing return statement at end of non-void function "vt::runnable::RunnableNew::runLambda(Callable &&, Args &&...) [with Callable=__nv_bool (NodeObj::*)(), Args=<NodeObj *>]"
          detected during:
            instantiation of "decltype(auto) vt::runnable::RunnableNew::runLambda(Callable &&, Args &&...) [with Callable=__nv_bool (NodeObj::*)(), Args=<NodeObj *>]" 
/vt/src/vt/runnable/make_run%0D%0A%0D%0A%0D%0A ==> And there is more. Read log. <==

Build log

PR tests (clang-9, ubuntu, mpich)

Build for 6b69c91 (2023-03-14 17:08:17 UTC)

Compilation - successful

Testing - passed

Build log

PR tests (clang-11, ubuntu, mpich)

Build for 6b69c91 (2023-03-14 17:08:17 UTC)

Compilation - successful

Testing - passed

Build log

PR tests (clang-12, ubuntu, mpich)

Build for 6b69c91 (2023-03-14 17:08:17 UTC)

Compilation - successful

Testing - passed

Build log

PR tests (gcc-10, ubuntu, openmpi, no LB)

Build for 6b69c91 (2023-03-14 17:08:17 UTC)

Compilation - successful

Testing - passed

Build log

PR tests (gcc-9, ubuntu, mpich, zoltan)

Build for 6b69c91 (2023-03-14 17:08:17 UTC)

Compilation - successful

Testing - passed

Build log

PR tests (gcc-7, ubuntu, mpich, trace runtime, LB)

Build for 6b69c91 (2023-03-14 17:08:17 UTC)

Compilation - successful

Testing - passed

Build log

PR tests (nvidia cuda 11.2, ubuntu, mpich)

Build for ( UTC)

/vt/src/vt/runnable/runnable.h(248): warning: missing return statement at end of non-void function "vt::runnable::RunnableNew::runLambda(Callable &&, Args &&...) [with Callable=__nv_bool (NodeObj::*)(), Args=<NodeObj *>]"
          detected during:
            instantiation of "decltype(auto) vt::runnable::RunnableNew::runLambda(Callable &&, Args &&...) [with Callable=__nv_bool (NodeObj::*)(), Args=<NodeObj *>]" 
/vt/src/vt/runnable/make_runnable.h(282): here
            instantiation of "auto vt::runnable::RunnableMaker<MsgT>::runLambda(Callable &&, Args &&...) [with MsgT=vt::BaseMsgType, Callable=__nv_bool (NodeObj::*)(), Args=<NodeObj *>]" 
/vt/src/vt/objgroup/manager.impl.h(228): here
            instantiation of "decltype(auto) vt::objgroup::ObjGroupManager::invoke<ObjT,f,Args...>(vt::objgroup::ObjGroupManager::ProxyElmType<ObjT>, Args &&...) [with ObjT=NodeObj, f=&NodeObj::isWorkFinished, Args=<>]" 
/vt/src/vt/objgroup/proxy/proxy_objgroup_elm.impl.h(114): here
            instantiation of "decltype(auto) vt::objgroup::proxy::ProxyElm<ObjT>::invoke<f,Args...>(Args &&...) const [with ObjT=NodeObj, f=&NodeObj::isWorkFinished, Args=<>]" 
/vt/examples/collection/jacobi1d_vt.cc(325): here

/vt/src/vt/runnable/runnable.h(248): warning: missing return statement at end of non-void function "vt::runnable::RunnableNew::runLambda(Callable &&, Args &&...) [with Callable=bool (NodeObj::*)(), Args=<NodeObj *>]"
          detected during:
            instantiation of "decltype(auto) vt::runnable::RunnableNew::runLambda(Callable &&, Args &&...) [with Callable=bool (NodeObj::*)(), Args=<NodeObj *>]" 
/vt/src/vt/runnable/make_runnable.h(282): here
            instantiation of "auto vt::runnable::RunnableMaker<MsgT>::runLambda(Callable &&, Args &&...) [with MsgT=vt::BaseMsgType, Callable=bool (NodeObj::*)(), Args=<NodeObj *>]" 
/vt/src/vt/objgroup/manager.impl.h(228): here
            instantiation of "decltype(auto) vt::objgroup::ObjGroupManager::invoke<ObjT,f,Args...>(vt::objgroup::ObjGroupManager::ProxyElmType<ObjT>, Args &&...) [with ObjT=NodeObj, f=&NodeObj::isWorkFinished, Args=<>]" 
/vt/src/vt/objgroup/proxy/proxy_objgroup_elm.impl.h(114): here
            instantiation of "decltype(auto) vt::objgroup::proxy::ProxyElm<ObjT>::invoke<f,Args...>(Args &&...) const [with ObjT=NodeObj, f=&NodeObj::isWorkFinished, Args=<>]" 
/vt/examples/collection/jacobi1d_vt.cc(325): here

/vt/src/vt/runnable/runnable.h(248): warning: missing return statement at end of non-void function "vt::runnable::RunnableNew::runLambda(Callable &&, Args &&...) [with Callable=__nv_bool (NodeObj::*)(), Args=<NodeObj *>]"
          detected during:
            instantiation of "decltype(auto) vt::runnable::RunnableNew::runLambda(Callable &&, Args &&...) [with Callable=__nv_bool (NodeObj::*)(), Args=<NodeObj *>]" 
/vt/src/vt/runnable/make_run%0D%0A%0D%0A%0D%0A ==> And there is more. Read log. <==

Build log

PR tests (clang-13, ubuntu, mpich)

Build for 6b69c91 (2023-03-14 17:08:17 UTC)

Compilation - successful

Testing - passed

Build log

PR tests (gcc-11, ubuntu, mpich, json schema test)

Build for 6b69c91 (2023-03-14 17:08:17 UTC)

Compilation - successful

Testing - passed

Build log

PR tests (gcc-12, ubuntu, mpich)

Build for 6b69c91 (2023-03-14 17:08:17 UTC)

Compilation - successful

Testing - passed

Build log

PR tests (gcc-8, ubuntu, mpich, address sanitizer)

Build for 6b69c91 (2023-03-14 17:08:17 UTC)

Compilation - successful

Testing - passed

Build log

PR tests (clang-14, ubuntu, mpich)

Build for 6b69c91 (2023-03-14 17:08:17 UTC)

Compilation - successful

Testing - passed

Build log

PR tests (intel icpc, ubuntu, mpich)

Build for 6b69c91 (2023-03-14 17:08:17 UTC)

intel-cc: remark #10441: The Intel(R) C++ Compiler Classic (ICC) is deprecated and will be removed from product release in the second half of 2023. The Intel(R) oneAPI DPC++/C++ Compiler (ICX) is the recommended compiler moving forward. Please transition to use this compiler. Use '-diag-disable=10441' to disable this message.
intel-cc: remark #10441: The Intel(R) C++ Compiler Classic (ICC) is deprecated and will be removed from product release in the second half of 2023. The Intel(R) oneAPI DPC++/C++ Compiler (ICX) is the recommended compiler moving forward. Please transition to use this compiler. Use '-diag-disable=10441' to disable this message.
intel-cc: remark #10441: The Intel(R) C++ Compiler Classic (ICC) is deprecated and will be removed from product release in the second half of 2023. The Intel(R) oneAPI DPC++/C++ Compiler (ICX) is the recommended compiler moving forward. Please transition to use this compiler. Use '-diag-disable=10441' to disable this message.
intel-cc: remark #10441: The Intel(R) C++ Compiler Classic (ICC) is deprecated and will be removed from product release in the second half of 2023. The Intel(R) oneAPI DPC++/C++ Compiler (ICX) is the recommended compiler moving forward. Please transition to use this compiler. Use '-diag-disable=10441' to disable this message.
intel-cc: remark #10441: The Intel(R) C++ Compiler Classic (ICC) is deprecated and will be removed from product release in the second half of 2023. The Intel(R) oneAPI DPC++/C++ Compiler (ICX) is the recommended compiler moving forward. Please transition to use this compiler. Use '-diag-disable=10441' to disable this message.
intel-cc: remark #10441: The Intel(R) C++ Compiler Classic (ICC) is deprecated and will be removed from product release in the second half of 2023. The Intel(R) oneAPI DPC++/C++ Compiler (ICX) is the recommended compiler moving forward. Please transition to use this compiler. Use '-diag-disable=10441' to disable this message.
intel-cc: remark #10441: The Intel(R) C++ Compiler Classic (ICC) is deprecated and will be removed from product release in the second half of 2023. The Intel(R) oneAPI DPC++/C++ Compiler (ICX) is the recommended compiler moving forward. Please transition to use this compiler. Use '-diag-disable=10441' to disable this message.
intel-cc: remark #10441: The Intel(R) C++ Compiler Classic (ICC) is deprecated and will be removed from product release in the second half of 2023. The Intel(R) oneAPI DPC++/C++ Compiler (ICX) is the recommended compiler moving forward. Please transition to use this compiler. Use '-diag-disable=10441' to disable this message.
intel-cc: remark #10441: The Intel(R) C++ Compiler Classic (ICC) is deprecated and will be removed from product release in the second half of 2023. The Intel(R) oneAPI DPC++/C++ Compiler (ICX) is the recommended compiler moving forward. Please transition to use this compiler. Use '-diag-disable=10441' to disable this message.-%0D%0A%0D%0A%0D%0A ==> And there is more. Read log. <==

Build log

PhilMiller · 2023-01-24T23:41:40Z

src/vt/registry/auto/auto_registry_general.h

@@ -154,10 +155,11 @@ struct FunctorAdapterArgs<ObjTypeT, MsgT> {

 // Need to provide a non-pointer overload for parameterization auto-registered
 // functions for GCC


Was this here to address a limitation in older versions of GCC, that may no longer be applicable?

Should checking this be converted into a separate issue? (to avoid delaying merge for this PR)

PhilMiller · 2023-01-26T17:09:23Z

Separately, I think it would be worthwhile to do a refactor pushing that strong Node type all the way down the stack, to the point that we don't call get() on it except in MPI call arguments.

lifflander · 2023-02-08T22:11:59Z

@thearusable Can you focus on the objgroup and collection send/broadcast APIs?

thearusable · 2023-02-09T12:59:33Z

@thearusable Can you focus on the objgroup and collection send/broadcast APIs?

Sure, will do

src/vt/group/group_manager.h

PhilMiller · 2023-02-28T22:47:34Z

I'd be a bit more comfortable if the changes for reduce and scatter were split out to follow-up PRs, just because their interface is a bit more involved.

PhilMiller · 2023-02-28T23:00:10Z

OK, scatter is fine to keep in.. That's small and similar to send/broadcast

src/vt/registry/auto/auto_registry_common.h

… in non-debug builds

lifflander linked an issue Jan 13, 2023 that may be closed by this pull request

Automatically synthesize message types for sends to arbitrary handlers #276

Closed

PhilMiller reviewed Jan 17, 2023

View reviewed changes

cmake/load_bundled_libraries.cmake Outdated Show resolved Hide resolved

lifflander force-pushed the 276-automatically-synthesize-message-types-for-sends-to-arbitrary-handlers branch from 1429af2 to 3020c27 Compare January 17, 2023 18:32

PhilMiller reviewed Jan 17, 2023

View reviewed changes

examples/hello_world/hello_world.cc Outdated Show resolved Hide resolved

lifflander force-pushed the 276-automatically-synthesize-message-types-for-sends-to-arbitrary-handlers branch from 1338c01 to cb28d28 Compare January 18, 2023 00:45

lifflander marked this pull request as ready for review January 24, 2023 18:32

lifflander force-pushed the 276-automatically-synthesize-message-types-for-sends-to-arbitrary-handlers branch from f90c90b to b600931 Compare January 24, 2023 18:32

PhilMiller reviewed Jan 24, 2023

View reviewed changes

PhilMiller approved these changes Jan 26, 2023

View reviewed changes

lifflander force-pushed the 276-automatically-synthesize-message-types-for-sends-to-arbitrary-handlers branch from 7bb32e4 to dcc89a1 Compare February 2, 2023 22:25

PhilMiller approved these changes Feb 2, 2023

View reviewed changes

thearusable force-pushed the 276-automatically-synthesize-message-types-for-sends-to-arbitrary-handlers branch 2 times, most recently from e69f41f to ab229fe Compare February 8, 2023 15:07

lifflander force-pushed the 276-automatically-synthesize-message-types-for-sends-to-arbitrary-handlers branch from ab229fe to 3502881 Compare February 8, 2023 21:36

thearusable force-pushed the 276-automatically-synthesize-message-types-for-sends-to-arbitrary-handlers branch 2 times, most recently from 57c81e5 to 96fbfe5 Compare February 10, 2023 18:21

nmm0 mentioned this pull request Feb 28, 2023

Meeting Agenda [do not close] #925

Open

PhilMiller reviewed Feb 28, 2023

View reviewed changes

src/vt/group/group_manager.h Outdated Show resolved Hide resolved

PhilMiller changed the title ~~276 automatically synthesize message types for sends to arbitrary handlers~~ 276 Automatically synthesize and infer message types for sends to arbitrary handlers Feb 28, 2023

PhilMiller changed the title ~~276 Automatically synthesize and infer message types for sends to arbitrary handlers~~ 276 Automatically synthesize and deduce message types for sends to arbitrary handlers Feb 28, 2023

PhilMiller reviewed Feb 28, 2023

View reviewed changes

src/vt/registry/auto/auto_registry_common.h Show resolved Hide resolved

PhilMiller reviewed Feb 28, 2023

View reviewed changes

src/vt/registry/auto/auto_registry_common.h Show resolved Hide resolved

PhilMiller reviewed Feb 28, 2023

View reviewed changes

src/vt/registry/auto/auto_registry_common.h Show resolved Hide resolved

lifflander and others added 26 commits March 14, 2023 10:08

#276: active: make Node parameter a strong type

b1441fe

#276: active: add get

e676f2d

#276: Automatically synthesize message type in vt/collective

0e7d5d3

#276: Automatically synthesize message type in vt/group

1a286a8

#276: Update calls to sendMsg and broadcastMsg

8664831

#276: Automatically synthesize Msg type in vt/objgroup

b314870

#276: Synthesize Msg type in vt/collection

46651bd

#276: vrt: add synthesis of messages for collections

289d084

#276: registry: cleanup with if constexpr

faba282

#276: registry: simplify code

d786976

#276: vrt: move function traits to utils

174c1b7

#276: vrt: implement for broadcasts and invokes

e206bb0

#276: vrt: swap ColT and MsgT for plain function handlers

181d8ce

#276: registry: simplify constexpr dispatch code

04de002

#276: vrt: use new handlers to remove messages

cd2e1a6

#276: collective: fix scatter

f224238

#276: objgroup: implement parameterization

629ead7

#276: objgroup: fix msg pointer access issues

09865e0

#276: examples: update for new interface

1109fba

#276: objgroup: fix epoch capture without message

b942097

#276: registry: add static assert for invalid paths

d7e91f0

#276: registry: switch to regular assert

36d0707

#276: examples: simplify with new syntax

47d3a32

#276: param: remove old component

df46d65

#276: Clarify boolean literal function argument

aeed39d

#276: Turn vtAssert(false, ...) into vtAbort(...) to avoid issues…

6b69c91

… in non-debug builds

lifflander force-pushed the 276-automatically-synthesize-message-types-for-sends-to-arbitrary-handlers branch from bead248 to 6b69c91 Compare March 14, 2023 17:08

PhilMiller added this pull request to the merge queue Mar 14, 2023

github-merge-queue bot removed this pull request from the merge queue due to no response for status checks Mar 14, 2023

lifflander merged commit edf105d into develop Mar 15, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

276 Automatically synthesize and deduce message types for sends to arbitrary handlers #2064

276 Automatically synthesize and deduce message types for sends to arbitrary handlers #2064

lifflander commented Jan 13, 2023

codecov bot commented Jan 24, 2023 •

edited

Loading

github-actions bot commented Jan 24, 2023 •

edited

Loading

PhilMiller Jan 24, 2023

cz4rs Mar 14, 2023

PhilMiller commented Jan 26, 2023

lifflander commented Feb 8, 2023

thearusable commented Feb 9, 2023

PhilMiller commented Feb 28, 2023

PhilMiller commented Feb 28, 2023

		@@ -154,10 +155,11 @@ struct FunctorAdapterArgs<ObjTypeT, MsgT> {

		// Need to provide a non-pointer overload for parameterization auto-registered
		// functions for GCC

276 Automatically synthesize and deduce message types for sends to arbitrary handlers #2064

276 Automatically synthesize and deduce message types for sends to arbitrary handlers #2064

Conversation

lifflander commented Jan 13, 2023

codecov bot commented Jan 24, 2023 • edited Loading

Codecov Report

github-actions bot commented Jan 24, 2023 • edited Loading

Pipelines results

PhilMiller Jan 24, 2023

Choose a reason for hiding this comment

cz4rs Mar 14, 2023

Choose a reason for hiding this comment

PhilMiller commented Jan 26, 2023

lifflander commented Feb 8, 2023

thearusable commented Feb 9, 2023

PhilMiller commented Feb 28, 2023

PhilMiller commented Feb 28, 2023

codecov bot commented Jan 24, 2023 •

edited

Loading

github-actions bot commented Jan 24, 2023 •

edited

Loading