Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement new data replication handle for use cases like NimbleSM #1554

Open
lifflander opened this issue Sep 7, 2021 · 0 comments · May be fixed by #1562
Open

Implement new data replication handle for use cases like NimbleSM #1554

lifflander opened this issue Sep 7, 2021 · 0 comments · May be fixed by #1562

Comments

@lifflander
Copy link
Collaborator

lifflander commented Sep 7, 2021

What Needs to be Done?

We would like to implement a data handle that can be read across multiple ranks, is efficiently cached, and exploits communication persistence to efficiently update the cache of data.

  • We are imagining a data handle vt::DR<X> my_handle, where X is a user's data type that may or may not be serializable. - vt::DR<X> has reference (or move-only) semantics on a given rank.
  • It can be migrated to another rank safely and it does not interfere with any reads.
  • When initialized, it is given a distributed token/name that can be used to read against at a later point.
vt::Reader<X> reader = vt::DR::read<X>(TOKEN);
while (not reader.isReady()) ;
X* val = reader.get();

Under the hood, the DR will build spanning trees to distribute data efficiently and cache data as necessary.

We are thinking that the most complex part is the update protocol.

  • When are updates allowed?
  • How does the user sequence readers (which may be cached) with respect to an update?
lifflander added a commit that referenced this issue Sep 21, 2021
@lifflander lifflander linked a pull request Sep 28, 2021 that will close this issue
lifflander added a commit that referenced this issue Sep 29, 2021
lifflander added a commit that referenced this issue Sep 29, 2021
lifflander pushed a commit that referenced this issue Oct 4, 2021
lifflander added a commit that referenced this issue Oct 4, 2021
lifflander added a commit that referenced this issue Oct 4, 2021
lifflander added a commit that referenced this issue Oct 5, 2021
lifflander added a commit that referenced this issue Oct 5, 2021
lifflander added a commit that referenced this issue Oct 5, 2021
lifflander added a commit that referenced this issue Oct 5, 2021
lifflander added a commit that referenced this issue Oct 5, 2021
lifflander pushed a commit that referenced this issue Oct 14, 2021
lifflander added a commit that referenced this issue Oct 14, 2021
lifflander added a commit that referenced this issue Oct 14, 2021
lifflander added a commit that referenced this issue Oct 14, 2021
lifflander added a commit that referenced this issue Oct 14, 2021
lifflander added a commit that referenced this issue Oct 14, 2021
lifflander added a commit that referenced this issue Oct 14, 2021
lifflander added a commit that referenced this issue Oct 14, 2021
lifflander pushed a commit that referenced this issue Oct 19, 2021
lifflander added a commit that referenced this issue Oct 19, 2021
lifflander added a commit that referenced this issue Oct 19, 2021
lifflander added a commit that referenced this issue Oct 19, 2021
lifflander added a commit that referenced this issue Oct 19, 2021
lifflander added a commit that referenced this issue Oct 19, 2021
lifflander added a commit that referenced this issue Oct 19, 2021
lifflander added a commit that referenced this issue Oct 19, 2021
lifflander added a commit that referenced this issue Oct 21, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant