Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VMFB buffer access test #368

Merged
merged 9 commits into from
Oct 30, 2024
Merged

Conversation

renxida
Copy link
Contributor

@renxida renxida commented Oct 29, 2024

This test mimics the kvcache indexing, reading, and writing in open-llama-3b-v2-f16.gguf.

It's a simple test to verify that shortfin can:

  • Create device arrays
  • Map and fill them
  • Invoke a VMFB and pass arguments to it
  • Properly retain changes made by the VMFB to device arrays provided as arguments

@renxida renxida changed the title VMFB buffer access test with numpy dependency removed VMFB buffer access test (numpy dependency removed) Oct 29, 2024
@renxida
Copy link
Contributor Author

renxida commented Oct 29, 2024

used to be #325, which was reverted because it broke CI

@renxida renxida force-pushed the sf-vmfb-buffer-access-test branch from 30f2a8e to d2128af Compare October 29, 2024 23:29
@renxida
Copy link
Contributor Author

renxida commented Oct 29, 2024

Compared to #325 this has 2 code changes:

  • no more np dependency
  • skipped for windows
    and some name / docstring /comment changes.

Copy link

@kumardeepakamd kumardeepakamd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should try to get it fixed for windows before merging.

@renxida renxida changed the title VMFB buffer access test (numpy dependency removed) VMFB buffer access test Oct 30, 2024
@renxida renxida force-pushed the sf-vmfb-buffer-access-test branch from 8f77093 to 3d552cb Compare October 30, 2024 00:05
@renxida renxida marked this pull request as ready for review October 30, 2024 00:21
Copy link

@kumardeepakamd kumardeepakamd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for enabling it for Windows too.

@renxida renxida merged commit 866691e into nod-ai:main Oct 30, 2024
11 checks passed
@renxida renxida deleted the sf-vmfb-buffer-access-test branch October 30, 2024 16:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants