New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

NPU Support ? #612

Open

gformcreation opened this issue Aug 21, 2024 · 2 comments

gformcreation commented Aug 21, 2024

Hi Guys,
Wanted to ask does this supports proper NPU ? for llm loading and processing?

Member

danemadsen commented Aug 24, 2024

No it doesn't as llama.cpp doesn't to the best of my knowledge. Once they support it, it will be supported here

Author

gformcreation commented Aug 24, 2024

FYI ...
abetlen/llama-cpp-python#1702

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment