transformers

LLMs

This repository consists of methods to run LLMs in PyTorch, ONNX and Llama.cpp with operators dispatch to NPU.

This is an early access flow, and expected to be upgraded in upcoming release.

Run LLMs

• LLMs on RyzenAI with Pytorch
• Speculative Decoding of LLMs in Pytorch
• LLMs on RyzenAI with ONNX
• LLMs on RyzenAI with llama.cpp