RunningLocalModels.md

Running Local Models

Install 'unifree' and activate the Python Virtual Environment

Linux/MacOS

git clone https://github.com/ProjectUnifree/unifree
cd unifree
./launch.sh
source venv/bin/activate

Windows

git clone https://github.com/ProjectUnifree/unifree
cd unifree
.\launch.bat
.\venv\Scripts\activate.bat

Install ctranformers

Please follow the ctransformers GPU install guide to install to enable Python bindings for the Transformer models on your local machine. We recommend using CUDA or Metal to increase the inference speed.

Use unifree as usual

See main README.md for usage.