Cortex.cpp: RAM & VRAM Management #1128

dan-homebrew started this conversation in Architecture Specs

dan-homebrew
Sep 6, 2024
Maintainer

Goal

Cortex.cpp should monitor free memory (e.g. RAM and VRAM)
Cortex.cpp should prevent user from ending up in OOM errors that hang their computer
e.g. OOM due to user loading model size > free memory

Replies: 0 comments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment