Add a note about needing to pass the returned tokens

utilityai · Feb 27, 2024 · 1a37ca0 · 1a37ca0
1 parent c6179bf
commit 1a37ca0
Showing 1 changed file with 2 additions and 0 deletions.
diff --git a/llama-cpp-2/src/context/session.rs b/llama-cpp-2/src/context/session.rs
@@ -58,6 +58,8 @@ impl LlamaContext<'_> {
     }
     /// Load a session file into the current context.
     ///
+    /// You still need to pass the returned tokens to the context for inference to work. What this function buys you is that the KV caches are already filled with the relevant data.
+    ///
     /// # Parameters
     ///
     /// * `path_session` - The file to load from. It must be a session file from a compatible context, otherwise the function will error.