Code

Context and model enhancements#510

Code

Merged

MarcusDunn merged 8 commits intoutilityai:mainutilityai/llama-cpp-rs:mainfrom brittlewis12:context-and-model-enhancementsbrittlewis12/llama-cpp-rs:context-and-model-enhancementsCopy head branch name to clipboardSep 28, 2024

+189-22

Commits

Commits on Sep 25, 2024

• Demonstrate using candidates in simple example

  brittlewis12

  committed
  5de722f

• Expose mirostat v1 sampling

  brittlewis12

  committed
  d8f80cc

• Expose llama_token_is_eog equivalent on Model

  Show description for 3316646
  brittlewis12

  committed
  3316646

• Demonstrate is_eog_token in simple example

  brittlewis12

  committed
  8ce9218

• Expose llama_kv_cache_seq_rm on Context

  Show description for 9c1ec6d
  brittlewis12

  committed
  9c1ec6d

• Expose flash attention

  brittlewis12

  committed
  fdc70c1

• Expose offload_kqv to control GPU KV cache & KQV ops

  brittlewis12

  committed
  b10bd0a

Commits on Sep 27, 2024

• Handle KV cache mutations for llama_pos values greater than u16

  Show description for 7d1b2d5
  brittlewis12

  committed
  7d1b2d5