build based on f29ae85

dev/.documenter-siteinfo.json

@@ -1 +1 @@
-{"documenter":{"julia_version":"1.10.0","generation_timestamp":"2024-01-30T21:17:11","documenter_version":"1.2.1"}}
+{"documenter":{"julia_version":"1.10.0","generation_timestamp":"2024-02-01T09:58:15","documenter_version":"1.2.1"}}

dev/examples/readme_examples/index.html

@@ -89,4 +89,4 @@
 msg = aigenerate(PT.MistralOpenAISchema(), &quot;Say Hi!&quot;; model=&quot;mistral-tiny&quot;, api_key=ENV[&quot;MISTRALAI_API_KEY&quot;])</code></pre><p>As you can see, we can load your API key either from the ENV or via the Preferences.jl mechanism (see <code>?PREFERENCES</code> for more information).</p><p>But MistralAI are not the only ones! There are many other exciting providers, eg, <a href="https://docs.perplexity.ai/">Perplexity.ai</a>, <a href="https://app.fireworks.ai/">Fireworks.ai</a>. As long as they are compatible with the OpenAI API (eg, sending <code>messages</code> with <code>role</code> and <code>content</code> keys), you can use them with PromptingTools.jl by using <code>schema = CustomOpenAISchema()</code>:</p><pre><code class="language-julia hljs"># Set your API key and the necessary base URL for the API
 api_key = &quot;...&quot;
 prompt = &quot;Say hi!&quot;
-msg = aigenerate(PT.CustomOpenAISchema(), prompt; model=&quot;my_model&quot;, api_key, api_kwargs=(; url=&quot;http://localhost:8081&quot;))</code></pre><p>As you can see, it also works for any local models that you might have running on your computer!</p><p>Note: At the moment, we only support <code>aigenerate</code> and <code>aiembed</code> functions for MistralAI and other OpenAI-compatible APIs. We plan to extend the support in the future.</p></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../../getting_started/">« Getting Started</a><a class="docs-footer-nextpage" href="../working_with_aitemplates/">Using AITemplates »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option><option value="auto">Automatic (OS)</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> version 1.2.1 on <span class="colophon-date" title="Tuesday 30 January 2024 21:17">Tuesday 30 January 2024</span>. Using Julia version 1.10.0.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>
+msg = aigenerate(PT.CustomOpenAISchema(), prompt; model=&quot;my_model&quot;, api_key, api_kwargs=(; url=&quot;http://localhost:8081&quot;))</code></pre><p>As you can see, it also works for any local models that you might have running on your computer!</p><p>Note: At the moment, we only support <code>aigenerate</code> and <code>aiembed</code> functions for MistralAI and other OpenAI-compatible APIs. We plan to extend the support in the future.</p></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../../getting_started/">« Getting Started</a><a class="docs-footer-nextpage" href="../working_with_aitemplates/">Using AITemplates »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option><option value="auto">Automatic (OS)</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> version 1.2.1 on <span class="colophon-date" title="Thursday 1 February 2024 09:58">Thursday 1 February 2024</span>. Using Julia version 1.10.0.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>

dev/examples/working_with_aitemplates/index.html

@@ -43,4 +43,4 @@
 PT.save_template(filename,
     tpl;
     description = "For asking data analysis questions in Julia language. Placeholders: `ask`")
-rm(filename) # cleanup if we don&#39;t like it</code></pre><p>When you create a new template, remember to re-load the templates with <code>load_templates!()</code> so that it&#39;s available for use.</p><pre><code class="language-julia hljs">PT.load_templates!();</code></pre><p>!!! If you have some good templates (or suggestions for the existing ones), please consider sharing them with the community by opening a PR to the <code>templates</code> directory!</p><hr/><p><em>This page was generated using <a href="https://github.com/fredrikekre/Literate.jl">Literate.jl</a>.</em></p></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../readme_examples/">« Various examples</a><a class="docs-footer-nextpage" href="../working_with_ollama/">Local models with Ollama.ai »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option><option value="auto">Automatic (OS)</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> version 1.2.1 on <span class="colophon-date" title="Tuesday 30 January 2024 21:17">Tuesday 30 January 2024</span>. Using Julia version 1.10.0.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>
+rm(filename) # cleanup if we don&#39;t like it</code></pre><p>When you create a new template, remember to re-load the templates with <code>load_templates!()</code> so that it&#39;s available for use.</p><pre><code class="language-julia hljs">PT.load_templates!();</code></pre><p>!!! If you have some good templates (or suggestions for the existing ones), please consider sharing them with the community by opening a PR to the <code>templates</code> directory!</p><hr/><p><em>This page was generated using <a href="https://github.com/fredrikekre/Literate.jl">Literate.jl</a>.</em></p></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../readme_examples/">« Various examples</a><a class="docs-footer-nextpage" href="../working_with_ollama/">Local models with Ollama.ai »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option><option value="auto">Automatic (OS)</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> version 1.2.1 on <span class="colophon-date" title="Thursday 1 February 2024 09:58">Thursday 1 February 2024</span>. Using Julia version 1.10.0.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>

dev/examples/working_with_custom_apis/index.html

@@ -7,4 +7,4 @@
 api_key = "..."
 provider_url = "..." # provider API URL
 prompt = "Say hi!"
-msg = aigenerate(PT.CustomOpenAISchema(), prompt; model=&quot;&lt;some-model&gt;&quot;, api_key, api_kwargs=(; url=provider_url))</code></pre><blockquote><p>[!TIP] If you register the model names with <code>PT.register_model!</code>, you won&#39;t have to keep providing the <code>schema</code> manually.</p></blockquote><p>Note: At the moment, we only support <code>aigenerate</code> and <code>aiembed</code> functions.</p><h2 id="Using-llama.cpp-server"><a class="docs-heading-anchor" href="#Using-llama.cpp-server">Using llama.cpp server</a><a id="Using-llama.cpp-server-1"></a><a class="docs-heading-anchor-permalink" href="#Using-llama.cpp-server" title="Permalink"></a></h2><p>In line with the above, you can also use the <a href="https://github.com/ggerganov/llama.cpp/blob/master/examples/server/README.md"><code>llama.cpp</code> server</a>. </p><p>It is a bit more technically demanding because you need to &quot;compile&quot; <code>llama.cpp</code> first, but it will always have the latest models and it is quite fast (eg, faster than Ollama, which uses llama.cpp under the hood but has some extra overhead).</p><p>Start your server in a command line (<code>-m</code> refers to the model file, <code>-c</code> is the context length, <code>-ngl</code> is the number of layers to offload to GPU):</p><pre><code class="language-bash hljs">./server -m models/mixtral-8x7b-instruct-v0.1.Q4_K_M.gguf -c 2048 -ngl 99</code></pre><p>Then simply access it via PromptingTools:</p><pre><code class="language-julia hljs">msg = aigenerate(PT.CustomOpenAISchema(), &quot;Count to 5 and say hi!&quot;; api_kwargs=(; url=&quot;http://localhost:8080/v1&quot;))</code></pre><blockquote><p>[!TIP] If you register the model names with <code>PT.register_model!</code>, you won&#39;t have to keep providing the <code>schema</code> manually. It can be any <code>model</code> name, because the model is actually selected when you start the server in the terminal.</p></blockquote></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../working_with_ollama/">« Local models with Ollama.ai</a><a class="docs-footer-nextpage" href="../building_RAG/">Building RAG Application »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option><option value="auto">Automatic (OS)</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> version 1.2.1 on <span class="colophon-date" title="Tuesday 30 January 2024 21:17">Tuesday 30 January 2024</span>. Using Julia version 1.10.0.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>
+msg = aigenerate(PT.CustomOpenAISchema(), prompt; model=&quot;&lt;some-model&gt;&quot;, api_key, api_kwargs=(; url=provider_url))</code></pre><blockquote><p>[!TIP] If you register the model names with <code>PT.register_model!</code>, you won&#39;t have to keep providing the <code>schema</code> manually.</p></blockquote><p>Note: At the moment, we only support <code>aigenerate</code> and <code>aiembed</code> functions.</p><h2 id="Using-llama.cpp-server"><a class="docs-heading-anchor" href="#Using-llama.cpp-server">Using llama.cpp server</a><a id="Using-llama.cpp-server-1"></a><a class="docs-heading-anchor-permalink" href="#Using-llama.cpp-server" title="Permalink"></a></h2><p>In line with the above, you can also use the <a href="https://github.com/ggerganov/llama.cpp/blob/master/examples/server/README.md"><code>llama.cpp</code> server</a>. </p><p>It is a bit more technically demanding because you need to &quot;compile&quot; <code>llama.cpp</code> first, but it will always have the latest models and it is quite fast (eg, faster than Ollama, which uses llama.cpp under the hood but has some extra overhead).</p><p>Start your server in a command line (<code>-m</code> refers to the model file, <code>-c</code> is the context length, <code>-ngl</code> is the number of layers to offload to GPU):</p><pre><code class="language-bash hljs">./server -m models/mixtral-8x7b-instruct-v0.1.Q4_K_M.gguf -c 2048 -ngl 99</code></pre><p>Then simply access it via PromptingTools:</p><pre><code class="language-julia hljs">msg = aigenerate(PT.CustomOpenAISchema(), &quot;Count to 5 and say hi!&quot;; api_kwargs=(; url=&quot;http://localhost:8080/v1&quot;))</code></pre><blockquote><p>[!TIP] If you register the model names with <code>PT.register_model!</code>, you won&#39;t have to keep providing the <code>schema</code> manually. It can be any <code>model</code> name, because the model is actually selected when you start the server in the terminal.</p></blockquote></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../working_with_ollama/">« Local models with Ollama.ai</a><a class="docs-footer-nextpage" href="../building_RAG/">Building RAG Application »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option><option value="auto">Automatic (OS)</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> version 1.2.1 on <span class="colophon-date" title="Thursday 1 February 2024 09:58">Thursday 1 February 2024</span>. Using Julia version 1.10.0.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>

dev/examples/working_with_ollama/index.html

@@ -38,4 +38,4 @@
     LinearAlgebra.normalize;
     model = &quot;openhermes2.5-mistral&quot;)</code></pre><pre><code class="nohighlight hljs">PromptingTools.DataMessage(Matrix{Float64} of size (4096, 2))</code></pre><p>Cosine similarity is then a simple multiplication</p><pre><code class="language-julia hljs">msg.content&#39; * msg.content[:, 1]</code></pre><pre><code class="nohighlight hljs">2-element Vector{Float64}:
  0.9999999999999982
- 0.40796033843072876</code></pre><hr/><p><em>This page was generated using <a href="https://github.com/fredrikekre/Literate.jl">Literate.jl</a>.</em></p></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../working_with_aitemplates/">« Using AITemplates</a><a class="docs-footer-nextpage" href="../working_with_custom_apis/">Custom APIs (Mistral, Llama.cpp) »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option><option value="auto">Automatic (OS)</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> version 1.2.1 on <span class="colophon-date" title="Tuesday 30 January 2024 21:17">Tuesday 30 January 2024</span>. Using Julia version 1.10.0.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>
+ 0.40796033843072876</code></pre><hr/><p><em>This page was generated using <a href="https://github.com/fredrikekre/Literate.jl">Literate.jl</a>.</em></p></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../working_with_aitemplates/">« Using AITemplates</a><a class="docs-footer-nextpage" href="../working_with_custom_apis/">Custom APIs (Mistral, Llama.cpp) »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option><option value="auto">Automatic (OS)</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> version 1.2.1 on <span class="colophon-date" title="Thursday 1 February 2024 09:58">Thursday 1 February 2024</span>. Using Julia version 1.10.0.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>