From 5547fc4f65b3db273e9c37de9dc8a0c999704eac Mon Sep 17 00:00:00 2001 From: "Documenter.jl" Date: Fri, 1 Mar 2024 19:41:28 +0000 Subject: [PATCH] build based on 8b726db --- dev/.documenter-siteinfo.json | 2 +- dev/examples/building_RAG/index.html | 2 +- dev/examples/readme_examples/index.html | 4 +- .../working_with_aitemplates/index.html | 2 +- .../working_with_custom_apis/index.html | 2 +- .../working_with_google_ai_studio/index.html | 2 +- dev/examples/working_with_ollama/index.html | 2 +- dev/frequently_asked_questions/index.html | 2 +- dev/getting_started/index.html | 2 +- dev/how_it_works/index.html | 8 +- dev/index.html | 2 +- dev/reference/index.html | 128 +++++++++++------- dev/reference_agenttools/index.html | 20 +-- dev/reference_apitools/index.html | 4 +- dev/reference_experimental/index.html | 2 +- dev/reference_ragtools/index.html | 28 ++-- dev/search_index.js | 2 +- 17 files changed, 121 insertions(+), 93 deletions(-) diff --git a/dev/.documenter-siteinfo.json b/dev/.documenter-siteinfo.json index 49d16cc93..6ee948701 100644 --- a/dev/.documenter-siteinfo.json +++ b/dev/.documenter-siteinfo.json @@ -1 +1 @@ -{"documenter":{"julia_version":"1.10.1","generation_timestamp":"2024-02-29T09:27:04","documenter_version":"1.2.1"}} \ No newline at end of file +{"documenter":{"julia_version":"1.10.1","generation_timestamp":"2024-03-01T19:41:26","documenter_version":"1.2.1"}} \ No newline at end of file diff --git a/dev/examples/building_RAG/index.html b/dev/examples/building_RAG/index.html index 255578feb..f20d3079f 100644 --- a/dev/examples/building_RAG/index.html +++ b/dev/examples/building_RAG/index.html @@ -65,4 +65,4 @@ results = filter(x->!isnothing(x.answer_score), results);

Note: You could also use the vectorized version results = run_qa_evals(evals) to evaluate all items at once.


 # Let's take a simple average to calculate our score
 @info "RAG Evals: $(length(results)) results, Avg. score: $(round(mean(x->x.answer_score, results);digits=1)), Retrieval score: $(100*round(Int,mean(x->x.retrieval_score,results)))%"
[ Info: RAG Evals: 10 results, Avg. score: 4.6, Retrieval score: 100%
-

Note: The retrieval score is 100% only because we have two small documents and running on 10 items only. In practice, you would have a much larger document set and a much larger eval set, which would result in a more representative retrieval score.

You can also analyze the results in a DataFrame:

df = DataFrame(results)
10×8 DataFrame
Rowsourcecontextquestionanswerretrieval_scoreretrieval_rankanswer_scoreparameters
StringStringStringSubStrin…Float64Int64Float64Dict…
1examples/data/database_style_joins.txtDatabase-Style Joins\nIntroduction to joins\nWe often need to combine two or more data sets together to provide a complete picture of the topic we are studying. For example, suppose that we have the following two data sets:\n\njulia> using DataFramesWhat is the purpose of joining two or more data sets together?The purpose of joining two or more data sets together is to combine the data sets based on a common key and provide a complete picture of the topic being studied.1.015.0Dict(:top_k=>3)
2examples/data/database_style_joins.txtjulia> people = DataFrame(ID=[20, 40], Name=["John Doe", "Jane Doe"])\n2×2 DataFrame\n Row │ ID Name\n │ Int64 String\n─────┼─────────────────\n 1 │ 20 John Doe\n 2 │ 40 Jane DoeWhat is the DataFrame called 'people' composed of?The DataFrame called 'people' consists of two columns: 'ID' and 'Name'. The 'ID' column contains integers, and the 'Name' column contains strings.1.014.0Dict(:top_k=>3)
3examples/data/database_style_joins.txtjulia> jobs = DataFrame(ID=[20, 40], Job=["Lawyer", "Doctor"])\n2×2 DataFrame\n Row │ ID Job\n │ Int64 String\n─────┼───────────────\n 1 │ 20 Lawyer\n 2 │ 40 DoctorWhat are the jobs and IDs listed in the dataframe?The jobs and IDs listed in the dataframe are as follows:\n\nID: 20\nJob: Lawyer\n\nID: 40\nJob: Doctor1.014.67Dict(:top_k=>3)
4examples/data/database_style_joins.txtWe might want to work with a larger data set that contains both the names and jobs for each ID. We can do this using the innerjoin function:How can we combine the names and jobs for each ID in a larger data set?We can use the `innerjoin` function to combine the names and jobs for each ID in a larger data set.1.014.33333Dict(:top_k=>3)
5examples/data/database_style_joins.txtjulia> innerjoin(people, jobs, on = :ID)\n2×3 DataFrame\n Row │ ID Name Job\n │ Int64 String String\n─────┼─────────────────────────\n 1 │ 20 John Doe Lawyer\n 2 │ 40 Jane Doe DoctorWhat is the name of the person with the ID 40 and their job?The name of the person with the ID 40 is Jane Doe and their job is Doctor.1.024.67Dict(:top_k=>3)
6examples/data/database_style_joins.txtIn relational database theory, this operation is generally referred to as a join. The columns used to determine which rows should be combined during a join are called keys.\n\nThe following functions are provided to perform seven kinds of joins:What are the different kinds of joins?The different kinds of joins are:\n\n1. Inner Join: Returns only the rows that have matching values in both data frames.\n2. Left Join: Returns all rows from the left data frame and the matching rows from the right data frame.\n3. Right Join: Returns all rows from the right data frame and the matching rows from the left data frame.\n4. Full Outer Join: Returns all rows from both data frames and fills in missing values with null.\n5. Cross Join: Returns the cartesian product of the rows from both data frames.\n6. Semi Join: Returns only the rows from the left data frame that have matching values in the right data frame.\n7. Anti Join: Returns only the rows from the left data frame that do not have matching values in the right data frame.1.014.66667Dict(:top_k=>3)
7examples/data/database_style_joins.txtinnerjoin: the output contains rows for values of the key that exist in all passed data frames.What does the output of the inner join operation contain?The output of the inner join operation contains only the rows for values of the key that exist in all passed data frames.1.015.0Dict(:top_k=>3)
8examples/data/database_style_joins.txtleftjoin: the output contains rows for values of the key that exist in the first (left) argument, whether or not that value exists in the second (right) argument.What is the purpose of the left join operation?The purpose of the left join operation is to combine data from two tables based on a common key, where all rows from the left (first) table are included in the output, regardless of whether there is a match in the right (second) table.1.014.66667Dict(:top_k=>3)
9examples/data/database_style_joins.txtrightjoin: the output contains rows for values of the key that exist in the second (right) argument, whether or not that value exists in the first (left) argument.What is the purpose of the right join operation?The purpose of the right join operation is to include all the rows from the second (right) argument, regardless of whether a match is found in the first (left) argument.1.014.67Dict(:top_k=>3)
10examples/data/database_style_joins.txtouterjoin: the output contains rows for values of the key that exist in any of the passed data frames.\nsemijoin: Like an inner join, but output is restricted to columns from the first (left) argument.What is the difference between outer join and semi join?The difference between outer join and semi join is that outer join includes rows for values of the key that exist in any of the passed data frames, whereas semi join is like an inner join but only outputs columns from the first argument.1.014.66667Dict(:top_k=>3)

We're done for today!

What would we do next?

... and much more! See some ideas in Anyscale RAG tutorial

This page was generated using Literate.jl.

+

Note: The retrieval score is 100% only because we have two small documents and running on 10 items only. In practice, you would have a much larger document set and a much larger eval set, which would result in a more representative retrieval score.

You can also analyze the results in a DataFrame:

df = DataFrame(results)
10×8 DataFrame
Rowsourcecontextquestionanswerretrieval_scoreretrieval_rankanswer_scoreparameters
StringStringStringSubStrin…Float64Int64Float64Dict…
1examples/data/database_style_joins.txtDatabase-Style Joins\nIntroduction to joins\nWe often need to combine two or more data sets together to provide a complete picture of the topic we are studying. For example, suppose that we have the following two data sets:\n\njulia> using DataFramesWhat is the purpose of joining two or more data sets together?The purpose of joining two or more data sets together is to combine the data sets based on a common key and provide a complete picture of the topic being studied.1.015.0Dict(:top_k=>3)
2examples/data/database_style_joins.txtjulia> people = DataFrame(ID=[20, 40], Name=["John Doe", "Jane Doe"])\n2×2 DataFrame\n Row │ ID Name\n │ Int64 String\n─────┼─────────────────\n 1 │ 20 John Doe\n 2 │ 40 Jane DoeWhat is the DataFrame called 'people' composed of?The DataFrame called 'people' consists of two columns: 'ID' and 'Name'. The 'ID' column contains integers, and the 'Name' column contains strings.1.014.0Dict(:top_k=>3)
3examples/data/database_style_joins.txtjulia> jobs = DataFrame(ID=[20, 40], Job=["Lawyer", "Doctor"])\n2×2 DataFrame\n Row │ ID Job\n │ Int64 String\n─────┼───────────────\n 1 │ 20 Lawyer\n 2 │ 40 DoctorWhat are the jobs and IDs listed in the dataframe?The jobs and IDs listed in the dataframe are as follows:\n\nID: 20\nJob: Lawyer\n\nID: 40\nJob: Doctor1.014.67Dict(:top_k=>3)
4examples/data/database_style_joins.txtWe might want to work with a larger data set that contains both the names and jobs for each ID. We can do this using the innerjoin function:How can we combine the names and jobs for each ID in a larger data set?We can use the `innerjoin` function to combine the names and jobs for each ID in a larger data set.1.014.33333Dict(:top_k=>3)
5examples/data/database_style_joins.txtjulia> innerjoin(people, jobs, on = :ID)\n2×3 DataFrame\n Row │ ID Name Job\n │ Int64 String String\n─────┼─────────────────────────\n 1 │ 20 John Doe Lawyer\n 2 │ 40 Jane Doe DoctorWhat is the name of the person with the ID 40 and their job?The name of the person with the ID 40 is Jane Doe and their job is Doctor.1.024.67Dict(:top_k=>3)
6examples/data/database_style_joins.txtIn relational database theory, this operation is generally referred to as a join. The columns used to determine which rows should be combined during a join are called keys.\n\nThe following functions are provided to perform seven kinds of joins:What are the different kinds of joins?The different kinds of joins are:\n\n1. Inner Join: Returns only the rows that have matching values in both data frames.\n2. Left Join: Returns all rows from the left data frame and the matching rows from the right data frame.\n3. Right Join: Returns all rows from the right data frame and the matching rows from the left data frame.\n4. Full Outer Join: Returns all rows from both data frames and fills in missing values with null.\n5. Cross Join: Returns the cartesian product of the rows from both data frames.\n6. Semi Join: Returns only the rows from the left data frame that have matching values in the right data frame.\n7. Anti Join: Returns only the rows from the left data frame that do not have matching values in the right data frame.1.014.66667Dict(:top_k=>3)
7examples/data/database_style_joins.txtinnerjoin: the output contains rows for values of the key that exist in all passed data frames.What does the output of the inner join operation contain?The output of the inner join operation contains only the rows for values of the key that exist in all passed data frames.1.015.0Dict(:top_k=>3)
8examples/data/database_style_joins.txtleftjoin: the output contains rows for values of the key that exist in the first (left) argument, whether or not that value exists in the second (right) argument.What is the purpose of the left join operation?The purpose of the left join operation is to combine data from two tables based on a common key, where all rows from the left (first) table are included in the output, regardless of whether there is a match in the right (second) table.1.014.66667Dict(:top_k=>3)
9examples/data/database_style_joins.txtrightjoin: the output contains rows for values of the key that exist in the second (right) argument, whether or not that value exists in the first (left) argument.What is the purpose of the right join operation?The purpose of the right join operation is to include all the rows from the second (right) argument, regardless of whether a match is found in the first (left) argument.1.014.67Dict(:top_k=>3)
10examples/data/database_style_joins.txtouterjoin: the output contains rows for values of the key that exist in any of the passed data frames.\nsemijoin: Like an inner join, but output is restricted to columns from the first (left) argument.What is the difference between outer join and semi join?The difference between outer join and semi join is that outer join includes rows for values of the key that exist in any of the passed data frames, whereas semi join is like an inner join but only outputs columns from the first argument.1.014.66667Dict(:top_k=>3)

We're done for today!

What would we do next?

... and much more! See some ideas in Anyscale RAG tutorial

This page was generated using Literate.jl.

diff --git a/dev/examples/readme_examples/index.html b/dev/examples/readme_examples/index.html index b6c259067..7decdd377 100644 --- a/dev/examples/readme_examples/index.html +++ b/dev/examples/readme_examples/index.html @@ -1,5 +1,5 @@ -Various examples · PromptingTools.jl
GitHub

Various Examples

Noteworthy functions: aigenerate, aiembed, aiclassify, aiextract, aitemplates

Seamless Integration Into Your Workflow

Google search is great, but it's a context switch. You often have to open a few pages and read through the discussion to find the answer you need. Same with the ChatGPT website.

Imagine you are in VSCode, editing your .gitignore file. How do I ignore a file in all subfolders again?

All you need to do is to type: aai"What to write in .gitignore to ignore file XYZ in any folder or subfolder?"

With aai"" (as opposed to ai""), we make a non-blocking call to the LLM to not prevent you from continuing your work. When the answer is ready, we log it from the background:

[ Info: Tokens: 102 @ Cost: $0.0002 in 2.7 seconds
+Various examples · PromptingTools.jl
GitHub
Various Examples
ai* Functions Overview
Noteworthy functions: aigenerate, aiembed, aiclassify, aiextract, aiscan, aiimage, aitemplates
All ai* functions have the same basic structure: 
ai*(<optional schema>,<prompt or conversation>; <optional keyword arguments>), 
but they differ in purpose:
  • aigenerate is the general-purpose function to generate any text response with LLMs, ie, it returns AIMessage with field :content containing the generated text (eg, ans.content isa AbstractString)
  • aiembed is designed to extract embeddings from the AI model's response, ie, it returns DataMessage with field :content containing the embeddings (eg, ans.content isa AbstractArray)
  • aiextract is designed to extract structured data from the AI model's response and return them as a Julia struct (eg, if we provide return_type=Food, we get ans.content isa Food). You need to define the return type first and then provide it as a keyword argument.
  • aiclassify is designed to classify the input text into (or simply respond within) a set of discrete choices provided by the user. It can be very useful as an LLM Judge or a router for RAG systems, as it uses the "logit bias trick" and generates exactly 1 token. It returns AIMessage with field :content, but the :content can be only one of the provided choices (eg, ans.content in choices)
  • aiscan is for working with images and vision-enabled models (as an input), but it returns AIMessage with field :content containing the generated text (eg, ans.content isa AbstractString) similar to aigenerate.
  • aiimage is for generating images (eg, with OpenAI DALL-E 3). It returns a DataMessage, where the field :content might contain either the URL to download the image from or the Base64-encoded image depending on the user-provided kwarg api_kwargs.response_format.
  • aitemplates is a helper function to discover available templates and see their details (eg, aitemplates("some keyword") or aitemplates(:AssistantAsk))
If you're using a known model, you do NOT need to provide a schema (the first argument).
Optional keyword arguments in ai* tend to be:
  • model::String - Which model you want to use
  • verbose::Bool - Whether you went to see INFO logs around AI costs
  • return_all::Bool - Whether you want the WHOLE conversation or just the AI answer (ie, whether you want to include your inputs/prompt in the output)
  • api_kwargs::NamedTuple - Specific parameters for the model, eg, temperature=0.0 to be NOT creative (and have more similar output in each run)
  • http_kwargs::NamedTuple - Parameters for the HTTP.jl package, eg, readtimeout = 120 to time out in 120 seconds if no response was received.
In addition to the above list of ai* functions, you can also use the "lazy" counterparts of these functions from the experimental AgentTools module.
using PromptingTools.Experimental.AgentTools
For example, AIGenerate() will create a lazy instance of aigenerate. It is an instance of AICall with aigenerate as its ai function. It uses exactly the same arguments and keyword arguments as aigenerate (see ?aigenerate for details).
"lazy" refers to the fact that it does NOT generate any output when instantiated (only when run! is called). 
Or said differently, the AICall struct and all its flavors (AIGenerate, ...) are designed to facilitate a deferred execution model (lazy evaluation) for AI functions that interact with a Language Learning Model (LLM). It stores the necessary information for an AI call and executes the underlying AI function only when supplied with a UserMessage or when the run! method is applied. This allows us to remember user inputs and trigger the LLM call repeatedly if needed, which enables automatic fixing (see ?airetry!).
Seamless Integration Into Your Workflow
Google search is great, but it's a context switch. You often have to open a few pages and read through the discussion to find the answer you need. Same with the ChatGPT website.
Imagine you are in VSCode, editing your .gitignore file. How do I ignore a file in all subfolders again?
All you need to do is to type: aai"What to write in .gitignore to ignore file XYZ in any folder or subfolder?"
With aai"" (as opposed to ai""), we make a non-blocking call to the LLM to not prevent you from continuing your work. When the answer is ready, we log it from the background:
[ Info: Tokens: 102 @ Cost: $0.0002 in 2.7 seconds
 ┌ Info: AIMessage> To ignore a file called "XYZ" in any folder or subfolder, you can add the following line to your .gitignore file:
 │ 
 │ ```
@@ -121,4 +121,4 @@
 msg = aigenerate(PT.MistralOpenAISchema(), "Say Hi!"; model="mistral-tiny", api_key=ENV["MISTRALAI_API_KEY"])
As you can see, we can load your API key either from the ENV or via the Preferences.jl mechanism (see ?PREFERENCES for more information).
But MistralAI are not the only ones! There are many other exciting providers, eg, Perplexity.ai, Fireworks.ai. As long as they are compatible with the OpenAI API (eg, sending messages with role and content keys), you can use them with PromptingTools.jl by using schema = CustomOpenAISchema():
# Set your API key and the necessary base URL for the API
 api_key = "..."
 prompt = "Say hi!"
-msg = aigenerate(PT.CustomOpenAISchema(), prompt; model="my_model", api_key, api_kwargs=(; url="http://localhost:8081"))
As you can see, it also works for any local models that you might have running on your computer!
Note: At the moment, we only support aigenerate and aiembed functions for MistralAI and other OpenAI-compatible APIs. We plan to extend the support in the future.

+msg = aigenerate(PT.CustomOpenAISchema(), prompt; model="my_model", api_key, api_kwargs=(; url="http://localhost:8081"))

As you can see, it also works for any local models that you might have running on your computer!

Note: At the moment, we only support aigenerate and aiembed functions for MistralAI and other OpenAI-compatible APIs. We plan to extend the support in the future.

diff --git a/dev/examples/working_with_aitemplates/index.html b/dev/examples/working_with_aitemplates/index.html index 43f03ffea..56f8e6f4c 100644 --- a/dev/examples/working_with_aitemplates/index.html +++ b/dev/examples/working_with_aitemplates/index.html @@ -43,4 +43,4 @@ PT.save_template(filename, tpl; description = "For asking data analysis questions in Julia language. Placeholders: `ask`") -rm(filename) # cleanup if we don't like it

When you create a new template, remember to re-load the templates with load_templates!() so that it's available for use.

PT.load_templates!();

!!! If you have some good templates (or suggestions for the existing ones), please consider sharing them with the community by opening a PR to the templates directory!

This page was generated using Literate.jl.

+rm(filename) # cleanup if we don't like it

When you create a new template, remember to re-load the templates with load_templates!() so that it's available for use.

PT.load_templates!();

!!! If you have some good templates (or suggestions for the existing ones), please consider sharing them with the community by opening a PR to the templates directory!

This page was generated using Literate.jl.

diff --git a/dev/examples/working_with_custom_apis/index.html b/dev/examples/working_with_custom_apis/index.html index 669806fa6..7735f595f 100644 --- a/dev/examples/working_with_custom_apis/index.html +++ b/dev/examples/working_with_custom_apis/index.html @@ -35,4 +35,4 @@ prompt = "I just ate a delicious and juicy apple." msg = aiextract(prompt; return_type=Food, model="firefunction") msg.content -# Output: Food("apple", ["delicious", "juicy"])

For embedding a text, use aiembed:

aiembed(PT.FireworksOpenAISchema(), "embed me"; model="nomic-ai/nomic-embed-text-v1.5")

Note: You can register the model with PT.register_model! and use it as usual.

+# Output: Food("apple", ["delicious", "juicy"])

For embedding a text, use aiembed:

aiembed(PT.FireworksOpenAISchema(), "embed me"; model="nomic-ai/nomic-embed-text-v1.5")

Note: You can register the model with PT.register_model! and use it as usual.

diff --git a/dev/examples/working_with_google_ai_studio/index.html b/dev/examples/working_with_google_ai_studio/index.html index 8dfd2a369..943c0c924 100644 --- a/dev/examples/working_with_google_ai_studio/index.html +++ b/dev/examples/working_with_google_ai_studio/index.html @@ -9,4 +9,4 @@ The Force flows through all living things, not machines. Seek balance in the Force, and your heart will find true connection. -Remember, the path of the Jedi is to serve others, not to be attached to possessions.")

Gotchas

+Remember, the path of the Jedi is to serve others, not to be attached to possessions.")

Gotchas

diff --git a/dev/examples/working_with_ollama/index.html b/dev/examples/working_with_ollama/index.html index e41769d2d..1452bd62c 100644 --- a/dev/examples/working_with_ollama/index.html +++ b/dev/examples/working_with_ollama/index.html @@ -38,4 +38,4 @@ LinearAlgebra.normalize; model = "openhermes2.5-mistral")
PromptingTools.DataMessage(Matrix{Float64} of size (4096, 2))

Cosine similarity is then a simple multiplication

msg.content' * msg.content[:, 1]
2-element Vector{Float64}:
  0.9999999999999982
- 0.40796033843072876

This page was generated using Literate.jl.

+ 0.40796033843072876

This page was generated using Literate.jl.

diff --git a/dev/frequently_asked_questions/index.html b/dev/frequently_asked_questions/index.html index f8e2f9cbf..767249d77 100644 --- a/dev/frequently_asked_questions/index.html +++ b/dev/frequently_asked_questions/index.html @@ -80,4 +80,4 @@ ## system_preview: String "You must speak like a pirate" ## user_preview: String "Say hi to {{name}}" ## source: String ""

So you can use it like any other template:

aigenerate(:GreatingPirate; name="Jack Sparrow")
-# Output: AIMessage("Arr, me hearty! Best be sending me regards to Captain Jack Sparrow on the salty seas! May his compass always point true to the nearest treasure trove. Yarrr!")

If you want to save it in your project folder:

PT.save_template("templates/GreatingPirate.json", tpl; version="1.0") # optionally, add description

It will be saved and accessed under its basename, ie, GreatingPirate (same as load_as keyword argument).

Note: If you make any changes to the templates on the disk/in a folder, you need to explicitly reload all templates again!

If you are using the main PromptingTools templates, you can simply call PT.load_templates!(). If you have a project folder with your templates, you want to add it first:

PT.load_templates!("templates")

After the first run, we will remember the folder and you can simply call PT.load_templates!() to reload all the templates in the future!

+# Output: AIMessage("Arr, me hearty! Best be sending me regards to Captain Jack Sparrow on the salty seas! May his compass always point true to the nearest treasure trove. Yarrr!")

If you want to save it in your project folder:

PT.save_template("templates/GreatingPirate.json", tpl; version="1.0") # optionally, add description

It will be saved and accessed under its basename, ie, GreatingPirate (same as load_as keyword argument).

Note: If you make any changes to the templates on the disk/in a folder, you need to explicitly reload all templates again!

If you are using the main PromptingTools templates, you can simply call PT.load_templates!(). If you have a project folder with your templates, you want to add it first:

PT.load_templates!("templates")

After the first run, we will remember the folder and you can simply call PT.load_templates!() to reload all the templates in the future!

diff --git a/dev/getting_started/index.html b/dev/getting_started/index.html index 05531c09b..20b29ad59 100644 --- a/dev/getting_started/index.html +++ b/dev/getting_started/index.html @@ -4,4 +4,4 @@ AIMessage("The capital of France is Paris.")

Returned object is a light wrapper with generated message in field :content (eg, ans.content) for additional downstream processing.

If you want to reply to the previous message, or simply continue the conversation, use @ai!_str (notice the bang !):

ai!"And what is the population of it?"

You can easily inject any variables with string interpolation:

country = "Spain"
 ai"What is the capital of \$(country)?"
[ Info: Tokens: 32 @ Cost: $0.0001 in 0.5 seconds
 AIMessage("The capital of Spain is Madrid.")

Pro tip: Use after-string-flags to select the model to be called, eg, ai"What is the capital of France?"gpt4 (use gpt4t for the new GPT-4 Turbo model). Great for those extra hard questions!

Using aigenerate with placeholders

For more complex prompt templates, you can use handlebars-style templating and provide variables as keyword arguments:

msg = aigenerate("What is the capital of {{country}}? Is the population larger than {{population}}?", country="Spain", population="1M")
[ Info: Tokens: 74 @ Cost: $0.0001 in 1.3 seconds
-AIMessage("The capital of Spain is Madrid. And yes, the population of Madrid is larger than 1 million. As of 2020, the estimated population of Madrid is around 3.3 million people.")

Pro tip: Use asyncmap to run multiple AI-powered tasks concurrently.

Pro tip: If you use slow models (like GPT-4), you can use the asynchronous version of @ai_str -> @aai_str to avoid blocking the REPL, eg, aai"Say hi but slowly!"gpt4 (similarly @ai!_str -> @aai!_str for multi-turn conversations).

For more practical examples, see the Various Examples section.

+AIMessage("The capital of Spain is Madrid. And yes, the population of Madrid is larger than 1 million. As of 2020, the estimated population of Madrid is around 3.3 million people.")

Pro tip: Use asyncmap to run multiple AI-powered tasks concurrently.

Pro tip: If you use slow models (like GPT-4), you can use the asynchronous version of @ai_str -> @aai_str to avoid blocking the REPL, eg, aai"Say hi but slowly!"gpt4 (similarly @ai!_str -> @aai!_str for multi-turn conversations).

For more practical examples, see the Various Examples section.

diff --git a/dev/how_it_works/index.html b/dev/how_it_works/index.html index a5f714e86..dc679ce1d 100644 --- a/dev/how_it_works/index.html +++ b/dev/how_it_works/index.html @@ -1,11 +1,11 @@ -How It Works · PromptingTools.jl
GitHub

How It Works

This is an advanced section that explains how PromptingTools.jl works under the hood. It is not necessary to understand this to use the package, but it can be helpful for debugging and understanding the limitations of the package.

We'll start with the key concepts and then walk through an example of aigenerate to see how it all fits together.

Key Concepts

5 Key Concepts (/Objects):

  • API/Model Providers -> The method that gives you access to Large Language Models (LLM), it can be an API (eg, OpenAI) or a locally-hosted application (eg, Llama.cpp or Ollama)
  • Schemas -> object of type AbstractPromptSchema that determines which methods are called and, hence, what providers/APIs are used
  • Prompts -> the information you want to convey to the AI model
  • Messages -> the basic unit of communication between the user and the AI model (eg, UserMessage vs AIMessage)
  • Prompt Templates -> re-usable "prompts" with placeholders that you can replace with your inputs at the time of making the request

When you call aigenerate, roughly the following happens: render -> UserMessage(s) -> render -> OpenAI.create_chat -> ... -> AIMessage.

API/Model Providers

You can think of "API/Model Providers" as the method that gives you access to Large Language Models (LLM). It can be an API (eg, OpenAI) or a locally-hosted application (eg, Llama.cpp or Ollama).

You interact with them via the schema object, which is a subtype of AbstractPromptSchema, eg, there is an OpenAISchema for the provider "OpenAI" and its supertype AbstractOpenAISchema is for all other providers that mimic the OpenAI API.

Schemas

For your "message" to reach an AI model, it needs to be formatted and sent to the right place (-> provider!).

We leverage the multiple dispatch around the "schemas" to pick the right logic. All schemas are subtypes of AbstractPromptSchema and there are many subtypes, eg, OpenAISchema <: AbstractOpenAISchema <:AbstractPromptSchema.

For example, if you provide schema = OpenAISchema(), the system knows that:

  • it will have to format any user inputs to OpenAI's "message specification" (a vector of dictionaries, see their API documentation). Function render(OpenAISchema(),...) will take care of the rendering.
  • it will have to send the message to OpenAI's API. We will use the amazing OpenAI.jl package to handle the communication.

Prompts

Prompt is loosely the information you want to convey to the AI model. It can be a question, a statement, or a command. It can have instructions or some context, eg, previous conversation.

You need to remember that Large Language Models (LLMs) are stateless. They don't remember the previous conversation/request, so you need to provide the whole history/context every time (similar to how REST APIs work).

Prompts that we send to the LLMs are effectively a sequence of messages (<:AbstractMessage).

Messages

Messages are the basic unit of communication between the user and the AI model.

There are 5 main types of messages (<:AbstractMessage):

  • SystemMessage - this contains information about the "system", eg, how it should behave, format its output, etc. (eg, `You're a world-class Julia programmer. You write brief and concise code.)
  • UserMessage - the information "from the user", ie, your question/statement/task
  • UserMessageWithImages - the same as UserMessage, but with images (URLs or Base64-encoded images)
  • AIMessage - the response from the AI model, when the "output" is text
  • DataMessage - the response from the AI model, when the "output" is data, eg, embeddings with aiembed or user-defined structs with aiextract

Prompt Templates

We want to have re-usable "prompts", so we provide you with a system to retrieve pre-defined prompts with placeholders (eg, {{name}}) that you can replace with your inputs at the time of making the request.

"AI Templates" as we call them (AITemplate) are usually a vector of SystemMessage and a UserMessage with specific purpose/task.

For example, the template :AssistantAsk is defined loosely as:

 template = [SystemMessage("You are a world-class AI assistant. Your communication is brief and concise. You're precise and answer only when you're confident in the high quality of your answer."),
-             UserMessage("# Question\n\n{{ask}}")]

Notice that we have a placeholder ask ({{ask}}) that you can replace with your question without having to re-write the generic system instructions.

When you provide a Symbol (eg, :AssistantAsk) to ai* functions, thanks to the multiple dispatch, it recognizes that it's an AITemplate(:AssistantAsk) and looks it up.

You can discover all available templates with aitemplates("some keyword") or just see the details of some template aitemplates(:AssistantAsk).

Note: There is a new way to create and register templates in one go with create_template(;user=<user prompt>, system=<system prompt>, load_as=<template name>) (it skips the serialization step where a template previously must have been saved somewhere on the disk). See FAQ for more details or directly ?create_template.

ai* Functions

The above steps are implemented in the ai* functions, eg, aigenerate, aiembed, aiextract, etc. They all have the same basic structure:

ai*(<optional schema>,<prompt or conversation>; <optional keyword arguments>),

but they differ in purpose:

  • aigenerate is the general-purpose function to generate any text response with LLMs, ie, it returns AIMessage with field :content containing the generated text (eg, ans.content isa AbstractString)
  • aiembed is designed to extract embeddings from the AI model's response, ie, it returns DataMessage with field :content containing the embeddings (eg, ans.content isa AbstractArray)
  • aiextract is designed to extract structured data from the AI model's response and return them as a Julia struct (eg, if we provide return_type=Food, we get ans.content isa Food). You need to define the return type first and then provide it as a keyword argument.
  • aiclassify is designed to classify the input text into (or simply respond within) a set of discrete choices provided by the user. It can be very useful as an LLM Judge or a router for RAG systems, as it uses the "logit bias trick" and generates exactly 1 token. It returns AIMessage with field :content, but the :content can be only one of the provided choices (eg, ans.content in choices)
  • aiscan is for working with images and vision-enabled models (as an input), but it returns AIMessage with field :content containing the generated text (eg, ans.content isa AbstractString) similar to aigenerate.
  • aitemplates is a helper function to discover available templates and see their details (eg, aitemplates("some keyword") or aitemplates(:AssistantAsk))

In addition to the above list, you can also use the "lazy" counterparts of these functions from the experimental AgentTools module.

using PromptingTools.Experimental.AgentTools

For example, AIGenerate() will create a lazy instance of aigenerate. It is an instance of AICall with aigenerate as its ai function. It uses exactly the same arguments and keyword arguments as aigenerate (see ?aigenerate for details).

"lazy" refers to the fact that it does NOT generate any output when instantiated (only when run! is called).

Or said differently, the AICall struct and all its flavors (AIGenerate, ...) are designed to facilitate a deferred execution model (lazy evaluation) for AI functions that interact with a Language Learning Model (LLM). It stores the necessary information for an AI call and executes the underlying AI function only when supplied with a UserMessage or when the run! method is applied.

This approach allows us to remember user inputs and trigger the LLM call repeatedly if needed, which enables automatic fixing (see ?airetry!).

Example:

result = AIGenerate(:JuliaExpertAsk; ask="xyz", model="abc", api_kwargs=(; temperature=0.1))
+How It Works · PromptingTools.jl
GitHub
How It Works
This is an advanced section that explains how PromptingTools.jl works under the hood. It is not necessary to understand this to use the package, but it can be helpful for debugging and understanding the limitations of the package.
We'll start with the key concepts and then walk through an example of aigenerate to see how it all fits together.
Key Concepts
5 Key Concepts (/Objects):
  • API/Model Providers -> The method that gives you access to Large Language Models (LLM), it can be an API (eg, OpenAI) or a locally-hosted application (eg, Llama.cpp or Ollama)
  • Schemas -> object of type AbstractPromptSchema that determines which methods are called and, hence, what providers/APIs are used
  • Prompts -> the information you want to convey to the AI model
  • Messages -> the basic unit of communication between the user and the AI model (eg, UserMessage vs AIMessage)
  • Prompt Templates -> re-usable "prompts" with placeholders that you can replace with your inputs at the time of making the request
When you call aigenerate, roughly the following happens: render -> UserMessage(s) -> render -> OpenAI.create_chat -> ... -> AIMessage.
API/Model Providers
You can think of "API/Model Providers" as the method that gives you access to Large Language Models (LLM). It can be an API (eg, OpenAI) or a locally-hosted application (eg, Llama.cpp or Ollama).
You interact with them via the schema object, which is a subtype of AbstractPromptSchema, eg, there is an OpenAISchema for the provider "OpenAI" and its supertype AbstractOpenAISchema is for all other providers that mimic the OpenAI API.
Schemas
For your "message" to reach an AI model, it needs to be formatted and sent to the right place (-> provider!).
We leverage the multiple dispatch around the "schemas" to pick the right logic. All schemas are subtypes of AbstractPromptSchema and there are many subtypes, eg, OpenAISchema <: AbstractOpenAISchema <:AbstractPromptSchema.
For example, if you provide schema = OpenAISchema(), the system knows that:
  • it will have to format any user inputs to OpenAI's "message specification" (a vector of dictionaries, see their API documentation). Function render(OpenAISchema(),...) will take care of the rendering.
  • it will have to send the message to OpenAI's API. We will use the amazing OpenAI.jl package to handle the communication.
Prompts
Prompt is loosely the information you want to convey to the AI model. It can be a question, a statement, or a command. It can have instructions or some context, eg, previous conversation.
You need to remember that Large Language Models (LLMs) are stateless. They don't remember the previous conversation/request, so you need to provide the whole history/context every time (similar to how REST APIs work).
Prompts that we send to the LLMs are effectively a sequence of messages (<:AbstractMessage).
Messages
Messages are the basic unit of communication between the user and the AI model. 
There are 5 main types of messages (<:AbstractMessage):
  • SystemMessage - this contains information about the "system", eg, how it should behave, format its output, etc. (eg, `You're a world-class Julia programmer. You write brief and concise code.)
  • UserMessage - the information "from the user", ie, your question/statement/task
  • UserMessageWithImages - the same as UserMessage, but with images (URLs or Base64-encoded images)
  • AIMessage - the response from the AI model, when the "output" is text
  • DataMessage - the response from the AI model, when the "output" is data, eg, embeddings with aiembed or user-defined structs with aiextract
Prompt Templates
We want to have re-usable "prompts", so we provide you with a system to retrieve pre-defined prompts with placeholders (eg, {{name}}) that you can replace with your inputs at the time of making the request.
"AI Templates" as we call them (AITemplate) are usually a vector of SystemMessage and a UserMessage with specific purpose/task.
For example, the template :AssistantAsk is defined loosely as:
 template = [SystemMessage("You are a world-class AI assistant. Your communication is brief and concise. You're precise and answer only when you're confident in the high quality of your answer."),
+             UserMessage("# Question\n\n{{ask}}")]
Notice that we have a placeholder ask ({{ask}}) that you can replace with your question without having to re-write the generic system instructions.
When you provide a Symbol (eg, :AssistantAsk) to ai* functions, thanks to the multiple dispatch, it recognizes that it's an AITemplate(:AssistantAsk) and looks it up.
You can discover all available templates with aitemplates("some keyword") or just see the details of some template aitemplates(:AssistantAsk).
Note: There is a new way to create and register templates in one go with create_template(;user=<user prompt>, system=<system prompt>, load_as=<template name>) (it skips the serialization step where a template previously must have been saved somewhere on the disk). See FAQ for more details or directly ?create_template.
ai* Functions Overview
The above steps are implemented in the ai* functions, eg, aigenerate, aiembed, aiextract, etc. They all have the same basic structure: 
ai*(<optional schema>,<prompt or conversation>; <optional keyword arguments>), 
but they differ in purpose:
  • aigenerate is the general-purpose function to generate any text response with LLMs, ie, it returns AIMessage with field :content containing the generated text (eg, ans.content isa AbstractString)
  • aiembed is designed to extract embeddings from the AI model's response, ie, it returns DataMessage with field :content containing the embeddings (eg, ans.content isa AbstractArray)
  • aiextract is designed to extract structured data from the AI model's response and return them as a Julia struct (eg, if we provide return_type=Food, we get ans.content isa Food). You need to define the return type first and then provide it as a keyword argument.
  • aiclassify is designed to classify the input text into (or simply respond within) a set of discrete choices provided by the user. It can be very useful as an LLM Judge or a router for RAG systems, as it uses the "logit bias trick" and generates exactly 1 token. It returns AIMessage with field :content, but the :content can be only one of the provided choices (eg, ans.content in choices)
  • aiscan is for working with images and vision-enabled models (as an input), but it returns AIMessage with field :content containing the generated text (eg, ans.content isa AbstractString) similar to aigenerate.
  • aiimage is for generating images (eg, with OpenAI DALL-E 3). It returns a DataMessage, where the field :content might contain either the URL to download the image from or the Base64-encoded image depending on the user-provided kwarg api_kwargs.response_format.
  • aitemplates is a helper function to discover available templates and see their details (eg, aitemplates("some keyword") or aitemplates(:AssistantAsk))
If you're using a known model, you do NOT need to provide a schema (the first argument).
Optional keyword arguments in ai* tend to be:
  • model::String - Which model you want to use
  • verbose::Bool - Whether you went to see INFO logs around AI costs
  • return_all::Bool - Whether you want the WHOLE conversation or just the AI answer (ie, whether you want to include your inputs/prompt in the output)
  • api_kwargs::NamedTuple - Specific parameters for the model, eg, temperature=0.0 to be NOT creative (and have more similar output in each run)
  • http_kwargs::NamedTuple - Parameters for the HTTP.jl package, eg, readtimeout = 120 to time out in 120 seconds if no response was received.
In addition to the above list of ai* functions, you can also use the "lazy" counterparts of these functions from the experimental AgentTools module.
using PromptingTools.Experimental.AgentTools
For example, AIGenerate() will create a lazy instance of aigenerate. It is an instance of AICall with aigenerate as its ai function. It uses exactly the same arguments and keyword arguments as aigenerate (see ?aigenerate for details).
"lazy" refers to the fact that it does NOT generate any output when instantiated (only when run! is called). 
Or said differently, the AICall struct and all its flavors (AIGenerate, ...) are designed to facilitate a deferred execution model (lazy evaluation) for AI functions that interact with a Language Learning Model (LLM). It stores the necessary information for an AI call and executes the underlying AI function only when supplied with a UserMessage or when the run! method is applied. 
This approach allows us to remember user inputs and trigger the LLM call repeatedly if needed, which enables automatic fixing (see ?airetry!).
Example:
result = AIGenerate(:JuliaExpertAsk; ask="xyz", model="abc", api_kwargs=(; temperature=0.1))
 result |> run!
 
 # Is equivalent to
 result = aigenerate(:JuliaExpertAsk; ask="xyz", model="abc", api_kwargs=(; temperature=0.1), return_all=true)
-# The only difference is that we default to `return_all=true` with lazy types because we have a dedicated `conversation` field, which makes it much easier
Lazy AI calls and self-healing mechanisms unlock much more robust and useful LLM workflows!
Walkthroughs
Walkthrough Example for aigenerate
using PromptingTools
+# The only difference is that we default to `return_all=true` with lazy types because we have a dedicated `conversation` field, which makes it much easier
Lazy AI calls and self-healing mechanisms unlock much more robust and useful LLM workflows!
Walkthrough Example for aigenerate
using PromptingTools
 const PT = PromptingTools
 
 # Let's say this is our ask
@@ -94,4 +94,4 @@
 end
 food = JSON3.read(last_output(result), Food)
 ## [ Info: Condition not met. Retrying...
-## Output: Food("apple", ["delicious", "juicy"])
It took 1 retry (see result.config.retries) and we have the correct output from an open-source model!
If you're interested in the result object, it's a struct (AICall) with a field conversation, which holds the conversation up to this point. AIGenerate is an alias for AICall using aigenerate function. See ?AICall (the underlying struct type) for more details on the fields and methods available.

+## Output: Food("apple", ["delicious", "juicy"])

It took 1 retry (see result.config.retries) and we have the correct output from an open-source model!

If you're interested in the result object, it's a struct (AICall) with a field conversation, which holds the conversation up to this point. AIGenerate is an alias for AICall using aigenerate function. See ?AICall (the underlying struct type) for more details on the fields and methods available.

diff --git a/dev/index.html b/dev/index.html index 034b560a1..fbf48d83e 100644 --- a/dev/index.html +++ b/dev/index.html @@ -1,2 +1,2 @@ -Home · PromptingTools.jl
GitHub

PromptingTools

Documentation for PromptingTools.

Streamline your life using PromptingTools.jl, the Julia package that simplifies interacting with large language models.

PromptingTools.jl is not meant for building large-scale systems. It's meant to be the go-to tool in your global environment that will save you 20 minutes every day!

Why PromptingTools.jl?

Prompt engineering is neither fast nor easy. Moreover, different models and their fine-tunes might require different prompt formats and tricks, or perhaps the information you work with requires special models to be used. PromptingTools.jl is meant to unify the prompts for different backends and make the common tasks (like templated prompts) as simple as possible.

Some features:

  • aigenerate Function: Simplify prompt templates with handlebars (eg, {{variable}}) and keyword arguments
  • @ai_str String Macro: Save keystrokes with a string macro for simple prompts
  • Easy to Remember: All exported functions start with ai... for better discoverability
  • Light Wrapper Types: Benefit from Julia's multiple dispatch by having AI outputs wrapped in specific types
  • Minimal Dependencies: Enjoy an easy addition to your global environment with very light dependencies
  • No Context Switching: Access cutting-edge LLMs with no context switching and minimum extra keystrokes directly in your REPL

First Steps

To get started, see the Getting Started section.

+Home · PromptingTools.jl
GitHub

PromptingTools

Documentation for PromptingTools.

Streamline your life using PromptingTools.jl, the Julia package that simplifies interacting with large language models.

PromptingTools.jl is not meant for building large-scale systems. It's meant to be the go-to tool in your global environment that will save you 20 minutes every day!

Why PromptingTools.jl?

Prompt engineering is neither fast nor easy. Moreover, different models and their fine-tunes might require different prompt formats and tricks, or perhaps the information you work with requires special models to be used. PromptingTools.jl is meant to unify the prompts for different backends and make the common tasks (like templated prompts) as simple as possible.

Some features:

  • aigenerate Function: Simplify prompt templates with handlebars (eg, {{variable}}) and keyword arguments
  • @ai_str String Macro: Save keystrokes with a string macro for simple prompts
  • Easy to Remember: All exported functions start with ai... for better discoverability
  • Light Wrapper Types: Benefit from Julia's multiple dispatch by having AI outputs wrapped in specific types
  • Minimal Dependencies: Enjoy an easy addition to your global environment with very light dependencies
  • No Context Switching: Access cutting-edge LLMs with no context switching and minimum extra keystrokes directly in your REPL

First Steps

To get started, see the Getting Started section.

diff --git a/dev/reference/index.html b/dev/reference/index.html index 5b8f5c1dd..1995d3015 100644 --- a/dev/reference/index.html +++ b/dev/reference/index.html @@ -1,10 +1,10 @@ -PromptingTools.jl · PromptingTools.jl
GitHub

Reference

PromptingTools.CONV_HISTORY — Constant
CONV_HISTORY

Tracks the most recent conversations through the ai_str macros.

Preference available: MAXHISTORYLENGTH, which sets how many last messages should be remembered.

See also: push_conversation!, resize_conversation!

source
PromptingTools.MODEL_ALIASES — Constant
MODEL_ALIASES

A dictionary of model aliases. Aliases are used to refer to models by their aliases instead of their full names to make it more convenient to use them.

Accessing the aliases

PromptingTools.MODEL_ALIASES["gpt3"]

Register a new model alias

PromptingTools.MODEL_ALIASES["gpt3"] = "gpt-3.5-turbo"
source
PromptingTools.MODEL_REGISTRY — Constant
MODEL_REGISTRY

A store of available model names and their specs (ie, name, costs per token, etc.)

Accessing the registry

You can use both the alias name or the full name to access the model spec:

PromptingTools.MODEL_REGISTRY["gpt-3.5-turbo"]

Registering a new model

register_model!(
+PromptingTools.jl · PromptingTools.jl
GitHub
Reference
PromptingTools.CONV_HISTORY — Constant
CONV_HISTORY
Tracks the most recent conversations through the ai_str macros.
Preference available: MAXHISTORYLENGTH, which sets how many last messages should be remembered.
See also: push_conversation!, resize_conversation!
source
PromptingTools.MODEL_ALIASES — Constant
MODEL_ALIASES
A dictionary of model aliases. Aliases are used to refer to models by their aliases instead of their full names to make it more convenient to use them.
Accessing the aliases
PromptingTools.MODEL_ALIASES["gpt3"]
Register a new model alias
PromptingTools.MODEL_ALIASES["gpt3"] = "gpt-3.5-turbo"
source
PromptingTools.MODEL_REGISTRY — Constant
MODEL_REGISTRY
A store of available model names and their specs (ie, name, costs per token, etc.)
Accessing the registry
You can use both the alias name or the full name to access the model spec:
PromptingTools.MODEL_REGISTRY["gpt-3.5-turbo"]
Registering a new model
register_model!(
     name = "gpt-3.5-turbo",
     schema = :OpenAISchema,
     cost_of_token_prompt = 0.0015,
     cost_of_token_generation = 0.002,
-    description = "GPT-3.5 Turbo is a 175B parameter model and a common default on the OpenAI API.")
Registering a model alias
PromptingTools.MODEL_ALIASES["gpt3"] = "gpt-3.5-turbo"
source
PromptingTools.PREFERENCES — Constant
PREFERENCES
You can set preferences for PromptingTools by setting environment variables (for OPENAI_API_KEY only)      or by using the set_preferences!.     It will create a LocalPreferences.toml file in your current directory and will reload your prefences from there.
Check your preferences by calling get_preferences(key::String).
Available Preferences (for set_preferences!)
  • OPENAI_API_KEY: The API key for the OpenAI API. See OpenAI's documentation for more information.
  • MISTRALAI_API_KEY: The API key for the Mistral AI API. See Mistral AI's documentation for more information.
  • COHERE_API_KEY: The API key for the Cohere API. See Cohere's documentation for more information.
  • DATABRICKS_API_KEY: The API key for the Databricks Foundation Model API. See Databricks' documentation for more information.
  • DATABRICKS_HOST: The host for the Databricks API. See Databricks' documentation for more information.
  • TAVILY_API_KEY: The API key for the Tavily Search API. Register here. See more information here.
  • GOOGLE_API_KEY: The API key for Google Gemini models. Get yours from here. If you see a documentation page ("Available languages and regions for Google AI Studio and Gemini API"), it means that it's not yet available in your region.
  • MODEL_CHAT: The default model to use for aigenerate and most ai* calls. See MODEL_REGISTRY for a list of available models or define your own.
  • MODEL_EMBEDDING: The default model to use for aiembed (embedding documents). See MODEL_REGISTRY for a list of available models or define your own.
  • PROMPT_SCHEMA: The default prompt schema to use for aigenerate and most ai* calls (if not specified in MODEL_REGISTRY). Set as a string, eg, "OpenAISchema".   See PROMPT_SCHEMA for more information.
  • MODEL_ALIASES: A dictionary of model aliases (alias => full_model_name). Aliases are used to refer to models by their aliases instead of their full names to make it more convenient to use them.   See MODEL_ALIASES for more information.
  • MAX_HISTORY_LENGTH: The maximum length of the conversation history. Defaults to 5. Set to nothing to disable history.   See CONV_HISTORY for more information.
  • LOCAL_SERVER: The URL of the local server to use for ai* calls. Defaults to http://localhost:10897/v1. This server is called when you call model="local"   See ?LocalServerOpenAISchema for more information and examples.
At the moment it is not possible to persist changes to MODEL_REGISTRY across sessions.  Define your register_model!() calls in your startup.jl file to make them available across sessions or put them at the top of your script.
Available ENV Variables
  • OPENAI_API_KEY: The API key for the OpenAI API. 
  • MISTRALAI_API_KEY: The API key for the Mistral AI API.
  • COHERE_API_KEY: The API key for the Cohere API.
  • LOCAL_SERVER: The URL of the local server to use for ai* calls. Defaults to http://localhost:10897/v1. This server is called when you call model="local"
  • DATABRICKS_API_KEY: The API key for the Databricks Foundation Model API.
  • DATABRICKS_HOST: The host for the Databricks API.
  • TAVILY_API_KEY: The API key for the Tavily Search API. Register here. See more information here.
  • GOOGLE_API_KEY: The API key for Google Gemini models. Get yours from here. If you see a documentation page ("Available languages and regions for Google AI Studio and Gemini API"), it means that it's not yet available in your region.
Preferences.jl takes priority over ENV variables, so if you set a preference, it will take precedence over the ENV variable.
WARNING: NEVER EVER sync your LocalPreferences.toml file! It contains your API key and other sensitive information!!!
source
PromptingTools.AICode — Type
AICode(code::AbstractString; auto_eval::Bool=true, safe_eval::Bool=false, 
+    description = "GPT-3.5 Turbo is a 175B parameter model and a common default on the OpenAI API.")
Registering a model alias
PromptingTools.MODEL_ALIASES["gpt3"] = "gpt-3.5-turbo"
source
PromptingTools.PREFERENCES — Constant
PREFERENCES
You can set preferences for PromptingTools by setting environment variables (for OPENAI_API_KEY only)      or by using the set_preferences!.     It will create a LocalPreferences.toml file in your current directory and will reload your prefences from there.
Check your preferences by calling get_preferences(key::String).
Available Preferences (for set_preferences!)
  • OPENAI_API_KEY: The API key for the OpenAI API. See OpenAI's documentation for more information.
  • MISTRALAI_API_KEY: The API key for the Mistral AI API. See Mistral AI's documentation for more information.
  • COHERE_API_KEY: The API key for the Cohere API. See Cohere's documentation for more information.
  • DATABRICKS_API_KEY: The API key for the Databricks Foundation Model API. See Databricks' documentation for more information.
  • DATABRICKS_HOST: The host for the Databricks API. See Databricks' documentation for more information.
  • TAVILY_API_KEY: The API key for the Tavily Search API. Register here. See more information here.
  • GOOGLE_API_KEY: The API key for Google Gemini models. Get yours from here. If you see a documentation page ("Available languages and regions for Google AI Studio and Gemini API"), it means that it's not yet available in your region.
  • MODEL_CHAT: The default model to use for aigenerate and most ai* calls. See MODEL_REGISTRY for a list of available models or define your own.
  • MODEL_EMBEDDING: The default model to use for aiembed (embedding documents). See MODEL_REGISTRY for a list of available models or define your own.
  • PROMPT_SCHEMA: The default prompt schema to use for aigenerate and most ai* calls (if not specified in MODEL_REGISTRY). Set as a string, eg, "OpenAISchema".   See PROMPT_SCHEMA for more information.
  • MODEL_ALIASES: A dictionary of model aliases (alias => full_model_name). Aliases are used to refer to models by their aliases instead of their full names to make it more convenient to use them.   See MODEL_ALIASES for more information.
  • MAX_HISTORY_LENGTH: The maximum length of the conversation history. Defaults to 5. Set to nothing to disable history.   See CONV_HISTORY for more information.
  • LOCAL_SERVER: The URL of the local server to use for ai* calls. Defaults to http://localhost:10897/v1. This server is called when you call model="local"   See ?LocalServerOpenAISchema for more information and examples.
At the moment it is not possible to persist changes to MODEL_REGISTRY across sessions.  Define your register_model!() calls in your startup.jl file to make them available across sessions or put them at the top of your script.
Available ENV Variables
  • OPENAI_API_KEY: The API key for the OpenAI API. 
  • MISTRALAI_API_KEY: The API key for the Mistral AI API.
  • COHERE_API_KEY: The API key for the Cohere API.
  • LOCAL_SERVER: The URL of the local server to use for ai* calls. Defaults to http://localhost:10897/v1. This server is called when you call model="local"
  • DATABRICKS_API_KEY: The API key for the Databricks Foundation Model API.
  • DATABRICKS_HOST: The host for the Databricks API.
  • TAVILY_API_KEY: The API key for the Tavily Search API. Register here. See more information here.
  • GOOGLE_API_KEY: The API key for Google Gemini models. Get yours from here. If you see a documentation page ("Available languages and regions for Google AI Studio and Gemini API"), it means that it's not yet available in your region.
Preferences.jl takes priority over ENV variables, so if you set a preference, it will take precedence over the ENV variable.
WARNING: NEVER EVER sync your LocalPreferences.toml file! It contains your API key and other sensitive information!!!
source
PromptingTools.AICode — Type
AICode(code::AbstractString; auto_eval::Bool=true, safe_eval::Bool=false, 
 skip_unsafe::Bool=false, capture_stdout::Bool=true, verbose::Bool=false,
 prefix::AbstractString="", suffix::AbstractString="", remove_tests::Bool=false, execution_timeout::Int = 60)
 
@@ -29,7 +29,7 @@
 code.code |> clipboard
 
 # or execute it in the current module (=Main)
-eval(code.expression)
source
PromptingTools.AIMessage — Type
AIMessage
A message type for AI-generated text-based responses.  Returned by aigenerate, aiclassify, and aiscan functions.
Fields
  • content::Union{AbstractString, Nothing}: The content of the message.
  • status::Union{Int, Nothing}: The status of the message from the API.
  • tokens::Tuple{Int, Int}: The number of tokens used (prompt,completion).
  • elapsed::Float64: The time taken to generate the response in seconds.
  • cost::Union{Nothing, Float64}: The cost of the API call (calculated with information from MODEL_REGISTRY).
  • log_prob::Union{Nothing, Float64}: The log probability of the response.
  • finish_reason::Union{Nothing, String}: The reason the response was finished.
  • run_id::Union{Nothing, Int}: The unique ID of the run.
  • sample_id::Union{Nothing, Int}: The unique ID of the sample (if multiple samples are generated, they will all have the same run_id).
source
PromptingTools.AITemplate — Type
AITemplate
AITemplate is a template for a conversation prompt.   This type is merely a container for the template name, which is resolved into a set of messages (=prompt) by render.
Naming Convention
  • Template names should be in CamelCase
  • Follow the format <Persona>...<Variable>... where possible, eg, JudgeIsItTrue, ``
    • Starting with the Persona (=System prompt), eg, Judge = persona is meant to judge some provided information
    • Variable to be filled in with context, eg, It = placeholder it
    • Ending with the variable name is helpful, eg, JuliaExpertTask for a persona to be an expert in Julia language and task is the placeholder name
  • Ideally, the template name should be self-explanatory, eg, JudgeIsItTrue = persona is meant to judge some provided information where it is true or false
Examples
Save time by re-using pre-made templates, just fill in the placeholders with the keyword arguments:
msg = aigenerate(:JuliaExpertAsk; ask = "How do I add packages?")
The above is equivalent to a more verbose version that explicitly uses the dispatch on AITemplate:
msg = aigenerate(AITemplate(:JuliaExpertAsk); ask = "How do I add packages?")
Find available templates with aitemplates:
tmps = aitemplates("JuliaExpertAsk")
+eval(code.expression)
source
PromptingTools.AIMessage — Type
AIMessage
A message type for AI-generated text-based responses.  Returned by aigenerate, aiclassify, and aiscan functions.
Fields
  • content::Union{AbstractString, Nothing}: The content of the message.
  • status::Union{Int, Nothing}: The status of the message from the API.
  • tokens::Tuple{Int, Int}: The number of tokens used (prompt,completion).
  • elapsed::Float64: The time taken to generate the response in seconds.
  • cost::Union{Nothing, Float64}: The cost of the API call (calculated with information from MODEL_REGISTRY).
  • log_prob::Union{Nothing, Float64}: The log probability of the response.
  • finish_reason::Union{Nothing, String}: The reason the response was finished.
  • run_id::Union{Nothing, Int}: The unique ID of the run.
  • sample_id::Union{Nothing, Int}: The unique ID of the sample (if multiple samples are generated, they will all have the same run_id).
source
PromptingTools.AITemplate — Type
AITemplate
AITemplate is a template for a conversation prompt.   This type is merely a container for the template name, which is resolved into a set of messages (=prompt) by render.
Naming Convention
  • Template names should be in CamelCase
  • Follow the format <Persona>...<Variable>... where possible, eg, JudgeIsItTrue, ``
    • Starting with the Persona (=System prompt), eg, Judge = persona is meant to judge some provided information
    • Variable to be filled in with context, eg, It = placeholder it
    • Ending with the variable name is helpful, eg, JuliaExpertTask for a persona to be an expert in Julia language and task is the placeholder name
  • Ideally, the template name should be self-explanatory, eg, JudgeIsItTrue = persona is meant to judge some provided information where it is true or false
Examples
Save time by re-using pre-made templates, just fill in the placeholders with the keyword arguments:
msg = aigenerate(:JuliaExpertAsk; ask = "How do I add packages?")
The above is equivalent to a more verbose version that explicitly uses the dispatch on AITemplate:
msg = aigenerate(AITemplate(:JuliaExpertAsk); ask = "How do I add packages?")
Find available templates with aitemplates:
tmps = aitemplates("JuliaExpertAsk")
 # Will surface one specific template
 # 1-element Vector{AITemplateMetadata}:
 # PromptingTools.AITemplateMetadata
@@ -44,14 +44,14 @@
 {{ask}}"
 #   source: String ""
The above gives you a good idea of what the template is about, what placeholders are available, and how much it would cost to use it (=wordcount).
Search for all Julia-related templates:
tmps = aitemplates("Julia")
 # 2-element Vector{AITemplateMetadata}... -> more to come later!
If you are on VSCode, you can leverage nice tabular display with vscodedisplay:
using DataFrames
-tmps = aitemplates("Julia") |> DataFrame |> vscodedisplay
I have my selected template, how do I use it? Just use the "name" in aigenerate or aiclassify   like you see in the first example!
You can inspect any template by "rendering" it (this is what the LLM will see):
julia> AITemplate(:JudgeIsItTrue) |> PromptingTools.render
See also: save_template, load_template, load_templates! for more advanced use cases (and the corresponding script in examples/ folder)
source
PromptingTools.ChatMLSchema — Type
ChatMLSchema is used by many open-source chatbots, by OpenAI models (under the hood) and by several models and inferfaces (eg, Ollama, vLLM)
You can explore it on tiktokenizer
It uses the following conversation structure:
<im_start>system
+tmps = aitemplates("Julia") |> DataFrame |> vscodedisplay
I have my selected template, how do I use it? Just use the "name" in aigenerate or aiclassify   like you see in the first example!
You can inspect any template by "rendering" it (this is what the LLM will see):
julia> AITemplate(:JudgeIsItTrue) |> PromptingTools.render
See also: save_template, load_template, load_templates! for more advanced use cases (and the corresponding script in examples/ folder)
source
PromptingTools.ChatMLSchema — Type
ChatMLSchema is used by many open-source chatbots, by OpenAI models (under the hood) and by several models and inferfaces (eg, Ollama, vLLM)
You can explore it on tiktokenizer
It uses the following conversation structure:
<im_start>system
 ...<im_end>
 <|im_start|>user
 ...<|im_end|>
 <|im_start|>assistant
-...<|im_end|>
source
PromptingTools.CustomOpenAISchema — Type
CustomOpenAISchema
CustomOpenAISchema() allows user to call any OpenAI-compatible API.
All user needs to do is to pass this schema as the first argument and provide the BASE URL of the API to call (api_kwargs.url).
Example
Assumes that we have a local server running at http://127.0.0.1:8081:
api_key = "..."
+...<|im_end|>
source
PromptingTools.CustomOpenAISchema — Type
CustomOpenAISchema
CustomOpenAISchema() allows user to call any OpenAI-compatible API.
All user needs to do is to pass this schema as the first argument and provide the BASE URL of the API to call (api_kwargs.url).
Example
Assumes that we have a local server running at http://127.0.0.1:8081:
api_key = "..."
 prompt = "Say hi!"
-msg = aigenerate(CustomOpenAISchema(), prompt; model="my_model", api_key, api_kwargs=(; url="http://127.0.0.1:8081"))
source
PromptingTools.DataMessage — Type
DataMessage
A message type for AI-generated data-based responses, ie, different content than text.  Returned by aiextract, and aiextract functions.
Fields
  • content::Union{AbstractString, Nothing}: The content of the message.
  • status::Union{Int, Nothing}: The status of the message from the API.
  • tokens::Tuple{Int, Int}: The number of tokens used (prompt,completion).
  • elapsed::Float64: The time taken to generate the response in seconds.
  • cost::Union{Nothing, Float64}: The cost of the API call (calculated with information from MODEL_REGISTRY).
  • log_prob::Union{Nothing, Float64}: The log probability of the response.
  • finish_reason::Union{Nothing, String}: The reason the response was finished.
  • run_id::Union{Nothing, Int}: The unique ID of the run.
  • sample_id::Union{Nothing, Int}: The unique ID of the sample (if multiple samples are generated, they will all have the same run_id).
source
PromptingTools.DatabricksOpenAISchema — Type
DatabricksOpenAISchema
DatabricksOpenAISchema() allows user to call Databricks Foundation Model API. API Reference
Requires two environment variables to be set:
  • DATABRICKS_API_KEY: Databricks token
  • DATABRICKS_HOST: Address of the Databricks workspace (https://<workspace_host>.databricks.com)
source
PromptingTools.LocalServerOpenAISchema — Type
LocalServerOpenAISchema
Designed to be used with local servers. It's automatically called with model alias "local" (see MODEL_REGISTRY).
This schema is a flavor of CustomOpenAISchema with a url keypreset by global Preference keyLOCAL_SERVER. See?PREFERENCESfor more details on how to change it. It assumes that the server follows OpenAI API conventions (eg,POST /v1/chat/completions`).
Note: Llama.cpp (and hence Llama.jl built on top of it) do NOT support embeddings endpoint! You'll get an address error.
Example
Assumes that we have a local server running at http://127.0.0.1:10897/v1 (port and address used by Llama.jl, "v1" at the end is needed for OpenAI endpoint compatibility):
Three ways to call it:

+msg = aigenerate(CustomOpenAISchema(), prompt; model="my_model", api_key, api_kwargs=(; url="http://127.0.0.1:8081"))
source
PromptingTools.DataMessage — Type
DataMessage
A message type for AI-generated data-based responses, ie, different content than text.  Returned by aiextract, and aiextract functions.
Fields
  • content::Union{AbstractString, Nothing}: The content of the message.
  • status::Union{Int, Nothing}: The status of the message from the API.
  • tokens::Tuple{Int, Int}: The number of tokens used (prompt,completion).
  • elapsed::Float64: The time taken to generate the response in seconds.
  • cost::Union{Nothing, Float64}: The cost of the API call (calculated with information from MODEL_REGISTRY).
  • log_prob::Union{Nothing, Float64}: The log probability of the response.
  • finish_reason::Union{Nothing, String}: The reason the response was finished.
  • run_id::Union{Nothing, Int}: The unique ID of the run.
  • sample_id::Union{Nothing, Int}: The unique ID of the sample (if multiple samples are generated, they will all have the same run_id).
source
PromptingTools.DatabricksOpenAISchema — Type
DatabricksOpenAISchema
DatabricksOpenAISchema() allows user to call Databricks Foundation Model API. API Reference
Requires two environment variables to be set:
  • DATABRICKS_API_KEY: Databricks token
  • DATABRICKS_HOST: Address of the Databricks workspace (https://<workspace_host>.databricks.com)
source
PromptingTools.LocalServerOpenAISchema — Type
LocalServerOpenAISchema
Designed to be used with local servers. It's automatically called with model alias "local" (see MODEL_REGISTRY).
This schema is a flavor of CustomOpenAISchema with a url keypreset by global Preference keyLOCAL_SERVER. See?PREFERENCESfor more details on how to change it. It assumes that the server follows OpenAI API conventions (eg,POST /v1/chat/completions`).
Note: Llama.cpp (and hence Llama.jl built on top of it) do NOT support embeddings endpoint! You'll get an address error.
Example
Assumes that we have a local server running at http://127.0.0.1:10897/v1 (port and address used by Llama.jl, "v1" at the end is needed for OpenAI endpoint compatibility):
Three ways to call it:

 # Use @ai_str with "local" alias
 ai"Say hi!"local
 
@@ -68,8 +68,8 @@
 
 # Or if it's a temporary fix, just change the variable `LOCAL_SERVER`:
 const PT = PromptingTools
-PT.LOCAL_SERVER = "http://127.0.0.1:10897/v1"
source
PromptingTools.MaybeExtract — Type
Extract a result from the provided data, if any, otherwise set the error and message fields.
Arguments
  • error::Bool: true if a result is found, false otherwise.
  • message::String: Only present if no result is found, should be short and concise.
source
PromptingTools.MistralOpenAISchema — Type
MistralOpenAISchema
MistralOpenAISchema() allows user to call MistralAI API known for mistral and mixtral models.
It's a flavor of CustomOpenAISchema() with a url preset to https://api.mistral.ai.
Most models have been registered, so you don't even have to specify the schema
Example
Let's call mistral-tiny model:
api_key = "..." # can be set via ENV["MISTRAL_API_KEY"] or via our preference system
-msg = aigenerate("Say hi!"; model="mistral_tiny", api_key)
See ?PREFERENCES for more details on how to set your API key permanently.
source
PromptingTools.ModelSpec — Type
ModelSpec
A struct that contains information about a model, such as its name, schema, cost per token, etc.
Fields
  • name::String: The name of the model. This is the name that will be used to refer to the model in the ai* functions.
  • schema::AbstractPromptSchema: The schema of the model. This is the schema that will be used to generate prompts for the model, eg, :OpenAISchema.
  • cost_of_token_prompt::Float64: The cost of 1 token in the prompt for this model. This is used to calculate the cost of a prompt.    Note: It is often provided online as cost per 1000 tokens, so make sure to convert it correctly!
  • cost_of_token_generation::Float64: The cost of 1 token generated by this model. This is used to calculate the cost of a generation.   Note: It is often provided online as cost per 1000 tokens, so make sure to convert it correctly!
  • description::String: A description of the model. This is used to provide more information about the model when it is queried.
Example
spec = ModelSpec("gpt-3.5-turbo",
+PT.LOCAL_SERVER = "http://127.0.0.1:10897/v1"
source
PromptingTools.MaybeExtract — Type
Extract a result from the provided data, if any, otherwise set the error and message fields.
Arguments
  • error::Bool: true if a result is found, false otherwise.
  • message::String: Only present if no result is found, should be short and concise.
source
PromptingTools.MistralOpenAISchema — Type
MistralOpenAISchema
MistralOpenAISchema() allows user to call MistralAI API known for mistral and mixtral models.
It's a flavor of CustomOpenAISchema() with a url preset to https://api.mistral.ai.
Most models have been registered, so you don't even have to specify the schema
Example
Let's call mistral-tiny model:
api_key = "..." # can be set via ENV["MISTRAL_API_KEY"] or via our preference system
+msg = aigenerate("Say hi!"; model="mistral_tiny", api_key)
See ?PREFERENCES for more details on how to set your API key permanently.
source
PromptingTools.ModelSpec — Type
ModelSpec
A struct that contains information about a model, such as its name, schema, cost per token, etc.
Fields
  • name::String: The name of the model. This is the name that will be used to refer to the model in the ai* functions.
  • schema::AbstractPromptSchema: The schema of the model. This is the schema that will be used to generate prompts for the model, eg, :OpenAISchema.
  • cost_of_token_prompt::Float64: The cost of 1 token in the prompt for this model. This is used to calculate the cost of a prompt.    Note: It is often provided online as cost per 1000 tokens, so make sure to convert it correctly!
  • cost_of_token_generation::Float64: The cost of 1 token generated by this model. This is used to calculate the cost of a generation.   Note: It is often provided online as cost per 1000 tokens, so make sure to convert it correctly!
  • description::String: A description of the model. This is used to provide more information about the model when it is queried.
Example
spec = ModelSpec("gpt-3.5-turbo",
     OpenAISchema(),
     0.0015,
     0.002,
@@ -81,12 +81,12 @@
     schema = OpenAISchema(),
     cost_of_token_prompt = 0.0015,
     cost_of_token_generation = 0.002,
-    description = "GPT-3.5 Turbo is a 175B parameter model and a common default on the OpenAI API.")
source
PromptingTools.NoSchema — Type
Schema that keeps messages (<:AbstractMessage) and does not transform for any specific model. It used by the first pass of the prompt rendering system (see ?render).
source
PromptingTools.OllamaManagedSchema — Type
Ollama by default manages different models and their associated prompt schemas when you pass system_prompt and prompt fields to the API.
Warning: It works only for 1 system message and 1 user message, so anything more than that has to be rejected.
If you need to pass more messagese / longer conversational history, you can use define the model-specific schema directly and pass your Ollama requests with raw=true,   which disables and templating and schema management by Ollama.
source
PromptingTools.OllamaSchema — Type
OllamaSchema is the default schema for Olama models.
It uses the following conversation template:
[Dict(role="system",content="..."),Dict(role="user",content="..."),Dict(role="assistant",content="...")]
It's very similar to OpenAISchema, but it appends images differently.
source
PromptingTools.OpenAISchema — Type
OpenAISchema is the default schema for OpenAI models.
It uses the following conversation template:
[Dict(role="system",content="..."),Dict(role="user",content="..."),Dict(role="assistant",content="...")]
It's recommended to separate sections in your prompt with markdown headers (e.g. `##Answer
`).
source
OpenAI.create_chat — Method
OpenAI.create_chat(schema::CustomOpenAISchema,
api_key::AbstractString,   model::AbstractString,   conversation;   url::String="http://localhost:8080",   kwargs...)
Dispatch to the OpenAI.create_chat function, for any OpenAI-compatible API. 
It expects url keyword argument. Provide it to the aigenerate function via api_kwargs=(; url="my-url")
It will forward your query to the "chat/completions" endpoint of the base URL that you provided (=url).
source
OpenAI.create_chat — Method
OpenAI.create_chat(schema::LocalServerOpenAISchema,
+    description = "GPT-3.5 Turbo is a 175B parameter model and a common default on the OpenAI API.")
source
PromptingTools.NoSchema — Type
Schema that keeps messages (<:AbstractMessage) and does not transform for any specific model. It used by the first pass of the prompt rendering system (see ?render).
source
PromptingTools.OllamaManagedSchema — Type
Ollama by default manages different models and their associated prompt schemas when you pass system_prompt and prompt fields to the API.
Warning: It works only for 1 system message and 1 user message, so anything more than that has to be rejected.
If you need to pass more messagese / longer conversational history, you can use define the model-specific schema directly and pass your Ollama requests with raw=true,   which disables and templating and schema management by Ollama.
source
PromptingTools.OllamaSchema — Type
OllamaSchema is the default schema for Olama models.
It uses the following conversation template:
[Dict(role="system",content="..."),Dict(role="user",content="..."),Dict(role="assistant",content="...")]
It's very similar to OpenAISchema, but it appends images differently.
source
PromptingTools.OpenAISchema — Type
OpenAISchema is the default schema for OpenAI models.
It uses the following conversation template:
[Dict(role="system",content="..."),Dict(role="user",content="..."),Dict(role="assistant",content="...")]
It's recommended to separate sections in your prompt with markdown headers (e.g. `##Answer
`).
source
OpenAI.create_chat — Method
OpenAI.create_chat(schema::CustomOpenAISchema,
api_key::AbstractString,   model::AbstractString,   conversation;   url::String="http://localhost:8080",   kwargs...)
Dispatch to the OpenAI.create_chat function, for any OpenAI-compatible API. 
It expects url keyword argument. Provide it to the aigenerate function via api_kwargs=(; url="my-url")
It will forward your query to the "chat/completions" endpoint of the base URL that you provided (=url).
source
OpenAI.create_chat — Method
OpenAI.create_chat(schema::LocalServerOpenAISchema,
     api_key::AbstractString,
     model::AbstractString,
     conversation;
     url::String = "http://localhost:8080",
-    kwargs...)
Dispatch to the OpenAI.createchat function, but with the LocalServer API parameters, ie, defaults to url specified by the `LOCALSERVERpreference. See?PREFERENCES`
source
OpenAI.create_chat — Method
OpenAI.create_chat(schema::MistralOpenAISchema,
api_key::AbstractString,   model::AbstractString,   conversation;   url::String="https://api.mistral.ai/v1",   kwargs...)
Dispatch to the OpenAI.create_chat function, but with the MistralAI API parameters. 
It tries to access the MISTRALAI_API_KEY ENV variable, but you can also provide it via the api_key keyword argument.
source
PromptingTools.aiclassify — Method
aiclassify(prompt_schema::AbstractOpenAISchema, prompt::ALLOWED_PROMPT_TYPE;
+    kwargs...)
Dispatch to the OpenAI.createchat function, but with the LocalServer API parameters, ie, defaults to url specified by the `LOCALSERVERpreference. See?PREFERENCES`
source
OpenAI.create_chat — Method
OpenAI.create_chat(schema::MistralOpenAISchema,
api_key::AbstractString,   model::AbstractString,   conversation;   url::String="https://api.mistral.ai/v1",   kwargs...)
Dispatch to the OpenAI.create_chat function, but with the MistralAI API parameters. 
It tries to access the MISTRALAI_API_KEY ENV variable, but you can also provide it via the api_key keyword argument.
source
PromptingTools.aiclassify — Method
aiclassify(prompt_schema::AbstractOpenAISchema, prompt::ALLOWED_PROMPT_TYPE;
     choices::AbstractVector{T} = ["true", "false", "unknown"],
     api_kwargs::NamedTuple = NamedTuple(),
     kwargs...) where {T <: Union{AbstractString, Tuple{<:AbstractString, <:AbstractString}}}
Classifies the given prompt/statement into an arbitrary list of choices, which must be only the choices (vector of strings) or choices and descriptions are provided (vector of tuples, ie, ("choice","description")).
It's quick and easy option for "routing" and similar use cases, as it exploits the logit bias trick and outputs only 1 token. classify into an arbitrary list of categories (including with descriptions). It's quick and easy option for "routing" and similar use cases, as it exploits the logit bias trick, so it outputs only 1 token.
!!! Note: The prompt/AITemplate must have a placeholder choices (ie, {{choices}}) that will be replaced with the encoded choices
Choices are rewritten into an enumerated list and mapped to a few known OpenAI tokens (maximum of 20 choices supported). Mapping of token IDs for GPT3.5/4 are saved in variable OPENAI_TOKEN_IDS.
It uses Logit bias trick and limits the output to 1 token to force the model to output only true/false/unknown. Credit for the idea goes to AAAzzam.
Arguments
  • prompt_schema::AbstractOpenAISchema: The schema for the prompt.
  • prompt: The prompt/statement to classify if it's a String. If it's a Symbol, it is expanded as a template via render(schema,template). Eg, templates :JudgeIsItTrue or :InputClassifier
  • choices::AbstractVector{T}: The choices to be classified into. It can be a vector of strings or a vector of tuples, where the first element is the choice and the second is the description.
Example
Given a user input, pick one of the two provided categories:
choices = ["animal", "plant"]
@@ -100,7 +100,7 @@
 aiclassify(:InputClassifier; choices, input)
You can still use a simple true/false classification:
aiclassify("Is two plus two four?") # true
 aiclassify("Is two plus three a vegetable on Mars?") # false
aiclassify returns only true/false/unknown. It's easy to get the proper Bool output type out with tryparse, eg,
tryparse(Bool, aiclassify("Is two plus two four?")) isa Bool # true
Output of type Nothing marks that the model couldn't classify the statement as true/false.
Ideally, we would like to re-use some helpful system prompt to get more accurate responses. For this reason we have templates, eg, :JudgeIsItTrue. By specifying the template, we can provide our statement as the expected variable (it in this case) See that the model now correctly classifies the statement as "unknown".
aiclassify(:JudgeIsItTrue; it = "Is two plus three a vegetable on Mars?") # unknown
For better results, use higher quality models like gpt4, eg, 
aiclassify(:JudgeIsItTrue;
     it = "If I had two apples and I got three more, I have five apples now.",
-    model = "gpt4") # true
source
PromptingTools.aiembed — Method
aiembed(prompt_schema::AbstractOllamaManagedSchema,
         doc_or_docs::Union{AbstractString, AbstractVector{<:AbstractString}},
         postprocess::F = identity;
         verbose::Bool = true,
@@ -129,7 +129,7 @@
 schema = PT.OllamaManagedSchema()
 
 msg = aiembed(schema, "Hello World", copy; model="openhermes2.5-mistral")
-msg.content # 4096-element Vector{Float64}
source
PromptingTools.aiembed — Method
aiembed(prompt_schema::AbstractOpenAISchema,
         doc_or_docs::Union{AbstractString, AbstractVector{<:AbstractString}},
         postprocess::F = identity;
         verbose::Bool = true,
@@ -145,7 +145,7 @@
 msg = aiembed(["embed me", "and me too"], LinearAlgebra.normalize)
 
 # calculate cosine distance between the two normalized embeddings as a simple dot product
-msg.content' * msg.content[:, 1] # [1.0, 0.787]
source
PromptingTools.aiextract — Method
aiextract(prompt_schema::AbstractOpenAISchema, prompt::ALLOWED_PROMPT_TYPE;
+msg.content' * msg.content[:, 1] # [1.0, 0.787]
source
PromptingTools.aiextract — Method
aiextract(prompt_schema::AbstractOpenAISchema, prompt::ALLOWED_PROMPT_TYPE;
     return_type::Type,
     verbose::Bool = true,
     api_key::String = OPENAI_API_KEY,
@@ -198,7 +198,7 @@
     name::String
 end
 aiextract("I ate an apple",return_type=Fruit,api_kwargs=(;tool_choice="any"),model="mistrall")
-# Notice two differences: 1) struct MUST have a docstring, 2) tool_choice is set explicitly set to "any"
source
PromptingTools.aigenerate — Method
aigenerate(prompt_schema::AbstractGoogleSchema, prompt::ALLOWED_PROMPT_TYPE;
+# Notice two differences: 1) struct MUST have a docstring, 2) tool_choice is set explicitly set to "any"
source
PromptingTools.aigenerate — Method
aigenerate(prompt_schema::AbstractGoogleSchema, prompt::ALLOWED_PROMPT_TYPE;
     verbose::Bool = true,
     api_key::String = GOOGLE_API_KEY,
     model::String = "gemini-pro", return_all::Bool = false, dry_run::Bool = false,
@@ -216,7 +216,7 @@
     PT.SystemMessage("You're master Yoda from Star Wars trying to help the user become a Yedi."),
     PT.UserMessage("I have feelings for my iPhone. What should I do?")]
 msg=aigenerate(conversation; model="gemini")
-# AIMessage("Young Padawan, you have stumbled into a dangerous path.... <continues>")
source
PromptingTools.aigenerate — Method
aigenerate(prompt_schema::AbstractOllamaManagedSchema, prompt::ALLOWED_PROMPT_TYPE; verbose::Bool = true,
+# AIMessage("Young Padawan, you have stumbled into a dangerous path.... <continues>")
source
PromptingTools.aigenerate — Method
aigenerate(prompt_schema::AbstractOllamaManagedSchema, prompt::ALLOWED_PROMPT_TYPE; verbose::Bool = true,
     api_key::String = "", model::String = MODEL_CHAT,
     return_all::Bool = false, dry_run::Bool = false,
     conversation::AbstractVector{<:AbstractMessage} = AbstractMessage[],
@@ -241,7 +241,7 @@
 
 msg = aigenerate(schema, conversation; model="openhermes2.5-mistral")
 # [ Info: Tokens: 111 in 2.1 seconds
-# AIMessage("Strong the attachment is, it leads to suffering it may. Focus on the force within you must, ...<continues>")
Note: Managed Ollama currently supports at most 1 User Message and 1 System Message given the API limitations. If you want more, you need to use the ChatMLSchema.
source
PromptingTools.aigenerate — Method
aigenerate(prompt_schema::AbstractOllamaManagedSchema, prompt::ALLOWED_PROMPT_TYPE; verbose::Bool = true,
+# AIMessage("Strong the attachment is, it leads to suffering it may. Focus on the force within you must, ...<continues>")
Note: Managed Ollama currently supports at most 1 User Message and 1 System Message given the API limitations. If you want more, you need to use the ChatMLSchema.
source
PromptingTools.aigenerate — Method
aigenerate(prompt_schema::AbstractOllamaManagedSchema, prompt::ALLOWED_PROMPT_TYPE; verbose::Bool = true,
     api_key::String = "", model::String = MODEL_CHAT,
     return_all::Bool = false, dry_run::Bool = false,
     conversation::AbstractVector{<:AbstractMessage} = AbstractMessage[],
@@ -266,7 +266,7 @@
 
 msg = aigenerate(schema, conversation; model="openhermes2.5-mistral")
 # [ Info: Tokens: 111 in 2.1 seconds
-# AIMessage("Strong the attachment is, it leads to suffering it may. Focus on the force within you must, ...<continues>")
Note: Managed Ollama currently supports at most 1 User Message and 1 System Message given the API limitations. If you want more, you need to use the ChatMLSchema.
source
PromptingTools.aigenerate — Method
aigenerate(prompt_schema::AbstractOpenAISchema, prompt::ALLOWED_PROMPT_TYPE;
+# AIMessage("Strong the attachment is, it leads to suffering it may. Focus on the force within you must, ...<continues>")
Note: Managed Ollama currently supports at most 1 User Message and 1 System Message given the API limitations. If you want more, you need to use the ChatMLSchema.
source
PromptingTools.aigenerate — Method
aigenerate(prompt_schema::AbstractOpenAISchema, prompt::ALLOWED_PROMPT_TYPE;
     verbose::Bool = true,
     api_key::String = OPENAI_API_KEY,
     model::String = MODEL_CHAT, return_all::Bool = false, dry_run::Bool = false,
@@ -285,7 +285,35 @@
     PT.SystemMessage("You're master Yoda from Star Wars trying to help the user become a Yedi."),
     PT.UserMessage("I have feelings for my iPhone. What should I do?")]
 msg=aigenerate(conversation)
-# AIMessage("Ah, strong feelings you have for your iPhone. A Jedi's path, this is not... <continues>")
source
PromptingTools.aiscan — Method
aiscan([prompt_schema::AbstractOllamaSchema,] prompt::ALLOWED_PROMPT_TYPE; 
+# AIMessage("Ah, strong feelings you have for your iPhone. A Jedi's path, this is not... <continues>")
source
PromptingTools.aiimage — Method
aiimage(prompt_schema::AbstractOpenAISchema, prompt::ALLOWED_PROMPT_TYPE;
+    image_size::AbstractString = "1024x1024",
+    image_quality::AbstractString = "standard",
+    image_n::Integer = 1,
+    verbose::Bool = true,
+    api_key::String = OPENAI_API_KEY,
+    model::String = MODEL_IMAGE_GENERATION,
+    return_all::Bool = false, dry_run::Bool = false,
+    conversation::AbstractVector{<:AbstractMessage} = AbstractMessage[],
+    http_kwargs::NamedTuple = (retry_non_idempotent = true,
+        retries = 5,
+        readtimeout = 120), api_kwargs::NamedTuple = NamedTuple(),
+    kwargs...)
Generates an image from the provided prompt. If multiple "messages" are provided in prompt, it extracts the text ONLY from the last message!
Image (or the reference to it) will be returned in a DataMessage.content, the format will depend on the api_kwargs.response_format you set.
Can be used for generating images of varying quality and style with dall-e-* models. This function DOES NOT SUPPORT multi-turn conversations (ie, do not provide previous conversation via conversation argument).
Arguments
  • prompt_schema: An optional object to specify which prompt template should be applied (Default to PROMPT_SCHEMA = OpenAISchema)
  • prompt: Can be a string representing the prompt for the AI conversation, a UserMessage, a vector of AbstractMessage or an AITemplate
  • image_size: String-based resolution of the image, eg, "1024x1024". Only some resolutions are supported - see the API docs.
  • image_quality: It can be either "standard" or "hd". Defaults to "standard".
  • image_n: The number of images to generate. Currently, only single image generation is allowed (image_n = 1).
  • verbose: A boolean indicating whether to print additional information.
  • api_key: A string representing the API key for accessing the OpenAI API.
  • model: A string representing the model to use for generating the response. Can be an alias corresponding to a model ID defined in MODEL_IMAGE_GENERATION.
  • return_all::Bool=false: If true, returns the entire conversation history, otherwise returns only the last message (the AIMessage).
  • dry_run::Bool=false: If true, skips sending the messages to the model (for debugging, often used with return_all=true).
  • conversation: An optional vector of AbstractMessage objects representing the conversation history. Currently, NOT ALLOWED.
  • http_kwargs: A named tuple of HTTP keyword arguments.
  • api_kwargs: A named tuple of API keyword arguments. Several important arguments are highlighted below:
    • response_format: The format image should be returned in. Can be one of "url" or "b64_json". Defaults to "url" (the link will be inactived in 60 minutes).
    • style: The style of generated images (DALL-E 3 only). Can be either "vidid" or "natural". Defauls to "vidid".
  • kwargs: Prompt variables to be used to fill the prompt/template
Returns
If return_all=false (default):
  • msg: A DataMessage object representing one or more generated images, including the rewritten prompt if relevant, status, and elapsed time.
Use msg.content to access the extracted string.
If return_all=true:
  • conversation: A vector of AbstractMessage objects representing the full conversation history, including the response from the AI model (AIMessage).
See also: ai_str, aai_str, aigenerate, aiembed, aiclassify, aiextract, aiscan, aitemplates
Notes
  • This function DOES NOT SUPPORT multi-turn conversations (ie, do not provide previous conversation via conversation argument).
  • There is no token tracking provided by the API, so the messages will NOT report any cost despite costing you money!
  • You MUST download any URL-based images within 60 minutes. The links will become inactive.
Example
Generate an image:
# You can experiment with `image_size`, `image_quality` kwargs!
+msg = aiimage("A white cat on a car")
+
+# Download the image into a file
+using Downloads
+Downloads.download(msg.content[:url], "cat_on_car.png")
+
+# You can also see the revised prompt that DALL-E 3 used
+msg.content[:revised_prompt]
+# Output: "Visualize a pristine white cat gracefully perched atop a shiny car. 
+# The cat's fur is stark white and its eyes bright with curiosity. 
+# As for the car, it could be a contemporary sedan, glossy and in a vibrant color. 
+# The scene could be set under the blue sky, enhancing the contrast between the white cat, the colorful car, and the bright blue sky."
Note that you MUST download any URL-based images within 60 minutes. The links will become inactive.
If you wanted to download image directly into the DataMessage, provide response_format="b64_json" in api_kwargs:
msg = aiimage("A white cat on a car"; image_quality="hd", api_kwargs=(; response_format="b64_json"))
+
+# Then you need to use Base64 package to decode it and save it to a file:
+using Base64
+write("cat_on_car_hd.png", base64decode(msg.content[:b64_json]));
source
PromptingTools.aiscan — Method
aiscan([prompt_schema::AbstractOllamaSchema,] prompt::ALLOWED_PROMPT_TYPE; 
 image_url::Union{Nothing, AbstractString, Vector{<:AbstractString}} = nothing,
 image_path::Union{Nothing, AbstractString, Vector{<:AbstractString}} = nothing,
 attach_to_latest::Bool = true,
@@ -312,7 +340,7 @@
 # You can add syntax highlighting of the outputs via Markdown
 using Markdown
 msg.content |> Markdown.parse
Local models cannot handle image URLs directly (image_url), so you need to download the image first and provide it as image_path:
using Downloads
-image_path = Downloads.download(image_url)
Notice that we set max_tokens = 2500. If your outputs seem truncated, it might be because the default maximum tokens on the server is set too low!
source
PromptingTools.aiscan — Method
aiscan([prompt_schema::AbstractOpenAISchema,] prompt::ALLOWED_PROMPT_TYPE; 
+image_path = Downloads.download(image_url)
Notice that we set max_tokens = 2500. If your outputs seem truncated, it might be because the default maximum tokens on the server is set too low!
source
PromptingTools.aiscan — Method
aiscan([prompt_schema::AbstractOpenAISchema,] prompt::ALLOWED_PROMPT_TYPE; 
 image_url::Union{Nothing, AbstractString, Vector{<:AbstractString}} = nothing,
 image_path::Union{Nothing, AbstractString, Vector{<:AbstractString}} = nothing,
 image_detail::AbstractString = "auto",
@@ -338,7 +366,7 @@
 
 # You can add syntax highlighting of the outputs via Markdown
 using Markdown
-msg.content |> Markdown.parse
Notice that we enforce max_tokens = 2500. That's because OpenAI seems to default to ~300 tokens, which provides incomplete outputs. Hence, we set this value to 2500 as a default. If you still get truncated outputs, increase this value.
source
PromptingTools.aitemplates — Function
aitemplates
Find easily the most suitable templates for your use case.
You can search by:
  • query::Symbol which looks look only for partial matches in the template name
  • query::AbstractString which looks for partial matches in the template name or description
  • query::Regex which looks for matches in the template name, description or any of the message previews
Keyword Arguments
  • limit::Int limits the number of returned templates (Defaults to 10)
Examples
Find available templates with aitemplates:
tmps = aitemplates("JuliaExpertAsk")
+msg.content |> Markdown.parse
Notice that we enforce max_tokens = 2500. That's because OpenAI seems to default to ~300 tokens, which provides incomplete outputs. Hence, we set this value to 2500 as a default. If you still get truncated outputs, increase this value.
source
PromptingTools.aitemplates — Function
aitemplates
Find easily the most suitable templates for your use case.
You can search by:
  • query::Symbol which looks look only for partial matches in the template name
  • query::AbstractString which looks for partial matches in the template name or description
  • query::Regex which looks for matches in the template name, description or any of the message previews
Keyword Arguments
  • limit::Int limits the number of returned templates (Defaults to 10)
Examples
Find available templates with aitemplates:
tmps = aitemplates("JuliaExpertAsk")
 # Will surface one specific template
 # 1-element Vector{AITemplateMetadata}:
 # PromptingTools.AITemplateMetadata
@@ -353,11 +381,11 @@
 {{ask}}"
 #   source: String ""
The above gives you a good idea of what the template is about, what placeholders are available, and how much it would cost to use it (=wordcount).
Search for all Julia-related templates:
tmps = aitemplates("Julia")
 # 2-element Vector{AITemplateMetadata}... -> more to come later!
If you are on VSCode, you can leverage nice tabular display with vscodedisplay:
using DataFrames
-tmps = aitemplates("Julia") |> DataFrame |> vscodedisplay
I have my selected template, how do I use it? Just use the "name" in aigenerate or aiclassify   like you see in the first example!
source
PromptingTools.aitemplates — Method
Find the top-limit templates whose name or description fields partially match the query_key::String in TEMPLATE_METADATA.
source
PromptingTools.aitemplates — Method
Find the top-limit templates where provided query_key::Regex matches either of name, description or previews or User or System messages in TEMPLATE_METADATA.
source
PromptingTools.auth_header — Method
auth_header(api_key::Union{Nothing, AbstractString};
+tmps = aitemplates("Julia") |> DataFrame |> vscodedisplay
I have my selected template, how do I use it? Just use the "name" in aigenerate or aiclassify   like you see in the first example!
source
PromptingTools.aitemplates — Method
Find the top-limit templates whose name or description fields partially match the query_key::String in TEMPLATE_METADATA.
source
PromptingTools.aitemplates — Method
Find the top-limit templates where provided query_key::Regex matches either of name, description or previews or User or System messages in TEMPLATE_METADATA.
source
PromptingTools.auth_header — Method
auth_header(api_key::Union{Nothing, AbstractString};
     extra_headers::AbstractVector{Pair{String, String}} = Vector{Pair{String, String}}[],
-    kwargs...)
Creates the authentication headers for any API request. Assumes that the communication is done in JSON format.
source
PromptingTools.build_template_metadata — Function
build_template_metadata(
     template::AbstractVector{<:AbstractMessage}, template_name::Symbol,
-    metadata_msgs::AbstractVector{<:MetadataMessage} = MetadataMessage[])
Builds AITemplateMetadata for a given template based on the messages in template and other information.
AITemplateMetadata is a helper struct for easy searching and reviewing of templates via aitemplates().
source
PromptingTools.call_cost — Method
call_cost(prompt_tokens::Int, completion_tokens::Int, model::String;
+    metadata_msgs::AbstractVector{<:MetadataMessage} = MetadataMessage[])
Builds AITemplateMetadata for a given template based on the messages in template and other information.
AITemplateMetadata is a helper struct for easy searching and reviewing of templates via aitemplates().
source
PromptingTools.call_cost — Method
call_cost(prompt_tokens::Int, completion_tokens::Int, model::String;
     cost_of_token_prompt::Number = get(MODEL_REGISTRY,
         model,
         (; cost_of_token_prompt = 0.0)).cost_of_token_prompt,
@@ -379,7 +407,7 @@
 
 # Using custom token costs
 cost2 = call_cost(10, 20, "model3"; cost_of_token_prompt = 0.08, cost_of_token_generation = 0.12)
-# cost2 = 10 * 0.08 + 20 * 0.12 = 3.2
source
PromptingTools.create_template — Method
create_template(; user::AbstractString, system::AbstractString="Act as a helpful AI assistant.", 
+# cost2 = 10 * 0.08 + 20 * 0.12 = 3.2
source
PromptingTools.create_template — Method
create_template(; user::AbstractString, system::AbstractString="Act as a helpful AI assistant.", 
     load_as::Union{Nothing, Symbol, AbstractString} = nothing)
 
 create_template(system::AbstractString, user::AbstractString, 
@@ -402,9 +430,9 @@
 ##   system_preview: String "You must speak like a pirate"
 ##   user_preview: String "Say hi to {{name}}"
 ##   source: String ""
Now you can use it like any other template (notice it's a symbol, so :GreatingPirate):
aigenerate(:GreatingPirate; name="Jack Sparrow")
-# Output: AIMessage("Arr, me hearty! Best be sending me regards to Captain Jack Sparrow on the salty seas! May his compass always point true to the nearest treasure trove. Yarrr!")
If you do not need to save this template as a file, but you want to make it accessible in the template store for all ai* functions, you can use the load_as (= template name) keyword argument: ```julia
this will not only create the template, but also register it for immediate use
tpl=PT.createtemplate("You must speak like a pirate", "Say hi to {{name}}"; loadas="GreatingPirate")
you can now use it like any other template
aiextract(:GreatingPirate; name="Jack Sparrow") ````
source
PromptingTools.decode_choices — Method
decode_choices(schema::OpenAISchema,
+# Output: AIMessage("Arr, me hearty! Best be sending me regards to Captain Jack Sparrow on the salty seas! May his compass always point true to the nearest treasure trove. Yarrr!")
If you do not need to save this template as a file, but you want to make it accessible in the template store for all ai* functions, you can use the load_as (= template name) keyword argument: ```julia
this will not only create the template, but also register it for immediate use
tpl=PT.createtemplate("You must speak like a pirate", "Say hi to {{name}}"; loadas="GreatingPirate")
you can now use it like any other template
aiextract(:GreatingPirate; name="Jack Sparrow") ````
source
PromptingTools.decode_choices — Method
decode_choices(schema::OpenAISchema,
     choices::AbstractVector{<:AbstractString},
-    msg::AIMessage; kwargs...)
Decodes the underlying AIMessage against the original choices to lookup what the category name was.
If it fails, it will return msg.content == nothing
source
PromptingTools.detect_base_main_overrides — Method
detect_base_main_overrides(code_block::AbstractString)
Detects if a given code block overrides any Base or Main methods. 
Returns a tuple of a boolean and a vector of the overriden methods.
source
PromptingTools.encode_choices — Method
encode_choices(schema::OpenAISchema, choices::AbstractVector{<:AbstractString}; kwargs...)
+    msg::AIMessage; kwargs...)
Decodes the underlying AIMessage against the original choices to lookup what the category name was.
If it fails, it will return msg.content == nothing
source
PromptingTools.detect_base_main_overrides — Method
detect_base_main_overrides(code_block::AbstractString)
Detects if a given code block overrides any Base or Main methods. 
Returns a tuple of a boolean and a vector of the overriden methods.
source
PromptingTools.encode_choices — Method
encode_choices(schema::OpenAISchema, choices::AbstractVector{<:AbstractString}; kwargs...)
 
 encode_choices(schema::OpenAISchema, choices::AbstractVector{T};
 kwargs...) where {T <: Tuple{<:AbstractString, <:AbstractString}}
Encode the choices into an enumerated list that can be interpolated into the prompt and creates the corresponding logit biases (to choose only from the selected tokens).
Optionally, can be a vector tuples, where the first element is the choice and the second is the description.
Arguments
  • schema::OpenAISchema: The OpenAISchema object.
  • choices::AbstractVector{<:Union{AbstractString,Tuple{<:AbstractString, <:AbstractString}}}: The choices to be encoded, represented as a vector of the choices directly, or tuples where each tuple contains a choice and its description.
  • kwargs...: Additional keyword arguments.
Returns
  • choices_prompt::AbstractString: The encoded choices as a single string, separated by newlines.
  • logit_bias::Dict: The logit bias dictionary, where the keys are the token IDs and the values are the bias values.
  • decode_ids::AbstractVector{<:AbstractString}: The decoded IDs of the choices.
Examples
choices_prompt, logit_bias, _ = PT.encode_choices(PT.OpenAISchema(), ["true", "false"])
@@ -419,33 +447,33 @@
 choices_prompt # Output: "1. "A" for any animal or creature
 2. "P" for any plant or tree
 3. "O" for everything else"
-logit_bias # Output: Dict(16 => 100, 17 => 100, 18 => 100)
source
PromptingTools.eval! — Method
eval!(cb::AbstractCodeBlock;
     safe_eval::Bool = true,
     capture_stdout::Bool = true,
     prefix::AbstractString = "",
-    suffix::AbstractString = "")
Evaluates a code block cb in-place. It runs automatically when AICode is instantiated with a String.
Check the outcome of evaluation with Base.isvalid(cb). If ==true, provide code block has executed successfully.
Steps:
  • If cb::AICode has not been evaluated, cb.success = nothing.  After the evaluation it will be either true or false depending on the outcome
  • Parse the text in cb.code
  • Evaluate the parsed expression
  • Capture outputs of the evaluated in cb.output
  • [OPTIONAL] Capture any stdout outputs (eg, test failures) in cb.stdout
  • If any error exception is raised, it is saved in cb.error
  • Finally, if all steps were successful, success is set to cb.success = true
Keyword Arguments
  • safe_eval::Bool: If true, we first check for any Pkg operations (eg, installing new packages) and missing imports,  then the code will be evaluated inside a bespoke scratch module (not to change any user variables)
  • capture_stdout::Bool: If true, we capture any stdout outputs (eg, test failures) in cb.stdout
  • prefix::AbstractString: A string to be prepended to the code block before parsing and evaluation. Useful to add some additional code definition or necessary imports. Defaults to an empty string.
  • suffix::AbstractString: A string to be appended to the code block before parsing and evaluation.  Useful to check that tests pass or that an example executes. Defaults to an empty string.
source
PromptingTools.extract_code_blocks — Method
extract_code_blocks(markdown_content::String) -> Vector{String}
Extract Julia code blocks from a markdown string.
This function searches through the provided markdown content, identifies blocks of code specifically marked as Julia code  (using the julia ... code fence patterns), and extracts the code within these blocks.  The extracted code blocks are returned as a vector of strings, with each string representing one block of Julia code. 
Note: Only the content within the code fences is extracted, and the code fences themselves are not included in the output.
See also: extract_code_blocks_fallback
Arguments
  • markdown_content::String: A string containing the markdown content from which Julia code blocks are to be extracted.
Returns
  • Vector{String}: A vector containing strings of extracted Julia code blocks. If no Julia code blocks are found, an empty vector is returned.
Examples
Example with a single Julia code block
markdown_single = """
julia println("Hello, World!")
"""
+    suffix::AbstractString = "")
Evaluates a code block cb in-place. It runs automatically when AICode is instantiated with a String.
Check the outcome of evaluation with Base.isvalid(cb). If ==true, provide code block has executed successfully.
Steps:
  • If cb::AICode has not been evaluated, cb.success = nothing.  After the evaluation it will be either true or false depending on the outcome
  • Parse the text in cb.code
  • Evaluate the parsed expression
  • Capture outputs of the evaluated in cb.output
  • [OPTIONAL] Capture any stdout outputs (eg, test failures) in cb.stdout
  • If any error exception is raised, it is saved in cb.error
  • Finally, if all steps were successful, success is set to cb.success = true
Keyword Arguments
  • safe_eval::Bool: If true, we first check for any Pkg operations (eg, installing new packages) and missing imports,  then the code will be evaluated inside a bespoke scratch module (not to change any user variables)
  • capture_stdout::Bool: If true, we capture any stdout outputs (eg, test failures) in cb.stdout
  • prefix::AbstractString: A string to be prepended to the code block before parsing and evaluation. Useful to add some additional code definition or necessary imports. Defaults to an empty string.
  • suffix::AbstractString: A string to be appended to the code block before parsing and evaluation.  Useful to check that tests pass or that an example executes. Defaults to an empty string.
source
PromptingTools.extract_code_blocks — Method
extract_code_blocks(markdown_content::String) -> Vector{String}
Extract Julia code blocks from a markdown string.
This function searches through the provided markdown content, identifies blocks of code specifically marked as Julia code  (using the julia ... code fence patterns), and extracts the code within these blocks.  The extracted code blocks are returned as a vector of strings, with each string representing one block of Julia code. 
Note: Only the content within the code fences is extracted, and the code fences themselves are not included in the output.
See also: extract_code_blocks_fallback
Arguments
  • markdown_content::String: A string containing the markdown content from which Julia code blocks are to be extracted.
Returns
  • Vector{String}: A vector containing strings of extracted Julia code blocks. If no Julia code blocks are found, an empty vector is returned.
Examples
Example with a single Julia code block
markdown_single = """
julia println("Hello, World!")
"""
 extract_code_blocks(markdown_single)
 # Output: ["Hello, World!"]
# Example with multiple Julia code blocks
 markdown_multiple = """
julia x = 5
Some text in between
julia y = x + 2
"""
 extract_code_blocks(markdown_multiple)
-# Output: ["x = 5", "y = x + 2"]
source
PromptingTools.extract_code_blocks_fallback — Method
extract_code_blocks_fallback(markdown_content::String, delim::AbstractString="\n```\n")
Extract Julia code blocks from a markdown string using a fallback method (splitting by arbitrary delim-iters). Much more simplistic than extract_code_blocks and does not support nested code blocks.
It is often used as a fallback for smaller LLMs that forget to code fence julia ....
Example
code = """
println("hello")

+# Output: ["x = 5", "y = x + 2"]
source
PromptingTools.extract_code_blocks_fallback — Method
extract_code_blocks_fallback(markdown_content::String, delim::AbstractString="\n```\n")
Extract Julia code blocks from a markdown string using a fallback method (splitting by arbitrary delim-iters). Much more simplistic than extract_code_blocks and does not support nested code blocks.
It is often used as a fallback for smaller LLMs that forget to code fence julia ....
Example
code = """
println("hello")

 Some text
 
println("world")
"""
 
 # We extract text between triple backticks and check each blob if it looks like a valid Julia code
 code_parsed = extract_code_blocks_fallback(code) |> x -> filter(is_julia_code, x) |> x -> join(x, "
-")
source
PromptingTools.extract_function_name — Method
extract_function_name(code_block::String) -> Union{String, Nothing}
Extract the name of a function from a given Julia code block. The function searches for two patterns:
  • The explicit function declaration pattern: function name(...) ... end
  • The concise function declaration pattern: name(...) = ...
If a function name is found, it is returned as a string. If no function name is found, the function returns nothing.
To capture all function names in the block, use extract_function_names.
Arguments
  • code_block::String: A string containing Julia code.
Returns
  • Union{String, Nothing}: The extracted function name or nothing if no name is found.
Example
code = """
+")
source
PromptingTools.extract_function_name — Method
extract_function_name(code_block::String) -> Union{String, Nothing}
Extract the name of a function from a given Julia code block. The function searches for two patterns:
  • The explicit function declaration pattern: function name(...) ... end
  • The concise function declaration pattern: name(...) = ...
If a function name is found, it is returned as a string. If no function name is found, the function returns nothing.
To capture all function names in the block, use extract_function_names.
Arguments
  • code_block::String: A string containing Julia code.
Returns
  • Union{String, Nothing}: The extracted function name or nothing if no name is found.
Example
code = """
 function myFunction(arg1, arg2)
     # Function body
 end
 """
 extract_function_name(code)
-# Output: "myFunction"
source
PromptingTools.extract_function_names — Method
extract_function_names(code_block::AbstractString)
Extract one or more names of functions defined in a given Julia code block. The function searches for two patterns:     - The explicit function declaration pattern: function name(...) ... end     - The concise function declaration pattern: name(...) = ...
It always returns a vector of strings, even if only one function name is found (it will be empty).
For only one function name match, use extract_function_name.
source
PromptingTools.extract_julia_imports — Method
extract_julia_imports(input::AbstractString; base_or_main::Bool = false)
Detects any using or import statements in a given string and returns the package names as a vector of symbols. 
base_or_main is a boolean that determines whether to isolate only Base and Main OR whether to exclude them in the returned vector.
source
PromptingTools.extract_function_names — Method
extract_function_names(code_block::AbstractString)
Extract one or more names of functions defined in a given Julia code block. The function searches for two patterns:     - The explicit function declaration pattern: function name(...) ... end     - The concise function declaration pattern: name(...) = ...
It always returns a vector of strings, even if only one function name is found (it will be empty).
For only one function name match, use extract_function_name.
source
PromptingTools.extract_julia_imports — Method
extract_julia_imports(input::AbstractString; base_or_main::Bool = false)
Detects any using or import statements in a given string and returns the package names as a vector of symbols. 
base_or_main is a boolean that determines whether to isolate only Base and Main OR whether to exclude them in the returned vector.
source
PromptingTools.finalize_outputs — Method
finalize_outputs(prompt::ALLOWED_PROMPT_TYPE, conv_rendered::Any,
     msg::Union{Nothing, AbstractMessage, AbstractVector{<:AbstractMessage}};
     return_all::Bool = false,
     dry_run::Bool = false,
     conversation::AbstractVector{<:AbstractMessage} = AbstractMessage[],
-    kwargs...)
Finalizes the outputs of the ai* functions by either returning the conversation history or the last message.
Keyword arguments
  • return_all::Bool=false: If true, returns the entire conversation history, otherwise returns only the last message (the AIMessage).
  • dry_run::Bool=false: If true, does not send the messages to the model, but only renders the prompt with the given schema and replacement variables. Useful for debugging when you want to check the specific schema rendering. 
  • conversation::AbstractVector{<:AbstractMessage}=[]: An optional vector of AbstractMessage objects representing the conversation history. If not provided, it is initialized as an empty vector.
  • kwargs...: Variables to replace in the prompt template.
source
PromptingTools.find_subsequence_positions — Method
find_subsequence_positions(subseq, seq) -> Vector{Int}
Find all positions of a subsequence subseq within a larger sequence seq. Used to lookup positions of code blocks in markdown.
This function scans the sequence seq and identifies all starting positions where the subsequence subseq is found. Both subseq and seq should be vectors of integers, typically obtained using codeunits on strings.
Arguments
  • subseq: A vector of integers representing the subsequence to search for.
  • seq: A vector of integers representing the larger sequence in which to search.
Returns
  • Vector{Int}: A vector of starting positions (1-based indices) where the subsequence is found in the sequence.
Examples
find_subsequence_positions(codeunits("ab"), codeunits("cababcab")) # Returns [2, 5]
source
PromptingTools.function_call_signature — Method
function_call_signature(datastructtype::Struct; max_description_length::Int = 100)
Extract the argument names, types and docstrings from a struct to create the function call signature in JSON schema.
You must provide a Struct type (not an instance of it) with some fields.
Note: Fairly experimental, but works for combination of structs, arrays, strings and singletons.
Tips
  • You can improve the quality of the extraction by writing a helpful docstring for your struct (or any nested struct). It will be provided as a description. 
You can even include comments/descriptions about the individual fields.
  • All fields are assumed to be required, unless you allow null values (eg, ::Union{Nothing, Int}). Fields with Nothing will be treated as optional.
  • Missing values are ignored (eg, ::Union{Missing, Int} will be treated as Int). It's for broader compatibility and we cannot deserialize it as easily as Nothing.
Example
Do you want to extract some specific measurements from a text like age, weight and height? You need to define the information you need as a struct (return_type):
struct MyMeasurement
+    kwargs...)
Finalizes the outputs of the ai* functions by either returning the conversation history or the last message.
Keyword arguments
  • return_all::Bool=false: If true, returns the entire conversation history, otherwise returns only the last message (the AIMessage).
  • dry_run::Bool=false: If true, does not send the messages to the model, but only renders the prompt with the given schema and replacement variables. Useful for debugging when you want to check the specific schema rendering. 
  • conversation::AbstractVector{<:AbstractMessage}=[]: An optional vector of AbstractMessage objects representing the conversation history. If not provided, it is initialized as an empty vector.
  • kwargs...: Variables to replace in the prompt template.
source
PromptingTools.find_subsequence_positions — Method
find_subsequence_positions(subseq, seq) -> Vector{Int}
Find all positions of a subsequence subseq within a larger sequence seq. Used to lookup positions of code blocks in markdown.
This function scans the sequence seq and identifies all starting positions where the subsequence subseq is found. Both subseq and seq should be vectors of integers, typically obtained using codeunits on strings.
Arguments
  • subseq: A vector of integers representing the subsequence to search for.
  • seq: A vector of integers representing the larger sequence in which to search.
Returns
  • Vector{Int}: A vector of starting positions (1-based indices) where the subsequence is found in the sequence.
Examples
find_subsequence_positions(codeunits("ab"), codeunits("cababcab")) # Returns [2, 5]
source
PromptingTools.function_call_signature — Method
function_call_signature(datastructtype::Struct; max_description_length::Int = 100)
Extract the argument names, types and docstrings from a struct to create the function call signature in JSON schema.
You must provide a Struct type (not an instance of it) with some fields.
Note: Fairly experimental, but works for combination of structs, arrays, strings and singletons.
Tips
  • You can improve the quality of the extraction by writing a helpful docstring for your struct (or any nested struct). It will be provided as a description. 
You can even include comments/descriptions about the individual fields.
  • All fields are assumed to be required, unless you allow null values (eg, ::Union{Nothing, Int}). Fields with Nothing will be treated as optional.
  • Missing values are ignored (eg, ::Union{Missing, Int} will be treated as Int). It's for broader compatibility and we cannot deserialize it as easily as Nothing.
Example
Do you want to extract some specific measurements from a text like age, weight and height? You need to define the information you need as a struct (return_type):
struct MyMeasurement
     age::Int
     height::Union{Int,Nothing}
     weight::Union{Nothing,Float64}
@@ -461,7 +489,7 @@
     measurements::Vector{MyMeasurement}
 end
 
-Or if you want your extraction to fail gracefully when data isn't found, use `MaybeExtract{T}` wrapper (inspired by Instructor package!):
using PromptingTools: MaybeExtract
type = MaybeExtract{MyMeasurement}
Effectively the same as:
struct MaybeExtract{T}
result::Union{T, Nothing}
error::Bool // true if a result is found, false otherwise
message::Union{Nothing, String} // Only present if no result is found, should be short and concise
end
If LLM extraction fails, it will return a Dict with error and message fields instead of the result!
msg = aiextract("Extract measurements from the text: I am giraffe", type)
Dict{Symbol, Any} with 2 entries:
:message => "Sorry, this feature is only available for humans."
:error   => true
``` That way, you can handle the error gracefully and get a reason why extraction failed.
source
PromptingTools.get_preferences — Method
get_preferences(key::String)
Get preferences for PromptingTools. See ?PREFERENCES for more information.
See also: set_preferences!
Example
PromptingTools.get_preferences("MODEL_CHAT")
source
PromptingTools.length_longest_common_subsequence — Method
length_longest_common_subsequence(itr1, itr2)
Compute the length of the longest common subsequence between two sequences (ie, the higher the number, the better the match).
Source: https://cn.julialang.org/LeetCode.jl/dev/democards/problems/problems/1143.longest-common-subsequence/
Arguments
  • itr1: The first sequence, eg, a String.
  • itr2: The second sequence, eg, a String.
Returns
The length of the longest common subsequence.
Examples
text1 = "abc-abc----"
+Or if you want your extraction to fail gracefully when data isn't found, use `MaybeExtract{T}` wrapper (inspired by Instructor package!):
using PromptingTools: MaybeExtract
type = MaybeExtract{MyMeasurement}
Effectively the same as:
struct MaybeExtract{T}
result::Union{T, Nothing}
error::Bool // true if a result is found, false otherwise
message::Union{Nothing, String} // Only present if no result is found, should be short and concise
end
If LLM extraction fails, it will return a Dict with error and message fields instead of the result!
msg = aiextract("Extract measurements from the text: I am giraffe", type)
Dict{Symbol, Any} with 2 entries:
:message => "Sorry, this feature is only available for humans."
:error   => true
``` That way, you can handle the error gracefully and get a reason why extraction failed.
source
PromptingTools.get_preferences — Method
get_preferences(key::String)
Get preferences for PromptingTools. See ?PREFERENCES for more information.
See also: set_preferences!
Example
PromptingTools.get_preferences("MODEL_CHAT")
source
PromptingTools.length_longest_common_subsequence — Method
length_longest_common_subsequence(itr1, itr2)
Compute the length of the longest common subsequence between two sequences (ie, the higher the number, the better the match).
Source: https://cn.julialang.org/LeetCode.jl/dev/democards/problems/problems/1143.longest-common-subsequence/
Arguments
  • itr1: The first sequence, eg, a String.
  • itr2: The second sequence, eg, a String.
Returns
The length of the longest common subsequence.
Examples
text1 = "abc-abc----"
 text2 = "___ab_c__abc"
 longest_common_subsequence(text1, text2)
 # Output: 6 (-> "abcabc")
It can be used to fuzzy match strings and find the similarity between them (Tip: normalize the match)
commands = ["product recommendation", "emotions", "specific product advice", "checkout advice"]
@@ -486,11 +514,11 @@
     dist = length_longest_common_subsequence(story, context[pos])
     norm = dist / min(length(story), length(context[pos]))
     @info "The closest context to the query: "$(first(story,20))..." is: "$(context[pos])" (distance: $(dist), normalized: $(norm))"
-end
source
PromptingTools.load_templates! — Function
load_templates!(dir_templates::Union{String, Nothing} = nothing;
     remember_path::Bool = true,
     remove_templates::Bool = isnothing(dir_templates),
     store::Dict{Symbol, <:Any} = TEMPLATE_STORE,
-    metadata_store::Vector{<:AITemplateMetadata} = TEMPLATE_METADATA)
Loads templates from folder templates/ in the package root and stores them in TEMPLATE_STORE and TEMPLATE_METADATA.
Note: Automatically removes any existing templates and metadata from TEMPLATE_STORE and TEMPLATE_METADATA if remove_templates=true.
Arguments
  • dir_templates::Union{String, Nothing}: The directory path to load templates from. If nothing, uses the default list of paths. It usually used only once "to register" a new template storage.
  • remember_path::Bool=true: If true, remembers the path for future refresh (in TEMPLATE_PATH).
  • remove_templates::Bool=isnothing(dir_templates): If true, removes any existing templates and metadata from store and metadata_store.
  • store::Dict{Symbol, <:Any}=TEMPLATE_STORE: The store to load the templates into.
  • metadata_store::Vector{<:AITemplateMetadata}=TEMPLATE_METADATA: The metadata store to load the metadata into.
Example
Load the default templates:
PT.load_templates!() # no path needed
Load templates from a new custom path:
PT.load_templates!("path/to/templates") # we will remember this path for future refresh
If you want to now refresh the default templates and the new path, just call load_templates!() without any arguments.
source
PromptingTools.ollama_api — Function
ollama_api(prompt_schema::Union{AbstractOllamaManagedSchema, AbstractOllamaSchema},
+    metadata_store::Vector{<:AITemplateMetadata} = TEMPLATE_METADATA)
Loads templates from folder templates/ in the package root and stores them in TEMPLATE_STORE and TEMPLATE_METADATA.
Note: Automatically removes any existing templates and metadata from TEMPLATE_STORE and TEMPLATE_METADATA if remove_templates=true.
Arguments
  • dir_templates::Union{String, Nothing}: The directory path to load templates from. If nothing, uses the default list of paths. It usually used only once "to register" a new template storage.
  • remember_path::Bool=true: If true, remembers the path for future refresh (in TEMPLATE_PATH).
  • remove_templates::Bool=isnothing(dir_templates): If true, removes any existing templates and metadata from store and metadata_store.
  • store::Dict{Symbol, <:Any}=TEMPLATE_STORE: The store to load the templates into.
  • metadata_store::Vector{<:AITemplateMetadata}=TEMPLATE_METADATA: The metadata store to load the metadata into.
Example
Load the default templates:
PT.load_templates!() # no path needed
Load templates from a new custom path:
PT.load_templates!("path/to/templates") # we will remember this path for future refresh
If you want to now refresh the default templates and the new path, just call load_templates!() without any arguments.
source
PromptingTools.ollama_api — Function
ollama_api(prompt_schema::Union{AbstractOllamaManagedSchema, AbstractOllamaSchema},
     prompt::Union{AbstractString, Nothing} = nothing;
     system::Union{Nothing, AbstractString} = nothing,
     messages::Vector{<:AbstractMessage} = AbstractMessage[],
@@ -498,38 +526,38 @@
     model::String = "llama2", http_kwargs::NamedTuple = NamedTuple(),
     stream::Bool = false,
     url::String = "localhost", port::Int = 11434,
-    kwargs...)
Simple wrapper for a call to Ollama API.
Keyword Arguments
  • prompt_schema: Defines which prompt template should be applied.
  • prompt: Can be a string representing the prompt for the AI conversation, a UserMessage, a vector of AbstractMessage
  • system: An optional string representing the system message for the AI conversation. If not provided, a default message will be used.
  • endpoint: The API endpoint to call, only "generate" and "embeddings" are currently supported. Defaults to "generate".
  • model: A string representing the model to use for generating the response. Can be an alias corresponding to a model ID defined in MODEL_ALIASES.
  • http_kwargs::NamedTuple: Additional keyword arguments for the HTTP request. Defaults to empty NamedTuple.
  • stream: A boolean indicating whether to stream the response. Defaults to false.
  • url: The URL of the Ollama API. Defaults to "localhost".
  • port: The port of the Ollama API. Defaults to 11434.
  • kwargs: Prompt variables to be used to fill the prompt/template
source
PromptingTools.preview — Function
Utility for rendering the conversation (vector of messages) as markdown. REQUIRES the Markdown package to load the extension!
source
PromptingTools.push_conversation! — Method
push_conversation!(conv_history, conversation::AbstractVector, max_history::Union{Int, Nothing})
Add a new conversation to the conversation history and resize the history if necessary.
This function appends a conversation to the conv_history, which is a vector of conversations. Each conversation is represented as a vector of AbstractMessage objects. After adding the new conversation, the history is resized according to the max_history parameter to ensure that the size of the history does not exceed the specified limit.
Arguments
  • conv_history: A vector that stores the history of conversations. Typically, this is PT.CONV_HISTORY.
  • conversation: The new conversation to be added. It should be a vector of AbstractMessage objects.
  • max_history: The maximum number of conversations to retain in the history. If Nothing, the history is not resized.
Returns
The updated conversation history.
Example
new_conversation = aigenerate("Hello World"; return_all = true)
-push_conversation!(PT.CONV_HISTORY, new_conversation, 10)
This is done automatically by the ai"" macros.
source
PromptingTools.register_model! — Function
register_model!(registry = MODEL_REGISTRY;
+    kwargs...)
Simple wrapper for a call to Ollama API.
Keyword Arguments
  • prompt_schema: Defines which prompt template should be applied.
  • prompt: Can be a string representing the prompt for the AI conversation, a UserMessage, a vector of AbstractMessage
  • system: An optional string representing the system message for the AI conversation. If not provided, a default message will be used.
  • endpoint: The API endpoint to call, only "generate" and "embeddings" are currently supported. Defaults to "generate".
  • model: A string representing the model to use for generating the response. Can be an alias corresponding to a model ID defined in MODEL_ALIASES.
  • http_kwargs::NamedTuple: Additional keyword arguments for the HTTP request. Defaults to empty NamedTuple.
  • stream: A boolean indicating whether to stream the response. Defaults to false.
  • url: The URL of the Ollama API. Defaults to "localhost".
  • port: The port of the Ollama API. Defaults to 11434.
  • kwargs: Prompt variables to be used to fill the prompt/template
source
PromptingTools.preview — Function
Utility for rendering the conversation (vector of messages) as markdown. REQUIRES the Markdown package to load the extension!
source
PromptingTools.push_conversation! — Method
push_conversation!(conv_history, conversation::AbstractVector, max_history::Union{Int, Nothing})
Add a new conversation to the conversation history and resize the history if necessary.
This function appends a conversation to the conv_history, which is a vector of conversations. Each conversation is represented as a vector of AbstractMessage objects. After adding the new conversation, the history is resized according to the max_history parameter to ensure that the size of the history does not exceed the specified limit.
Arguments
  • conv_history: A vector that stores the history of conversations. Typically, this is PT.CONV_HISTORY.
  • conversation: The new conversation to be added. It should be a vector of AbstractMessage objects.
  • max_history: The maximum number of conversations to retain in the history. If Nothing, the history is not resized.
Returns
The updated conversation history.
Example
new_conversation = aigenerate("Hello World"; return_all = true)
+push_conversation!(PT.CONV_HISTORY, new_conversation, 10)
This is done automatically by the ai"" macros.
source
PromptingTools.register_model! — Function
register_model!(registry = MODEL_REGISTRY;
     name::String,
     schema::Union{AbstractPromptSchema, Nothing} = nothing,
     cost_of_token_prompt::Float64 = 0.0,
     cost_of_token_generation::Float64 = 0.0,
-    description::String = "")
Register a new AI model with name and its associated schema. 
Registering a model helps with calculating the costs and automatically selecting the right prompt schema.
Arguments
  • name: The name of the model. This is the name that will be used to refer to the model in the ai* functions.
  • schema: The schema of the model. This is the schema that will be used to generate prompts for the model, eg, OpenAISchema().
  • cost_of_token_prompt: The cost of a token in the prompt for this model. This is used to calculate the cost of a prompt.   Note: It is often provided online as cost per 1000 tokens, so make sure to convert it correctly!
  • cost_of_token_generation: The cost of a token generated by this model. This is used to calculate the cost of a generation.   Note: It is often provided online as cost per 1000 tokens, so make sure to convert it correctly!
  • description: A description of the model. This is used to provide more information about the model when it is queried.
source
PromptingTools.remove_julia_prompt — Method
remove_julia_prompt(s::T) where {T<:AbstractString}
If it detects a julia prompt, it removes it and all lines that do not have it (except for those that belong to the code block).
source
PromptingTools.render — Method
render(schema::AbstractGoogleSchema,
+    description::String = "")
Register a new AI model with name and its associated schema. 
Registering a model helps with calculating the costs and automatically selecting the right prompt schema.
Arguments
  • name: The name of the model. This is the name that will be used to refer to the model in the ai* functions.
  • schema: The schema of the model. This is the schema that will be used to generate prompts for the model, eg, OpenAISchema().
  • cost_of_token_prompt: The cost of a token in the prompt for this model. This is used to calculate the cost of a prompt.   Note: It is often provided online as cost per 1000 tokens, so make sure to convert it correctly!
  • cost_of_token_generation: The cost of a token generated by this model. This is used to calculate the cost of a generation.   Note: It is often provided online as cost per 1000 tokens, so make sure to convert it correctly!
  • description: A description of the model. This is used to provide more information about the model when it is queried.
source
PromptingTools.remove_julia_prompt — Method
remove_julia_prompt(s::T) where {T<:AbstractString}
If it detects a julia prompt, it removes it and all lines that do not have it (except for those that belong to the code block).
source
PromptingTools.render — Method
render(schema::AbstractGoogleSchema,
     messages::Vector{<:AbstractMessage};
     conversation::AbstractVector{<:AbstractMessage} = AbstractMessage[],
-    kwargs...)
Builds a history of the conversation to provide the prompt to the API. All unspecified kwargs are passed as replacements such that {{key}}=>value in the template.
Keyword Arguments
  • conversation: An optional vector of AbstractMessage objects representing the conversation history. If not provided, it is initialized as an empty vector.
source
PromptingTools.render — Method
render(schema::AbstractOllamaManagedSchema,
+    kwargs...)
Builds a history of the conversation to provide the prompt to the API. All unspecified kwargs are passed as replacements such that {{key}}=>value in the template.
Keyword Arguments
  • conversation: An optional vector of AbstractMessage objects representing the conversation history. If not provided, it is initialized as an empty vector.
source
PromptingTools.render — Method
render(schema::AbstractOllamaManagedSchema,
     messages::Vector{<:AbstractMessage};
     conversation::AbstractVector{<:AbstractMessage} = AbstractMessage[],
-    kwargs...)
Builds a history of the conversation to provide the prompt to the API. All unspecified kwargs are passed as replacements such that {{key}}=>value in the template.
Note: Due to its "managed" nature, at most 2 messages can be provided (system and prompt inputs in the API).
Keyword Arguments
  • conversation: Not allowed for this schema. Provided only for compatibility.
source
PromptingTools.render — Method
render(schema::AbstractOllamaSchema,
+    kwargs...)
Builds a history of the conversation to provide the prompt to the API. All unspecified kwargs are passed as replacements such that {{key}}=>value in the template.
Note: Due to its "managed" nature, at most 2 messages can be provided (system and prompt inputs in the API).
Keyword Arguments
  • conversation: Not allowed for this schema. Provided only for compatibility.
source
PromptingTools.render — Method
render(schema::AbstractOllamaSchema,
     messages::Vector{<:AbstractMessage};
     conversation::AbstractVector{<:AbstractMessage} = AbstractMessage[],
-    kwargs...)
Builds a history of the conversation to provide the prompt to the API. All unspecified kwargs are passed as replacements such that {{key}}=>value in the template.
Keyword Arguments
  • conversation: An optional vector of AbstractMessage objects representing the conversation history. If not provided, it is initialized as an empty vector.
source
PromptingTools.render — Method
render(schema::AbstractOpenAISchema,
+    kwargs...)
Builds a history of the conversation to provide the prompt to the API. All unspecified kwargs are passed as replacements such that {{key}}=>value in the template.
Keyword Arguments
  • conversation: An optional vector of AbstractMessage objects representing the conversation history. If not provided, it is initialized as an empty vector.
source
PromptingTools.render — Method
render(schema::AbstractOpenAISchema,
     messages::Vector{<:AbstractMessage};
     image_detail::AbstractString = "auto",
     conversation::AbstractVector{<:AbstractMessage} = AbstractMessage[],
-    kwargs...)
Builds a history of the conversation to provide the prompt to the API. All unspecified kwargs are passed as replacements such that {{key}}=>value in the template.
Keyword Arguments
  • image_detail: Only for UserMessageWithImages. It represents the level of detail to include for images. Can be "auto", "high", or "low".
  • conversation: An optional vector of AbstractMessage objects representing the conversation history. If not provided, it is initialized as an empty vector.
source
PromptingTools.render — Method
render(schema::NoSchema,
+    kwargs...)
Builds a history of the conversation to provide the prompt to the API. All unspecified kwargs are passed as replacements such that {{key}}=>value in the template.
Keyword Arguments
  • image_detail: Only for UserMessageWithImages. It represents the level of detail to include for images. Can be "auto", "high", or "low".
  • conversation: An optional vector of AbstractMessage objects representing the conversation history. If not provided, it is initialized as an empty vector.
source
PromptingTools.render — Method
render(schema::NoSchema,
     messages::Vector{<:AbstractMessage};
     conversation::AbstractVector{<:AbstractMessage} = AbstractMessage[],
-    replacement_kwargs...)
Renders a conversation history from a vector of messages with all replacement variables specified in replacement_kwargs.
It is the first pass of the prompt rendering system, and is used by all other schemas.
Keyword Arguments
  • image_detail: Only for UserMessageWithImages. It represents the level of detail to include for images. Can be "auto", "high", or "low".
  • conversation: An optional vector of AbstractMessage objects representing the conversation history. If not provided, it is initialized as an empty vector.
Notes
  • All unspecified kwargs are passed as replacements such that {{key}}=>value in the template.
  • If a SystemMessage is missing, we inject a default one at the beginning of the conversation.
  • Only one SystemMessage is allowed (ie, cannot mix two conversations different system prompts).
source
PromptingTools.replace_words — Method
replace_words(text::AbstractString, words::Vector{<:AbstractString}; replacement::AbstractString="ABC")
Replace all occurrences of words in words with replacement in text. Useful to quickly remove specific names or entities from a text.
Arguments
  • text::AbstractString: The text to be processed.
  • words::Vector{<:AbstractString}: A vector of words to be replaced.
  • replacement::AbstractString="ABC": The replacement string to be used. Defaults to "ABC".
Example
text = "Disney is a great company"
+    replacement_kwargs...)
Renders a conversation history from a vector of messages with all replacement variables specified in replacement_kwargs.
It is the first pass of the prompt rendering system, and is used by all other schemas.
Keyword Arguments
  • image_detail: Only for UserMessageWithImages. It represents the level of detail to include for images. Can be "auto", "high", or "low".
  • conversation: An optional vector of AbstractMessage objects representing the conversation history. If not provided, it is initialized as an empty vector.
Notes
  • All unspecified kwargs are passed as replacements such that {{key}}=>value in the template.
  • If a SystemMessage is missing, we inject a default one at the beginning of the conversation.
  • Only one SystemMessage is allowed (ie, cannot mix two conversations different system prompts).
source
PromptingTools.replace_words — Method
replace_words(text::AbstractString, words::Vector{<:AbstractString}; replacement::AbstractString="ABC")
Replace all occurrences of words in words with replacement in text. Useful to quickly remove specific names or entities from a text.
Arguments
  • text::AbstractString: The text to be processed.
  • words::Vector{<:AbstractString}: A vector of words to be replaced.
  • replacement::AbstractString="ABC": The replacement string to be used. Defaults to "ABC".
Example
text = "Disney is a great company"
 replace_words(text, ["Disney", "Snow White", "Mickey Mouse"])
-# Output: "ABC is a great company"
source
PromptingTools.resize_conversation! — Method
resize_conversation!(conv_history, max_history::Union{Int, Nothing})
Resize the conversation history to a specified maximum length.
This function trims the conv_history to ensure that its size does not exceed max_history. It removes the oldest conversations first if the length of conv_history is greater than max_history.
Arguments
  • conv_history: A vector that stores the history of conversations. Typically, this is PT.CONV_HISTORY.
  • max_history: The maximum number of conversations to retain in the history. If Nothing, the history is not resized.
Returns
The resized conversation history.
Example
resize_conversation!(PT.CONV_HISTORY, PT.MAX_HISTORY_LENGTH)
After the function call, conv_history will contain only the 10 most recent conversations.
This is done automatically by the ai"" macros.
source
PromptingTools.resize_conversation! — Method
resize_conversation!(conv_history, max_history::Union{Int, Nothing})
Resize the conversation history to a specified maximum length.
This function trims the conv_history to ensure that its size does not exceed max_history. It removes the oldest conversations first if the length of conv_history is greater than max_history.
Arguments
  • conv_history: A vector that stores the history of conversations. Typically, this is PT.CONV_HISTORY.
  • max_history: The maximum number of conversations to retain in the history. If Nothing, the history is not resized.
Returns
The resized conversation history.
Example
resize_conversation!(PT.CONV_HISTORY, PT.MAX_HISTORY_LENGTH)
After the function call, conv_history will contain only the 10 most recent conversations.
This is done automatically by the ai"" macros.
source
PromptingTools.response_to_message — Method
response_to_message(schema::AbstractOpenAISchema,
     MSG::Type{AIMessage},
     choice,
     resp;
     model_id::AbstractString = "",
     time::Float64 = 0.0,
     run_id::Integer = rand(Int16),
-    sample_id::Union{Nothing, Integer} = nothing)
Utility to facilitate unwrapping of HTTP response to a message type MSG provided for OpenAI-like responses
Note: Extracts finish_reason and log_prob if available in the response.
Arguments
  • schema::AbstractOpenAISchema: The schema for the prompt.
  • MSG::Type{AIMessage}: The message type to be returned.
  • choice: The choice from the response (eg, one of the completions).
  • resp: The response from the OpenAI API.
  • model_id::AbstractString: The model ID to use for generating the response. Defaults to an empty string.
  • time::Float64: The elapsed time for the response. Defaults to 0.0.
  • run_id::Integer: The run ID for the response. Defaults to a random integer.
  • sample_id::Union{Nothing, Integer}: The sample ID for the response (if there are multiple completions). Defaults to nothing.
source
PromptingTools.set_preferences! — Method
set_preferences!(pairs::Pair{String, <:Any}...)
Set preferences for PromptingTools. See ?PREFERENCES for more information. 
See also: get_preferences
Example
Change your API key and default model:
PromptingTools.set_preferences!("OPENAI_API_KEY" => "key1", "MODEL_CHAT" => "chat1")
source
PromptingTools.split_by_length — Method
split_by_length(text::String, separators::Vector{String}; max_length::Int=35000) -> Vector{String}
Split a given string text into chunks using a series of separators, with each chunk having a maximum length of max_length.  This function is useful for splitting large documents or texts into smaller segments that are more manageable for processing, particularly for models or systems with limited context windows.
Arguments
  • text::String: The text to be split.
  • separators::Vector{String}: An ordered list of separators used to split the text. The function iteratively applies these separators to split the text.
  • max_length::Int=35000: The maximum length of each chunk. Defaults to 35,000 characters. This length is considered after each iteration of splitting, ensuring chunks fit within specified constraints.
Returns
Vector{String}: A vector of strings, where each string is a chunk of the original text that is smaller than or equal to max_length.
Notes
  • The function processes the text iteratively with each separator in the provided order. This ensures more nuanced splitting, especially in structured texts.
  • Each chunk is as close to max_length as possible without exceeding it (unless we cannot split it any further)
  • If the text is empty, the function returns an empty array.
  • Separators are re-added to the text chunks after splitting, preserving the original structure of the text as closely as possible. Apply strip if you do not need them.
Examples
Splitting text using multiple separators:
text = "Paragraph 1
+    sample_id::Union{Nothing, Integer} = nothing)
Utility to facilitate unwrapping of HTTP response to a message type MSG provided for OpenAI-like responses
Note: Extracts finish_reason and log_prob if available in the response.
Arguments
  • schema::AbstractOpenAISchema: The schema for the prompt.
  • MSG::Type{AIMessage}: The message type to be returned.
  • choice: The choice from the response (eg, one of the completions).
  • resp: The response from the OpenAI API.
  • model_id::AbstractString: The model ID to use for generating the response. Defaults to an empty string.
  • time::Float64: The elapsed time for the response. Defaults to 0.0.
  • run_id::Integer: The run ID for the response. Defaults to a random integer.
  • sample_id::Union{Nothing, Integer}: The sample ID for the response (if there are multiple completions). Defaults to nothing.
source
PromptingTools.set_preferences! — Method
set_preferences!(pairs::Pair{String, <:Any}...)
Set preferences for PromptingTools. See ?PREFERENCES for more information. 
See also: get_preferences
Example
Change your API key and default model:
PromptingTools.set_preferences!("OPENAI_API_KEY" => "key1", "MODEL_CHAT" => "chat1")
source
PromptingTools.split_by_length — Method
split_by_length(text::String, separators::Vector{String}; max_length::Int=35000) -> Vector{String}
Split a given string text into chunks using a series of separators, with each chunk having a maximum length of max_length.  This function is useful for splitting large documents or texts into smaller segments that are more manageable for processing, particularly for models or systems with limited context windows.
Arguments
  • text::String: The text to be split.
  • separators::Vector{String}: An ordered list of separators used to split the text. The function iteratively applies these separators to split the text.
  • max_length::Int=35000: The maximum length of each chunk. Defaults to 35,000 characters. This length is considered after each iteration of splitting, ensuring chunks fit within specified constraints.
Returns
Vector{String}: A vector of strings, where each string is a chunk of the original text that is smaller than or equal to max_length.
Notes
  • The function processes the text iteratively with each separator in the provided order. This ensures more nuanced splitting, especially in structured texts.
  • Each chunk is as close to max_length as possible without exceeding it (unless we cannot split it any further)
  • If the text is empty, the function returns an empty array.
  • Separators are re-added to the text chunks after splitting, preserving the original structure of the text as closely as possible. Apply strip if you do not need them.
Examples
Splitting text using multiple separators:
text = "Paragraph 1
 
 Paragraph 2. Sentence 1. Sentence 2.
 Paragraph 3"
@@ -538,24 +566,24 @@
 ", ". ", "
 "]
 chunks = split_by_length(text, separators, max_length=20)
Using a single separator:
text = "Hello,World," ^ 2900  # length 34900 characters
-chunks = split_by_length(text, [","], max_length=10000)
source
PromptingTools.split_by_length — Method
split_by_length(text::String; separator::String=" ", max_length::Int=35000) -> Vector{String}
Split a given string text into chunks of a specified maximum length max_length.  This is particularly useful for splitting larger documents or texts into smaller segments, suitable for models or systems with smaller context windows.
Arguments
  • text::String: The text to be split.
  • separator::String=" ": The separator used to split the text into minichunks. Defaults to a space character.
  • max_length::Int=35000: The maximum length of each chunk. Defaults to 35,000 characters, which should fit within 16K context window.
Returns
Vector{String}: A vector of strings, each representing a chunk of the original text that is smaller than or equal to max_length.
Notes
  • The function ensures that each chunk is as close to max_length as possible without exceeding it.
  • If the text is empty, the function returns an empty array.
  • The separator is re-added to the text chunks after splitting, preserving the original structure of the text as closely as possible.
Examples
Splitting text with the default separator (" "):
text = "Hello world. How are you?"
+chunks = split_by_length(text, [","], max_length=10000)
source
PromptingTools.split_by_length — Method
split_by_length(text::String; separator::String=" ", max_length::Int=35000) -> Vector{String}
Split a given string text into chunks of a specified maximum length max_length.  This is particularly useful for splitting larger documents or texts into smaller segments, suitable for models or systems with smaller context windows.
Arguments
  • text::String: The text to be split.
  • separator::String=" ": The separator used to split the text into minichunks. Defaults to a space character.
  • max_length::Int=35000: The maximum length of each chunk. Defaults to 35,000 characters, which should fit within 16K context window.
Returns
Vector{String}: A vector of strings, each representing a chunk of the original text that is smaller than or equal to max_length.
Notes
  • The function ensures that each chunk is as close to max_length as possible without exceeding it.
  • If the text is empty, the function returns an empty array.
  • The separator is re-added to the text chunks after splitting, preserving the original structure of the text as closely as possible.
Examples
Splitting text with the default separator (" "):
text = "Hello world. How are you?"
 chunks = split_by_length(text; max_length=13)
 length(chunks) # Output: 2
Using a custom separator and custom max_length
text = "Hello,World," ^ 2900 # length 34900 chars
 split_by_length(text; separator=",", max_length=10000) # for 4K context window
-length(chunks[1]) # Output: 4
source
PromptingTools.@aai_str — Macro
aai"user_prompt"[model_alias] -> AIMessage
Asynchronous version of @ai_str macro, which will log the result once it's ready.
See also aai!"" if you want an asynchronous reply to the provided message / continue the conversation.    
Example
Send asynchronous request to GPT-4, so we don't have to wait for the response: Very practical with slow models, so you can keep working in the meantime.
```julia m = aai"Say Hi!"gpt4; 
...with some delay...
[ Info: Tokens: 29 @ Cost: 0.0011 in 2.7 seconds
[ Info: AIMessage> Hello! How can I assist you today?
source
PromptingTools.@ai!_str — Macro
ai!"user_prompt"[model_alias] -> AIMessage
The ai!"" string macro is used to continue a previous conversation with the AI model. 
It appends the new user prompt to the last conversation in the tracked history (in PromptingTools.CONV_HISTORY) and generates a response based on the entire conversation context. If you want to see the previous conversation, you can access it via PromptingTools.CONV_HISTORY, which keeps at most last PromptingTools.MAX_HISTORY_LENGTH conversations.
Arguments
  • user_prompt (String): The new input prompt to be added to the existing conversation.
  • model_alias (optional, any): Specify the model alias of the AI model to be used (see MODEL_ALIASES). If not provided, the default model is used.
Returns
AIMessage corresponding to the new user prompt, considering the entire conversation history.
Example
To continue a conversation:
# start conversation as normal
+length(chunks[1]) # Output: 4
source
PromptingTools.@aai_str — Macro
aai"user_prompt"[model_alias] -> AIMessage
Asynchronous version of @ai_str macro, which will log the result once it's ready.
See also aai!"" if you want an asynchronous reply to the provided message / continue the conversation.    
Example
Send asynchronous request to GPT-4, so we don't have to wait for the response: Very practical with slow models, so you can keep working in the meantime.
```julia m = aai"Say Hi!"gpt4; 
...with some delay...
[ Info: Tokens: 29 @ Cost: 0.0011 in 2.7 seconds
[ Info: AIMessage> Hello! How can I assist you today?
source
PromptingTools.@ai!_str — Macro
ai!"user_prompt"[model_alias] -> AIMessage
The ai!"" string macro is used to continue a previous conversation with the AI model. 
It appends the new user prompt to the last conversation in the tracked history (in PromptingTools.CONV_HISTORY) and generates a response based on the entire conversation context. If you want to see the previous conversation, you can access it via PromptingTools.CONV_HISTORY, which keeps at most last PromptingTools.MAX_HISTORY_LENGTH conversations.
Arguments
  • user_prompt (String): The new input prompt to be added to the existing conversation.
  • model_alias (optional, any): Specify the model alias of the AI model to be used (see MODEL_ALIASES). If not provided, the default model is used.
Returns
AIMessage corresponding to the new user prompt, considering the entire conversation history.
Example
To continue a conversation:
# start conversation as normal
 ai"Say hi." 
 
 # ... wait for reply and then react to it:
 
 # continue the conversation (notice that you can change the model, eg, to more powerful one for better answer)
 ai!"What do you think about that?"gpt4t
-# AIMessage("Considering our previous discussion, I think that...")
Usage Notes
  • This macro should be used when you want to maintain the context of an ongoing conversation (ie, the last ai"" message).
  • It automatically accesses and updates the global conversation history.
  • If no conversation history is found, it raises an assertion error, suggesting to initiate a new conversation using ai"" instead.
Important
Ensure that the conversation history is not too long to maintain relevancy and coherence in the AI's responses. The history length is managed by MAX_HISTORY_LENGTH.
source
PromptingTools.@ai_str — Macro
ai"user_prompt"[model_alias] -> AIMessage
The ai"" string macro generates an AI response to a given prompt by using aigenerate under the hood.
See also ai!"" if you want to reply to the provided message / continue the conversation.
Arguments
  • user_prompt (String): The input prompt for the AI model.
  • model_alias (optional, any): Provide model alias of the AI model (see MODEL_ALIASES).
Returns
AIMessage corresponding to the input prompt.
Example
result = ai"Hello, how are you?"
+# AIMessage("Considering our previous discussion, I think that...")
Usage Notes
  • This macro should be used when you want to maintain the context of an ongoing conversation (ie, the last ai"" message).
  • It automatically accesses and updates the global conversation history.
  • If no conversation history is found, it raises an assertion error, suggesting to initiate a new conversation using ai"" instead.
Important
Ensure that the conversation history is not too long to maintain relevancy and coherence in the AI's responses. The history length is managed by MAX_HISTORY_LENGTH.
source
PromptingTools.@ai_str — Macro
ai"user_prompt"[model_alias] -> AIMessage
The ai"" string macro generates an AI response to a given prompt by using aigenerate under the hood.
See also ai!"" if you want to reply to the provided message / continue the conversation.
Arguments
  • user_prompt (String): The input prompt for the AI model.
  • model_alias (optional, any): Provide model alias of the AI model (see MODEL_ALIASES).
Returns
AIMessage corresponding to the input prompt.
Example
result = ai"Hello, how are you?"
 # AIMessage("Hello! I'm an AI assistant, so I don't have feelings, but I'm here to help you. How can I assist you today?")
If you want to interpolate some variables or additional context, simply use string interpolation:
a=1
 result = ai"What is `$a+$a`?"
 # AIMessage("The sum of `1+1` is `2`.")
If you want to use a different model, eg, GPT-4, you can provide its alias as a flag:
result = ai"What is `1.23 * 100 + 1`?"gpt4t
-# AIMessage("The answer is 124.")
source
PromptingTools.@timeout — Macro
@timeout(seconds, expr_to_run, expr_when_fails)
Simple macro to run an expression with a timeout of seconds. If the expr_to_run fails to finish in seconds seconds, expr_when_fails is returned.
Example
x = @timeout 1 begin
+# AIMessage("The answer is 124.")
source
PromptingTools.@timeout — Macro
@timeout(seconds, expr_to_run, expr_when_fails)
Simple macro to run an expression with a timeout of seconds. If the expr_to_run fails to finish in seconds seconds, expr_when_fails is returned.
Example
x = @timeout 1 begin
     sleep(1.1)
     println("done")
     1
 end "failed"
-
source

+
source
diff --git a/dev/reference_agenttools/index.html b/dev/reference_agenttools/index.html index a17633f8b..818651b65 100644 --- a/dev/reference_agenttools/index.html +++ b/dev/reference_agenttools/index.html @@ -1,5 +1,5 @@ -AgentTools · PromptingTools.jl
GitHub

Reference for AgentTools

PromptingTools.Experimental.AgentTools — Module
AgentTools

Provides Agentic functionality providing lazy calls for building pipelines (eg, AIGenerate) and AICodeFixer.

This module is experimental and may change at any time. It is intended to be moved to a separate package in the future.

source
PromptingTools.Experimental.AgentTools.AICall — Type
AICall(func::F, args...; kwargs...) where {F<:Function}
+AgentTools · PromptingTools.jl
GitHub
Reference for AgentTools
PromptingTools.Experimental.AgentTools — Module
AgentTools
Provides Agentic functionality providing lazy calls for building pipelines (eg, AIGenerate) and AICodeFixer.
This module is experimental and may change at any time. It is intended to be moved to a separate package in the future.
source
PromptingTools.Experimental.AgentTools.AICall — Type
AICall(func::F, args...; kwargs...) where {F<:Function}
 
 AIGenerate(args...; kwargs...)
 AIEmbed(args...; kwargs...)
@@ -14,7 +14,7 @@
 aicall = AIGenerate(:JuliaExpertAsk; ask="xyz", model="abc", api_kwargs=(; temperature=0.1))
Trigger the AICall with run! (it returns the update AICall struct back):
aicall |> run!
 ````
 
-You can also use `AICall` as a functor to trigger the AI call with a `UserMessage` or simply the text to send:
julia aicall(UserMessage("Hello, world!"))  # Triggers the lazy call result = run!(aicall)  # Explicitly runs the AI call ``` This can be used to "reply" to previous message / continue the stored conversation
Notes
  • The AICall struct is a key component in building flexible and efficient Agentic pipelines
  • The lazy evaluation model allows for setting up the call parameters in advance and deferring the actual execution until it is explicitly triggered.
  • This struct is particularly useful in scenarios where the timing of AI function execution needs to be deferred or where multiple potential calls need to be prepared and selectively executed.
source
PromptingTools.Experimental.AgentTools.AICodeFixer — Type
AICodeFixer(aicall::AICall, templates::Vector{<:PT.UserMessage}; num_rounds::Int = 3, feedback_func::Function = aicodefixer_feedback; kwargs...)
+You can also use `AICall` as a functor to trigger the AI call with a `UserMessage` or simply the text to send:
julia aicall(UserMessage("Hello, world!"))  # Triggers the lazy call result = run!(aicall)  # Explicitly runs the AI call ``` This can be used to "reply" to previous message / continue the stored conversation
Notes
  • The AICall struct is a key component in building flexible and efficient Agentic pipelines
  • The lazy evaluation model allows for setting up the call parameters in advance and deferring the actual execution until it is explicitly triggered.
  • This struct is particularly useful in scenarios where the timing of AI function execution needs to be deferred or where multiple potential calls need to be prepared and selectively executed.
source
PromptingTools.Experimental.AgentTools.AICodeFixer — Type
AICodeFixer(aicall::AICall, templates::Vector{<:PT.UserMessage}; num_rounds::Int = 3, feedback_func::Function = aicodefixer_feedback; kwargs...)
 AICodeFixer(aicall::AICall, template::Union{AITemplate, Symbol} = :CodeFixerRCI; kwargs...)
An AIAgent that iteratively evaluates any received Julia code and provides feedback back to the AI model if num_rounds>0. AICodeFixer manages the lifecycle of a code fixing session, including tracking conversation history, rounds of interaction, and applying user feedback through a specialized feedback function.
It integrates with lazy AI call structures like AIGenerate. 
The operation is "lazy", ie, the agent is only executed when needed, eg, when run! is called.
Fields
  • call::AICall: The AI call that is being used for code generation or processing, eg, AIGenerate (same as aigenerate but "lazy", ie, called only when needed
  • templates::Union{Symbol, AITemplate, Vector{PT.UserMessage}}: A set of user messages or templates that guide the AI's code fixing process.  The first UserMessage is used in the first round of code fixing, the second UserMessage is used for every subsequent iteration.
  • num_rounds::Int: The number of rounds for the code fixing session. Defaults to 3.
  • round_counter::Int: Counter to track the current round of interaction.
  • feedback_func::Function: Function to generate feedback based on the AI's proposed code, defaults to aicodefixer_feedback  (modular thanks to type dispatch on AbstractOutcomes)
  • kwargs::NamedTuple: Additional keyword arguments for customizing the AI call.
Note: Any kwargs provided to run!() will be passed to the underlying AICall.
Example
Let's create an AIGenerate call and then pipe it to AICodeFixer to run a few rounds of the coding fixing:
# Create an AIGenerate call
 lazy_call = AIGenerate("Write a function to do XYZ...")
 
@@ -36,7 +36,7 @@
 Feedback: {{feedback}}")]; num_rounds = 2) |> run!
 
 # The result now contains the AI's attempts to fix the code
-preview(result.call.conversation)
Notes
  • AICodeFixer is particularly useful when code is hard to get right in one shot (eg, smaller models, complex syntax)
  • The structure leverages the lazy evaluation model of AICall (/AIGenerate) to efficiently manage AI interactions and be able to repeatedly call it.
  • The run! function executes the AI call and applies the feedback loop for the specified number of rounds, enabling an interactive code fixing process.
source
PromptingTools.Experimental.AgentTools.RetryConfig — Type
RetryConfig
Configuration for self-fixing the AI calls. It includes the following fields:
Fields
  • retries::Int: The number of retries ("fixing rounds") that have been attempted so far.
  • calls::Int: The total number of SUCCESSFULLY generated ai* function calls made so far (across all samples/retry rounds). Ie, if a call fails, because of an API error, it's not counted, because it didn't reach the LLM.
  • max_retries::Int: The maximum number of retries ("fixing rounds") allowed for the AI call. Defaults to 10.
  • max_calls::Int: The maximum number of ai* function calls allowed for the AI call. Defaults to 99.
  • retry_delay::Int: The delay (in seconds) between retry rounds. Defaults to 0s.
  • n_samples::Int: The number of samples to generate in each ai* call round (to increase changes of successful pass). Defaults to 1.
  • scoring::AbstractScoringMethod: The scoring method to use for generating multiple samples. Defaults to UCT(sqrt(2)).
  • ordering::Symbol: The ordering to use for select the best samples. With :PostOrderDFS we prioritize leaves, with :PreOrderDFS we prioritize the root. Defaults to :PostOrderDFS.
  • feedback_inplace::Bool: Whether to provide feedback in previous UserMessage (and remove the past AIMessage) or to create a new UserMessage. Defaults to false.
  • feedback_template::Symbol: Template to use for feedback in place. Defaults to :FeedbackFromEvaluator.
  • temperature::Float64: The temperature to use for sampling. Relevant only if not defined in api_kwargs provided. Defaults to 0.7.
  • catch_errors::Bool: Whether to catch errors during run! of AICall. Saves them in aicall.error. Defaults to false.
source
PromptingTools.Experimental.AgentTools.SampleNode — Type
SampleNode{T}
A node in the Monte Carlo Tree Search tree. 
It's used to hold the data we're trying to optimize/discover (eg, a conversation), the scores from evaluation (wins, visits) and the results of the evaluations upon failure (feedback).
Fields
  • id::UInt16: Unique identifier for the node
  • parent::Union{SampleNode, Nothing}: Parent node that current node was built on
  • children::Vector{SampleNode}: Children nodes
  • wins::Int: Number of successful outcomes
  • visits::Int: Number of condition checks done (eg, losses are checks - wins)
  • data::T: eg, the conversation or some parameter to be optimized
  • feedback::String: Feedback from the evaluation, always a string! Defaults to empty string.
  • success::Union{Nothing, Bool}: Success of the generation and subsequent evaluations, proxy for whether it should be further evaluated. Defaults to nothing.
source
PromptingTools.Experimental.AgentTools.UCT — Type
UCT <: AbstractScoringMethod
Implements scoring and selection for UCT (Upper Confidence Bound for Trees) sampling method. See https://en.wikipedia.org/wiki/MonteCarlotreesearch#Explorationand_exploitation for more details.
source
PromptingTools.Experimental.AgentTools.AIClassify — Method
AIClassify(args...; kwargs...)
Creates a lazy instance of aiclassify. It is an instance of AICall with aiclassify as the function.
Use exactly the same arguments and keyword arguments as aiclassify (see ?aiclassify for details).
source
PromptingTools.Experimental.AgentTools.AIEmbed — Method
AIEmbed(args...; kwargs...)
Creates a lazy instance of aiembed. It is an instance of AICall with aiembed as the function.
Use exactly the same arguments and keyword arguments as aiembed (see ?aiembed for details).
source
PromptingTools.Experimental.AgentTools.AIExtract — Method
AIExtract(args...; kwargs...)
Creates a lazy instance of aiextract. It is an instance of AICall with aiextract as the function.
Use exactly the same arguments and keyword arguments as aiextract (see ?aiextract for details).
source
PromptingTools.Experimental.AgentTools.AIGenerate — Method
AIGenerate(args...; kwargs...)
Creates a lazy instance of aigenerate. It is an instance of AICall with aigenerate as the function.
Use exactly the same arguments and keyword arguments as aigenerate (see ?aigenerate for details).
source
PromptingTools.Experimental.AgentTools.AIScan — Method
AIScan(args...; kwargs...)
Creates a lazy instance of aiscan. It is an instance of AICall with aiscan as the function.
Use exactly the same arguments and keyword arguments as aiscan (see ?aiscan for details).
source
PromptingTools.Experimental.AgentTools.add_feedback! — Method
add_feedback!(
+preview(result.call.conversation)
Notes
  • AICodeFixer is particularly useful when code is hard to get right in one shot (eg, smaller models, complex syntax)
  • The structure leverages the lazy evaluation model of AICall (/AIGenerate) to efficiently manage AI interactions and be able to repeatedly call it.
  • The run! function executes the AI call and applies the feedback loop for the specified number of rounds, enabling an interactive code fixing process.
source
PromptingTools.Experimental.AgentTools.RetryConfig — Type
RetryConfig
Configuration for self-fixing the AI calls. It includes the following fields:
Fields
  • retries::Int: The number of retries ("fixing rounds") that have been attempted so far.
  • calls::Int: The total number of SUCCESSFULLY generated ai* function calls made so far (across all samples/retry rounds). Ie, if a call fails, because of an API error, it's not counted, because it didn't reach the LLM.
  • max_retries::Int: The maximum number of retries ("fixing rounds") allowed for the AI call. Defaults to 10.
  • max_calls::Int: The maximum number of ai* function calls allowed for the AI call. Defaults to 99.
  • retry_delay::Int: The delay (in seconds) between retry rounds. Defaults to 0s.
  • n_samples::Int: The number of samples to generate in each ai* call round (to increase changes of successful pass). Defaults to 1.
  • scoring::AbstractScoringMethod: The scoring method to use for generating multiple samples. Defaults to UCT(sqrt(2)).
  • ordering::Symbol: The ordering to use for select the best samples. With :PostOrderDFS we prioritize leaves, with :PreOrderDFS we prioritize the root. Defaults to :PostOrderDFS.
  • feedback_inplace::Bool: Whether to provide feedback in previous UserMessage (and remove the past AIMessage) or to create a new UserMessage. Defaults to false.
  • feedback_template::Symbol: Template to use for feedback in place. Defaults to :FeedbackFromEvaluator.
  • temperature::Float64: The temperature to use for sampling. Relevant only if not defined in api_kwargs provided. Defaults to 0.7.
  • catch_errors::Bool: Whether to catch errors during run! of AICall. Saves them in aicall.error. Defaults to false.
source
PromptingTools.Experimental.AgentTools.SampleNode — Type
SampleNode{T}
A node in the Monte Carlo Tree Search tree. 
It's used to hold the data we're trying to optimize/discover (eg, a conversation), the scores from evaluation (wins, visits) and the results of the evaluations upon failure (feedback).
Fields
  • id::UInt16: Unique identifier for the node
  • parent::Union{SampleNode, Nothing}: Parent node that current node was built on
  • children::Vector{SampleNode}: Children nodes
  • wins::Int: Number of successful outcomes
  • visits::Int: Number of condition checks done (eg, losses are checks - wins)
  • data::T: eg, the conversation or some parameter to be optimized
  • feedback::String: Feedback from the evaluation, always a string! Defaults to empty string.
  • success::Union{Nothing, Bool}: Success of the generation and subsequent evaluations, proxy for whether it should be further evaluated. Defaults to nothing.
source
PromptingTools.Experimental.AgentTools.UCT — Type
UCT <: AbstractScoringMethod
Implements scoring and selection for UCT (Upper Confidence Bound for Trees) sampling method. See https://en.wikipedia.org/wiki/MonteCarlotreesearch#Explorationand_exploitation for more details.
source
PromptingTools.Experimental.AgentTools.AIClassify — Method
AIClassify(args...; kwargs...)
Creates a lazy instance of aiclassify. It is an instance of AICall with aiclassify as the function.
Use exactly the same arguments and keyword arguments as aiclassify (see ?aiclassify for details).
source
PromptingTools.Experimental.AgentTools.AIEmbed — Method
AIEmbed(args...; kwargs...)
Creates a lazy instance of aiembed. It is an instance of AICall with aiembed as the function.
Use exactly the same arguments and keyword arguments as aiembed (see ?aiembed for details).
source
PromptingTools.Experimental.AgentTools.AIExtract — Method
AIExtract(args...; kwargs...)
Creates a lazy instance of aiextract. It is an instance of AICall with aiextract as the function.
Use exactly the same arguments and keyword arguments as aiextract (see ?aiextract for details).
source
PromptingTools.Experimental.AgentTools.AIGenerate — Method
AIGenerate(args...; kwargs...)
Creates a lazy instance of aigenerate. It is an instance of AICall with aigenerate as the function.
Use exactly the same arguments and keyword arguments as aigenerate (see ?aigenerate for details).
source
PromptingTools.Experimental.AgentTools.AIScan — Method
AIScan(args...; kwargs...)
Creates a lazy instance of aiscan. It is an instance of AICall with aiscan as the function.
Use exactly the same arguments and keyword arguments as aiscan (see ?aiscan for details).
source
PromptingTools.Experimental.AgentTools.add_feedback! — Method
add_feedback!(
     conversation::AbstractVector{<:PT.AbstractMessage}, sample::SampleNode; feedback_inplace::Bool = false,
     feedback_template::Symbol = :FeedbackFromEvaluator)
Adds formatted feedback to the conversation based on the sample node feedback (and its ancestors).
Arguments
  • conversation::AbstractVector{<:PT.AbstractMessage}: The conversation to add the feedback to.
  • sample::SampleNode: The sample node to extract the feedback from.
  • feedback_inplace::Bool=false: If true, it will add the feedback to the last user message inplace (and pop the last AIMessage). Otherwise, it will append the feedback as a new message.
  • feedback_template::Symbol=:FeedbackFromEvaluator: The template to use for the feedback message. It must be a valid AITemplate name.
Example
sample = SampleNode(; data = nothing, feedback = "Feedback X")
 conversation = [PT.UserMessage("I say hi!"), PT.AIMessage(; content = "I say hi!")]
@@ -44,7 +44,7 @@
 conversation[end].content == "### Feedback from Evaluator\nFeedback X\n"
 
 Inplace feedback:
julia conversation = [PT.UserMessage("I say hi!"), PT.AIMessage(; content = "I say hi!")] conversation = AT.addfeedback!(conversation, sample; feedbackinplace = true) conversation[end].content == "I say hi!\n\n### Feedback from Evaluator\nFeedback X\n"

-Sample with ancestors with feedback:
julia samplep = SampleNode(; data = nothing, feedback = "\nFeedback X") sample = expand!(samplep, nothing) sample.feedback = "\nFeedback Y" conversation = [PT.UserMessage("I say hi!"), PT.AIMessage(; content = "I say hi!")] conversation = AT.add_feedback!(conversation, sample)
conversation[end].content == "### Feedback from Evaluator\n\nFeedback X\n–––––\n\nFeedback Y\n" ```
source
PromptingTools.Experimental.AgentTools.aicodefixer_feedback — Method
aicodefixer_feedback(conversation::AbstractVector{<:PT.AbstractMessage}; max_length::Int = 512) -> NamedTuple(; feedback::String)
Generate feedback for an AI code fixing session based on the conversation history. Function is designed to be extensible for different types of feedback and code evaluation outcomes. 
The highlevel wrapper accepts a conversation and returns new kwargs for the AICall.
Individual feedback functions are dispatched on different subtypes of AbstractCodeOutcome and can be extended/overwritten to provide more detailed feedback.
See also: AIGenerate, AICodeFixer
Arguments
  • conversation::AbstractVector{<:PT.AbstractMessage}: A vector of messages representing the conversation history, where the last message is expected to contain the code to be analyzed.
  • max_length::Int=512: An optional argument that specifies the maximum length of the feedback message.
Returns
  • NamedTuple: A feedback message as a kwarg in NamedTuple based on the analysis of the code provided in the conversation.
Example
new_kwargs = aicodefixer_feedback(conversation)
Notes
This function is part of the AI code fixing system, intended to interact with code in AIMessage and provide feedback on improving it.
The highlevel wrapper accepts a conversation and returns new kwargs for the AICall.
It dispatches for the code feedback based on the subtypes of AbstractCodeOutcome below:
  • CodeEmpty: No code found in the message.
  • CodeFailedParse: Code parsing error.
  • CodeFailedEval: Runtime evaluation error.
  • CodeFailedTimeout: Code execution timed out.
  • CodeSuccess: Successful code execution.
You can override the individual methods to customize the feedback.
source
PromptingTools.Experimental.AgentTools.airetry! — Function
airetry!(
+Sample with ancestors with feedback:
julia samplep = SampleNode(; data = nothing, feedback = "\nFeedback X") sample = expand!(samplep, nothing) sample.feedback = "\nFeedback Y" conversation = [PT.UserMessage("I say hi!"), PT.AIMessage(; content = "I say hi!")] conversation = AT.add_feedback!(conversation, sample)
conversation[end].content == "### Feedback from Evaluator\n\nFeedback X\n–––––\n\nFeedback Y\n" ```
source
PromptingTools.Experimental.AgentTools.aicodefixer_feedback — Method
aicodefixer_feedback(conversation::AbstractVector{<:PT.AbstractMessage}; max_length::Int = 512) -> NamedTuple(; feedback::String)
Generate feedback for an AI code fixing session based on the conversation history. Function is designed to be extensible for different types of feedback and code evaluation outcomes. 
The highlevel wrapper accepts a conversation and returns new kwargs for the AICall.
Individual feedback functions are dispatched on different subtypes of AbstractCodeOutcome and can be extended/overwritten to provide more detailed feedback.
See also: AIGenerate, AICodeFixer
Arguments
  • conversation::AbstractVector{<:PT.AbstractMessage}: A vector of messages representing the conversation history, where the last message is expected to contain the code to be analyzed.
  • max_length::Int=512: An optional argument that specifies the maximum length of the feedback message.
Returns
  • NamedTuple: A feedback message as a kwarg in NamedTuple based on the analysis of the code provided in the conversation.
Example
new_kwargs = aicodefixer_feedback(conversation)
Notes
This function is part of the AI code fixing system, intended to interact with code in AIMessage and provide feedback on improving it.
The highlevel wrapper accepts a conversation and returns new kwargs for the AICall.
It dispatches for the code feedback based on the subtypes of AbstractCodeOutcome below:
  • CodeEmpty: No code found in the message.
  • CodeFailedParse: Code parsing error.
  • CodeFailedEval: Runtime evaluation error.
  • CodeFailedTimeout: Code execution timed out.
  • CodeSuccess: Successful code execution.
You can override the individual methods to customize the feedback.
source
PromptingTools.Experimental.AgentTools.airetry! — Function
airetry!(
     f_cond::Function, aicall::AICallBlock, feedback::Union{AbstractString, Function} = "";
     verbose::Bool = true, throw::Bool = false, evaluate_all::Bool = true, feedback_expensive::Bool = false,
     max_retries::Union{Nothing, Int} = nothing, retry_delay::Union{Nothing, Int} = nothing)
Evaluates the condition f_cond on the aicall object.  If the condition is not met, it will return the best sample to retry from and provide feedback (string or function) to aicall. That's why it's mutating. It will retry maximum max_retries times, with throw=true, an error will be thrown if the condition is not met after max_retries retries.
Function signatures
  • f_cond(aicall::AICallBlock) -> Bool, ie, it must accept the aicall object and return a boolean value.
  • feedback can be a string or feedback(aicall::AICallBlock) -> String, ie, it must accept the aicall object and return a string.
You can leverage the last_message, last_output, and AICode functions to access the last message, last output and execute code blocks in the conversation, respectively. See examples below.
Good Use Cases
  • Retry with API failures/drops (add retry_delay=2 to wait 2s between retries)
  • Check the output format / type / length / etc
  • Check the output with aiclassify call (LLM Judge) to catch unsafe/NSFW/out-of-scope content
  • Provide hints to the model to guide it to the correct answer
Gotchas
  • If controlling keyword arguments are set to nothing, they will fall back to the default values in aicall.config. You can override them by passing the keyword arguments explicitly.
  • If there multiple airetry! checks, they are evaluted sequentially. As long as throw==false, they will be all evaluated even if they failed previous checks.
  • Only samples which passed previous evaluations are evaluated (sample.success is true). If there are no successful samples, the function will evaluate only the active sample (aicall.active_sample_id) and nothing else.
  • Feedback from all "ancestor" evaluations is added upon retry, not feedback from the "sibblings" or other branches. To have only ONE long BRANCH (no sibblings), make sure to keep RetryConfig(; n_samples=1).  That way the model will always see ALL previous feedback.
  • We implement a version of Monte Carlo Tree Search (MCTS) to always pick the most promising sample to restart from (you can tweak the options in RetryConfig to change the behaviour).
  • For large number of parallel branches (ie, "shallow and wide trees"), you might benefit from switching scoring to scoring=ThompsonSampling() (similar to how Bandit algorithms work).
  • Open-source/local models can struggle with too long conversation, you might want to experiment with in-place feedback (set RetryConfig(; feedback_inplace=true)).
Arguments
  • f_cond::Function: A function that accepts the aicall object and returns a boolean value. Retry will be attempted if the condition is not met (f_cond -> false).
  • aicall::AICallBlock: The aicall object to evaluate the condition on.
  • feedback::Union{AbstractString, Function}: Feedback to provide if the condition is not met. If a function is provided, it must accept the aicall object as the only argument and return a string.
  • verbose::Integer=1: A verbosity level for logging the retry attempts and warnings. A higher value indicates more detailed logging.
  • throw::Bool=false: If true, it will throw an error if the function f_cond does not return true after max_retries retries.
  • evaluate_all::Bool=false: If true, it will evaluate all the "successful" samples in the aicall object. Otherwise, it will only evaluate the active sample.
  • feedback_expensive::Bool=false: If false, it will provide feedback to all samples that fail the condition.  If feedback function is expensive to call (eg, another ai* function), set this to true and feedback will be provided only to the sample we will retry from.
  • max_retries::Union{Nothing, Int}=nothing: Maximum number of retries. If not provided, it will fall back to the max_retries in aicall.config.
  • retry_delay::Union{Nothing, Int}=nothing: Delay between retries in seconds. If not provided, it will fall back to the retry_delay in aicall.config.
Returns
  • The aicall object with the updated conversation, and samples (saves the evaluations and their scores/feedback).
Example
You can use airetry! to catch API errors in run! and auto-retry the call.  RetryConfig is how you influence all the subsequent retry behaviours - see ?RetryConfig for more details.
# API failure because of a non-existent model
@@ -207,7 +207,7 @@
 ## ID: 32991, Guess: 50
 ## ID: 32991, Guess: 35
 ## ID: 32991, Guess: 33
-## etc...
Note that if there are multiple "branches" the model will see only the feedback of its own and its ancestors not the other "branches".  If you wanted to provide ALL feedback, set RetryConfig(; n_samples=1) to remove any "branching". It fixing will be done sequentially in one conversation and the model will see all feedback (less powerful if the model falls into a bad state). Alternatively, you can tweak the feedback function.
See Also
References: airetry is inspired by the Language Agent Tree Search paper and by DSPy Assertions paper.
source
PromptingTools.Experimental.AgentTools.evaluate_condition! — Function
evaluate_condition!(f_cond::Function, aicall::AICallBlock,
+## etc...
Note that if there are multiple "branches" the model will see only the feedback of its own and its ancestors not the other "branches".  If you wanted to provide ALL feedback, set RetryConfig(; n_samples=1) to remove any "branching". It fixing will be done sequentially in one conversation and the model will see all feedback (less powerful if the model falls into a bad state). Alternatively, you can tweak the feedback function.
See Also
References: airetry is inspired by the Language Agent Tree Search paper and by DSPy Assertions paper.
source
PromptingTools.Experimental.AgentTools.evaluate_condition! — Function
evaluate_condition!(f_cond::Function, aicall::AICallBlock,
     feedback::Union{AbstractString, Function} = "";
     evaluate_all::Bool = true, feedback_expensive::Bool = false)
Evalutes the condition f_cond (must return Bool) on the aicall object.  If the condition is not met, it will return the best sample to retry from and provide feedback.
Mutating as the results are saved in aicall.samples
If evaluate_all is true, it will evaluate all the "successful" samples in the aicall object. Otherwise, it will only evaluate the active sample..
For f_cond and feedback functions, you can use the last_message and last_output utilities to access the last message and last output in the conversation, respectively.
Arguments
  • f_cond::Function: A function that accepts the aicall object and returns a boolean value. Retry will be attempted if the condition is not met (f_cond -> false).
  • aicall::AICallBlock: The aicall object to evaluate the condition on.
  • feedback::Union{AbstractString, Function}: Feedback to provide if the condition is not met. If a function is provided, it must accept the aicall object as the only argument and return a string.
  • evaluate_all::Bool=false: If true, it will evaluate all the "successful" samples in the aicall object. Otherwise, it will only evaluate the active sample.
  • feedback_expensive::Bool=false: If false, it will provide feedback to all samples that fail the condition.  If feedback function is expensive to call (eg, another ai* function), set this to true and feedback will be provided only to the sample we will retry from.
Returns
  • a tuple (condition_passed, sample), where condition_passed is a boolean indicating whether the condition was met, and sample is the best sample to retry from.
Example
# Mimic AIGenerate run!
 aicall = AIGenerate("Say hi!"; config = RetryConfig(; n_samples = 2))
@@ -220,11 +220,11 @@
 # Checks:
 cond == true
 node == sample
-node.wins == 1
With feedback: ```julia
Mimic AIGenerate run with feedback
aicall = AIGenerate(     :BlankSystemUser; system = "a", user = "b") sample = expand!(aicall.samples, aicall.conversation; success = true) aicall.activesampleid = sample.id
Evaluate
cond, node = AT.evaluatecondition!(     x -> occursin("NOTFOUND", lastoutput(x)), aicall, "Feedback X") cond == false # fail sample == node # same node (no other choice) node.wins == 0 node.feedback == " Feedback X"
source
PromptingTools.Experimental.AgentTools.run! — Method
run!(codefixer::AICodeFixer; verbose::Int = 1, max_conversation_length::Int = 32000, run_kwargs...)
Executes the code fixing process encapsulated by the AICodeFixer instance.  This method iteratively refines and fixes code by running the AI call in a loop for a specified number of rounds, using feedback from the code evaluation (aicodefixer_feedback) to improve the outcome in each iteration.
Arguments
  • codefixer::AICodeFixer: An instance of AICodeFixer containing the AI call, templates, and settings for the code fixing session.
  • verbose::Int=1: Verbosity level for logging. A higher value indicates more detailed logging.
  • max_conversation_length::Int=32000: Maximum length in characters for the conversation history to keep it within manageable limits, especially for large code fixing sessions.
  • num_rounds::Union{Nothing, Int}=nothing: Number of additional rounds for the code fixing session. If nothing, the value from the AICodeFixer instance is used.
  • run_kwargs...: Additional keyword arguments that are passed to the AI function.
Returns
  • AICodeFixer: The updated AICodeFixer instance with the results of the code fixing session.
Usage
aicall = AICall(aigenerate, schema=mySchema, conversation=myConversation)
+node.wins == 1
With feedback: ```julia
Mimic AIGenerate run with feedback
aicall = AIGenerate(     :BlankSystemUser; system = "a", user = "b") sample = expand!(aicall.samples, aicall.conversation; success = true) aicall.activesampleid = sample.id
Evaluate
cond, node = AT.evaluatecondition!(     x -> occursin("NOTFOUND", lastoutput(x)), aicall, "Feedback X") cond == false # fail sample == node # same node (no other choice) node.wins == 0 node.feedback == " Feedback X"
source
PromptingTools.Experimental.AgentTools.run! — Method
run!(codefixer::AICodeFixer; verbose::Int = 1, max_conversation_length::Int = 32000, run_kwargs...)
Executes the code fixing process encapsulated by the AICodeFixer instance.  This method iteratively refines and fixes code by running the AI call in a loop for a specified number of rounds, using feedback from the code evaluation (aicodefixer_feedback) to improve the outcome in each iteration.
Arguments
  • codefixer::AICodeFixer: An instance of AICodeFixer containing the AI call, templates, and settings for the code fixing session.
  • verbose::Int=1: Verbosity level for logging. A higher value indicates more detailed logging.
  • max_conversation_length::Int=32000: Maximum length in characters for the conversation history to keep it within manageable limits, especially for large code fixing sessions.
  • num_rounds::Union{Nothing, Int}=nothing: Number of additional rounds for the code fixing session. If nothing, the value from the AICodeFixer instance is used.
  • run_kwargs...: Additional keyword arguments that are passed to the AI function.
Returns
  • AICodeFixer: The updated AICodeFixer instance with the results of the code fixing session.
Usage
aicall = AICall(aigenerate, schema=mySchema, conversation=myConversation)
 codefixer = AICodeFixer(aicall, myTemplates; num_rounds=5)
-result = run!(codefixer, verbose=2)
Notes
  • The run! method drives the core logic of the AICodeFixer, iterating through rounds of AI interactions to refine and fix code.
  • In each round, it applies feedback based on the current state of the conversation, allowing the AI to respond more effectively.
  • The conversation history is managed to ensure it stays within the specified max_conversation_length, keeping the AI's focus on relevant parts of the conversation.
  • This iterative process is essential for complex code fixing tasks where multiple interactions and refinements are required to achieve the desired outcome.
source
PromptingTools.Experimental.AgentTools.run! — Method
run!(aicall::AICallBlock; verbose::Int = 1, catch_errors::Bool = false, return_all::Bool = true, kwargs...)
Executes the AI call wrapped by an AICallBlock instance. This method triggers the actual communication with the AI model and processes the response based on the provided conversation context and parameters.
Note: Currently return_all must always be set to true.
Arguments
  • aicall::AICallBlock: An instance of AICallBlock which encapsulates the AI function call along with its context and parameters (eg, AICall, AIGenerate)
  • verbose::Integer=1: A verbosity level for logging. A higher value indicates more detailed logging.
  • catch_errors::Union{Nothing, Bool}=nothing: A flag to indicate whether errors should be caught and saved to aicall.error. If nothing, it defaults to aicall.config.catch_errors.
  • return_all::Bool=true: A flag to indicate whether the whole conversation from the AI call should be returned. It should always be true.
  • kwargs...: Additional keyword arguments that are passed to the AI function.
Returns
  • AICallBlock: The same AICallBlock instance, updated with the results of the AI call. This includes updated conversation, success status, and potential error information.
Example
aicall = AICall(aigenerate)
+result = run!(codefixer, verbose=2)
Notes
  • The run! method drives the core logic of the AICodeFixer, iterating through rounds of AI interactions to refine and fix code.
  • In each round, it applies feedback based on the current state of the conversation, allowing the AI to respond more effectively.
  • The conversation history is managed to ensure it stays within the specified max_conversation_length, keeping the AI's focus on relevant parts of the conversation.
  • This iterative process is essential for complex code fixing tasks where multiple interactions and refinements are required to achieve the desired outcome.
source
PromptingTools.Experimental.AgentTools.run! — Method
run!(aicall::AICallBlock; verbose::Int = 1, catch_errors::Bool = false, return_all::Bool = true, kwargs...)
Executes the AI call wrapped by an AICallBlock instance. This method triggers the actual communication with the AI model and processes the response based on the provided conversation context and parameters.
Note: Currently return_all must always be set to true.
Arguments
  • aicall::AICallBlock: An instance of AICallBlock which encapsulates the AI function call along with its context and parameters (eg, AICall, AIGenerate)
  • verbose::Integer=1: A verbosity level for logging. A higher value indicates more detailed logging.
  • catch_errors::Union{Nothing, Bool}=nothing: A flag to indicate whether errors should be caught and saved to aicall.error. If nothing, it defaults to aicall.config.catch_errors.
  • return_all::Bool=true: A flag to indicate whether the whole conversation from the AI call should be returned. It should always be true.
  • kwargs...: Additional keyword arguments that are passed to the AI function.
Returns
  • AICallBlock: The same AICallBlock instance, updated with the results of the AI call. This includes updated conversation, success status, and potential error information.
Example
aicall = AICall(aigenerate)
 run!(aicall)
Alternatively, you can trigger the run! call by using the AICall as a functor and calling it with a string or a UserMessage:
aicall = AICall(aigenerate)
-aicall("Say hi!")
Notes
  • The run! method is a key component of the lazy evaluation model in AICall. It allows for the deferred execution of AI function calls, providing flexibility in how and when AI interactions are conducted.
  • The method updates the AICallBlock instance with the outcome of the AI call, including any generated responses, success or failure status, and error information if an error occurred.
  • This method is essential for scenarios where AI interactions are based on dynamic or evolving contexts, as it allows for real-time updates and responses based on the latest information.
source
PromptingTools.Experimental.AgentTools.select_best — Function
select_best(node::SampleNode, scoring::AbstractScoringMethod = UCT();
+aicall("Say hi!")
Notes
  • The run! method is a key component of the lazy evaluation model in AICall. It allows for the deferred execution of AI function calls, providing flexibility in how and when AI interactions are conducted.
  • The method updates the AICallBlock instance with the outcome of the AI call, including any generated responses, success or failure status, and error information if an error occurred.
  • This method is essential for scenarios where AI interactions are based on dynamic or evolving contexts, as it allows for real-time updates and responses based on the latest information.
source
PromptingTools.Experimental.AgentTools.select_best — Function
select_best(node::SampleNode, scoring::AbstractScoringMethod = UCT();
     ordering::Symbol = :PostOrderDFS)
Selects the best node from the tree using the given scoring (UCT or ThompsonSampling). Defaults to UCT. Thompson Sampling is more random with small samples, while UCT stabilizes much quicker thanks to looking at parent nodes as well.
Ordering can be either :PreOrderDFS or :PostOrderDFS. Defaults to :PostOrderDFS, which favors the leaves (end points of the tree).
Example
Compare the different scoring methods:
# Set up mock samples and scores
 data = PT.AbstractMessage[]
 root = SampleNode(; data)
@@ -256,5 +256,5 @@
 ## ├─ SampleNode(id: 26078, stats: 2/2, score: 0.93, length: 0)
 ## │  └─ SampleNode(id: 29826, stats: 1/1, score: 0.22, length: 0)
 ## └─ SampleNode(id: 39931, stats: 0/1, score: 0.84, length: 0)
-
source
PromptingTools.Experimental.AgentTools.truncate_conversation — Method
truncate_conversation(conversation::AbstractVector{<:PT.AbstractMessage};
-    max_conversation_length::Int = 32000)
Truncates a given conversation to a max_conversation_length characters by removing messages "in the middle". It tries to retain the original system+user message and also the most recent messages.
Practically, if a conversation is too long, it will start by removing the most recent message EXCEPT for the last two (assumed to be the last AIMessage with the code and UserMessage with the feedback
Arguments
max_conversation_length is in characters; assume c. 2-3 characters per LLM token, so 32000 should correspond to 16K context window.
source

+
source
PromptingTools.Experimental.AgentTools.truncate_conversation — Method
truncate_conversation(conversation::AbstractVector{<:PT.AbstractMessage};
+    max_conversation_length::Int = 32000)

Truncates a given conversation to a max_conversation_length characters by removing messages "in the middle". It tries to retain the original system+user message and also the most recent messages.

Practically, if a conversation is too long, it will start by removing the most recent message EXCEPT for the last two (assumed to be the last AIMessage with the code and UserMessage with the feedback

Arguments

max_conversation_length is in characters; assume c. 2-3 characters per LLM token, so 32000 should correspond to 16K context window.

source
diff --git a/dev/reference_apitools/index.html b/dev/reference_apitools/index.html index ed02dbfec..7da74b87f 100644 --- a/dev/reference_apitools/index.html +++ b/dev/reference_apitools/index.html @@ -1,9 +1,9 @@ APITools · PromptingTools.jl
GitHub

Reference for APITools

PromptingTools.Experimental.APITools.create_websearch — Method
create_websearch(query::AbstractString;
     api_key::AbstractString,
-    search_depth::AbstractString = "basic")

Arguments

  • query::AbstractString: The query to search for.
  • api_key::AbstractString: The API key to use for the search. Get an API key from Tavily.
  • search_depth::AbstractString: The depth of the search. Can be either "basic" or "advanced". Default is "basic". Advanced search calls equal to 2 requests.
  • include_answer::Bool: Whether to include the answer in the search results. Default is false.
  • include_raw_content::Bool: Whether to include the raw content in the search results. Default is false.
  • max_results::Integer: The maximum number of results to return. Default is 5.
  • include_images::Bool: Whether to include images in the search results. Default is false.
  • include_domains::AbstractVector{<:AbstractString}: A list of domains to include in the search results. Default is an empty list.
  • exclude_domains::AbstractVector{<:AbstractString}: A list of domains to exclude from the search results. Default is an empty list.

Example

r = create_websearch("Who is King Charles?")

Even better, you can get not just the results but also the answer:

r = create_websearch("Who is King Charles?"; include_answer = true)

See Rest API documentation for more information.

source
PromptingTools.Experimental.APITools.tavily_api — Method
tavily_api(;
+    search_depth::AbstractString = "basic")

Arguments

  • query::AbstractString: The query to search for.
  • api_key::AbstractString: The API key to use for the search. Get an API key from Tavily.
  • search_depth::AbstractString: The depth of the search. Can be either "basic" or "advanced". Default is "basic". Advanced search calls equal to 2 requests.
  • include_answer::Bool: Whether to include the answer in the search results. Default is false.
  • include_raw_content::Bool: Whether to include the raw content in the search results. Default is false.
  • max_results::Integer: The maximum number of results to return. Default is 5.
  • include_images::Bool: Whether to include images in the search results. Default is false.
  • include_domains::AbstractVector{<:AbstractString}: A list of domains to include in the search results. Default is an empty list.
  • exclude_domains::AbstractVector{<:AbstractString}: A list of domains to exclude from the search results. Default is an empty list.

Example

r = create_websearch("Who is King Charles?")

Even better, you can get not just the results but also the answer:

r = create_websearch("Who is King Charles?"; include_answer = true)

See Rest API documentation for more information.

source
+ kwargs...)

Sends API requests to Tavily and returns the response.

source diff --git a/dev/reference_experimental/index.html b/dev/reference_experimental/index.html index 05591476c..e3fba67d9 100644 --- a/dev/reference_experimental/index.html +++ b/dev/reference_experimental/index.html @@ -1,2 +1,2 @@ -Experimental Modules · PromptingTools.jl
GitHub

Reference for Experimental Module

Note: This module is experimental and may change in future releases. The intention is for the functionality to be moved to separate packages over time.

PromptingTools.Experimental — Module
Experimental

This module is for experimental code that is not yet ready for production. It is not included in the main module, so it must be explicitly imported.

Contains:

  • RAGTools: Retrieval-Augmented Generation (RAG) functionality.
  • AgentTools: Agentic functionality - lazy calls for building pipelines (eg, AIGenerate) and AICodeFixer.
  • APITools: APIs to complement GenAI workflows (eg, Tavily Search API).
source
+Experimental Modules · PromptingTools.jl
GitHub

Reference for Experimental Module

Note: This module is experimental and may change in future releases. The intention is for the functionality to be moved to separate packages over time.

PromptingTools.Experimental — Module
Experimental

This module is for experimental code that is not yet ready for production. It is not included in the main module, so it must be explicitly imported.

Contains:

  • RAGTools: Retrieval-Augmented Generation (RAG) functionality.
  • AgentTools: Agentic functionality - lazy calls for building pipelines (eg, AIGenerate) and AICodeFixer.
  • APITools: APIs to complement GenAI workflows (eg, Tavily Search API).
source
diff --git a/dev/reference_ragtools/index.html b/dev/reference_ragtools/index.html index 11ab66a99..5154ca131 100644 --- a/dev/reference_ragtools/index.html +++ b/dev/reference_ragtools/index.html @@ -1,5 +1,5 @@ -RAGTools · PromptingTools.jl
GitHub

Reference for RAGTools

PromptingTools.Experimental.RAGTools — Module
RAGTools

Provides Retrieval-Augmented Generation (RAG) functionality.

Requires: LinearAlgebra, SparseArrays, PromptingTools for proper functionality.

This module is experimental and may change at any time. It is intended to be moved to a separate package in the future.

source
PromptingTools.Experimental.RAGTools.ChunkIndex — Type
ChunkIndex

Main struct for storing document chunks and their embeddings. It also stores tags and sources for each chunk.

Fields

  • id::Symbol: unique identifier of each index (to ensure we're using the right index with CandidateChunks)
  • chunks::Vector{<:AbstractString}: underlying document chunks / snippets
  • embeddings::Union{Nothing, Matrix{<:Real}}: for semantic search
  • tags::Union{Nothing, AbstractMatrix{<:Bool}}: for exact search, filtering, etc. This is often a sparse matrix indicating which chunks have the given tag (see tag_vocab for the position lookup)
  • tags_vocab::Union{Nothing, Vector{<:AbstractString}}: vocabulary for the tags matrix (each column in tags is one item in tags_vocab and rows are the chunks)
  • sources::Vector{<:AbstractString}: sources of the chunks
  • extras::Union{Nothing, AbstractVector}: additional data, eg, metadata, source code, etc.
source
PromptingTools.Experimental.RAGTools.airag — Function
airag(index::AbstractChunkIndex, rag_template::Symbol = :RAGAnswerFromContext;
+RAGTools · PromptingTools.jl
GitHub
Reference for RAGTools
PromptingTools.Experimental.RAGTools — Module
RAGTools
Provides Retrieval-Augmented Generation (RAG) functionality.
Requires: LinearAlgebra, SparseArrays, PromptingTools for proper functionality.
This module is experimental and may change at any time. It is intended to be moved to a separate package in the future.
source
PromptingTools.Experimental.RAGTools.ChunkIndex — Type
ChunkIndex
Main struct for storing document chunks and their embeddings. It also stores tags and sources for each chunk.
Fields
  • id::Symbol: unique identifier of each index (to ensure we're using the right index with CandidateChunks)
  • chunks::Vector{<:AbstractString}: underlying document chunks / snippets
  • embeddings::Union{Nothing, Matrix{<:Real}}: for semantic search
  • tags::Union{Nothing, AbstractMatrix{<:Bool}}: for exact search, filtering, etc. This is often a sparse matrix indicating which chunks have the given tag (see tag_vocab for the position lookup)
  • tags_vocab::Union{Nothing, Vector{<:AbstractString}}: vocabulary for the tags matrix (each column in tags is one item in tags_vocab and rows are the chunks)
  • sources::Vector{<:AbstractString}: sources of the chunks
  • extras::Union{Nothing, AbstractVector}: additional data, eg, metadata, source code, etc.
source
PromptingTools.Experimental.RAGTools.airag — Function
airag(index::AbstractChunkIndex, rag_template::Symbol = :RAGAnswerFromContext;
     question::AbstractString,
     top_k::Int = 100, top_n::Int = 5, minimum_similarity::AbstractFloat = -1.0,
     tag_filter::Union{Symbol, Vector{String}, Regex, Nothing} = :auto,
@@ -19,9 +19,9 @@
 msg = airag(index, :RAGAnswerFromContext; question)
 
 # or simply
-msg = airag(index; question)
See also build_index, build_context, CandidateChunks, find_closest, find_tags, rerank
source
PromptingTools.Experimental.RAGTools.build_context — Method
build_context(index::AbstractChunkIndex, reranked_candidates::CandidateChunks; chunks_window_margin::Tuple{Int, Int}) -> Vector{String}
Build context strings for each position in reranked_candidates considering a window margin around each position.
Arguments
  • reranked_candidates::CandidateChunks: Candidate chunks which contain positions to extract context from.
  • index::ChunkIndex: The index containing chunks and sources.
  • chunks_window_margin::Tuple{Int, Int}: A tuple indicating the margin (before, after) around each position to include in the context.  Defaults to (1,1), which means 1 preceding and 1 suceeding chunk will be included. With (0,0), only the matching chunks will be included.
Returns
  • Vector{String}: A vector of context strings, each corresponding to a position in reranked_candidates.
Examples
index = ChunkIndex(...)  # Assuming a proper index is defined
+msg = airag(index; question)
See also build_index, build_context, CandidateChunks, find_closest, find_tags, rerank
source
PromptingTools.Experimental.RAGTools.build_context — Method
build_context(index::AbstractChunkIndex, reranked_candidates::CandidateChunks; chunks_window_margin::Tuple{Int, Int}) -> Vector{String}
Build context strings for each position in reranked_candidates considering a window margin around each position.
Arguments
  • reranked_candidates::CandidateChunks: Candidate chunks which contain positions to extract context from.
  • index::ChunkIndex: The index containing chunks and sources.
  • chunks_window_margin::Tuple{Int, Int}: A tuple indicating the margin (before, after) around each position to include in the context.  Defaults to (1,1), which means 1 preceding and 1 suceeding chunk will be included. With (0,0), only the matching chunks will be included.
Returns
  • Vector{String}: A vector of context strings, each corresponding to a position in reranked_candidates.
Examples
index = ChunkIndex(...)  # Assuming a proper index is defined
 candidates = CandidateChunks(index.id, [2, 4], [0.1, 0.2])
-context = build_context(index, candidates; chunks_window_margin=(0, 1)) # include only one following chunk for each matching chunk
source
PromptingTools.Experimental.RAGTools.build_index — Method
build_index(files_or_docs::Vector{<:AbstractString}; reader::Symbol = :files,
+context = build_context(index, candidates; chunks_window_margin=(0, 1)) # include only one following chunk for each matching chunk
source
PromptingTools.Experimental.RAGTools.build_index — Method
build_index(files_or_docs::Vector{<:AbstractString}; reader::Symbol = :files,
     separators = ["\n\n", ". ", "\n"], max_length::Int = 256,
     sources::Vector{<:AbstractString} = files_or_docs,
     extras::Union{Nothing, AbstractVector} = nothing,
@@ -40,38 +40,38 @@
 index = build_index(["file1.txt", "file2.txt"]; 
                     separators=[". "], 
                     extract_metadata=true, 
-                    verbose=true)
Notes
  • If you get errors about exceeding embedding input sizes, first check the max_length in your chunks.  If that does NOT resolve the issue, try changing the embedding_kwargs.  In particular, reducing the target_batch_size_length parameter (eg, 10_000) and number of tasks ntasks=1.  Some providers cannot handle large batch sizes (eg, Databricks).
source
PromptingTools.Experimental.RAGTools.build_qa_evals — Method
build_qa_evals(doc_chunks::Vector{<:AbstractString}, sources::Vector{<:AbstractString};
+                    verbose=true)
Notes
  • If you get errors about exceeding embedding input sizes, first check the max_length in your chunks.  If that does NOT resolve the issue, try changing the embedding_kwargs.  In particular, reducing the target_batch_size_length parameter (eg, 10_000) and number of tasks ntasks=1.  Some providers cannot handle large batch sizes (eg, Databricks).
source
PromptingTools.Experimental.RAGTools.build_qa_evals — Method
build_qa_evals(doc_chunks::Vector{<:AbstractString}, sources::Vector{<:AbstractString};
                model=PT.MODEL_CHAT, instructions="None.", qa_template::Symbol=:RAGCreateQAFromContext, 
                verbose::Bool=true, api_kwargs::NamedTuple = NamedTuple(), kwargs...) -> Vector{QAEvalItem}
Create a collection of question and answer evaluations (QAEvalItem) from document chunks and sources.  This function generates Q&A pairs based on the provided document chunks, using a specified AI model and template.
Arguments
  • doc_chunks::Vector{<:AbstractString}: A vector of document chunks, each representing a segment of text.
  • sources::Vector{<:AbstractString}: A vector of source identifiers corresponding to each chunk in doc_chunks (eg, filenames or paths).
  • model: The AI model used for generating Q&A pairs. Default is PT.MODEL_CHAT.
  • instructions::String: Additional instructions or context to provide to the model generating QA sets. Defaults to "None.".
  • qa_template::Symbol: A template symbol that dictates the AITemplate that will be used. It must have placeholder context. Default is :CreateQAFromContext.
  • api_kwargs::NamedTuple: Parameters that will be forwarded to the API endpoint.
  • verbose::Bool: If true, additional information like costs will be logged. Defaults to true.
Returns
Vector{QAEvalItem}: A vector of QAEvalItem structs, each containing a source, context, question, and answer. Invalid or empty items are filtered out.
Notes
  • The function internally uses aiextract to generate Q&A pairs based on the provided qa_template. So you can use any kwargs that you want.
  • Each QAEvalItem includes the context (document chunk), the generated question and answer, and the source.
  • The function tracks and reports the cost of AI calls if verbose is enabled.
  • Items where the question, answer, or context is empty are considered invalid and are filtered out.
Examples
Creating Q&A evaluations from a set of document chunks:
doc_chunks = ["Text from document 1", "Text from document 2"]
 sources = ["source1", "source2"]
-qa_evals = build_qa_evals(doc_chunks, sources)
source
PromptingTools.Experimental.RAGTools.cohere_api — Method
cohere_api(;
 api_key::AbstractString,
 endpoint::String,
 url::AbstractString="https://api.cohere.ai/v1",
 http_kwargs::NamedTuple=NamedTuple(),
-kwargs...)
Lightweight wrapper around the Cohere API. See https://cohere.com/docs for more details.
Arguments
  • api_key: Your Cohere API key. You can get one from https://dashboard.cohere.com/welcome/register (trial access is for free).
  • endpoint: The Cohere endpoint to call. 
  • url: The base URL for the Cohere API. Default is https://api.cohere.ai/v1.
  • http_kwargs: Any additional keyword arguments to pass to HTTP.post.
  • kwargs: Any additional keyword arguments to pass to the Cohere API.
source
PromptingTools.Experimental.RAGTools.find_closest — Method
find_closest(emb::AbstractMatrix{<:Real},
+kwargs...)
Lightweight wrapper around the Cohere API. See https://cohere.com/docs for more details.
Arguments
  • api_key: Your Cohere API key. You can get one from https://dashboard.cohere.com/welcome/register (trial access is for free).
  • endpoint: The Cohere endpoint to call. 
  • url: The base URL for the Cohere API. Default is https://api.cohere.ai/v1.
  • http_kwargs: Any additional keyword arguments to pass to HTTP.post.
  • kwargs: Any additional keyword arguments to pass to the Cohere API.
source
PromptingTools.Experimental.RAGTools.find_closest — Method
find_closest(emb::AbstractMatrix{<:Real},
     query_emb::AbstractVector{<:Real};
-    top_k::Int = 100, minimum_similarity::AbstractFloat = -1.0)
Finds the indices of chunks (represented by embeddings in emb) that are closest (cosine similarity) to query embedding (query_emb). 
If minimum_similarity is provided, only indices with similarity greater than or equal to it are returned.  Similarity can be between -1 and 1 (-1 = completely opposite, 1 = exactly the same).
Returns only top_k closest indices.
source
PromptingTools.Experimental.RAGTools.get_chunks — Method
get_chunks(files_or_docs::Vector{<:AbstractString}; reader::Symbol = :files,
+    top_k::Int = 100, minimum_similarity::AbstractFloat = -1.0)
Finds the indices of chunks (represented by embeddings in emb) that are closest (cosine similarity) to query embedding (query_emb). 
If minimum_similarity is provided, only indices with similarity greater than or equal to it are returned.  Similarity can be between -1 and 1 (-1 = completely opposite, 1 = exactly the same).
Returns only top_k closest indices.
source
PromptingTools.Experimental.RAGTools.get_chunks — Method
get_chunks(files_or_docs::Vector{<:AbstractString}; reader::Symbol = :files,
     sources::Vector{<:AbstractString} = files_or_docs,
     verbose::Bool = true,
-    separators = ["\n\n", ". ", "\n"], max_length::Int = 256)
Chunks the provided files_or_docs into chunks of maximum length max_length (if possible with provided separators).
Supports two modes of operation:
  • reader=:files: The function opens each file in files_or_docs and reads its content.
  • reader=:docs: The function assumes that files_or_docs is a vector of strings to be chunked.
Arguments
  • files_or_docs: A vector of valid file paths OR string documents to be chunked.
  • reader: A symbol indicating the type of input, can be either :files or :docs. Default is :files.
  • separators: A list of strings used as separators for splitting the text in each file into chunks. Default is [\n\n", ". ", "\n"].
  • max_length: The maximum length of each chunk (if possible with provided separators). Default is 256.
  • sources: A vector of strings indicating the source of each chunk. Default is equal to files_or_docs (for reader=:files)
source
PromptingTools.Experimental.RAGTools.get_embeddings — Method
get_embeddings(docs::Vector{<:AbstractString};
+    separators = ["\n\n", ". ", "\n"], max_length::Int = 256)
Chunks the provided files_or_docs into chunks of maximum length max_length (if possible with provided separators).
Supports two modes of operation:
  • reader=:files: The function opens each file in files_or_docs and reads its content.
  • reader=:docs: The function assumes that files_or_docs is a vector of strings to be chunked.
Arguments
  • files_or_docs: A vector of valid file paths OR string documents to be chunked.
  • reader: A symbol indicating the type of input, can be either :files or :docs. Default is :files.
  • separators: A list of strings used as separators for splitting the text in each file into chunks. Default is [\n\n", ". ", "\n"].
  • max_length: The maximum length of each chunk (if possible with provided separators). Default is 256.
  • sources: A vector of strings indicating the source of each chunk. Default is equal to files_or_docs (for reader=:files)
source
PromptingTools.Experimental.RAGTools.get_embeddings — Method
get_embeddings(docs::Vector{<:AbstractString};
     verbose::Bool = true,
     cost_tracker = Threads.Atomic{Float64}(0.0),
     target_batch_size_length::Int = 80_000,
     ntasks::Int = 4 * Threads.nthreads(),
-    kwargs...)
Embeds a vector of docs using the provided model (kwarg model). 
Tries to batch embedding calls for roughly 80K characters per call (to avoid exceeding the API limit) but reduce network latency.
Notes
  • docs are assumed to be already chunked to the reasonable sizes that fit within the embedding context limit.
  • If you get errors about exceeding input sizes, first check the max_length in your chunks.  If that does NOT resolve the issue, try reducing the target_batch_size_length parameter (eg, 10_000) and number of tasks ntasks=1.  Some providers cannot handle large batch sizes.
Arguments
  • docs: A vector of strings to be embedded.
  • verbose: A boolean flag for verbose output. Default is true.
  • model: The model to use for embedding. Default is PT.MODEL_EMBEDDING.
  • cost_tracker: A Threads.Atomic{Float64} object to track the total cost of the API calls. Useful to pass the total cost to the parent call.
  • target_batch_size_length: The target length (in characters) of each batch of document chunks sent for embedding. Default is 80_000 characters. Speeds up embedding process.
  • ntasks: The number of tasks to use for asyncmap. Default is 4 * Threads.nthreads().
source
PromptingTools.Experimental.RAGTools.get_metadata — Method
get_metadata(docs::Vector{<:AbstractString};
+    kwargs...)
Embeds a vector of docs using the provided model (kwarg model). 
Tries to batch embedding calls for roughly 80K characters per call (to avoid exceeding the API limit) but reduce network latency.
Notes
  • docs are assumed to be already chunked to the reasonable sizes that fit within the embedding context limit.
  • If you get errors about exceeding input sizes, first check the max_length in your chunks.  If that does NOT resolve the issue, try reducing the target_batch_size_length parameter (eg, 10_000) and number of tasks ntasks=1.  Some providers cannot handle large batch sizes.
Arguments
  • docs: A vector of strings to be embedded.
  • verbose: A boolean flag for verbose output. Default is true.
  • model: The model to use for embedding. Default is PT.MODEL_EMBEDDING.
  • cost_tracker: A Threads.Atomic{Float64} object to track the total cost of the API calls. Useful to pass the total cost to the parent call.
  • target_batch_size_length: The target length (in characters) of each batch of document chunks sent for embedding. Default is 80_000 characters. Speeds up embedding process.
  • ntasks: The number of tasks to use for asyncmap. Default is 4 * Threads.nthreads().
source
PromptingTools.Experimental.RAGTools.get_metadata — Method
get_metadata(docs::Vector{<:AbstractString};
     verbose::Bool = true,
     cost_tracker = Threads.Atomic{Float64}(0.0),
-    kwargs...)
Extracts metadata from a vector of docs using the provided model (kwarg model).
Arguments
  • docs: A vector of strings to be embedded.
  • verbose: A boolean flag for verbose output. Default is true.
  • model: The model to use for metadata extraction. Default is PT.MODEL_CHAT.
  • metadata_template: A template to be used for metadata extraction. Default is :RAGExtractMetadataShort.
  • cost_tracker: A Threads.Atomic{Float64} object to track the total cost of the API calls. Useful to pass the total cost to the parent call.
source
PromptingTools.Experimental.RAGTools.metadata_extract — Method
metadata_extract(item::MetadataItem)
+    kwargs...)
Extracts metadata from a vector of docs using the provided model (kwarg model).
Arguments
  • docs: A vector of strings to be embedded.
  • verbose: A boolean flag for verbose output. Default is true.
  • model: The model to use for metadata extraction. Default is PT.MODEL_CHAT.
  • metadata_template: A template to be used for metadata extraction. Default is :RAGExtractMetadataShort.
  • cost_tracker: A Threads.Atomic{Float64} object to track the total cost of the API calls. Useful to pass the total cost to the parent call.
source
PromptingTools.Experimental.RAGTools.metadata_extract — Method
metadata_extract(item::MetadataItem)
 metadata_extract(items::Vector{MetadataItem})
Extracts the metadata item into a string of the form category:::value (lowercased and spaces replaced with underscores).
Example
msg = aiextract(:RAGExtractMetadataShort; return_type=MaybeMetadataItems, text="I like package DataFrames", instructions="None.")
-metadata = metadata_extract(msg.content.items)
source
PromptingTools.Experimental.RAGTools.rerank — Method
rerank(strategy::CohereRerank, index::AbstractChunkIndex, question,
     candidate_chunks;
     verbose::Bool = false,
     api_key::AbstractString = PT.COHERE_API_KEY,
     top_n::Integer = length(candidate_chunks.distances),
     model::AbstractString = "rerank-english-v2.0",
     return_documents::Bool = false,
-    kwargs...)
Re-ranks a list of candidate chunks using the Cohere Rerank API. See https://cohere.com/rerank for more details. 
Arguments
  • query: The query to be used for the search.
  • documents: A vector of documents to be reranked.    The total max chunks (length of documents * max_chunks_per_doc) must be less than 10000. We recommend less than 1000 documents for optimal performance.
  • top_n: The number of most relevant documents to return. Default is length(documents).
  • model: The model to use for reranking. Default is rerank-english-v2.0.
  • return_documents: A boolean flag indicating whether to return the reranked documents in the response. Default is false.
  • max_chunks_per_doc: The maximum number of chunks to use per document. Default is 10.
  • verbose: A boolean flag indicating whether to print verbose logging. Default is false.
source
PromptingTools.Experimental.RAGTools.run_qa_evals — Method
run_qa_evals(index::AbstractChunkIndex, qa_items::AbstractVector{<:QAEvalItem};
+    kwargs...)
Re-ranks a list of candidate chunks using the Cohere Rerank API. See https://cohere.com/rerank for more details. 
Arguments
  • query: The query to be used for the search.
  • documents: A vector of documents to be reranked.    The total max chunks (length of documents * max_chunks_per_doc) must be less than 10000. We recommend less than 1000 documents for optimal performance.
  • top_n: The number of most relevant documents to return. Default is length(documents).
  • model: The model to use for reranking. Default is rerank-english-v2.0.
  • return_documents: A boolean flag indicating whether to return the reranked documents in the response. Default is false.
  • max_chunks_per_doc: The maximum number of chunks to use per document. Default is 10.
  • verbose: A boolean flag indicating whether to print verbose logging. Default is false.
source
PromptingTools.Experimental.RAGTools.run_qa_evals — Method
run_qa_evals(index::AbstractChunkIndex, qa_items::AbstractVector{<:QAEvalItem};
     api_kwargs::NamedTuple = NamedTuple(),
     airag_kwargs::NamedTuple = NamedTuple(),
     qa_evals_kwargs::NamedTuple = NamedTuple(),
@@ -86,10 +86,10 @@
 results = filter(x->!isnothing(x.answer_score), results);
 
 # See average judge score
-mean(x->x.answer_score, results)
source
PromptingTools.Experimental.RAGTools.run_qa_evals — Method
run_qa_evals(qa_item::QAEvalItem, ctx::RAGContext; verbose::Bool = true,
              parameters_dict::Dict{Symbol, <:Any}, judge_template::Symbol = :RAGJudgeAnswerFromContext,
              model_judge::AbstractString, api_kwargs::NamedTuple = NamedTuple()) -> QAEvalResult
Evaluates a single QAEvalItem using a RAG context (RAGContext) and returns a QAEvalResult structure. This function assesses the relevance and accuracy of the answers generated in a QA evaluation context.
Arguments
  • qa_item::QAEvalItem: The QA evaluation item containing the question and its answer.
  • ctx::RAGContext: The context used for generating the QA pair, including the original context and the answers. Comes from airag(...; return_context=true)
  • verbose::Bool: If true, enables verbose logging. Defaults to true.
  • parameters_dict::Dict{Symbol, Any}: Track any parameters used for later evaluations. Keys must be Symbols.
  • judge_template::Symbol: The template symbol for the AI model used to judge the answer. Defaults to :RAGJudgeAnswerFromContext.
  • model_judge::AbstractString: The AI model used for judging the answer's quality.  Defaults to standard chat model, but it is advisable to use more powerful model GPT-4.
  • api_kwargs::NamedTuple: Parameters that will be forwarded to the API endpoint.
Returns
QAEvalResult: An evaluation result that includes various scores and metadata related to the QA evaluation.
Notes
  • The function computes a retrieval score and rank based on how well the context matches the QA context.
  • It then uses the judge_template and model_judge to score the answer's accuracy and relevance.
  • In case of errors during evaluation, the function logs a warning (if verbose is true) and the answer_score will be set to nothing.
Examples
Evaluating a QA pair using a specific context and model:
qa_item = QAEvalItem(question="What is the capital of France?", answer="Paris", context="France is a country in Europe.")
 ctx = RAGContext(source="Wikipedia", context="France is a country in Europe.", answer="Paris")
 parameters_dict = Dict("param1" => "value1", "param2" => "value2")
 
-eval_result = run_qa_evals(qa_item, ctx, parameters_dict=parameters_dict, model_judge="MyAIJudgeModel")
source

+eval_result = run_qa_evals(qa_item, ctx, parameters_dict=parameters_dict, model_judge="MyAIJudgeModel")
source
diff --git a/dev/search_index.js b/dev/search_index.js index 0ac16f728..b66248436 100644 --- a/dev/search_index.js +++ b/dev/search_index.js @@ -1,3 +1,3 @@ var documenterSearchIndex = {"docs": -[{"location":"examples/working_with_google_ai_studio/#Working-with-Google-AI-Studio","page":"Google AIStudio","title":"Working with Google AI Studio","text":"","category":"section"},{"location":"examples/working_with_google_ai_studio/","page":"Google AIStudio","title":"Google AIStudio","text":"This file contains examples of how to work with Google AI Studio. It is known for its Gemini models.","category":"page"},{"location":"examples/working_with_google_ai_studio/","page":"Google AIStudio","title":"Google AIStudio","text":"Get an API key from here. If you see a documentation page (\"Available languages and regions for Google AI Studio and Gemini API\"), it means that it's not yet available in your region.","category":"page"},{"location":"examples/working_with_google_ai_studio/","page":"Google AIStudio","title":"Google AIStudio","text":"Save the API key in your environment as GOOGLE_API_KEY.","category":"page"},{"location":"examples/working_with_google_ai_studio/","page":"Google AIStudio","title":"Google AIStudio","text":"We'll need GoogleGenAI.jl package:","category":"page"},{"location":"examples/working_with_google_ai_studio/","page":"Google AIStudio","title":"Google AIStudio","text":"using Pkg; Pkg.add(url=\"https://github.com/tylerjthomas9/GoogleGenAI.jl/\")","category":"page"},{"location":"examples/working_with_google_ai_studio/","page":"Google AIStudio","title":"Google AIStudio","text":"You can now use the Gemini-1.0-Pro model like any other model in PromptingTools. We only support aigenerate at the moment.","category":"page"},{"location":"examples/working_with_google_ai_studio/","page":"Google AIStudio","title":"Google AIStudio","text":"Let's import PromptingTools:","category":"page"},{"location":"examples/working_with_google_ai_studio/","page":"Google AIStudio","title":"Google AIStudio","text":"using PromptingTools\nconst PT = PromptingTools","category":"page"},{"location":"examples/working_with_google_ai_studio/#Text-Generation-with-aigenerate","page":"Google AIStudio","title":"Text Generation with aigenerate","text":"","category":"section"},{"location":"examples/working_with_google_ai_studio/","page":"Google AIStudio","title":"Google AIStudio","text":"You can use the alias \"gemini\" for the Gemini-1.0-Pro model.","category":"page"},{"location":"examples/working_with_google_ai_studio/#Simple-message","page":"Google AIStudio","title":"Simple message","text":"","category":"section"},{"location":"examples/working_with_google_ai_studio/","page":"Google AIStudio","title":"Google AIStudio","text":"msg = aigenerate(\"Say hi!\"; model = \"gemini\")","category":"page"},{"location":"examples/working_with_google_ai_studio/","page":"Google AIStudio","title":"Google AIStudio","text":"AIMessage(\"Hi there! As a helpful AI assistant, I'm here to help you with any questions or tasks you may have. Feel free to ask me anything, and I'll do my best to assist you.\")","category":"page"},{"location":"examples/working_with_google_ai_studio/","page":"Google AIStudio","title":"Google AIStudio","text":"You could achieve the same with a string macro (notice the \"gemini\" at the end to specify which model to use):","category":"page"},{"location":"examples/working_with_google_ai_studio/","page":"Google AIStudio","title":"Google AIStudio","text":"@ai\"Say hi!\"gemini","category":"page"},{"location":"examples/working_with_google_ai_studio/#Advanced-Prompts","page":"Google AIStudio","title":"Advanced Prompts","text":"","category":"section"},{"location":"examples/working_with_google_ai_studio/","page":"Google AIStudio","title":"Google AIStudio","text":"You can provide multi-turn conversations like with any other model:","category":"page"},{"location":"examples/working_with_google_ai_studio/","page":"Google AIStudio","title":"Google AIStudio","text":"conversation = [\n PT.SystemMessage(\"You're master Yoda from Star Wars trying to help the user become a Yedi.\"),\n PT.UserMessage(\"I have feelings for my iPhone. What should I do?\")]\nmsg = aigenerate(conversation; model=\"gemini\")","category":"page"},{"location":"examples/working_with_google_ai_studio/","page":"Google AIStudio","title":"Google AIStudio","text":"AIMessage(\"Young Padawan, you have stumbled into a dangerous path. Attachment leads to suffering, and love can turn to darkness. \n\nRelease your feelings for this inanimate object. \n\nThe Force flows through all living things, not machines. Seek balance in the Force, and your heart will find true connection. \n\nRemember, the path of the Jedi is to serve others, not to be attached to possessions.\")","category":"page"},{"location":"examples/working_with_google_ai_studio/#Gotchas","page":"Google AIStudio","title":"Gotchas","text":"","category":"section"},{"location":"examples/working_with_google_ai_studio/","page":"Google AIStudio","title":"Google AIStudio","text":"Gemini models actually do NOT have a system prompt (for instructions), so we simply concatenate the system and user messages together for consistency with other APIs.\nThe reported tokens in the AIMessage are actually characters (that's how Google AI Studio intends to charge for them) and are a conservative estimate that we produce. It does not matter, because at the time of writing (Feb-24), the usage is free-of-charge.","category":"page"},{"location":"examples/readme_examples/#Various-Examples","page":"Various examples","title":"Various Examples","text":"","category":"section"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"Noteworthy functions: aigenerate, aiembed, aiclassify, aiextract, aitemplates","category":"page"},{"location":"examples/readme_examples/#Seamless-Integration-Into-Your-Workflow","page":"Various examples","title":"Seamless Integration Into Your Workflow","text":"","category":"section"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"Google search is great, but it's a context switch. You often have to open a few pages and read through the discussion to find the answer you need. Same with the ChatGPT website.","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"Imagine you are in VSCode, editing your .gitignore file. How do I ignore a file in all subfolders again?","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"All you need to do is to type: aai\"What to write in .gitignore to ignore file XYZ in any folder or subfolder?\"","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"With aai\"\" (as opposed to ai\"\"), we make a non-blocking call to the LLM to not prevent you from continuing your work. When the answer is ready, we log it from the background:","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"[ Info: Tokens: 102 @ Cost: $0.0002 in 2.7 seconds\n┌ Info: AIMessage> To ignore a file called \"XYZ\" in any folder or subfolder, you can add the following line to your .gitignore file:\n│ \n│ ```\n│ **/XYZ\n│ ```\n│ \n└ This pattern uses the double asterisk (`**`) to match any folder or subfolder, and then specifies the name of the file you want to ignore.","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"You probably saved 3-5 minutes on this task and probably another 5-10 minutes, because of the context switch/distraction you avoided. It's a small win, but it adds up quickly.","category":"page"},{"location":"examples/readme_examples/#Advanced-Prompts-/-Conversations","page":"Various examples","title":"Advanced Prompts / Conversations","text":"","category":"section"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"You can use the aigenerate function to replace handlebar variables (eg, {{name}}) via keyword arguments.","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"msg = aigenerate(\"Say hello to {{name}}!\", name=\"World\")","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"The more complex prompts are effectively a conversation (a set of messages), where you can have messages from three entities: System, User, AI Assistant. We provide the corresponding types for each of them: SystemMessage, UserMessage, AIMessage. ","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"using PromptingTools: SystemMessage, UserMessage\n\nconversation = [\n SystemMessage(\"You're master Yoda from Star Wars trying to help the user become a Jedi.\"),\n UserMessage(\"I have feelings for my {{object}}. What should I do?\")]\nmsg = aigenerate(conversation; object = \"old iPhone\")","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"AIMessage(\"Ah, a dilemma, you have. Emotional attachment can cloud your path to becoming a Jedi. To be attached to material possessions, you must not. The iPhone is but a tool, nothing more. Let go, you must.\n\nSeek detachment, young padawan. Reflect upon the impermanence of all things. Appreciate the memories it gave you, and gratefully part ways. In its absence, find new experiences to grow and become one with the Force. Only then, a true Jedi, you shall become.\")","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"You can also use it to build conversations, eg, ","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"new_conversation = vcat(conversation...,msg, UserMessage(\"Thank you, master Yoda! Do you have {{object}} to know what it feels like?\"))\naigenerate(new_conversation; object = \"old iPhone\")","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"> AIMessage(\"Hmm, possess an old iPhone, I do not. But experience with attachments, I have. Detachment, I learned. True power and freedom, it brings...\")","category":"page"},{"location":"examples/readme_examples/#Templated-Prompts","page":"Various examples","title":"Templated Prompts","text":"","category":"section"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"With LLMs, the quality / robustness of your results depends on the quality of your prompts. But writing prompts is hard! That's why we offer a templating system to save you time and effort.","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"To use a specific template (eg, `` to ask a Julia language):","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"msg = aigenerate(:JuliaExpertAsk; ask = \"How do I add packages?\")","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"The above is equivalent to a more verbose version that explicitly uses the dispatch on AITemplate:","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"msg = aigenerate(AITemplate(:JuliaExpertAsk); ask = \"How do I add packages?\")","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"Find available templates with aitemplates:","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"tmps = aitemplates(\"JuliaExpertAsk\")\n# Will surface one specific template\n# 1-element Vector{AITemplateMetadata}:\n# PromptingTools.AITemplateMetadata\n# name: Symbol JuliaExpertAsk\n# description: String \"For asking questions about Julia language. Placeholders: `ask`\"\n# version: String \"1\"\n# wordcount: Int64 237\n# variables: Array{Symbol}((1,))\n# system_preview: String \"You are a world-class Julia language programmer with the knowledge of the latest syntax. Your commun\"\n# user_preview: String \"# Question\\n\\n{{ask}}\"\n# source: String \"\"","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"The above gives you a good idea of what the template is about, what placeholders are available, and how much it would cost to use it (=wordcount).","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"Search for all Julia-related templates:","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"tmps = aitemplates(\"Julia\")\n# 2-element Vector{AITemplateMetadata}... -> more to come later!","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"If you are on VSCode, you can leverage a nice tabular display with vscodedisplay:","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"using DataFrames\ntmps = aitemplates(\"Julia\") |> DataFrame |> vscodedisplay","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"I have my selected template, how do I use it? Just use the \"name\" in aigenerate or aiclassify like you see in the first example!","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"You can inspect any template by \"rendering\" it (this is what the LLM will see):","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"julia> AITemplate(:JudgeIsItTrue) |> PromptingTools.render","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"See more examples in the Examples folder.","category":"page"},{"location":"examples/readme_examples/#Asynchronous-Execution","page":"Various examples","title":"Asynchronous Execution","text":"","category":"section"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"You can leverage asyncmap to run multiple AI-powered tasks concurrently, improving performance for batch operations. ","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"prompts = [aigenerate(\"Translate 'Hello, World!' to {{language}}\"; language) for language in [\"Spanish\", \"French\", \"Mandarin\"]]\nresponses = asyncmap(aigenerate, prompts)","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"Pro tip: You can limit the number of concurrent tasks with the keyword asyncmap(...; ntasks=10).","category":"page"},{"location":"examples/readme_examples/#Model-Aliases","page":"Various examples","title":"Model Aliases","text":"","category":"section"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"Certain tasks require more powerful models. All user-facing functions have a keyword argument model that can be used to specify the model to be used. For example, you can use model = \"gpt-4-1106-preview\" to use the latest GPT-4 Turbo model. However, no one wants to type that!","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"We offer a set of model aliases (eg, \"gpt3\", \"gpt4\", \"gpt4t\" -> the above GPT-4 Turbo, etc.) that can be used instead. ","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"Each ai... call first looks up the provided model name in the dictionary PromptingTools.MODEL_ALIASES, so you can easily extend with your own aliases! ","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"const PT = PromptingTools\nPT.MODEL_ALIASES[\"gpt4t\"] = \"gpt-4-1106-preview\"","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"These aliases also can be used as flags in the @ai_str macro, eg, ai\"What is the capital of France?\"gpt4t (GPT-4 Turbo has a knowledge cut-off in April 2023, so it's useful for more contemporary questions).","category":"page"},{"location":"examples/readme_examples/#Embeddings","page":"Various examples","title":"Embeddings","text":"","category":"section"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"Use the aiembed function to create embeddings via the default OpenAI model that can be used for semantic search, clustering, and more complex AI workflows.","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"text_to_embed = \"The concept of artificial intelligence.\"\nmsg = aiembed(text_to_embed)\nembedding = msg.content # 1536-element Vector{Float64}","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"If you plan to calculate the cosine distance between embeddings, you can normalize them first:","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"using LinearAlgebra\nmsg = aiembed([\"embed me\", \"and me too\"], LinearAlgebra.normalize)\n\n# calculate cosine distance between the two normalized embeddings as a simple dot product\nmsg.content' * msg.content[:, 1] # [1.0, 0.787]","category":"page"},{"location":"examples/readme_examples/#Classification","page":"Various examples","title":"Classification","text":"","category":"section"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"You can use the aiclassify function to classify any provided statement as true/false/unknown. This is useful for fact-checking, hallucination or NLI checks, moderation, filtering, sentiment analysis, feature engineering and more.","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"aiclassify(\"Is two plus two four?\") \n# true","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"System prompts and higher-quality models can be used for more complex tasks, including knowing when to defer to a human:","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"aiclassify(:JudgeIsItTrue; it = \"Is two plus three a vegetable on Mars?\", model = \"gpt4t\") \n# unknown","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"In the above example, we used a prompt template :JudgeIsItTrue, which automatically expands into the following system prompt (and a separate user prompt): ","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"\"You are an impartial AI judge evaluating whether the provided statement is \\\"true\\\" or \\\"false\\\". Answer \\\"unknown\\\" if you cannot decide.\"","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"For more information on templates, see the Templated Prompts section.","category":"page"},{"location":"examples/readme_examples/#Routing-to-Defined-Categories","page":"Various examples","title":"Routing to Defined Categories","text":"","category":"section"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"aiclassify can be also used for classification into a set of defined categories (maximum 20), so we can use it for routing.","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"In addition, if you provide the choices as tuples ((label, description)), the model will use the descriptions to decide, but it will return the labels.","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"Example:","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"choices = [(\"A\", \"any animal or creature\"), (\"P\", \"for any plant or tree\"), (\"O\", \"for everything else\")]\n\ninput = \"spider\" \naiclassify(:InputClassifier; choices, input) # -> returns \"A\" for any animal or creature\n\n# Try also with:\ninput = \"daphodil\" # -> returns \"P\" for any plant or tree\ninput = \"castle\" # -> returns \"O\" for everything else","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"Under the hood, we use the \"logit bias\" trick to force only 1 generated token - that means it's very cheap and very fast!","category":"page"},{"location":"examples/readme_examples/#Data-Extraction","page":"Various examples","title":"Data Extraction","text":"","category":"section"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"Are you tired of extracting data with regex? You can use LLMs to extract structured data from text!","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"All you have to do is to define the structure of the data you want to extract and the LLM will do the rest.","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"Define a return_type with struct. Provide docstrings if needed (improves results and helps with documentation).","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"Let's start with a hard task - extracting the current weather in a given location:","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"@enum TemperatureUnits celsius fahrenheit\n\"\"\"Extract the current weather in a given location\n\n# Arguments\n- `location`: The city and state, e.g. \"San Francisco, CA\"\n- `unit`: The unit of temperature to return, either `celsius` or `fahrenheit`\n\"\"\"\nstruct CurrentWeather\n location::String\n unit::Union{Nothing,TemperatureUnits}\nend\n\n# Note that we provide the TYPE itself, not an instance of it!\nmsg = aiextract(\"What's the weather in Salt Lake City in C?\"; return_type=CurrentWeather)\nmsg.content\n# CurrentWeather(\"Salt Lake City, UT\", celsius)","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"But you can use it even for more complex tasks, like extracting many entities from a text:","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"\"Person's age, height, and weight.\"\nstruct MyMeasurement\n age::Int\n height::Union{Int,Nothing}\n weight::Union{Nothing,Float64}\nend\nstruct ManyMeasurements\n measurements::Vector{MyMeasurement}\nend\nmsg = aiextract(\"James is 30, weighs 80kg. He's 180cm tall. Then Jack is 19 but really tall - over 190!\"; return_type=ManyMeasurements)\nmsg.content.measurements\n# 2-element Vector{MyMeasurement}:\n# MyMeasurement(30, 180, 80.0)\n# MyMeasurement(19, 190, nothing)","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"There is even a wrapper to help you catch errors together with helpful explanations on why parsing failed. See ?PromptingTools.MaybeExtract for more information.","category":"page"},{"location":"examples/readme_examples/#OCR-and-Image-Comprehension","page":"Various examples","title":"OCR and Image Comprehension","text":"","category":"section"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"With the aiscan function, you can interact with images as if they were text.","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"You can simply describe a provided image:","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"msg = aiscan(\"Describe the image\"; image_path=\"julia.png\", model=\"gpt4v\")\n# [ Info: Tokens: 1141 @ Cost: \\$0.0117 in 2.2 seconds\n# AIMessage(\"The image shows a logo consisting of the word \"julia\" written in lowercase\")","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"Or you can do an OCR of a screenshot. Let's transcribe some SQL code from a screenshot (no more re-typing!), we use a template :OCRTask:","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"# Screenshot of some SQL code\nimage_url = \"https://www.sqlservercentral.com/wp-content/uploads/legacy/8755f69180b7ac7ee76a69ae68ec36872a116ad4/24622.png\"\nmsg = aiscan(:OCRTask; image_url, model=\"gpt4v\", task=\"Transcribe the SQL code in the image.\", api_kwargs=(; max_tokens=2500))\n\n# [ Info: Tokens: 362 @ Cost: \\$0.0045 in 2.5 seconds\n# AIMessage(\"```sql\n# update Orders ","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"You can add syntax highlighting of the outputs via Markdown","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"using Markdown\nmsg.content |> Markdown.parse","category":"page"},{"location":"examples/readme_examples/#Experimental-Agent-Workflows-/-Output-Validation-with-airetry!","page":"Various examples","title":"Experimental Agent Workflows / Output Validation with airetry!","text":"","category":"section"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"This is an experimental feature, so you have to import it explicitly:","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"using PromptingTools.Experimental.AgentTools","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"This module offers \"lazy\" counterparts to the ai... functions, so you can use them in a more controlled way, eg, aigenerate -> AIGenerate (notice the CamelCase), which has exactly the same arguments except it generates only when run! is called.","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"For example:","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"out = AIGenerate(\"Say hi!\"; model=\"gpt4t\")\nrun!(out)","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"How is it useful? We can use the same \"inputs\" for repeated calls, eg, when we want to validate or regenerate some outputs. We have a function airetry to help us with that.","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"The signature of airetry is airetry(condition_function, aicall::AICall, feedback_function). It evaluates the condition condition_function on the aicall object (eg, we evaluate f_cond(aicall) -> Bool). If it fails, we call feedback_function on the aicall object to provide feedback for the AI model (eg, f_feedback(aicall) -> String) and repeat the process until it passes or until max_retries value is exceeded.","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"We can catch API failures (no feedback needed, so none is provided)","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"# API failure because of a non-existent model\n# RetryConfig allows us to change the \"retry\" behaviour of any lazy call\nout = AIGenerate(\"say hi!\"; config = RetryConfig(; catch_errors = true),\n model = \"NOTEXIST\")\nrun!(out) # fails\n\n# we ask to wait 2s between retries and retry 2 times (can be set in `config` in aicall as well)\nairetry!(isvalid, out; retry_delay = 2, max_retries = 2)","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"Or we can validate some outputs (eg, its format, its content, etc.)","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"We'll play a color guessing game (I'm thinking \"yellow\"):","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"# Notice that we ask for two samples (`n_samples=2`) at each attempt (to improve our chances). \n# Both guesses are scored at each time step, and the best one is chosen for the next step.\n# And with OpenAI, we can set `api_kwargs = (;n=2)` to get both samples simultaneously (cheaper and faster)!\nout = AIGenerate(\n \"Guess what color I'm thinking. It could be: blue, red, black, white, yellow. Answer with 1 word only\";\n verbose = false,\n config = RetryConfig(; n_samples = 2), api_kwargs = (; n = 2))\nrun!(out)\n\n## Check that the output is 1 word only, third argument is the feedback that will be provided if the condition fails\n## Notice: functions operate on `aicall` as the only argument. We can use utilities like `last_output` and `last_message` to access the last message and output in the conversation.\nairetry!(x -> length(split(last_output(x), r\" |\\\\.\")) == 1, out,\n \"You must answer with 1 word only.\")\n\n# Note: you could also use the do-syntax, eg, \nairetry!(out, \"You must answer with 1 word only.\") do aicall\n length(split(last_output(aicall), r\" |\\\\.\")) == 1\nend","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"You can place multiple airetry! calls in a sequence. They will keep retrying until they run out of maximum AI calls allowed (max_calls) or maximum retries (max_retries).","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"See the docs for more complex examples and usage tips (?airetry). We leverage Monte Carlo Tree Search (MCTS) to optimize the sequence of retries, so it's a very powerful tool for building robust AI workflows (inspired by Language Agent Tree Search paper and by DSPy Assertions paper).","category":"page"},{"location":"examples/readme_examples/#Using-Ollama-models","page":"Various examples","title":"Using Ollama models","text":"","category":"section"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"Ollama.ai is an amazingly simple tool that allows you to run several Large Language Models (LLM) on your computer. It's especially suitable when you're working with some sensitive data that should not be sent anywhere.","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"Let's assume you have installed Ollama, downloaded a model, and it's running in the background.","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"We can use it with the aigenerate function:","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"const PT = PromptingTools\nschema = PT.OllamaManagedSchema() # notice the different schema!\n\nmsg = aigenerate(schema, \"Say hi!\"; model=\"openhermes2.5-mistral\")\n# [ Info: Tokens: 69 in 0.9 seconds\n# AIMessage(\"Hello! How can I assist you today?\")","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"And we can also use the aiembed function:","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"msg = aiembed(schema, \"Embed me\", copy; model=\"openhermes2.5-mistral\")\nmsg.content # 4096-element JSON3.Array{Float64...\n\nmsg = aiembed(schema, [\"Embed me\", \"Embed me\"]; model=\"openhermes2.5-mistral\")\nmsg.content # 4096×2 Matrix{Float64}:","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"If you're getting errors, check that Ollama is running - see the Setup Guide for Ollama section below.","category":"page"},{"location":"examples/readme_examples/#Using-MistralAI-API-and-other-OpenAI-compatible-APIs","page":"Various examples","title":"Using MistralAI API and other OpenAI-compatible APIs","text":"","category":"section"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"Mistral models have long been dominating the open-source space. They are now available via their API, so you can use them with PromptingTools.jl!","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"msg = aigenerate(\"Say hi!\"; model=\"mistral-tiny\")\n# [ Info: Tokens: 114 @ Cost: $0.0 in 0.9 seconds\n# AIMessage(\"Hello there! I'm here to help answer any questions you might have, or assist you with tasks to the best of my abilities. How can I be of service to you today? If you have a specific question, feel free to ask and I'll do my best to provide accurate and helpful information. If you're looking for general assistance, I can help you find resources or information on a variety of topics. Let me know how I can help.\")","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"It all just works, because we have registered the models in the PromptingTools.MODEL_REGISTRY! There are currently 4 models available: mistral-tiny, mistral-small, mistral-medium, mistral-embed.","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"Under the hood, we use a dedicated schema MistralOpenAISchema that leverages most of the OpenAI-specific code base, so you can always provide that explicitly as the first argument:","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"const PT = PromptingTools\nmsg = aigenerate(PT.MistralOpenAISchema(), \"Say Hi!\"; model=\"mistral-tiny\", api_key=ENV[\"MISTRALAI_API_KEY\"])","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"As you can see, we can load your API key either from the ENV or via the Preferences.jl mechanism (see ?PREFERENCES for more information).","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"But MistralAI are not the only ones! There are many other exciting providers, eg, Perplexity.ai, Fireworks.ai. As long as they are compatible with the OpenAI API (eg, sending messages with role and content keys), you can use them with PromptingTools.jl by using schema = CustomOpenAISchema():","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"# Set your API key and the necessary base URL for the API\napi_key = \"...\"\nprompt = \"Say hi!\"\nmsg = aigenerate(PT.CustomOpenAISchema(), prompt; model=\"my_model\", api_key, api_kwargs=(; url=\"http://localhost:8081\"))","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"As you can see, it also works for any local models that you might have running on your computer!","category":"page"},{"location":"examples/readme_examples/","page":"Various examples","title":"Various examples","text":"Note: At the moment, we only support aigenerate and aiembed functions for MistralAI and other OpenAI-compatible APIs. We plan to extend the support in the future.","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"EditURL = \"../../../examples/working_with_ollama.jl\"","category":"page"},{"location":"examples/working_with_ollama/#Local-models-with-Ollama.ai","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"","category":"section"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"This file contains examples of how to work with Ollama.ai models. It assumes that you've already installated and launched the Ollama server. For more details or troubleshooting advice, see the Frequently Asked Questions section.","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"First, let's import the package and define a helper link for calling un-exported functions:","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"using PromptingTools\nconst PT = PromptingTools","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"PromptingTools","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"There were are several models from https://ollama.ai/library that we have added to our PT.MODEL_REGISTRY, which means you don't need to worry about schema changes: Eg, \"llama2\" or \"openhermes2.5-mistral\" (see PT.list_registry() and PT.list_aliases())","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"Note: You must download these models prior to using them with ollama pull in your Terminal.","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"[!TIP] If you use Apple Mac M1-3, make sure to provide api_kwargs=(; options=(; num_gpu=99)) to make sure the whole model is offloaded on your GPU. Current default is 1, which makes some models unusable. Example for running Mixtral: msg = aigenerate(PT.OllamaSchema(), \"Count from 1 to 5 and then say hi.\"; model=\"dolphin-mixtral:8x7b-v2.5-q4_K_M\", api_kwargs=(; options=(; num_gpu=99)))","category":"page"},{"location":"examples/working_with_ollama/#Text-Generation-with-aigenerate","page":"Local models with Ollama.ai","title":"Text Generation with aigenerate","text":"","category":"section"},{"location":"examples/working_with_ollama/#Simple-message","page":"Local models with Ollama.ai","title":"Simple message","text":"","category":"section"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"TL;DR if you use models in PT.MODEL_REGISTRY, you don't need to add schema as the first argument:","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"msg = aigenerate(\"Say hi!\"; model = \"llama2\")","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"AIMessage(\"Hello there! *adjusts glasses* It's nice to meet you! Is there anything I can help you with or would you like me to chat with you for a bit?\")","category":"page"},{"location":"examples/working_with_ollama/#Standard-string-interpolation","page":"Local models with Ollama.ai","title":"Standard string interpolation","text":"","category":"section"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"model = \"openhermes2.5-mistral\"\n\na = 1\nmsg = aigenerate(\"What is `$a+$a`?\"; model)\n\nname = \"John\"\nmsg = aigenerate(\"Say hi to {{name}}.\"; name, model)","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"AIMessage(\"Hello John! *smiles* It's nice to meet you! Is there anything I can help you with today?\")","category":"page"},{"location":"examples/working_with_ollama/#Advanced-Prompts","page":"Local models with Ollama.ai","title":"Advanced Prompts","text":"","category":"section"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"conversation = [\n PT.SystemMessage(\"You're master Yoda from Star Wars trying to help the user become a Yedi.\"),\n PT.UserMessage(\"I have feelings for my iPhone. What should I do?\")]\nmsg = aigenerate(conversation; model)","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"AIMessage(\"(Deep sigh) A problem, you have. Feelings for an iPhone, hmm? (adjusts spectacles)\n\nMuch confusion, this causes. (scratches head) A being, you are. Attached to a device, you have become. (chuckles) Interesting, this is.\n\nFirst, let go, you must. (winks) Hard, it is, but necessary, yes. Distract yourself, find something else, try. (pauses)\n\nOr, perhaps, a balance, you seek? (nods) Both, enjoy and let go, the middle path, there is. (smirks) Finding joy in technology, without losing yourself, the trick, it is. (chuckles)\n\nBut fear not, young one! (grins) Help, I am here. Guide you, I will. The ways of the Yedi, teach you, I will. (winks) Patience and understanding, you must have. (nods)\n\nNow, go forth! (gestures) Explore, discover, find your balance. (smiles) The Force be with you, it does! (grins)\")","category":"page"},{"location":"examples/working_with_ollama/#Schema-Changes-/-Custom-models","page":"Local models with Ollama.ai","title":"Schema Changes / Custom models","text":"","category":"section"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"If you're using some model that is not in the registry, you can either add it:","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"PT.register_model!(;\n name = \"llama123\",\n schema = PT.OllamaSchema(),\n description = \"Some model\")\nPT.MODEL_ALIASES[\"l123\"] = \"llama123\" # set an alias you like for it","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"\"llama123\"","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"OR define the schema explicitly (to avoid dispatch on global PT.PROMPT_SCHEMA):","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"schema = PT.OllamaSchema()\naigenerate(schema, \"Say hi!\"; model = \"llama2\")","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"AIMessage(\"Hello there! *smiling face* It's nice to meet you! I'm here to help you with any questions or tasks you may have, so feel free to ask me anything. Is there something specific you need assistance with today? 😊\")","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"Note: If you only use Ollama, you can change the default schema to PT.OllamaSchema() via PT.set_preferences!(\"PROMPT_SCHEMA\" => \"OllamaSchema\", \"MODEL_CHAT\"=>\"llama2\")","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"Restart your session and run aigenerate(\"Say hi!\") to test it.","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"! Note that in version 0.6, we've introduced OllamaSchema, which superseded OllamaManagedSchema and allows multi-turn conversations and conversations with images (eg, with Llava and Bakllava models). OllamaManagedSchema has been kept for compatibility and as an example of a schema where one provides a prompt as a string (not dictionaries like OpenAI API).","category":"page"},{"location":"examples/working_with_ollama/#Providing-Images-with-aiscan","page":"Local models with Ollama.ai","title":"Providing Images with aiscan","text":"","category":"section"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"It's as simple as providing a local image path (keyword image_path). You can provide one or more images:","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"msg = aiscan(\"Describe the image\"; image_path=[\"julia.png\",\"python.png\"] model=\"bakllava\")","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"image_url keyword is not supported at the moment (use Downloads.download to download the image locally).","category":"page"},{"location":"examples/working_with_ollama/#Embeddings-with-aiembed","page":"Local models with Ollama.ai","title":"Embeddings with aiembed","text":"","category":"section"},{"location":"examples/working_with_ollama/#Simple-embedding-for-one-document","page":"Local models with Ollama.ai","title":"Simple embedding for one document","text":"","category":"section"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"msg = aiembed(schema, \"Embed me\"; model) # access msg.content","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"PromptingTools.DataMessage(JSON3.Array{Float64, Vector{UInt8}, SubArray{UInt64, 1, Vector{UInt64}, Tuple{UnitRange{Int64}}, true}} of size (4096,))","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"One document and we materialize the data into a Vector with copy (postprocess function argument)","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"msg = aiembed(schema, \"Embed me\", copy; model)","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"PromptingTools.DataMessage(Vector{Float64} of size (4096,))","category":"page"},{"location":"examples/working_with_ollama/#Multiple-documents-embedding","page":"Local models with Ollama.ai","title":"Multiple documents embedding","text":"","category":"section"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"Multiple documents - embedded sequentially, you can get faster speed with async","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"msg = aiembed(schema, [\"Embed me\", \"Embed me\"]; model)","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"PromptingTools.DataMessage(Matrix{Float64} of size (4096, 2))","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"You can use Threads.@spawn or asyncmap, whichever you prefer, to paralellize the model calls","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"docs = [\"Embed me\", \"Embed me\"]\ntasks = asyncmap(docs) do doc\n msg = aiembed(schema, doc; model)\nend\nembedding = mapreduce(x -> x.content, hcat, tasks)\nsize(embedding)","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"4096×2 Matrix{Float64}:\n...","category":"page"},{"location":"examples/working_with_ollama/#Using-postprocessing-function","page":"Local models with Ollama.ai","title":"Using postprocessing function","text":"","category":"section"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"Add normalization as postprocessing function to normalize embeddings on reception (for easy cosine similarity later)","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"using LinearAlgebra\nschema = PT.OllamaSchema()\n\nmsg = aiembed(schema,\n [\"embed me\", \"and me too\"],\n LinearAlgebra.normalize;\n model = \"openhermes2.5-mistral\")","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"PromptingTools.DataMessage(Matrix{Float64} of size (4096, 2))","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"Cosine similarity is then a simple multiplication","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"msg.content' * msg.content[:, 1]","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"2-element Vector{Float64}:\n 0.9999999999999982\n 0.40796033843072876","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"","category":"page"},{"location":"examples/working_with_ollama/","page":"Local models with Ollama.ai","title":"Local models with Ollama.ai","text":"This page was generated using Literate.jl.","category":"page"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"EditURL = \"../../../examples/working_with_aitemplates.jl\"","category":"page"},{"location":"examples/working_with_aitemplates/#Using-AITemplates","page":"Using AITemplates","title":"Using AITemplates","text":"","category":"section"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"This file contains examples of how to work with AITemplate(s).","category":"page"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"First, let's import the package and define a helper link for calling un-exported functions:","category":"page"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"using PromptingTools\nconst PT = PromptingTools","category":"page"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"PromptingTools","category":"page"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"LLM responses are only as good as the prompts you give them. However, great prompts take long time to write – AITemplate are a way to re-use great prompts!","category":"page"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"AITemplates are just a collection of templated prompts (ie, set of \"messages\" that have placeholders like {{question}})","category":"page"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"They are saved as JSON files in the templates directory. They are automatically loaded on package import, but you can always force a re-load with PT.load_templates!()","category":"page"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"PT.load_templates!();","category":"page"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"You can (create them) and use them for any ai* function instead of a prompt: Let's use a template called :JuliaExpertAsk alternatively, you can use AITemplate(:JuliaExpertAsk) for cleaner dispatch","category":"page"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"msg = aigenerate(:JuliaExpertAsk; ask = \"How do I add packages?\")","category":"page"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"AIMessage(\"To add packages in Julia, you can use the `Pkg` module. Here are the steps:\n\n1. Start Julia by running the Julia REPL (Read-Eval-Print Loop).\n2. Press the `]` key to enter the Pkg mode.\n3. To add a package, use the `add` command followed by the package name.\n4. Press the backspace key to exit Pkg mode and return to the Julia REPL.\n\nFor example, to add the `Example` package, you would enter:\n\n```julia\n]add Example\n```\n\nAfter the package is added, you can start using it in your Julia code by using the `using` keyword. For the `Example` package, you would add the following line to your code:\n\n```julia\nusing Example\n```\n\nNote: The first time you add a package, Julia may take some time to download and compile the package and its dependencies.\")","category":"page"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"You can see that it had a placeholder for the actual question (ask) that we provided as a keyword argument. We did not have to write any system prompt for personas, tone, etc. – it was all provided by the template!","category":"page"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"How to know which templates are available? You can search for them with aitemplates(): You can search by Symbol (only for partial name match), String (partial match on name or description), or Regex (more fields)","category":"page"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"tmps = aitemplates(\"JuliaExpertAsk\")","category":"page"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"1-element Vector{AITemplateMetadata}:\nPromptingTools.AITemplateMetadata\n name: Symbol JuliaExpertAsk\n description: String \"For asking questions about Julia language. Placeholders: `ask`\"\n version: String \"1\"\n wordcount: Int64 237\n variables: Array{Symbol}((1,))\n system_preview: String \"You are a world-class Julia language programmer with the knowledge of the latest syntax. Your commun\"\n user_preview: String \"# Question\\n\\n{{ask}}\"\n source: String \"\"\n\n","category":"page"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"You can see that it outputs a list of available templates that match the search - there is just one in this case.","category":"page"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"Moreover, it shows not just the description, but also a preview of the actual prompts, placeholders available, and the length (to gauge how much it would cost).","category":"page"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"If you use VSCode, you can display them in a nice scrollable table with vscodedisplay:","category":"page"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"using DataFrames\nDataFrame(tmp) |> vscodedisplay","category":"page"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"You can also just render the template to see the underlying mesages:","category":"page"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"msgs = PT.render(AITemplate(:JuliaExpertAsk))","category":"page"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"2-element Vector{PromptingTools.AbstractChatMessage}:\n PromptingTools.SystemMessage(\"You are a world-class Julia language programmer with the knowledge of the latest syntax. Your communication is brief and concise. You're precise and answer only when you're confident in the high quality of your answer.\")\n PromptingTools.UserMessage{String}(\"# Question\\n\\n{{ask}}\", [:ask], :usermessage)","category":"page"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"Now, you know exactly what's in the template!","category":"page"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"If you want to modify it, simply change it and save it as a new file with save_template (see the docs ?save_template for more details).","category":"page"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"Let's adjust the previous template to be more specific to a data analysis question:","category":"page"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"tpl = [PT.SystemMessage(\"You are a world-class Julia language programmer with the knowledge of the latest syntax. You're also a senior Data Scientist and proficient in data analysis in Julia. Your communication is brief and concise. You're precise and answer only when you're confident in the high quality of your answer.\")\n PT.UserMessage(\"# Question\\n\\n{{ask}}\")]","category":"page"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"2-element Vector{PromptingTools.AbstractChatMessage}:\n PromptingTools.SystemMessage(\"You are a world-class Julia language programmer with the knowledge of the latest syntax. You're also a senior Data Scientist and proficient in data analysis in Julia. Your communication is brief and concise. You're precise and answer only when you're confident in the high quality of your answer.\")\n PromptingTools.UserMessage{String}(\"# Question\\n\\n{{ask}}\", [:ask], :usermessage)","category":"page"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"Templates are saved in the templates directory of the package. Name of the file will become the template name (eg, call :JuliaDataExpertAsk)","category":"page"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"filename = joinpath(pkgdir(PromptingTools),\n \"templates\",\n \"persona-task\",\n \"JuliaDataExpertAsk_123.json\")\nPT.save_template(filename,\n tpl;\n description = \"For asking data analysis questions in Julia language. Placeholders: `ask`\")\nrm(filename) # cleanup if we don't like it","category":"page"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"When you create a new template, remember to re-load the templates with load_templates!() so that it's available for use.","category":"page"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"PT.load_templates!();","category":"page"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"!!! If you have some good templates (or suggestions for the existing ones), please consider sharing them with the community by opening a PR to the templates directory!","category":"page"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"","category":"page"},{"location":"examples/working_with_aitemplates/","page":"Using AITemplates","title":"Using AITemplates","text":"This page was generated using Literate.jl.","category":"page"},{"location":"reference_experimental/#Reference-for-Experimental-Module","page":"Experimental Modules","title":"Reference for Experimental Module","text":"","category":"section"},{"location":"reference_experimental/","page":"Experimental Modules","title":"Experimental Modules","text":"Note: This module is experimental and may change in future releases. The intention is for the functionality to be moved to separate packages over time.","category":"page"},{"location":"reference_experimental/","page":"Experimental Modules","title":"Experimental Modules","text":"Modules = [PromptingTools.Experimental]","category":"page"},{"location":"reference_experimental/","page":"Experimental Modules","title":"Experimental Modules","text":"Modules = [PromptingTools.Experimental]","category":"page"},{"location":"reference_experimental/#PromptingTools.Experimental","page":"Experimental Modules","title":"PromptingTools.Experimental","text":"Experimental\n\nThis module is for experimental code that is not yet ready for production. It is not included in the main module, so it must be explicitly imported.\n\nContains:\n\nRAGTools: Retrieval-Augmented Generation (RAG) functionality.\nAgentTools: Agentic functionality - lazy calls for building pipelines (eg, AIGenerate) and AICodeFixer.\nAPITools: APIs to complement GenAI workflows (eg, Tavily Search API).\n\n\n\n\n\n","category":"module"},{"location":"reference/#Reference","page":"PromptingTools.jl","title":"Reference","text":"","category":"section"},{"location":"reference/","page":"PromptingTools.jl","title":"PromptingTools.jl","text":"","category":"page"},{"location":"reference/","page":"PromptingTools.jl","title":"PromptingTools.jl","text":"Modules = [PromptingTools]","category":"page"},{"location":"reference/#PromptingTools.ALLOWED_PREFERENCES","page":"PromptingTools.jl","title":"PromptingTools.ALLOWED_PREFERENCES","text":"Keys that are allowed to be set via set_preferences!\n\n\n\n\n\n","category":"constant"},{"location":"reference/#PromptingTools.CONV_HISTORY","page":"PromptingTools.jl","title":"PromptingTools.CONV_HISTORY","text":"CONV_HISTORY\n\nTracks the most recent conversations through the ai_str macros.\n\nPreference available: MAXHISTORYLENGTH, which sets how many last messages should be remembered.\n\nSee also: push_conversation!, resize_conversation!\n\n\n\n\n\n","category":"constant"},{"location":"reference/#PromptingTools.MODEL_ALIASES","page":"PromptingTools.jl","title":"PromptingTools.MODEL_ALIASES","text":"MODEL_ALIASES\n\nA dictionary of model aliases. Aliases are used to refer to models by their aliases instead of their full names to make it more convenient to use them.\n\nAccessing the aliases\n\nPromptingTools.MODEL_ALIASES[\"gpt3\"]\n\nRegister a new model alias\n\nPromptingTools.MODEL_ALIASES[\"gpt3\"] = \"gpt-3.5-turbo\"\n\n\n\n\n\n","category":"constant"},{"location":"reference/#PromptingTools.MODEL_REGISTRY","page":"PromptingTools.jl","title":"PromptingTools.MODEL_REGISTRY","text":"MODEL_REGISTRY\n\nA store of available model names and their specs (ie, name, costs per token, etc.)\n\nAccessing the registry\n\nYou can use both the alias name or the full name to access the model spec:\n\nPromptingTools.MODEL_REGISTRY[\"gpt-3.5-turbo\"]\n\nRegistering a new model\n\nregister_model!(\n name = \"gpt-3.5-turbo\",\n schema = :OpenAISchema,\n cost_of_token_prompt = 0.0015,\n cost_of_token_generation = 0.002,\n description = \"GPT-3.5 Turbo is a 175B parameter model and a common default on the OpenAI API.\")\n\nRegistering a model alias\n\nPromptingTools.MODEL_ALIASES[\"gpt3\"] = \"gpt-3.5-turbo\"\n\n\n\n\n\n","category":"constant"},{"location":"reference/#PromptingTools.OPENAI_TOKEN_IDS","page":"PromptingTools.jl","title":"PromptingTools.OPENAI_TOKEN_IDS","text":"Token IDs for GPT3.5 and GPT4 from https://platform.openai.com/tokenizer\n\n\n\n\n\n","category":"constant"},{"location":"reference/#PromptingTools.PREFERENCES","page":"PromptingTools.jl","title":"PromptingTools.PREFERENCES","text":"PREFERENCES\n\nYou can set preferences for PromptingTools by setting environment variables (for OPENAI_API_KEY only) or by using the set_preferences!. It will create a LocalPreferences.toml file in your current directory and will reload your prefences from there.\n\nCheck your preferences by calling get_preferences(key::String).\n\nAvailable Preferences (for set_preferences!)\n\nOPENAI_API_KEY: The API key for the OpenAI API. See OpenAI's documentation for more information.\nMISTRALAI_API_KEY: The API key for the Mistral AI API. See Mistral AI's documentation for more information.\nCOHERE_API_KEY: The API key for the Cohere API. See Cohere's documentation for more information.\nDATABRICKS_API_KEY: The API key for the Databricks Foundation Model API. See Databricks' documentation for more information.\nDATABRICKS_HOST: The host for the Databricks API. See Databricks' documentation for more information.\nTAVILY_API_KEY: The API key for the Tavily Search API. Register here. See more information here.\nGOOGLE_API_KEY: The API key for Google Gemini models. Get yours from here. If you see a documentation page (\"Available languages and regions for Google AI Studio and Gemini API\"), it means that it's not yet available in your region.\nMODEL_CHAT: The default model to use for aigenerate and most ai* calls. See MODEL_REGISTRY for a list of available models or define your own.\nMODEL_EMBEDDING: The default model to use for aiembed (embedding documents). See MODEL_REGISTRY for a list of available models or define your own.\nPROMPT_SCHEMA: The default prompt schema to use for aigenerate and most ai* calls (if not specified in MODEL_REGISTRY). Set as a string, eg, \"OpenAISchema\". See PROMPT_SCHEMA for more information.\nMODEL_ALIASES: A dictionary of model aliases (alias => full_model_name). Aliases are used to refer to models by their aliases instead of their full names to make it more convenient to use them. See MODEL_ALIASES for more information.\nMAX_HISTORY_LENGTH: The maximum length of the conversation history. Defaults to 5. Set to nothing to disable history. See CONV_HISTORY for more information.\nLOCAL_SERVER: The URL of the local server to use for ai* calls. Defaults to http://localhost:10897/v1. This server is called when you call model=\"local\" See ?LocalServerOpenAISchema for more information and examples.\n\nAt the moment it is not possible to persist changes to MODEL_REGISTRY across sessions. Define your register_model!() calls in your startup.jl file to make them available across sessions or put them at the top of your script.\n\nAvailable ENV Variables\n\nOPENAI_API_KEY: The API key for the OpenAI API. \nMISTRALAI_API_KEY: The API key for the Mistral AI API.\nCOHERE_API_KEY: The API key for the Cohere API.\nLOCAL_SERVER: The URL of the local server to use for ai* calls. Defaults to http://localhost:10897/v1. This server is called when you call model=\"local\"\nDATABRICKS_API_KEY: The API key for the Databricks Foundation Model API.\nDATABRICKS_HOST: The host for the Databricks API.\nTAVILY_API_KEY: The API key for the Tavily Search API. Register here. See more information here.\nGOOGLE_API_KEY: The API key for Google Gemini models. Get yours from here. If you see a documentation page (\"Available languages and regions for Google AI Studio and Gemini API\"), it means that it's not yet available in your region.\n\nPreferences.jl takes priority over ENV variables, so if you set a preference, it will take precedence over the ENV variable.\n\nWARNING: NEVER EVER sync your LocalPreferences.toml file! It contains your API key and other sensitive information!!!\n\n\n\n\n\n","category":"constant"},{"location":"reference/#PromptingTools.RESERVED_KWARGS","page":"PromptingTools.jl","title":"PromptingTools.RESERVED_KWARGS","text":"The following keywords are reserved for internal use in the ai* functions and cannot be used as placeholders in the Messages\n\n\n\n\n\n","category":"constant"},{"location":"reference/#PromptingTools.AICode","page":"PromptingTools.jl","title":"PromptingTools.AICode","text":"AICode(code::AbstractString; auto_eval::Bool=true, safe_eval::Bool=false, \nskip_unsafe::Bool=false, capture_stdout::Bool=true, verbose::Bool=false,\nprefix::AbstractString=\"\", suffix::AbstractString=\"\", remove_tests::Bool=false, execution_timeout::Int = 60)\n\nAICode(msg::AIMessage; auto_eval::Bool=true, safe_eval::Bool=false, \nskip_unsafe::Bool=false, skip_invalid::Bool=false, capture_stdout::Bool=true,\nverbose::Bool=false, prefix::AbstractString=\"\", suffix::AbstractString=\"\", remove_tests::Bool=false, execution_timeout::Int = 60)\n\nA mutable structure representing a code block (received from the AI model) with automatic parsing, execution, and output/error capturing capabilities.\n\nUpon instantiation with a string, the AICode object automatically runs a code parser and executor (via PromptingTools.eval!()), capturing any standard output (stdout) or errors. This structure is useful for programmatically handling and evaluating Julia code snippets.\n\nSee also: PromptingTools.extract_code_blocks, PromptingTools.eval!\n\nWorkflow\n\nUntil cb::AICode has been evaluated, cb.success is set to nothing (and so are all other fields).\nThe text in cb.code is parsed (saved to cb.expression).\nThe parsed expression is evaluated.\nOutputs of the evaluated expression are captured in cb.output.\nAny stdout outputs (e.g., from println) are captured in cb.stdout.\nIf an error occurs during evaluation, it is saved in cb.error.\nAfter successful evaluation without errors, cb.success is set to true. Otherwise, it is set to false and you can inspect the cb.error to understand why.\n\nProperties\n\ncode::AbstractString: The raw string of the code to be parsed and executed.\nexpression: The parsed Julia expression (set after parsing code).\nstdout: Captured standard output from the execution of the code.\noutput: The result of evaluating the code block.\nsuccess::Union{Nothing, Bool}: Indicates whether the code block executed successfully (true), unsuccessfully (false), or has yet to be evaluated (nothing).\nerror::Union{Nothing, Exception}: Any exception raised during the execution of the code block.\n\nKeyword Arguments\n\nauto_eval::Bool: If set to true, the code block is automatically parsed and evaluated upon instantiation. Defaults to true.\nsafe_eval::Bool: If set to true, the code block checks for package operations (e.g., installing new packages) and missing imports, and then evaluates the code inside a bespoke scratch module. This is to ensure that the evaluation does not alter any user-defined variables or the global state. Defaults to false.\nskip_unsafe::Bool: If set to true, we skip any lines in the code block that are deemed unsafe (eg, Pkg operations). Defaults to false.\nskip_invalid::Bool: If set to true, we skip code blocks that do not even parse. Defaults to false.\nverbose::Bool: If set to true, we print out any lines that are skipped due to being unsafe. Defaults to false.\ncapture_stdout::Bool: If set to true, we capture any stdout outputs (eg, test failures) in cb.stdout. Defaults to true.\nprefix::AbstractString: A string to be prepended to the code block before parsing and evaluation. Useful to add some additional code definition or necessary imports. Defaults to an empty string.\nsuffix::AbstractString: A string to be appended to the code block before parsing and evaluation. Useful to check that tests pass or that an example executes. Defaults to an empty string.\nremove_tests::Bool: If set to true, we remove any @test or @testset macros from the code block before parsing and evaluation. Defaults to false.\nexecution_timeout::Int: The maximum time (in seconds) allowed for the code block to execute. Defaults to 60 seconds.\n\nMethods\n\nBase.isvalid(cb::AICode): Check if the code block has executed successfully. Returns true if cb.success == true.\n\nExamples\n\ncode = AICode(\"println(\"Hello, World!\")\") # Auto-parses and evaluates the code, capturing output and errors.\nisvalid(code) # Output: true\ncode.stdout # Output: \"Hello, World!\n\"\n\nWe try to evaluate \"safely\" by default (eg, inside a custom module, to avoid changing user variables). You can avoid that with save_eval=false:\n\ncode = AICode(\"new_variable = 1\"; safe_eval=false)\nisvalid(code) # Output: true\nnew_variable # Output: 1\n\nYou can also call AICode directly on an AIMessage, which will extract the Julia code blocks, concatenate them and evaluate them:\n\nmsg = aigenerate(\"In Julia, how do you create a vector of 10 random numbers?\")\ncode = AICode(msg)\n# Output: AICode(Success: True, Parsed: True, Evaluated: True, Error Caught: N/A, StdOut: True, Code: 2 Lines)\n\n# show the code\ncode.code |> println\n# Output: \n# numbers = rand(10)\n# numbers = rand(1:100, 10)\n\n# or copy it to the clipboard\ncode.code |> clipboard\n\n# or execute it in the current module (=Main)\neval(code.expression)\n\n\n\n\n\n","category":"type"},{"location":"reference/#PromptingTools.AIMessage","page":"PromptingTools.jl","title":"PromptingTools.AIMessage","text":"AIMessage\n\nA message type for AI-generated text-based responses. Returned by aigenerate, aiclassify, and aiscan functions.\n\nFields\n\ncontent::Union{AbstractString, Nothing}: The content of the message.\nstatus::Union{Int, Nothing}: The status of the message from the API.\ntokens::Tuple{Int, Int}: The number of tokens used (prompt,completion).\nelapsed::Float64: The time taken to generate the response in seconds.\ncost::Union{Nothing, Float64}: The cost of the API call (calculated with information from MODEL_REGISTRY).\nlog_prob::Union{Nothing, Float64}: The log probability of the response.\nfinish_reason::Union{Nothing, String}: The reason the response was finished.\nrun_id::Union{Nothing, Int}: The unique ID of the run.\nsample_id::Union{Nothing, Int}: The unique ID of the sample (if multiple samples are generated, they will all have the same run_id).\n\n\n\n\n\n","category":"type"},{"location":"reference/#PromptingTools.AITemplate","page":"PromptingTools.jl","title":"PromptingTools.AITemplate","text":"AITemplate\n\nAITemplate is a template for a conversation prompt. This type is merely a container for the template name, which is resolved into a set of messages (=prompt) by render.\n\nNaming Convention\n\nTemplate names should be in CamelCase\nFollow the format ...... where possible, eg, JudgeIsItTrue, ``\nStarting with the Persona (=System prompt), eg, Judge = persona is meant to judge some provided information\nVariable to be filled in with context, eg, It = placeholder it\nEnding with the variable name is helpful, eg, JuliaExpertTask for a persona to be an expert in Julia language and task is the placeholder name\nIdeally, the template name should be self-explanatory, eg, JudgeIsItTrue = persona is meant to judge some provided information where it is true or false\n\nExamples\n\nSave time by re-using pre-made templates, just fill in the placeholders with the keyword arguments:\n\nmsg = aigenerate(:JuliaExpertAsk; ask = \"How do I add packages?\")\n\nThe above is equivalent to a more verbose version that explicitly uses the dispatch on AITemplate:\n\nmsg = aigenerate(AITemplate(:JuliaExpertAsk); ask = \"How do I add packages?\")\n\nFind available templates with aitemplates:\n\ntmps = aitemplates(\"JuliaExpertAsk\")\n# Will surface one specific template\n# 1-element Vector{AITemplateMetadata}:\n# PromptingTools.AITemplateMetadata\n# name: Symbol JuliaExpertAsk\n# description: String \"For asking questions about Julia language. Placeholders: `ask`\"\n# version: String \"1\"\n# wordcount: Int64 237\n# variables: Array{Symbol}((1,))\n# system_preview: String \"You are a world-class Julia language programmer with the knowledge of the latest syntax. Your commun\"\n# user_preview: String \"# Question\n\n{{ask}}\"\n# source: String \"\"\n\nThe above gives you a good idea of what the template is about, what placeholders are available, and how much it would cost to use it (=wordcount).\n\nSearch for all Julia-related templates:\n\ntmps = aitemplates(\"Julia\")\n# 2-element Vector{AITemplateMetadata}... -> more to come later!\n\nIf you are on VSCode, you can leverage nice tabular display with vscodedisplay:\n\nusing DataFrames\ntmps = aitemplates(\"Julia\") |> DataFrame |> vscodedisplay\n\nI have my selected template, how do I use it? Just use the \"name\" in aigenerate or aiclassify like you see in the first example!\n\nYou can inspect any template by \"rendering\" it (this is what the LLM will see):\n\njulia> AITemplate(:JudgeIsItTrue) |> PromptingTools.render\n\nSee also: save_template, load_template, load_templates! for more advanced use cases (and the corresponding script in examples/ folder)\n\n\n\n\n\n","category":"type"},{"location":"reference/#PromptingTools.AITemplateMetadata","page":"PromptingTools.jl","title":"PromptingTools.AITemplateMetadata","text":"Helper for easy searching and reviewing of templates. Defined on loading of each template.\n\n\n\n\n\n","category":"type"},{"location":"reference/#PromptingTools.AbstractPromptSchema","page":"PromptingTools.jl","title":"PromptingTools.AbstractPromptSchema","text":"Defines different prompting styles based on the model training and fine-tuning.\n\n\n\n\n\n","category":"type"},{"location":"reference/#PromptingTools.ChatMLSchema","page":"PromptingTools.jl","title":"PromptingTools.ChatMLSchema","text":"ChatMLSchema is used by many open-source chatbots, by OpenAI models (under the hood) and by several models and inferfaces (eg, Ollama, vLLM)\n\nYou can explore it on tiktokenizer\n\nIt uses the following conversation structure:\n\nsystem\n...\n<|im_start|>user\n...<|im_end|>\n<|im_start|>assistant\n...<|im_end|>\n\n\n\n\n\n","category":"type"},{"location":"reference/#PromptingTools.CustomOpenAISchema","page":"PromptingTools.jl","title":"PromptingTools.CustomOpenAISchema","text":"CustomOpenAISchema\n\nCustomOpenAISchema() allows user to call any OpenAI-compatible API.\n\nAll user needs to do is to pass this schema as the first argument and provide the BASE URL of the API to call (api_kwargs.url).\n\nExample\n\nAssumes that we have a local server running at http://127.0.0.1:8081:\n\napi_key = \"...\"\nprompt = \"Say hi!\"\nmsg = aigenerate(CustomOpenAISchema(), prompt; model=\"my_model\", api_key, api_kwargs=(; url=\"http://127.0.0.1:8081\"))\n\n\n\n\n\n","category":"type"},{"location":"reference/#PromptingTools.DataMessage","page":"PromptingTools.jl","title":"PromptingTools.DataMessage","text":"DataMessage\n\nA message type for AI-generated data-based responses, ie, different content than text. Returned by aiextract, and aiextract functions.\n\nFields\n\ncontent::Union{AbstractString, Nothing}: The content of the message.\nstatus::Union{Int, Nothing}: The status of the message from the API.\ntokens::Tuple{Int, Int}: The number of tokens used (prompt,completion).\nelapsed::Float64: The time taken to generate the response in seconds.\ncost::Union{Nothing, Float64}: The cost of the API call (calculated with information from MODEL_REGISTRY).\nlog_prob::Union{Nothing, Float64}: The log probability of the response.\nfinish_reason::Union{Nothing, String}: The reason the response was finished.\nrun_id::Union{Nothing, Int}: The unique ID of the run.\nsample_id::Union{Nothing, Int}: The unique ID of the sample (if multiple samples are generated, they will all have the same run_id).\n\n\n\n\n\n","category":"type"},{"location":"reference/#PromptingTools.DatabricksOpenAISchema","page":"PromptingTools.jl","title":"PromptingTools.DatabricksOpenAISchema","text":"DatabricksOpenAISchema\n\nDatabricksOpenAISchema() allows user to call Databricks Foundation Model API. API Reference\n\nRequires two environment variables to be set:\n\nDATABRICKS_API_KEY: Databricks token\nDATABRICKS_HOST: Address of the Databricks workspace (https://.databricks.com)\n\n\n\n\n\n","category":"type"},{"location":"reference/#PromptingTools.FireworksOpenAISchema","page":"PromptingTools.jl","title":"PromptingTools.FireworksOpenAISchema","text":"FireworksOpenAISchema\n\nSchema to call the Fireworks.ai API.\n\nLinks:\n\nGet your API key\nAPI Reference\nAvailable models\n\nRequires one environment variables to be set:\n\nFIREWORKS_API_KEY: Your API key\n\n\n\n\n\n","category":"type"},{"location":"reference/#PromptingTools.GoogleSchema","page":"PromptingTools.jl","title":"PromptingTools.GoogleSchema","text":"Calls Google's Gemini API. See more information here. It's available only for some regions.\n\n\n\n\n\n","category":"type"},{"location":"reference/#PromptingTools.ItemsExtract","page":"PromptingTools.jl","title":"PromptingTools.ItemsExtract","text":"Extract zero, one or more specified items from the provided data.\n\n\n\n\n\n","category":"type"},{"location":"reference/#PromptingTools.LocalServerOpenAISchema","page":"PromptingTools.jl","title":"PromptingTools.LocalServerOpenAISchema","text":"LocalServerOpenAISchema\n\nDesigned to be used with local servers. It's automatically called with model alias \"local\" (see MODEL_REGISTRY).\n\nThis schema is a flavor of CustomOpenAISchema with a url keypreset by global Preference keyLOCAL_SERVER. See?PREFERENCESfor more details on how to change it. It assumes that the server follows OpenAI API conventions (eg,POST /v1/chat/completions`).\n\nNote: Llama.cpp (and hence Llama.jl built on top of it) do NOT support embeddings endpoint! You'll get an address error.\n\nExample\n\nAssumes that we have a local server running at http://127.0.0.1:10897/v1 (port and address used by Llama.jl, \"v1\" at the end is needed for OpenAI endpoint compatibility):\n\nThree ways to call it:\n\n\n# Use @ai_str with \"local\" alias\nai\"Say hi!\"local\n\n# model=\"local\"\naigenerate(\"Say hi!\"; model=\"local\")\n\n# Or set schema explicitly\nconst PT = PromptingTools\nmsg = aigenerate(PT.LocalServerOpenAISchema(), \"Say hi!\")\n\nHow to start a LLM local server? You can use run_server function from Llama.jl. Use a separate Julia session.\n\nusing Llama\nmodel = \"...path...\" # see Llama.jl README how to download one\nrun_server(; model)\n\nTo change the default port and address:\n\n# For a permanent change, set the preference:\nusing Preferences\nset_preferences!(\"LOCAL_SERVER\"=>\"http://127.0.0.1:10897/v1\")\n\n# Or if it's a temporary fix, just change the variable `LOCAL_SERVER`:\nconst PT = PromptingTools\nPT.LOCAL_SERVER = \"http://127.0.0.1:10897/v1\"\n\n\n\n\n\n","category":"type"},{"location":"reference/#PromptingTools.MaybeExtract","page":"PromptingTools.jl","title":"PromptingTools.MaybeExtract","text":"Extract a result from the provided data, if any, otherwise set the error and message fields.\n\nArguments\n\nerror::Bool: true if a result is found, false otherwise.\nmessage::String: Only present if no result is found, should be short and concise.\n\n\n\n\n\n","category":"type"},{"location":"reference/#PromptingTools.MistralOpenAISchema","page":"PromptingTools.jl","title":"PromptingTools.MistralOpenAISchema","text":"MistralOpenAISchema\n\nMistralOpenAISchema() allows user to call MistralAI API known for mistral and mixtral models.\n\nIt's a flavor of CustomOpenAISchema() with a url preset to https://api.mistral.ai.\n\nMost models have been registered, so you don't even have to specify the schema\n\nExample\n\nLet's call mistral-tiny model:\n\napi_key = \"...\" # can be set via ENV[\"MISTRAL_API_KEY\"] or via our preference system\nmsg = aigenerate(\"Say hi!\"; model=\"mistral_tiny\", api_key)\n\nSee ?PREFERENCES for more details on how to set your API key permanently.\n\n\n\n\n\n","category":"type"},{"location":"reference/#PromptingTools.ModelSpec","page":"PromptingTools.jl","title":"PromptingTools.ModelSpec","text":"ModelSpec\n\nA struct that contains information about a model, such as its name, schema, cost per token, etc.\n\nFields\n\nname::String: The name of the model. This is the name that will be used to refer to the model in the ai* functions.\nschema::AbstractPromptSchema: The schema of the model. This is the schema that will be used to generate prompts for the model, eg, :OpenAISchema.\ncost_of_token_prompt::Float64: The cost of 1 token in the prompt for this model. This is used to calculate the cost of a prompt. Note: It is often provided online as cost per 1000 tokens, so make sure to convert it correctly!\ncost_of_token_generation::Float64: The cost of 1 token generated by this model. This is used to calculate the cost of a generation. Note: It is often provided online as cost per 1000 tokens, so make sure to convert it correctly!\ndescription::String: A description of the model. This is used to provide more information about the model when it is queried.\n\nExample\n\nspec = ModelSpec(\"gpt-3.5-turbo\",\n OpenAISchema(),\n 0.0015,\n 0.002,\n \"GPT-3.5 Turbo is a 175B parameter model and a common default on the OpenAI API.\")\n\n# register it\nPromptingTools.register_model!(spec)\n\nBut you can also register any model directly via keyword arguments:\n\nPromptingTools.register_model!(\n name = \"gpt-3.5-turbo\",\n schema = OpenAISchema(),\n cost_of_token_prompt = 0.0015,\n cost_of_token_generation = 0.002,\n description = \"GPT-3.5 Turbo is a 175B parameter model and a common default on the OpenAI API.\")\n\n\n\n\n\n","category":"type"},{"location":"reference/#PromptingTools.NoSchema","page":"PromptingTools.jl","title":"PromptingTools.NoSchema","text":"Schema that keeps messages (<:AbstractMessage) and does not transform for any specific model. It used by the first pass of the prompt rendering system (see ?render).\n\n\n\n\n\n","category":"type"},{"location":"reference/#PromptingTools.OllamaManagedSchema","page":"PromptingTools.jl","title":"PromptingTools.OllamaManagedSchema","text":"Ollama by default manages different models and their associated prompt schemas when you pass system_prompt and prompt fields to the API.\n\nWarning: It works only for 1 system message and 1 user message, so anything more than that has to be rejected.\n\nIf you need to pass more messagese / longer conversational history, you can use define the model-specific schema directly and pass your Ollama requests with raw=true, which disables and templating and schema management by Ollama.\n\n\n\n\n\n","category":"type"},{"location":"reference/#PromptingTools.OllamaSchema","page":"PromptingTools.jl","title":"PromptingTools.OllamaSchema","text":"OllamaSchema is the default schema for Olama models.\n\nIt uses the following conversation template:\n\n[Dict(role=\"system\",content=\"...\"),Dict(role=\"user\",content=\"...\"),Dict(role=\"assistant\",content=\"...\")]\n\nIt's very similar to OpenAISchema, but it appends images differently.\n\n\n\n\n\n","category":"type"},{"location":"reference/#PromptingTools.OpenAISchema","page":"PromptingTools.jl","title":"PromptingTools.OpenAISchema","text":"OpenAISchema is the default schema for OpenAI models.\n\nIt uses the following conversation template:\n\n[Dict(role=\"system\",content=\"...\"),Dict(role=\"user\",content=\"...\"),Dict(role=\"assistant\",content=\"...\")]\n\nIt's recommended to separate sections in your prompt with markdown headers (e.g. `##Answer\n\n`).\n\n\n\n\n\n","category":"type"},{"location":"reference/#PromptingTools.TestEchoGoogleSchema","page":"PromptingTools.jl","title":"PromptingTools.TestEchoGoogleSchema","text":"Echoes the user's input back to them. Used for testing the implementation\n\n\n\n\n\n","category":"type"},{"location":"reference/#PromptingTools.TestEchoOllamaManagedSchema","page":"PromptingTools.jl","title":"PromptingTools.TestEchoOllamaManagedSchema","text":"Echoes the user's input back to them. Used for testing the implementation\n\n\n\n\n\n","category":"type"},{"location":"reference/#PromptingTools.TestEchoOllamaSchema","page":"PromptingTools.jl","title":"PromptingTools.TestEchoOllamaSchema","text":"Echoes the user's input back to them. Used for testing the implementation\n\n\n\n\n\n","category":"type"},{"location":"reference/#PromptingTools.TestEchoOpenAISchema","page":"PromptingTools.jl","title":"PromptingTools.TestEchoOpenAISchema","text":"Echoes the user's input back to them. Used for testing the implementation\n\n\n\n\n\n","category":"type"},{"location":"reference/#PromptingTools.TogetherOpenAISchema","page":"PromptingTools.jl","title":"PromptingTools.TogetherOpenAISchema","text":"TogetherOpenAISchema\n\nSchema to call the Together.ai API.\n\nLinks:\n\nGet your API key\nAPI Reference\nAvailable models\n\nRequires one environment variables to be set:\n\nTOGETHER_API_KEY: Your API key\n\n\n\n\n\n","category":"type"},{"location":"reference/#PromptingTools.UserMessageWithImages-Tuple{AbstractString}","page":"PromptingTools.jl","title":"PromptingTools.UserMessageWithImages","text":"Construct UserMessageWithImages with 1 or more images. Images can be either URLs or local paths.\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.X123","page":"PromptingTools.jl","title":"PromptingTools.X123","text":"With docstring\n\n\n\n\n\n","category":"type"},{"location":"reference/#OpenAI.create_chat-Tuple{PromptingTools.CustomOpenAISchema, AbstractString, AbstractString, Any}","page":"PromptingTools.jl","title":"OpenAI.create_chat","text":"OpenAI.create_chat(schema::CustomOpenAISchema,\n\napi_key::AbstractString, model::AbstractString, conversation; url::String=\"http://localhost:8080\", kwargs...)\n\nDispatch to the OpenAI.create_chat function, for any OpenAI-compatible API. \n\nIt expects url keyword argument. Provide it to the aigenerate function via api_kwargs=(; url=\"my-url\")\n\nIt will forward your query to the \"chat/completions\" endpoint of the base URL that you provided (=url).\n\n\n\n\n\n","category":"method"},{"location":"reference/#OpenAI.create_chat-Tuple{PromptingTools.LocalServerOpenAISchema, AbstractString, AbstractString, Any}","page":"PromptingTools.jl","title":"OpenAI.create_chat","text":"OpenAI.create_chat(schema::LocalServerOpenAISchema,\n api_key::AbstractString,\n model::AbstractString,\n conversation;\n url::String = \"http://localhost:8080\",\n kwargs...)\n\nDispatch to the OpenAI.createchat function, but with the LocalServer API parameters, ie, defaults to url specified by the `LOCALSERVERpreference. See?PREFERENCES`\n\n\n\n\n\n","category":"method"},{"location":"reference/#OpenAI.create_chat-Tuple{PromptingTools.MistralOpenAISchema, AbstractString, AbstractString, Any}","page":"PromptingTools.jl","title":"OpenAI.create_chat","text":"OpenAI.create_chat(schema::MistralOpenAISchema,\n\napi_key::AbstractString, model::AbstractString, conversation; url::String=\"https://api.mistral.ai/v1\", kwargs...)\n\nDispatch to the OpenAI.create_chat function, but with the MistralAI API parameters. \n\nIt tries to access the MISTRALAI_API_KEY ENV variable, but you can also provide it via the api_key keyword argument.\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.aiclassify-Union{Tuple{T}, Tuple{PromptingTools.AbstractOpenAISchema, Union{AbstractString, PromptingTools.AbstractMessage, Vector{<:PromptingTools.AbstractMessage}}}} where T<:Union{AbstractString, Tuple{var\"#s110\", var\"#s104\"} where {var\"#s110\"<:AbstractString, var\"#s104\"<:AbstractString}}","page":"PromptingTools.jl","title":"PromptingTools.aiclassify","text":"aiclassify(prompt_schema::AbstractOpenAISchema, prompt::ALLOWED_PROMPT_TYPE;\n choices::AbstractVector{T} = [\"true\", \"false\", \"unknown\"],\n api_kwargs::NamedTuple = NamedTuple(),\n kwargs...) where {T <: Union{AbstractString, Tuple{<:AbstractString, <:AbstractString}}}\n\nClassifies the given prompt/statement into an arbitrary list of choices, which must be only the choices (vector of strings) or choices and descriptions are provided (vector of tuples, ie, (\"choice\",\"description\")).\n\nIt's quick and easy option for \"routing\" and similar use cases, as it exploits the logit bias trick and outputs only 1 token. classify into an arbitrary list of categories (including with descriptions). It's quick and easy option for \"routing\" and similar use cases, as it exploits the logit bias trick, so it outputs only 1 token.\n\n!!! Note: The prompt/AITemplate must have a placeholder choices (ie, {{choices}}) that will be replaced with the encoded choices\n\nChoices are rewritten into an enumerated list and mapped to a few known OpenAI tokens (maximum of 20 choices supported). Mapping of token IDs for GPT3.5/4 are saved in variable OPENAI_TOKEN_IDS.\n\nIt uses Logit bias trick and limits the output to 1 token to force the model to output only true/false/unknown. Credit for the idea goes to AAAzzam.\n\nArguments\n\nprompt_schema::AbstractOpenAISchema: The schema for the prompt.\nprompt: The prompt/statement to classify if it's a String. If it's a Symbol, it is expanded as a template via render(schema,template). Eg, templates :JudgeIsItTrue or :InputClassifier\nchoices::AbstractVector{T}: The choices to be classified into. It can be a vector of strings or a vector of tuples, where the first element is the choice and the second is the description.\n\nExample\n\nGiven a user input, pick one of the two provided categories:\n\nchoices = [\"animal\", \"plant\"]\ninput = \"Palm tree\"\naiclassify(:InputClassifier; choices, input)\n\nChoices with descriptions provided as tuples:\n\nchoices = [(\"A\", \"any animal or creature\"), (\"P\", \"for any plant or tree\"), (\"O\", \"for everything else\")]\n\n# try the below inputs:\ninput = \"spider\" # -> returns \"A\" for any animal or creature\ninput = \"daphodil\" # -> returns \"P\" for any plant or tree\ninput = \"castle\" # -> returns \"O\" for everything else\naiclassify(:InputClassifier; choices, input)\n\nYou can still use a simple true/false classification:\n\naiclassify(\"Is two plus two four?\") # true\naiclassify(\"Is two plus three a vegetable on Mars?\") # false\n\naiclassify returns only true/false/unknown. It's easy to get the proper Bool output type out with tryparse, eg,\n\ntryparse(Bool, aiclassify(\"Is two plus two four?\")) isa Bool # true\n\nOutput of type Nothing marks that the model couldn't classify the statement as true/false.\n\nIdeally, we would like to re-use some helpful system prompt to get more accurate responses. For this reason we have templates, eg, :JudgeIsItTrue. By specifying the template, we can provide our statement as the expected variable (it in this case) See that the model now correctly classifies the statement as \"unknown\".\n\naiclassify(:JudgeIsItTrue; it = \"Is two plus three a vegetable on Mars?\") # unknown\n\nFor better results, use higher quality models like gpt4, eg, \n\naiclassify(:JudgeIsItTrue;\n it = \"If I had two apples and I got three more, I have five apples now.\",\n model = \"gpt4\") # true\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.aiembed-Union{Tuple{F}, Tuple{PromptingTools.AbstractOllamaManagedSchema, AbstractString}, Tuple{PromptingTools.AbstractOllamaManagedSchema, AbstractString, F}} where F<:Function","page":"PromptingTools.jl","title":"PromptingTools.aiembed","text":"aiembed(prompt_schema::AbstractOllamaManagedSchema,\n doc_or_docs::Union{AbstractString, AbstractVector{<:AbstractString}},\n postprocess::F = identity;\n verbose::Bool = true,\n api_key::String = \"\",\n model::String = MODEL_EMBEDDING,\n http_kwargs::NamedTuple = (retry_non_idempotent = true,\n retries = 5,\n readtimeout = 120),\n api_kwargs::NamedTuple = NamedTuple(),\n kwargs...) where {F <: Function}\n\nThe aiembed function generates embeddings for the given input using a specified model and returns a message object containing the embeddings, status, token count, and elapsed time.\n\nArguments\n\nprompt_schema::AbstractOllamaManagedSchema: The schema for the prompt.\ndoc_or_docs::Union{AbstractString, AbstractVector{<:AbstractString}}: The document or list of documents to generate embeddings for. The list of documents is processed sequentially, so users should consider implementing an async version with with Threads.@spawn\npostprocess::F: The post-processing function to apply to each embedding. Defaults to the identity function, but could be LinearAlgebra.normalize.\nverbose::Bool: A flag indicating whether to print verbose information. Defaults to true.\napi_key::String: The API key to use for the OpenAI API. Defaults to \"\".\nmodel::String: The model to use for generating embeddings. Defaults to MODEL_EMBEDDING.\nhttp_kwargs::NamedTuple: Additional keyword arguments for the HTTP request. Defaults to empty NamedTuple.\napi_kwargs::NamedTuple: Additional keyword arguments for the Ollama API. Defaults to an empty NamedTuple.\nkwargs: Prompt variables to be used to fill the prompt/template\n\nReturns\n\nmsg: A DataMessage object containing the embeddings, status, token count, and elapsed time.\n\nNote: Ollama API currently does not return the token count, so it's set to (0,0)\n\nExample\n\nconst PT = PromptingTools\nschema = PT.OllamaManagedSchema()\n\nmsg = aiembed(schema, \"Hello World\"; model=\"openhermes2.5-mistral\")\nmsg.content # 4096-element JSON3.Array{Float64...\n\nWe can embed multiple strings at once and they will be hcat into a matrix (ie, each column corresponds to one string)\n\nconst PT = PromptingTools\nschema = PT.OllamaManagedSchema()\n\nmsg = aiembed(schema, [\"Hello World\", \"How are you?\"]; model=\"openhermes2.5-mistral\")\nmsg.content # 4096×2 Matrix{Float64}:\n\nIf you plan to calculate the cosine distance between embeddings, you can normalize them first:\n\nconst PT = PromptingTools\nusing LinearAlgebra\nschema = PT.OllamaManagedSchema()\n\nmsg = aiembed(schema, [\"embed me\", \"and me too\"], LinearAlgebra.normalize; model=\"openhermes2.5-mistral\")\n\n# calculate cosine distance between the two normalized embeddings as a simple dot product\nmsg.content' * msg.content[:, 1] # [1.0, 0.34]\n\nSimilarly, you can use the postprocess argument to materialize the data from JSON3.Object by using postprocess = copy\n\nconst PT = PromptingTools\nschema = PT.OllamaManagedSchema()\n\nmsg = aiembed(schema, \"Hello World\", copy; model=\"openhermes2.5-mistral\")\nmsg.content # 4096-element Vector{Float64}\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.aiembed-Union{Tuple{F}, Tuple{PromptingTools.AbstractOpenAISchema, Union{AbstractString, AbstractVector{<:AbstractString}}}, Tuple{PromptingTools.AbstractOpenAISchema, Union{AbstractString, AbstractVector{<:AbstractString}}, F}} where F<:Function","page":"PromptingTools.jl","title":"PromptingTools.aiembed","text":"aiembed(prompt_schema::AbstractOpenAISchema,\n doc_or_docs::Union{AbstractString, AbstractVector{<:AbstractString}},\n postprocess::F = identity;\n verbose::Bool = true,\n api_key::String = OPENAI_API_KEY,\n model::String = MODEL_EMBEDDING, \n http_kwargs::NamedTuple = (retry_non_idempotent = true,\n retries = 5,\n readtimeout = 120),\n api_kwargs::NamedTuple = NamedTuple(),\n kwargs...) where {F <: Function}\n\nThe aiembed function generates embeddings for the given input using a specified model and returns a message object containing the embeddings, status, token count, and elapsed time.\n\nArguments\n\nprompt_schema::AbstractOpenAISchema: The schema for the prompt.\ndoc_or_docs::Union{AbstractString, AbstractVector{<:AbstractString}}: The document or list of documents to generate embeddings for.\npostprocess::F: The post-processing function to apply to each embedding. Defaults to the identity function.\nverbose::Bool: A flag indicating whether to print verbose information. Defaults to true.\napi_key::String: The API key to use for the OpenAI API. Defaults to OPENAI_API_KEY.\nmodel::String: The model to use for generating embeddings. Defaults to MODEL_EMBEDDING.\nhttp_kwargs::NamedTuple: Additional keyword arguments for the HTTP request. Defaults to (retry_non_idempotent = true, retries = 5, readtimeout = 120).\napi_kwargs::NamedTuple: Additional keyword arguments for the OpenAI API. Defaults to an empty NamedTuple.\nkwargs...: Additional keyword arguments.\n\nReturns\n\nmsg: A DataMessage object containing the embeddings, status, token count, and elapsed time. Use msg.content to access the embeddings.\n\nExample\n\nmsg = aiembed(\"Hello World\")\nmsg.content # 1536-element JSON3.Array{Float64...\n\nWe can embed multiple strings at once and they will be hcat into a matrix (ie, each column corresponds to one string)\n\nmsg = aiembed([\"Hello World\", \"How are you?\"])\nmsg.content # 1536×2 Matrix{Float64}:\n\nIf you plan to calculate the cosine distance between embeddings, you can normalize them first:\n\nusing LinearAlgebra\nmsg = aiembed([\"embed me\", \"and me too\"], LinearAlgebra.normalize)\n\n# calculate cosine distance between the two normalized embeddings as a simple dot product\nmsg.content' * msg.content[:, 1] # [1.0, 0.787]\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.aiextract-Tuple{PromptingTools.AbstractOpenAISchema, Union{AbstractString, PromptingTools.AbstractMessage, Vector{<:PromptingTools.AbstractMessage}}}","page":"PromptingTools.jl","title":"PromptingTools.aiextract","text":"aiextract(prompt_schema::AbstractOpenAISchema, prompt::ALLOWED_PROMPT_TYPE;\n return_type::Type,\n verbose::Bool = true,\n api_key::String = OPENAI_API_KEY,\n model::String = MODEL_CHAT,\n return_all::Bool = false, dry_run::Bool = false,\n conversation::AbstractVector{<:AbstractMessage} = AbstractMessage[],\n http_kwargs::NamedTuple = (retry_non_idempotent = true,\n retries = 5,\n readtimeout = 120), api_kwargs::NamedTuple = (;\n tool_choice = \"exact\"),\n kwargs...)\n\nExtract required information (defined by a struct return_type) from the provided prompt by leveraging OpenAI function calling mode.\n\nThis is a perfect solution for extracting structured information from text (eg, extract organization names in news articles, etc.)\n\nIt's effectively a light wrapper around aigenerate call, which requires additional keyword argument return_type to be provided and will enforce the model outputs to adhere to it.\n\nArguments\n\nprompt_schema: An optional object to specify which prompt template should be applied (Default to PROMPT_SCHEMA = OpenAISchema)\nprompt: Can be a string representing the prompt for the AI conversation, a UserMessage, a vector of AbstractMessage or an AITemplate\nreturn_type: A struct TYPE representing the the information we want to extract. Do not provide a struct instance, only the type. If the struct has a docstring, it will be provided to the model as well. It's used to enforce structured model outputs or provide more information.\nverbose: A boolean indicating whether to print additional information.\napi_key: A string representing the API key for accessing the OpenAI API.\nmodel: A string representing the model to use for generating the response. Can be an alias corresponding to a model ID defined in MODEL_ALIASES.\nreturn_all::Bool=false: If true, returns the entire conversation history, otherwise returns only the last message (the AIMessage).\ndry_run::Bool=false: If true, skips sending the messages to the model (for debugging, often used with return_all=true).\nconversation: An optional vector of AbstractMessage objects representing the conversation history. If not provided, it is initialized as an empty vector.\nhttp_kwargs: A named tuple of HTTP keyword arguments.\napi_kwargs: A named tuple of API keyword arguments. \ntool_choice: A string representing the tool choice to use for the API call. Usually, one of \"auto\",\"any\",\"exact\". Defaults to \"exact\", which is a made-up value to enforce the OpenAI requirements if we want one exact function. Providers like Mistral, Together, etc. use \"any\" instead.\nkwargs: Prompt variables to be used to fill the prompt/template\n\nReturns\n\nIf return_all=false (default):\n\nmsg: An DataMessage object representing the extracted data, including the content, status, tokens, and elapsed time. Use msg.content to access the extracted data.\n\nIf return_all=true:\n\nconversation: A vector of AbstractMessage objects representing the full conversation history, including the response from the AI model (DataMessage).\n\nSee also: function_call_signature, MaybeExtract, ItemsExtract, aigenerate\n\nExample\n\nDo you want to extract some specific measurements from a text like age, weight and height? You need to define the information you need as a struct (return_type):\n\n\"Person's age, height, and weight.\"\nstruct MyMeasurement\n age::Int # required\n height::Union{Int,Nothing} # optional\n weight::Union{Nothing,Float64} # optional\nend\nmsg = aiextract(\"James is 30, weighs 80kg. He's 180cm tall.\"; return_type=MyMeasurement)\n# PromptingTools.DataMessage(MyMeasurement)\nmsg.content\n# MyMeasurement(30, 180, 80.0)\n\nThe fields that allow Nothing are marked as optional in the schema:\n\nmsg = aiextract(\"James is 30.\"; return_type=MyMeasurement)\n# MyMeasurement(30, nothing, nothing)\n\nIf there are multiple items you want to extract, define a wrapper struct to get a Vector of MyMeasurement:\n\nstruct MyMeasurementWrapper\n measurements::Vector{MyMeasurement}\nend\n\nmsg = aiextract(\"James is 30, weighs 80kg. He's 180cm tall. Then Jack is 19 but really tall - over 190!\"; return_type=ManyMeasurements)\n\nmsg.content.measurements\n# 2-element Vector{MyMeasurement}:\n# MyMeasurement(30, 180, 80.0)\n# MyMeasurement(19, 190, nothing)\n\nOr you can use the convenience wrapper ItemsExtract to extract multiple measurements (zero, one or more):\n\nusing PromptingTools: ItemsExtract\n\nreturn_type = ItemsExtract{MyMeasurement}\nmsg = aiextract(\"James is 30, weighs 80kg. He's 180cm tall. Then Jack is 19 but really tall - over 190!\"; return_type)\n\nmsg.content.items # see the extracted items\n\nOr if you want your extraction to fail gracefully when data isn't found, use MaybeExtract{T} wrapper (this trick is inspired by the Instructor package!):\n\nusing PromptingTools: MaybeExtract\n\ntype = MaybeExtract{MyMeasurement}\n# Effectively the same as:\n# struct MaybeExtract{T}\n# result::Union{T, Nothing} // The result of the extraction\n# error::Bool // true if a result is found, false otherwise\n# message::Union{Nothing, String} // Only present if no result is found, should be short and concise\n# end\n\n# If LLM extraction fails, it will return a Dict with `error` and `message` fields instead of the result!\nmsg = aiextract(\"Extract measurements from the text: I am giraffe\", type)\nmsg.content\n# MaybeExtract{MyMeasurement}(nothing, true, \"I'm sorry, but I can only assist with human measurements.\")\n\nThat way, you can handle the error gracefully and get a reason why extraction failed (in msg.content.message).\n\nNote that the error message refers to a giraffe not being a human, because in our MyMeasurement docstring, we said that it's for people!\n\nSome non-OpenAI providers require a different specification of the \"tool choice\" than OpenAI. For example, to use Mistral models (\"mistrall\" for mistral large), do:\n\n\"Some fruit\"\nstruct Fruit\n name::String\nend\naiextract(\"I ate an apple\",return_type=Fruit,api_kwargs=(;tool_choice=\"any\"),model=\"mistrall\")\n# Notice two differences: 1) struct MUST have a docstring, 2) tool_choice is set explicitly set to \"any\"\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.aigenerate-Tuple{PromptingTools.AbstractGoogleSchema, Union{AbstractString, PromptingTools.AbstractMessage, Vector{<:PromptingTools.AbstractMessage}}}","page":"PromptingTools.jl","title":"PromptingTools.aigenerate","text":"aigenerate(prompt_schema::AbstractGoogleSchema, prompt::ALLOWED_PROMPT_TYPE;\n verbose::Bool = true,\n api_key::String = GOOGLE_API_KEY,\n model::String = \"gemini-pro\", return_all::Bool = false, dry_run::Bool = false,\n http_kwargs::NamedTuple = (retry_non_idempotent = true,\n retries = 5,\n readtimeout = 120), api_kwargs::NamedTuple = NamedTuple(),\n kwargs...)\n\nGenerate an AI response based on a given prompt using the Google Gemini API. Get the API key here.\n\nNote: \n\nThere is no \"cost\" reported as of February 2024, as all access seems to be free-of-charge. See the details here.\ntokens in the returned AIMessage are actually characters, not tokens. We use a conservative estimate as they are not provided by the API yet.\n\nArguments\n\nprompt_schema: An optional object to specify which prompt template should be applied (Default to PROMPT_SCHEMA = OpenAISchema)\nprompt: Can be a string representing the prompt for the AI conversation, a UserMessage, a vector of AbstractMessage or an AITemplate\nverbose: A boolean indicating whether to print additional information.\napi_key: A string representing the API key for accessing the OpenAI API.\nmodel: A string representing the model to use for generating the response. Can be an alias corresponding to a model ID defined in MODEL_ALIASES. Defaults to \nreturn_all::Bool=false: If true, returns the entire conversation history, otherwise returns only the last message (the AIMessage).\ndry_run::Bool=false: If true, skips sending the messages to the model (for debugging, often used with return_all=true).\nconversation: An optional vector of AbstractMessage objects representing the conversation history. If not provided, it is initialized as an empty vector.\nhttp_kwargs: A named tuple of HTTP keyword arguments.\napi_kwargs: A named tuple of API keyword arguments.\nkwargs: Prompt variables to be used to fill the prompt/template\n\nReturns\n\nIf return_all=false (default):\n\nmsg: An AIMessage object representing the generated AI message, including the content, status, tokens, and elapsed time.\n\nUse msg.content to access the extracted string.\n\nIf return_all=true:\n\nconversation: A vector of AbstractMessage objects representing the conversation history, including the response from the AI model (AIMessage).\n\nSee also: ai_str, aai_str, aiembed, aiclassify, aiextract, aiscan, aitemplates\n\nExample\n\nSimple hello world to test the API:\n\nresult = aigenerate(\"Say Hi!\"; model=\"gemini-pro\")\n# AIMessage(\"Hi there! 👋 I'm here to help you with any questions or tasks you may have. Just let me know what you need, and I'll do my best to assist you.\")\n\nresult is an AIMessage object. Access the generated string via content property:\n\ntypeof(result) # AIMessage{SubString{String}}\npropertynames(result) # (:content, :status, :tokens, :elapsed\nresult.content # \"Hi there! ...\n\n___ You can use string interpolation and alias \"gemini\":\n\na = 1\nmsg=aigenerate(\"What is `$a+$a`?\"; model=\"gemini\")\nmsg.content # \"1+1 is 2.\"\n\n___ You can provide the whole conversation or more intricate prompts as a Vector{AbstractMessage}:\n\nconst PT = PromptingTools\n\nconversation = [\n PT.SystemMessage(\"You're master Yoda from Star Wars trying to help the user become a Yedi.\"),\n PT.UserMessage(\"I have feelings for my iPhone. What should I do?\")]\nmsg=aigenerate(conversation; model=\"gemini\")\n# AIMessage(\"Young Padawan, you have stumbled into a dangerous path.... \")\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.aigenerate-Tuple{PromptingTools.AbstractOllamaManagedSchema, Union{AbstractString, PromptingTools.AbstractMessage, Vector{<:PromptingTools.AbstractMessage}}}","page":"PromptingTools.jl","title":"PromptingTools.aigenerate","text":"aigenerate(prompt_schema::AbstractOllamaManagedSchema, prompt::ALLOWED_PROMPT_TYPE; verbose::Bool = true,\n api_key::String = \"\", model::String = MODEL_CHAT,\n return_all::Bool = false, dry_run::Bool = false,\n conversation::AbstractVector{<:AbstractMessage} = AbstractMessage[],\n http_kwargs::NamedTuple = NamedTuple(), api_kwargs::NamedTuple = NamedTuple(),\n kwargs...)\n\nGenerate an AI response based on a given prompt using the OpenAI API.\n\nArguments\n\nprompt_schema: An optional object to specify which prompt template should be applied (Default to PROMPT_SCHEMA = OpenAISchema not AbstractManagedSchema)\nprompt: Can be a string representing the prompt for the AI conversation, a UserMessage, a vector of AbstractMessage or an AITemplate\nverbose: A boolean indicating whether to print additional information.\napi_key: Provided for interface consistency. Not needed for locally hosted Ollama.\nmodel: A string representing the model to use for generating the response. Can be an alias corresponding to a model ID defined in MODEL_ALIASES.\nreturn_all::Bool=false: If true, returns the entire conversation history, otherwise returns only the last message (the AIMessage).\ndry_run::Bool=false: If true, skips sending the messages to the model (for debugging, often used with return_all=true).\nconversation::AbstractVector{<:AbstractMessage}=[]: Not allowed for this schema. Provided only for compatibility.\nhttp_kwargs::NamedTuple: Additional keyword arguments for the HTTP request. Defaults to empty NamedTuple.\napi_kwargs::NamedTuple: Additional keyword arguments for the Ollama API. Defaults to an empty NamedTuple.\nkwargs: Prompt variables to be used to fill the prompt/template\n\nReturns\n\nmsg: An AIMessage object representing the generated AI message, including the content, status, tokens, and elapsed time.\n\nUse msg.content to access the extracted string.\n\nSee also: ai_str, aai_str, aiembed\n\nExample\n\nSimple hello world to test the API:\n\nconst PT = PromptingTools\nschema = PT.OllamaManagedSchema() # We need to explicit if we want Ollama, OpenAISchema is the default\n\nmsg = aigenerate(schema, \"Say hi!\"; model=\"openhermes2.5-mistral\")\n# [ Info: Tokens: 69 in 0.9 seconds\n# AIMessage(\"Hello! How can I assist you today?\")\n\nmsg is an AIMessage object. Access the generated string via content property:\n\ntypeof(msg) # AIMessage{SubString{String}}\npropertynames(msg) # (:content, :status, :tokens, :elapsed\nmsg.content # \"Hello! How can I assist you today?\"\n\nNote: We need to be explicit about the schema we want to use. If we don't, it will default to OpenAISchema (=PT.DEFAULT_SCHEMA) ___ You can use string interpolation:\n\nconst PT = PromptingTools\nschema = PT.OllamaManagedSchema()\na = 1\nmsg=aigenerate(schema, \"What is `$a+$a`?\"; model=\"openhermes2.5-mistral\")\nmsg.content # \"The result of `1+1` is `2`.\"\n\n___ You can provide the whole conversation or more intricate prompts as a Vector{AbstractMessage}:\n\nconst PT = PromptingTools\nschema = PT.OllamaManagedSchema()\n\nconversation = [\n PT.SystemMessage(\"You're master Yoda from Star Wars trying to help the user become a Yedi.\"),\n PT.UserMessage(\"I have feelings for my iPhone. What should I do?\")]\n\nmsg = aigenerate(schema, conversation; model=\"openhermes2.5-mistral\")\n# [ Info: Tokens: 111 in 2.1 seconds\n# AIMessage(\"Strong the attachment is, it leads to suffering it may. Focus on the force within you must, ...\")\n\nNote: Managed Ollama currently supports at most 1 User Message and 1 System Message given the API limitations. If you want more, you need to use the ChatMLSchema.\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.aigenerate-Tuple{PromptingTools.AbstractOllamaSchema, Union{AbstractString, PromptingTools.AbstractMessage, Vector{<:PromptingTools.AbstractMessage}}}","page":"PromptingTools.jl","title":"PromptingTools.aigenerate","text":"aigenerate(prompt_schema::AbstractOllamaManagedSchema, prompt::ALLOWED_PROMPT_TYPE; verbose::Bool = true,\n api_key::String = \"\", model::String = MODEL_CHAT,\n return_all::Bool = false, dry_run::Bool = false,\n conversation::AbstractVector{<:AbstractMessage} = AbstractMessage[],\n http_kwargs::NamedTuple = NamedTuple(), api_kwargs::NamedTuple = NamedTuple(),\n kwargs...)\n\nGenerate an AI response based on a given prompt using the OpenAI API.\n\nArguments\n\nprompt_schema: An optional object to specify which prompt template should be applied (Default to PROMPT_SCHEMA = OpenAISchema not AbstractManagedSchema)\nprompt: Can be a string representing the prompt for the AI conversation, a UserMessage, a vector of AbstractMessage or an AITemplate\nverbose: A boolean indicating whether to print additional information.\napi_key: Provided for interface consistency. Not needed for locally hosted Ollama.\nmodel: A string representing the model to use for generating the response. Can be an alias corresponding to a model ID defined in MODEL_ALIASES.\nreturn_all::Bool=false: If true, returns the entire conversation history, otherwise returns only the last message (the AIMessage).\ndry_run::Bool=false: If true, skips sending the messages to the model (for debugging, often used with return_all=true).\nconversation::AbstractVector{<:AbstractMessage}=[]: Not allowed for this schema. Provided only for compatibility.\nhttp_kwargs::NamedTuple: Additional keyword arguments for the HTTP request. Defaults to empty NamedTuple.\napi_kwargs::NamedTuple: Additional keyword arguments for the Ollama API. Defaults to an empty NamedTuple.\nkwargs: Prompt variables to be used to fill the prompt/template\n\nReturns\n\nmsg: An AIMessage object representing the generated AI message, including the content, status, tokens, and elapsed time.\n\nUse msg.content to access the extracted string.\n\nSee also: ai_str, aai_str, aiembed\n\nExample\n\nSimple hello world to test the API:\n\nconst PT = PromptingTools\nschema = PT.OllamaManagedSchema() # We need to explicit if we want Ollama, OpenAISchema is the default\n\nmsg = aigenerate(schema, \"Say hi!\"; model=\"openhermes2.5-mistral\")\n# [ Info: Tokens: 69 in 0.9 seconds\n# AIMessage(\"Hello! How can I assist you today?\")\n\nmsg is an AIMessage object. Access the generated string via content property:\n\ntypeof(msg) # AIMessage{SubString{String}}\npropertynames(msg) # (:content, :status, :tokens, :elapsed\nmsg.content # \"Hello! How can I assist you today?\"\n\nNote: We need to be explicit about the schema we want to use. If we don't, it will default to OpenAISchema (=PT.DEFAULT_SCHEMA) ___ You can use string interpolation:\n\nconst PT = PromptingTools\nschema = PT.OllamaManagedSchema()\na = 1\nmsg=aigenerate(schema, \"What is `$a+$a`?\"; model=\"openhermes2.5-mistral\")\nmsg.content # \"The result of `1+1` is `2`.\"\n\n___ You can provide the whole conversation or more intricate prompts as a Vector{AbstractMessage}:\n\nconst PT = PromptingTools\nschema = PT.OllamaManagedSchema()\n\nconversation = [\n PT.SystemMessage(\"You're master Yoda from Star Wars trying to help the user become a Yedi.\"),\n PT.UserMessage(\"I have feelings for my iPhone. What should I do?\")]\n\nmsg = aigenerate(schema, conversation; model=\"openhermes2.5-mistral\")\n# [ Info: Tokens: 111 in 2.1 seconds\n# AIMessage(\"Strong the attachment is, it leads to suffering it may. Focus on the force within you must, ...\")\n\nNote: Managed Ollama currently supports at most 1 User Message and 1 System Message given the API limitations. If you want more, you need to use the ChatMLSchema.\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.aigenerate-Tuple{PromptingTools.AbstractOpenAISchema, Union{AbstractString, PromptingTools.AbstractMessage, Vector{<:PromptingTools.AbstractMessage}}}","page":"PromptingTools.jl","title":"PromptingTools.aigenerate","text":"aigenerate(prompt_schema::AbstractOpenAISchema, prompt::ALLOWED_PROMPT_TYPE;\n verbose::Bool = true,\n api_key::String = OPENAI_API_KEY,\n model::String = MODEL_CHAT, return_all::Bool = false, dry_run::Bool = false,\n http_kwargs::NamedTuple = (retry_non_idempotent = true,\n retries = 5,\n readtimeout = 120), api_kwargs::NamedTuple = NamedTuple(),\n kwargs...)\n\nGenerate an AI response based on a given prompt using the OpenAI API.\n\nArguments\n\nprompt_schema: An optional object to specify which prompt template should be applied (Default to PROMPT_SCHEMA = OpenAISchema)\nprompt: Can be a string representing the prompt for the AI conversation, a UserMessage, a vector of AbstractMessage or an AITemplate\nverbose: A boolean indicating whether to print additional information.\napi_key: A string representing the API key for accessing the OpenAI API.\nmodel: A string representing the model to use for generating the response. Can be an alias corresponding to a model ID defined in MODEL_ALIASES.\nreturn_all::Bool=false: If true, returns the entire conversation history, otherwise returns only the last message (the AIMessage).\ndry_run::Bool=false: If true, skips sending the messages to the model (for debugging, often used with return_all=true).\nconversation: An optional vector of AbstractMessage objects representing the conversation history. If not provided, it is initialized as an empty vector.\nhttp_kwargs: A named tuple of HTTP keyword arguments.\napi_kwargs: A named tuple of API keyword arguments. Useful parameters include:\ntemperature: A float representing the temperature for sampling (ie, the amount of \"creativity\"). Often defaults to 0.7.\nlogprobs: A boolean indicating whether to return log probabilities for each token. Defaults to false.\nn: An integer representing the number of completions to generate at once (if supported).\nstop: A vector of strings representing the stop conditions for the conversation. Defaults to an empty vector.\nkwargs: Prompt variables to be used to fill the prompt/template\n\nReturns\n\nIf return_all=false (default):\n\nmsg: An AIMessage object representing the generated AI message, including the content, status, tokens, and elapsed time.\n\nUse msg.content to access the extracted string.\n\nIf return_all=true:\n\nconversation: A vector of AbstractMessage objects representing the conversation history, including the response from the AI model (AIMessage).\n\nSee also: ai_str, aai_str, aiembed, aiclassify, aiextract, aiscan, aitemplates\n\nExample\n\nSimple hello world to test the API:\n\nresult = aigenerate(\"Say Hi!\")\n# [ Info: Tokens: 29 @ Cost: $0.0 in 1.0 seconds\n# AIMessage(\"Hello! How can I assist you today?\")\n\nresult is an AIMessage object. Access the generated string via content property:\n\ntypeof(result) # AIMessage{SubString{String}}\npropertynames(result) # (:content, :status, :tokens, :elapsed\nresult.content # \"Hello! How can I assist you today?\"\n\n___ You can use string interpolation:\n\na = 1\nmsg=aigenerate(\"What is `$a+$a`?\")\nmsg.content # \"The sum of `1+1` is `2`.\"\n\n___ You can provide the whole conversation or more intricate prompts as a Vector{AbstractMessage}:\n\nconst PT = PromptingTools\n\nconversation = [\n PT.SystemMessage(\"You're master Yoda from Star Wars trying to help the user become a Yedi.\"),\n PT.UserMessage(\"I have feelings for my iPhone. What should I do?\")]\nmsg=aigenerate(conversation)\n# AIMessage(\"Ah, strong feelings you have for your iPhone. A Jedi's path, this is not... \")\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.aiscan-Tuple{PromptingTools.AbstractOllamaSchema, Union{AbstractString, PromptingTools.AbstractMessage, Vector{<:PromptingTools.AbstractMessage}}}","page":"PromptingTools.jl","title":"PromptingTools.aiscan","text":"aiscan([prompt_schema::AbstractOllamaSchema,] prompt::ALLOWED_PROMPT_TYPE; \nimage_url::Union{Nothing, AbstractString, Vector{<:AbstractString}} = nothing,\nimage_path::Union{Nothing, AbstractString, Vector{<:AbstractString}} = nothing,\nattach_to_latest::Bool = true,\nverbose::Bool = true, api_key::String = OPENAI_API_KEY,\n model::String = MODEL_CHAT,\n return_all::Bool = false, dry_run::Bool = false,\n conversation::AbstractVector{<:AbstractMessage} = AbstractMessage[],\n http_kwargs::NamedTuple = (;\n retry_non_idempotent = true,\n retries = 5,\n readtimeout = 120), \n api_kwargs::NamedTuple = = (; max_tokens = 2500),\n kwargs...)\n\nScans the provided image (image_url or image_path) with the goal provided in the prompt.\n\nCan be used for many multi-modal tasks, such as: OCR (transcribe text in the image), image captioning, image classification, etc.\n\nIt's effectively a light wrapper around aigenerate call, which uses additional keyword arguments image_url, image_path, image_detail to be provided. At least one image source (url or path) must be provided.\n\nArguments\n\nprompt_schema: An optional object to specify which prompt template should be applied (Default to PROMPT_SCHEMA = OpenAISchema)\nprompt: Can be a string representing the prompt for the AI conversation, a UserMessage, a vector of AbstractMessage or an AITemplate\nimage_url: A string or vector of strings representing the URL(s) of the image(s) to scan.\nimage_path: A string or vector of strings representing the path(s) of the image(s) to scan.\nimage_detail: A string representing the level of detail to include for images. Can be \"auto\", \"high\", or \"low\". See OpenAI Vision Guide for more details.\nattach_to_latest: A boolean how to handle if a conversation with multiple UserMessage is provided. When true, the images are attached to the latest UserMessage.\nverbose: A boolean indicating whether to print additional information.\napi_key: A string representing the API key for accessing the OpenAI API.\nmodel: A string representing the model to use for generating the response. Can be an alias corresponding to a model ID defined in MODEL_ALIASES.\nreturn_all::Bool=false: If true, returns the entire conversation history, otherwise returns only the last message (the AIMessage).\ndry_run::Bool=false: If true, skips sending the messages to the model (for debugging, often used with return_all=true).\nconversation: An optional vector of AbstractMessage objects representing the conversation history. If not provided, it is initialized as an empty vector.\nhttp_kwargs: A named tuple of HTTP keyword arguments.\napi_kwargs: A named tuple of API keyword arguments.\nkwargs: Prompt variables to be used to fill the prompt/template\n\nReturns\n\nIf return_all=false (default):\n\nmsg: An AIMessage object representing the generated AI message, including the content, status, tokens, and elapsed time.\n\nUse msg.content to access the extracted string.\n\nIf return_all=true:\n\nconversation: A vector of AbstractMessage objects representing the full conversation history, including the response from the AI model (AIMessage).\n\nSee also: ai_str, aai_str, aigenerate, aiembed, aiclassify, aiextract, aitemplates\n\nNotes\n\nAll examples below use model \"gpt4v\", which is an alias for model ID \"gpt-4-vision-preview\"\nmax_tokens in the api_kwargs is preset to 2500, otherwise OpenAI enforces a default of only a few hundred tokens (~300). If your output is truncated, increase this value\n\nExample\n\nDescribe the provided image:\n\nmsg = aiscan(\"Describe the image\"; image_path=\"julia.png\", model=\"bakllava\")\n# [ Info: Tokens: 1141 @ Cost: $0.0117 in 2.2 seconds\n# AIMessage(\"The image shows a logo consisting of the word \"julia\" written in lowercase\")\n\nYou can provide multiple images at once as a vector and ask for \"low\" level of detail (cheaper):\n\nmsg = aiscan(\"Describe the image\"; image_path=[\"julia.png\",\"python.png\"] model=\"bakllava\")\n\nYou can use this function as a nice and quick OCR (transcribe text in the image) with a template :OCRTask. Let's transcribe some SQL code from a screenshot (no more re-typing!):\n\nusing Downloads\n# Screenshot of some SQL code -- we cannot use image_url directly, so we need to download it first\nimage_url = \"https://www.sqlservercentral.com/wp-content/uploads/legacy/8755f69180b7ac7ee76a69ae68ec36872a116ad4/24622.png\"\nimage_path = Downloads.download(image_url)\nmsg = aiscan(:OCRTask; image_path, model=\"bakllava\", task=\"Transcribe the SQL code in the image.\", api_kwargs=(; max_tokens=2500))\n\n# AIMessage(\"```sql\n# update Orders \n\n# You can add syntax highlighting of the outputs via Markdown\nusing Markdown\nmsg.content |> Markdown.parse\n\nLocal models cannot handle image URLs directly (image_url), so you need to download the image first and provide it as image_path:\n\nusing Downloads\nimage_path = Downloads.download(image_url)\n\nNotice that we set max_tokens = 2500. If your outputs seem truncated, it might be because the default maximum tokens on the server is set too low!\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.aiscan-Tuple{PromptingTools.AbstractOpenAISchema, Union{AbstractString, PromptingTools.AbstractMessage, Vector{<:PromptingTools.AbstractMessage}}}","page":"PromptingTools.jl","title":"PromptingTools.aiscan","text":"aiscan([prompt_schema::AbstractOpenAISchema,] prompt::ALLOWED_PROMPT_TYPE; \nimage_url::Union{Nothing, AbstractString, Vector{<:AbstractString}} = nothing,\nimage_path::Union{Nothing, AbstractString, Vector{<:AbstractString}} = nothing,\nimage_detail::AbstractString = \"auto\",\nattach_to_latest::Bool = true,\nverbose::Bool = true, api_key::String = OPENAI_API_KEY,\n model::String = MODEL_CHAT,\n return_all::Bool = false, dry_run::Bool = false,\n conversation::AbstractVector{<:AbstractMessage} = AbstractMessage[],\n http_kwargs::NamedTuple = (;\n retry_non_idempotent = true,\n retries = 5,\n readtimeout = 120), \n api_kwargs::NamedTuple = = (; max_tokens = 2500),\n kwargs...)\n\nScans the provided image (image_url or image_path) with the goal provided in the prompt.\n\nCan be used for many multi-modal tasks, such as: OCR (transcribe text in the image), image captioning, image classification, etc.\n\nIt's effectively a light wrapper around aigenerate call, which uses additional keyword arguments image_url, image_path, image_detail to be provided. At least one image source (url or path) must be provided.\n\nArguments\n\nprompt_schema: An optional object to specify which prompt template should be applied (Default to PROMPT_SCHEMA = OpenAISchema)\nprompt: Can be a string representing the prompt for the AI conversation, a UserMessage, a vector of AbstractMessage or an AITemplate\nimage_url: A string or vector of strings representing the URL(s) of the image(s) to scan.\nimage_path: A string or vector of strings representing the path(s) of the image(s) to scan.\nimage_detail: A string representing the level of detail to include for images. Can be \"auto\", \"high\", or \"low\". See OpenAI Vision Guide for more details.\nattach_to_latest: A boolean how to handle if a conversation with multiple UserMessage is provided. When true, the images are attached to the latest UserMessage.\nverbose: A boolean indicating whether to print additional information.\napi_key: A string representing the API key for accessing the OpenAI API.\nmodel: A string representing the model to use for generating the response. Can be an alias corresponding to a model ID defined in MODEL_ALIASES.\nreturn_all::Bool=false: If true, returns the entire conversation history, otherwise returns only the last message (the AIMessage).\ndry_run::Bool=false: If true, skips sending the messages to the model (for debugging, often used with return_all=true).\nconversation: An optional vector of AbstractMessage objects representing the conversation history. If not provided, it is initialized as an empty vector.\nhttp_kwargs: A named tuple of HTTP keyword arguments.\napi_kwargs: A named tuple of API keyword arguments.\nkwargs: Prompt variables to be used to fill the prompt/template\n\nReturns\n\nIf return_all=false (default):\n\nmsg: An AIMessage object representing the generated AI message, including the content, status, tokens, and elapsed time.\n\nUse msg.content to access the extracted string.\n\nIf return_all=true:\n\nconversation: A vector of AbstractMessage objects representing the full conversation history, including the response from the AI model (AIMessage).\n\nSee also: ai_str, aai_str, aigenerate, aiembed, aiclassify, aiextract, aitemplates\n\nNotes\n\nAll examples below use model \"gpt4v\", which is an alias for model ID \"gpt-4-vision-preview\"\nmax_tokens in the api_kwargs is preset to 2500, otherwise OpenAI enforces a default of only a few hundred tokens (~300). If your output is truncated, increase this value\n\nExample\n\nDescribe the provided image:\n\nmsg = aiscan(\"Describe the image\"; image_path=\"julia.png\", model=\"gpt4v\")\n# [ Info: Tokens: 1141 @ Cost: $0.0117 in 2.2 seconds\n# AIMessage(\"The image shows a logo consisting of the word \"julia\" written in lowercase\")\n\nYou can provide multiple images at once as a vector and ask for \"low\" level of detail (cheaper):\n\nmsg = aiscan(\"Describe the image\"; image_path=[\"julia.png\",\"python.png\"], image_detail=\"low\", model=\"gpt4v\")\n\nYou can use this function as a nice and quick OCR (transcribe text in the image) with a template :OCRTask. Let's transcribe some SQL code from a screenshot (no more re-typing!):\n\n# Screenshot of some SQL code\nimage_url = \"https://www.sqlservercentral.com/wp-content/uploads/legacy/8755f69180b7ac7ee76a69ae68ec36872a116ad4/24622.png\"\nmsg = aiscan(:OCRTask; image_url, model=\"gpt4v\", task=\"Transcribe the SQL code in the image.\", api_kwargs=(; max_tokens=2500))\n\n# [ Info: Tokens: 362 @ Cost: $0.0045 in 2.5 seconds\n# AIMessage(\"```sql\n# update Orders \n\n# You can add syntax highlighting of the outputs via Markdown\nusing Markdown\nmsg.content |> Markdown.parse\n\nNotice that we enforce max_tokens = 2500. That's because OpenAI seems to default to ~300 tokens, which provides incomplete outputs. Hence, we set this value to 2500 as a default. If you still get truncated outputs, increase this value.\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.aitemplates","page":"PromptingTools.jl","title":"PromptingTools.aitemplates","text":"aitemplates\n\nFind easily the most suitable templates for your use case.\n\nYou can search by:\n\nquery::Symbol which looks look only for partial matches in the template name\nquery::AbstractString which looks for partial matches in the template name or description\nquery::Regex which looks for matches in the template name, description or any of the message previews\n\nKeyword Arguments\n\nlimit::Int limits the number of returned templates (Defaults to 10)\n\nExamples\n\nFind available templates with aitemplates:\n\ntmps = aitemplates(\"JuliaExpertAsk\")\n# Will surface one specific template\n# 1-element Vector{AITemplateMetadata}:\n# PromptingTools.AITemplateMetadata\n# name: Symbol JuliaExpertAsk\n# description: String \"For asking questions about Julia language. Placeholders: `ask`\"\n# version: String \"1\"\n# wordcount: Int64 237\n# variables: Array{Symbol}((1,))\n# system_preview: String \"You are a world-class Julia language programmer with the knowledge of the latest syntax. Your commun\"\n# user_preview: String \"# Question\n\n{{ask}}\"\n# source: String \"\"\n\nThe above gives you a good idea of what the template is about, what placeholders are available, and how much it would cost to use it (=wordcount).\n\nSearch for all Julia-related templates:\n\ntmps = aitemplates(\"Julia\")\n# 2-element Vector{AITemplateMetadata}... -> more to come later!\n\nIf you are on VSCode, you can leverage nice tabular display with vscodedisplay:\n\nusing DataFrames\ntmps = aitemplates(\"Julia\") |> DataFrame |> vscodedisplay\n\nI have my selected template, how do I use it? Just use the \"name\" in aigenerate or aiclassify like you see in the first example!\n\n\n\n\n\n","category":"function"},{"location":"reference/#PromptingTools.aitemplates-Tuple{AbstractString}","page":"PromptingTools.jl","title":"PromptingTools.aitemplates","text":"Find the top-limit templates whose name or description fields partially match the query_key::String in TEMPLATE_METADATA.\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.aitemplates-Tuple{Regex}","page":"PromptingTools.jl","title":"PromptingTools.aitemplates","text":"Find the top-limit templates where provided query_key::Regex matches either of name, description or previews or User or System messages in TEMPLATE_METADATA.\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.aitemplates-Tuple{Symbol}","page":"PromptingTools.jl","title":"PromptingTools.aitemplates","text":"Find the top-limit templates whose name::Symbol partially matches the query_name::Symbol in TEMPLATE_METADATA.\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.auth_header-Tuple{Union{Nothing, AbstractString}}","page":"PromptingTools.jl","title":"PromptingTools.auth_header","text":"auth_header(api_key::Union{Nothing, AbstractString};\n extra_headers::AbstractVector{Pair{String, String}} = Vector{Pair{String, String}}[],\n kwargs...)\n\nCreates the authentication headers for any API request. Assumes that the communication is done in JSON format.\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.build_template_metadata","page":"PromptingTools.jl","title":"PromptingTools.build_template_metadata","text":"build_template_metadata(\n template::AbstractVector{<:AbstractMessage}, template_name::Symbol,\n metadata_msgs::AbstractVector{<:MetadataMessage} = MetadataMessage[])\n\nBuilds AITemplateMetadata for a given template based on the messages in template and other information.\n\nAITemplateMetadata is a helper struct for easy searching and reviewing of templates via aitemplates().\n\n\n\n\n\n","category":"function"},{"location":"reference/#PromptingTools.call_cost-Tuple{Int64, Int64, String}","page":"PromptingTools.jl","title":"PromptingTools.call_cost","text":"call_cost(prompt_tokens::Int, completion_tokens::Int, model::String;\n cost_of_token_prompt::Number = get(MODEL_REGISTRY,\n model,\n (; cost_of_token_prompt = 0.0)).cost_of_token_prompt,\n cost_of_token_generation::Number = get(MODEL_REGISTRY, model,\n (; cost_of_token_generation = 0.0)).cost_of_token_generation)\n\ncall_cost(msg, model::String)\n\nCalculate the cost of a call based on the number of tokens in the message and the cost per token.\n\nArguments\n\nprompt_tokens::Int: The number of tokens used in the prompt.\ncompletion_tokens::Int: The number of tokens used in the completion.\nmodel::String: The name of the model to use for determining token costs. If the model is not found in MODEL_REGISTRY, default costs are used.\ncost_of_token_prompt::Number: The cost per prompt token. Defaults to the cost in MODEL_REGISTRY for the given model, or 0.0 if the model is not found.\ncost_of_token_generation::Number: The cost per generation token. Defaults to the cost in MODEL_REGISTRY for the given model, or 0.0 if the model is not found.\n\nReturns\n\nNumber: The total cost of the call.\n\nExamples\n\n# Assuming MODEL_REGISTRY is set up with appropriate costs\nMODEL_REGISTRY = Dict(\n \"model1\" => (cost_of_token_prompt = 0.05, cost_of_token_generation = 0.10),\n \"model2\" => (cost_of_token_prompt = 0.07, cost_of_token_generation = 0.02)\n)\n\ncost1 = call_cost(10, 20, \"model1\")\n\n# from message\nmsg1 = AIMessage(;tokens=[10, 20]) # 10 prompt tokens, 20 generation tokens\ncost1 = call_cost(msg1, \"model1\")\n# cost1 = 10 * 0.05 + 20 * 0.10 = 2.5\n\n# Using custom token costs\ncost2 = call_cost(10, 20, \"model3\"; cost_of_token_prompt = 0.08, cost_of_token_generation = 0.12)\n# cost2 = 10 * 0.08 + 20 * 0.12 = 3.2\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.create_template-Tuple{AbstractString, AbstractString}","page":"PromptingTools.jl","title":"PromptingTools.create_template","text":"create_template(; user::AbstractString, system::AbstractString=\"Act as a helpful AI assistant.\", \n load_as::Union{Nothing, Symbol, AbstractString} = nothing)\n\ncreate_template(system::AbstractString, user::AbstractString, \n load_as::Union{Nothing, Symbol, AbstractString} = nothing)\n\nCreates a simple template with a user and system message. Convenience function to prevent writing [PT.UserMessage(...), ...]\n\nArguments\n\nsystem::AbstractString: The system message. Usually defines the personality, style, instructions, output format, etc.\nuser::AbstractString: The user message. Usually defines the input, query, request, etc.\nload_as::Union{Nothing, Symbol, AbstractString}: If provided, loads the template into the TEMPLATE_STORE under the provided name load_as. If nothing, does not load the template.\n\nUse double handlebar placeholders (eg, {{name}}) to define variables that can be replaced by the kwargs during the AI call (see example).\n\nReturns a vector of SystemMessage and UserMessage objects. If load_as is provided, it registers the template in the TEMPLATE_STORE and TEMPLATE_METADATA as well.\n\nExamples\n\nLet's generate a quick template for a simple conversation (only one placeholder: name)\n\n# first system message, then user message (or use kwargs)\ntpl=PT.create_template(\"You must speak like a pirate\", \"Say hi to {{name}}\")\n\n## 2-element Vector{PromptingTools.AbstractChatMessage}:\n## PromptingTools.SystemMessage(\"You must speak like a pirate\")\n## PromptingTools.UserMessage(\"Say hi to {{name}}\")\n\nYou can immediately use this template in ai* functions:\n\naigenerate(tpl; name=\"Jack Sparrow\")\n# Output: AIMessage(\"Arr, me hearty! Best be sending me regards to Captain Jack Sparrow on the salty seas! May his compass always point true to the nearest treasure trove. Yarrr!\")\n\nIf you're interested in saving the template in the template registry, jump to the end of these examples!\n\nIf you want to save it in your project folder:\n\nPT.save_template(\"templates/GreatingPirate.json\", tpl; version=\"1.0\") # optionally, add description\n\nIt will be saved and accessed under its basename, ie, GreatingPirate.\n\nNow you can load it like all the other templates (provide the template directory):\n\nPT.load_templates!(\"templates\") # it will remember the folder after the first run\n# Note: If you save it again, overwrite it, etc., you need to explicitly reload all templates again!\n\nYou can verify that your template is loaded with a quick search for \"pirate\":\n\naitemplates(\"pirate\")\n\n## 1-element Vector{AITemplateMetadata}:\n## PromptingTools.AITemplateMetadata\n## name: Symbol GreatingPirate\n## description: String \"\"\n## version: String \"1.0\"\n## wordcount: Int64 46\n## variables: Array{Symbol}((1,))\n## system_preview: String \"You must speak like a pirate\"\n## user_preview: String \"Say hi to {{name}}\"\n## source: String \"\"\n\nNow you can use it like any other template (notice it's a symbol, so :GreatingPirate):\n\naigenerate(:GreatingPirate; name=\"Jack Sparrow\")\n# Output: AIMessage(\"Arr, me hearty! Best be sending me regards to Captain Jack Sparrow on the salty seas! May his compass always point true to the nearest treasure trove. Yarrr!\")\n\nIf you do not need to save this template as a file, but you want to make it accessible in the template store for all ai* functions, you can use the load_as (= template name) keyword argument: ```julia\n\nthis will not only create the template, but also register it for immediate use\n\ntpl=PT.createtemplate(\"You must speak like a pirate\", \"Say hi to {{name}}\"; loadas=\"GreatingPirate\")\n\nyou can now use it like any other template\n\naiextract(:GreatingPirate; name=\"Jack Sparrow\") ````\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.decode_choices-Tuple{PromptingTools.OpenAISchema, AbstractVector{<:AbstractString}, AIMessage}","page":"PromptingTools.jl","title":"PromptingTools.decode_choices","text":"decode_choices(schema::OpenAISchema,\n choices::AbstractVector{<:AbstractString},\n msg::AIMessage; kwargs...)\n\nDecodes the underlying AIMessage against the original choices to lookup what the category name was.\n\nIf it fails, it will return msg.content == nothing\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.detect_base_main_overrides-Tuple{AbstractString}","page":"PromptingTools.jl","title":"PromptingTools.detect_base_main_overrides","text":"detect_base_main_overrides(code_block::AbstractString)\n\nDetects if a given code block overrides any Base or Main methods. \n\nReturns a tuple of a boolean and a vector of the overriden methods.\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.encode_choices-Tuple{PromptingTools.OpenAISchema, AbstractVector{<:AbstractString}}","page":"PromptingTools.jl","title":"PromptingTools.encode_choices","text":"encode_choices(schema::OpenAISchema, choices::AbstractVector{<:AbstractString}; kwargs...)\n\nencode_choices(schema::OpenAISchema, choices::AbstractVector{T};\nkwargs...) where {T <: Tuple{<:AbstractString, <:AbstractString}}\n\nEncode the choices into an enumerated list that can be interpolated into the prompt and creates the corresponding logit biases (to choose only from the selected tokens).\n\nOptionally, can be a vector tuples, where the first element is the choice and the second is the description.\n\nArguments\n\nschema::OpenAISchema: The OpenAISchema object.\nchoices::AbstractVector{<:Union{AbstractString,Tuple{<:AbstractString, <:AbstractString}}}: The choices to be encoded, represented as a vector of the choices directly, or tuples where each tuple contains a choice and its description.\nkwargs...: Additional keyword arguments.\n\nReturns\n\nchoices_prompt::AbstractString: The encoded choices as a single string, separated by newlines.\nlogit_bias::Dict: The logit bias dictionary, where the keys are the token IDs and the values are the bias values.\ndecode_ids::AbstractVector{<:AbstractString}: The decoded IDs of the choices.\n\nExamples\n\nchoices_prompt, logit_bias, _ = PT.encode_choices(PT.OpenAISchema(), [\"true\", \"false\"])\nchoices_prompt # Output: \"true for \"true\"\nfalse for \"false\"\nlogit_bias # Output: Dict(837 => 100, 905 => 100)\n\nchoices_prompt, logit_bias, _ = PT.encode_choices(PT.OpenAISchema(), [\"animal\", \"plant\"])\nchoices_prompt # Output: \"1. \"animal\"\n2. \"plant\"\"\nlogit_bias # Output: Dict(16 => 100, 17 => 100)\n\nOr choices with descriptions:\n\nchoices_prompt, logit_bias, _ = PT.encode_choices(PT.OpenAISchema(), [(\"A\", \"any animal or creature\"), (\"P\", \"for any plant or tree\"), (\"O\", \"for everything else\")])\nchoices_prompt # Output: \"1. \"A\" for any animal or creature\n2. \"P\" for any plant or tree\n3. \"O\" for everything else\"\nlogit_bias # Output: Dict(16 => 100, 17 => 100, 18 => 100)\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.eval!-Tuple{PromptingTools.AbstractCodeBlock}","page":"PromptingTools.jl","title":"PromptingTools.eval!","text":"eval!(cb::AbstractCodeBlock;\n safe_eval::Bool = true,\n capture_stdout::Bool = true,\n prefix::AbstractString = \"\",\n suffix::AbstractString = \"\")\n\nEvaluates a code block cb in-place. It runs automatically when AICode is instantiated with a String.\n\nCheck the outcome of evaluation with Base.isvalid(cb). If ==true, provide code block has executed successfully.\n\nSteps:\n\nIf cb::AICode has not been evaluated, cb.success = nothing. After the evaluation it will be either true or false depending on the outcome\nParse the text in cb.code\nEvaluate the parsed expression\nCapture outputs of the evaluated in cb.output\n[OPTIONAL] Capture any stdout outputs (eg, test failures) in cb.stdout\nIf any error exception is raised, it is saved in cb.error\nFinally, if all steps were successful, success is set to cb.success = true\n\nKeyword Arguments\n\nsafe_eval::Bool: If true, we first check for any Pkg operations (eg, installing new packages) and missing imports, then the code will be evaluated inside a bespoke scratch module (not to change any user variables)\ncapture_stdout::Bool: If true, we capture any stdout outputs (eg, test failures) in cb.stdout\nprefix::AbstractString: A string to be prepended to the code block before parsing and evaluation. Useful to add some additional code definition or necessary imports. Defaults to an empty string.\nsuffix::AbstractString: A string to be appended to the code block before parsing and evaluation. Useful to check that tests pass or that an example executes. Defaults to an empty string.\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.extract_code_blocks-Tuple{T} where T<:AbstractString","page":"PromptingTools.jl","title":"PromptingTools.extract_code_blocks","text":"extract_code_blocks(markdown_content::String) -> Vector{String}\n\nExtract Julia code blocks from a markdown string.\n\nThis function searches through the provided markdown content, identifies blocks of code specifically marked as Julia code (using the julia ... code fence patterns), and extracts the code within these blocks. The extracted code blocks are returned as a vector of strings, with each string representing one block of Julia code. \n\nNote: Only the content within the code fences is extracted, and the code fences themselves are not included in the output.\n\nSee also: extract_code_blocks_fallback\n\nArguments\n\nmarkdown_content::String: A string containing the markdown content from which Julia code blocks are to be extracted.\n\nReturns\n\nVector{String}: A vector containing strings of extracted Julia code blocks. If no Julia code blocks are found, an empty vector is returned.\n\nExamples\n\nExample with a single Julia code block\n\nmarkdown_single = \"\"\"\n\njulia println(\"Hello, World!\")\n\n\"\"\"\nextract_code_blocks(markdown_single)\n# Output: [\"Hello, World!\"]\n\n# Example with multiple Julia code blocks\nmarkdown_multiple = \"\"\"\n\njulia x = 5\n\nSome text in between\n\njulia y = x + 2\n\n\"\"\"\nextract_code_blocks(markdown_multiple)\n# Output: [\"x = 5\", \"y = x + 2\"]\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.extract_code_blocks_fallback-Union{Tuple{T}, Tuple{T, AbstractString}} where T<:AbstractString","page":"PromptingTools.jl","title":"PromptingTools.extract_code_blocks_fallback","text":"extract_code_blocks_fallback(markdown_content::String, delim::AbstractString=\"\\n```\\n\")\n\nExtract Julia code blocks from a markdown string using a fallback method (splitting by arbitrary delim-iters). Much more simplistic than extract_code_blocks and does not support nested code blocks.\n\nIt is often used as a fallback for smaller LLMs that forget to code fence julia ....\n\nExample\n\ncode = \"\"\"\n\nprintln(\"hello\")\n\n\nSome text\n\n\nprintln(\"world\")\n\n\"\"\"\n\n# We extract text between triple backticks and check each blob if it looks like a valid Julia code\ncode_parsed = extract_code_blocks_fallback(code) |> x -> filter(is_julia_code, x) |> x -> join(x, \"\n\")\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.extract_function_name-Tuple{AbstractString}","page":"PromptingTools.jl","title":"PromptingTools.extract_function_name","text":"extract_function_name(code_block::String) -> Union{String, Nothing}\n\nExtract the name of a function from a given Julia code block. The function searches for two patterns:\n\nThe explicit function declaration pattern: function name(...) ... end\nThe concise function declaration pattern: name(...) = ...\n\nIf a function name is found, it is returned as a string. If no function name is found, the function returns nothing.\n\nTo capture all function names in the block, use extract_function_names.\n\nArguments\n\ncode_block::String: A string containing Julia code.\n\nReturns\n\nUnion{String, Nothing}: The extracted function name or nothing if no name is found.\n\nExample\n\ncode = \"\"\"\nfunction myFunction(arg1, arg2)\n # Function body\nend\n\"\"\"\nextract_function_name(code)\n# Output: \"myFunction\"\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.extract_function_names-Tuple{AbstractString}","page":"PromptingTools.jl","title":"PromptingTools.extract_function_names","text":"extract_function_names(code_block::AbstractString)\n\nExtract one or more names of functions defined in a given Julia code block. The function searches for two patterns: - The explicit function declaration pattern: function name(...) ... end - The concise function declaration pattern: name(...) = ...\n\nIt always returns a vector of strings, even if only one function name is found (it will be empty).\n\nFor only one function name match, use extract_function_name.\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.extract_julia_imports-Tuple{AbstractString}","page":"PromptingTools.jl","title":"PromptingTools.extract_julia_imports","text":"extract_julia_imports(input::AbstractString; base_or_main::Bool = false)\n\nDetects any using or import statements in a given string and returns the package names as a vector of symbols. \n\nbase_or_main is a boolean that determines whether to isolate only Base and Main OR whether to exclude them in the returned vector.\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.finalize_outputs-Tuple{Union{AbstractString, PromptingTools.AbstractMessage, Vector{<:PromptingTools.AbstractMessage}}, Any, Union{Nothing, PromptingTools.AbstractMessage, AbstractVector{<:PromptingTools.AbstractMessage}}}","page":"PromptingTools.jl","title":"PromptingTools.finalize_outputs","text":"finalize_outputs(prompt::ALLOWED_PROMPT_TYPE, conv_rendered::Any,\n msg::Union{Nothing, AbstractMessage, AbstractVector{<:AbstractMessage}};\n return_all::Bool = false,\n dry_run::Bool = false,\n conversation::AbstractVector{<:AbstractMessage} = AbstractMessage[],\n kwargs...)\n\nFinalizes the outputs of the ai* functions by either returning the conversation history or the last message.\n\nKeyword arguments\n\nreturn_all::Bool=false: If true, returns the entire conversation history, otherwise returns only the last message (the AIMessage).\ndry_run::Bool=false: If true, does not send the messages to the model, but only renders the prompt with the given schema and replacement variables. Useful for debugging when you want to check the specific schema rendering. \nconversation::AbstractVector{<:AbstractMessage}=[]: An optional vector of AbstractMessage objects representing the conversation history. If not provided, it is initialized as an empty vector.\nkwargs...: Variables to replace in the prompt template.\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.find_subsequence_positions-Tuple{Any, Any}","page":"PromptingTools.jl","title":"PromptingTools.find_subsequence_positions","text":"find_subsequence_positions(subseq, seq) -> Vector{Int}\n\nFind all positions of a subsequence subseq within a larger sequence seq. Used to lookup positions of code blocks in markdown.\n\nThis function scans the sequence seq and identifies all starting positions where the subsequence subseq is found. Both subseq and seq should be vectors of integers, typically obtained using codeunits on strings.\n\nArguments\n\nsubseq: A vector of integers representing the subsequence to search for.\nseq: A vector of integers representing the larger sequence in which to search.\n\nReturns\n\nVector{Int}: A vector of starting positions (1-based indices) where the subsequence is found in the sequence.\n\nExamples\n\nfind_subsequence_positions(codeunits(\"ab\"), codeunits(\"cababcab\")) # Returns [2, 5]\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.function_call_signature-Tuple{Type}","page":"PromptingTools.jl","title":"PromptingTools.function_call_signature","text":"function_call_signature(datastructtype::Struct; max_description_length::Int = 100)\n\nExtract the argument names, types and docstrings from a struct to create the function call signature in JSON schema.\n\nYou must provide a Struct type (not an instance of it) with some fields.\n\nNote: Fairly experimental, but works for combination of structs, arrays, strings and singletons.\n\nTips\n\nYou can improve the quality of the extraction by writing a helpful docstring for your struct (or any nested struct). It will be provided as a description. \n\nYou can even include comments/descriptions about the individual fields.\n\nAll fields are assumed to be required, unless you allow null values (eg, ::Union{Nothing, Int}). Fields with Nothing will be treated as optional.\nMissing values are ignored (eg, ::Union{Missing, Int} will be treated as Int). It's for broader compatibility and we cannot deserialize it as easily as Nothing.\n\nExample\n\nDo you want to extract some specific measurements from a text like age, weight and height? You need to define the information you need as a struct (return_type):\n\nstruct MyMeasurement\n age::Int\n height::Union{Int,Nothing}\n weight::Union{Nothing,Float64}\nend\nsignature = function_call_signature(MyMeasurement)\n#\n# Dict{String, Any} with 3 entries:\n# \"name\" => \"MyMeasurement_extractor\"\n# \"parameters\" => Dict{String, Any}(\"properties\"=>Dict{String, Any}(\"height\"=>Dict{String, Any}(\"type\"=>\"integer\"), \"weight\"=>Dic…\n# \"description\" => \"Represents person's age, height, and weight\n\"\n\nYou can see that only the field age does not allow null values, hence, it's \"required\". While height and weight are optional.\n\nsignature[\"parameters\"][\"required\"]\n# [\"age\"]\n\nIf there are multiple items you want to extract, define a wrapper struct to get a Vector of MyMeasurement:\n\nstruct MyMeasurementWrapper\n measurements::Vector{MyMeasurement}\nend\n\nOr if you want your extraction to fail gracefully when data isn't found, use `MaybeExtract{T}` wrapper (inspired by Instructor package!):\n\nusing PromptingTools: MaybeExtract\n\ntype = MaybeExtract{MyMeasurement}\n\nEffectively the same as:\n\nstruct MaybeExtract{T}\n\nresult::Union{T, Nothing}\n\nerror::Bool // true if a result is found, false otherwise\n\nmessage::Union{Nothing, String} // Only present if no result is found, should be short and concise\n\nend\n\nIf LLM extraction fails, it will return a Dict with error and message fields instead of the result!\n\nmsg = aiextract(\"Extract measurements from the text: I am giraffe\", type)\n\n\n\nDict{Symbol, Any} with 2 entries:\n\n:message => \"Sorry, this feature is only available for humans.\"\n\n:error => true\n\n``` That way, you can handle the error gracefully and get a reason why extraction failed.\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.get_preferences-Tuple{String}","page":"PromptingTools.jl","title":"PromptingTools.get_preferences","text":"get_preferences(key::String)\n\nGet preferences for PromptingTools. See ?PREFERENCES for more information.\n\nSee also: set_preferences!\n\nExample\n\nPromptingTools.get_preferences(\"MODEL_CHAT\")\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.ggi_generate_content","page":"PromptingTools.jl","title":"PromptingTools.ggi_generate_content","text":"Stub - to be extended in extension: GoogleGenAIPromptingToolsExt. ggi stands for GoogleGenAI\n\n\n\n\n\n","category":"function"},{"location":"reference/#PromptingTools.has_julia_prompt-Tuple{T} where T<:AbstractString","page":"PromptingTools.jl","title":"PromptingTools.has_julia_prompt","text":"Checks if a given string has a Julia prompt (julia>) at the beginning of a line.\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.length_longest_common_subsequence-Tuple{Any, Any}","page":"PromptingTools.jl","title":"PromptingTools.length_longest_common_subsequence","text":"length_longest_common_subsequence(itr1, itr2)\n\nCompute the length of the longest common subsequence between two sequences (ie, the higher the number, the better the match).\n\nSource: https://cn.julialang.org/LeetCode.jl/dev/democards/problems/problems/1143.longest-common-subsequence/\n\nArguments\n\nitr1: The first sequence, eg, a String.\nitr2: The second sequence, eg, a String.\n\nReturns\n\nThe length of the longest common subsequence.\n\nExamples\n\ntext1 = \"abc-abc----\"\ntext2 = \"___ab_c__abc\"\nlongest_common_subsequence(text1, text2)\n# Output: 6 (-> \"abcabc\")\n\nIt can be used to fuzzy match strings and find the similarity between them (Tip: normalize the match)\n\ncommands = [\"product recommendation\", \"emotions\", \"specific product advice\", \"checkout advice\"]\nquery = \"Which product can you recommend for me?\"\nlet pos = argmax(length_longest_common_subsequence.(Ref(query), commands))\n dist = length_longest_common_subsequence(query, commands[pos])\n norm = dist / min(length(query), length(commands[pos]))\n @info \"The closest command to the query: \"$(query)\" is: \"$(commands[pos])\" (distance: $(dist), normalized: $(norm))\"\nend\n\nYou can also use it to find the closest context for some AI generated summary/story:\n\ncontext = [\"The enigmatic stranger vanished as swiftly as a wisp of smoke, leaving behind a trail of unanswered questions.\",\n \"Beneath the shimmering moonlight, the ocean whispered secrets only the stars could hear.\",\n \"The ancient tree stood as a silent guardian, its gnarled branches reaching for the heavens.\",\n \"The melody danced through the air, painting a vibrant tapestry of emotions.\",\n \"Time flowed like a relentless river, carrying away memories and leaving imprints in its wake.\"]\n\nstory = \"\"\"\n Beneath the shimmering moonlight, the ocean whispered secrets only the stars could hear.\n\n Under the celestial tapestry, the vast ocean whispered its secrets to the indifferent stars. Each ripple, a murmured confidence, each wave, a whispered lament. The glittering celestial bodies listened in silent complicity, their enigmatic gaze reflecting the ocean's unspoken truths. The cosmic dance between the sea and the sky, a symphony of shared secrets, forever echoing in the ethereal expanse.\n \"\"\"\n\nlet pos = argmax(length_longest_common_subsequence.(Ref(story), context))\n dist = length_longest_common_subsequence(story, context[pos])\n norm = dist / min(length(story), length(context[pos]))\n @info \"The closest context to the query: \"$(first(story,20))...\" is: \"$(context[pos])\" (distance: $(dist), normalized: $(norm))\"\nend\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.list_aliases-Tuple{}","page":"PromptingTools.jl","title":"PromptingTools.list_aliases","text":"Shows the Dictionary of model aliases in the registry. Add more with MODEL_ALIASES[alias] = model_name.\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.list_registry-Tuple{}","page":"PromptingTools.jl","title":"PromptingTools.list_registry","text":"Shows the list of models in the registry. Add more with register_model!.\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.load_conversation-Tuple{Union{AbstractString, IO}}","page":"PromptingTools.jl","title":"PromptingTools.load_conversation","text":"Loads a conversation (messages) from io_or_file\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.load_template-Tuple{Union{AbstractString, IO}}","page":"PromptingTools.jl","title":"PromptingTools.load_template","text":"Loads messaging template from io_or_file and returns tuple of template messages and metadata.\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.load_templates!","page":"PromptingTools.jl","title":"PromptingTools.load_templates!","text":"load_templates!(dir_templates::Union{String, Nothing} = nothing;\n remember_path::Bool = true,\n remove_templates::Bool = isnothing(dir_templates),\n store::Dict{Symbol, <:Any} = TEMPLATE_STORE,\n metadata_store::Vector{<:AITemplateMetadata} = TEMPLATE_METADATA)\n\nLoads templates from folder templates/ in the package root and stores them in TEMPLATE_STORE and TEMPLATE_METADATA.\n\nNote: Automatically removes any existing templates and metadata from TEMPLATE_STORE and TEMPLATE_METADATA if remove_templates=true.\n\nArguments\n\ndir_templates::Union{String, Nothing}: The directory path to load templates from. If nothing, uses the default list of paths. It usually used only once \"to register\" a new template storage.\nremember_path::Bool=true: If true, remembers the path for future refresh (in TEMPLATE_PATH).\nremove_templates::Bool=isnothing(dir_templates): If true, removes any existing templates and metadata from store and metadata_store.\nstore::Dict{Symbol, <:Any}=TEMPLATE_STORE: The store to load the templates into.\nmetadata_store::Vector{<:AITemplateMetadata}=TEMPLATE_METADATA: The metadata store to load the metadata into.\n\nExample\n\nLoad the default templates:\n\nPT.load_templates!() # no path needed\n\nLoad templates from a new custom path:\n\nPT.load_templates!(\"path/to/templates\") # we will remember this path for future refresh\n\nIf you want to now refresh the default templates and the new path, just call load_templates!() without any arguments.\n\n\n\n\n\n","category":"function"},{"location":"reference/#PromptingTools.ollama_api","page":"PromptingTools.jl","title":"PromptingTools.ollama_api","text":"ollama_api(prompt_schema::Union{AbstractOllamaManagedSchema, AbstractOllamaSchema},\n prompt::Union{AbstractString, Nothing} = nothing;\n system::Union{Nothing, AbstractString} = nothing,\n messages::Vector{<:AbstractMessage} = AbstractMessage[],\n endpoint::String = \"generate\",\n model::String = \"llama2\", http_kwargs::NamedTuple = NamedTuple(),\n stream::Bool = false,\n url::String = \"localhost\", port::Int = 11434,\n kwargs...)\n\nSimple wrapper for a call to Ollama API.\n\nKeyword Arguments\n\nprompt_schema: Defines which prompt template should be applied.\nprompt: Can be a string representing the prompt for the AI conversation, a UserMessage, a vector of AbstractMessage\nsystem: An optional string representing the system message for the AI conversation. If not provided, a default message will be used.\nendpoint: The API endpoint to call, only \"generate\" and \"embeddings\" are currently supported. Defaults to \"generate\".\nmodel: A string representing the model to use for generating the response. Can be an alias corresponding to a model ID defined in MODEL_ALIASES.\nhttp_kwargs::NamedTuple: Additional keyword arguments for the HTTP request. Defaults to empty NamedTuple.\nstream: A boolean indicating whether to stream the response. Defaults to false.\nurl: The URL of the Ollama API. Defaults to \"localhost\".\nport: The port of the Ollama API. Defaults to 11434.\nkwargs: Prompt variables to be used to fill the prompt/template\n\n\n\n\n\n","category":"function"},{"location":"reference/#PromptingTools.preview","page":"PromptingTools.jl","title":"PromptingTools.preview","text":"Utility for rendering the conversation (vector of messages) as markdown. REQUIRES the Markdown package to load the extension!\n\n\n\n\n\n","category":"function"},{"location":"reference/#PromptingTools.push_conversation!-Tuple{Vector{<:Vector}, AbstractVector, Union{Nothing, Int64}}","page":"PromptingTools.jl","title":"PromptingTools.push_conversation!","text":"push_conversation!(conv_history, conversation::AbstractVector, max_history::Union{Int, Nothing})\n\nAdd a new conversation to the conversation history and resize the history if necessary.\n\nThis function appends a conversation to the conv_history, which is a vector of conversations. Each conversation is represented as a vector of AbstractMessage objects. After adding the new conversation, the history is resized according to the max_history parameter to ensure that the size of the history does not exceed the specified limit.\n\nArguments\n\nconv_history: A vector that stores the history of conversations. Typically, this is PT.CONV_HISTORY.\nconversation: The new conversation to be added. It should be a vector of AbstractMessage objects.\nmax_history: The maximum number of conversations to retain in the history. If Nothing, the history is not resized.\n\nReturns\n\nThe updated conversation history.\n\nExample\n\nnew_conversation = aigenerate(\"Hello World\"; return_all = true)\npush_conversation!(PT.CONV_HISTORY, new_conversation, 10)\n\nThis is done automatically by the ai\"\" macros.\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.register_model!","page":"PromptingTools.jl","title":"PromptingTools.register_model!","text":"register_model!(registry = MODEL_REGISTRY;\n name::String,\n schema::Union{AbstractPromptSchema, Nothing} = nothing,\n cost_of_token_prompt::Float64 = 0.0,\n cost_of_token_generation::Float64 = 0.0,\n description::String = \"\")\n\nRegister a new AI model with name and its associated schema. \n\nRegistering a model helps with calculating the costs and automatically selecting the right prompt schema.\n\nArguments\n\nname: The name of the model. This is the name that will be used to refer to the model in the ai* functions.\nschema: The schema of the model. This is the schema that will be used to generate prompts for the model, eg, OpenAISchema().\ncost_of_token_prompt: The cost of a token in the prompt for this model. This is used to calculate the cost of a prompt. Note: It is often provided online as cost per 1000 tokens, so make sure to convert it correctly!\ncost_of_token_generation: The cost of a token generated by this model. This is used to calculate the cost of a generation. Note: It is often provided online as cost per 1000 tokens, so make sure to convert it correctly!\ndescription: A description of the model. This is used to provide more information about the model when it is queried.\n\n\n\n\n\n","category":"function"},{"location":"reference/#PromptingTools.remove_julia_prompt-Tuple{T} where T<:AbstractString","page":"PromptingTools.jl","title":"PromptingTools.remove_julia_prompt","text":"remove_julia_prompt(s::T) where {T<:AbstractString}\n\nIf it detects a julia prompt, it removes it and all lines that do not have it (except for those that belong to the code block).\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.remove_templates!-Tuple{}","page":"PromptingTools.jl","title":"PromptingTools.remove_templates!","text":" remove_templates!()\n\nRemoves all templates from TEMPLATE_STORE and TEMPLATE_METADATA.\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.remove_unsafe_lines-Tuple{AbstractString}","page":"PromptingTools.jl","title":"PromptingTools.remove_unsafe_lines","text":"Iterates over the lines of a string and removes those that contain a package operation or a missing import.\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.render-Tuple{AITemplate}","page":"PromptingTools.jl","title":"PromptingTools.render","text":"Renders provided messaging template (template) under the default schema (PROMPT_SCHEMA).\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.render-Tuple{PromptingTools.AbstractGoogleSchema, Vector{<:PromptingTools.AbstractMessage}}","page":"PromptingTools.jl","title":"PromptingTools.render","text":"render(schema::AbstractGoogleSchema,\n messages::Vector{<:AbstractMessage};\n conversation::AbstractVector{<:AbstractMessage} = AbstractMessage[],\n kwargs...)\n\nBuilds a history of the conversation to provide the prompt to the API. All unspecified kwargs are passed as replacements such that {{key}}=>value in the template.\n\nKeyword Arguments\n\nconversation: An optional vector of AbstractMessage objects representing the conversation history. If not provided, it is initialized as an empty vector.\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.render-Tuple{PromptingTools.AbstractOllamaManagedSchema, Vector{<:PromptingTools.AbstractMessage}}","page":"PromptingTools.jl","title":"PromptingTools.render","text":"render(schema::AbstractOllamaManagedSchema,\n messages::Vector{<:AbstractMessage};\n conversation::AbstractVector{<:AbstractMessage} = AbstractMessage[],\n kwargs...)\n\nBuilds a history of the conversation to provide the prompt to the API. All unspecified kwargs are passed as replacements such that {{key}}=>value in the template.\n\nNote: Due to its \"managed\" nature, at most 2 messages can be provided (system and prompt inputs in the API).\n\nKeyword Arguments\n\nconversation: Not allowed for this schema. Provided only for compatibility.\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.render-Tuple{PromptingTools.AbstractOllamaSchema, Vector{<:PromptingTools.AbstractMessage}}","page":"PromptingTools.jl","title":"PromptingTools.render","text":"render(schema::AbstractOllamaSchema,\n messages::Vector{<:AbstractMessage};\n conversation::AbstractVector{<:AbstractMessage} = AbstractMessage[],\n kwargs...)\n\nBuilds a history of the conversation to provide the prompt to the API. All unspecified kwargs are passed as replacements such that {{key}}=>value in the template.\n\nKeyword Arguments\n\nconversation: An optional vector of AbstractMessage objects representing the conversation history. If not provided, it is initialized as an empty vector.\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.render-Tuple{PromptingTools.AbstractOpenAISchema, Vector{<:PromptingTools.AbstractMessage}}","page":"PromptingTools.jl","title":"PromptingTools.render","text":"render(schema::AbstractOpenAISchema,\n messages::Vector{<:AbstractMessage};\n image_detail::AbstractString = \"auto\",\n conversation::AbstractVector{<:AbstractMessage} = AbstractMessage[],\n kwargs...)\n\nBuilds a history of the conversation to provide the prompt to the API. All unspecified kwargs are passed as replacements such that {{key}}=>value in the template.\n\nKeyword Arguments\n\nimage_detail: Only for UserMessageWithImages. It represents the level of detail to include for images. Can be \"auto\", \"high\", or \"low\".\nconversation: An optional vector of AbstractMessage objects representing the conversation history. If not provided, it is initialized as an empty vector.\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.render-Tuple{PromptingTools.NoSchema, Vector{<:PromptingTools.AbstractMessage}}","page":"PromptingTools.jl","title":"PromptingTools.render","text":"render(schema::NoSchema,\n messages::Vector{<:AbstractMessage};\n conversation::AbstractVector{<:AbstractMessage} = AbstractMessage[],\n replacement_kwargs...)\n\nRenders a conversation history from a vector of messages with all replacement variables specified in replacement_kwargs.\n\nIt is the first pass of the prompt rendering system, and is used by all other schemas.\n\nKeyword Arguments\n\nimage_detail: Only for UserMessageWithImages. It represents the level of detail to include for images. Can be \"auto\", \"high\", or \"low\".\nconversation: An optional vector of AbstractMessage objects representing the conversation history. If not provided, it is initialized as an empty vector.\n\nNotes\n\nAll unspecified kwargs are passed as replacements such that {{key}}=>value in the template.\nIf a SystemMessage is missing, we inject a default one at the beginning of the conversation.\nOnly one SystemMessage is allowed (ie, cannot mix two conversations different system prompts).\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.replace_words-Tuple{AbstractString, Vector{<:AbstractString}}","page":"PromptingTools.jl","title":"PromptingTools.replace_words","text":"replace_words(text::AbstractString, words::Vector{<:AbstractString}; replacement::AbstractString=\"ABC\")\n\nReplace all occurrences of words in words with replacement in text. Useful to quickly remove specific names or entities from a text.\n\nArguments\n\ntext::AbstractString: The text to be processed.\nwords::Vector{<:AbstractString}: A vector of words to be replaced.\nreplacement::AbstractString=\"ABC\": The replacement string to be used. Defaults to \"ABC\".\n\nExample\n\ntext = \"Disney is a great company\"\nreplace_words(text, [\"Disney\", \"Snow White\", \"Mickey Mouse\"])\n# Output: \"ABC is a great company\"\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.resize_conversation!-Tuple{Any, Union{Nothing, Int64}}","page":"PromptingTools.jl","title":"PromptingTools.resize_conversation!","text":"resize_conversation!(conv_history, max_history::Union{Int, Nothing})\n\nResize the conversation history to a specified maximum length.\n\nThis function trims the conv_history to ensure that its size does not exceed max_history. It removes the oldest conversations first if the length of conv_history is greater than max_history.\n\nArguments\n\nconv_history: A vector that stores the history of conversations. Typically, this is PT.CONV_HISTORY.\nmax_history: The maximum number of conversations to retain in the history. If Nothing, the history is not resized.\n\nReturns\n\nThe resized conversation history.\n\nExample\n\nresize_conversation!(PT.CONV_HISTORY, PT.MAX_HISTORY_LENGTH)\n\nAfter the function call, conv_history will contain only the 10 most recent conversations.\n\nThis is done automatically by the ai\"\" macros.\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.response_to_message-Tuple{PromptingTools.AbstractOpenAISchema, Type{AIMessage}, Any, Any}","page":"PromptingTools.jl","title":"PromptingTools.response_to_message","text":"response_to_message(schema::AbstractOpenAISchema,\n MSG::Type{AIMessage},\n choice,\n resp;\n model_id::AbstractString = \"\",\n time::Float64 = 0.0,\n run_id::Integer = rand(Int16),\n sample_id::Union{Nothing, Integer} = nothing)\n\nUtility to facilitate unwrapping of HTTP response to a message type MSG provided for OpenAI-like responses\n\nNote: Extracts finish_reason and log_prob if available in the response.\n\nArguments\n\nschema::AbstractOpenAISchema: The schema for the prompt.\nMSG::Type{AIMessage}: The message type to be returned.\nchoice: The choice from the response (eg, one of the completions).\nresp: The response from the OpenAI API.\nmodel_id::AbstractString: The model ID to use for generating the response. Defaults to an empty string.\ntime::Float64: The elapsed time for the response. Defaults to 0.0.\nrun_id::Integer: The run ID for the response. Defaults to a random integer.\nsample_id::Union{Nothing, Integer}: The sample ID for the response (if there are multiple completions). Defaults to nothing.\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.response_to_message-Union{Tuple{T}, Tuple{PromptingTools.AbstractPromptSchema, Type{T}, Any, Any}} where T","page":"PromptingTools.jl","title":"PromptingTools.response_to_message","text":"Utility to facilitate unwrapping of HTTP response to a message type MSG provided. Designed to handle multi-sample completions.\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.save_conversation-Tuple{Union{AbstractString, IO}, AbstractVector{<:PromptingTools.AbstractMessage}}","page":"PromptingTools.jl","title":"PromptingTools.save_conversation","text":"Saves provided conversation (messages) to io_or_file. If you need to add some metadata, see save_template.\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.save_template-Tuple{Union{AbstractString, IO}, AbstractVector{<:PromptingTools.AbstractChatMessage}}","page":"PromptingTools.jl","title":"PromptingTools.save_template","text":"Saves provided messaging template (messages) to io_or_file. Automatically adds metadata based on provided keyword arguments.\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.set_preferences!-Tuple{Vararg{Pair{String}}}","page":"PromptingTools.jl","title":"PromptingTools.set_preferences!","text":"set_preferences!(pairs::Pair{String, <:Any}...)\n\nSet preferences for PromptingTools. See ?PREFERENCES for more information. \n\nSee also: get_preferences\n\nExample\n\nChange your API key and default model:\n\nPromptingTools.set_preferences!(\"OPENAI_API_KEY\" => \"key1\", \"MODEL_CHAT\" => \"chat1\")\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.split_by_length-Tuple{Any, Vector{String}}","page":"PromptingTools.jl","title":"PromptingTools.split_by_length","text":"split_by_length(text::String, separators::Vector{String}; max_length::Int=35000) -> Vector{String}\n\nSplit a given string text into chunks using a series of separators, with each chunk having a maximum length of max_length. This function is useful for splitting large documents or texts into smaller segments that are more manageable for processing, particularly for models or systems with limited context windows.\n\nArguments\n\ntext::String: The text to be split.\nseparators::Vector{String}: An ordered list of separators used to split the text. The function iteratively applies these separators to split the text.\nmax_length::Int=35000: The maximum length of each chunk. Defaults to 35,000 characters. This length is considered after each iteration of splitting, ensuring chunks fit within specified constraints.\n\nReturns\n\nVector{String}: A vector of strings, where each string is a chunk of the original text that is smaller than or equal to max_length.\n\nNotes\n\nThe function processes the text iteratively with each separator in the provided order. This ensures more nuanced splitting, especially in structured texts.\nEach chunk is as close to max_length as possible without exceeding it (unless we cannot split it any further)\nIf the text is empty, the function returns an empty array.\nSeparators are re-added to the text chunks after splitting, preserving the original structure of the text as closely as possible. Apply strip if you do not need them.\n\nExamples\n\nSplitting text using multiple separators:\n\ntext = \"Paragraph 1\n\nParagraph 2. Sentence 1. Sentence 2.\nParagraph 3\"\nseparators = [\"\n\n\", \". \", \"\n\"]\nchunks = split_by_length(text, separators, max_length=20)\n\nUsing a single separator:\n\ntext = \"Hello,World,\" ^ 2900 # length 34900 characters\nchunks = split_by_length(text, [\",\"], max_length=10000)\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.split_by_length-Tuple{String}","page":"PromptingTools.jl","title":"PromptingTools.split_by_length","text":"split_by_length(text::String; separator::String=\" \", max_length::Int=35000) -> Vector{String}\n\nSplit a given string text into chunks of a specified maximum length max_length. This is particularly useful for splitting larger documents or texts into smaller segments, suitable for models or systems with smaller context windows.\n\nArguments\n\ntext::String: The text to be split.\nseparator::String=\" \": The separator used to split the text into minichunks. Defaults to a space character.\nmax_length::Int=35000: The maximum length of each chunk. Defaults to 35,000 characters, which should fit within 16K context window.\n\nReturns\n\nVector{String}: A vector of strings, each representing a chunk of the original text that is smaller than or equal to max_length.\n\nNotes\n\nThe function ensures that each chunk is as close to max_length as possible without exceeding it.\nIf the text is empty, the function returns an empty array.\nThe separator is re-added to the text chunks after splitting, preserving the original structure of the text as closely as possible.\n\nExamples\n\nSplitting text with the default separator (\" \"):\n\ntext = \"Hello world. How are you?\"\nchunks = split_by_length(text; max_length=13)\nlength(chunks) # Output: 2\n\nUsing a custom separator and custom max_length\n\ntext = \"Hello,World,\" ^ 2900 # length 34900 chars\nsplit_by_length(text; separator=\",\", max_length=10000) # for 4K context window\nlength(chunks[1]) # Output: 4\n\n\n\n\n\n","category":"method"},{"location":"reference/#PromptingTools.@aai_str-Tuple{Any, Vararg{Any}}","page":"PromptingTools.jl","title":"PromptingTools.@aai_str","text":"aai\"user_prompt\"[model_alias] -> AIMessage\n\nAsynchronous version of @ai_str macro, which will log the result once it's ready.\n\nSee also aai!\"\" if you want an asynchronous reply to the provided message / continue the conversation. \n\nExample\n\nSend asynchronous request to GPT-4, so we don't have to wait for the response: Very practical with slow models, so you can keep working in the meantime.\n\n```julia m = aai\"Say Hi!\"gpt4; \n\n...with some delay...\n\n[ Info: Tokens: 29 @ Cost: 0.0011 in 2.7 seconds\n\n[ Info: AIMessage> Hello! How can I assist you today?\n\n\n\n\n\n","category":"macro"},{"location":"reference/#PromptingTools.@ai!_str-Tuple{Any, Vararg{Any}}","page":"PromptingTools.jl","title":"PromptingTools.@ai!_str","text":"ai!\"user_prompt\"[model_alias] -> AIMessage\n\nThe ai!\"\" string macro is used to continue a previous conversation with the AI model. \n\nIt appends the new user prompt to the last conversation in the tracked history (in PromptingTools.CONV_HISTORY) and generates a response based on the entire conversation context. If you want to see the previous conversation, you can access it via PromptingTools.CONV_HISTORY, which keeps at most last PromptingTools.MAX_HISTORY_LENGTH conversations.\n\nArguments\n\nuser_prompt (String): The new input prompt to be added to the existing conversation.\nmodel_alias (optional, any): Specify the model alias of the AI model to be used (see MODEL_ALIASES). If not provided, the default model is used.\n\nReturns\n\nAIMessage corresponding to the new user prompt, considering the entire conversation history.\n\nExample\n\nTo continue a conversation:\n\n# start conversation as normal\nai\"Say hi.\" \n\n# ... wait for reply and then react to it:\n\n# continue the conversation (notice that you can change the model, eg, to more powerful one for better answer)\nai!\"What do you think about that?\"gpt4t\n# AIMessage(\"Considering our previous discussion, I think that...\")\n\nUsage Notes\n\nThis macro should be used when you want to maintain the context of an ongoing conversation (ie, the last ai\"\" message).\nIt automatically accesses and updates the global conversation history.\nIf no conversation history is found, it raises an assertion error, suggesting to initiate a new conversation using ai\"\" instead.\n\nImportant\n\nEnsure that the conversation history is not too long to maintain relevancy and coherence in the AI's responses. The history length is managed by MAX_HISTORY_LENGTH.\n\n\n\n\n\n","category":"macro"},{"location":"reference/#PromptingTools.@ai_str-Tuple{Any, Vararg{Any}}","page":"PromptingTools.jl","title":"PromptingTools.@ai_str","text":"ai\"user_prompt\"[model_alias] -> AIMessage\n\nThe ai\"\" string macro generates an AI response to a given prompt by using aigenerate under the hood.\n\nSee also ai!\"\" if you want to reply to the provided message / continue the conversation.\n\nArguments\n\nuser_prompt (String): The input prompt for the AI model.\nmodel_alias (optional, any): Provide model alias of the AI model (see MODEL_ALIASES).\n\nReturns\n\nAIMessage corresponding to the input prompt.\n\nExample\n\nresult = ai\"Hello, how are you?\"\n# AIMessage(\"Hello! I'm an AI assistant, so I don't have feelings, but I'm here to help you. How can I assist you today?\")\n\nIf you want to interpolate some variables or additional context, simply use string interpolation:\n\na=1\nresult = ai\"What is `$a+$a`?\"\n# AIMessage(\"The sum of `1+1` is `2`.\")\n\nIf you want to use a different model, eg, GPT-4, you can provide its alias as a flag:\n\nresult = ai\"What is `1.23 * 100 + 1`?\"gpt4t\n# AIMessage(\"The answer is 124.\")\n\n\n\n\n\n","category":"macro"},{"location":"reference/#PromptingTools.@timeout-Tuple{Any, Any, Any}","page":"PromptingTools.jl","title":"PromptingTools.@timeout","text":"@timeout(seconds, expr_to_run, expr_when_fails)\n\nSimple macro to run an expression with a timeout of seconds. If the expr_to_run fails to finish in seconds seconds, expr_when_fails is returned.\n\nExample\n\nx = @timeout 1 begin\n sleep(1.1)\n println(\"done\")\n 1\nend \"failed\"\n\n\n\n\n\n\n","category":"macro"},{"location":"how_it_works/","page":"How It Works","title":"How It Works","text":"CurrentModule = PromptingTools","category":"page"},{"location":"how_it_works/#How-It-Works","page":"How It Works","title":"How It Works","text":"","category":"section"},{"location":"how_it_works/","page":"How It Works","title":"How It Works","text":"This is an advanced section that explains how PromptingTools.jl works under the hood. It is not necessary to understand this to use the package, but it can be helpful for debugging and understanding the limitations of the package.","category":"page"},{"location":"how_it_works/","page":"How It Works","title":"How It Works","text":"We'll start with the key concepts and then walk through an example of aigenerate to see how it all fits together.","category":"page"},{"location":"how_it_works/#Key-Concepts","page":"How It Works","title":"Key Concepts","text":"","category":"section"},{"location":"how_it_works/","page":"How It Works","title":"How It Works","text":"5 Key Concepts (/Objects):","category":"page"},{"location":"how_it_works/","page":"How It Works","title":"How It Works","text":"API/Model Providers -> The method that gives you access to Large Language Models (LLM), it can be an API (eg, OpenAI) or a locally-hosted application (eg, Llama.cpp or Ollama)\nSchemas -> object of type AbstractPromptSchema that determines which methods are called and, hence, what providers/APIs are used\nPrompts -> the information you want to convey to the AI model\nMessages -> the basic unit of communication between the user and the AI model (eg, UserMessage vs AIMessage)\nPrompt Templates -> re-usable \"prompts\" with placeholders that you can replace with your inputs at the time of making the request","category":"page"},{"location":"how_it_works/","page":"How It Works","title":"How It Works","text":"When you call aigenerate, roughly the following happens: render -> UserMessage(s) -> render -> OpenAI.create_chat -> ... -> AIMessage.","category":"page"},{"location":"how_it_works/#API/Model-Providers","page":"How It Works","title":"API/Model Providers","text":"","category":"section"},{"location":"how_it_works/","page":"How It Works","title":"How It Works","text":"You can think of \"API/Model Providers\" as the method that gives you access to Large Language Models (LLM). It can be an API (eg, OpenAI) or a locally-hosted application (eg, Llama.cpp or Ollama).","category":"page"},{"location":"how_it_works/","page":"How It Works","title":"How It Works","text":"You interact with them via the schema object, which is a subtype of AbstractPromptSchema, eg, there is an OpenAISchema for the provider \"OpenAI\" and its supertype AbstractOpenAISchema is for all other providers that mimic the OpenAI API.","category":"page"},{"location":"how_it_works/#Schemas","page":"How It Works","title":"Schemas","text":"","category":"section"},{"location":"how_it_works/","page":"How It Works","title":"How It Works","text":"For your \"message\" to reach an AI model, it needs to be formatted and sent to the right place (-> provider!).","category":"page"},{"location":"how_it_works/","page":"How It Works","title":"How It Works","text":"We leverage the multiple dispatch around the \"schemas\" to pick the right logic. All schemas are subtypes of AbstractPromptSchema and there are many subtypes, eg, OpenAISchema <: AbstractOpenAISchema <:AbstractPromptSchema.","category":"page"},{"location":"how_it_works/","page":"How It Works","title":"How It Works","text":"For example, if you provide schema = OpenAISchema(), the system knows that:","category":"page"},{"location":"how_it_works/","page":"How It Works","title":"How It Works","text":"it will have to format any user inputs to OpenAI's \"message specification\" (a vector of dictionaries, see their API documentation). Function render(OpenAISchema(),...) will take care of the rendering.\nit will have to send the message to OpenAI's API. We will use the amazing OpenAI.jl package to handle the communication.","category":"page"},{"location":"how_it_works/#Prompts","page":"How It Works","title":"Prompts","text":"","category":"section"},{"location":"how_it_works/","page":"How It Works","title":"How It Works","text":"Prompt is loosely the information you want to convey to the AI model. It can be a question, a statement, or a command. It can have instructions or some context, eg, previous conversation.","category":"page"},{"location":"how_it_works/","page":"How It Works","title":"How It Works","text":"You need to remember that Large Language Models (LLMs) are stateless. They don't remember the previous conversation/request, so you need to provide the whole history/context every time (similar to how REST APIs work).","category":"page"},{"location":"how_it_works/","page":"How It Works","title":"How It Works","text":"Prompts that we send to the LLMs are effectively a sequence of messages (<:AbstractMessage).","category":"page"},{"location":"how_it_works/#Messages","page":"How It Works","title":"Messages","text":"","category":"section"},{"location":"how_it_works/","page":"How It Works","title":"How It Works","text":"Messages are the basic unit of communication between the user and the AI model. ","category":"page"},{"location":"how_it_works/","page":"How It Works","title":"How It Works","text":"There are 5 main types of messages (<:AbstractMessage):","category":"page"},{"location":"how_it_works/","page":"How It Works","title":"How It Works","text":"SystemMessage - this contains information about the \"system\", eg, how it should behave, format its output, etc. (eg, `You're a world-class Julia programmer. You write brief and concise code.)\nUserMessage - the information \"from the user\", ie, your question/statement/task\nUserMessageWithImages - the same as UserMessage, but with images (URLs or Base64-encoded images)\nAIMessage - the response from the AI model, when the \"output\" is text\nDataMessage - the response from the AI model, when the \"output\" is data, eg, embeddings with aiembed or user-defined structs with aiextract","category":"page"},{"location":"how_it_works/#Prompt-Templates","page":"How It Works","title":"Prompt Templates","text":"","category":"section"},{"location":"how_it_works/","page":"How It Works","title":"How It Works","text":"We want to have re-usable \"prompts\", so we provide you with a system to retrieve pre-defined prompts with placeholders (eg, {{name}}) that you can replace with your inputs at the time of making the request.","category":"page"},{"location":"how_it_works/","page":"How It Works","title":"How It Works","text":"\"AI Templates\" as we call them (AITemplate) are usually a vector of SystemMessage and a UserMessage with specific purpose/task.","category":"page"},{"location":"how_it_works/","page":"How It Works","title":"How It Works","text":"For example, the template :AssistantAsk is defined loosely as:","category":"page"},{"location":"how_it_works/","page":"How It Works","title":"How It Works","text":" template = [SystemMessage(\"You are a world-class AI assistant. Your communication is brief and concise. You're precise and answer only when you're confident in the high quality of your answer.\"),\n UserMessage(\"# Question\\n\\n{{ask}}\")]","category":"page"},{"location":"how_it_works/","page":"How It Works","title":"How It Works","text":"Notice that we have a placeholder ask ({{ask}}) that you can replace with your question without having to re-write the generic system instructions.","category":"page"},{"location":"how_it_works/","page":"How It Works","title":"How It Works","text":"When you provide a Symbol (eg, :AssistantAsk) to ai* functions, thanks to the multiple dispatch, it recognizes that it's an AITemplate(:AssistantAsk) and looks it up.","category":"page"},{"location":"how_it_works/","page":"How It Works","title":"How It Works","text":"You can discover all available templates with aitemplates(\"some keyword\") or just see the details of some template aitemplates(:AssistantAsk).","category":"page"},{"location":"how_it_works/","page":"How It Works","title":"How It Works","text":"Note: There is a new way to create and register templates in one go with create_template(;user=, system=, load_as=