Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Template Functionality #2

Merged
merged 4 commits into from
Nov 16, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,5 @@
/docs/Manifest.toml
/docs/build/

/.DS_Store # macOS folder metadata
/.vscode
**/.DS_Store
**/.vscode
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Changelog
All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]
### Added
- Add support for prompt templates with `AITemplate` struct. Search for suitable templates with `aitemplates("query string")` and then simply use them with `aigenerate(AITemplate(:TemplateABC); variableX = "some value") -> AIMessage` or use a dispatch on the template name as a `Symbol`, eg, `aigenerate(:TemplateABC; variableX = "some value") -> AIMessage`. Templates are saved as JSON files in the folder `templates/`. If you add new templates, you can reload them with `load_templates!()` (notice the exclamation mark to override the existing `TEMPLATE_STORE`).
4 changes: 4 additions & 0 deletions Project.toml
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,15 @@ version = "0.2.0-DEV"
HTTP = "cd3eb016-35fb-5094-929b-558a96fad6f3"
JSON3 = "0f8b85d8-7281-11e9-16c2-39a750bddbf1"
OpenAI = "e9f21f70-7185-4079-aca2-91159181367c"
PrecompileTools = "aea7be01-6a6a-4083-8856-8a6e6704d82a"

[compat]
Aqua = "0.7"
HTTP = "1"
JSON3 = "1"
OpenAI = "0.8.7"
PrecompileTools = "1"
Test = "<0.0.1, 1"
julia = "1.9,1.10"

[extras]
Expand Down
94 changes: 87 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ ai"What is the capital of \$(country)?"
# AIMessage("The capital of Spain is Madrid.")
```

Pro tip: Use after-string-flags to select the model to be called, eg, `ai"What is the capital of France?"gpt4`. Great for those extra hard questions!
Pro tip: Use after-string-flags to select the model to be called, eg, `ai"What is the capital of France?"gpt4` (use `gpt4t` for the new GPT-4 Turbo model). Great for those extra hard questions!

For more complex prompt templates, you can use handlebars-style templating and provide variables as keyword arguments:

Expand All @@ -50,14 +50,18 @@ Pro tip: Use `asyncmap` to run multiple AI-powered tasks concurrently.

Pro tip: If you use slow models (like GPT-4), you can use async version of `@ai_str` -> `@aai_str` to avoid blocking the REPL, eg, `aai"Say hi but slowly!"gpt4`

For more practical examples, see the `examples/` folder and the [Advanced Examples](#advanced-examples) section below.

## Table of Contents

- [PromptingTools.jl: "Your Daily Dose of AI Efficiency."](#promptingtoolsjl-your-daily-dose-of-ai-efficiency)
- [Quick Start with `@ai_str` and Easy Templating](#quick-start-with-ai_str-and-easy-templating)
- [Table of Contents](#table-of-contents)
- [Why PromptingTools.jl](#why-promptingtoolsjl)
- [Advanced Examples](#advanced-examples)
- [Seamless Integration Into Your Workflow](#seamless-integration-into-your-workflow)
- [Advanced Prompts / Conversations](#advanced-prompts--conversations)
- [Templated Prompts](#templated-prompts)
- [Asynchronous Execution](#asynchronous-execution)
- [Model Aliases](#model-aliases)
- [Embeddings](#embeddings)
Expand Down Expand Up @@ -90,11 +94,32 @@ Some features:

## Advanced Examples

TODO:
TODOs:

- [ ] Add more practical examples (with DataFrames!)
- [ ] Add mini tasks with structured extraction
- [ ] Add an example of how to build a RAG app in 50 lines

### Seamless Integration Into Your Workflow
Google search is great, but it's a context switch. You often have to open a few pages and read through the discussion to find the answer you need. Same with the ChatGPT website.

Imagine you are in VSCode, editing your `.gitignore` file. How do I ignore a file in all subfolders again?

[ ] Add more practical examples (DataFrames!)
[ ] Add mini tasks with structured extraction
[ ] Add an example of how to build a RAG app in 50 lines
All you need to do is to type:
`aai"What to write in .gitignore to ignore file XYZ in any folder or subfolder?"`

With `aai""` (as opposed to `ai""`), we make a non-blocking call to the LLM to not prevent you from continuing your work. When the answer is ready, we log it from the background:

> [ Info: Tokens: 102 @ Cost: $0.0002 in 2.7 seconds
> ┌ Info: AIMessage> To ignore a file called "XYZ" in any folder or subfolder, you can add the following line to your .gitignore file:
> │
> │ ```
> │ **/XYZ
> │ ```
> │
> └ This pattern uses the double asterisk (`**`) to match any folder or subfolder, and then specifies the name of the file you want to ignore.

You probably saved 3-5 minutes on this task and probably another 5-10 minutes, because of the context switch/distraction you avoided. It's a small win, but it adds up quickly.

### Advanced Prompts / Conversations

Expand Down Expand Up @@ -126,6 +151,59 @@ aigenerate(new_conversation; object = "old iPhone")
```
> AIMessage("Hmm, possess an old iPhone, I do not. But experience with attachments, I have. Detachment, I learned. True power and freedom, it brings...")

### Templated Prompts

With LLMs, the quality / robustness of your results depends on the quality of your prompts. But writing prompts is hard! That's why we offer a templating system to save you time and effort.

To use a specific template (eg, `` to ask a Julia language):
```julia
msg = aigenerate(:JuliaExpertAsk; ask = "How do I add packages?")
```

The above is equivalent to a more verbose version that explicitly uses the dispatch on `AITemplate`:
```julia
msg = aigenerate(AITemplate(:JuliaExpertAsk); ask = "How do I add packages?")
```

Find available templates with `aitemplates`:
```julia
tmps = aitemplates("JuliaExpertAsk")
# Will surface one specific template
# 1-element Vector{AITemplateMetadata}:
# PromptingTools.AITemplateMetadata
# name: Symbol JuliaExpertAsk
# description: String "For asking questions about Julia language. Placeholders: `ask`"
# version: String "1"
# wordcount: Int64 237
# variables: Array{Symbol}((1,))
# system_preview: String "You are a world-class Julia language programmer with the knowledge of the latest syntax. Your commun"
# user_preview: String "# Question\n\n{{ask}}"
# source: String ""
```
The above gives you a good idea of what the template is about, what placeholders are available, and how much it would cost to use it (=wordcount).

Search for all Julia-related templates:
```julia
tmps = aitemplates("Julia")
# 2-element Vector{AITemplateMetadata}... -> more to come later!
```

If you are on VSCode, you can leverage nice tabular display with `vscodedisplay`:
```julia
using DataFrames
tmps = aitemplates("Julia") |> DataFrame |> vscodedisplay
```

I have my selected template, how do I use it? Just use the "name" in `aigenerate` or `aiclassify`
like you see in the first example!

You can inspect any template by "rendering" it (this is what the LLM will see):
```julia
julia> AITemplate(:JudgeIsItTrue) |> PromptingTools.render
```

See more examples in the [examples/](examples/) folder.

### Asynchronous Execution

You can leverage `asyncmap` to run multiple AI-powered tasks concurrently, improving performance for batch operations.
Expand Down Expand Up @@ -183,14 +261,16 @@ aiclassify("Is two plus two four?")
System prompts and higher-quality models can be used for more complex tasks, including knowing when to defer to a human:

```julia
aiclassify(:IsStatementTrue; statement = "Is two plus three a vegetable on Mars?", model = "gpt4")
aiclassify(:JudgeIsItTrue; it = "Is two plus three a vegetable on Mars?", model = "gpt4t")
# unknown
```

In the above example, we used a prompt template `:IsStatementTrue`, which automatically expands into the following system prompt (and a separate user prompt):
In the above example, we used a prompt template `:JudgeIsItTrue`, which automatically expands into the following system prompt (and a separate user prompt):

> "You are an impartial AI judge evaluating whether the provided statement is \"true\" or \"false\". Answer \"unknown\" if you cannot decide."

For more information on templates, see the [Templated Prompts](#templated-prompts) section.

### Data Extraction

!!! Experimental
Expand Down
73 changes: 73 additions & 0 deletions examples/working_with_aitemplates.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# This file contains examples of how to work with AITemplate(s).

using PromptingTools
const PT = PromptingTools

# LLM responses are only as good as the prompts you give them. However, great prompts take long time to write -- AITemplate are a way to re-use great prompts!
#
# AITemplates are just a collection of templated prompts (ie, set of "messages" that have placeholders like {{question}})
#
# They are saved as JSON files in the `templates` directory.
# They are automatically loaded on package import, but you can always force a re-load with `PT.load_templates!()`
PT.load_templates!();

# You can (create them) and use them for any ai* function instead of a prompt:
# Let's use a template called :JuliaExpertAsk
# alternatively, you can use `AITemplate(:JuliaExpertAsk)` for cleaner dispatch
msg = aigenerate(:JuliaExpertAsk; ask = "How do I add packages?")
# ... some response from GPT3.5
#
# You can see that it had a placeholder for the actual question (`ask`) that we provided as a keyword argument.
# We did not have to write any system prompt for personas, tone, etc. -- it was all provided by the template!
#
# How to know which templates are available? You can search for them with `aitemplates()`:
# You can search by Symbol (only for partial name match), String (partial match on name or description), or Regex (more fields)
tmps = aitemplates("JuliaExpertAsk")
# Outputs a list of available templates that match the search -- there is just one in this case:
#
# 1-element Vector{AITemplateMetadata}:
# PromptingTools.AITemplateMetadata
# name: Symbol JuliaExpertAsk
# description: String "For asking questions about Julia language. Placeholders: `ask`"
# version: String "1"
# wordcount: Int64 237
# variables: Array{Symbol}((1,))
# system_preview: String "You are a world-class Julia language programmer with the knowledge of the latest syntax. Your commun"
# user_preview: String "# Question\n\n{{ask}}"
# source: String ""
#
# You see not just the description, but also a preview of the actual prompts, placeholders available, and the length (to gauge how much it would cost).
#
# If you use VSCode, you can display them in a nice scrollable table with `vscodedisplay`:
using DataFrames
DataFrame(tmp) |> vscodedisplay
#
#
# You can also just `render` the template to see the underlying mesages:
msgs = PT.render(AITemplate(:JuliaExpertAsk))
#
# 2-element Vector{PromptingTools.AbstractChatMessage}:
# PromptingTools.SystemMessage("You are a world-class Julia language programmer with the knowledge of the latest syntax. Your communication is brief and concise. You're precise and answer only when you're confident in the high quality of your answer.")
# PromptingTools.UserMessage("# Question\n\n{{ask}}")
#
# Now, you know exactly what's in the template!
#
# If you want to modify it, simply change it and save it as a new file with `save_template` (see the docs `?save_template` for more details).
#
# Let's adjust the previous template to be more specific to a data analysis question:
tpl = [PT.SystemMessage("You are a world-class Julia language programmer with the knowledge of the latest syntax. You're also a senior Data Scientist and proficient in data analysis in Julia. Your communication is brief and concise. You're precise and answer only when you're confident in the high quality of your answer.")
PT.UserMessage("# Question\n\n{{ask}}")]
# Templates are saved in the `templates` directory of the package. Name of the file will become the template name (eg, call `:JuliaDataExpertAsk`)
filename = joinpath(pkgdir(PromptingTools),
"templates",
"persona-task",
"JuliaDataExpertAsk.json")
PT.save_template(filename,
tpl;
description = "For asking data analysis questions in Julia language. Placeholders: `ask`")
rm(filename) # cleanup if we don't like it
#
# When you create a new template, remember to re-load the templates with `load_templates!()` so that it's available for use.
PT.load_templates!();
#
# !!! If you have some good templates (or suggestions for the existing ones), please consider sharing them with the community by opening a PR to the `templates` directory!
18 changes: 17 additions & 1 deletion src/PromptingTools.jl
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,9 @@ module PromptingTools

using OpenAI
using JSON3
using JSON3: StructTypes
using HTTP
using PrecompileTools

# GLOBALS
const MODEL_CHAT = "gpt-3.5-turbo"
Expand All @@ -20,7 +22,7 @@ const MODEL_ALIASES = Dict("gpt3" => "gpt-3.5-turbo",
"gpt4t" => "gpt-4-1106-preview", # 4t is for "4 turbo"
"gpt3t" => "gpt-3.5-turbo-1106", # 3t is for "3 turbo"
"ada" => "text-embedding-ada-002")
# below is defined in llm_interace.jl !
# the below default is defined in llm_interace.jl !
# const PROMPT_SCHEMA = OpenAISchema()

include("utils.jl")
Expand All @@ -34,11 +36,25 @@ export AIMessage
# export UserMessage, SystemMessage, DataMessage # for debugging only
include("messages.jl")

export aitemplates, AITemplate
include("templates.jl")

const TEMPLATE_STORE = Dict{Symbol, Any}()
const TEMPLATE_METADATA = Vector{AITemplateMetadata}()

## Individual interfaces
include("llm_openai.jl")

## Convenience utils
export @ai_str, @aai_str
include("macros.jl")

function __init__()
# Load templates
load_templates!()
end

# Enable precompilation to reduce start time
@compile_workload include("precompilation.jl")

end # module PromptingTools
20 changes: 17 additions & 3 deletions src/llm_interface.jl
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,9 @@ end

abstract type AbstractChatMLSchema <: AbstractPromptSchema end
"""
ChatMLSchema is used by many open-source chatbots, by OpenAI models under the hood and by several models and inferfaces (eg, Ollama, vLLM)
ChatMLSchema is used by many open-source chatbots, by OpenAI models (under the hood) and by several models and inferfaces (eg, Ollama, vLLM)

You can explore it on [tiktokenizer](https://tiktokenizer.vercel.app/)

It uses the following conversation structure:
```
Expand All @@ -54,11 +56,23 @@ It uses the following conversation structure:
"""
struct ChatMLSchema <: AbstractChatMLSchema end

## Dispatch into defaults
abstract type AbstractManagedSchema <: AbstractPromptSchema end

"""
Ollama by default manages different models and their associated prompt schemas when you pass `system_prompt` and `prompt` fields to the API.

Warning: It works only for 1 system message and 1 user message, so anything more than that has to be rejected.

If you need to pass more messagese / longer conversational history, you can use define the model-specific schema directly and pass your Ollama requests with `raw=true`,
which disables and templating and schema management by Ollama.
"""
struct OllamaManagedSchema <: AbstractManagedSchema end

## Dispatch into default schema
const PROMPT_SCHEMA = OpenAISchema()

aigenerate(prompt; kwargs...) = aigenerate(PROMPT_SCHEMA, prompt; kwargs...)
function aiembed(doc_or_docs, args...; kwargs...)
aiembed(PROMPT_SCHEMA, doc_or_docs, args...; kwargs...)
end
aiclassify(prompt; kwargs...) = aiclassify(PROMPT_SCHEMA, prompt; kwargs...)
aiclassify(prompt; kwargs...) = aiclassify(PROMPT_SCHEMA, prompt; kwargs...)
Loading
Loading