Aiclassify arbitrary choices

[svilupp]

• Rewrite of aiclassify to classify into an arbitrary list of categories (including with descriptions). It's a quick and easy option for "routing" and similar use cases, as it exploits the logit bias trick and outputs only 1 token. Currently only OpenAISchema is supported. See ?aiclassify for more information.

Given a user input, pick one of the two provided categories:

choices = ["animal", "plant"]
input = "Palm tree"
aiclassify(:InputClassifier; choices, input)

Choices with descriptions provided as tuples:

choices = [("A", "any animal or creature"), ("P", "for any plant or tree"), ("O", "for everything else")]
input = "spider"
input = "daphodil"
input = "castle"
aiclassify(:InputClassifier; choices, input)

You can still use a simple true/false classification:

aiclassify("Is two plus two four?") # true
aiclassify("Is two plus three a vegetable on Mars?") # false

aiclassify returns only true/false/unknown. It's easy to get the proper Bool output type out with tryparse, eg,

tryparse(Bool, aiclassify("Is two plus two four?")) isa Bool # true

Output of type Nothing marks that the model couldn't classify the statement as true/false.

Ideally, we would like to re-use some helpful system prompt to get more accurate responses.
For this reason we have templates, eg, :JudgeIsItTrue. By specifying the template, we can provide our statement as the expected variable (it in this case)
See that the model now correctly classifies the statement as "unknown".

aiclassify(:JudgeIsItTrue; it = "Is two plus three a vegetable on Mars?") # unknown

For better results, use higher quality models like gpt4, eg,

aiclassify(:JudgeIsItTrue;
    it = "If I had two apples and I got three more, I have five apples now.",
    model = "gpt4") # true

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (9c667c8) 91.71% compared to head (3c7f356) 92.02%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #78      +/-   ##
==========================================
+ Coverage   91.71%   92.02%   +0.30%     
==========================================
  Files          36       36              
  Lines        1750     1818      +68     
==========================================
+ Hits         1605     1673      +68     
  Misses        145      145              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.