aici support (#269)

* add string template node * string template node * removed error var * add aici rendering * add AICI to api * add cli tracing * note about escaping * better formatting * percolate aici * more plumbing * weave aici into openai call * capitalize * more plumbing * more wiring * wiring of chat messages * make key format more generic * retreive aici token * key mess * more plumbing * fix tokens * add version to token * parse aici keys * pickup secret per template * going further * move logging * tracing generated aici * better function names * don't trace aici * update call * convert more messages * support for system prompts using aici * log url * rename url to base in token * fixed codegen issue * encoding regexes * fixes for azure connection * more key parsing * fixed azure key * more docs * better codegen to avoid backgracking * expose AICI.gen({ storeVar: ... }) as fences * fix typo * example of substring constraint * add `genVars` to output * support for runPrompt * merge issues * updated d.ts --------- Co-authored-by: Michal Moskal <[email protected]>
microsoft · Mar 21, 2024 · 469ab4d · 469ab4d
1 parent 56f49e4
commit 469ab4d
Show file tree

Hide file tree

Showing 46 changed files with 2,493 additions and 346 deletions.
diff --git a/README.md b/README.md
@@ -2,7 +2,7 @@
 
 # GenAIScript
 
-Scripting environment with convinient tooling for file ingestion, prompt development and structured data extraction.
+Scripting environment with convenient tooling for file ingestion, prompt development and structured data extraction.
 
 ```js wrap
 // metadata and model configuration

diff --git a/docs/src/content/docs/reference/cli/commands.md b/docs/src/content/docs/reference/cli/commands.md
@@ -81,17 +81,6 @@ Options:
   -h, --help                         display help for command
 ```
 
-## `keys`
-
-```
-Usage: genaiscript keys [options]
-
-Parse and show current key information
-
-Options:
-  -h, --help  display help for command
-```
-
 ## `scripts`
 
 ```

diff --git a/docs/src/content/docs/reference/scripts/aici.md b/docs/src/content/docs/reference/scripts/aici.md
@@ -0,0 +1,73 @@
+---
+title: AICI
+sidebar:
+    order: 200
+---
+
+[Microsoft AICI](https://github.com/microsoft/aici/) allows to constrain the output of a LLM using WASM. In particular, it is possible to send JavaScript program to describe the prompt.
+
+GenAIScript support executing scripts and converting the output into a AICI compatible JavaScript program, which will them generate constrainted output.
+
+:::caution
+
+This feature is experimental and may change in the future.
+
+:::
+
+Let's take a look at an example.
+
+```js title="answer-to-everything.genai.js"
+$`Ultimate answer is to the life, universe 
+and everything is ${AICI.gen({ regex: /\d\d/ })}`
+```
+
+The execution of this script is converted into a AICI JavaScript program.
+
+```js title="answer-to-everything.aici.js"
+async function aiciregex() {
+  await $`Ultimate answer is to the life, universe and everything is `
+  await gen({regex: /\d\d/})
+}
+
+async function main() {
+  await aiciregex()
+}
+start(main)
+```
+
+And AICI comes back with the following log.
+
+```txt
+FIXED "Ultimate answer is to the life, universe and everything is "
+GEN-OPT {regex: /\d\d/}
+regex constraint: "\\d\\d"
+dfa: 160 bytes
+
+GEN "42"
+JsCtrl: done
+```
+
+And the text output is `42`.
+
+
+## Metadata
+
+An AICI template should set the `aici: true` field in the `script` function.
+
+```js title="answer-to-everything.genai.js"
+script({ ...
+    aici: true,
+})
+```
+
+## `gen`
+
+The `AICI.gen` function creates a constrain in the prompt flow.
+
+## Token
+
+AICI uses `AICI_API_KEY`, `AICI_API_BASE` and `AICI_API_VERSION` (default `v1`) to compose the API URL.
+
+```
+<base>/<model>/<version>/run
+```
diff --git a/docs/src/content/docs/reference/scripts/custom-output.md b/docs/src/content/docs/reference/scripts/custom-output.md
@@ -6,6 +6,13 @@ sidebar:
 
 The `defOutput` function registers a callback to do custom processing of the LLM output at the end of the generation process. This function allows to create new files or modify the existing ones.
 
+:::caution
+
+This feature is experimental and may change in the future.
+
+:::
+
+
 ```js
 // compute a filepath
 const output = path.join(path.dirname(env.spec), "output.txt")
@@ -20,7 +27,7 @@ defOutput(output => {
 })
 ```
 
-## Clearning generated files
+## Cleaning generated files
 
 This example clears the `fileEdits` object, which contains the parsed file updates.
 

diff --git a/packages/cli/src/main.ts b/packages/cli/src/main.ts
@@ -231,10 +231,6 @@ async function batch(
         `tool: ${script.id} (${script.title}), files: ${specFiles.size}, out: ${resolve(out)}`
     )
 
-    spinner.start(`validating token`)
-    const tok = await initToken() // ensure we have a token early
-    spinner.succeed(`LLM: ${tok.url}`)
-
     let errors = 0
     let totalTokens = 0
     if (removeOut) await emptyDir(out)
@@ -935,20 +931,6 @@ async function main() {
         )
         .action(batch)
 
-    program
-        .command("keys")
-        .description("Parse and show current key information")
-        .action(async () => {
-            const key = await host.getSecretToken()
-            console.log(
-                key
-                    ? `${
-                          key.isOpenAI ? "OpenAI" : key.url
-                      } (from ${key.source})`
-                    : "no key set"
-            )
-        })
-
     program
         .command("scripts")
         .description("List all available scripts in workspace")
@@ -961,7 +943,7 @@ async function main() {
         .argument("<file...>", "Files to index")
         .option("-ef, --excluded-files <string...>", "excluded files")
         .option("-n, --name <string>", "index name")
-//        .option("-s, --summary", "use LLM-generated summaries")
+        //        .option("-s, --summary", "use LLM-generated summaries")
         .option("-cs, --chunk-size <number>", "chunk size")
         .option("-co, --chunk-overlap <number>", "chunk overlap")
         .option("-m, --model <string>", "model for embeddings (default gpt-4)")
@@ -977,13 +959,13 @@ async function main() {
         .option("-ef, --excluded-files <string...>", "excluded files")
         .option("-tk, --top-k <number>", "maximum number of embeddings")
         .option("-n, --name <string>", "index name")
-//        .option("-s, --summary", "use LLM-generated summaries")
+        //        .option("-s, --summary", "use LLM-generated summaries")
         .action(retreivalSearch)
     retreival
         .command("clear")
         .description("Clear index to force re-indexing")
         .option("-n, --name <string>", "index name")
-//        .option("-s, --summary", "use LLM-generated summaries")
+        //        .option("-s, --summary", "use LLM-generated summaries")
         .action(retreivalClear)
 
     retreival

diff --git a/packages/cli/src/nodehost.ts b/packages/cli/src/nodehost.ts
@@ -58,8 +58,8 @@ export class NodeHost implements Host {
         return process.env[name]
     }
 
-    async getSecretToken(): Promise<OAIToken> {
-        return await parseTokenFromEnv(process.env)
+    async getSecretToken(template: ModelOptions): Promise<OAIToken> {
+        return await parseTokenFromEnv(process.env, template)
     }
 
     clearVirtualFiles(): void {

diff --git a/packages/core/prompt_type.d.ts b/packages/core/prompt_type.d.ts
@@ -93,6 +93,16 @@ declare var fs: FileSystem
  */
 declare var YAML: YAML
 
+/**
+ * INI parsing and stringifying.
+ */
+declare var INI: INI
+
+/**
+ * AICI operations
+ */
+declare var AICI: AICI
+
 /**
  * Fetches a given URL and returns the response.
  * @param url