Fix artificial error (#738)

* artificial error

* Add GitHub Action guide, enhance filename filtering, and add spell checker script

* Fix grammar and punctuation in documentation and refine AI analysis instructions

* Add file chunk size option, replace unique with uniq from es-toolkit and update dependencies

* revert file chunks

* Update CLI docs, improve spell checker script, and modify type definitions

* Update spell checker script and documentation for clarity and changelog integration

* Update script to fetch staged Markdown files for spell-checking and refine documentation

* Fix syntax error by adding closing backtick in poem.genai.mts

docs/src/content/docs/reference/cli/commands.md

@@ -235,9 +235,9 @@ Options:
 
 ## `retrieval`
 
-```
 Usage: genaiscript retrieval|retreival [options] [command]
 
+RAG support
 RAG support
 
 Options:

docs/src/content/docs/samples/cmt.mdx

@@ -9,10 +9,10 @@ import { Code } from "@astrojs/starlight/components"
 import source from "../../../../../packages/vscode/genaisrc/cmt.genai.mts?raw"
 
 This sample automates adding comments to source code using an LLM
-and validate the changes haven't introduce any code modifications.
+and validates the changes haven't introduced any code modifications.
 
 To do so, we could use a combination of tools to validate the transformer: source formatters,
-compilers, linters or LLM-as-judge.
+compilers, linters, or LLM-as-judge.
 
 The algorithm could be summarized as follows:
 
@@ -45,8 +45,7 @@ if (files.length === 0)
 
 ### Processing Each File
 
-We process each file separately, to avoid exploding the token context and keep the AI focused.
-We can use [inline prompts](/genaiscript/reference/scripts/inline-prompts) to make inner queries.
+We process each file separately to avoid exploding the token context and keep the AI focused. We can use [inline prompts](/genaiscript/reference/scripts/inline-prompts) to make inner queries.
 
 ```ts
 for (const file of files) {
@@ -73,12 +72,11 @@ const res = await runPrompt(
 )
 ```
 
-We provide a detailed set of instructions to the AI for how to analyze and comment on the code.
+We provide a detailed set of instructions to the AI on how to analyze and comment on the code.
 
 ### Format, build, lint
 
-At this point, we have a modified source code by an LLM. We should try to use all the available tools to validate the changes.
-It is best to start with like formatters and compilers as they are deterministic and typically fast.
+At this point, we have a modified source code by an LLM. We should try to use all the available tools to validate the changes. It is best to start with formatters and compilers as they are deterministic and typically fast.
 
 ### Judge results with LLM
 
@@ -115,7 +113,7 @@ genaiscript run cmt
 
 ## Format and build
 
-One important aspect is to normalize and valid the AI generated code. The user can provide a `format` command to run a formatter
+One important aspect is to normalize and validate the AI-generated code. The user can provide a `format` command to run a formatter
 and a `build` command to check if the code is still valid.
 
 ```ts

docs/src/content/docs/samples/gai.mdx

@@ -0,0 +1,17 @@
+---
+title: GitHub Action Investigator
+description: Investigate GitHub Actions failures
+sidebar:
+    order: 100
+---
+import { Code } from "@astrojs/starlight/components"
+import source from "../../../../../packages/sample/genaisrc/gai.genai.mts?raw"
+import gasource from "../../../../../.github/workflows/genai-investigator.yml?raw"
+
+This is an in-depth guide to build a script that interactively investigates GitHub Actions failures.
+
+## Full source ([GitHub](https://github.com/microsoft/genaiscript/blob/main/packages/sample/genaisrc/gai.genai.mts))
+
+<Code code={source} wrap={true} lang="ts" title="gai.genai.mts" />
+
+<Code code={gasource} wrap={true} lang="yaml" title="gai.yml" />

docs/src/content/docs/samples/sc.mdx

@@ -0,0 +1,94 @@
+---
+title: Spell Checker
+description: Spell check a document
+sidebar:
+    order: 101
+---
+
+import { Code } from "@astrojs/starlight/components"
+import source from "../../../../../packages/sample/genaisrc/sc.genai.mts?raw"
+
+Automating and improving the efficiency of proofreading documents is a common need among developers and writers. This script addresses this need by checking and correcting spelling and grammar in Markdown files.
+
+## Code Explanation
+
+Starting at the top of the script, we see that it's a GenAI script, which is evident from the `.mts` extension and the `script` function call.
+
+```ts
+script({
+    title: "Spell checker",
+    system: ["system", "system.files", "system.diff"],
+    temperature: 0.1,
+})
+```
+
+This block sets the title of the script to "Spell checker" and specifies that it uses several system prompts, such as file operations and diff generation. The `temperature` is set to `0.1`, indicating that the script will generate output with low creativity, favoring precision.
+
+### Fetching Files for Checking
+
+Next, we check for files to process, first from the environment and then from Git if none are provided.
+
+```ts
+let files = env.files
+if (files.length === 0) {
+    const gitStatus = await host.exec("git diff --name-only --cached")
+    files = await Promise.all(
+        gitStatus.stdout
+            .split(/\r?\n/g)
+            .filter((filename) => /\.(md|mdx)$/.test(filename))
+            .map(async (filename) => await workspace.readText(filename))
+    )
+}
+```
+
+In this block, we're assigning files from the `env` variable, which would contain any files passed to the script. If no files are provided, we execute a Git command to get a list of all cached (staged) modified files and filter them to include only `.md` and `.mdx` files. We then read the content of these files for processing.
+
+### Defining the File Types to Work on
+
+Following this, there's a `def` call:
+
+```ts
+def("FILES", files, { endsWith: [".md", ".mdx"] })
+```
+
+This line defines `FILES` to be the array of files we gathered. The options object `{ endsWith: [".md", ".mdx"] }` tells GenAI that we're only interested in files ending with `.md` or `.mdx`.
+
+The `$`-prefixed backtick notation is used to write the prompt template:
+
+```ts
+$`Fix the spelling and grammar of the content of FILES. Use diff format for small changes.
+
+- do NOT fix the frontmatter
+- do NOT fix code regions
+- do NOT fix \`code\` and \`\`\`code\`\`\`
+- in .mdx files, do NOT fix inline typescript code
+`
+```
+
+This prompt instructs GenAI to fix spelling and grammar in the content of the defined `FILES`, outputting small changes in diff format. It also specifies constraints, such as not fixing the frontmatter, code regions, inline code in markdown, and inline TypeScript code in MDX files.
+
+Finally, there is a `defFileOutput` call:
+
+```ts
+defFileOutput(files, "fixed markdown or mdx files")
+```
+
+This call declares the intent that the script will generate "fixed markdown or mdx files" based on the input files.
+
+## How to Run the Script with GenAIScript CLI
+
+Running this spell checker script is straightforward with the GenAIScript CLI. First, ensure you have the CLI installed by following the instructions in the [GenAIScript documentation](https://microsoft.github.io/genaiscript/getting-started/installation).
+
+Once you have the CLI installed, navigate to your local copy of the script in your terminal or command line interface. Run the following command to execute the spell checker:
+
+```shell
+genaiscript run sc
+```
+
+Remember, you do not need to specify the `.genai.mts` extension when using the `run` command.
+
+And there you have it—a detailed walkthrough of a GenAI spell checker script for markdown files. Happy coding and perfecting your documents!
+
+## Full source ([GitHub](https://github.com/microsoft/genaiscript/blob/main/packages/sample/genaisrc/sc.genai.mts))
+
+<Code code={source} wrap={true} lang="ts" title="sc.genai.mts" />

packages/cli/package.json

@@ -75,6 +75,7 @@
     "commander": "^12.1.0",
     "diff": "^7.0.0",
     "dotenv": "^16.4.5",
+    "es-toolkit": "^1.22.0",
     "esbuild": "^0.24.0",
     "execa": "^9.4.0",
     "fs-extra": "^11.2.0",

packages/cli/src/nodehost.ts

@@ -39,7 +39,7 @@ import {
     ResponseStatus,
 } from "../../core/src/host"
 import { AbortSignalOptions, TraceOptions } from "../../core/src/trace"
-import { logVerbose, unique } from "../../core/src/util"
+import { logVerbose } from "../../core/src/util"
 import { parseModelIdentifier } from "../../core/src/models"
 import {
     AuthenticationToken,
@@ -51,6 +51,7 @@ import { errorMessage } from "../../core/src/error"
 import { BrowserManager } from "./playwright"
 import { shellConfirm, shellInput, shellSelect } from "./input"
 import { shellQuote } from "../../core/src/shell"
+import { uniq } from "es-toolkit"
 
 class NodeServerManager implements ServerManager {
     async start(): Promise<void> {
@@ -251,7 +252,7 @@ export class NodeHost implements RuntimeHost {
             const gitignore = await tryReadText(".gitignore")
             files = await filterGitIgnore(gitignore, files)
         }
-        return unique(files)
+        return uniq(files)
     }
     async writeFile(name: string, content: Uint8Array): Promise<void> {
         await ensureDir(dirname(name))

packages/core/package.json

@@ -40,6 +40,7 @@
     "csv-stringify": "^6.5.1",
     "diff": "^7.0.0",
     "dotenv": "^16.4.5",
+    "es-toolkit": "^1.22.0",
     "esbuild": "^0.24.0",
     "fast-xml-parser": "^4.5.0",
     "fetch-retry": "^6.0.0",

packages/core/src/expander.ts

@@ -1,5 +1,5 @@
 import { Project, PromptScript } from "./ast"
-import { assert, normalizeFloat, normalizeInt, unique } from "./util"
+import { assert, normalizeFloat, normalizeInt } from "./util"
 import { MarkdownTrace } from "./trace"
 import { errorMessage, isCancelError } from "./error"
 import {

packages/core/src/fs.ts

@@ -1,6 +1,7 @@
 import { HTTPS_REGEX } from "./constants"
 import { host } from "./host"
-import { unique, utf8Decode, utf8Encode } from "./util"
+import { utf8Decode, utf8Encode } from "./util"
+import { uniq } from "es-toolkit"
 
 export async function readText(fn: string) {
     const curr = await host.readFile(fn)
@@ -60,7 +61,7 @@ export async function expandFiles(files: string[], excludedFiles?: string[]) {
         files.filter((f) => !HTTPS_REGEX.test(f)),
         { ignore: excludedFiles?.filter((f) => !HTTPS_REGEX.test(f)) }
     )
-    return unique([...urls, ...others])
+    return uniq([...urls, ...others])
 }
 
 export function filePathOrUrlToWorkspaceFile(f: string) {

packages/core/src/genaisrc/system.changelog.genai.js

@@ -17,7 +17,7 @@ index N in the above snippets, and then be prefixed with exactly the same whites
 the original snippets above. See also the following examples of the expected response format.
 
 CHANGELOG:
-\`\`\`changelog
+\`\`\`\`\`changelog
 ChangeLog:1@<file>
 Description: <summary>.
 OriginalCode@4-6:
@@ -47,5 +47,5 @@ OriginalCode@23-23:
 [23] <white space> <original code line>
 ChangedCode@23-23:
 [23] <white space> <changed code line>
-\`\`\`
+\`\`\`\`\`
 `

packages/core/src/glob.ts

@@ -11,7 +11,10 @@ import { arrayify } from "./util"
  * @param patterns - A single glob pattern or an array of glob patterns to match against.
  * @returns A boolean indicating if the filename matches any of the patterns.
  */
-export function isGlobMatch(filename: string, patterns: string | string[]) {
+export function isGlobMatch(
+    filename: string,
+    patterns: ElementOrArray<string>
+) {
     // Convert patterns to an array and check if any pattern matches the filename
     return arrayify(patterns).some((pattern) => {
         // Perform the match using minimatch with specific options