Skip to content

Commit

Permalink
added workspace.readJSON
Browse files Browse the repository at this point in the history
  • Loading branch information
pelikhan committed Jul 16, 2024
1 parent 51106a7 commit 0cb01b1
Show file tree
Hide file tree
Showing 21 changed files with 113 additions and 5 deletions.
6 changes: 6 additions & 0 deletions demo/genaisrc/genaiscript.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions docs/genaisrc/genaiscript.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 8 additions & 0 deletions docs/src/content/docs/reference/scripts/files.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,14 @@ const content = file.content

It will automatically convert PDFs and DOCX files to text.

### `readJSON`

Reads the content of a file as JSON (using a [JSON5](https://json5.org/) parser)

```ts
const data = await workspace.readJSON("data.json")
```

### `writeText`

Writes text to a file, relative to the workspace root.
Expand Down
6 changes: 6 additions & 0 deletions genaisrc/genaiscript.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions packages/core/src/fs.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { DOT_ENV_REGEX, HTTPS_REGEX } from "./constants"
import { NotSupportedError, errorMessage } from "./error"
import { resolveFileContent } from "./file"
import { host } from "./host"
import { JSON5parse } from "./json5"
import { logVerbose, unique, utf8Decode, utf8Encode } from "./util"
import ignorer from "ignore"

Expand Down Expand Up @@ -105,6 +106,11 @@ export function createFileSystem(): Omit<WorkspaceFileSystem, "grep"> {
}
return file
},
readJSON: async (f: string | WorkspaceFile) => {
const file = await fs.readText(f)
const res = JSON5parse(file.content)
return res
},
}
;(fs as any).readFile = readText
return Object.freeze(fs)
Expand Down
6 changes: 6 additions & 0 deletions packages/core/src/genaisrc/genaiscript.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions packages/core/src/promptcontext.ts
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@ export function createPromptContext(
const path = runtimeHost.path
const workspace: WorkspaceFileSystem = {
readText: (f) => runtimeHost.workspace.readText(f),
readJSON: (f) => runtimeHost.workspace.readJSON(f),
writeText: (f, c) => runtimeHost.workspace.writeText(f, c),
findFiles: async (pattern, options) => {
const res = await runtimeHost.workspace.findFiles(pattern, options)
Expand Down
6 changes: 6 additions & 0 deletions packages/core/src/types/prompt_template.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -492,6 +492,12 @@ interface WorkspaceFileSystem {
*/
readText(path: string | WorkspaceFile): Promise<WorkspaceFile>

/**
* Reads the content of a file and parses to JSON, using the JSON5 parser.
* @param path
*/
readJSON(path: string | WorkspaceFile): Promise<any>

/**
* Writes a file as text to the file system
* @param path
Expand Down
6 changes: 6 additions & 0 deletions packages/sample/genaisrc/genaiscript.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion packages/sample/genaisrc/git-release-notes.genai.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ script({ system: ["system"], temperature: 0.5, model: "openai:gpt-4-turbo" })
const product = env.vars.product || "GenAIScript"

// find previous tag
const pkg = JSON.parse((await workspace.readText("package.json")).content)
const pkg = await workspace.readJSON("package.json")
const { version } = pkg
const { stdout: tag } = await host.exec("git", [
"describe",
Expand Down
6 changes: 6 additions & 0 deletions packages/sample/genaisrc/node/genaiscript.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions packages/sample/genaisrc/python/genaiscript.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions packages/sample/genaisrc/style/genaiscript.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 4 additions & 1 deletion packages/sample/genaisrc/writefile.genai.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,12 @@ script({
model: "openai:gpt-3.5-turbo",
})
const fn = `temp/${Math.random() + ""}.txt`
const content = Math.random() + ""
const content = JSON.stringify({ val: Math.random() + "" })
await workspace.writeText(fn, content)
const res = await workspace.readText(fn)
if (content !== res.content) throw new Error("file write error")

const jres = await workspace.readJSON(fn)
if (JSON.stringify(jres) !== content) throw new Error("readJSON error")

$`All good!`
6 changes: 6 additions & 0 deletions packages/sample/src/aici/genaiscript.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions packages/sample/src/errors/genaiscript.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions packages/sample/src/makecode/genaiscript.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions packages/sample/src/makecode/makecode-loc.genai.js
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ const strings = JSON.parse(content)

// find the existing translation and remove existing translations
const trfn = path.join(dir, langCode, path.basename(filename))
const translated = parsers.JSON5(await workspace.readText(trfn))
const translated = await workspace.readJSON(trfn)
if (translated)
for (const k of Object.keys(strings)) if (translated[k]) delete strings[k]

Expand Down Expand Up @@ -113,8 +113,8 @@ defFileMerge((filename, label, before, generated) => {
// parse out kv
const news = generated
.split(/\n/g)
.map(line => /^([^=]+)=(.+)$/.exec(line))
.filter(m => !!m)
.map((line) => /^([^=]+)=(.+)$/.exec(line))
.filter((m) => !!m)
.reduce((o, m) => {
const [, key, value] = m
// assign
Expand Down
6 changes: 6 additions & 0 deletions packages/sample/src/tla/genaiscript.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions packages/sample/src/vision/genaiscript.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions slides/genaisrc/genaiscript.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

0 comments on commit 0cb01b1

Please sign in to comment.