Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Notebook support #540

Merged
merged 26 commits into from
Jun 13, 2024
Merged
Show file tree
Hide file tree
Changes from 24 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
},
"features": {
"ghcr.io/devcontainers/features/docker-in-docker:2": {},
"ghcr.io/devcontainers/features/azure-cli:1.2.5": {}
"ghcr.io/devcontainers/features/azure-cli:1.2.5": {},
"ghcr.io/devcontainers/features/python:1.6.1": {}
}
}
58 changes: 58 additions & 0 deletions demo/genaisrc/poem.genaibook

Large diffs are not rendered by default.

13 changes: 13 additions & 0 deletions demo/web-app-basic-linux/summary_and_diagrams.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,16 @@ stateDiagram
[*] --> AppServicePlan
AppServicePlan --> WebApp
```

```js title="foo.genai.js"
$`Write a one sentence poem.`
```

```md title="output"
Beneath the silver moon's soft glow, the silent lake whispers secrets to the night.
```


```js
$`Write another poem`
```
15 changes: 15 additions & 0 deletions docs/src/content/docs/getting-started/your-first-genai-script.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,21 @@ The resulting file will be placed in the `genaisrc` folder in your project.
</ol>
</Steps>

## the prompt

The execution of the GenAIScript generates a prompt (and more)
that gets sent to the LLM model.

The ` $``...`` ` template string function formats and write the string to the prompt; which gets sent to the LLM.
pelikhan marked this conversation as resolved.
Show resolved Hide resolved

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The template string function is incorrectly represented as $``...```. It should be ${...}` to correctly denote a template literal in JavaScript.

generated by pr-docs-review-commit template_literal


```js title="poem.genai.js"
$`Write a one sentence poem.`
```

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code block starting on line 50 should specify the language for syntax highlighting. It should be js instead of js title="poem.genai.js".

generated by pr-docs-review-commit code_fence_info_string


```md title="output" wrap
pelikhan marked this conversation as resolved.
Show resolved Hide resolved
Beneath the silver moon's gentle gaze, the tranquil sea whispers secrets to the shore.
```

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The wrap attribute in the markdown code block is unnecessary and not standard markdown syntax.

generated by pr-docs-review-commit unnecessary_wrap_attribute

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code block starting on line 54 should specify the language for syntax highlighting. It should be md instead of md title="🤖" wrap. Additionally, the "wrap" option is not standard and may not be recognized by the renderer.

generated by pr-docs-review-commit code_fence_info_string

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code fence for the markdown block is incorrectly formatted; it should not include the title and wrap annotations.

generated by pr-docs-review-commit incorrect_code_fence



## the Context

Expand Down
4 changes: 0 additions & 4 deletions packages/core/src/ast.ts
Original file line number Diff line number Diff line change
Expand Up @@ -78,10 +78,6 @@ export class Fragment {
Object.assign(this, init)
if (!this.fullId) this.fullId = this.id
}
pelikhan marked this conversation as resolved.
Show resolved Hide resolved

get project() {
return this.file.project
}
}
pelikhan marked this conversation as resolved.
Show resolved Hide resolved

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The project getter method has been removed from the Fragment class but there is no replacement provided. If other parts of the codebase are using this method, this will cause a breakage.

generated by pr-review-commit unused_code


export function templateGroup(template: PromptScript) {
Expand Down
2 changes: 1 addition & 1 deletion packages/core/src/chat.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@ import {
renderFencedVariables,
} from "./fence"
import { validateFencesWithSchema, validateJSONWithSchema } from "./schema"
import dedent from "ts-dedent"
import {
CHAT_CACHE,
DEFAULT_MODEL,
Expand All @@ -28,6 +27,7 @@ import { fenceMD } from "./markdown"
import { YAMLStringify } from "./yaml"
import { estimateChatTokens } from "./tokens"
import { createChatTurnGenerationContext } from "./runpromptcontext"
import { dedent } from "./indent"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The import of dedent has been changed from ts-dedent to ./indent. If the implementation of dedent in ./indent is different from ts-dedent, this could potentially introduce bugs.

generated by pr-review-commit import_change


export type ChatCompletionTool = OpenAI.Chat.Completions.ChatCompletionTool

Expand Down
2 changes: 2 additions & 0 deletions packages/core/src/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,8 @@ export const XLSX_MIME_TYPE =
export const JSON_MIME_TYPE = "application/json"
export const JSON_SCHEMA_MIME_TYPE = "application/schema+json"
export const JAVASCRIPT_MIME_TYPE = "application/javascript"
export const MARKDOWN_MIME_TYPE = "text/markdown"
export const YAML_MIME_TYPE = "application/yaml"

export const JSON_META_SCHEMA_URI =
"https://json-schema.org/draft/2020-12/schema"
Expand Down
10 changes: 10 additions & 0 deletions packages/core/src/indent.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
import tsDedent from "ts-dedent"

export function indent(text: string, indentation: string) {
return text
?.split(/\r?\n/g)
.map((line) => indentation + line)
.join("\n")
}

export const dedent = tsDedent
3 changes: 2 additions & 1 deletion packages/core/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -63,4 +63,5 @@ export * from "./parameters"
export * from "./scripts"
export * from "./math"
export * from "./fence"
export * from "./ollama"
export * from "./ollama"
export * from "./indent"
8 changes: 8 additions & 0 deletions packages/core/src/markdown.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,11 @@ export function fenceMD(t: string, contentType?: string) {
export function link(text: string, href: string) {
return href ? `[${text}](${href})` : text
}

export function details(summary: string, body: string) {
return `\n<details><summary>${summary}</summary>

${body}

</summary></details>\n`
}
2 changes: 1 addition & 1 deletion packages/core/src/promptdom.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@ import { YAMLStringify } from "./yaml"
import { MARKDOWN_PROMPT_FENCE, PROMPT_FENCE } from "./constants"
import { fenceMD } from "./markdown"
import { parseModelIdentifier } from "./models"
import dedent from "ts-dedent"
import {
ChatCompletionAssistantMessageParam,
ChatCompletionMessageParam,
Expand All @@ -18,6 +17,7 @@ import {
import { errorMessage } from "./error"
import { tidyData } from "./tidy"
import { inspect } from "./logging"
import { dedent } from "./indent"

export interface PromptNode extends ContextExpansionOptions {
type?:
Expand Down
4 changes: 4 additions & 0 deletions packages/sample/genaisrc/pr-review-commit.genai.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
script({
model: "openai:gpt-4-32k",
temperature: 0,
files: [],
title: "pull request commit review",
system: [
Expand Down Expand Up @@ -32,11 +33,14 @@ Provide feedback to the author using annotations.

Think step by step and for each annotation explain your result.

## Guidelines

- Assume the TypeScript code is type correct. do NOT report issues that the TypeScript type checker would find.
- report 3 most serious errors only, ignore notes and warnings
- only report issues you are absolutely certain about
- do NOT repeat the same issue multiple times
- do NOT report common convention issues
- do NOT report deleted code since you cannot review the entire codebase
- use a friendly tone
- use emojis
- do NOT cross-reference annotations, assume they are all independent
Expand Down
38 changes: 38 additions & 0 deletions packages/sample/src/jupytest.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"ename": "",
"evalue": "",
"output_type": "error",
"traceback": [
"\u001b[1;31mRunning cells with 'Python 3.11.2' requires the ipykernel package.\n",
"\u001b[1;31mRun the following command to install 'ipykernel' into the Python environment. \n",
"\u001b[1;31mCommand: '/bin/python3 -m pip install ipykernel -U --user --force-reinstall'"
]
}
],
"source": [
"x = 1\n",
"\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"name": "python",
"version": "3.11.2"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
41 changes: 41 additions & 0 deletions packages/sample/src/node.nnb
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
{
"cells": [
{
"language": "typescript",
"source": [
"var x = 2\ny =1"
],
"outputs": [
{
"items": [
{
"mime": "text/plain",
"value": [
"\u001b[33m1\u001b[39m"
]
}
]
}
]
},
{
"language": "typescript",
"source": [
"// <reference path=\"../genaisrc/genaiscript.d.ts\" />\nconsole.log({ x,y})\n"
],
"outputs": [
{
"items": [
{
"mime": "application/vnd.code.notebook.stdout",
"value": [
"{ x: 2, y: 1 }",
""
]
}
]
}
]
}
]
}
48 changes: 48 additions & 0 deletions packages/sample/src/poem.genaibook

Large diffs are not rendered by default.

15 changes: 14 additions & 1 deletion packages/vscode/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,8 @@
},
"activationEvents": [
"workspaceContains:**/*.gpspec.md",
"workspaceContains:**/*.genai.js"
"workspaceContains:**/*.genai.js",
"onNotebook:genaiscript"
],
"main": "./built/extension.js",
"bugs": {
Expand All @@ -56,6 +57,18 @@
"./icon-dark.svg"
],
"contributes": {
"notebooks": [
{
"type": "genaiscript",
"displayName": "GenAIScript Markdown Notebook",
"priority": "option",
"selector": [
{
"filenamePattern": "*.{md,mdx,markdown}"
}
]
}
],
"markdown.previewStyles": [
"./markdown.css"
],
Expand Down
2 changes: 2 additions & 0 deletions packages/vscode/src/extension.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import { activateOpenAIRequestTreeDataProvider } from "./openairequesttree"
import { activateAIRequestTreeDataProvider } from "./airequesttree"
import { activateTestController } from "./testcontroller"
import { activateModelCompletionProvider } from "./modelcompletionprovider"
import { activateNotebook } from "./notebook"

export async function activate(context: ExtensionContext) {
const state = new ExtensionState(context)
Expand All @@ -22,6 +23,7 @@ export async function activate(context: ExtensionContext) {
activateOpenAIRequestTreeDataProvider(state)
activateStatusBar(state)
activateModelCompletionProvider(state)
activateNotebook(state)
// activateChatParticipant(state)

context.subscriptions.push(
Expand Down
Loading
Loading