github.getfile, searchcode (#735)

* Add getFileRevision method to GitHubClient for retrieving file content by ref

* Add concurrency to workflow and refactor GitHub file retrieval method

* Add code search functionality and sample script for GitHub integration

* Remove env.secrets section and add defDiff documentation for comparing data

* Add language specification for `def` in context documentation.

* Add GitHub querying support with helper functions and examples

.github/workflows/genai-investigator.yml

@@ -4,6 +4,10 @@ on:
         workflows: ["build"]
         types:
             - completed
+concurrency:
+    group: ${{ github.workflow }}-${{ github.ref }}-${{ github.event.workflow_run.event }}-${{ github.event.workflow_run.conclusion }}
+    cancel-in-progress: true
+
 permissions:
     contents: read
     actions: read

docs/src/content/docs/reference/scripts/context.md

@@ -70,16 +70,6 @@ const locale = env.vars.locale || "en-US"
 
 Read more about [variables](/genaiscript/reference/scripts/variables).
 
-### `env.secrets`
-
-The `secrets` property contains the secrets that have been defined in the script execution context.
-
-```javascript
-const token = env.secrets.SECRET_TOKEN
-```
-
-Read more about [secrets](/genaiscript/reference/scripts/secrets).
-
 ## Definition (`def`)
 
 The `def("FILE", file)` function is a shorthand for generating a fenced variable output.
@@ -104,6 +94,15 @@ The `def` function can also be used with an array of files, such as `env.files`.
 def("FILE", env.files)
 ```
 
+### Language
+
+You can specify the language of the text contained in `def`. This can help GenAIScript optimize the rendering of the text.
+
+```js 'language: "diff"'
+// hint that the output is a diff
+def("DIFF", gitdiff, { language: "diff" })
+```
+
 ### Referencing
 
 The `def` function returns a variable name that can be used in the prompt.
@@ -210,3 +209,22 @@ defData("DATA", data, {
     sliceSample: 100,
 })
 ```
+
+## Diff Definition (`defDiff`)
+
+It is very common to compare two piece of data and ask the LLM to analyze the differences. Using diffs is a great way
+to naturally compress the information since we only reason about differences!
+
+The `defDiff` takes care of formatting the diff in a way that helps LLM reason. It behaves similarly to `def` and assigns
+a name to the diff.
+
+```js
+// diff files
+defDiff("DIFF", env.files[0], env.files[1])
+
+// diff strings
+defDiff("DIFF", "cat", "dog")
+
+// diff objects
+defDiff("DIFF", { name: "cat" }, { name: "dog" })
+```

docs/src/content/docs/reference/scripts/github.md

@@ -0,0 +1,99 @@
+---
+title: GitHub
+description: Support for querying GitHub
+sidebar:
+    order: 50
+---
+
+The `github` provides several helper function to query github. It also provides the connection information for more advanced usage.
+
+## Configuration
+
+The `github` configuration is automatically sniffed from the environment and git.
+
+The GitHub token is read from the `GITHUB_TOKEN` environment variable. Some queries might work without authentication for public repositories.
+
+### GitHub CodeSpaces
+
+In a GitHub CodeSpace, the `GITHUB_TOKEN` is automatically provisioned.
+
+### GitHub Actions
+
+In GitHub Actions, you might will to add permissions to the
+workspace to access workflow logs and pull requests. You also need to pass the `secret.GITHUB_TOKEN` to the genaiscript script run.
+
+```yml title="genai.yml" 'actions: read' 'GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}'
+permissions:
+    contents: read
+    actions: read
+    pull-requests: read # or write if you plan to create comments
+...
+    - run: npx --yes genaiscript ...
+      env:
+        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        ...
+```
+
+## Functions
+
+### Issues
+
+You can query issues and issue comments using `listIssues`, `listIssueComments`.
+
+```js
+const issues = await github.listIssues({ per_page: 5 })
+console.log(issues.map((i) => i.title))
+
+// use number!
+const issueComments = await github.listIssueComments(issues[0].number)
+console.log(issueComments)
+```
+
+### Pull Requests
+
+You can query pull requests and pull request reviews comments using `listPullRequests`, `listPullRequestReviewComments`.
+
+```js
+const prs = await github.listPullRequests({ pre_page: 5 })
+console.log(prs.map((i) => i.title))
+
+// use number!
+const prcs = await github.listPullRequestReviewComments(prs[0].number)
+console.log(prcs.map((i) => i.body))
+```
+
+In GitHub Actions, you need to give the `pull-request: read` permission.
+
+### Workflow runs
+
+Access the log of workflow runs to analyze failures.
+
+```js
+// list runs
+const runs = await github.listWorkflowRuns("build.yml", { per_page: 5 })
+console.log(runs.map((i) => i.status))
+
+const jobs = await github.listWorkflowJobs(runs[0].id)
+// redacted job log
+console.log(jobs[0].content)
+```
+
+In GitHub Actions, you need to give the `actions: read` permission.
+
+### Search Code
+
+Use `searchCode` for a code search on the default branch in the same repo.
+
+```js
+const res = await github.searchCode("HTMLToText")
+console.log(res)
+```
+
+### Get file content
+
+Gets the file content for a given ref, tag or commit sha.
+
+```js
+const pkg = await github.getFile("package.json", "main")
+console.log(pkg.content.slice(0, 50) + "...")
+```