The Basics of GenAIScript
+
Introduction
+
GenAIScript is a framework that empowers teams, including
+non-developers, to create and use AI-enhanced scripts to support their
+workflows. GenAIScript provides support for authoring and debugging
+JavaScript scripts that incorporate calls to foundation models and LLMs 1
+in their execution. GenAIScript is a programming framework that
+allows its users to author AI scripts (which we call a GenAIScript),
+debug those scripts in a development environment that is an extension of
+VS Code, and package those scripts in a command-line interface that can
+be deployed in many contexts.
+
Our VS Code extension supports easy authoring of a GenAIScript by
+writing natural language in markdown syntax plus a small amount of
+stylized JavaScript programming. Our framework allows users to leverage
+multiple LLM models, parameterize the calls to the models, execute and
+debug scripts, trace the construction of the LLM prompts and provide a
+full trace of execution from prompt construction to LLM generation to
+parsing the LLM result. Our framework also supports extracting multiple
+forms of output from LLM generations, including output in files of
+different types, outputs intended as edits to existing files and outputs
+in structured formats, such as JSON.
+
Key terms
+
GenAIScript A stylized JavaScript program that defines the context
+for the LLM call, allows arbitrary JavaScript code execution, packages
+the prompt input for the LLM, calls the LLM, and unpacks that LLM output
+based on the directions given in the prompt.
+
GPVM: A runtime system that given a GenAIScript and an optional
+GPSpec, executes the GenAIScript, which involves integrating the context
+into a prompt, calling the specified LLM, and extracting content from
+the LLM result.
+
VS Code GenAIScript extension An add-in to VS Code that provides
+users with easy methods for creating, editing, running and debugging
+GenAIScript.
+
Foundation models and LLMs While GenAIScript currently supports
+different LLMs, in the future we anticipate that we will incorporate
+additional foundation models beyond large language models.
+
Capabilities
+
System behavior
+
GenAIScript is a general-purpose AI-script authoring framework for
+seamlessly integrating code execution and foundation model/LLM
+invocations. A GenAIScript is a JavaScript program in a stylized format
+that allows users to easily specify the LLM context and prompt, invoked
+a specified model, and parse the resulting output according to user
+specifications. This functionality allows even users who are not
+programmers to inspect model results and double check them for
+correctness.
+
GenAIScript can be written in any IDE but the VS Code GenAIScript add-in
+makes creating, executing and debugging GenAIScript especially easy.
+GenAIScript users can implement tools that generate and edit multiple
+files with a single tool and our integration with VS Code leverages
+existing functionality in for refactoring to allow users to easily see
+the results of the tool execution. The add-in supports creating a new
+GenAIScript, invoking a given GenAIScript, tracing the execution of the GenAIScript in establishing the LLM
+context and final prompt, and unparsing the LLM output into the
+user-specified elements. Examples of all of these capabilities can be
+viewed in the documents in the GenAIScript repository:
+microsoft/GenAIScript: Generative AI Scripting
+(github.com)
+
The goal of GenAIScript is to empower a broad range of potential users
+to innovate with building AI-powered scripts and identify new ways to
+leverage AI for their daily tasks. We expect that professional
+developers, who are familiar with writing and using scripts to enhance
+their productivity will be the early adopters of GenAIScript.
+GenAIScript will give these users benefit because GenAIScript can do
+many things that existing scripts written in traditional scripting
+languages like JavaScript and Python cannot do. While developers can
+leverage other frameworks, such as langchain and Semantic Kernel, that
+integrate calls to LLMs into languages like Python, these
+frameworks require more user effort and have less IDE support than
+GenAIScript. Ultimately, because our goal is to make GenAIScript easy to
+author, modify, debug and run, we anticipate that they will be useful
+far beyond professional developers. A major impact of GenAIScript will
+be to enable non-developers to innovate and build GenAIScripts that
+enhance their productivity. We illustrate this point with examples
+below.
+
Documentation
+
To help users get started with GenAIScript, we include documentation in
+our repository that illustrates in code snippets the contents of several
+different GenAIScripts. The documentation shows both what the example
+GenAIScript looks like as well as what the effect is from the
+GenAIScript acting on a particular input. While these examples are
+intended to explain the technology, they are not intended to be the
+basis for user-written tools.
+
Use cases
+
Intended uses
+
GenAIScript can be used in any context where a command line script
+written in another programming language might be used but the use cases
+are much more ambitious because the LLM can do much more than ordinary
+code. Here are some examples:
+
+-
+
Checking for potential inconsistencies in a collection of
+configuration files or other content. Using the LLM, a GenAIScript
+can inspect configuration files and leverage the LLMโs understanding
+of common configuration errors to detect and report them. Before
+LLMs, professional developers would write tools, such as lint2,
+which are complex programs that detect inconsistencies in the syntax
+of their code files. With GenAIScript, checking tools can be written
+for much richer scenarios (such as checking for inappropriate
+variable names), and by individuals who are not professional
+developers.
+
+-
+
Automating document translation: Given documentation in a
+repository written in one natural language, a GenAIScript can be
+written to translate that documentation into another language. For a
+specific example of why GenAIScript is important for this use,
+consider the task of maintaining the localization of the
+MakeCode3 documentation. MakeCode documentation has nearly 2M
+files, which are typically markdown with a mix of code snippets.
+Many documents are partially translated (at the paragraph level). To check the correctness of
+document translations, there are
+3500 registered volunteer translators for 35+ languages. One cannot
+just apply Bing translate for this use case, as it typically destroys the code
+snippets. With GenAIScript, we can have a script that goes through
+every documentation file, pulls the current localized version and
+assembles a prompt to ask the LLM to fill in the missing
+translations, while leaving the existing ones alone. Because the LLM model we use has already been trained on
+MakeCode examples and documentation it is aware of the syntax.
+
+-
+
Generating executable code from a natural language
+specification. A GPSpec file can be used to specify the task being
+performed and a GenAIScript that specializes in code generation can
+translate the spec into code.
+
+-
+
Creating a short version of a longer white paper by summarizing
+each chapter. LLMs are quite effective at summarizing documents. A
+GenAIScript can be written to take each chapter of a long document
+and summarize it in a section of a shorter document.
+
+-
+
Translating a monolog to a dialog. Given a monolog from a video
+transcript, a GenAIScript can be written to rewrite the monolog into
+a dialog between two individuals (akin to sports announcers talking
+to each other) to make the video more interesting and accessible.
+
+
+
Unintended uses
+
GenAIScript is a general framework for authoring scripts. As a result,
+an adversary can use GenAIScript to author adversarial scripts that
+could be used for malicious purposes. All of the adversarial uses of
+GenAIScript could also be implemented in other LLM language extension
+frameworks such as Sematic Kernel, autogen, and langchain, so the danger
+from unintended uses of GenAIScript stems from possibility that it might
+make it easier to author adversarial scripts. This issue is present in
+any infrastructure that makes programming easier, including languages
+such as PowerShell, JavaScript, and Python, as well as IDEs such as VS
+Code and Visual Studio. While we cannot prevent unintended uses, we will
+encourage users to consider Responsible AI practices when they build
+GenAIScripts. We provide more details about issues related to security and trust in security and trust.
+
Foundation model best practices
+
We strongly encourage GenAIScript users to use foundation models and
+LLMs that support robust Responsible AI mitigations, such as the Azure
+Open AI (AOAI) services. Such services continually update the safety and
+RAI mitigations to track our up-to-date understanding on how to deploy
+and use foundation models most responsibly. Here are resources to help
+understand and use best practices when employing foundations models
+for scripts and applications:
+
+
Limitations
+
GenAIScript is an evolving framework that will improve based on input
+from users. Existing limitations in the framework include integration into only one IDE
+(VS code), and internal support for OpenAI APIs plus a relatively small
+number of other LLMs. We intend to allow users to integrate calls to
+external services (such as RAG) in GenAIScript to provide the LLM with
+more context. We anticipate adding support for more foundation models as the use cases evolve.
+
We also anticipate that the on-ramp to using GenAIScript will evolve. We
+have explored supporting invoking the GenAIScript framework as part of a VS
+Code Copilot Chat experience (hosted in VS Code Insiderโs Edition). We also understand that some developers would prefer to
+implement their GenAIScript using Python instead of JavaScript. We
+anticipate building a Python binding form authoring GenAIScripts in the
+future.
+
Technical limitations, operational factors and ranges
+
GenAIScript does not use any AI model in executing the framework itself.
+Individuals using GenAIScript to author their own AI scripts will be
+subject to the technical limitations, operational factors, and ranges of
+the AI LLM their script uses.
+
+
GenAIScript encourages users to consult the best practices for authoring
+effective prompts for the specific LLM they are invoking in their tool.
+
Learn more about responsible AI
+
Microsoft AI
+principles
+
Microsoft responsible AI
+resources
+
Microsoft Azure Learning courses on responsible
+AI
+
Learn more about the GenAIScript
+
Read more about GenAIScript at our GitHub site, microsoft/GenAIScript: GenAI
+Scripting (github.com)
+
+
Give us feedback on this document: zorn@microsoft.com,
+jhalleux@microsoft.com
+
About this document
+
ยฉย 2024 Microsoft Corporation. All rights reserved. This document is
+provided โas-isโ and for informational purposes only. Information and
+views expressed in this document, including URL and other Internet Web
+site references, may change without notice. You bear the risk of using
+it. Some examples are for illustration only and are fictitious. No real
+association is intended or inferred.
+
This document is not intended to be, and should not be construed as
+providing. legal advice. The jurisdiction in which youโre operating may
+have various regulatory or legal requirements that apply to your AI
+system. Consult a legal specialist if you are uncertain about laws or
+regulations that might apply to your system, especially if you think
+those might impact these recommendations. Be aware that not all of these
+recommendations and resources will be appropriate for every scenario,
+and conversely, these recommendations and resources may be insufficient
+for some scenarios.
+
+-
+
Published: March 18, 2024
+
+-
+
Last updated: March 18, 2024
+
+
+
+
%3b}%23mermaid-0 .cluster rect{fill:%23ffffde%3bstroke:%23aaaa33%3bstroke-width:1px%3b}%23mermaid-0 .cluster text{fill:%23333%3b}%23mermaid-0 .cluster span%2c%23mermaid-0 p{color:%23333%3b}%23mermaid-0 div.mermaidTooltip{position:absolute%3btext-align:center%3bmax-width:200px%3bpadding:2px%3bfont-family:arial%2csans-serif%3bfont-size:12px%3bbackground:hsl(80%2c 100%%2c 96.2745098039%)%3bborder:1px solid %23aaaa33%3bborder-radius:2px%3bpointer-events:none%3bz-index:100%3b}%23mermaid-0 .flowchartTitleText{text-anchor:middle%3bfont-size:18px%3bfill:%23333%3b}%23mermaid-0 :root{--mermaid-font-family:arial%2csans-serif%3b}</style><g><marker orient='auto' markerHeight='12' markerWidth='12' markerUnits='userSpaceOnUse' refY='5' refX='6' viewBox='0 0 10 10' class='marker flowchart' id='mermaid-0_flowchart-pointEnd'><path style='stroke-width: 1%3b stroke-dasharray: 1%2c 0%3b' class='arrowMarkerPath' d='M 0 0 L 10 5 L 0 10 z'/></marker><marker orient='auto' markerHeight='12' markerWidth='12' markerUnits='userSpaceOnUse' refY='5' refX='4.5' viewBox='0 0 10 10' class='marker flowchart' id='mermaid-0_flowchart-pointStart'><path style='stroke-width: 1%3b stroke-dasharray: 1%2c 0%3b' class='arrowMarkerPath' d='M 0 5 L 10 10 L 10 0 z'/></marker><marker orient='auto' markerHeight='11' markerWidth='11' markerUnits='userSpaceOnUse' refY='5' refX='11' viewBox='0 0 10 10' class='marker flowchart' id='mermaid-0_flowchart-circleEnd'><circle style='stroke-width: 1%3b stroke-dasharray: 1%2c 0%3b' class='arrowMarkerPath' r='5' cy='5' cx='5'/></marker><marker orient='auto' markerHeight='11' markerWidth='11' markerUnits='userSpaceOnUse' refY='5' refX='-1' viewBox='0 0 10 10' class='marker flowchart' id='mermaid-0_flowchart-circleStart'><circle style='stroke-width: 1%3b stroke-dasharray: 1%2c 0%3b' class='arrowMarkerPath' r='5' cy='5' cx='5'/></marker><marker orient='auto' markerHeight='11' markerWidth='11' markerUnits='userSpaceOnUse' refY='5.2' refX='12' viewBox='0 0 11 11' class='marker cross flowchart' id='mermaid-0_flowchart-crossEnd'><path style='stroke-width: 2%3b stroke-dasharray: 1%2c 0%3b' class='arrowMarkerPath' d='M 1%2c1 l 9%2c9 M 10%2c1 l -9%2c9'/></marker><marker orient='auto' markerHeight='11' markerWidth='11' markerUnits='userSpaceOnUse' refY='5.2' refX='-1' viewBox='0 0 11 11' class='marker cross flowchart' id='mermaid-0_flowchart-crossStart'><path style='stroke-width: 2%3b stroke-dasharray: 1%2c 0%3b' class='arrowMarkerPath' d='M 1%2c1 l 9%2c9 M 10%2c1 l -9%2c9'/></marker><g class='root'><g class='clusters'><g id='agent' class='cluster default flowchart-label'><rect height='166' width='213.921875' y='202' x='0' ry='0' rx='0' style=''/><g transform='translate(48.703125%2c 202)' class='cluster-label'><foreignObject height='18' width='116.515625'><div style='display: inline-block%3b white-space: nowrap%3b' xmlns='http://www.w3.org/1999/xhtml'><span class='nodeLabel'>agent user (tool)</span></div></foreignObject></g></g></g><g class='edgePaths'><path marker-end='url(%23mermaid-0_flowchart-pointEnd)' style='fill:none%3b' class='edge-thickness-normal edge-pattern-solid flowchart-link LS-query LE-LLM' id='L-query-LLM-0' d='M106.961%2c33L106.961%2c38.667C106.961%2c44.333%2c106.961%2c55.667%2c106.961%2c66.117C106.961%2c76.567%2c106.961%2c86.133%2c106.961%2c90.917L106.961%2c95.7'/><path marker-end='url(%23mermaid-0_flowchart-pointEnd)' style='fill:none%3b' class='edge-thickness-normal edge-pattern-solid flowchart-link LS-LLM LE-agent_user_LLM' id='L-LLM-agent_user_LLM-0' d='M106.961%2c134L106.961%2c139.667C106.961%2c145.333%2c106.961%2c156.667%2c106.961%2c168C106.961%2c179.333%2c106.961%2c190.667%2c106.961%2c199.617C106.961%2c208.567%2c106.961%2c215.133%2c106.961%2c218.417L106.961%2c221.7'/><path marker-end='url(%23mermaid-0_flowchart-pointEnd)' style='fill:none%3b' class='edge-thickness-normal edge-pattern-solid flowchart-link LS-agent_user_LLM LE-user_tools' id='L-agent_user_LLM-user_tools-0' d='M106.961%2c260L106.961%2c264.167C106.961%2c268.333%2c106.961%2c276.667%2c106.961%2c284.117C106.961%2c291.567%2c106.961%2c298.133%2c106.961%2c301.417L106.961%2c304.7'/><path marker-end='url(%23mermaid-0_flowchart-pointEnd)' style='fill:none%3b' class='edge-thickness-normal edge-pattern-solid flowchart-link LS-user_tools LE-user' id='L-user_tools-user-0' d='M106.961%2c343L106.961%2c347.167C106.961%2c351.333%2c106.961%2c359.667%2c106.961%2c369.5C106.961%2c379.333%2c106.961%2c390.667%2c106.961%2c401.117C106.961%2c411.567%2c106.961%2c421.133%2c106.961%2c425.917L106.961%2c430.7'/></g><g class='edgeLabels'><g transform='translate(106.9609375%2c 67)' class='edgeLabel'><g transform='translate(-60.4609375%2c -9)' class='label'><foreignObject height='18' width='120.921875'><div style='display: inline-block%3b white-space: nowrap%3b' xmlns='http://www.w3.org/1999/xhtml'><span class='edgeLabel'>confirm with user</span></div></foreignObject></g></g><g transform='translate(106.9609375%2c 168)' class='edgeLabel'><g transform='translate(-102.7109375%2c -9)' class='label'><foreignObject height='18' width='205.421875'><div style='display: inline-block%3b white-space: nowrap%3b' xmlns='http://www.w3.org/1999/xhtml'><span class='edgeLabel'>ask user to confirm yes or no</span></div></foreignObject></g></g><g class='edgeLabel'><g transform='translate(0%2c 0)' class='label'><foreignObject height='0' width='0'><div style='display: inline-block%3b white-space: nowrap%3b' xmlns='http://www.w3.org/1999/xhtml'><span class='edgeLabel'></span></div></foreignObject></g></g><g transform='translate(106.9609375%2c 402)' class='edgeLabel'><g transform='translate(-64.046875%2c -9)' class='label'><foreignObject height='18' width='128.09375'><div style='display: inline-block%3b white-space: nowrap%3b' xmlns='http://www.w3.org/1999/xhtml'><span class='edgeLabel'>are you sure%3f Yes</span></div></foreignObject></g></g></g><g class='nodes'><g transform='translate(106.9609375%2c 326.5)' data-id='user_tools' data-node='true' id='flowchart-user_tools-5' class='node default default flowchart-label'><rect height='33' width='143.921875' y='-16.5' x='-71.9609375' ry='0' rx='0' style='' class='basic label-container'/><g transform='translate(-64.4609375%2c -9)' style='' class='label'><rect/><foreignObject height='18' width='128.921875'><div style='display: inline-block%3b white-space: nowrap%3b' xmlns='http://www.w3.org/1999/xhtml'><span class='nodeLabel'>user confirm (tool)</span></div></foreignObject></g></g><g transform='translate(106.9609375%2c 243.5)' data-id='agent_user_LLM' data-node='true' id='flowchart-agent_user_LLM-3' class='node default default flowchart-label'><rect height='33' width='90.609375' y='-16.5' x='-45.3046875' ry='0' rx='0' style='' class='basic label-container'/><g transform='translate(-37.8046875%2c -9)' style='' class='label'><rect/><foreignObject height='18' width='75.609375'><div style='display: inline-block%3b white-space: nowrap%3b' xmlns='http://www.w3.org/1999/xhtml'><span class='nodeLabel'>agent LLM</span></div></foreignObject></g></g><g transform='translate(106.9609375%2c 16.5)' data-id='query' data-node='true' id='flowchart-query-0' class='node default default flowchart-label'><rect height='33' width='55.03125' y='-16.5' x='-27.515625' ry='0' rx='0' style='' class='basic label-container'/><g transform='translate(-20.015625%2c -9)' style='' class='label'><rect/><foreignObject height='18' width='40.03125'><div style='display: inline-block%3b white-space: nowrap%3b' xmlns='http://www.w3.org/1999/xhtml'><span class='nodeLabel'>query</span></div></foreignObject></g></g><g transform='translate(106.9609375%2c 117.5)' data-id='LLM' data-node='true' id='flowchart-LLM-1' class='node default default flowchart-label'><rect height='33' width='46.125' y='-16.5' x='-23.0625' ry='0' rx='0' style='' class='basic label-container'/><g transform='translate(-15.5625%2c -9)' style='' class='label'><rect/><foreignObject height='18' width='31.125'><div style='display: inline-block%3b white-space: nowrap%3b' xmlns='http://www.w3.org/1999/xhtml'><span class='nodeLabel'>LLM</span></div></foreignObject></g></g><g transform='translate(106.9609375%2c 452.5)' data-id='user' data-node='true' id='flowchart-user-7' class='node default default flowchart-label'><rect height='33' width='46.125' y='-16.5' x='-23.0625' ry='0' rx='0' style='' class='basic label-container'/><g transform='translate(-15.5625%2c -9)' style='' class='label'><rect/><foreignObject height='18' width='31.125'><div style='display: inline-block%3b white-space: nowrap%3b' xmlns='http://www.w3.org/1999/xhtml'><span class='nodeLabel'>user</span></div></foreignObject></g></g></g></g></g></svg>)
+
+
+%3b%7d%23mermaid-0 .cluster rect%7bfill:%23ffffde%3bstroke:%23aaaa33%3bstroke-width:1px%3b%7d%23mermaid-0 .cluster text%7bfill:%23333%3b%7d%23mermaid-0 .cluster span%2c%23mermaid-0 p%7bcolor:%23333%3b%7d%23mermaid-0 div.mermaidTooltip%7bposition:absolute%3btext-align:center%3bmax-width:200px%3bpadding:2px%3bfont-family:arial%2csans-serif%3bfont-size:12px%3bbackground:hsl(80%2c 100%25%2c 96.2745098039%25)%3bborder:1px solid %23aaaa33%3bborder-radius:2px%3bpointer-events:none%3bz-index:100%3b%7d%23mermaid-0 .flowchartTitleText%7btext-anchor:middle%3bfont-size:18px%3bfill:%23333%3b%7d%23mermaid-0 :root%7b--mermaid-font-family:arial%2csans-serif%3b%7d%3c/style%3e%3cg%3e%3cmarker orient='auto' markerHeight='12' markerWidth='12' markerUnits='userSpaceOnUse' refY='5' refX='6' viewBox='0 0 10 10' class='marker flowchart' id='mermaid-0_flowchart-pointEnd'%3e%3cpath style='stroke-width: 1%3b stroke-dasharray: 1%2c 0%3b' class='arrowMarkerPath' d='M 0 0 L 10 5 L 0 10 z'/%3e%3c/marker%3e%3cmarker orient='auto' markerHeight='12' markerWidth='12' markerUnits='userSpaceOnUse' refY='5' refX='4.5' viewBox='0 0 10 10' class='marker flowchart' id='mermaid-0_flowchart-pointStart'%3e%3cpath style='stroke-width: 1%3b stroke-dasharray: 1%2c 0%3b' class='arrowMarkerPath' d='M 0 5 L 10 10 L 10 0 z'/%3e%3c/marker%3e%3cmarker orient='auto' markerHeight='11' markerWidth='11' markerUnits='userSpaceOnUse' refY='5' refX='11' viewBox='0 0 10 10' class='marker flowchart' id='mermaid-0_flowchart-circleEnd'%3e%3ccircle style='stroke-width: 1%3b stroke-dasharray: 1%2c 0%3b' class='arrowMarkerPath' r='5' cy='5' cx='5'/%3e%3c/marker%3e%3cmarker orient='auto' markerHeight='11' markerWidth='11' markerUnits='userSpaceOnUse' refY='5' refX='-1' viewBox='0 0 10 10' class='marker flowchart' id='mermaid-0_flowchart-circleStart'%3e%3ccircle style='stroke-width: 1%3b stroke-dasharray: 1%2c 0%3b' class='arrowMarkerPath' r='5' cy='5' cx='5'/%3e%3c/marker%3e%3cmarker orient='auto' markerHeight='11' markerWidth='11' markerUnits='userSpaceOnUse' refY='5.2' refX='12' viewBox='0 0 11 11' class='marker cross flowchart' id='mermaid-0_flowchart-crossEnd'%3e%3cpath style='stroke-width: 2%3b stroke-dasharray: 1%2c 0%3b' class='arrowMarkerPath' d='M 1%2c1 l 9%2c9 M 10%2c1 l -9%2c9'/%3e%3c/marker%3e%3cmarker orient='auto' markerHeight='11' markerWidth='11' markerUnits='userSpaceOnUse' refY='5.2' refX='-1' viewBox='0 0 11 11' class='marker cross flowchart' id='mermaid-0_flowchart-crossStart'%3e%3cpath style='stroke-width: 2%3b stroke-dasharray: 1%2c 0%3b' class='arrowMarkerPath' d='M 1%2c1 l 9%2c9 M 10%2c1 l -9%2c9'/%3e%3c/marker%3e%3cg class='root'%3e%3cg class='clusters'/%3e%3cg class='edgePaths'%3e%3cpath marker-end='url(%23mermaid-0_flowchart-pointEnd)' style='fill:none%3b' class='edge-thickness-normal edge-pattern-solid flowchart-link LS-query LE-LLM' id='L-query-LLM-0' d='M539.688%2c33L539.688%2c38.667C539.688%2c44.333%2c539.688%2c55.667%2c539.688%2c66.117C539.688%2c76.567%2c539.688%2c86.133%2c539.688%2c90.917L539.688%2c95.7'/%3e%3cpath marker-end='url(%23mermaid-0_flowchart-pointEnd)' style='fill:none%3b' class='edge-thickness-normal edge-pattern-solid flowchart-link LS-LLM LE-agent_github' id='L-LLM-agent_github-0' d='M516.625%2c120.506L444.382%2c129.922C372.138%2c139.337%2c227.651%2c158.169%2c167.807%2c174.329C107.963%2c190.489%2c132.762%2c203.978%2c145.161%2c210.723L157.561%2c217.468'/%3e%3cpath marker-end='url(%23mermaid-0_flowchart-pointEnd)' style='fill:none%3b' class='edge-thickness-normal edge-pattern-solid flowchart-link LS-agent_github LE-agent_github_LLM' id='L-agent_github-agent_github_LLM-0' d='M188.575%2c253L187.571%2c257.167C186.567%2c261.333%2c184.559%2c269.667%2c184.352%2c277.141C184.145%2c284.616%2c185.739%2c291.232%2c186.536%2c294.54L187.333%2c297.847'/%3e%3cpath marker-end='url(%23mermaid-0_flowchart-pointEnd)' style='fill:none%3b' class='edge-thickness-normal edge-pattern-solid flowchart-link LS-agent_github LE-LLM' id='L-agent_github-LLM-0' d='M222.885%2c220L236.06%2c212.833C249.236%2c205.667%2c275.587%2c191.333%2c323.686%2c175.426C371.786%2c159.519%2c441.635%2c142.039%2c476.559%2c133.299L511.484%2c124.558'/%3e%3cpath marker-end='url(%23mermaid-0_flowchart-pointEnd)' style='fill:none%3b' class='edge-thickness-normal edge-pattern-solid flowchart-link LS-agent_github_LLM LE-github_tools' id='L-agent_github_LLM-github_tools-0' d='M188.575%2c336L187.571%2c340.167C186.567%2c344.333%2c184.559%2c352.667%2c184.352%2c360.141C184.145%2c367.616%2c185.739%2c374.232%2c186.536%2c377.54L187.333%2c380.847'/%3e%3cpath marker-end='url(%23mermaid-0_flowchart-pointEnd)' style='fill:none%3b' class='edge-thickness-normal edge-pattern-solid flowchart-link LS-github_tools LE-agent_github_LLM' id='L-github_tools-agent_github_LLM-0' d='M240.973%2c393.293L269.279%2c387.911C297.586%2c382.529%2c354.199%2c371.764%2c350.84%2c360.361C347.482%2c348.958%2c284.151%2c336.917%2c252.485%2c330.896L220.82%2c324.875'/%3e%3cpath marker-end='url(%23mermaid-0_flowchart-pointEnd)' style='fill:none%3b' class='edge-thickness-normal edge-pattern-solid flowchart-link LS-agent_github_LLM LE-agent_github' id='L-agent_github_LLM-agent_github-0' d='M215.613%2c315.115L248.146%2c308.929C280.68%2c302.743%2c345.746%2c290.372%2c351.36%2c279.067C356.974%2c267.763%2c303.135%2c257.526%2c276.216%2c252.408L249.297%2c247.29'/%3e%3cpath marker-end='url(%23mermaid-0_flowchart-pointEnd)' style='fill:none%3b' class='edge-thickness-normal edge-pattern-solid flowchart-link LS-LLM LE-agent_git' id='L-LLM-agent_git-0' d='M539.688%2c134L539.688%2c141.167C539.688%2c148.333%2c539.688%2c162.667%2c550.852%2c176.545C562.017%2c190.423%2c584.347%2c203.846%2c595.512%2c210.558L606.677%2c217.269'/%3e%3cpath marker-end='url(%23mermaid-0_flowchart-pointEnd)' style='fill:none%3b' class='edge-thickness-normal edge-pattern-solid flowchart-link LS-agent_git LE-agent_git_LLM' id='L-agent_git-agent_git_LLM-0' d='M600.473%2c244.126L572.196%2c249.772C543.919%2c255.417%2c487.366%2c266.709%2c489.022%2c278.331C490.678%2c289.953%2c550.543%2c301.905%2c580.475%2c307.881L610.408%2c313.858'/%3e%3cpath marker-end='url(%23mermaid-0_flowchart-pointEnd)' style='fill:none%3b' class='edge-thickness-normal edge-pattern-solid flowchart-link LS-agent_git LE-LLM' id='L-agent_git-LLM-0' d='M666.116%2c220L678.038%2c212.833C689.96%2c205.667%2c713.804%2c191.333%2c697.423%2c175.66C681.041%2c159.986%2c624.433%2c142.972%2c596.129%2c134.464L567.826%2c125.957'/%3e%3cpath marker-end='url(%23mermaid-0_flowchart-pointEnd)' style='fill:none%3b' class='edge-thickness-normal edge-pattern-solid flowchart-link LS-agent_git_LLM LE-git_tools' id='L-agent_git_LLM-git_tools-0' d='M615.605%2c324.105L584.807%2c330.254C554.008%2c336.403%2c492.41%2c348.702%2c489.543%2c360.427C486.675%2c372.153%2c542.538%2c383.307%2c570.469%2c388.884L598.4%2c394.46'/%3e%3cpath marker-end='url(%23mermaid-0_flowchart-pointEnd)' style='fill:none%3b' class='edge-thickness-normal edge-pattern-solid flowchart-link LS-git_tools LE-agent_git_LLM' id='L-git_tools-agent_git_LLM-0' d='M642.644%2c386L643.648%2c381.833C644.652%2c377.667%2c646.66%2c369.333%2c646.867%2c361.859C647.074%2c354.384%2c645.48%2c347.768%2c644.683%2c344.46L643.885%2c341.153'/%3e%3cpath marker-end='url(%23mermaid-0_flowchart-pointEnd)' style='fill:none%3b' class='edge-thickness-normal edge-pattern-solid flowchart-link LS-agent_git_LLM LE-agent_git' id='L-agent_git_LLM-agent_git-0' d='M642.644%2c303L643.648%2c298.833C644.652%2c294.667%2c646.66%2c286.333%2c646.867%2c278.859C647.074%2c271.384%2c645.48%2c264.768%2c644.683%2c261.46L643.885%2c258.153'/%3e%3cpath marker-end='url(%23mermaid-0_flowchart-pointEnd)' style='fill:none%3b' class='edge-thickness-normal edge-pattern-solid flowchart-link LS-LLM LE-response' id='L-LLM-response-0' d='M562.75%2c120.93L625.586%2c130.275C688.422%2c139.62%2c814.094%2c158.31%2c876.93%2c173.938C939.766%2c189.567%2c939.766%2c202.133%2c939.766%2c208.417L939.766%2c214.7'/%3e%3c/g%3e%3cg class='edgeLabels'%3e%3cg transform='translate(539.6875%2c 67)' class='edgeLabel'%3e%3cg transform='translate(-76.4921875%2c -9)' class='label'%3e%3cforeignObject height='18' width='152.984375'%3e%3cdiv style='display: inline-block%3b white-space: nowrap%3b' xmlns='http://www.w3.org/1999/xhtml'%3e%3cspan class='edgeLabel'%3ewhy did the build fail%3f%3c/span%3e%3c/div%3e%3c/foreignObject%3e%3c/g%3e%3c/g%3e%3cg transform='translate(83.1640625%2c 177)' class='edgeLabel'%3e%3cg transform='translate(-83.1640625%2c -9)' class='label'%3e%3cforeignObject height='18' width='166.328125'%3e%3cdiv style='display: inline-block%3b white-space: nowrap%3b' xmlns='http://www.w3.org/1999/xhtml'%3e%3cspan class='edgeLabel'%3equery the last failed run%3c/span%3e%3c/div%3e%3c/foreignObject%3e%3c/g%3e%3c/g%3e%3cg class='edgeLabel'%3e%3cg transform='translate(0%2c 0)' class='label'%3e%3cforeignObject height='0' width='0'%3e%3cdiv style='display: inline-block%3b white-space: nowrap%3b' xmlns='http://www.w3.org/1999/xhtml'%3e%3cspan class='edgeLabel'%3e%3c/span%3e%3c/div%3e%3c/foreignObject%3e%3c/g%3e%3c/g%3e%3cg transform='translate(301.9375%2c 177)' class='edgeLabel'%3e%3cg transform='translate(-115.609375%2c -18)' class='label'%3e%3cforeignObject height='36' width='231.21875'%3e%3cdiv style='display: inline-block%3b white-space: nowrap%3b' xmlns='http://www.w3.org/1999/xhtml'%3e%3cspan class='edgeLabel'%3ecommit failed_sha is responsible%3cbr /%3esuccess_sha is last known good%3c/span%3e%3c/div%3e%3c/foreignObject%3e%3c/g%3e%3c/g%3e%3cg class='edgeLabel'%3e%3cg transform='translate(0%2c 0)' class='label'%3e%3cforeignObject height='0' width='0'%3e%3cdiv style='display: inline-block%3b white-space: nowrap%3b' xmlns='http://www.w3.org/1999/xhtml'%3e%3cspan class='edgeLabel'%3e%3c/span%3e%3c/div%3e%3c/foreignObject%3e%3c/g%3e%3c/g%3e%3cg class='edgeLabel'%3e%3cg transform='translate(0%2c 0)' class='label'%3e%3cforeignObject height='0' width='0'%3e%3cdiv style='display: inline-block%3b white-space: nowrap%3b' xmlns='http://www.w3.org/1999/xhtml'%3e%3cspan class='edgeLabel'%3e%3c/span%3e%3c/div%3e%3c/foreignObject%3e%3c/g%3e%3c/g%3e%3cg class='edgeLabel'%3e%3cg transform='translate(0%2c 0)' class='label'%3e%3cforeignObject height='0' width='0'%3e%3cdiv style='display: inline-block%3b white-space: nowrap%3b' xmlns='http://www.w3.org/1999/xhtml'%3e%3cspan class='edgeLabel'%3e%3c/span%3e%3c/div%3e%3c/foreignObject%3e%3c/g%3e%3c/g%3e%3cg transform='translate(539.6875%2c 177)' class='edgeLabel'%3e%3cg transform='translate(-102.140625%2c -9)' class='label'%3e%3cforeignObject height='18' width='204.28125'%3e%3cdiv style='display: inline-block%3b white-space: nowrap%3b' xmlns='http://www.w3.org/1999/xhtml'%3e%3cspan class='edgeLabel'%3ediff success_sha...failed_sha%3c/span%3e%3c/div%3e%3c/foreignObject%3e%3c/g%3e%3c/g%3e%3cg class='edgeLabel'%3e%3cg transform='translate(0%2c 0)' class='label'%3e%3cforeignObject height='0' width='0'%3e%3cdiv style='display: inline-block%3b white-space: nowrap%3b' xmlns='http://www.w3.org/1999/xhtml'%3e%3cspan class='edgeLabel'%3e%3c/span%3e%3c/div%3e%3c/foreignObject%3e%3c/g%3e%3c/g%3e%3cg transform='translate(737.6484375%2c 177)' class='edgeLabel'%3e%3cg transform='translate(-75.8203125%2c -9)' class='label'%3e%3cforeignObject height='18' width='151.640625'%3e%3cdiv style='display: inline-block%3b white-space: nowrap%3b' xmlns='http://www.w3.org/1999/xhtml'%3e%3cspan class='edgeLabel'%3ein file: %5b50%5d %2b foo.bar()%3c/span%3e%3c/div%3e%3c/foreignObject%3e%3c/g%3e%3c/g%3e%3cg class='edgeLabel'%3e%3cg transform='translate(0%2c 0)' class='label'%3e%3cforeignObject height='0' width='0'%3e%3cdiv style='display: inline-block%3b white-space: nowrap%3b' xmlns='http://www.w3.org/1999/xhtml'%3e%3cspan class='edgeLabel'%3e%3c/span%3e%3c/div%3e%3c/foreignObject%3e%3c/g%3e%3c/g%3e%3cg class='edgeLabel'%3e%3cg transform='translate(0%2c 0)' class='label'%3e%3cforeignObject height='0' width='0'%3e%3cdiv style='display: inline-block%3b white-space: nowrap%3b' xmlns='http://www.w3.org/1999/xhtml'%3e%3cspan class='edgeLabel'%3e%3c/span%3e%3c/div%3e%3c/foreignObject%3e%3c/g%3e%3c/g%3e%3cg class='edgeLabel'%3e%3cg transform='translate(0%2c 0)' class='label'%3e%3cforeignObject height='0' width='0'%3e%3cdiv style='display: inline-block%3b white-space: nowrap%3b' xmlns='http://www.w3.org/1999/xhtml'%3e%3cspan class='edgeLabel'%3e%3c/span%3e%3c/div%3e%3c/foreignObject%3e%3c/g%3e%3c/g%3e%3cg transform='translate(939.765625%2c 177)' class='edgeLabel'%3e%3cg transform='translate(-106.296875%2c -9)' class='label'%3e%3cforeignObject height='18' width='212.59375'%3e%3cdiv style='display: inline-block%3b white-space: nowrap%3b' xmlns='http://www.w3.org/1999/xhtml'%3e%3cspan class='edgeLabel'%3ethe change at line 50 is wrong%3c/span%3e%3c/div%3e%3c/foreignObject%3e%3c/g%3e%3c/g%3e%3c/g%3e%3cg class='nodes'%3e%3cg transform='translate(539.6875%2c 16.5)' data-id='query' data-node='true' id='flowchart-query-0' class='node default default flowchart-label'%3e%3crect height='33' width='55.03125' y='-16.5' x='-27.515625' ry='0' rx='0' style='' class='basic label-container'/%3e%3cg transform='translate(-20.015625%2c -9)' style='' class='label'%3e%3crect/%3e%3cforeignObject height='18' width='40.03125'%3e%3cdiv style='display: inline-block%3b white-space: nowrap%3b' xmlns='http://www.w3.org/1999/xhtml'%3e%3cspan class='nodeLabel'%3equery%3c/span%3e%3c/div%3e%3c/foreignObject%3e%3c/g%3e%3c/g%3e%3cg transform='translate(539.6875%2c 117.5)' data-id='LLM' data-node='true' id='flowchart-LLM-1' class='node default default flowchart-label'%3e%3crect height='33' width='46.125' y='-16.5' x='-23.0625' ry='0' rx='0' style='' class='basic label-container'/%3e%3cg transform='translate(-15.5625%2c -9)' style='' class='label'%3e%3crect/%3e%3cforeignObject height='18' width='31.125'%3e%3cdiv style='display: inline-block%3b white-space: nowrap%3b' xmlns='http://www.w3.org/1999/xhtml'%3e%3cspan class='nodeLabel'%3eLLM%3c/span%3e%3c/div%3e%3c/foreignObject%3e%3c/g%3e%3c/g%3e%3cg transform='translate(192.55078125%2c 236.5)' data-id='agent_github' data-node='true' id='flowchart-agent_github-3' class='node default default flowchart-label'%3e%3crect height='33' width='103.078125' y='-16.5' x='-51.5390625' ry='0' rx='0' style='' class='basic label-container'/%3e%3cg transform='translate(-44.0390625%2c -9)' style='' class='label'%3e%3crect/%3e%3cforeignObject height='18' width='88.078125'%3e%3cdiv style='display: inline-block%3b white-space: nowrap%3b' xmlns='http://www.w3.org/1999/xhtml'%3e%3cspan class='nodeLabel'%3eagent github%3c/span%3e%3c/div%3e%3c/foreignObject%3e%3c/g%3e%3c/g%3e%3cg transform='translate(192.55078125%2c 319.5)' data-id='agent_github_LLM' data-node='true' id='flowchart-agent_github_LLM-5' class='node default default flowchart-label'%3e%3crect height='33' width='46.125' y='-16.5' x='-23.0625' ry='0' rx='0' style='' class='basic label-container'/%3e%3cg transform='translate(-15.5625%2c -9)' style='' class='label'%3e%3crect/%3e%3cforeignObject height='18' width='31.125'%3e%3cdiv style='display: inline-block%3b white-space: nowrap%3b' xmlns='http://www.w3.org/1999/xhtml'%3e%3cspan class='nodeLabel'%3eLLM%3c/span%3e%3c/div%3e%3c/foreignObject%3e%3c/g%3e%3c/g%3e%3cg transform='translate(192.55078125%2c 402.5)' data-id='github_tools' data-node='true' id='flowchart-github_tools-9' class='node default default flowchart-label'%3e%3crect height='33' width='96.84375' y='-16.5' x='-48.421875' ry='0' rx='0' style='' class='basic label-container'/%3e%3cg transform='translate(-40.921875%2c -9)' style='' class='label'%3e%3crect/%3e%3cforeignObject height='18' width='81.84375'%3e%3cdiv style='display: inline-block%3b white-space: nowrap%3b' xmlns='http://www.w3.org/1999/xhtml'%3e%3cspan class='nodeLabel'%3egithub tools%3c/span%3e%3c/div%3e%3c/foreignObject%3e%3c/g%3e%3c/g%3e%3cg transform='translate(638.66796875%2c 236.5)' data-id='agent_git' data-node='true' id='flowchart-agent_git-15' class='node default default flowchart-label'%3e%3crect height='33' width='76.390625' y='-16.5' x='-38.1953125' ry='0' rx='0' style='' class='basic label-container'/%3e%3cg transform='translate(-30.6953125%2c -9)' style='' class='label'%3e%3crect/%3e%3cforeignObject height='18' width='61.390625'%3e%3cdiv style='display: inline-block%3b white-space: nowrap%3b' xmlns='http://www.w3.org/1999/xhtml'%3e%3cspan class='nodeLabel'%3eagent git%3c/span%3e%3c/div%3e%3c/foreignObject%3e%3c/g%3e%3c/g%3e%3cg transform='translate(638.66796875%2c 319.5)' data-id='agent_git_LLM' data-node='true' id='flowchart-agent_git_LLM-17' class='node default default flowchart-label'%3e%3crect height='33' width='46.125' y='-16.5' x='-23.0625' ry='0' rx='0' style='' class='basic label-container'/%3e%3cg transform='translate(-15.5625%2c -9)' style='' class='label'%3e%3crect/%3e%3cforeignObject height='18' width='31.125'%3e%3cdiv style='display: inline-block%3b white-space: nowrap%3b' xmlns='http://www.w3.org/1999/xhtml'%3e%3cspan class='nodeLabel'%3eLLM%3c/span%3e%3c/div%3e%3c/foreignObject%3e%3c/g%3e%3c/g%3e%3cg transform='translate(638.66796875%2c 402.5)' data-id='git_tools' data-node='true' id='flowchart-git_tools-21' class='node default default flowchart-label'%3e%3crect height='33' width='70.140625' y='-16.5' x='-35.0703125' ry='0' rx='0' style='' class='basic label-container'/%3e%3cg transform='translate(-27.5703125%2c -9)' style='' class='label'%3e%3crect/%3e%3cforeignObject height='18' width='55.140625'%3e%3cdiv style='display: inline-block%3b white-space: nowrap%3b' xmlns='http://www.w3.org/1999/xhtml'%3e%3cspan class='nodeLabel'%3egit tools%3c/span%3e%3c/div%3e%3c/foreignObject%3e%3c/g%3e%3c/g%3e%3cg transform='translate(939.765625%2c 236.5)' data-id='response' data-node='true' id='flowchart-response-27' class='node default default flowchart-label'%3e%3crect height='33' width='80.828125' y='-16.5' x='-40.4140625' ry='0' rx='0' style='' class='basic label-container'/%3e%3cg transform='translate(-32.9140625%2c -9)' style='' class='label'%3e%3crect/%3e%3cforeignObject height='18' width='65.828125'%3e%3cdiv style='display: inline-block%3b white-space: nowrap%3b' xmlns='http://www.w3.org/1999/xhtml'%3e%3cspan class='nodeLabel'%3eresponse%3c/span%3e%3c/div%3e%3c/foreignObject%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/svg%3e)
+