Skip to content

Commit

Permalink
Fix bug where Gemini STOP token causes crash (#68)
Browse files Browse the repository at this point in the history
Sometimes Gemini sends a STOP token here during normal processing. When this happens, `candidate.content.parts` is undefined, which causes the loop below to fail and crash generation with an exception:

```
Error summarizing message TypeError: candidate.content.parts is not iterable
    at file:///Users/josh/Projects/bigb/api/node_modules/.deno/@ax-llm[email protected]/node_modules/@ax-llm/ax/ai/google-gemini/api.js:266:50
    at Array.map (<anonymous>)
    at AxAIGoogleGemini.generateChatResp (file:///Users/josh/Projects/bigb/api/node_modules/.deno/@ax-llm[email protected]/node_modules/@ax-llm/ax/ai/google-gemini/api.js:249:42)
    at generateChatStreamResp (file:///Users/josh/Projects/bigb/api/node_modules/.deno/@ax-llm[email protected]/node_modules/@ax-llm/ax/ai/google-gemini/api.js:300:21)
    at TypeTransformer.transformFn (file:///Users/josh/Projects/bigb/api/node_modules/.deno/@ax-llm[email protected]/node_modules/@ax-llm/ax/ai/base.js:188:29)
    at TypeTransformer.transform (file:///Users/josh/Projects/bigb/api/node_modules/.deno/@ax-llm[email protected]/node_modules/@ax-llm/ax/util/transform.js:12:26)
    at Module.invokeCallbackFunction (ext:deno_webidl/00_webidl.js:982:16)
    at TransformStreamDefaultController.transformAlgorithm (ext:deno_web/06_streams.js:3789:14)
    at transformStreamDefaultControllerPerformTransform (ext:deno_web/06_streams.js:4058:59)
    at transformStreamDefaultSinkWriteAlgorithm (ext:deno_web/06_streams.js:4161:10)
```

The `candidate` object looks like this in these cases:

```
{ content: { role: "model" }, finishReason: "STOP", index: 0 }
```
  • Loading branch information
joshvfleming authored Nov 2, 2024
1 parent f82ea5d commit 11b07c6
Showing 1 changed file with 4 additions and 0 deletions.
4 changes: 4 additions & 0 deletions src/ax/ai/google-gemini/api.ts
Original file line number Diff line number Diff line change
Expand Up @@ -378,6 +378,10 @@ export class AxAIGoogleGemini extends AxBaseAI<
throw new Error('Finish reason: RECITATION');
}

if (!candidate.content.parts) {
return result;
}

for (const part of candidate.content.parts) {
if ('text' in part) {
result.content = part.text;
Expand Down

0 comments on commit 11b07c6

Please sign in to comment.