Skip to content

Commit

Permalink
Fix for negative tokens
Browse files Browse the repository at this point in the history
Signed-off-by: Igor Gitman <[email protected]>
  • Loading branch information
Kipok committed Nov 24, 2024
1 parent e69d19b commit b9710ee
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion nemo_skills/inference/server/code_execution_model.py
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,8 @@ def _generate_single(
request['tokens_to_generate'] -= num_generated_tokens
# TODO: currently we don't account for tokens in the code output that we add to the prompt
# in most cases the output should be small though

if request['tokens_to_generate'] <= 0:
break
# .rfind(code_end, 0, -1) searches for the second-to-last occurrence of code_end and checks
# that the last code_begin is not closed to ensure that we are inside the code block
if output.endswith(code_end) and output.rfind(code_begin) > output.rfind(code_end, 0, -1):
Expand Down

0 comments on commit b9710ee

Please sign in to comment.