Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better handle very long transcripts #6

Open
domdomegg opened this issue Oct 9, 2024 · 0 comments
Open

Better handle very long transcripts #6

domdomegg opened this issue Oct 9, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@domdomegg
Copy link
Member

There is some limit of how big a request you can send to Airtable / how much data you can put in a field. If an application is very long, and then the AI evaluator copies it 5 times to evaluate it for 5 different criteria, and then tries to put it in the one notes field (with the (optional) Logs field thing), the API rejects it.

It would be nice to either:

  • store the logs without copying the application so much
  • if it will be too long, truncating it before putting it in the field / not setting it at all? (this seems better than just silently failing - especially because the errors are not obvious, see Better surface errors to users #4. and maybe better than noisily failing, because then you need to re-run the evaluator)

The workaround for now is to run it on everyone with the logs field set, and this will work for everyone apart from the long applications. Then for the long applications we re-run evaluations with the logs field unset. But this adds extra effort and means we have no logs for these people.

@domdomegg domdomegg added the enhancement New feature or request label Oct 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant