Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add columns with cost details to Results #1254

Open
rbyh opened this issue Nov 10, 2024 · 3 comments
Open

Add columns with cost details to Results #1254

rbyh opened this issue Nov 10, 2024 · 3 comments
Assignees
Labels
enhancement New feature or request

Comments

@rbyh
Copy link
Contributor

rbyh commented Nov 10, 2024

We should add add the following columns to results, especially as they are not presented identically in raw responses across models:

Prefix: "cost" (keep this single to clarify it is per response)
Sub-fields:

  • "input_tokens": input tokens for the individual result
  • "output_tokens": output tokens for the individual result
  • "cache_used": true / false
  • "credits": total credits consumed to generate the result (either when it was cached or in current results, so that non-owner can estimate costs)
@rbyh rbyh added the enhancement New feature or request label Nov 10, 2024
@johnjosephhorton
Copy link
Contributor

Thanks @rbyh - I like this - just trying to think about the right syntax. Right now, because the 'fields" have to be globally unique, I append a string to a question-specific field e.g., I do generated_tokens.how_feeling_generated_tokens
How would you like this organized? Maybe give some examples of the syntax you'd like to use?

@rbyh
Copy link
Contributor Author

rbyh commented Nov 11, 2024

cost.how_feeling_input_tokens
cost.how_feeling_output_tokens
cost.how_feeling_cache_used
cost.how_feeling_credits

@rbyh
Copy link
Contributor Author

rbyh commented Nov 14, 2024

Related open issue: #998

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants