Expose cached tokens in the RequestUsage class along with input/output tokens #7111
isotopes-sbv started this conversation in Feature suggestions
Replies: 0 comments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
The total number of cached tokens would really help us to see the efficiency of the model.
Beta Was this translation helpful? Give feedback.
All reactions