Skip to content

fix: use full Kiro payload for prompt token counting#102

Open
juslintek wants to merge 1 commit intojwadow:mainfrom
juslintek:fix/prompt-token-counting
Open

fix: use full Kiro payload for prompt token counting#102
juslintek wants to merge 1 commit intojwadow:mainfrom
juslintek:fix/prompt-token-counting

Conversation

@juslintek
Copy link

Problem

The prompt_tokens reported to clients (used by Claude Code's /context command) were wildly inaccurate because they were derived from Kiro's contextUsagePercentage, which returns unreliable values.

Solution

Count tokens from the complete serialized Kiro request payload using tiktoken. This includes system prompt, messages, tools, and all other payload fields — matching what actually gets sent to the API.

Changes

  • Replace request_messages/request_tools params with pre-counted prompt_tokens across all streaming functions
  • Count tokens from full kiro_request_body in both OpenAI and Anthropic route handlers
  • Remove dependency on contextUsagePercentage for token counting
  • Update tests to match new function signatures
The prompt_tokens reported to clients (used by Claude Code's /context command) were wildly inaccurate because they were derived from Kiro's contextUsagePercentage, which returns unreliable values. Instead, count tokens from the complete serialized Kiro request payload using tiktoken. This includes system prompt, messages, tools, and all other payload fields — matching what actually gets sent to the API. - Replace request_messages/request_tools params with pre-counted prompt_tokens across all streaming functions - Count tokens from full kiro_request_body in both OpenAI and Anthropic route handlers - Remove dependency on contextUsagePercentage for token counting - Update tests to match new function signatures
@cla-bot
Copy link

cla-bot bot commented Mar 20, 2026

Thanks for the PR! 🎉

Before merge, we need a one-time CLA confirmation.
It confirms that you have the right to contribute this code and allow the project to use it.

Full CLA text:
https://github.com/jwadow/kiro-gateway/blob/main/CLA.md

Please reply once with:

I have read the CLA and I accept its terms 

You need to write once, all further messages from me can be ignored.

@juslintek
Copy link
Author

I have read the CLA and I accept its terms

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants