🤖 feat: add 5% buffer between auto-compaction threshold and force-compaction #840
Conversation
ethanndickson
force-pushed
the
force-compaction-threshold-buffer
branch
from
9b4ae34 to
8c70240
Compare
ethanndickson
force-pushed
the
force-compaction-threshold-buffer
branch
from
8c70240 to
0f6c5ae
Compare
|
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
…ase) Previously shouldShowWarning only used lastUsagePercentage, so the warning never appeared during the first streaming response. Now uses max of last and live usage, ensuring the countdown renders when it should.
|
@codex review |
|
Codex Review: Didn't find any major issues. You're on a roll. ℹ️ About Codex in GitHubCodex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback". |
1 participant
Background
Auto-compaction runs after a response completes when context usage exceeds a user-configured threshold (default 70%). It summarizes the conversation to free up context space while preserving important information.
Force-compaction is a safety mechanism that triggers during streaming when context usage gets dangerously high. Unlike auto-compaction which waits for a natural break point, force-compaction interrupts the current response to prevent hitting the context window limit.
The Problem
Previously, force-compaction triggered based on a fixed token buffer from the context window limit. This meant force-compaction timing was disconnected from the user's configured auto-compaction threshold—changing your threshold didn't affect when force-compaction would kick in.
The Change
Force-compaction now triggers at threshold + 5%. With a 70% threshold, force-compaction happens at 75%.
This gives users a predictable buffer zone between when auto-compaction would run (after the response) and when force-compaction will run (during streaming). If a response slightly overshoots the threshold, the chat has some leeway—it won't immediately force-compact just because usage landed a bit over. This avoids unnecessary force-compactions when the stream ends and usage settles back within acceptable bounds.
The buffer is intentionally small (5%) to balance user control with safety margins as context approaches capacity.
UI Change
Also adds a "Force-compacting in N%" countdown during streaming when usage is in the buffer zone, so users know how much room remains before force-compaction triggers.
Generated with
mux