Technical report · TR-2026-08-29
Context tokens sent to the model, with and without Hexum
Last updated 12 September 2026 · Estimator ceil(utf8_bytes / 4) · Not a provider invoice
Figure 1
Published card
| Pass | ||||
|---|---|---|---|---|
| 220,393 | 27,352 | 87.6% | 24 / 24 | |
| 18,200 | 3,100 | 83.0% | n/a | |
| 132,807 | 14,379 | 89.2% | 12 / 12 | |
| 87,586 | 12,973 | 85.2% | 12 / 12 |
Study I
MBPP and HumanEval
24-task card. 193,041 tokens not sent. $0.88 → $0.11 at $4 / 1M input.
Study II
LedgerKit · five scenarios
Where is overage calculated, who calls it, what breaks if the free-tier cap moves?
Default inspect 32,768 → 3,220 (90.2%). Hot file stays almost full: 1,276 → 1,242. Budget 3,000 is a second measured point (92.9%), not an interpolation.
- vs dump
- 32,768 → 3,220 90.2%
- vs focused
- 32,768 → 3,220 90.2%
- edit bandwidth
- — → — —
- S1 budget 3k
- 32,768 → 2,331 92.9%
Dump analog in the 89–93% band. This is the published “up to 90%” regime.
Figure 3
Cut matrix
Figure 4
When 90% shows up
The 80–93% band is dump-vs-shrink or splice-vs-file. Left of that is authoring and “I already know the four files.” Click a tick to load that row in Fig. 1.
Figure 5
Scale Study I combined
10,000 × 24-task suites · 220,393 vs 27,352 tokens each
Protocol
| Item | Study I | Study II |
|---|---|---|
| What is counted | Tokens sent to the model | Same estimator on dump vs shrink / expand / splice |
| Estimator | ceil(utf8_bytes / 4) | ceil(utf8_bytes / 4) |
| n | 24 tasks (12 + 12) | 5 scenarios × 2 sides on one 43-file seed |
| Model | GPT-5.6 Sol | Engine only (no LLM in the token count) |
| Pass | 24 / 24 official tests | Planted overage test for S2/S5 |
| Not counted | Thought tokens, SLOC, disk KB, provider invoices, skill dialect | |
Output-token savings from the skill are not a published study. Tiny prompts may save little.
Hosted MCP rejects more than 64 files and currently forces since_hash to none.
Full write-up: methodology.
FAQ
How does Hexum cut what I pay the model?
Your agent stops dumping the whole repo into the prompt. Hexum sends this task, not every file. On the published card, MBPP context dropped 85.2% and the combined 24-task cut is 87.6%, with 24/24 tests still passing. Tokens are ceil(utf8_bytes/4), not a provider invoice. See /benchmarks.
Did the tests still pass?
Yes on the published card: 12 / 12 MBPP and 12 / 12 HumanEval. LedgerKit S2/S5 is a planted overage assertion; the splice restores extra == 0 at the free-tier cap.
Is that a cloud invoice?
No. Tokens sent to the model, estimated as ceil(utf8_bytes / 4). Output-token savings from the skill are not a published study.
When is the cut not ~85%?
Greenfield authoring (S4) is 0%: new source cannot be shrunk. A focused agent that already opens four files can still beat a whole-repo shrink (S3). Dump-vs-shrink on LedgerKit inspect is 90.2% by default, 92.9% with a 3,000-token budget.
What does the dashboard mean by tokens saved?
How much smaller the reply was than what the agent sent, when that is a cut. Your invoice is still based on what the agent sent Hexum, not on the savings.