Compressing Base Context from 30k to 10k for Local LLM Prefill: Coding Harness 18.3.0 Optimization
Coding agent harness 18.3.0 slashes base prompt token usage from ~30k to 10k for faster local model prefill, consolidates tools into read/write/bash, enables de
Coding agent harness developer Can Bölük (@_can1357) has shared a major efficiency update in version 18.3.0, cutting baseline prompt context from approximately 30,000 to 10,000 tokens. The release significantly accelerates local model prefill times by trimming prompt overhead, consolidating tool interfaces, compressing skill descriptions by default, and integrating Anthropic's latest on-demand compaction.
Image source: Can Bölük (@_can1357) / X
When deploying coding agents against locally hosted LLMs, one of the most prominent performance bottlenecks has been the substantial prefill latency caused by heavy initial system instructions and sprawling tool schemas. Version 18.3.0 directly tackles this operational friction, cutting baseline context load to one-third of its previous size to deliver a markedly more responsive local inference workflow.
Slashing Base Tokens from 30k to 10k and Streamlining Core Tools
Even on dedicated workstation GPUs, computing the prompt prefill for a 30k base token context on every turn introduces perceptible latency and heavy VRAM pressure. The 18.3.0 release executes an aggressive diet across default system prompts and schemas:
- Baseline Token Usage Reduced from ~30k to 10k: By refactoring foundational guidelines and eliminating unnecessary verbosity, initial prompt size is restricted to approximately 10,000 tokens. As noted by the developer, this provides a much more pleasant experience with local models regarding prefill latency.
- Folding Multiple Tools into
read/write/bash: Granular file manipulation and helper tools have been consolidated into three atomic tool primitives (read,write, andbash). This removes substantial tool definition schema overhead from the context window, reducing model confusion and improving tool-calling precision. - Skill Descriptions Compressed by Default: Extensible skills and agent capabilities now have their catalog descriptions compressed by default, preventing context bloat as developers register additional specialized workflows.
Anthropic On-Demand Compaction and Refusal Mitigation via Entitlement Detection
The update also refines interactions with cloud frontier models and specialized security workflows:
- Anthropic On-Demand Compaction Support: The harness natively incorporates the latest on-demand compaction mechanism introduced by Anthropic, helping long-running interactive sessions stay within context window budgets without losing critical conversational state.
- Codex Cyber Access Entitlement Detection: The system now detects whether a user account holds Codex cyber access entitlements and explicitly requests those capabilities even when querying non-daybreak models. This targeted signaling helps eligible developers avoid false-positive refusals during legitimate security-related tasks.
Apple AFM Support, TypeSafe Jev 1.13 Offline Counting, and Gateway Expansion
In addition to local server optimizations, version 18.3.0 broadens on-device runtime coverage and multi-modal tool endpoints:
- Apple Foundational Models (AFM) Support: Initial integration for Apple Foundational Models is now supported. Because AFM currently provides an 8K context window, it is primarily viable as a tiny local model. Can Bölük added in an author note that "Apple AFM 3 Core Advanced can only do 8K ctx at 5 toks/s," establishing clear hardware throughput baselines.
- TypeSafe Jev 1.13 Offline Token Counting: The release integrates local, offline token counting for the TypeSafe Jev 1.13 decision engine, removing external API calls and network latency for token tallying.
- Unified Media Auth Gateway Support: The harness broadens its authenticated gateway layer to support media generation and processing services, covering
systemone, image generation (image-gen), video generation (video-gen), and transcription (transcription).
The architectural adjustments in harness 18.3.0 highlight a decisive trend in agent engineering: reducing systemic overhead through concise core primitives, compressed tool schemas, and streamlined context windows to make both local models and high-end cloud LLMs operate with greater efficiency.