REQ-O-049: LLM Token Budget Flags

Tier: Opt-In | Priority: P2

Source: §4 Verbosity & Token Cost · §43 Tool Output Result Size Unboundedness

Addresses: Severity: Medium–Critical / Token Spend: High / Time: Low / Context: High

Description

The framework MUST provide three flags that let an LLM caller manage how much output lands in its context window:

--token-limit <n>: truncate output to the first n tokens and emit meta.truncated: true with meta.token_limit: n in the response envelope. The truncation MUST be clean (never mid-token or mid-field) and MUST be indicated by a truncation sentinel appended to the data.
--token-count: instead of emitting the command output, emit only the token count that the output would consume (meta.token_count: n, data: null). The command MUST execute fully to produce the count, but MUST NOT write the payload to stdout.
--token-offset <n>: skip the first n tokens of output before beginning emission. When combined with --token-limit, this enables sliding-window access over output that exceeds the caller's context budget.

Token counting MUST use the same tokenizer as the primary consumer (typically cl100k_base or the framework's declared default). If the tokenizer is configurable, --tokenizer <name> selects it.

Command-specific formats. A command that accepts --format values beyond the framework's lists them in its manifest entry's output_formats. Beside it, output_media_types maps a format value to the media type it writes ({"toon": "text/plain"}). It is required for every output_formats value that neither the spec's media type table (REQ-O-001) nor the root format flag's media_types covers, and an entry overrides the root map for that command. Every key is a value the command accepts for --format. output_formats keeps its string-list type.

These flags are distinct from --fields (which filters by key) and the framework's byte-level hard cap (REQ-F-052). They operate on the token dimension, which is what LLM callers actually spend.

Acceptance Criteria

--token-limit 500 produces output that, when tokenized, contains ≤500 tokens; meta.truncated: true and meta.token_limit: 500 are present in the envelope
--token-count returns data: null and meta.token_count: N without writing the payload
--token-offset 200 --token-limit 200 returns the second window of 200 tokens
All three flags are available on every command without per-command implementation
meta.token_count is present in every response when --token-count is passed, regardless of --format value
A command whose output_formats lists a value covered by neither the spec's media type table nor the root media_types declares it in output_media_types

Schema

Type: response-envelope.md

Requirement-specific meta fields:

Field	Type	When present
`meta.truncated`	boolean	When `--token-limit` truncated the payload
`meta.token_limit`	integer	When `--token-limit` was passed
`meta.token_count`	integer	When `--token-count` was passed
`meta.token_offset`	integer	When `--token-offset` was nonzero

Type: manifest-response.md

CommandEntry.output_formats (string array) lists command-specific format values, and CommandEntry.output_media_types (map from format value to media type) declares what each writes.

Wire Format

$ tool list-events --token-limit 500 --format json

{
  "ok": true,
  "data": ["...first 500 tokens of events...", "[TRUNCATED]"],
  "error": null,
  "warnings": [],
  "meta": {
    "exit_code": 0,
    "truncated": true,
    "token_limit": 500,
    "token_offset": 0,
    "duration_ms": 84
  }
}

$ tool list-events --token-count --format json

{
  "ok": true,
  "data": null,
  "error": null,
  "warnings": [],
  "meta": {
    "exit_code": 0,
    "token_count": 4217,
    "duration_ms": 61
  }
}

Sliding window over large output:

$ tool list-events --token-offset 500 --token-limit 500 --format json
# Returns tokens 500–999 of the full output

Example

Opt-in at the framework level; token counting uses the declared tokenizer.

app = Framework("tool")
app.enable_token_budget(tokenizer="cl100k_base")

# Agent decides whether to fetch before committing context budget:
count_result = run("tool list-events --token-count --format json")
if count_result["meta"]["token_count"] > context_budget:
    # Fetch in windows instead
    run(f"tool list-events --token-offset 0 --token-limit {context_budget} --format json")
else:
    run("tool list-events --format json")

Requirement	Tier	Relationship
REQ-F-052	F	Provides: byte-level hard cap; token budget flags operate on the token dimension instead
REQ-O-002	O	Composes: `--fields` reduces the key space before token limiting reduces the token count
REQ-O-001	O	Composes: token budget flags apply after format selection, not before
schemas/manifest-response.md	Exposes: `output_formats` and `output_media_types` in `CommandEntry` declare when LLM-optimized formats are available and what each writes