resend — CLI Agent Evaluation
Evaluated against the CLI Agent Spec — a specification defining 74 failure modes for CLI tools used under AI agent orchestration.
CLI version: resend-cli v2.8.1 Evaluated: 2026-07-06 Scope: Critical (22 of 74 failure modes)
Note: this run covered the 22 Critical failure modes as scoped at evaluation time. §75 (Safe-Default Execution Mode Absent), a Critical mode added in the same cycle, was not evaluated.
Scores
| Metric | Result |
|---|---|
| Failure mode score | 1.2/3 — 4 passing · 12 partial · 5 failing |
| Readiness score | 14/15 [A] |
| Observed bugs | 6 confirmed during live evaluation |
| Worst gaps | §1 Exit Codes, §25 Prompt Injection, §43 Output Size, §64 Headless GUI, §74 Credential Scopes |
Key Findings
whoamican report a fake flag-provided key as authenticated, so credential presence can be mistaken for validity.- Browser-opening commands still invoke the OS opener even in quiet/JSON contexts and do not emit a structured URL fallback.
- Dry-run output can include large file contents verbatim with no truncation metadata or output cap.
- User-provided HTML is returned without an untrusted-content marker, creating prompt-injection risk for agents.
- Resend is strong on proactive readiness: it ships a co-versioned agent skill, detailed references, workflows, and a JSON command tree.
Files
| File | What it is |
|---|---|
| report-index.md | Full scorecard — all failure modes, readiness breakdown, links to all reports |
| report-issues.md | Concrete bugs and gaps agents will hit when using this CLI as-is |
| report-runtime.md | Compact operational brief — what to set, what to avoid, what to watch for |
| report-agent-dev.md | Integration guide — invocation invariants and per-gap workarounds for agent developers |
| report-dev.md | Fix list for CLI authors — what to implement, mapped to spec requirements |
| findings.md | Raw scorecard — one row per evaluated failure mode |
| issues.md | Observed bugs recorded during live evaluation |
| trace.md | Audit trail — exact check commands, exit codes, stdout/stderr per §N |
| environment.md | CLI environment profile — binary path, version, flags, timeout method |
| readiness.md | Proactive readiness scores across 5 dimensions |
Generated by cli-agent-audit · CLI Agent Spec