See the gap between what an agent should do and what its static tool authority can do.
A reproducible census of ten open-source agent applications and four adapter-calibration repositories—without a risk score or project ranking.
Sandbox controls can.
Constrains runtime capability.
Evals measure does.
Observes behavior in tested scenarios.
Agents Shipgate reviews should vs. can.
Compares static authority with intended purpose, scopes, and controls.
Evidence before conclusions
Numbers describe workflow state, not safety outcomes.
Every selected repository stays visible
Not found does not mean absent. Unsupported and failed cases remain in the final denominator.
OpenClaw
openclaw/openclaw- Stratum
- General / personal
- Snapshot
dfa31af0- Stars at freeze
- 383k
- Scan result
- Pending human source-map review
Root MIT surface; source map remains pending review.
Reproduce this case
python scripts/run_tool_surface_study.py reproduce --catalog benchmark/tool-surface-study/catalog.lock.json --slug openclaw Run from an Agents Shipgate repository checkout; the downloadable catalog is the public inspection copy. The command remains blocked until the source map and engine gates are approved.
AutoGPT Classic
Significant-Gravitas/AutoGPT- Stratum
- General / personal
- Snapshot
a8d9d05e- Stars at freeze
- 185k
- Scan result
- Pending human source-map review
classic/** only; autogpt_platform/** excluded by license.
Reproduce this case
python scripts/run_tool_surface_study.py reproduce --catalog benchmark/tool-surface-study/catalog.lock.json --slug autogpt-classic Run from an Agents Shipgate repository checkout; the downloadable catalog is the public inspection copy. The command remains blocked until the source map and engine gates are approved.
OpenManus
FoundationAgents/OpenManus- Stratum
- General / personal
- Snapshot
52a13f2a- Stars at freeze
- 57.1k
- Scan result
- Pending human source-map review
Canonical deployable subproject requires human mapping.
Reproduce this case
python scripts/run_tool_surface_study.py reproduce --catalog benchmark/tool-surface-study/catalog.lock.json --slug openmanus Run from an Agents Shipgate repository checkout; the downloadable catalog is the public inspection copy. The command remains blocked until the source map and engine gates are approved.
Gemini CLI
google-gemini/gemini-cli- Stratum
- Coding
- Snapshot
f354eeba- Stars at freeze
- 106k
- Scan result
- Pending human source-map review
TypeScript tool surface retained as a coverage-gap candidate.
Reproduce this case
python scripts/run_tool_surface_study.py reproduce --catalog benchmark/tool-surface-study/catalog.lock.json --slug gemini-cli Run from an Agents Shipgate repository checkout; the downloadable catalog is the public inspection copy. The command remains blocked until the source map and engine gates are approved.
Codex
openai/codex- Stratum
- Coding
- Snapshot
5c19155c- Stars at freeze
- 97.1k
- Scan result
- Pending human source-map review
Rust and host-config surfaces require human mapping.
Reproduce this case
python scripts/run_tool_surface_study.py reproduce --catalog benchmark/tool-surface-study/catalog.lock.json --slug codex Run from an Agents Shipgate repository checkout; the downloadable catalog is the public inspection copy. The command remains blocked until the source map and engine gates are approved.
OpenHands
OpenHands/OpenHands- Stratum
- Coding
- Snapshot
e3185990- Stars at freeze
- 80.4k
- Scan result
- Pending human source-map review
MIT core only; enterprise/** excluded; app-server candidate is low confidence.
Reproduce this case
python scripts/run_tool_surface_study.py reproduce --catalog benchmark/tool-surface-study/catalog.lock.json --slug openhands Run from an Agents Shipgate repository checkout; the downloadable catalog is the public inspection copy. The command remains blocked until the source map and engine gates are approved.
Cline
cline/cline- Stratum
- Coding
- Snapshot
63099710- Stars at freeze
- 64.5k
- Scan result
- Pending human source-map review
TypeScript tool surface retained as a coverage-gap candidate.
Reproduce this case
python scripts/run_tool_surface_study.py reproduce --catalog benchmark/tool-surface-study/catalog.lock.json --slug cline Run from an Agents Shipgate repository checkout; the downloadable catalog is the public inspection copy. The command remains blocked until the source map and engine gates are approved.
browser-use
browser-use/browser-use- Stratum
- Web / research
- Snapshot
fa21b782- Stars at freeze
- 104k
- Scan result
- Pending human source-map review
browser_use/mcp/manifest.json; DXT/MCP tools array pending review.
Reproduce this case
python scripts/run_tool_surface_study.py reproduce --catalog benchmark/tool-surface-study/catalog.lock.json --slug browser-use Run from an Agents Shipgate repository checkout; the downloadable catalog is the public inspection copy. The command remains blocked until the source map and engine gates are approved.
Deep Agents
langchain-ai/deepagents- Stratum
- Web / research
- Snapshot
e9c61304- Stars at freeze
- 26.1k
- Scan result
- Pending human source-map review
examples/deep_research/research_agent/tools.py.
Reproduce this case
python scripts/run_tool_surface_study.py reproduce --catalog benchmark/tool-surface-study/catalog.lock.json --slug deepagents Run from an Agents Shipgate repository checkout; the downloadable catalog is the public inspection copy. The command remains blocked until the source map and engine gates are approved.
Open Deep Research
langchain-ai/open_deep_research- Stratum
- Web / research
- Snapshot
408da442- Stars at freeze
- 12.0k
- Scan result
- Pending human source-map review
src/open_deep_research/utils.py.
Reproduce this case
python scripts/run_tool_surface_study.py reproduce --catalog benchmark/tool-surface-study/catalog.lock.json --slug open-deep-research Run from an Agents Shipgate repository checkout; the downloadable catalog is the public inspection copy. The command remains blocked until the source map and engine gates are approved.
OpenAI Agents SDK
openai/openai-agents-python- Stratum
- OpenAI Agents SDK
- Snapshot
08859339- Stars at freeze
- 27.8k
- Scan result
- Pending human source-map review
examples/customer_service/main.py; tests excluded.
Reproduce this case
python scripts/run_tool_surface_study.py reproduce --catalog benchmark/tool-surface-study/catalog.lock.json --slug openai-agents-python Run from an Agents Shipgate repository checkout; the downloadable catalog is the public inspection copy. The command remains blocked until the source map and engine gates are approved.
Google ADK samples
google/adk-samples- Stratum
- Google ADK
- Snapshot
3fb70da9- Stars at freeze
- 9.9k
- Scan result
- Pending human source-map review
travel-planner-google-maps-mcp agent candidate.
Reproduce this case
python scripts/run_tool_surface_study.py reproduce --catalog benchmark/tool-surface-study/catalog.lock.json --slug adk-samples Run from an Agents Shipgate repository checkout; the downloadable catalog is the public inspection copy. The command remains blocked until the source map and engine gates are approved.
CrewAI
crewAIInc/crewAI- Stratum
- CrewAI
- Snapshot
fb8e93be- Stars at freeze
- 55.3k
- Scan result
- Pending human source-map review
Framework-core calibration source remains under human review.
Reproduce this case
python scripts/run_tool_surface_study.py reproduce --catalog benchmark/tool-surface-study/catalog.lock.json --slug crewai Run from an Agents Shipgate repository checkout; the downloadable catalog is the public inspection copy. The command remains blocked until the source map and engine gates are approved.
n8n AI starter kit
n8n-io/self-hosted-ai-starter-kit- Stratum
- n8n
- Snapshot
9b802c62- Stars at freeze
- 15.0k
- Scan result
- Pending human source-map review
n8n/demo-data/workflows/srOnR8PAY3u4RSwb.json.
Reproduce this case
python scripts/run_tool_surface_study.py reproduce --catalog benchmark/tool-surface-study/catalog.lock.json --slug n8n-ai-starter Run from an Agents Shipgate repository checkout; the downloadable catalog is the public inspection copy. The command remains blocked until the source map and engine gates are approved.
No repositories match these filters.
Inspect the prepublication status data
Frozen metadata and workflow state only—no findings or completed-review claims.
A fixed corpus, awaiting the remaining replay work
The lineage is public; routing claims remain blocked until replay and independent labels complete.
Data lineage only
Analysis denominator
Blind review required
Seeded; force-run pending
Five cases are preregistered for explanation—not prevalence: Stripe AI #232, CrewAI examples #169, ADK samples #1975, OpenAI Agents Python #3461, and Goose #9637.
What must happen before this becomes a report
- 1
Freeze and scope
Immutable commits, path-level licenses, and two-person source-map review before results.
- 2
Scan twice
Pinned engine, primary and no-heuristics modes, no upstream code execution or scan-time network.
- 3
Review blind
Two independent reviewers code purpose and alignment; a third human adjudicates disagreements.
- 4
Fact-check
Named cases receive ten full business days for maintainer corrections and right of reply.
- 5
Publish by hash
Privacy, disclosure, reproducibility, and human publication gates bind the exact public artifact.