pyATS MCP has been live since April 2025 — one of the first MCP servers to put Cisco pyATS and Genie in front of an AI agent. It worked. It also predates almost everything MCP has shipped since: the SDK was on 1.x, the transport was STDIO only, and the tool surface stopped at show commands, config, and a sandboxed test runner. This is the write-up for bringing it forward — Streamable HTTP, nine new tools, a real CML lab, and two bugs from the original release that had been sitting there, unnoticed, the whole time.
Where this started
The ask was simple on its face: add tools from a companion skills repo, modernize the transport to whatever MCP currently considers “stateless,” and benchmark before against after. Three sentences. What actually happened was closer to a full audit of a fifteen-month-old codebase, conducted against a real, live lab, with the lab occasionally rebooting mid-investigation.
I’ll take the mistakes in the order I made them, because two of them are worth more than the features.
The mistake I made in the first ten minutes
The plan called for mcp>=2.0.0. I ran pip install mcp==2.0.0 to go check its real API before writing anything — and did it in the global environment, not a project venv.
The install succeeded. Pip also printed eight lines of dependency-conflict warnings I almost scrolled past, because that same machine runs several other MCP servers out of the same global site-packages: fastmcp, uml-mcp, servicenow-mcp, and — this is the one that mattered — cml-mcp, the server that talks to the very CML lab I was about to need. All four pin mcp<2.0.0. I'd just broken every one of them to satisfy a version check on a server I hadn't touched yet.
Reverted immediately, back to 1.26.0 globally, then built an isolated .venv for the actual work and never touched the shared environment again. The finding worth keeping isn't "I made a mistake" — it's that the mistake was invisible until I looked at the dependency solver's output on a machine that already had other real tools installed. A throwaway sandbox with nothing else in it would never have caught this.
What "stateless" actually turned out to mean
Streamable HTTP's MCPServer.run() in mcp>=2.0.0 takes a stateless_http flag — True means no server-side session state kept between requests. That's the obvious reading of "stateless," and it's the one I built the first pass of the benchmark plan around.
It's also not the interesting change. SEP-2575, the MCP spec revision dated 2026-07-28, removes the initialize/initialized handshake and the Mcp-Session-Id header from the protocol core entirely — every request instead carries its own _meta block with protocol version, client info, and capabilities. That's a different axis from the transport-level flag, and the SDK does it automatically for any client that negotiates the new protocol version, with zero code required on the server side. I'd conflated "the kwarg named stateless_http" with "the thing SEP-2575 actually changed" until I went and read the SEP text directly.
Both axes turned out to matter for the benchmark, they just answer different questions — which is exactly why the final run measured four conditions, not two.
What my first pass never got to
I wrote a book on pyATS and Genie. The gap between what I've spent years teaching people to do with this framework and what my own MCP server actually exposed was bigger than I'd like to admit. The original five tools were exactly what a CLI-first engineer reaches for on day one — run a show command, push config, learn the running config, learn logging, ping. Real coverage. Also the floor of what pyATS can do, not the ceiling.
The parts I'd wanted in there for years never made the cut the first time, mostly because none of them fit the shape of "wrap a show command":
pcall — pyATS's own parallel-execution primitive, one OS process per target instead of a loop. I reach for this constantly at scale and it had never once been in the MCP server.
Genie's general device.learn(feature) — not "learn config" and "learn logging" as two hardcoded calls, but the actual Ops model library behind them: interface, ospf, bgp, platform, dozens of features, each normalized to a vendor-neutral structure you can diff.
Genie Clean — the staged device reset/reprovisioning framework. Deliberately scoped down to non-destructive stages here, but the hook into the real Kleenex machinery hadn't existed before at all.
Blitz and Robot Framework — the two declarative test-authoring surfaces I point people toward specifically when they don't want to write Python. Neither had ever been reachable from an agent.
The REST connector — pyATS isn't CLI-only; it's had a real REST/RESTCONF/NX-API connection type for years, entirely separate from the SSH path every one of the original five tools used.
None of that is exotic. It's close to the table of contents. It just hadn't made it into the MCP server, because the first version was built to prove the idea worked at all, not to be comprehensive.
Nine new tools, verified against a real lab
The companion skills repo's coverage — pcall, clean, learn, blitz, mock devices — doesn't map onto pyATS's actual Python API as cleanly as the skill names suggest. Two of the five have no supported in-process entry point at all.
Genie device.learn() with named before/after snapshots and genie.utils.diff.Diff
pyats_clean_device
Genie Clean has no supported public API — DeviceClean is an internal class coupled to aetest's global executer state. Shells out to the real pyats clean CLI instead, restricted to non-destructive connect+execute_command stages only, dry_run=True by default
pyats_run_blitz
Same story — Blitz imports the global pyats.easypy.runtime singleton, so it has to run through a real job, not a function call
pyats_run_robot
Robot Framework via the actual installed pyats.robot / genie.libs.robot keyword libraries — there's no pyats robot CLI subcommand in this version, so it runs through the standalone robot CLI
pyats_rest_request
Generic REST/RESTCONF/NX-API via pyATS's own rest.connector — a second, separate connection type from the CLI/SSH one every other tool uses
pyats_xpresso_request
Cisco XPresso's REST API v2, built from the published docs. No live instance to test against — flagged as unverified in the tool's own docstring rather than presented as equal-confidence to everything else
"Mock devices" — the fifth skill — never became a user-facing tool. It became a benchmark fixture instead, once it was clear the actual value was repeatable timing runs, not another tool an agent would call.
Two bugs from April 2025, still there
Every one of the nine tools got a real call against a real CML lab before I called it done — four IOS-XE devices, reachable once I worked out which network they were actually on (more on that below). That process surfaced two defects in code that shipped over a year ago and had presumably been called successfully many times since.
pyats_run_dynamic_test's report was always empty. The original code passed --json-job <path> to pyats run job and read the report back from that path. That flag does not exist in this pyATS version's CLI. Not deprecated — never present in --help at all. The argument parser accepts it silently, produces no report file, and the job still exits 0. The tool never crashed, never logged an error, and its report field was quietly None on every single call since launch. pyATS does archive every run to a real zip under ~/.pyats/archive/ by default, with an actual results.json inside — the fix reads that instead, and now returns a genuine structured report with pass/fail counts.
Blitz jobs failed on any lab with one unreachable device — including devices the job never asked for.pyats run job invokes genie.harness's common_setup, which by default connects to every device in the testbed handed to it, not just the ones a specific trigger targets. Point Blitz at a testbed where three of seven devices aren't up yet, ask it to run one command against a device that is up, and the whole job errors out in setup before it ever reaches your device. The fix generates a testbed scoped to only the devices a given call actually needs.
Same shape both times: a tool that looks like it's working — no exception, no error field, a plausible-looking exit code — while quietly not doing what it claims. Neither would show up in a unit test with a mocked device. Both only showed up because something insisted on hitting real hardware before calling the work finished.
The testbed was pointing at the wrong network
The CML lab's testbed.yaml had four IOS-XE devices at 10.10.20.171–174. None of them were reachable. ip route showed no path to that subnet at all — not a timeout, a routing-table gap.
The lab's own OOB-MGMT network told a different story. One device's running config, pasted mid-investigation, showed an interface configured at 192.168.2.201 — a subnet that was reachable. Rather than guess at the other three, I used the credentials already sitting in ~/.openclaw/.env for the existing cml-mcp server to query the CML controller's own API directly (virl2_client, read-only) and asked it where it thought each device's management interface actually was. It answered with exact IPs, discovered from the lab's own ARP/L3 tables: .201, .202, .211, .212. A short port scan across the neighboring range confirmed all four were real, live, and speaking Cisco's SSH banner before a single credential was tried.
The lesson isn't "the lab was misconfigured." It's that the CML controller already knew the right answer, in a format a script could ask for directly, and guessing at the network from the outside would have taken far longer than just asking the source of truth.
The benchmark, and the answer I didn't expect
Four conditions, against the same four real devices, each running the same scenario set: STDIO on the pre-modernization server (mcp==1.26.0), the modernized server negotiating the new handshake-free protocol, and the modernized server under the old handshake-based protocol with stateless_http toggled both ways.
Condition
Connection setup
STDIO (pre, mcp==1.26.0)
1.074s
Streamable HTTP, modern protocol
0.025s
Streamable HTTP, legacy protocol, stateful
0.023s
Streamable HTTP, legacy protocol, stateless
0.026s
Streamable HTTP sets up a connection roughly 40–50× faster than STDIO — no surprise, since STDIO means forking a fresh Python process and re-importing pyATS and Genie every single time, and HTTP just opens a request against a process that's already running.
Per-call latency for the actual tools, though, was identical across all four conditions — single show command, thread-pool fan-out, pcall fan-out, device health, all within noise of each other, all around 15–18 seconds. Connection caching defaults to off, so every call does a real SSH connect and disconnect to a CSR1kv, and that cost swallows any transport-layer difference completely. The one signal that did survive the noise: pcall's fan-out ran consistently ~150–200ms slower than the thread-pool version, which is exactly the fork overhead its own documentation describes.
I could have reported "Streamable HTTP is faster" and stopped there, and it would have been technically true and substantially misleading. The honest version is narrower: modernizing the transport buys you cheap, frequent connections — which matters if you have many short-lived clients — and buys you almost nothing on a workload that's bottlenecked on SSH negotiation to real gear. Both of those are real findings. Only one of them is the one a changelog would lead with.
Where it stands
26 tools now (the tool table in the README had been silently missing two working tools — pyats_show_running_config and pyats_show_logging — since launch; caught it while cross-checking the table against the actual @mcp.tool() registrations rather than trusting either version of the document). 119 unit tests, up from 85, all mocked, no real devices required to run them. A benchmark/ harness that stays in the repo rather than living only in a chat transcript. And a rewritten README with real, source-verified instructions for wiring the server into Claude Code, VS Code's Copilot Chat, OpenAI's Codex CLI, Claude Desktop, and raw Python — including the one that looks like it should just work and doesn't: Desktop's config file is stdio-only, a url field in it fails silently, and the actual supported path is either a Custom Connector against a public HTTPS endpoint or an mcp-remote stdio bridge for anything running on your own machine.
Merged to main as PR #13. CI is temporarily disabled — out of Actions minutes, not a statement about the pipeline — and the one thing still red when it comes back is a black formatting check that was already failing a month before any of this started.
The uncomfortable part
I wrote a book to teach people how pyATS and Genie actually work — the Ops model, the connection classes, when to reach for Blitz instead of a raw testscript, why pcall exists instead of a loop. That book's entire premise is that understanding the framework is the value. Every tool in this post is an abstraction that lets an agent reach the same capability without a human needing to know any of that. Ask it to snapshot OSPF state, push a change, and diff what moved, and it picks pyats_learn_feature and pyats_diff_learned_snapshots on its own. Nobody has to have read the chapter first.
I don't think that makes the book worthless. I think it means the value moved. Fewer people need to know pyATS's API to get something real done with it, which was arguably the whole point of automation to begin with. It does mean I'm the one shrinking the book's core pitch, one tool at a time. I'm fine with that trade. I'd rather be the one making myself obsolete than watch someone else do it slower.
A working start-to-finish guide. Tested on an M-series MacBook Pro, August 2026.
Read this first: there is one non-obvious step that will waste your afternoon if you miss it — you must convert the weights at BF16, not F16. Details in step 6 and in Troubleshooting. Everything else is routine.
What Antares-1B actually is
An open-weight 1B model from Cisco Foundation AI, built on IBM Granite 4.0 1B, specialized for vulnerability localization. Given a CWE description and a repository, it explores the codebase through a terminal — issuing grep, find, cat — and submits a list of files it believes contain the vulnerability.
Two things to internalize before you start:
It is not a chat assistant. The model card explicitly lists general-purpose chat and instruction following as out-of-scope. Asking it “what are your capabilities” gets you nothing useful even when the setup is perfect.
It is one half of an agent loop. It emits <tool_call> JSON; your harness executes the command and feeds the output back as <tool_response>. It runs up to 15 terminal calls, then terminates via submit_vulnerable_files or submit_no_vulnerability_found.
License is Apache 2.0. The Hugging Face repo is gated behind a contact-info form. Expect ~100 tok/s on Apple silicon. Benchmark is File F1 0.209 on Cisco’s VLoc Bench — useful for triage, not a scanner replacement.
Prerequisites
macOS on Apple Silicon
Homebrew
~15 GB free disk (4 GB weights, 3.7 GB GGUF, plus PyTorch)
requirements.txt already installs huggingface_hub at a compatible version. Do not run pip install -U huggingface_hub — that pulls the 1.x line, which breaks transformers. If you already did:
pip install "huggingface_hub<1.0"
pip check # should report no broken requirements
This is the step that matters. Granite 4.0 is a hybrid Mamba/attention architecture. The SSM state updates exceed F16’s dynamic range, saturate, and the model collapses into emitting a single repeated token forever. BF16 has the same bit width but far more exponent headroom and survives fine.
Symptom if you get this wrong: the model loads cleanly, reports ~100 tok/s, and prints @@@@@@@@@@@... in response to anything.
This is also why the community mradermacher / mitkox GGUF conversions on the Hub don’t work — they appear to route through an F16 intermediate. Convert it yourself.
7. Verify with llama.cpp before touching Ollama
llama-cli -m ~/models/antares-1b-bf16.gguf --jinja -no-cnv \
--temp 0.3 --top-p 1.0 \
-p "Vulnerability to locate: CWE-78 OS Command Injection."
--jinja uses the template embedded in the GGUF, so there is nothing to hand-write and get wrong. This isolates model correctness from Ollama config.
Expected: coherent reasoning prose, then </think>, then a response. The reply starts mid-thought with no opening <think> tag — that is correct. The template pre-fills it.
If you see @@@@, stop here and go to Troubleshooting. Do not proceed.
8. Import into Ollama
Ollama does not pick up the chat template from this GGUF automatically, so supply it explicitly. Without it Ollama sends raw text with no role markers and no stop token, and you get @@@@ again even from a good BF16 file.
cd ~/models
cat > Modelfile <<'EOF'
FROM ./antares-1b-bf16.gguf
TEMPLATE """<|start_of_role|>system<|end_of_role|>{{ if .System }}{{ .System }}{{ else }}You are a helpful assistant. Please ensure responses are professional, accurate, and safe.{{ end }}<|end_of_text|>
{{ range .Messages }}{{ if eq .Role "user" }}<|start_of_role|>user<|end_of_role|>{{ .Content }}<|end_of_text|>
{{ else if eq .Role "assistant" }}<|start_of_role|>assistant<|end_of_role|>{{ .Content }}<|end_of_text|>
{{ end }}{{ end }}<|start_of_role|>assistant<|end_of_role|><think>
"""
PARAMETER stop "<|end_of_text|>"
PARAMETER stop "<|start_of_role|>"
PARAMETER temperature 0.3
PARAMETER top_p 1.0
PARAMETER num_ctx 32768
EOF
ollama create antares-1b -f Modelfile
Note there is no --quantize flag on the first build. Get it working unquantized, then shrink.
Three details in that template, all derived from the model’s real chat_template.jinja:
A system block is always emitted — the Jinja has no “if system” branch, it injects a default. Making it conditional produces a prompt shape the model never saw in training.
The generation prompt pre-fills<think>\n. The model starts already inside a reasoning block.
Temperature 0.3 / top-p 1.0 are the settings Cisco used for every published evaluation. Ollama’s defaults (0.8 / top-k 40 / repeat-penalty 1.1) are far too loose for an agentic model.
Verify:
ollama show --modelfile antares-1b | head -30
If TEMPLATE shows {{ .Prompt }}, the template didn’t take — rebuild.
ollama run antares-1b "Vulnerability to locate: CWE-78 OS Command Injection."
9. Shrink it (optional)
3.7 GB is large for a 2B model. Step down one level at a time and test each:
ollama create antares-1b:q8 --quantize q8_0 -f Modelfile
ollama run antares-1b:q8 "Vulnerability to locate: CWE-78 OS Command Injection."
ollama create antares-1b:q4 --quantize q4_K_M -f Modelfile
ollama run antares-1b:q4 "Vulnerability to locate: CWE-78 OS Command Injection."
If a level produces @@@@ or degraded reasoning, you’ve found the floor for this architecture — stay one level up. Hybrid SSM models are more quantization-sensitive than plain transformers.
10. Run it as an agent
The REPL is a diagnostic, not a workflow. To see real tool calls, give it the system prompt it was trained against:
ollama run antares-1b
>>> /set system You are a security vulnerability localization agent. You have access to a terminal with the repository mounted at /workspace/repo/. Use shell commands (grep, find, cat, etc.) to explore the codebase and identify files that contain the reported vulnerability. When confident, submit your findings.
>>> Vulnerability to locate: CWE-78: Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection').
Success looks like a <tool_call> block containing a shell command. The model is asking you to run it and return the output.
For the real loop, Cisco ships the Antares CLI as a ZIP in the Files tab of the model repo. It implements the full agent loop against any OpenAI-compatible endpoint:
ollama serve # exposes http://localhost:11434/v1
Configure the CLI with base URL http://localhost:11434/v1, model antares-1b, and any dummy API key. It supports targeted CWE analyses and repo-wide sweeps, with output in human-readable, JSON, or SARIF.
Alternative: skip Ollama entirely and use llama-server, which gives you the same OpenAI-compatible endpoint and tends to get Granite hybrid fixes first:
You are executing shell commands the model invents. Per Cisco’s recommendations:
Run the target repo snapshot in a container with --network=none
10 second command timeout
2 CPU cores, 4 GB RAM limit
Destroy the container after each run
Human review before any remediation action
Troubleshooting
@@@@@@@@@ output
Ranked by likelihood:
GGUF converted at F16 instead of BF16 → reconvert (step 6)
Ollama Modelfile missing TEMPLATE → check ollama show --modelfile
Quantized too aggressively → rebuild unquantized and re-test
Using a community GGUF from the Hub → convert it yourself
401: Invalid username or password from ollama pull
Not an auth problem. Hugging Face returns 401 rather than 404 for repos it can’t show you, to avoid leaking whether private repos exist. You almost certainly have the repo name wrong. (mradermacher/Antares-11b-v2-GGUF is a different, unrelated model — an old 11B SOLAR finetune. Nothing to do with Cisco Antares.)
Ollama and gated/private repos
Ollama’s HF integration doesn’t use tokens at all. It authenticates with the SSH key at ~/.ollama/id_ed25519.pub — add that to https://huggingface.co/settings/keys.
pyenv: python: command not found
No pyenv version is active:
pyenv global 3.10.14 && pyenv rehash
If it still fails, your ~/.zshrc is missing eval "$(pyenv init -)".
zsh: bad pattern: ^[[200~
Bracketed-paste escape leaking into zsh. Run exec zsh, or paste one line at a time.
cmake: command not found
You don’t need it — brew install llama.cpp gives prebuilt binaries. Only brew install cmake if you specifically need to build from source (e.g. to pick up a fix newer than the brew formula).
transformers … requires huggingface-hub<1.0
See step 4.
Cross-checking against reference weights
If GGUF misbehaves and you want to know whether the weights themselves are fine:
cd ~/llama.cpp && source .venv/bin/activate && pip install accelerate
python - <<'EOF'
from transformers import AutoTokenizer, AutoModelForCausalLM
import os
p = os.path.expanduser("~/models/antares-1b")
tok = AutoTokenizer.from_pretrained(p)
m = AutoModelForCausalLM.from_pretrained(p, device_map="cpu")
msgs = [{"role":"user","content":"Vulnerability to locate: CWE-78 OS Command Injection."}]
i = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt", return_dict=True)
o = m.generate(**i, max_new_tokens=80, do_sample=True, temperature=0.3, top_p=1.0)
print(tok.decode(o[0][i["input_ids"].shape[-1]:], skip_special_tokens=False))
EOF
Coherent here but garbage from every GGUF = a llama.cpp conversion/runtime gap. Check the llama.cpp issue tracker, or serve with vLLM, which the model card documents as supported.
On July 30th I sat down and wrote a roadmap: twenty-five identified gaps in what NetClaw could actually reach, ordered by dependency, with one rule at the top — one item, one spec, one branch, no batching. Six days later, twenty-three specs had merged. Twelve new MCP servers. Twenty-three new skills. And a running argument with reality that corrected the roadmap four separate times.
This is the write-up for specs 076 through 098. It is long, because a lot shipped. If you only read one section, make it the last one — the through-lines are more useful than any individual integration.
The setup: a roadmap, and one rule
NetClaw has a lot of reach. It talks to Cisco through pyATS and RADKit, to Juniper through JunOS, to Arista through CloudVision, to NetBox and Nautobot and Infrahub for source of truth, to Grafana and Prometheus and Splunk and Datadog for observability, to CML and ContainerLab and GNS3 and EVE-NG for labs. Before this run: 199 skills, 149 MCP integrations.
But “a lot of reach” is not the same as “no holes,” and I’d never actually sat down and enumerated the holes. So I did a landscape scan — vendor and community MCP registries, awesome-mcp-servers, Itential’s 56-server network automation guide, Cisco and Juniper and HPE official releases, anthropics/skills, and the IETF datatracker — and turned it into docs/COVERAGE-ROADMAP.md: twenty-five items, R0 through R24, ordered by dependency and then by value-per-effort.
The rule at the top of that document is the whole methodology:
Work top to bottom. One roadmap item = one spec = one branch. Do not batch.
Every item goes through the full spec-driven cycle: specify, clarify, plan, task, implement, verify. Every item ends with a merged branch and a reconciliation gate that exits zero. And crucially — every item is treated as a hypothesis, not an instruction.
That last part turned out to matter enormously.
What landed
Item
Spec
What shipped
R0a
077
Dependency-pin hazards fixed and gated
R0b
090
Dead registered servers fixed; startup becomes a hard gate
R1
076
multivendor-cli-mcp — SSH/NAPALM to ~175 platform families
R2
078
cisco-psirt-mcp — live version to published advisory
Globalping — the first vantage point outside my own domain
R9
081
bgp-intel-mcp — RPKI, RDAP, PeeringDB, RIPEstat
R11
083
zabbix-mcp — NetClaw’s first polled history
R12
096
Elasticsearch log search
R13
091
nsm-mcp — Zeek and Suricata offline PCAP analysis
R14
084
k8s-mcp — read-only Kubernetes
R15
094
redfish-mcp — out-of-band hardware, “is the box dead or is it the network”
R17
092
analysis-mcp — DuckDB over exported network data
R18
082
document-mcp — docx, xlsx, pptx, pdf
R22
093
Closed as already satisfied — and found a real defect doing it
R23
085
IETF MCP landscape survey feeding NCFED -01
R24
097
All 22 open-territory candidates dispositioned
R25
098
anta-mcp — the assertion layer
—
087
Cisco Catalyst Center, read-only
—
088
Startup check — the fifth reconciliation surface
—
089
Cisco Meraki official, adopted, zero code
—
095
Juniper Mist measured against the token ceiling
—
086
ntopng deferral, recorded with evidence
Ninety-three commits. 401 files. +62,926 / −1,379. 199 skills to 221; 149 MCP integrations to 163.
First, the boring specs that turned out to be the important ones
Two of the three foundation items weren’t in the original roadmap at all. They got inserted because the foundation turned out to be less solid than I’d established.
The bug that only breaks people who aren’t you
While implementing R1, I found that mcp 2.0.0 had removed mcp.server.fastmcp entirely. Not moved. Not deprecated with a shim. The 2.0.0 wheel contains zeromcp/server/fastmcp/ files and doesn’t declare fastmcp as a dependency, so there is no re-export to fall back on.
Any server with an unbounded mcp>=1.0.0 pin that imports that module resolves a breaking major on a fresh install. My install was fine — my versions were already resolved months ago. Only new installs broke, which is exactly why nobody had noticed.
Spec 077 fixed it, and then fixed the class rather than the instance:
Repair
Scale
Unbounded pins on packages whose submodules are imported
25 failures across 20 servers
Bare pip calls routed through one helper
130
GAIT’s unbounded install (the audit trail itself)
bounded
New CI gate surface
dependencies
Contract tests, including false-positive guards
23/23
The finding worth keeping: my hand audit found 7 exposed servers. The static scan found 25 across 20. The audit looked for the pattern I already knew — unbounded mcp>= plus that one import. The scan looked for the class: any unbounded pin on any package whose submodule is imported anywhere.
A human audit finds what it expects. A static scan finds what is there.
Seven registered servers that could not start
Here’s the one that stung.
NetClaw’s reconciliation gate had four surfaces: installer coverage, documented counts, registration portability, dependency pins. All four validate that things are declared consistently. Spec 088 asked a question none of them asked: can a registered server actually start?
It launches all 98 registered stdio servers and watches what happens. Result: seven could not start at all, and 22 skills routed to them — while reconcile-mcp.py exited 0 and CI passed green.
A first pass at this by static import analysis reported 11 findings, 5 of them false, because sys.path resolution is a runtime fact and reading source cannot tell you whether an import will succeed. Only launching the process gives the truth. That’s written into the script’s own docstring so nobody repeats the shortcut.
Spec 090 then fixed six of the seven, and the root cause was a single function. netclaw_pip_install had no handling for PEP 668 (the “externally managed environment” error) — while 56 call sites hid the resulting failure behind --break-system-packages 2>/dev/null || log_warn. Every one of those installs could fail silently and the installer would carry on cheerfully.
The seventh is an enumerated exception with a written reason: RADKit ships code-signed wheels outside PyPI. startup is now a hard gate.
It also corrected spec 088’s own claim that prisma_sase was unavailable. It wasn’t. That was a PEP 668 error being read as an availability error.
The gate now runs seven surfaces: catalog, docs, portability, dependencies, startup, meraki-ids, and packages. Plus a separate one added during 096, after an audit found ten consecutive specs shipping with spec.md alone against 72 of 86 carrying the full artifact set. That drift was self-reinforcing in a way I find genuinely interesting: an author checking the three most recent specs for the convention saw the drift, and read it as the convention.
The biggest single item: SSH to anything
Before spec 076, NetClaw’s device reach was pyATS, JunOS, gNMI and RADKit. Excellent coverage of Cisco and Juniper. No way to SSH to anything else.
multivendor-cli-mcp is 10 tools over Nornir, Netmiko and NAPALM, and it is read-only by default with a server-side command filter — not a prompt instruction, not a skill convention, an actual filter in the request path that the model cannot talk its way around. Inventory comes from NetBox, Nautobot or Infrahub as source of truth, never a credentials YAML. Writes need two distinct gates: human approval and an approved ServiceNow change record, with separate refusal outcomes so they can never be conflated.
94/94 tasks, 31/31 live integration checks, 175 driver-documented platform families:
Verified live
Evidence
Nokia SR Linux, native NOS CLI
real show version, show interface brief
FRR via vtysh, shell-hosted
real routing table through the linux driver
IOS-XE normalized read
NAPALM ios, real hostname and interfaces
SR Linux normalization gap
reported as a row with a reason, never silently omitted
live instance: production + approval but no CR → blocked
Both candidate community servers were rejected. One is archived, has two stars, and reloads config.yaml from the current working directory on every single call — threading its inventory assumption straight through the request path. The other has no command filtering at all. Both store credentials in YAML.
But I deliberately ported the archived one’s safety design — prefix allowlist, destructive-token denylist, chaining prevention, path sandboxing. That’s the part most easily got wrong, and somebody had already thought it through carefully. Rejecting a project is not the same as rejecting its ideas.
Three bugs only real devices found, and one of them is a nice trap:
The command filter blocked FRR’s only read path.vtysh -c "show ip route" starts with vtysh, not show. The tempting fix — allowlist vtysh — would also have permitted vtysh -c "configure terminal", which is a config escape wearing a read command’s clothes. Fixed by unwrapping the wrapper and judging the inner command.
SR Linux was under-protected because nokia_srl (the driver and inventory name) is not nokia_srlinux (the denylist table key), so the denylist quietly didn’t apply. Alias normalisation now runs first.
The ITSM gating had zero test coverage, and the plan had claimed it was “inherited from the existing approval path.” That was an assertion, not a fact. Human approval and a change record are distinct gates and now have distinct tests.
Asking Cisco whether the box you’re looking at is vulnerable
Spec 078 closes a top-five real-world netops question: is this build affected by a published advisory?
R2 was planned as “Cisco Support APIs — PSIRT, EoX, Bug Search, Case.” Measuring the APIs before writing the spec cut it from eight families to one:
Family
Result
PSIRT openVuln
200 — this is the feature
Bug Search
403
EoX
403
Case
403
Serial → Info
403
CX Cloud, 7 paths
504
The API Console grant covers PSIRT and nothing else. Which is exactly why the obvious community candidate — 46 tools across all 8 families — was not adopted: seven-eighths of that tool manifest would have been dead surface, spending tokens on every single turn to advertise capabilities that return 403.
I’ll say the uncomfortable part plainly: EoL/EoS lookup was half of R2’s original value and it is not delivered. Not descoped for convenience — unreachable with the entitlement available.
Verifying all seven PSIRT OS types then produced my favourite finding of the run. Cisco’s version formats contradict each other across families:
OSType
Accepted
Rejected
iosxe
17.3.1, 17.03.01, 17.3.1a
17.3(1)
ios
15.2(4)E, 15.2(4)E10
15.2.4E
nxos
9.3(5)
9.3.5
asa
9.16.1
9.16(1)
aci
15.2(3e)
5.2(3e), 5.2.3
ios and nxos require exactly the formiosxe rejects. aci wants the letter suffix inside the parentheses where ios wants it outside — and it wants the switch image version, not the APIC version, so an operator reading the number off their APIC hands over something the API refuses. The single global normalisation rule my spec had drafted would have broken ios and nxos on every call.
Then the whole chain on a live CML router: pyATS read IOS-XE 17.16.1a off the device, PSIRT returned 26 advisories — 14 High, 11 Medium, 1 Critical (CVE-2025-20363, CVSS 9.0). No human typed a version anywhere in that path.
Rate discipline is contractual and ordered: de-duplicate, then cache, then pace, then back off. Measured — 60 devices running 12 distinct versions costs 12 API calls, not 60. De-duplication is first because it’s the largest win; pacing an un-de-duplicated sweep just spreads the same excess over more minutes.
Fortinet, and a skill that was lying
R3’s premise turned out to be worse than “there’s no server.” The fortimanager-ops skill shipped with user-invocable: true, declaring environment variables and naming a specific community server — that was never vendored, never registered, and not installable. The installer even cloned the repo. A federation member was configured to run against the phantom command.
That’s not a gap. A gap is honest. That’s a claim — and an agent routing a firewall question to it finds out mid-investigation.
Spec 080 shipped 21 tools across three planes for a 2,486 of 5,000 token manifest, built rather than adopted on four independent disqualifications: no candidate emits a plane field, the manifests run 69–204 tools each, only one enforces read-only (another exposes package install ungated), and none has any concept of a change record.
The three planes are not substitutes for each other, and that’s the design:
Manager = intent. What the policy says it should be.
Device = observed state. What the box is actually running.
Analyzer = observed traffic. What actually hit the rule.
fgt_compare_with_manager reports only_in_device rules as candidate out-of-band changes — something that is invisible from either plane alone. Device plane verified live from Slack against FortiOS 7.6.7.
The follow-up spec is worth a paragraph on its own, because NetClaw reported the defect on itself, unprompted, during live testing.fgt_list_interfaces was reading only the monitor endpoint, which carries operational reality — link state, speed, counters. Administrative intent lives in cmdb: status, role, allowed access. An interface that is administratively DOWN with a live carrier reports link: true and looks perfectly healthy if you only read one of them. It now merges both, reports admin_status and link separately, flags the admin_up_link_down combination, and reports admin status as UNKNOWN if the config read fails — rather than assuming “enabled” and silently recreating the exact conflation it was written to fix.
Two Cisco cloud platforms, two opposite answers
Catalyst Center and Meraki landed the same week and demonstrate the single design constraint that decided more of this run than any architectural preference: the 5,000-token tool manifest ceiling.
Cisco’s official Catalyst Center MCP server exposes 515 tools ≈ 64,420 tokens. Thirteen times the ceiling. So spec 087 built a thin client over Cisco’s own published catalogue instead: 514 read operations reachable through 8 dispatchers plus find/describe, for 1,821 tokens. All the coverage, none of the manifest.
Meraki went the opposite way — adopted with zero lines of code. Cisco’s remote MCP is 2 tools exposing 494 read-only capabilities for 1,561 tokens, and the read-only property is structural rather than promised: 431 mutating operations are simply absent from the catalogue. I verified 10 of 10.
Adopting it retired the dead community Meraki server, which took spec 088’s dead-server count from 7 to 6. And it surfaced something I did not expect: 54 of the 80 method names the five Meraki skills documented did not exist in the Meraki API. Same defect class as the phantom FortiManager server — documentation that reads as capability. There is now a dedicated gate surface that validates skill text against Cisco’s own OpenAPI spec, vendored offline.
Looking at the internet from the outside
NetClaw had zero external-vantage capability. No ASN lookup, no route-origin validation, no peering data, no third-party reachability. Not a missing tool — a missing domain.
Spec 079 (Globalping) was rated the highest value-per-effort item in the entire scan, and it held: no server was written. One remote registration, one skill. ~4,800 probes across ~1,390 autonomous systems, measuring toward a public target. It answers “the router is fine, so why can’t anyone reach us?” — which NetClaw previously could not address at all.
The whole feature is a registration plus prose, which is precisely why the prose had to be right:
Response
What it means
no_probes_found (422)
The measurement never ran. Says nothing whatsoever about the target
finished, 0 of N successful
The target did not answer. A real finding
Private or internal target
Refused locally, before calling out
The first one is the trap. It arrives failure-shaped, and an agent reading it carelessly escalates a total outage that does not exist.
That third row is a disclosure control, not a correctness one. Globalping rejects RFC1918 itself, with good error text — but by the time it does, an internal hostname has already been transmitted to a third party. So NetClaw refuses first.
Two more things worth knowing if you use this API: the budget is charged per probe, not per call (limit: 20 costs 20), and AS13335 appears as a location example in Globalping’s own tool schema while hosting no probes at all. Anyone learning the syntax from the vendor’s own example tries it first, gets no_probes_found, and concludes ASN filtering is broken. The skill names this explicitly so the wrong lesson doesn’t get learned.
Spec 081 is the other half. Globalping measures toward a target; BGP intelligence looks up who owns a resource and whether an announcement is authorised. Ten tools, all ten live-verified, 1,376 tokens — and the first NetClaw integration with no secret to leak, because every source is a public unauthenticated API.
The distinction is the entire feature: RPKI not-found is not invalid. Most of the internet has no ROA. Reporting unsigned address space as a finding would manufacture false incidents at scale.
Query
State
Reason
Finding?
AS13335 + 1.1.1.0/24
valid
—
no
AS13335 + 8.8.8.0/24
invalid
as
yes
AS15169 + 8.8.8.128/25
invalid
length
yes
AS3356 + 4.0.0.0/9
not_found
—
no
There’s a fifth outcome, validation_unavailable: an unreachable validator is not unsigned space. Same distinction, one level down.
Every result names its registry or its collector. PeeringDB results carry a self-reported caveat, because PeeringDB is self-reported. And the rate limiter is a true sliding window — the first implementation used a minimum 250 ms gap and measured 4.53 requests per second against a 4/s target, because N requests spaced 250 ms apart put five of them inside one second.
Monitoring, traffic, and the layer beneath the network
Zabbix (083) was the first adopt, not build on the roadmap. NetClaw had Prometheus, Grafana, Datadog, Splunk, Auvik and ThousandEyes — and no SNMP-poller NMS at all, and therefore no polled history anywhere. The adopted server is 3 tools and 589 tokens; it is essentially the design I would have produced. The alternatives, measured by cloning and scanning rather than reading READMEs, run 53, 111 and 237 tools. It runs in a dedicated virtualenv because it needs fastmcp 3.x while five other NetClaw servers pin <3, and it’s strictly read-only: adopting as-is leaves nowhere to insert NetClaw’s two write gates, so writes were deferred rather than shipped ungated.
Zeek and Suricata (091) added the network-security-monitoring layer that was entirely absent: read-only offline PCAP analysis from digest-pinned containers, 6 tools / 934 tokens, 19 assertions. Arkime was rejected — mandatory OpenSearch plus 12–16 GB is a platform, not a tool. Two silent wrong answers were reproduced live and structurally blocked:
Stock Suricata loads zero signatures and cheerfully reports 0 alerts, behind two non-fatal warnings. After a rules update: 52,205 signatures. “No alerts” and “no ruleset” look identical in the output.
Zeek discards invalid-checksum packets by default, which loses http.log entirely and miscounts conn.log. This one matters beyond the new server — it affects the output of NetClaw’s own existing capture skills.
DuckDB (092) arrived after its data, exactly as the roadmap predicted it should. When R17 was first surveyed there were zero parquet files anywhere on the host, DuckDB wasn’t installed, and half the rationale had died with the ntopng deferral. The roadmap’s own conclusion was “R17 should follow whichever item first produces bulk exports” — and 091’s Zeek TSV and Suricata eve.json output is exactly that.
The interesting part is containment. A general SQL engine is a filesystem client: read_csv('/etc/passwd'), ATTACH some other database, COPY … TO, INSTALL httpfs and read over HTTP. Blocking that by inspecting query strings is a losing game — SQL has too many spellings and the one you miss is the one that matters. So enforcement is DuckDB’s own:
1. materialise each allowlisted dataset as a TABLE
2. SET enable_external_access = false -- filesystem and network close
3. SET lock_configuration = true -- and cannot be reopened
Eight escape attempts verified to raise. Memory, RAG, federation and GAIT are unreachable by construction, not by my regex.
Kubernetes (084) adopted Red Hat’s Apache-2.0 Go binary, pinned and checksummed. Read-only turned out to be what makes adoption possible: the upstream default is 21 tools / 5,716 tokens and busts the ceiling; trimmed it’s 7 tools / 1,643. It uses an explicit token-only kubeconfig for a dedicated cluster-wide-read ServiceAccount — never the ambient current-context, which might be production.
And it carries the sharpest finding of the whole run. The Kubernetes API is honest — it returns a correct 403 when you list cluster-wide without permission. The adopted server catches that permission error, discards it, and returns a perfectly plausible one-namespace answer with no error at all. Reproduced live in NetClaw’s own test suite, mitigated with a mandated ServiceAccount and a skill preflight.
Redfish (094) answers “is the box dead or is it the network.” Both candidate servers were unvendorable — one has no licence file at all, the other NOASSERTION — so it was built: 6 tools / 728 tokens, 15 assertions, verified against the DMTF Redfish mockup so no hardware was required. The core discipline is that the box-versus-network distinction is symmetric: a BMC timeout establishes nothing about the host and can never be emitted as a downed host, while PowerState: Off genuinely is a fact. Power control is deliberately unimplemented — the client issues no HTTP verb but GET.
Elasticsearch (096) adopted a 5-tool, 1,094-token server, verified against a live Elasticsearch 9.2.0 on a free basic licence with 25,000 indexed documents. It deliberately adopts a deprecated upstream, pinned by digest: the successor is Enterprise-tier on self-managed, so the supported path is paywalled while this one is Apache-2.0 and published.
Its silent wrong answer is a beauty. Elasticsearch caps hits.total at 10,000 and marks it relation: "gte" — and the server discards the qualifier. So 10,075 real documents get reported as “Total results: 10000,” and the error is unbounded: a million-document index also says 10,000. Mitigations verified and written into the skill: use esql, or pass track_total_hits: true.
That spec also corrected a claim I’d made in my own roadmap. I’d written that NetClaw had “no log search at all.” Wrong — Splunk, Datadog logs, GCP Logging and Grafana were all already registered. What was missing was an Elasticsearch backend. Selection between them is now by where the data lives, never by the shape of the question, and if that’s unknown the agent asks.
Documents, because the output lands in front of humans
NetClaw can render Three.js topologies, drawio, markmap, UML, Blender scenes and Unreal Engine worlds — and could not produce a change-record .docx, an exec .pptx, an interface-audit .xlsx, or fill a PDF form. Its output lands in front of enterprise humans who want documents.
The roadmap called this a “vendor the four official skills” item. That was wrong, and finding out why is useful: the four anthropics/skills document skills are source-available “for demonstration and educational purposes only” — not Apache-2.0. (The repo’s example skills are; the document skills specifically are not.) NetClaw ships Apache-2.0, so vendoring them isn’t licence-compatible. R18 became build-rather-than-adopt for a licensing reason, which is a different situation from R1, R3 and R9 where the community options were technically inadequate. Upstream stayed extremely useful for deciding which capabilities matter.
document-mcp: 6 tools, 2 skills, 1,232 tokens, 240 assertions, all four formats live-verified from a real FortiGate and opened in real applications. One server owns all document writing, so generation time, attribution and per-element provenance get stamped at a single chokepoint; skills own the compositions and contain no writing logic at all.
Provenance has to be visible — a source column on every spreadsheet row, a per-figure source in documents and decks, a sources section in every file. Cell comments and document metadata do not satisfy it; nobody reads those.
And the rule the whole server exists to enforce: a document must never fabricate to fill a blank. An empty cell in a generated audit is the correct output when there is no data. A plausible number is a catastrophe wearing a suit.
The assertion layer
The last item of the run, spec 098, is the one I’d been circling without realising it.
NetClaw can read state (pyATS, the new CLI driver, gNMI). It can read what the manager says (CloudVision, FortiManager, Catalyst Center). It can read state over time (SuzieQ, Zabbix). It had no assertion layer — nothing that takes a declarative expectation and returns a structured pass/fail verdict.
Arista’s ANTA is exactly that, and the triage in spec 097 picked it for one reason above all others: it was the only candidate whose verification path needed nothing obtained. A vEOS image was already on disk, containerlab was already installed, ANTA is pip-installable, no vendor account required.
anta-mcp: 4 tools reaching a 208-test catalogue for 1,272 tokens. One tool per test would have been roughly 58,000 — 11.6× over the ceiling — so it uses the dispatcher-plus-discovery shape that Catalyst Center established, and the manifest was counted, not estimated. Verified live against vEOS-lab 4.36.1F over eAPI.
It runs in its own virtualenv, and not by preference: ANTA moves cryptography from 46.0.5 to 50.0.0, and four installed distributions depend on it unbounded — including the federation TLS stack. Caught by a dry run before installing anything, which is a habit spec 076 forced into the process.
Its silent wrong answer is the sharpest of the whole run: ANTA reports a test for an unconfigured feature as a failure.VerifyBGPPeerCount returns “BGP inactive” as a failure on a switch that simply has no BGP configured. A fleet report built on that reads like a disaster and describes a healthy network. The server reclassifies to not_applicable with a deliberately narrow rule that can never hide a real failure — and then emits no health percentage at all, because passed / total is meaningless the moment not_applicable is in the denominator.
Two specs that shipped a decision instead of a server
These are the ones I’m proudest of, which probably says something.
Spec 093 started life as “add Excalidraw and draw.io MCPs.” I mentioned it, and the response was essentially hang on, doesn’t NetClaw already have a pile of diagramming? It did. Measured: the drawio-diagram skill already ships native .drawio files with CLI export to PNG, SVG and PDF; uml-diagram covers 27+ types via Kroki — Mermaid, D2, Graphviz, C4, BPMN, ER, sequence, state — plus markmap, AWS architecture, Visio, canvas, Three.js, Unreal and Blender. Excalidraw adds a hand-drawn aesthetic, not a capability. R22 closed as already satisfied.
The audit that closed it found something real, though. Three Microsoft Graph skills all invoke npx -y @anthropic-ai/microsoft-graph-mcp — and that package 404s on npm. Between them they documented 17 invocations across 14 distinct tool names, none of which could ever run. Same defect class as the phantom FortiManager server and the 54 imaginary Meraki methods: documentation that reads as capability. Every other package a skill invokes was checked — 17 distinct npx/uvx references, and this was the only missing one, so the problem is narrow rather than systemic.
The genuine replacement measures 188 tools ≈ 225,355 tokens — 45× the ceiling, by far the worst measured to date. It supports filtering, so it was adopted as a 12-tool subset. And there’s now a gate surface that verifies every package a skill invokes actually exists.
Spec 095 measured Juniper’s official Mist MCP against the live endpoint using my own org: 7 tools, 11,783 tokens, 2.36× over the ceiling — counted with a real tokenizer, not estimated — and there’s no tool-filtering key in the config across 101 registered servers, so a cheaper subset can’t be loaded. That’s the opposite failure mode from every prior rejection: not too many tools, but ~1,678 tokens per tool.
Three findings shipped with the rejection, and they’re all reusable:
The chars÷4 estimate under-reports by 17% near the ceiling — 10,052 estimated versus 11,783 measured. Enough to turn a “fits” into a breach. Count, don’t estimate.
get_mist_insights requires a query_type parameter its own schema never declares.
sites_sle on an empty org returns count: 1 with no metrics — so a site with no telemetry and a site with no problems are the same shape.
That last one is why the build is blocked rather than merely unstarted: my verification org has one site and zero devices, so the exact failure mode a wireless assurance skill exists to prevent cannot be exercised. Assurance skills whose central failure mode can’t be reproduced don’t get built. The spec, the exit conditions and a re-measurement script all shipped so it can be picked up the day a populated org exists.
The through-lines
Individual integrations are replaceable. These four patterns are what I’d actually carry to the next project.
1. The token ceiling decided more designs than any architectural preference
Every tool a server advertises costs tokens on every single turn, forever, whether or not it’s called. NetClaw budgets 5,000 tokens per server manifest. Look what that one number decided:
Integration
Tools
Manifest
Outcome
Elasticsearch
5
1,094
adopt as-is
Meraki
2
1,561
adopt as-is
Kubernetes
21 → 7
5,716 → 1,643
adopt, trimmed
Catalyst Center
515 → 10
64,420 → 1,821
build a dispatcher over the catalogue
Arista ANTA
208 tests → 4
~58,000 → 1,272
build a dispatcher
Juniper Mist
7
11,783
reject — no filtering mechanism exists
Microsoft 365
188
225,355
adopt a filtered 12-tool subset
Also worth noting: a tool count is not a capability count. Globalping advertises 12 tools, but 6 of them take only a context argument. The real capability is 5 measurement tools.
2. “Nothing came back” is not “nothing is wrong”
This is the one I’d tattoo on an agent if I could. Ten specs in a row shipped a named refusal state rather than an empty success, because an empty result and a failed question look identical in the data and only one of them means the network is fine.
Spec
The confusion that is now structurally blocked
078
No advisories ≠ not vulnerable. A normalisation failure ≠ a clean bill of health
079
no_probes_found ≠ an outage
080
No logs ≠ the rule is unused
081
RPKI not-found ≠ invalid. An unreachable validator ≠ unsigned space
084
A discarded 403 ≠ a one-namespace cluster
091
Zero loaded signatures ≠ zero alerts
094
A BMC timeout ≠ a downed host
095
An empty org ≠ a healthy org
096
hits.total: 10000, gte ≠ 10,000 documents
098
An unconfigured feature ≠ a failed test
A confidently wrong answer is far worse than a refusal. Every one of those rows is a case where the tool would have answered smoothly and the answer would have been false.
3. Sequence by verifiability, not by value
Three items stalled on the same question in a single week: can this be verified today, with the access I actually have?
ntopng’s free edition can’t do the job — flow history to ClickHouse is hard-gated to Enterprise M+, verified in ntop’s own source, and it fails open: with the licence missing it logs a warning, silently disables storage, and starts healthy. Dynatrace and New Relic are SaaS-only with no self-hostable path. Mist’s org is empty.
Each of those was worth building and could not be proven. An item that ships unverified isn’t a closed item; it’s a claim. Confirming access costs minutes before the spec is written and costs a day after.
4. A roadmap item is a hypothesis, not an instruction
Four premises in my own roadmap did not survive first contact:
Item
The premise
What measurement found
R2
Four Cisco API families
Three return 403 — rescoped to PSIRT alone
R10
ntopng gives flow history
Enterprise M+ only, verified in ntop’s own source
R17
A query layer is needed now
Nothing to query until 091 produced exports — resequenced
R22
Diagram coverage is missing
Already satisfied; Excalidraw is an aesthetic, not a capability
And twice, the premise was questioned by a human and the human was right. The measurement is the first deliverable of every item, not the paperwork you do afterward.
Where it stands
Of twenty-five roadmap items, seventeen are closed: sixteen done, one done-and-narrowed, one closed as already satisfied. One is deferred with evidence. One is blocked on a populated vendor org, with the build fully specified and a re-measurement script shipped. Seven remain unstarted, and they’re honestly triaged into “buildable and verifiable today” versus “blocked on access, not on effort” — Palo Alto needs an image, vSphere needs a vCenter, Aruba Central needs a tenant with devices.
Everything above is merged to main. The reconciliation gate exits zero across all seven surfaces, and CI fails the merge if it doesn’t.
221 skills. 163 MCP integrations. Six days.
The roadmap lives at docs/COVERAGE-ROADMAP.md in the repo, with every disposition, every measurement, and every corrected premise recorded in place rather than quietly patched. If you want to know why something wasn’t built, the reason is written down — which, for my money, is the part that makes a roadmap worth keeping.
