pyATS MCP has been live since April 2025 — one of the first MCP servers to put Cisco pyATS and Genie in front of an AI agent. It worked. It also predates almost everything MCP has shipped since: the SDK was on 1.x, the transport was STDIO only, and the tool surface stopped at show commands, config, and a sandboxed test runner. This is the write-up for bringing it forward — Streamable HTTP, nine new tools, a real CML lab, and two bugs from the original release that had been sitting there, unnoticed, the whole time.
Where this started
The ask was simple on its face: add tools from a companion skills repo, modernize the transport to whatever MCP currently considers “stateless,” and benchmark before against after. Three sentences. What actually happened was closer to a full audit of a fifteen-month-old codebase, conducted against a real, live lab, with the lab occasionally rebooting mid-investigation.
I’ll take the mistakes in the order I made them, because two of them are worth more than the features.
The mistake I made in the first ten minutes
The plan called for mcp>=2.0.0. I ran pip install mcp==2.0.0 to go check its real API before writing anything — and did it in the global environment, not a project venv.
The install succeeded. Pip also printed eight lines of dependency-conflict warnings I almost scrolled past, because that same machine runs several other MCP servers out of the same global site-packages: fastmcp, uml-mcp, servicenow-mcp, and — this is the one that mattered — cml-mcp, the server that talks to the very CML lab I was about to need. All four pin mcp<2.0.0. I'd just broken every one of them to satisfy a version check on a server I hadn't touched yet.
Reverted immediately, back to 1.26.0 globally, then built an isolated .venv for the actual work and never touched the shared environment again. The finding worth keeping isn't "I made a mistake" — it's that the mistake was invisible until I looked at the dependency solver's output on a machine that already had other real tools installed. A throwaway sandbox with nothing else in it would never have caught this.
What "stateless" actually turned out to mean
Streamable HTTP's MCPServer.run() in mcp>=2.0.0 takes a stateless_http flag — True means no server-side session state kept between requests. That's the obvious reading of "stateless," and it's the one I built the first pass of the benchmark plan around.
It's also not the interesting change. SEP-2575, the MCP spec revision dated 2026-07-28, removes the initialize/initialized handshake and the Mcp-Session-Id header from the protocol core entirely — every request instead carries its own _meta block with protocol version, client info, and capabilities. That's a different axis from the transport-level flag, and the SDK does it automatically for any client that negotiates the new protocol version, with zero code required on the server side. I'd conflated "the kwarg named stateless_http" with "the thing SEP-2575 actually changed" until I went and read the SEP text directly.
Both axes turned out to matter for the benchmark, they just answer different questions — which is exactly why the final run measured four conditions, not two.
What my first pass never got to
I wrote a book on pyATS and Genie. The gap between what I've spent years teaching people to do with this framework and what my own MCP server actually exposed was bigger than I'd like to admit. The original five tools were exactly what a CLI-first engineer reaches for on day one — run a show command, push config, learn the running config, learn logging, ping. Real coverage. Also the floor of what pyATS can do, not the ceiling.
The parts I'd wanted in there for years never made the cut the first time, mostly because none of them fit the shape of "wrap a show command":
pcall— pyATS's own parallel-execution primitive, one OS process per target instead of a loop. I reach for this constantly at scale and it had never once been in the MCP server.- Genie's general
device.learn(feature)— not "learn config" and "learn logging" as two hardcoded calls, but the actual Ops model library behind them: interface, ospf, bgp, platform, dozens of features, each normalized to a vendor-neutral structure you can diff. - Genie Clean — the staged device reset/reprovisioning framework. Deliberately scoped down to non-destructive stages here, but the hook into the real Kleenex machinery hadn't existed before at all.
- Blitz and Robot Framework — the two declarative test-authoring surfaces I point people toward specifically when they don't want to write Python. Neither had ever been reachable from an agent.
- The REST connector — pyATS isn't CLI-only; it's had a real REST/RESTCONF/NX-API connection type for years, entirely separate from the SSH path every one of the original five tools used.
None of that is exotic. It's close to the table of contents. It just hadn't made it into the MCP server, because the first version was built to prove the idea worked at all, not to be comprehensive.
Nine new tools, verified against a real lab
The companion skills repo's coverage — pcall, clean, learn, blitz, mock devices — doesn't map onto pyATS's actual Python API as cleanly as the skill names suggest. Two of the five have no supported in-process entry point at all.
| Tool | What it actually is |
|---|---|
pyats_pcall_show_command / pyats_pcall_configure_devices |
Real pyats.async_.pcall — one forked OS process per device, not the thread pool the existing multi-device tools use |
pyats_learn_feature / pyats_diff_learned_snapshots |
Genie device.learn() with named before/after snapshots and genie.utils.diff.Diff |
pyats_clean_device |
Genie Clean has no supported public API — DeviceClean is an internal class coupled to aetest's global executer state. Shells out to the real pyats clean CLI instead, restricted to non-destructive connect+execute_command stages only, dry_run=True by default |
pyats_run_blitz |
Same story — Blitz imports the global pyats.easypy.runtime singleton, so it has to run through a real job, not a function call |
pyats_run_robot |
Robot Framework via the actual installed pyats.robot / genie.libs.robot keyword libraries — there's no pyats robot CLI subcommand in this version, so it runs through the standalone robot CLI |
pyats_rest_request |
Generic REST/RESTCONF/NX-API via pyATS's own rest.connector — a second, separate connection type from the CLI/SSH one every other tool uses |
pyats_xpresso_request |
Cisco XPresso's REST API v2, built from the published docs. No live instance to test against — flagged as unverified in the tool's own docstring rather than presented as equal-confidence to everything else |
"Mock devices" — the fifth skill — never became a user-facing tool. It became a benchmark fixture instead, once it was clear the actual value was repeatable timing runs, not another tool an agent would call.
Two bugs from April 2025, still there
Every one of the nine tools got a real call against a real CML lab before I called it done — four IOS-XE devices, reachable once I worked out which network they were actually on (more on that below). That process surfaced two defects in code that shipped over a year ago and had presumably been called successfully many times since.
pyats_run_dynamic_test's report was always empty. The original code passed --json-job <path> to pyats run job and read the report back from that path. That flag does not exist in this pyATS version's CLI. Not deprecated — never present in --help at all. The argument parser accepts it silently, produces no report file, and the job still exits 0. The tool never crashed, never logged an error, and its report field was quietly None on every single call since launch. pyATS does archive every run to a real zip under ~/.pyats/archive/ by default, with an actual results.json inside — the fix reads that instead, and now returns a genuine structured report with pass/fail counts.
Blitz jobs failed on any lab with one unreachable device — including devices the job never asked for. pyats run job invokes genie.harness's common_setup, which by default connects to every device in the testbed handed to it, not just the ones a specific trigger targets. Point Blitz at a testbed where three of seven devices aren't up yet, ask it to run one command against a device that is up, and the whole job errors out in setup before it ever reaches your device. The fix generates a testbed scoped to only the devices a given call actually needs.
Same shape both times: a tool that looks like it's working — no exception, no error field, a plausible-looking exit code — while quietly not doing what it claims. Neither would show up in a unit test with a mocked device. Both only showed up because something insisted on hitting real hardware before calling the work finished.
The testbed was pointing at the wrong network
The CML lab's testbed.yaml had four IOS-XE devices at 10.10.20.171–174. None of them were reachable. ip route showed no path to that subnet at all — not a timeout, a routing-table gap.
The lab's own OOB-MGMT network told a different story. One device's running config, pasted mid-investigation, showed an interface configured at 192.168.2.201 — a subnet that was reachable. Rather than guess at the other three, I used the credentials already sitting in ~/.openclaw/.env for the existing cml-mcp server to query the CML controller's own API directly (virl2_client, read-only) and asked it where it thought each device's management interface actually was. It answered with exact IPs, discovered from the lab's own ARP/L3 tables: .201, .202, .211, .212. A short port scan across the neighboring range confirmed all four were real, live, and speaking Cisco's SSH banner before a single credential was tried.
The lesson isn't "the lab was misconfigured." It's that the CML controller already knew the right answer, in a format a script could ask for directly, and guessing at the network from the outside would have taken far longer than just asking the source of truth.
The benchmark, and the answer I didn't expect
Four conditions, against the same four real devices, each running the same scenario set: STDIO on the pre-modernization server (mcp==1.26.0), the modernized server negotiating the new handshake-free protocol, and the modernized server under the old handshake-based protocol with stateless_http toggled both ways.
| Condition | Connection setup |
|---|---|
STDIO (pre, mcp==1.26.0) |
1.074s |
| Streamable HTTP, modern protocol | 0.025s |
| Streamable HTTP, legacy protocol, stateful | 0.023s |
| Streamable HTTP, legacy protocol, stateless | 0.026s |
Streamable HTTP sets up a connection roughly 40–50× faster than STDIO — no surprise, since STDIO means forking a fresh Python process and re-importing pyATS and Genie every single time, and HTTP just opens a request against a process that's already running.
Per-call latency for the actual tools, though, was identical across all four conditions — single show command, thread-pool fan-out, pcall fan-out, device health, all within noise of each other, all around 15–18 seconds. Connection caching defaults to off, so every call does a real SSH connect and disconnect to a CSR1kv, and that cost swallows any transport-layer difference completely. The one signal that did survive the noise: pcall's fan-out ran consistently ~150–200ms slower than the thread-pool version, which is exactly the fork overhead its own documentation describes.
I could have reported "Streamable HTTP is faster" and stopped there, and it would have been technically true and substantially misleading. The honest version is narrower: modernizing the transport buys you cheap, frequent connections — which matters if you have many short-lived clients — and buys you almost nothing on a workload that's bottlenecked on SSH negotiation to real gear. Both of those are real findings. Only one of them is the one a changelog would lead with.
Where it stands
26 tools now (the tool table in the README had been silently missing two working tools — pyats_show_running_config and pyats_show_logging — since launch; caught it while cross-checking the table against the actual @mcp.tool() registrations rather than trusting either version of the document). 119 unit tests, up from 85, all mocked, no real devices required to run them. A benchmark/ harness that stays in the repo rather than living only in a chat transcript. And a rewritten README with real, source-verified instructions for wiring the server into Claude Code, VS Code's Copilot Chat, OpenAI's Codex CLI, Claude Desktop, and raw Python — including the one that looks like it should just work and doesn't: Desktop's config file is stdio-only, a url field in it fails silently, and the actual supported path is either a Custom Connector against a public HTTPS endpoint or an mcp-remote stdio bridge for anything running on your own machine.
Merged to main as PR #13. CI is temporarily disabled — out of Actions minutes, not a statement about the pipeline — and the one thing still red when it comes back is a black formatting check that was already failing a month before any of this started.
The uncomfortable part
I wrote a book to teach people how pyATS and Genie actually work — the Ops model, the connection classes, when to reach for Blitz instead of a raw testscript, why pcall exists instead of a loop. That book's entire premise is that understanding the framework is the value. Every tool in this post is an abstraction that lets an agent reach the same capability without a human needing to know any of that. Ask it to snapshot OSPF state, push a change, and diff what moved, and it picks pyats_learn_feature and pyats_diff_learned_snapshots on its own. Nobody has to have read the chapter first.
I don't think that makes the book worthless. I think it means the value moved. Fewer people need to know pyATS's API to get something real done with it, which was arguably the whole point of automation to begin with. It does mean I'm the one shrinking the book's core pitch, one tool at a time. I'm fine with that trade. I'd rather be the one making myself obsolete than watch someone else do it slower.