Modern AI has its own concepts, vocabulary, and ways of working that are unfamiliar even to experienced engineers. This course is the technical front door to the catalogue: it gives practitioners the grounding they need before entering any of the intermediate and advanced tracks, so they are not learning fundamentals and specifics at the same time. Students build an accurate mental model of what today’s AI is, how language models behave, what the application landscape looks like, and how quality and responsibility are handled. The emphasis is on genuine understanding rather than hype or hand-waving. Labs are gentle, guided exercises that let students touch each concept directly, so they finish able to reason about AI systems and ready to go deeper in DEV, MLE, or the other tracks.
Audience Profile • Developers and engineers new to AI • Technical staff about to work on AI projects • Practitioners who need fundamentals before deeper courses • Anyone wanting an accurate technical grounding in AI
Prerequisites • General software or technical familiarity • Comfort with basic programming for the light labs • No prior AI or machine-learning experience required • Curiosity and a willingness to question hype
You ask your coding agent to add a new endpoint. It churns for a minute and produces beautiful, idiomatic code that compiles on the first try. You merge it. Then you deploy, and three services fall over. The agent ignored your authentication layer, bypassed the validation patterns every other handler uses, and pulled in a dependency that conflicts with your stack.
None of that was in your prompt. You assumed the agent would just know — it’s been trained on millions of repositories, after all. But it had never seen your repository, your conventions, or your production constraints. The prompt was fine. The context was the problem.
This is the realisation that reorganised how serious teams work with coding agents in 2026: the bottleneck is no longer the model’s ability to write code. Claude Code, Cursor, and Copilot can all generate code fluently. Their ceiling is the quality, relevance, and durability of the context they receive. Managing that context is an engineering discipline in its own right, and it has a name.
What context engineering is
Anthropic formalised the term in September 2025, and the definition that’s stuck is deliberately economical: context engineering is finding “the smallest possible set of high-signal tokens that maximise the likelihood of some desired outcome.” The academic framing (Mohsenimofidi et al., October 2025) is broader — “the deliberate process of designing, structuring, and providing relevant information to LLMs” — and by early 2026 Gartner had its own enterprise version. They all point at the same thing.
The cleanest contrast is with the discipline it replaced:
Prompt engineering optimises the phrasing of a single request. It’s a writing skill.
Context engineering optimises the entire information environment the model sees across a multi-step session — system instructions, tool definitions, retrieved code, memory, prior tool outputs, and the task itself. It’s a systems engineering discipline.
The punchline that makes the distinction land: a perfect prompt inside a bloated, irrelevant context still produces mediocre output, while a mediocre prompt inside a surgically curated context often produces great output. Once you’ve seen that a few times, you stop polishing prompts and start engineering context.
The mental model: context is RAM
Andrej Karpathy, who helped popularise the term in mid-2025, offers the analogy that makes it click: an LLM is like a new kind of operating system. The model is the CPU, and its context window is the RAM — the working memory. Just as an operating system carefully curates what fits into limited RAM, context engineering decides what fits into the limited context window on every single turn.
That framing reframes the whole job. You’re not writing instructions; you’re managing a scarce, shared resource — working memory — under contention from many sources, all competing for the same finite space.
Why more space doesn’t solve it: context rot
The natural objection is “context windows are huge now — just put everything in.” This is the most expensive mistake in the field, because of a phenomenon called context rot: model accuracy measurably degrades as the input grows, regardless of the window’s size.
Chroma’s 2025 study tested 18 frontier models and found all of them degrade as context lengthens — through “lost in the middle” (information buried mid-context gets overlooked), attention dilution, and distractor interference (irrelevant tokens actively pull the model off course). Bigger windows raise the ceiling; they do not repeal the rot.
The practical consequence is counterintuitive but firm: filling the window hurts you. A 15-step refactor — tool calls, file reads, shell output, sub-agent hand-offs — can balloon past 100,000 tokens before the model does any real thinking, and quality drops accordingly. Even with a 2-million-token window, you want the model to see only what’s useful for the current step, because latency, cost, and rot all scale with what you load. The job is curation, not accumulation.
The four strategies (the map for this series)
LangChain’s Lance Martin distilled the community’s practices into a four-strategy taxonomy that has become the standard vocabulary. Every technique in this series is one of these four:
Write — persist context outside the window so the agent can pull it back later. Context files (Claude Code’s CLAUDE.md, Cursor/Windsurf rules files), scratchpads (todo.md), and memory stores live here.
Select — pull the right context in at the right moment: retrieval over your codebase, memory selection, choosing which files matter for this task.
Compress — keep only the tokens a step actually needs: summarisation, compaction, returning references instead of blobs.
Isolate — separate unrelated context to prevent interference: sub-agents with their own windows, per-task stores, the file system as external memory.
What you’re actually supplying
This series is organised around the four kinds of context the model can’t get from pretraining — the things that make code yours:
Coding style & conventions — your house patterns, architecture invariants, the “never do X” rules. (Mostly Write)
Internal libraries & code — your private APIs, utilities, and the right way to call them. (Mostly Select / retrieval )
Institutional knowledge — why decisions were made, runbooks, tribal knowledge that lives in people’s heads. (Write + Select, plus memory)
Production data — real schemas, configs, and live state, supplied safely. (Select + Isolate, under governance)
A coding agent that compiles but takes down production is one that received zero of these. The discipline is making sure it receives exactly the right slice of each, on each turn, without drowning in the rest.
Write: context files that actually change behavior
The simplest, highest-leverage context engineering you can do is also the most under-used: write down what the agent should know in a file it reads on every session. Claude Code calls this CLAUDE.md; Cursor and Windsurf use rules files; and a cross-tool convention, AGENTS.md, has emerged so the same context works across agents. This is procedural memory — the standing instructions that encode how your team builds software.
The mistake teams make is treating these files like documentation. They’re not docs; they’re a token budget you’re spending on the agent’s behavior, so every line has to earn its place (remember context rot from above — a 5,000-line CLAUDE.md is itself bloat). Good context files are short, specific, and imperative.
# AGENTS.md — payments-service
## Architecture invariants (never violate) - All money is integer minor units (cents). Never use floats for currency. - Every handler goes through `withAuth()` and `validate(schema)` — no exceptions. - DB access only via `src/db/repo.ts`. Never write raw SQL in handlers.
## Conventions - Errors: throw `AppError(code, msg)`, never return null on failure. - Tests: co-located `*.test.ts`, use the `makeFixture()` factory. - Logging: `logger.info({event, ...fields})` — structured, never `console.log`.
## Project facts - Default branch `main`; CI is GitHub Actions; Node 22. - Public API contract lives in `openapi.yaml` — update it when routes change.
## Never - Add a dependency without checking `package.json` for an existing one. - Touch `legacy/` — it's frozen pending migration.
Notice what this does: it front-loads the invariants (the auth and validation rules whose absence took down production in previous story), states conventions as commands, and ends with explicit prohibitions. It’s the institutional knowledge that normally lives in a senior engineer’s head, made machine-actionable.
Layer your context files. Context isn’t monolithic — it cascades from broad to specific, and the agent should load the relevant layers for wherever it’s working:
A practical loader concatenates the layers from general to specific, so the most local rules win:
from pathlib import Path
def build_context_files(target_file: Path, repo_root: Path) -> str: """Compose context from org → repo → directory, most-specific last.""" layers = [] org = Path.home() / ".agent" / "GLOBAL.md" if org.exists(): layers.append(("organisation", org.read_text())) repo_md = repo_root / "AGENTS.md" if repo_md.exists(): layers.append(("repository", repo_md.read_text())) # walk from repo root down to the target's directory, picking up local rules for parent in [target_file.parent, *target_file.parents]: if parent < repo_root: break local = parent / "AGENTS.md" if local.exists() and local != repo_md: layers.append((f"local:{parent.name}", local.read_text())) return "\n\n".join(f"<!-- {name} -->\n{body}" for name, body in layers)
Write also covers scratchpads. For state within a long task, have the agent persist a todo.md or plan file outside the window and reload only the summary. This is how agents track multi-step progress without carrying every intermediate thought in context — a Write strategy that directly fights the bloat as seen before.
Select: retrieving your internal libraries
Context files handle the knowledge that fits in a page. They can’t hold your entire codebase — and they shouldn’t try. For “how do we call the internal billing client?” the agent needs retrieval: pulling the relevant code into context on demand. This is where teams reach for RAG, and where a naive implementation quietly fails.
The trap: embedding search alone doesn’t scale on code. As Windsurf’s team documented, indexing is not retrieval. Embedding-based similarity search becomes an unreliable heuristic as a codebase grows — semantically similar code isn’t necessarily the code you need, and chunking by line count shreds functions mid-body. The fix the strong code agents converged on is a hybrid pipeline:
The three retrievers cover each other’s blind spots: grep nails exact symbol names, embeddings catch “code that does something similar,” and the code graph follows real relationships (who calls this, what it imports). A re-ranking pass then orders the merged candidates by genuine relevance so only a tight top-k enters context — keeping the window small and high-signal.
def retrieve_code(query: str, k: int = 8) -> list[Snippet]: # 1. fan out to complementary retrievers by_keyword = grep_search(query) # exact symbols, fast, precise by_meaning = embedding_search(query, chunks) # AST-aware chunks, semantic by_graph = code_graph_neighbours(query) # callers/callees/imports
# 3. re-rank by true relevance, then take a tight top-k ranked = rerank(query, candidates) # cross-encoder or LLM judge return ranked[:k]
Two implementation notes that matter more than the retriever choice. Chunk along AST boundaries (whole functions/classes), never fixed line counts, so a retrieved snippet is self-contained. And return the smallest useful unit — a function plus its signature and docstring, not the whole 2,000-line file — because what you retrieve, you pay for in context-rot terms.
You can expose this pipeline to the agent as a tool it calls (search_code(query)), or wire an existing code-graph source through MCP. Either way, the agent now pulls your internal libraries on demand instead of hallucinating their interfaces.
Select: institutional knowledge and memory
The third kind of context — why things are the way they are, runbooks, decisions, tribal knowledge — is partly Write (put durable facts in context files) and partly memory: knowledge that accumulates across sessions and is selected back when relevant.
For semantic memory (a growing store of facts and relationships), you select with the same retrieval machinery as code, indexed by embeddings or a knowledge graph. A few frameworks exist — mem0 and Letta are the visible ones — but most production teams build a thin memory layer over a key-value store plus a summarisation pass, because requirements vary. The pattern:
def recall(task: str, store, k: int = 5) -> str: """Select only memories relevant to the current task.""" hits = store.search(task, k=k) # semantic match return "\n".join(f"- {m.text}" for m in hits) # inject just these
def remember(fact: str, store): """Write a durable fact learned this session for future selection.""" store.upsert(fact, embedding=embed(fact))
The discipline is selective recall. Don’t dump the whole memory store into context — that’s how you recreate context rot. Select the handful of memories relevant to the task, the same way you select code. (And beware over-eager memory: an agent that silently injects stale or irrelevant “facts” is worse than one with none.)
Putting Write and Select together
A well-contexted coding agent, at the start of a task, assembles: the layered context files for where it’s working (Write), the top-k retrieved code for the symbols it’ll touch (Select), and the relevant memories about prior decisions (Select) — and nothing else. That assembled context is small, specific, and high-signal. The same agent without this receives a generic prompt and guesses — which is exactly how you get idiomatic code that ignores your auth layer.
Compress: fighting context rot in long sessions
Everything from above degrades over a long session. Each tool call, file read, and shell output adds tokens, and by step 15 you’re back in the context-rot zone — accuracy sliding as the window fills with spent material. Compress keeps the window lean by retaining only what each step still needs.
Three techniques, in rough order of how often you’ll reach for them:
Compaction is the workhorse. When a session approaches the context limit, summarise the conversation so far at high fidelity and restart a fresh window seeded with that summary. Long-range coherence survives; the token-heavy raw history doesn’t. Most production agents do this automatically, but you control what the summary preserves — and that’s a context-engineering decision: keep decisions, open threads, and invariants; drop resolved sub-tasks and verbose tool dumps.
def maybe_compact(messages, model, limit=120_000): if count_tokens(messages) < limit: return messages summary = model.summarize( messages, keep="decisions made, current plan, open questions, files modified, " "invariants discovered; DROP resolved steps and raw tool output", ) system = messages[0] return [system, {"role": "user", "content": f"[Compacted context]\n{summary}"}]
Offloading moves bulky content out of the window and keeps a reference. Compress a 50,000-token file or webpage to a 500-token summary plus a path/URL; the agent retrieves the full content only if a later step needs it. Teams report agents naturally adopting this with todo.md and scratch files — the file system as recoverable, near-infinite memory.
Structured tool outputs. Design tools to return the smallest useful result — an ID, a count, a path — not a giant JSON blob the agent has to carry. A tool that returns {"matches": 412, "sample": [...3 items...]} is far kinder to the window than one that returns all 412 rows.
Isolate: sub-agents with their own windows
Some work doesn’t belong in the main agent’s context at all. Isolate gives a sub-task its own fresh window, so its intermediate tokens never pollute the parent’s working memory. The parent delegates a goal and gets back only the result.
The payoff is large and well-documented: Anthropic’s multi-agent research system divided work among sub-agents that each explored independently with their own context windows, then had a lead agent synthesise — a 90.2% performance improvement over the single-agent baseline on their evaluation, precisely because each sub-agent reasoned in a clean, focused window instead of one shared, crowded one. (It used more total tokens — isolation trades token volume for quality and parallelism, which is usually the right trade for hard tasks.)
Isolation shines for parallelisable, mostly read-only work: “find every call site of this deprecated API,” “research how three modules implement caching,” “review these twelve files for the same issue.” Each becomes a sub-agent with a narrow brief and a clean window; the parent stays uncluttered.
def isolated_subtask(brief: str, tools) -> str: """Run a sub-agent in its own fresh window; return only the distilled result.""" sub = Agent(system=focused_system_prompt(brief), tools=tools) # clean context result = sub.run(brief) return result.summary # only this crosses back into the parent's window
The judgement call: isolate when a sub-task’s process is verbose but its result is compact. Don’t isolate tightly-coupled work where the parent needs the intermediate reasoning — you’ll just pay coordination overhead.
Supplying production data — safely
The fourth kind of context from before — real schemas, configs, live state — is the most valuable and the most dangerous. An agent that can see production data can be far more accurate; an agent that can see too much production data is a breach waiting to happen. Supply it under hard controls:
Read-only, least-privilege access. The agent gets a scoped, read-only credential to exactly the tables/resources the task needs — never a broad production role, never write access by default. This is the same containment discipline a shell agent needs, applied to data.
Redaction and PII handling at the boundary. Strip or tokenise secrets and personal data before it enters the context window. A retrieval tool over production should return schemas and shapes, sampled/synthetic rows, and aggregates — not raw customer records.
Prefer metadata over payloads. Most of the time the agent needs the schema (“what columns exist, what types”), not the data. Supply the structure; withhold the contents unless a task genuinely requires them.
Audit every access. Log what the agent read, when, and why. Production-data access through an agent should be as auditable as any other privileged access.
Route this through a controlled tool or an MCP server with these guarantees built in, rather than handing the agent a live connection string. Production data is exactly the kind of governed, cross-trust-boundary access where a structured, auditable interface earns its overhead.
ContextOps: govern it like code
Here’s the part that separates teams that win with coding agents from teams that generate expensive technical debt. The context you’ve built — files, retrieval config, memory, data policies — is infrastructure, and infrastructure needs governance. The gap is real: ~91% of engineering organisations have adopted at least one AI coding tool, but few have governance matching that adoption, and roughly 48% of AI-generated code has been found to carry security vulnerabilities. Ungoverned context is where that risk compounds.
Treat context as a first-class, version-controlled asset:
Version control your context files.AGENTS.md and rules files live in the repo, reviewed in PRs like any code. A convention change is a diff, not a Slack message.
Enforce at pre-commit. The invariants you wrote into context files should also be checked mechanically — linters, schema validation, dependency rules — so the agent’s adherence is verified, not trusted. Context tells the agent the rule; the gate confirms it followed it.
Monitor for drift. Codebases evolve; context files rot. Watch for context that’s gone stale (a convention the code no longer follows, a retrieval index that’s behind) and treat it as a bug.
Assign ownership. Someone owns each layer of context, the way someone owns a service. Orphaned context decays.
The reason this matters compounds over time: a well-governed context environment doesn’t just improve this quarter’s output. It accumulates your institutional knowledge in a form every agent on the team can act on — and as agents get more autonomous, that asset is what determines whether autonomy amplifies good patterns or replicates chaos across entire features with no human in the loop to catch it.
The whole system
Put all three parts together and a production coding agent’s context, on any given turn, is assembled like this:
Style and knowledge come in through Write and Select. Production data comes in through governed, least-privilege Select. Compress and Isolate keep the window small as work grows. And the governance loop keeps the whole thing honest over time. What reaches the model is a tight, high-signal context — your conventions, the relevant libraries, the pertinent decisions, the necessary data shape — and nothing else.
That’s the discipline. Not a clever prompt, but an engineered information environment: curated on every turn, governed like code, and compounding into the single most valuable asset a team building with agents can own.
Here is the benchmark that’s been driving architecture decisions across the industry in 2026: a typical CLI command costs an agent around 200 tokens. The equivalent operation through an MCP server costs 32,000 to 82,000 tokens.
That’s not a typo, and it’s not cherry-picked. Independent benchmarks from Scalekit, Apideck, and others keep landing in the same range — roughly a 35× overhead for MCP on identical tasks. When “MCP is dead. Long live the CLI” hit the top of Hacker News and Perplexity’s CTO publicly described moving away from MCP internally over context waste, they were all pointing at the same arithmetic.
This series is about that arithmetic — where it’s real, where it isn’t, and how to design around it. We start with the cost itself, because once you see where the tokens go, every later decision gets easier.
Where the tokens actually go
The gap comes down to one architectural difference: when does the agent pay for a tool’s definition?
MCP loads everything, always. When an agent connects to an MCP server, the entire tool catalog — every tool’s name, description, and full input/output JSON schema — is injected into the context window. It sits there on every single completion request, whether the agent calls ten tools or zero. The GitHub MCP server exposes ~93 tools; loading it costs roughly 55,000 tokens before the agent reads its first instruction. The agent is carrying schemas for creating gists, configuring webhooks, and managing PR reviews even when all it wants is the repo’s primary language.
CLI pays only when it calls. A command-line agent starts with zero tool context. When it needs GitHub, it runs gh repo view — and the model already knows gh from training, so the command plus its output might cost 200 tokens. No catalog. No schema. No discovery step that loads 92 tools it will never touch.
The stacking problem
A single server is survivable. The trouble is that real agents connect several. Add GitHub, a database connector, a project tracker, and a cloud provider, and a widely-cited Apideck measurement shows three MCP servers consuming 143,000 of a 200,000-token window — about 72% gone before the agent reads its first user message.
Now do the cost math at production scale. At roughly \$3 per million input tokens, 55,000 tokens of schema is about \$0.16 per session. Run 10,000 automated sessions a day — an unremarkable volume for a production pipeline — and you’re spending ~\$1,600 every day just loading tool definitions, before the agent solves anything. That’s the line item teams started calling the “MCP tax.”
The cost you can’t see: reasoning budget
Token cost is the headline, but it’s not the most important number. The deeper problem is cognitive.
A context window is also the agent’s working memory. Every token spent on tool schemas is a token not available for reasoning about the actual task. When 70% of the window is consumed by definitions, the model is trying to think in the cramped space that’s left — and quality degrades, especially late in a long task when accumulated tool output has pushed important context toward the edges of the window where attention is weakest.
This is why the cost gap reappears as a reliability gap. In Scalekit’s benchmark, CLI agents completed tasks with 100% reliability while the MCP equivalents came in at 72% — and most of the MCP failures weren’t logic errors but connection timeouts to a remote server. On a token-efficiency score (work completed per token spent), CLI scored 202 to MCP’s 152, a 33% advantage: the CLI agent spent its tokens on solving the problem instead of on protocol overhead.
Why CLI is good, not just cheap
It’s tempting to stop at “CLI uses fewer tokens,” but that misses the real reason it works so well. Models have been trained on decades of terminal interactions — Stack Overflow answers, GitHub histories, Dockerfiles, jq pipelines, git invocations, Kubernetes manifests. Shell tooling lives in the model’s weights as latent knowledge. When an agent composes gh pr list --json number,title | jq '.[] | select(...)', it’s operating from prior knowledge, not parsing a schema it met for the first time three tokens ago.
MCP schemas, by contrast, carry zero pretraining advantage. They’re custom JSON the model has never seen, that must be read and interpreted fresh on every run. The token savings of CLI are almost a side effect; the structural win is that the model already fluently speaks the interface.
Measure your own context budget
Before we build anything, do the one exercise that makes this concrete for your stack: measure what your tool integrations cost on idle. Here’s a quick way to tally MCP schema overhead using the same tokenizer your model uses:
# pip install tiktoken import json, tiktoken
enc = tiktoken.get_encoding("cl100k_base") # close enough for an estimate
# Paste in the tool catalog your MCP client advertises (the result of tools/list, # including each tool's full JSON schema). Many clients can dump this. with open("mcp_tools_dump.json") as f: catalog = json.load(f)
total = 0 for tool in catalog["tools"]: cost = tokens(json.dumps(tool)) # name + description + input/output schema total += cost print(f"{tool['name']:<32} {cost:>6} tokens")
Now compare against the CLI baseline: the discovery cost of a CLI tool is whatever your-tool --help returns — usually 150–600 tokens, paid once, only if the agent is unsure. Run this against your real tool catalog and the abstract benchmark becomes your actual API bill.
The honest caveat (so you don’t over-correct)
Everything above is real, but it benchmarks the slice of the world where a CLI exists and the agent controls execution. That’s a large and important slice — and it’s exactly where production pipelines live — but it isn’t everything. There is no Workday CLI, no Greenhouse CLI; for multi-tenant products where an agent acts on behalf of a specific user, the schema tax is buying something real (identity, scope, audit) that a raw shell can’t provide. We’ll give that side a full and fair hearing a bit ahead. For now, hold the cost picture clearly, because it’s the force pushing terminal-based agents into production pipelines — and it’s earned.
CLI Agents vs AI-Native IDEs: Building CLI-First Agents
A CLI-first agent is almost embarrassingly simple in concept: instead of wiring the agent to a catalog of pre-declared tools, you give it one tool — a shell — and let it compose commands. The sophistication isn’t in the plumbing; it’s in how you shape the agent’s knowledge and contain its blast radius. Let’s build it up piece by piece.
The core loop: one tool, a whole toolbox
The entire tool surface is a single bash function. The model writes a command; you run it; you feed back the output. That’s it.
import subprocess from anthropic import Anthropic
client = Anthropic()
BASH_TOOL = { "name": "bash", "description": "Run a shell command and return its stdout/stderr.", "input_schema": { "type": "object", "properties": {"cmd": {"type": "string"}}, "required": ["cmd"], }, }
def run(cmd: str) -> str: r = subprocess.run(cmd, shell=True, capture_output=True, text=True, timeout=120) return (r.stdout + r.stderr)[:10_000] # cap to protect the context window
def agent(task: str, system: str): messages = [{"role": "user", "content": task}] while True: resp = client.messages.create( model="claude-sonnet-4-6", max_tokens=2000, system=system, tools=[BASH_TOOL], messages=messages, ) messages.append({"role": "assistant", "content": resp.content}) tool_calls = [b for b in resp.content if b.type == "tool_use"] if not tool_calls: return resp # agent is done results = [] for call in tool_calls: output = run(call.input["cmd"]) results.append({"type": "tool_result", "tool_use_id": call.id, "content": output}) messages.append({"role": "user", "content": results})
Notice what’s absent: no per-tool schemas, no MCP catalog, no discovery payload. The agent already knows gh, git, jq, az, kubectl, psql, and a thousand other commands from pretraining. The single bash tool definition costs a few dozen tokens, and that’s the entire fixed overhead — versus the 55,000 tokens a GitHub MCP server would put on every request.
Composability: the property MCP can’t match
Token cost is the famous argument, but composability is the one practitioners care about more once they’ve felt it. MCP tools don’t chain — you can’t pipe one tool’s output into another; each call is a separate round-trip with the agent shuttling intermediate state back and forth, paying tokens and latency every hop.
A shell pipeline does the whole thing in one shot:
That’s “how many PRs merged since May 1st” as a single command. The agent composes the pipe, the shell executes it locally, and one number comes back. The MCP equivalent is: call list_pull_requests, get a large JSON blob into context, reason over it, maybe call again with pagination, filter in the agent’s head. More tokens, more latency, more failure surface.
This is just the Unix philosophy — small tools that do one thing, composed with pipes — and it turns out agents are excellent at it, because the training data is full of exactly these one-liners.
The 800-token skill file (the best ROI in the benchmark)
Raw CLI works, but there’s a refinement that’s almost free and pays for itself immediately. Instead of loading 28,000 tokens of MCP schema, give the agent a tiny skill file: a few hundred tokens of plain-markdown tips about the tools it has.
In Scalekit’s benchmark, an 800-token markdown file of gh tips beat 28,000 tokens of MCP schemas — the skill-augmented agent made about a third fewer tool calls and finished about a third faster than even the naive CLI agent. It was the single best return on investment they measured. A skill file looks like this:
# GitHub (gh) tips for this repo
- Auth is already configured; never run `gh auth`. - Prefer `--json <fields>` + `jq` over scraping human-readable output. - Useful fields: number, title, state, mergedAt, author, labels. - List recent merged PRs: gh pr list --state merged --limit 50 --json number,title,mergedAt - This repo's default branch is `main`. CI is GitHub Actions. - NEVER use: `gh repo delete`, `git push --force`.
You load that string into the system prompt. It costs a rounding error in tokens and dramatically sharpens behavior, because it encodes the two things training data can’t: your conventions and the specific flags that matter for this repo. This is also the cleanest place to put guardrails the agent should always honor.
SKILL = open("skills/github.md").read() # ~800 tokens SYSTEM = ( "You are a CLI agent. You have a bash tool. Prefer composing pipelines. " "Use --help if unsure about a command's flags.\n\n" + SKILL ) agent("How many PRs merged since May 1st?", SYSTEM)
--help as just-in-time documentation
What about a CLI the agent doesn’t know well, or a tool with unusual flags? You don’t pre-load its manual. You let the agent fetch documentation on demand: if it’s unsure, it runs some-tool --help and pays ~200 tokens for exactly the information it needs, exactly when it needs it. This is the same pay-per-use principle we’ve seen previously, applied to documentation: progressive disclosure instead of always-on schema. The agent’s instinct to do this is worth encouraging explicitly in the system prompt, as above.
The part nobody likes to talk about: a shell is a loaded gun
Here’s the honest cost of all this power. The single bash tool that makes CLI agents efficient and composable also hands the model rm -rf, git push --force, DROP TABLE, and arbitrary code execution. MCP’s much-maligned schema overhead is partly buying something: an agent can only call tools that were explicitly declared, so the blast radius is bounded by design. A shell has no such boundary. One bad generation is one bad generation away from something you can’t undo.
So a CLI-first agent is only production-ready with containment. The non-negotiables:
Sandbox execution. Run commands inside a container or VM with no access to production credentials, scoped to a throwaway working directory — the same isolation discipline any agentic system needs.
Least privilege. The environment gets only the credentials and network access the task requires. An agent summarizing PRs does not need write access to the repo or your database URL in its environment.
A deny list and/or approval gate. Block destructive verbs outright (rm -rf, force-push, DROP, DELETE FROM without a WHERE), and require human approval for anything that mutates state. The skill file’s “NEVER use” section helps, but never rely on the model’s compliance as your only control.
Output caps and timeouts. Bound stdout (as in run() above) so a runaway command can’t flood — and evict — the context window.
def run(cmd: str) -> str: low = f" {cmd.lower()} " if any(bad in low for bad in DENY): return "BLOCKED: destructive command requires human approval." # ...then execute inside the sandbox as before
A deny list is a backstop, not a security boundary — the real boundary is the sandbox. Treat the shell’s power as something you deliberately fence in, not something you trust the model to wield carefully.
What we’ve built — and what it can’t do
We now have a CLI-first agent that’s cheap (one tool, no schema tax), composable (it pipes), well-informed (a tiny skill file), and contained (sandbox + deny list). For developer-facing work and deterministic production pipelines where a mature CLI exists, this design is hard to beat.
But re-read that sentence: where a mature CLI exists and where you control execution. The moment your agent needs to act on behalf of a specific customer, inside a multi-tenant SaaS system that ships only an OAuth API and no shell, this whole approach runs out of road — and the schema tax you’ve been avoiding turns out to be the price of something you now actually need. That boundary, and how to decide on which side of it any given integration falls, is what we’ll see next.
CLI Agents vs AI-Native IDEs: When to Use Which
If you’ve read the above, you might think the verdict is in: CLI is cheaper, more reliable, more composable, so use it everywhere. That conclusion is wrong, and the teams that act on it create a different, quieter class of problem. The token benchmark is real — but it measures roughly the 5% of integrations where a CLI even exists and where you control execution. The other 95% of enterprise surfaces are a different world. Now, we’ll give that world its due and then hand you a framework that makes the choice mechanical.
Where MCP and AI-native IDEs actually win
Be fair to the other side, because the other side is right about several things.
Services with no shell. There is no Workday CLI, no Greenhouse CLI, no BambooHR CLI — and there never will be. These are SaaS systems with OAuth APIs, custom subdomain routing, refresh tokens, and org-level access control. MCP was built precisely for these. When the only integration a vendor ships is an API behind OAuth, the “just use the CLI” advice has nothing to point at.
Acting on behalf of a specific user. A CLI agent runs in your shell with your ambient credentials. That’s fine when you are the user. It’s a non-starter when an agent acts for a specific customer across a specific tenant. MCP’s model — explicit tool declarations, per-user OAuth 2.1 with PKCE, scope enforcement, the ability to revoke one user without touching everyone else — is buying governance the schema tax pays for. As one widely-shared analysis put it: the properties that make MCP expensive are the same properties that make it governable.
Audit and compliance. Structured tool calls with declared inputs produce clean audit trails. “The agent ran some bash” does not. In regulated workflows, that structure isn’t overhead — it’s the requirement.
The interactive IDE experience. AI-native IDEs (Cursor, Windsurf, Copilot-style tools) lean on always-on rich context and MCP integrations on purpose: it’s what makes inline exploration, hovering, and conversational iteration feel seamless for a human in the loop. A Sales Director shouldn’t have to read a stderr traceback. The token cost buys a UX that a headless shell simply doesn’t offer. The catch is that this advantage is about interactive use — which is exactly the part a production pipeline doesn’t have.
And the reliability gap from above deserves an asterisk: most MCP failures in the benchmarks were connection timeouts to remote servers — infrastructure problems, not protocol problems. An MCP gateway (one that filters schemas down to the relevant tools, pools connections, and centralizes auth) closes much of both the cost and reliability gap. So does lazy schema loading (Anthropic’s Tool Search, shipped late 2025), which defers pulling a tool’s full schema until it’s actually needed. The naive 55,000-token connection is a worst case, not a law of nature.
The reframe: it was never “CLI vs MCP”
Here’s the insight that makes the whole debate dissolve. MCP and CLI don’t sit on the same axis. Treating them as competing transports is a category error — like arguing whether to use an enterprise service bus or an API. They operate on different planes of the agent stack, and most well-designed systems use all of them at once.
Execution plane → CLI. Developer-facing agents, local tooling, code operations, infrastructure-as-code — anything with a mature shell interface the base model has seen in training. Accept the modest cold-start of describing the tools; harvest the long tail of pretraining familiarity.
Governance plane → MCP. Customer-facing agents, multi-tenant SaaS, systems of record, regulated workflows — any surface that requires per-request identity, scope enforcement, or audit. Spend the schema tokens here; they’re buying compliance.
Knowledge plane → Skills. Domain procedures, company conventions, playbooks. These aren’t tools at all. They belong in skill files and prompts. Wrapping a procedure in an MCP schema or a CLI is the most common over-engineering mistake — it’s instructions cosplaying as a transport.
Confusing the planes produces the exact pathologies the industry has been cataloguing all year: MCP servers wrapping shell commands that burn tokens for zero governance benefit; CLIs bolted onto SaaS integrations that leak credentials and lose audit trails; skills written as MCP tools, duplicating schema that should have been three lines of markdown.
The decision framework
Stop asking “MCP or CLI?” Ask three questions about each tool integration, in order:
Does this tool have a mature shell interface the base model already knows? (git, gh, kubectl, psql, az, aws, jq…) → Use CLI. The token savings are a side effect; the real win is operating from pretrained knowledge.
Does this action cross a trust boundary needing per-user identity, scope enforcement, or audit? (a customer’s CRM, a tenant’s billing) → Use MCP, ideally behind a gateway. The schema cost is buying something no shell provides.
Is this actually a procedure or convention dressed up as a tool? (a runbook, a house style, a multi-step playbook) → Put it in a skill or prompt. Don’t wrap it in any transport.
Decide this per integration, not per system. Your agent will almost certainly use all three.
What this means for production pipelines
The reason terminal-based agents are winning production pipelines specifically falls right out of the framework. A pipeline is headless and batched — there’s no human enjoying the IDE’s interactive UX, so that entire side of MCP’s value proposition is absent. And pipelines are dominated by deterministic operations: run tests, build, lint, query a database, transform files, hit git and gh. Those are textbook execution-plane work — CLI territory — where the token efficiency compounds across thousands of runs and the 100% reliability matters because nobody’s watching.
But “pipeline = all CLI” is still too simple. A mature pipeline is a hybrid:
Deterministic steps run as CLI / scripts / hooks — the bulk of the work, cheap and reliable.
The few steps that touch a governed external system go through MCP — fetching a customer record, posting to a per-tenant SaaS — ideally via a gateway that filters schemas so you pay for the three tools you use, not the ninety you don’t.
The pipeline’s domain logic lives in skills — what “done” means, your conventions, the order of operations — not baked into either transport.
That’s the real end state. Not “CLI beat MCP,” but a pipeline where each integration sits on its correct plane, the deterministic majority runs as efficient shell commands, and the governed minority pays the schema tax precisely where it buys something.
The bottom line
If the integration is…
Use
Because
A tool with a mature CLI the model knows
CLI
Pretrained fluency + ~200 tokens vs ~35× for MCP
A SaaS system with no shell, behind OAuth
MCP
The only option that handles tenant identity
Acting on behalf of a specific customer
MCP (gateway)
Per-user auth, scope, revocation, audit
A deterministic step in a headless pipeline
CLI
Cheaper, 100% reliable, composable, no UX needed
A procedure, convention, or playbook
Skill
It’s instructions, not a tool — no transport
Multi-tenant infra needing both
Both
CLI execution plane + MCP governance plane
The token economics are what made everyone look, and they’re genuinely the reason CLI agents are taking over production pipelines. But the durable lesson is the one underneath: match each integration to its plane. Do that and you stop paying the MCP tax where it buys nothing, stop leaking credentials where you need governance, and stop wrapping instructions in schemas. The transport stops being a religion and goes back to being an implementation detail — which is exactly where it belongs.
