Engineering notes
Since 2023 we have documented what actually works in AI systems — with numbers, sources and honest limits.
On March 26, 2025, OpenAI announced support for Anthropic's Model Context Protocol across its products, starting with the Agents SDK. We examine what the protocol standardizes, what it deliberately leaves open, why cross-vendor convergence changes integration strategy, and what the simultaneous specification revision means for remote MCP servers.
Manus launched in March 2025 with viral demos and state-of-the-art GAIA claims. We compare the self-reported numbers with the first independent tests, explain what the benchmark does not measure, and argue why constrained agents with approval gates ship to production today while general autonomy does not — yet.
How we build guardrails for LLM systems as of early 2025: layered policies with known miss rates, PII redaction with Presidio, output filtering with Llama Guard 3 and moderation APIs, human approval gates for irreversible actions, and guardrails versioned as code with attack, benign, and regression test suites in CI.
Deep research agents plan, browse and synthesize for minutes instead of answering in one shot. We define the pattern, examine Google's Gemini Deep Research from December 2024, and quantify the weak point: citation quality. Measured baselines show why the verification duty stays with the reader — and what that means for engineering teams.
Classic RAG retrieves once and generates once. Agentic RAG lets the model decompose queries, retrieve iteratively, and critique its own context. We define the pattern, summarize the evidence from IRCoT, Self-RAG, and CRAG, state when a retrieval loop beats one-shot lookup, and show how to keep loop costs bounded.
A retrospective of 2024 through four dated releases: OpenAI Structured Outputs in August, o1-preview in September, Anthropic computer use in October, and the Model Context Protocol in November. We summarize the verified numbers, the limits each capability still has, the lessons production teams drew from them, and our predictions for 2025.
OpenAI's o1 made test-time reasoning the frontier capability of late 2024 — closed, expensive, and API-only. DeepSeek's R1-Lite-Preview and Alibaba's QwQ-32B-Preview signal that open alternatives are weeks away, not years. We examine what reinforcement-learned reasoning actually does, what open weights would do to its cost structure, and why that matters for European data sovereignty.
Anthropic has open-sourced the Model Context Protocol: a JSON-RPC standard that connects AI applications to data sources through servers, clients, tools, resources, and prompts. We explain the architecture, the stdio transport, what the protocol deliberately leaves unsolved, and why one integration surface could replace N-by-M custom connectors.
Microsoft Research's LazyGraphRAG defers all LLM summarization from indexing to query time. Indexing costs drop to 0.1% of full GraphRAG — identical to vector RAG — while answer quality matches or exceeds GraphRAG global search at a fraction of the query cost. We examine the mechanism, the benchmark numbers, and the trade-offs for production systems.
AutoGen, CrewAI, and LangGraph matured through 2024, yet multi-agent systems still pay a steep tax: coordination overhead, compounding errors, and fragmented shared context. We survey the framework landscape as of November 2024, quantify where the overhead comes from, and argue that a single agent with good tools remains the right default for most production tasks.
On October 22, 2024, Anthropic released computer use in public beta: Claude 3.5 Sonnet operates desktops via screenshots, mouse, and keyboard. We explain the agent loop, the OSWorld results (14.9 percent versus roughly 72 percent for humans), known failure modes, safety gating, and which workflows this technique should and should not automate today.
Phi-3 and Gemma 2 show that models with 3 to 9 billion parameters can carry a large share of production traffic at a fraction of frontier cost. We examine which tasks small models cover, what on-premises and edge deployment requires, how routing directs easy traffic to cheap models, and how to measure quality per euro.