07 Jun 2026 AI Engineering 6 min read

GraphRAG — When Knowledge Graphs Pay Off

GraphRAG turns a document corpus into an LLM-extracted knowledge graph with Leiden communities and pre-written summaries, so corpus-wide questions become answerable. We check the verified numbers — 70–80% comprehensiveness win rates, LazyGraphRAG's 0.1% indexing cost and 700x cheaper global queries — and name the cases where plain hybrid RAG stays cheaper, faster, and more accurate. With five decision rules for production use.

Why Vector Search Fails Global Questions

Vector RAG retrieves the top-k text chunks whose embeddings are most similar to the query and hands them to a language model. This works when the answer sits inside a few chunks — a local question such as "who signed contract X?". It fails structurally on global questions such as "what are the main themes across these 10,000 documents?". Answering those requires considering the entire corpus. That is a query-focused summarization task, not a retrieval task, and no reranker or larger top-k fixes it.

Microsoft Research formalized this in "From Local to Global" (Edge et al., arXiv:2404.16130, first published April 2024). Their diagnosis: naive top-k retrieval matches chunks that look like the question rather than the texts needed to answer it, so it produces plausible but misleading summaries. GraphRAG is their response — build structure over the corpus once, so that global questions become answerable.

Entities and relations are extracted from every document. 1/4

What Microsoft GraphRAG Actually Builds

The open-source graphrag pipeline (on GitHub since July 2, 2024; version 3.1.1 as of July 2026) works in stages. It splits documents into TextUnits, then uses an LLM to extract entities, relationships, and claims from every unit, merging the results into a corpus-wide knowledge graph whose nodes and edges carry LLM-written descriptions.

It then partitions this graph with Leiden community detection (Traag et al. 2019), a hierarchical clustering algorithm that finds groups of densely connected entities. For each community, at every level of the hierarchy, an LLM writes a summary bottom-up: leaf summaries feed into parent summaries. The result is a layered, pre-computed description of what the corpus contains — before anyone has asked a question.

Global and Local Query Modes

At query time, graphrag offers four modes. Global search answers corpus-wide questions via map-reduce: community summaries are grouped into context windows, each group produces a partial answer (map), and the partial answers are combined into a final answer (reduce). Local search starts from the entities matched in the question and fans out to their neighbors, text units, and community reports — suited to questions about specific entities.

DRIFT search combines local fan-out with community context. Basic search is plain top-k vector retrieval. The library ships it because its authors acknowledge that many queries are best answered by baseline RAG — an honest scoping statement worth taking seriously when you evaluate the framework.

What the Evaluations Measure

The original evaluation used two corpora of roughly one million tokens each (podcast transcripts and news articles) and an LLM judge comparing answers pairwise on comprehensiveness, diversity, empowerment, and directness. GraphRAG's global search beat naive vector RAG with roughly 70–80% win rates on comprehensiveness and diversity. Using root-level community summaries, it stayed competitive with full map-reduce summarization over the source texts at roughly 2–3% of the per-query token cost.

Two caveats. The win rates come from an LLM judge, not from ground-truth accuracy — and naive RAG won on directness. An independent systematic evaluation (arXiv:2502.11371) found the mirror image on classic QA benchmarks: RAG was better on single-hop, detail-oriented questions, while GraphRAG's global search lost detail and hallucinated on unanswerable queries instead of replying "insufficient information".

The Honest Cost Accounting

Full GraphRAG indexing pushes every chunk through LLM extraction prompts, and extraction dominates the token bill. Microsoft's own cost walkthrough makes the gap concrete: embedding the complete text of The Wizard of Oz cost $0.0056, while graph construction on GPT-4o mini runs about $0.011 per 1,000 words — roughly 80 times the embedding cost — and multiples of that on GPT-4-Turbo, the model of the original paper. Global queries took 20–24 seconds end to end.

The graph is also a derived artifact, not a store. When documents change, extraction must rerun or be patched incrementally, and pre-computed community summaries go stale. Extraction is lossy too: the systematic evaluation above measured that only 65.8% of answer entities existed in the constructed graph for HotpotQA. Every missing entity is a question the graph cannot answer, however elegant the traversal.

LazyGraphRAG Shifts the Economics

LazyGraphRAG (Microsoft Research, November 25, 2024) inverts the design: it defers almost all LLM work to query time. Indexing uses lightweight NLP — noun-phrase extraction and co-occurrence analysis instead of LLM calls — so data indexing costs are identical to vector RAG and 0.1% of the costs of full GraphRAG.

At query time it combines best-first and breadth-first search under a single parameter, the relevance test budget. Microsoft reports answer quality comparable to GraphRAG global search at more than 700 times lower query cost; at 4% of the global-search query cost, LazyGraphRAG significantly outperformed all eight tested conditions on both local and global queries. In the BenchmarkQED follow-up (June 5, 2025), it won all 96 comparisons against same-model baselines — including vector RAG with a one-million-token context window.

Approach	Index cost	Query cost	Strong at	Weak at
Hybrid vector RAG	Embeddings only (minimal)	Low	Single-hop lookups and details	Corpus-wide synthesis
GraphRAG local search	Full LLM graph build	Moderate	Entity-centric multi-hop questions	Index cost and staleness
GraphRAG global search	Full LLM graph build plus summaries	High (map-reduce over communities)	Themes and global synthesis	Details; unanswerable queries
LazyGraphRAG	~Vector RAG (0.1% of full GraphRAG)	>700x below global search at equal quality	Local and global queries	Query-time LLM latency

When Hybrid RAG Remains Better

None of this repeals the base case. A 2025 analysis (arXiv:2506.05690) collates findings from prior studies: GraphRAG scored 13.4% lower accuracy than vanilla RAG on Natural Questions, added roughly 2.3x average latency, and gained only 4.5% on HotpotQA multi-hop questions. If your query log is dominated by single-hop lookups — "what is the notice period in contract X?" — hybrid retrieval (BM25 plus vectors plus a reranker) is cheaper, faster, and often more accurate.

The methods are complementary, not competing. The systematic evaluation showed that routing each query to RAG or GraphRAG via a classifier (Selection), or feeding both contexts to the generator (Integration), improved MultiHop-RAG accuracy by up to 6.4 points over the best single method. A graph earns its place for a segment of queries; it rarely replaces the vector index.

Five Decision Rules

1. Start with hybrid RAG and instrument it. Log queries and classify them: local lookup, multi-hop reasoning, corpus-wide synthesis. Decide on data, not on demos. 2. Stay with hybrid RAG if fewer than roughly 10% of queries need cross-document reasoning or summarization — the graph's operating costs will not amortize on the remainder.

3. Trial LazyGraphRAG before full GraphRAG when global questions matter. Its indexing cost matches vector RAG, so the experiment is nearly free; raise the relevance test budget until answer quality plateaus. 4. Reserve full GraphRAG with pre-computed community summaries for read-heavy, stable corpora with many repeated global queries — that is where up-front summarization amortizes.

5. Treat a production graph as a data product. It needs an owner, a refresh pipeline, and extraction-quality metrics. If you cannot staff that, do not build it — a stale graph answers from a reality that no longer exists, and it does so confidently.

Where Graph Retrieval Is Heading

The indexing-cost objection is dissolving; the bottleneck moves to extraction quality and governance. Graph and vector retrieval are converging into a single spectrum, tunable per query through a budget. Agentic search adds implicit structure at query time: a 2026 benchmark study (arXiv:2604.09666) shows it narrows the multi-hop gap between dense RAG and GraphRAG but does not close it.

We expect retrieval stacks that decide per query how much structure to build — and knowledge graphs increasingly serving as shared, inspectable memory for multi-agent systems, as a general architectural pattern. The decision rules above survive that shift: structure where the questions demand it, and nowhere else.