28 Nov 2024 AI Engineering 5 min read

LazyGraphRAG: The Economics of Deferred Summarization

Microsoft Research's LazyGraphRAG defers all LLM summarization from indexing to query time. Indexing costs drop to 0.1% of full GraphRAG — identical to vector RAG — while answer quality matches or exceeds GraphRAG global search at a fraction of the query cost. We examine the mechanism, the benchmark numbers, and the trade-offs for production systems.

The price of a graph index

Vector-based RAG retrieves text chunks by embedding similarity. This works for local questions whose answers sit in a few passages. It fails on global questions — "What are the main themes in this corpus?" — because no single chunk contains the answer. GraphRAG, published by Microsoft Research in April 2024, addresses this: an LLM extracts an entity graph from the corpus and pre-summarizes clusters of related entities. Answer quality on global questions improves measurably.

The price is the index. Every chunk passes through an LLM before the first question can be asked. Indexing costs grow linearly with corpus size, and much of the summarized content may never be touched by any query. For large or frequently changing corpora, this up-front bill has been the main argument against graph RAG in production.

Entities and relations are extracted from every document. 1/4

What full GraphRAG computes up front

The GraphRAG pipeline chunks the corpus, uses an LLM to extract entities and relationships, partitions the resulting graph into hierarchical communities with the Leiden algorithm, and generates a report for every community — bottom-up, level by level. At query time, a map-reduce pass over these reports produces the global answer. The method has been open source since July 2, 2024.

The dominant cost driver is the LLM-based extraction and summarization. Microsoft Research stated in July 2024 that it was investigating NLP-based approximations of the knowledge graph to enable evaluation "with minimal upfront indexing costs". LazyGraphRAG, presented in November 2024 by Darren Edge, Ha Trinh, and Jonathan Larson, is the result of that line of work.

The lazy inversion

LazyGraphRAG removes the LLM from indexing entirely. The index consists of embedded text chunks plus a concept graph built from noun-phrase extraction and co-occurrence statistics — classical NLP, no generation. Community detection still runs on this graph. Microsoft quantifies the effect: indexing costs are identical to vector RAG and 0.1% of the costs of full GraphRAG.

Summarization is not eliminated; it is deferred. All LLM work happens at query time, and only for the regions of the corpus a concrete query actually touches. Nothing is summarized that nobody asks about. That is the entire idea — and it is an economic one, not an algorithmic breakthrough in model quality.

Query time as a search process

A query is first expanded by an LLM into subqueries and matched against the concept graph. Candidate chunks are ranked by embedding similarity and community structure. A low-cost LLM then performs sentence-level relevance tests on the ranked chunks, while the search descends the community hierarchy in an iterative-deepening pattern that combines best-first and breadth-first dynamics. Branches that stop yielding relevant chunks are pruned. Relevant claims are extracted, ranked, and synthesized into the answer.

One parameter governs everything: the relevance test budget. It caps the number of LLM relevance tests per query and thereby sets the cost-quality trade-off on a single axis. A budget of 100 with a low-cost model costs roughly as much per query as standard vector RAG; budgets of 500 and 1,500 buy measurably better answers.

Benchmark results in numbers

Microsoft evaluated LazyGraphRAG on 5,590 AP news articles with 100 synthetic queries (50 local, 50 global) against eight conditions: vector RAG with 8K and 64K token contexts, RAPTOR, and GraphRAG local, global (levels C1, C2, C3 with dynamic community selection), and DRIFT search. Metrics were comprehensiveness, diversity, and empowerment, judged by an LLM in head-to-head comparisons.

Two caveats belong next to these numbers. They come from a single dataset with synthetic queries, and the metrics are LLM-as-judge preferences, not verified factual accuracy. The results are strong; they are not yet independently reproduced. Domain corpora behave differently from news text.

Configuration	Relevance test budget	Query cost	Reported result
Z100_Lite	100 (low-cost LLM throughout)	≈ vector RAG (8K context)	Best on local queries; matches GraphRAG global quality at over 700x lower query cost
Z500	500 (low-cost tests; stronger LLM for answers)	4% of GraphRAG global search (C2)	Significantly outperforms all eight competing conditions on local and global queries
Z1500	1,500 (same model split)	Still a fraction of C2	Win rates increase further with budget

What LazyGraphRAG does not solve

LazyGraphRAG produces no durable knowledge artifacts. Full GraphRAG leaves behind a typed entity graph and readable community reports that analysts can browse; these are products in their own right. A noun-phrase concept graph is noisier and not meant for human consumption. Teams that need the graph itself — for exploration, audit, or downstream analytics — still need the expensive index.

The cost moves; it does not vanish. Each query pays for up to hundreds of relevance tests, which adds latency and per-query spend. For a stable corpus with very high query volume, an amortized GraphRAG index can still be the cheaper total. And as of today the implementation is not released — Microsoft has announced it for the open-source GraphRAG library.

Consequences for system design

For our own retrieval systems at Blue IT Systems, the decision rule becomes: lazy by default. One-off analyses, exploratory work, and streaming corpora get the cheap index; precomputed summaries are justified only by high query volume on a stable corpus or by an explicit need for browsable reports. The relevance test budget turns retrieval cost into a per-request engineering decision — a property neither vector RAG nor full GraphRAG offered.

Microsoft also positions LazyGraphRAG as a benchmark instrument: any new RAG approach should state which budget level it beats. We consider that discipline useful. A method that cannot outperform a lazy baseline with a budget of 100 has no cost argument left to make.

Outlook from November 2024

As of November 2024, the code is announced but not shipped. We expect the implementation to land in the open-source GraphRAG library within months, and hybrid designs to follow: a cheap NLP index everywhere, with LLM summarization applied selectively to communities that queries visit often — a cache warmed by demand rather than built on speculation.

If the numbers hold on domain corpora, the sentence "graph RAG is too expensive" stops being true this month. The larger pattern looks durable: spend model capacity at query time, guided by what is actually asked, instead of precomputing answers to questions nobody has posed. We expect more systems to be built this way in 2025.