Retrieval Augmented Generation Compositionality

The paper evaluates Retrieval Augmented Generation techniques in combination rather than in isolation. Across 450 configurations on eight benchmarks with corpora from 5K to 200K documents, it reports that three retrieval techniques predicted +47% NDCG@10 together but delivered +17%. Faithfulness gains combined nearly additively, and the best-performing design on small corpora trailed by 13.7% NDCG@10 at 200K. The authors note their pre-declared hypothesis on retrieval gain shrinkage was not met, and results use one English test bed and proprietary generators. They release the open-source LASAGNA framework.

Why it matters

Gains measured for RAG components in isolation may not predict performance of full pipelines, which bears on how new techniques should be evaluated before adoption.

Read with care

Pre-declared hypothesis H1 was not met, the interaction effect is described as small, and results rely on one English benchmark setup with proprietary generators.