A user asks, “Can I drink seawater after boiling it?” A document about the evaporation of salt water may help answer the question: water evaporates when heated, while salt remains. The document says nothing directly about drinking seawater. Recognizing its relevance requires considering whether boiling removes salt.

Reasoning-intensive retrieval (RIR) concerns retrieval tasks where assessing relevance requires inference. The difficulty is that a system needs some understanding of the knowledge required to solve a problem before it has found the useful documents. In a pipeline with a fixed set of candidates, evidence missed during retrieval remains unavailable to the answer generator, however much reasoning it does afterward.

The RIR Task

The seawater question and the document about evaporation are not paraphrases. One asks whether something is safe to drink, while the other explains a physical process. Connecting them requires identifying the conditions in the question and deciding whether the document’s principle applies. Substituting synonyms or adding terms from the same topic may not establish that connection.

Similar relationships arise in code and mathematical retrieval. Connecting an error report to function documentation may require understanding the operation the code was meant to perform. Two problems with different wording may use the same algorithm or theorem. Relevance here depends on how the knowledge can be used. A shared topic provides a clue, but does not establish applicability.

The upper part of the figure spells out this connection: start with whether seawater is drinkable, consider whether dissolved salt remains after boiling, then look for the principle governing the heating of salt water. Even finding a single document can require this inference. RIR is therefore not defined by the number of retrieval rounds, the length of a chain of thought, or whether the model writes out its reasoning.

This also distinguishes the retrieval task from reasoning over evidence to produce a RAG answer. Finding evidence and solving a problem after reading it are abilities that can be assessed separately. BRIGHT, the benchmark used in most of the experiments below, provides queries, a document corpus, and relevance labels. Systems return ranked documents without generating final answers. Relevant materials include reference pages from real question-answering discussions, syntax documentation needed for code, example problems sharing an algorithm, and applicable theorems.

Candidate Retrieval and Relevance Assessment

An LLM given both a question and a document can analyze whether a principle applies or whether code performs the required operation. This supports a fuller relevance assessment than lexical matching. Reasoning rerankers take this approach, but analyzing every document is too expensive for a large corpus.

Systems commonly retrieve a small set of candidates using vector search, then pass them to a reranker for closer reading. Encoding queries and documents separately makes searching the full corpus efficient, but their complex applicability relationships must be reflected in vector similarity. If this stage misses a useful document, the reranker never gets to assess it.

The system has to select candidates before reading them closely, yet close reading may change its understanding of the knowledge it needs. Improving the reranker’s reasoning alone does not expand the candidate set it can see. Letting those judgments affect candidate selection requires changes to encoding, the text being matched, or the search process.

The works below can be grouped by what they change:

These changes can be combined. Query expansion still needs a retriever that can use the expanded text, and search still depends on the model’s judgments about material it has read. They do not form a mandatory sequence of stages.

Synthetic Data and Representation Learning

ReasonIR starts with training data. An LLM constructs pairs of questions and applicable documents, which are then used to train a retrieval encoder. The LLM generates the data, while the trained retriever still produces vectors.

For example, given a document about circadian rhythms, generating “What is a circadian rhythm?” would make the source easy to find through word matching. ReasonIR instead generates a question asking what could explain night-shift workers having trouble sleeping during the day and feeling tired at work.

This example appears at the bottom of the figure. The circadian-rhythm document explains the observations in the query and serves as the positive. The negative at the lower right also discusses night shifts, body temperature, and daytime sleep, but lacks the required explanation. Training encourages the encoder to assign higher similarity to the circadian-rhythm document even when the negative repeats more words from the query.

This data category is called HQ (hard query). Starting from a real document, the LLM considers situations where its knowledge could be useful and writes a question that can be understood without seeing the source. It then generates a hard negative separately: a document that looks relevant but does not help solve the problem. The positive remains a real document. The query and negative are synthetic.

The other category, VL (varied-length), appears in the middle of the figure. The LLM generates both the query and positive document, then generates the negative separately. This data covers longer queries containing more information. Complex questions often include background and conditions, and LLM rewriting can make them longer still. VL exposes the encoder to such inputs during training.

Both categories are mixed with existing public retrieval data for contrastive training, increasing query similarity to positives and decreasing similarity to negatives. After training, ReasonIR remains a bi-encoder: queries and documents are encoded separately for vector retrieval. It does not have to generate reasoning text for every search.

Query rewriting is a separate, optional step. The table below compares training data combinations using either original queries or GPT-4 rewrites, with the same initial model and number of training steps. The metric is BRIGHT nDCG@10, where higher scores indicate better ranking quality among the top ten results.

Using both categories gives the highest scores in both columns. HQ alone helps with original queries but adds no improvement for rewritten queries. VL changes sample content as well as length, so the gains cannot all be attributed to training on longer queries.

ReasonIR changes the relevance relationships learned by vector scores without requiring online reasoning over each document. The further gains from query rewriting also show that learning these relationships is compatible with supplying more query information.

Index Enrichment and Query Expansion

Implicit connections can also be written out before matching, without retraining the retrieval encoder. This can proceed from either end: start with a document and describe the problems it can help solve, or start with the current problem and infer what knowledge to seek. SPIKE and DIVER-QExpand work on these two sides, respectively.

Scenario Indexing

SPIKE adds usage scenarios to document entries in the index while keeping the retrieval encoder unchanged. A piece of code can then be found through the needs it satisfies as well as through its original text.

The lower part of the figure shows a pandas example. A user wants to combine multiple DataFrames, or tabular data objects, but encounters an error involving a list input. The document on the right contains pd.concat code and its output. The scenarios in the middle describe its uses: concatenating several DataFrames at once and aligning data by column names.

A user unfamiliar with pd.concat can match these descriptions through the need to combine tables and reach the original document. Generating scenarios therefore requires considering what problems would call for this code. Summarizing what the code does may not produce that description.

The model first identifies the document’s main topic M and key aspects K. From these, it proposes potential information needs I, then writes explanations E of how the document satisfies those needs. Each explanation is combined with the main topic as M+E, encoded separately, and stored in the index. The original document vector is retained, so one document can have multiple scenario vectors.

At retrieval time, the system computes the query’s similarity to the original document and to each scenario. It takes the highest scenario score and combines it with the original document score using a weighted sum. A single matching use can therefore improve the document’s rank.

SPIKE also uses synthetic data, but trains a different component from ReasonIR. It first uses GPT-4o to write scenarios for a set of documents, then trains a smaller Llama-3.2-3B-Instruct scenario generator on those samples. This generator produces scenarios for the corpus. The retrieval encoder needs no retraining, and scenarios need not be generated for each incoming query.

To control scoring costs, the experiments first retrieve the top 1,000 documents using their original vectors, then compute scenario scores only for those documents. Scenario enrichment can reorder that candidate set, but cannot recover documents outside it.

Query Expansion with Document Feedback

SPIKE prepares scenarios before knowing what users will ask. Query expansion can instead add concepts for the current question or draft an answer to use as search input. A model working from the question alone can still head in the wrong direction. DIVER-QExpand, the query expansion module in DIVER, reads retrieved documents and uses their information to revise the expansion. The experiments use two rounds of expansion with document feedback, followed by a final retrieval:

1. Retrieve with the original query and take the top five documents. The LLM identifies helpful information and combines it with its own knowledge to write an answer-style expansion.
2. Retrieve again using the expansion, excluding documents already retrieved. The LLM reads the original question, new material, and previous expansion, then revises the text.
3. Perform the final retrieval using the original query plus the last expansion, without concatenating all intermediate versions.

The expansion’s “answer” is a draft for retrieval, subject to revision as further material arrives. The prompt also warns that most retrieved documents may be irrelevant and should not simply be copied.

DIVER compares this module with ThinkQE, another query expansion method that uses feedback. In these comparisons, ThinkQE performs slightly better with BM25, while DIVER-QExpand performs better with ReasonIR. The expanded text is still input to a retriever. Which information is added and whether the retriever can use it have to be considered together. A complete explanation does not automatically make an effective search request.

Reasoning-Guided Iterative Search

DIVER-QExpand revises the search request after reading documents, leaving the retriever to select the next candidates from that request. Another approach lets the model directly influence where candidates come from: after judging a document or topic useful, it explores nearby documents or the corresponding directory.

Reranking can then alternate with search rather than wait until all candidates have been collected. The system reads some material, assesses its usefulness, and decides where to spend further computation. RGS and LATTICE both use this interaction, with different search entry points.

Document Proximity Graph Search

Reranker-Guided Search (RGS) uses reranking results to choose where to search next, reaching other candidates through highly ranked documents.

RGS first builds a proximity graph from document vectors. Each node is a document, with edges connecting documents that are close in the embedding space. For an incoming query, vector retrieval supplies initial candidates for an LLM to rerank. The system then visits the neighbors of the highest-ranked document that has not yet been expanded, adds the new documents, and updates the ranking.

The question in the figure uses guitar amplitude as its setting, but requires evaluating a limit. Initial candidate D1 introduces sine functions, while C1 discusses simple harmonic motion and amplitude. Both resemble the problem statement. The needed document D3 covers L’Hôpital’s rule, but is absent from the candidate set and cannot be found by reranking alone.

On the right, the reranker judges that C1 lacks the required mathematical relationship. It prioritizes D1, whose neighbors lead to D2. D2 has little word overlap with the query but contains a useful solution method. Expanding its neighbors then leads to D3.

The search selects this path as it proceeds. Edges indicate vector proximity, without guaranteeing that a neighbor helps solve the problem, so each new batch requires another relevance assessment. Vectors now serve a different role: even when the query and target document are not close enough, document-to-document proximity may provide a route that the reranker can guide.

RGS repeatedly expands neighbors, adds candidates, and reranks them until it reaches the limit on distinct documents visited, then returns the best current results. The experiments use BGE-Large to encode queries and documents and Gemini-2.0-Flash for reranking. BRIGHT nDCG@10 scores are shown below against reranking a fixed candidate set.

The limit counts distinct documents, and a document may be reranked repeatedly. This is not a comparison at equal token usage or latency. Average scores improve, but the LeetCode subset regresses. On the Biology subset, some positive documents are visited but still excluded from the final results.

Hierarchical Index Search

LATTICE organizes the corpus as a directory tree. Search selects potentially relevant document groups before reaching individual documents. The root represents the full corpus, internal nodes represent document groups, and leaves are the original documents. Unlike RGS, the model can read a group’s description and decide whether to enter that branch without starting from a particular document’s neighbors.

The directory is built before queries arrive. The model generates summaries of each document at several levels of detail, then groups documents using those summaries. Brief topics are used near the root, with more detailed summaries used to subdivide smaller groups. The LLM performing the grouping can thus work with descriptions at a suitable level of detail instead of reading all source documents at once. Large corpora can also be clustered by embeddings into broad groups before LLM processing.

From left to right, the figure shows three search rounds. The first compares directories directly below the root, the second expands two highly scored branches, and the third reaches some documents. Dark green nodes are candidates prioritized for expansion in the current round, and yellow circles mark target documents.

The system keeps a queue of branches awaiting expansion. Each round selects branches with high priority and asks the LLM to score their children’s descriptions. New document groups return to the queue alongside other branches, so search does not have to follow one path all the way down. Documents reached enter the result set. After a preset number of rounds, the system returns the highest-ranked documents found.

Scores in the queue come from different batches and may not be directly comparable. The same material may score highly alongside weak candidates and lower alongside stronger ones, affecting which branch is selected next.

LATTICE includes recurring reference nodes across batches. The dashed circles in the figure mark these references, and the arrows across branches show previously scored nodes included in another comparison. Using the shared references and previous scores, the system estimates how much each batch’s scores are biased upward or downward and calibrates the node scores.

A node’s search priority is determined by its path score, a weighted combination of its calibrated score and its parent’s path score. Scores assigned to higher-level directories therefore also influence the order in which documents below them are explored.

Directory descriptions can become stale. Some LATTICE evaluation tasks exclude documents for each query, although those documents were present when the directory was built. A summary may describe material that is no longer accessible, directing the model toward a branch without usable evidence.

On four BRIGHT subsets, LATTICE is also compared with a strong retriever followed by sliding-window reranking. Sliding-window reranking reads overlapping groups of candidates to progressively reorder the full list. Both methods use the same LLM and scoring prompt. In these comparisons, retrieval followed by reranking performs better at low token budgets. Directory search begins to show the benefit of further exploration as the budget increases.

Summary

ReasonIR trains vectors to reflect applicability relationships. SPIKE and DIVER-QExpand add information needed for matching to the text. RGS retains vector neighbors and guides traversal with more detailed relevance assessments. Similarity remains useful, but need not handle every step from understanding a problem to finding its evidence.

The RGS comparison illustrates this division of work. With the encoder and reranker unchanged, changing how candidates are acquired improves the average score in these experiments. Applying the same assessment capability to different candidates can produce different retrieval results. Positive documents that are visited but not selected on Biology also show that relevance assessment can remain a constraint after search coverage improves.

This division suggests studying how to allocate computation according to what a retrieval process still needs. A query with sufficient evidence may not need more search. One with only material on the same topic may benefit from further rewriting or exploration. The difficulty is judging whether the evidence is sufficient and what further search might add. RGS’s distinct-document budget and LATTICE’s fixed number of rounds do not directly answer that question.

References

The seawater example and first figure come from A Survey of Reasoning-Intensive Retrieval: Progress and Challenges. Method descriptions and experiments were checked against the original papers below. Links point to the versions used in this article.

Task and evaluation: BRIGHT .

Training and indexing: ReasonIR , SPIKE .

Query expansion and search: DIVER , Reranker-Guided Search , LATTICE .

The remaining figures come, in order, from ReasonIR’s training data diagram, SPIKE’s scenario example, RGS’s theorem search example, and LATTICE’s online tree search diagram.