Hybrid Lexical + Semantic Retrieval: Chunk Size, Fusion Weight and Latency vs Recall

Hypothesis

Speaker-aware chunks with overlap will beat fixed-size chunks on recall for meeting questions, and fusing keyword and semantic rankings will beat either alone, especially on questions that name projects, people or IDs.

Problem

The retrieval layer's current settings (chunk sizes and overlaps, the fusion weighting between vector and keyword rankings, and how many passages reach the model) were chosen by judgement, not measured. Every one of them trades recall against latency and token cost.

Setup

Current configuration, as the baseline: meeting chunks of 800 words with 150 overlapping, other documents 600 with 100; vector and keyword rankings fused with Reciprocal Rank Fusion at k=60, weighted 0.7 to vectors and 0.3 to keywords; the top 15 passages above a minimum similarity go to the model. Variants to test: chunk sizes from small to large, with and without speaker boundaries; vector-only, keyword-only and fused retrieval across a range of weights; and different numbers of passages sent to the model. Metrics: recall@k against labelled relevant passages, answer correctness against expected answers, retrieval latency, end-to-end latency and tokens per answer.

Input

An evaluation set of real questions over a fixed snapshot of transcripts, each labelled with the passages that should be found and the expected answer.

Output

Recall-vs-chunk-size curves, a fusion-weight sweep, and a latency vs recall chart to pick the operating point.

Decision

Status: designed, not run. The baseline values are the pipeline's configured settings, not measured results.

Next Step

Label the evaluation set, run the sweeps offline against a frozen index, then publish the curves and the chosen settings here.

Use Cases

• ▸Tuning retrieval for meeting and document Q&A
• ▸Deciding whether keyword search earns its place next to vectors