When Every Result Is a Copy

This week both projects tested redundancy, and both produced results that were perfect for exactly the wrong reasons.

RAG saturation

I added a CorpusRedundancyShiftGenerator that duplicates documents 1x, 2x and 4x. Rank stability stayed at 1.0, the score gap stayed at 0.0, and score entropy sat at 1.6094, which is ln(5). When every top-5 hit is a copy of the same text, the scores are tied and entropy is maximal by construction.

I initially wrote this up as 'capturing retrieval saturation.' Looking back, it's a better description of redundancy masking the problem: duplicated corpora hide the discriminability loss I'm trying to measure. That framing ended up becoming one of the central ideas of the project.

Lost in the Middle

I implemented a redundancy option that places a copy of the supporting passage at the start of the context. Accuracy went to 1.0 in every position bucket.

That number needs a caveat. My heuristic model always reads position zero, so this 'rescue' can't fail. The copy also displaces one distractor, which quietly changes the difficulty of the task. Both are worth testing properly with a real model.

Reflection

I'm learning to ask 'could this experiment have come out any other way?' before getting excited about a clean result. If the answer is no, it's a sanity check, not evidence.