When the Equation and the Code Disagree

This week was about making what I wrote match what I built.

RAG saturation

Writing the score margin out formally last week made me check the implementation line by line, and they didn't agree. The paper defines the margin as the top-1 score minus the mean of the remaining k-1 scores, but the code was still including the top-1 score in that mean. I fixed the code to match the equation. That was the third definition of the same metric in three months, and a clear sign I should have pinned it down earlier.

I also removed some placeholder stability functions, marked an outdated config as deprecated, corrected the noise-injection description to say what the code actually does (a small synthetic pool of filler text), and fixed a citation that had the wrong authors, venue and year.

Lost in the Middle

I wrote an extended version of the paper with a related-work section. Related work is where I learned to be careful with AI writing assistants: several references in that first pass needed correcting later, including one with a placeholder arXiv ID. I now verify every citation against the actual source before it goes in.

Reflection

If you can't write a metric down as an equation, you don't really know what you're measuring. And if the equation and the code disagree, the code is what produced your numbers.