· 1 min read
temperature zero is not determinism
I pinned an eval to exact-match at temp 0 and it flaked anyway. Where the wobble actually comes from and what to assert instead.
#evals#sampling#testing
Tagged: evals
3 articles
· 2 min read
Write the eval before the prompt, you animal
Prompt engineering without an eval set is interior decorating in the dark. Twenty labeled examples beat two hundred vibes.
#evals#testing#workflow
· 3 min read
Your RAG problem is a retrieval problem wearing a trench coat
When the answer is wrong, everyone tunes the prompt. The prompt is fine. Nobody looked at what the retriever actually returned.
#rag#retrieval#evals