Evaluating answers without fooling myself
It's easy to feel that answers got better after a change. A small, boring test set tells me whether they actually did.
Hi, I'm Daan. I'm a backend developer, and in my spare time I'm building aistwin: a small, local assistant that answers questions from my own notes — and says “I don't know” when they don't contain the answer.
This is where I write down what I learn along the way. More about the project.
It's easy to feel that answers got better after a change. A small, boring test set tells me whether they actually did.