---
title: "About these notes"
url: https://aimodelscompare.com/about/
description: "What the working notes on model comparison cover, how they are written and what they deliberately leave out."
---

# About these notes

About

What this river covers, how the notes are written, and what they leave out on purpose.

These are working notes about how language models get compared: what a comparison has to fix before it starts, how published numbers are produced, and which measurements survive a change of model version. The river runs newest first, and every note is dated because the subject moves.

## How the notes are written

Each note describes method rather than standing. The aim is that a note stays usable after the particular systems in view have been replaced, which means the writing concentrates on procedure - what to hold constant, what to record, what a number can and cannot support - and stays away from the question of which system is currently ahead.

## What is deliberately absent

There are no scores, rankings or verdicts on named systems here, and no figures presented as measurements. A published number without the harness that produced it cannot be checked by a reader, and an unverifiable number is worse than none: it carries the authority of measurement without the substance. Where a note needs an example it uses an unnamed candidate, because the point being made is about the procedure and not about the product.

## Reading order

The notes are independent, but three of them carry most of the weight and are worth reading first: the note on fixing a task before choosing a model, the note on building a small private evaluation set, and the note on what a published benchmark score leaves out. The glossary collects the vocabulary the rest of the notes assume.

[Back to the river](https://aimodelscompare.com/) · [Glossary](https://aimodelscompare.com/glossary/)