# A benchmark anyone can repeat

Our [results](results.md) come from internal tests whose data is not published. So that nobody has to take them
on trust, there is a public benchmark you can run yourself, with and without Recipe360, on data that is public
and invented:

**https://github.com/sebastianarangouribe-coder/recipe360-mcp/tree/main/benchmark**

## What is in it

- **The task**: a monthly review of a card portfolio, written as a person would brief it. Nine rules over 400
  cards, 28 numbers to report. It is the process this version's example recipe (`card-review`) runs.
- **The data**: `cards.csv`, the example's own file. Every value is invented, and a script writes it again from
  a seed, or writes a fresh file nobody has seen.
- **An independent reference**: the right answers worked out in under 150 lines of plain Python, from the rules
  as written, using nothing from Recipe360.
- **Exact scoring**: an attempt is correct only when every field is right.

## With Recipe360

```bash
python with_recipe.py --runs 10
```

Every run prints `EXACT` and the same delivery fingerprint, which starts with `4989f850` for the published file
and cut-off 2026-10-31. The example runs without a licence.

## Without a recipe

The benchmark's `without_recipe.md` says how to give the same task to the same agent program and model on its
own: the same request, a new conversation each time, each answer scored with `check.py`.

## What to compare

How many runs are exact in each arm, whether the result is the same every time, and the time and tokens each
run takes. Your numbers will depend on the model, the agent program and the task; this benchmark is a way to
check the question yourself, not a repeat of our internal tests.