Flywheel: Run any model, and check its answer before you rely on it.
Flywheel runs AI tasks with the model you choose and checks each answer against the record it came from. A wrong answer is held before your team acts on it.
The baron's trick
The baron said he pulled himself out of the swamp by his own hair. A model that grades its own answer tells the same story, and it tells it with a straight face.
So Flywheel never lets the model accept itself. A check outside the model, a test run or a source comparison, decides. The model proposes; the record disposes.1
- 1. The regress has a name: the Münchhausen trilemma, from Hans Albert, Treatise on Critical Reason, 1968. Every justification ends in a circle, an endless regress or a stop by decree. Back to the text
The record says 41
tests_passed: 42, source: the test record
Fail, held
The value disagrees with the record.
tests_passed: 41, source: the test record
Pass, released
The value agrees with the record.
tests_passed: 41, no source
Unverifiable, held
The value agrees, and the answer does not say where it came from.
How an answer gets accepted
- Model proposesCheap, replaceable, asked for more when needed.
- Outside checkNo learned model here. It accepts or rejects.
- ReceiptNamed by its own fingerprint; re-check it offline.
- Stronger tierNamed when the budget runs out. You decide.
- Proof cacheRe-checked before it is served again.
Same check, two hands
In yours
A green tick you can take apart: the record, the check and the receipt.
In the wrong ones
A green tick that launders a false result.
What keeps it in yours: the check runs outside the model, the checker never certifies itself, and a corpus of known false answers tests the checker on every release.
Install it
The core package needs only the Python standard library. Use hosted models with your keys, or run offline against local weights. Source on GitHub.
pip install flywheel-verify flywheel up