Run any model, and check its answer before you rely on it.
Flywheel checks a model's answer against the record that decides it, and keeps a receipt that anyone can re-check offline.
By Zain Dana HarperVersion 1.1.0Released 28 September 2026
Check one number against its record
The summary says 42 tests passed. The supplied record says 41. Watch Flywheel check that one structured field before the answer reaches your team. The video uses synthetic inputs and shows the checker's real output.
No sound is needed. Captions and the text version on this page cover the same steps.
The demo in text
Flywheel checks three prepared answers against the record. A held answer stays back from your team until someone resolves it.
42 claimed; 41 in the record
Fails. Held back. The number disagrees with the record.
41 claimed, with its source
Passes. Released. The number matches its cited source.
41 claimed, without a citation
Unverifiable. Held back. The number matches, but no source is named.
Demo details and limits
Does not prove
Released is the answer decision for this one structured field. It is not a software release or general correctness verdict. No model or real CI run was used. You choose the fields and sources, and this check does not establish source trust. The run ID is a lookup key and does not authenticate identity.
Checker output
The checker's own words for the three answers, in order: FAIL / HOLD with 0 of 1 fields confirmed, PASS / RELEASE with 1 of 1 fields confirmed, and UNVERIFIABLE / HOLD with 0 of 1 fields confirmed. The record belongs to run demo-run-001.
The video
The 36-second video sets the checker's result excerpts in type. It is not a screen recording, and it does not show an AI repairing itself. The checks ran on 5 September 2026 on the published Flywheel source of that day. On 25 September 2026 the frames were redrawn in the site's type, with the same checker output, captions and timing.
Need agent summaries checked against test outputs or other records you trust? A scoped paid pilot starts with agreement on one input format, the checks, the expected output and the acceptance criteria.
What Flywheel is
Flywheel is an AI workstation and coding harness, the program that runs a coding model and its tools, and you host it yourself. It runs any frontier or local model behind one interface, and it keeps its records and receipts on your machine. It comes with the Rowan desktop assistant, a coding agent that acts only within the permissions you grant, and a roster of seventeen integrations for research, memory and writing. The release notes state which actions run in the installed app and which need setup; Telos is excluded from this build.
Its check-output command grades an answer against the source that decides it. Each accepted result carries a sealed receipt that lists the inputs, the check that ran and its result.
See how it works, step by step: an animated walk through the capability check on real commands and the recheck of a sealed result.
Any provider
Flywheel runs local weights, a local server or any hosted provider you hold a key for, and it sends every answer through the same check. Its list of providers shows whether each key is present and does not display the key itself.
A record you can keep
Receipts are files you keep. Each one carries fingerprints of its contents, so a change to any part of it shows when someone checks it later.
Answer locally, escalate the hard part
Flywheel answers locally, at near-zero cost, when it can check the answer itself. When its checks fail and the attempt budget runs out, it names a stronger model for you to try.
Works offline
With your own weights, Flywheel runs a coding model of just under 9 GB on your machine with no network connection. Hosted models need your keys and a connection.
How an answer gets accepted
A model you choose proposes an answer. A check defined outside the model accepts or rejects it, and the run records the result.
From a proposed answer to a kept receipt
Only the external check can accept an answer, and a failed check sends the model back for more candidates.
Bar on the left: the step that decides
Ringed: the external check.
The local model proposes answers. It is cheap and replaceable, and asking it for more candidates widens the pool.
No learned model sits on this step. An external check, such as a test run or a source comparison, accepts or rejects the answer.
A pass writes a receipt named by a fingerprint of its own contents, so anyone can re-check it offline, and a change to any byte makes that check fail.
When the attempt budget runs out below the confidence Flywheel needs, it names a stronger tier, a more capable model you can choose to call. It never calls that tier for you.
Passed answers go into a proof cache with their receipts. The next identical request costs nothing, and the cache re-checks the answer before it serves it.
How we know: the loop drawing
Does not prove
The drawing shows the order of the steps. It does not show that the loop raises accuracy. On the shipped benchmark the verified loop shows no measured accuracy uplift over single-shot, and the interval includes zero.
Method
A model cannot reliably pick its own correct answer, so the check that accepts one sits outside the model. In our runs a self-test added nothing. The external check is the part that can change the outcome, because it can reject an answer, and so a pass carries information. The benchmark further down measured no accuracy gain from the loop. Every accepted answer, cached or fresh, carries the same receipt.
Sheet 1 of 2
Describes Flywheel 1.0.1
Walk the loop yourself
Set the attempt budget and the first candidate that passes, then watch where the answer goes. A short recall check follows each part.
From a proposed answer to a kept receipt, a 65-second explainer with captions and a local voice. Its build receipt lets anyone rebuild it and compare the hashes.Transcript
From a proposed answer to a kept receipt.
A local model proposes answers. It is cheap and replaceable, and asking it for more candidates widens the pool.
No learned model sits on this step. An external check, such as a test run or a source comparison, accepts or rejects the answer.
A pass writes a receipt named by a fingerprint of its own contents, so anyone can re-check it offline, and a change to any byte makes that check fail.
When the attempt budget runs out, Flywheel names a stronger tier, a more capable model you can choose to call. It never calls that tier for you.
Passed answers go into a proof cache with their receipts. The next identical request costs nothing, and the cache re-checks the answer before it serves it.
The drawing shows the order of the steps. It does not show that the loop raises accuracy. On the shipped benchmark the interval includes zero.
What the evidence measures
Flywheel has not yet been measured against a named market baseline. The figure and the notes below publish the results that do exist, including the negative result and the incomplete run across five setups.
The offline benchmark record ran six test suites with no model attached. Five scored 100 percent on their own small test sets. The sixth re-reads the saved per-task results of the model comparison drawn below. There, Flywheel Local Coder 14B scored 3.05 points below the base model it was trained from, a gap inside the noise. Read the offline benchmark record.
Same tasks, two models
Flywheel Local Coder 14B passed 136 of 164 tasks: five fewer than its base model, 3.05 points lower, a gap within noise.
95% interval: the range likely to hold the true rate
Pass rate on 164 tasks. The axis starts at 60%.
60%70%80%90%100%
Base model
Qwen 2.5 Coder 14B Instruct, 4-bit
86.0%141 of 164
Flywheel Local Coder 14B
the base model, further trained
82.9%136 of 164
The two rates sit inside each other's 95 percent intervals, and a paired test finds no reliable difference.
This result does not show a statistically significant improvement, general coding superiority, agentic reliability, production readiness, or independent validation. It compares two models in the same harness. It ranks no harness and no market product.
Limits
The base artifact digest is not pinned in this result file. The result covers one 164-task code-completion suite and does not measure agentic tool use. Hardware, runtime version, latency, cost, and resource use are not recorded in this result file.
Method
Both models ran the same 164 code-completion tasks in the same harness: pass@1, greedy decoding, temperature 0. The capped lines are Wilson 95 percent intervals. The paired test is McNemar with continuity correction: 9 tasks passed only on Flywheel, 14 passed only on the base model, and p = 0.404.
Tally
Tasks passed by each model, out of 164.
Model
Passed
Pass rate
95% interval
Base model, Qwen 2.5 Coder 14B Instruct, 4-bit
141 of 164
86.0%
79.8 to 90.5%
Flywheel Local Coder 14B, the base model, further trained
136 of 164
82.9%
76.4 to 87.9%
Check it yourself
Model tags: ollama:qwen2.5-coder:14b-instruct-q4_K_M and ollama:flywheel-local-coder-14b. Source: the tracked result file.
Sheet 2 of 2
Retrieved 28 August 2026
On 4 September 2026 we gave the same seven agent tasks to five setups: three agent harnesses and two local models. Of 35 attempts, 11 produced an answer a checker could score, and 6 passed. Read the full run across the five setups, which gives the reason each of the other 24 attempts went unscored.
How we know: the offline benchmark record and the five-setup run
Does not prove
The offline benchmark record ranks no coding harness. Zero peer harnesses were executed. Every suite runs offline, so none of it scores capability against a frontier model.
The five-setup run of 4 September 2026 measures neither market performance nor quality, and its order is not a ranking. Each setup ran each task once.
Current public evidence does not establish market superiority or feature uniqueness across competing harnesses.
Limits
Five more suites need a live model endpoint and did not run. The record names them, so an absent number reads as unmeasured and never as zero. The 100 percent scores come from test sets of 6 to 26 items, so their 95 percent intervals stay wide.
Offline record
Each suite in the offline record, with its size, result and interval.
Suite
Size
Result
95% interval
Accountability
8 dimensions
100%
none, a mean score
Governed agent
6 scenarios
100%
61 to 100%
Agent recovery
6 scenarios
100%
61 to 100%
Provider swap
10 checks
100%
72 to 100%
Source-mined checks
26 cases
100%
87 to 100%
Flywheel Local Coder 14B against its base model
164 tasks
3.05 points lower
none in the record
Five-setup run
The five setups ran the same seven agent tasks once each, with identical prompts and runtime contexts. All 35 receipts verified. The run names the reason each ungraded attempt failed, because an endpoint that never answered and an answer a checker rejected are different failures.
Each setup on the seven tasks.
Setup
Attempts
Reached a grader
Passed
Codex
7
4
3
Flywheel
7
4
2
Claude Code
7
2
1
Flywheel Local Coder 32B
7
1
0
Flywheel Local Coder 14B
7
0
0
The retired benchmark, kept as history
We retired this comparison on 26 July 2026. Its design let the checked method reuse the single-shot answer, the model's one and only attempt, so its results cannot show whether checking helps.
The model in it is Flywheel Local Coder 14B, just under 9 GB at 4-bit.
Why we retired it
Does not prove
In the retired table, verified inference scored +0.100 above single-shot. The 95 percent interval on that difference is [-0.236, +0.420], which includes zero, and plain best-of-4 sampling tied it. So we do not claim a capability uplift, and the correction below voids the difference as a comparison outright.
The retired table
Four ways of answering the same ten hard tasks, each with its 95 percent interval.
Method
Result
95% interval
Single-shot
8 of 10 (80%)
49.0 to 94.3%
Verified inference
9 of 10 (90%)
59.6 to 98.2%
Best-of-4
9 of 10 (90%)
59.6 to 98.2%
Single plus oracle
8 of 10 (80%)
49.0 to 94.3%
Every method kept 100 percent receipt reproducibility.
Correction
The benchmark ran the baseline as one attempt and the treatment as several, and the treatment's first attempt is the same call as the baseline's only attempt. The treatment therefore cannot score lower. Across all thirteen runs ever made, no task passed the baseline and failed the treatment. That pattern is what one arm containing the other produces, and it carries no evidence of an effect. The quantity measured is verified pass@k, a useful engineering number that says nothing about uplift. Every uplift figure this benchmark produced is void as a comparison for the same reason.
A second finding: artifacts recorded before 26 July 2026 cannot distinguish a model failure from a generation failure that was silently retried. The defect is fixed, and generation failures are now recorded and excluded from the denominator. The older artifacts lack that field, so their per-task failures cannot be re-attributed after the fact.
The model
Flywheel Local Coder 14B is a trained model with a full provenance chain.
What we claim
We claim, and can measure, four things: 100 percent receipt reproducibility, so every accepted answer re-checks; pass parity with the models it routes to; availability on your own schedule, from weights you hold; and local cost. Each claim needs its own evidence, and a limit stated for one claim tells you nothing about the others.
Next measurement
A rebuilt measurement is in progress. It draws the baseline and the treatment from one shared pool of independent attempts, and it adds a compute-matched arm that picks at random and ignores the verifier. It has produced no result yet. If the verifier earns nothing over random selection at the same compute, we will publish that result.
Five-setup run
The five-setup run of 4 September 2026 is described under What the evidence measures, with its results by role.
Check it yourself
The running app serves the receipt at /artifacts/flywheel-local-coder-14b-benchmark-ci.json, and you can re-check it offline. This page updates the number only when the evidence behind it changes.
Install Flywheel
Install the Python engine from PyPI with the command below. For the desktop route, download the unsigned Windows installer from the GitHub release. Test a workflow before you depend on it.
pip install flywheel-verify
flywheel up
The core package needs nothing beyond the Python standard library. Route online with your keys, or run fully offline against local weights. flywheel up starts Flywheel and prints the address of its page, http://127.0.0.1:8799 by default. Open that address in your browser.
The installer is unsigned. Compare your download with the published checksums before running it. A match checks the file's bytes; it does not authenticate the publisher.
The release reports that 15 of its 17 lanes reached the classes expected by its installed-app check. Index fell below its expected class without Git, and Telos is excluded from the build. Some lanes need a model server, project folder or other setup.
The engine command line can inspect, export and delete the agent traces Flywheel holds, with encryption at rest where its supported OS key store is available. The Windows app has no traces screen yet. Desktop chat remains plaintext, and trace deletion does not reach every lane's separate store.
Release evidence and limits
What was checked
The release's installed-app check used a Windows Server runner and a stub model. It called tools through the app's routes and approvals without driving screens. The measured lane table distinguishes actions that ran, actions needing setup and provider paths left untested.
Does not prove
The check does not measure answer quality or establish compatibility with consumer Windows 11, offline use or mobile devices. On the shipped benchmark the verified loop shows no measured accuracy uplift over single-shot; the interval includes zero.
Known issues
The release notes record problems with the engine's lane-install command: its help flag installs lanes, it selects package executables from PATH, and its Windows npm launch fails. The notes also warn about executable selection from the current folder. They say the bundled Windows installer is unaffected. Read those instructions before installing lanes separately.
Earlier release correction
The dated correction to 1.0.4 records where the Windows lane claims exceeded what the app delivered. The earlier demo and checks retain their original versions; they are not evidence for this release.
Compare the downloaded installer with the release's checksum file. The release notes include the measured lane table, setup requirements and remaining limits.
License
Source-available under FSL-1.1-MIT. From a source checkout, python scripts/run_harness_cli.py app --port 8799 starts the same page.
Companion tools
Three tools ship on their own release schedule, each with its own version and limits. None of them changes the Flywheel version.
A skill that names the main reader issue in a prose passage and asks for the smallest useful revision, keeping the author's voice.
Version 0.1.0. Experimental.
Downloads, setup and limits for each tool
Service Desk Incident Environment 0.2.0
Service Desk Incident Environment is a separately installable product with its own state, action log and verifier. Its offline HTML report keeps four things apart: recorded claims, whether the stored fingerprints still match, recomputed task outcomes, and contradictions between recorded authorization, responses and mutations. Its scripted cases use synthetic data and local HTTP services. They do not call a model provider or connect to ServiceNow.
Extract the sample ZIP and open the passing example first, then the example where a denied change still went through. That second example records a mutation despite denied authorization and an unsuccessful response. Its fingerprints were recomputed so they match, and the report still rejects the contradiction.
To create your own report, use Python 3.11 or newer in a virtual environment. Install a compatible Flywheel engine (the environment requires flywheel-verify >=0.6.1,<0.7), then download the Service Desk 0.2.0 wheel into your current directory. The add-on ships on GitHub. It does not update the desktop app, and this page claims no PyPI listing for it.
Open the review page locally. The output must be a new file outside the evidence directory, and review never overwrites an existing file. Review exits 1 when checks fail and 2 for output errors. Inspect the HTML before you share it: the common credential and path redaction does not catch everything, and JSON output keeps local paths.
Does not prove
Matching the wheel hash checks download integrity. Coherent fabricated records can still pass. These checks do not authenticate a producer, measure model alignment, or establish production ServiceNow compatibility.
Limits
The report uses no scripts or external assets and does not execute submitted log actions. Keep the original run artifacts when you investigate a failure. The 0.2.0 source archive is also available.
Check it yourself
The sample reports are positive.html and denied-but-applied.html. The review command writes review-output/review.html. See the 0.2.0 release notes and the download checksums.
Give your agent a claim about Flywheel or Bulletin to check. A release note or a public feedback thread works too. Flywheel Evidence Task helps it find the supporting sources and separate checked facts from reported claims, and it shows what remains unknown. You can use the skill without installing the Flywheel engine.
Use flywheel-evidence-task to check this claim: <claim and source URL>. Show the sources, what they support, what is still unknown, and the next useful action.
Does not prove
The package includes an Agent Skills folder and Codex and Claude Code plugin manifests. Package checks passed. Live host installation has not been tested.
Reader Flow Review is an experimental standalone skill for reviewing or revising a selected prose passage. It names the main reader-flow issue first and asks for the smallest useful revision. When you ask it to polish, it keeps the author's voice, qualifiers, dialogue and factual uncertainty intact.
Extract the ZIP and copy the folder inside it into the skills directory your agent host documents. Invoke it by name when you want a passage critique or a scoped revision. The archive does not install itself or add a marketplace listing.
Use $reader-flow-review to review this passage for reader flow while preserving voice and uncertainty: "The release is useful, but its evidence is developmental. The section should help a reader decide whether to try it without implying the skill verifies facts or certifies publication." Give the dominant reader issue first, then one smallest useful revision.
Does not prove
Developmental evaluation used synthetic same-task checks. It does not prove host installation, automatic invocation, held-out writing quality, factual correctness, third-party marketplace approval, or readiness for any specific writer's style.
Sources
The GitHub release holds five public files. It does not change the Flywheel platform version.
Check it yourself
The folder inside the ZIP is named reader-flow-review. The ZIP is 6,379 bytes. See the download checksums.
Spec at a glance
What you install and what it does.
Part
Detail
Desktop app
Windows x64 installer with the engine bundled, so no Python is required. Version 1.1.0. The installer is unsigned.
Dependencies
None. Python standard library only.
Network
Your choice per call: route online with hosted-provider keys, or run fully offline against local weights.
Entry point
flywheel up
Opens as
One page in your browser, served from your own machine.
Local model
Flywheel Local Coder 14B, about 9 GB at 4-bit. A GPU is optional.
Who accepts an answer
An external check. No learned model sits on the step that accepts.
Record
Each record is named by a fingerprint of its contents, so every accepted answer can be re-checked offline.
Tool lanes
Flywheel starts and connects separate tools as lanes, among them Gather, Index, Forum, Crucible and Articulate. The system map shows how they connect.
Check packs
Finance, medicine and law, for the check-output command.
Proofs
The check-output command can emit a Lean 4 proof.
License
Source-available under FSL-1.1-MIT.
What stays stable
You can swap the model, the route or the external check. The acceptance criteria you recorded stay the same, and so does the run history, where each run is named by a fingerprint of its contents. A reviewer can then compare a later run with an earlier one on the same task and evidence.