Bring supported sources in. Record how each item arrived.

It brings information in, from anywhere, and records how.

Network access, third-party tools, and credentials stay isolated behind source adapters. After intake, a run applies its scope filter, folds item receipts into one witnessed digest, and hands that digest to downstream indexing, refinement, or claim evaluation.

Every kind of intake is one small adapter behind a single shape: one string in, a list of receipted items out. The adapter can use the network, a tool, a credential, a headless browser, whatever the source demands; nothing else in gather imports any of that. Awkward access is an adapter problem, not a system problem, and the core stays pure standard library because of it.

Reach anywhere. Then say exactly how.

gather ships a source adapter for each kind of intake, all behind the one shape. The cheap, local sources are pure standard library; the harder ones shell out to their own external tool, never a Python dependency. Each adapter records the method it used, so the accountability is in place before the harder reach is trusted. The method on every item is the load-bearing field: it keeps a direct read and a derived inference from ever being read as the same thing.

The honesty matters most exactly where the reach is widest. The web adapter runs no JavaScript, so it is honest that a client-rendered page gives back only its shell. The browser adapter runs a real headless browser, and it is the most exposed adapter in the system: its host guard covers only the first navigation, and the rendered page then follows its own redirects and sub-requests unguarded. So do not point it at untrusted URLs in an environment where internal services are reachable. The browser-extract method is honest that JavaScript ran, but it cannot make the fetch safe. The threat model says so plainly, in the open, rather than hide it.

The receipt is the differentiator.

A provenance receipt rides on every item: the source, the reference, the method, the time, and a sha256 of the item’s own content. Re-hash the content and you can confirm it is what was obtained, unaltered. The method keeps the harder distinction on the record, yt-dlp, browser-extract, ocr, transcribe, synthesized, so a quote pulled straight from a source and an inference pieced together from fragments are both valid items, but they are never equally direct, and the receipt never lets one be read as the other.

A fact pieced together is not a fact pulled straight from the source. The receipt keeps them apart.

A derived item, one assembled or inferred from other items rather than fetched, is the sharp case, and the receipt is built for it. Its sha256 fingerprints the inference itself, not its sources, because it is a new statement that can only witness itself; a derived_from field records the content hash of each input, a re-checkable pointer back to the exact source content. The honesty is mechanical where it can be: the synthesized label is reachable only through one seam, and the bare builder refuses to stamp it and defaults to compiled, so a plain call can never forge a synthesis. With no model wired in, the default compiles the inputs verbatim and invents nothing. The digest seal folds in the method and the inputs alongside the hash, so relabelling an inference as a direct fetch, or quietly rewriting what it was built from, breaks the seal exactly as altering the content does.

Standard library at the core. The impurity is fenced at the edge.

The core runs on Python’s standard library with no third-party runtime dependency. An adapter may pull in whatever its source demands, a browser binary, an OCR engine, a transcription CLI, but only behind the Source shape and only at the impure edge. Clocks are injected everywhere a time is recorded, so given the same fetched items a run is deterministic and replayable, and verification stays trustworthy. The four stages below are the spine of a run, each one re-checkable from disk.

A full gather run folds fetch, scope, optional synthesis, digest, and store into one re-checkable RunRecord with its own seal over what it did. Recall queries a stored corpus and re-verifies every body it returns, so a missing or corrupt item is reported, never handed back as if intact. The scope filter, the synthesizer, the store, and the external provenance verdict are all seams that default to a Null, so gather stands alone and a peer system plugs in without the core ever importing it.

Current state. How to run it.

Version 1.9.1, published as gather-engine, installs the gather command and the gather package. New corpus writes preserve exact UTF-8 source text while selection uses a readable LF-normalized view, so mixed-newline text can be quoted with its source identity intact. Selections stay pinned to the witnessed corpus digest used by --expect-digest. The core remains pure standard library; optional adapters call an external tool (yt-dlp, pdftotext, headless chromium, tesseract, or a Whisper-style CLI) only when that route is selected.

Version 1.8.1 added protection for retained corpus reads against directory replacement on unsupported Linux mounts. On WSL, keep corpora on native Linux storage or use native Windows Python: Windows-drive 9p/v9fs mounts are refused with UNSAFE_PATH. Python hosts can pass a retained same-process corpus descriptor; CLI and MCP continue to accept directory strings.

Install 1.9.1 or later. Every earlier release is inside the range of at least one of the four Gather advisories listed on the Security page. A tool found only inside the working folder now needs its absolute path in its GATHER_<TOOL> variable, and an MCP host that relied on the old gather.run or gather.pilot behavior adds a grant to its launch configuration.

Create sample.txt with this exact two-line body, then store it and select only the useful slice.

sample.txt body
Offline quickstart: Gather keeps acquired context separate from conclusions.
Decision fact: the release can select useful text from a stored corpus.

After the inspect command, replace ROW_REF and CORPUS_DIGEST with the values from the inspect JSON. 0:76 selects the first line of this fixture only.

$ pip install gather-engine==1.9.1
$ gather docs sample.txt --store corpus --json
$ gather corpus context corpus --json --excerpt-chars 90
$ gather corpus context corpus --json --select ROW_REF:0:76 --expect-digest CORPUS_DIGEST

Sample selected text: Offline quickstart: Gather keeps acquired context separate from conclusions.