The format, on one page.

Five members, one of them optional.

handbook.tome
├── manifest.json          # identity, embedder, prompt, licenses
├── checksums.sha256       # SHA-256 of every other member
├── signature.json         # optional Ed25519 signature
├── models/
│   └── embed.gguf         # absent when the embedder is a reference
└── knowledge/
    └── vectors.db         # SQLite + sqlite-vec

A .tbox uses the same tree and packs a chat model into the same file, either as models/chat.gguf or as a SHA-256 reference. Both types share every rule below. The receiver checks every hash before they query.

Plain ZIP, no surprises.

• Every member uses compression method 0 (store). Nothing is encrypted.
• Names are UTF-8, relative, and use /. A name that is absolute or has a .. segment is rejected.
• Writers use ZIP64 only when the archive needs it. Readers accept both.
• The reference writer sorts members by name and gives them a fixed timestamp, 1980-01-01.

What the file says about itself.

manifest.json is UTF-8 JSON. Unknown fields are ignored. A missing required field is an error.

The SHA-256 of the archive identifies its bytes and is not written in the manifest. Subscribers follow id and revision.

One SQLite file. Text in the clear.

knowledge/vectors.db is written with journal_mode=DELETE and checkpointed, so no -wal or -shm file exists. Readers open it read-only.

CREATE TABLE tome_meta (key TEXT PRIMARY KEY, value TEXT NOT NULL);

CREATE TABLE chunks (
  id     INTEGER PRIMARY KEY,   -- equals vec_chunks.rowid
  text   TEXT NOT NULL,
  source TEXT NOT NULL,         -- path relative to the build input
  start  INTEGER NOT NULL,      -- character offsets in that file
  "end"  INTEGER NOT NULL,
  sha256 TEXT NOT NULL          -- hash of text
);

CREATE VIRTUAL TABLE vec_chunks USING vec0(
  embedding float[384] distance_metric=cosine   -- float[dimensions]
);

tome_meta repeats package_id, revision, dimensions, metric, embedding_id, and embedding_sha256 from the manifest, and records schema_version and the vec_version used at build time. Retrieval is one query:

SELECT c.id, c.text, c.source, c.start, c."end", v.distance
FROM vec_chunks AS v JOIN chunks AS c ON c.id = v.rowid
WHERE v.embedding MATCH ? AND k = ?
ORDER BY v.distance;

Checksums always. A signature when you sign.

checksums.sha256 lists every member except itself and signature.json, in GNU sha256sum form, sorted by path. A reader verifies every line before it reads the database or loads a model.

signature.json is an Ed25519 signature (RFC 8032) over the exact bytes of checksums.sha256:

{
  "algorithm": "ed25519",
  "signer": "com.example",
  "public_key": "<base64, 32 bytes>",
  "payload": "checksums.sha256",
  "signature": "<base64, 64 bytes>"
}

A valid signature proves the bytes are unchanged since signing. It proves who signed only when the receiver already trusts that key. The reference CLI keeps trusted keys in ~/.config/tomefile/trusted/<signer>; --require-signature refuses unsigned files and any other key.

Weights travel once.

A model is either a member or a reference, {"sha256": "…", "filename": "…"}. A reference resolves from a content-addressed cache, ~/.cache/tomefile/models/<sha256>.gguf. A file that references its embedder weighs about as much as its database, and several files share one copy of the weights.

Whatever form it takes, the embedder must hash to embedding.sha256. A different model with the same dimensionality would return plausible but wrong neighbours. Chat weights are embedded only when the publisher marks them redistributable.

Seven rules for a reader.

Keywords follow RFC 2119. Implement these and your program opens every .tome.

1. MUST reject a member that is compressed, has an absolute name, or has a .. path segment.
2. MUST verify every hash in checksums.sha256, and that the listed names equal the archive members other than checksums.sha256 and signature.json, before it reads the database or loads a model.
3. MUST resolve the embedder, member or reference, and refuse to query unless its SHA-256 equals embedding.sha256.
4. MUST open vectors.db read-only and refuse it when tome_meta disagrees with the manifest.
5. MUST embed the raw query text with that embedder, with no model-specific prefix, and run the query in §4 with k = retrieve_top_k.
6. SHOULD report source, start, and end of every chunk it uses.
7. MUST reject a present signature that does not verify, and MAY require a trusted signer.

An independent reader written to these rules takes 83 lines of Python. It is listed in the paper.

Graph tables.

New databases also contain graph_nodes and graph_edges. They ship empty, vector search does not read them, and a file without them is valid. When filled, a reader may add one hop from the vector results. Graph retrieval is in early access.

How the format changes.

Every file this page describes has version 1.3 and schema_version 1. A change that an existing reader would misread gets a new version number. Proposals are GitHub issues, and a change lands with specification text, the reference implementation, and a test.