etch

Etch Store Operations

Etch is Convex's content-addressed store: values are keyed by hash, and a single root hash reaches everything retained. Peers keep their state here.

Every subcommand takes the store file with -e / --etch:

java -jar convex.jar etch info -e /path/to/store.etch

Inspecting

info, dump and read take -o / --output-file to write to a file rather than the terminal — use it for dump on any real store, which is large.

Start a diagnosis with info, then validate. If validation reports failures, capture the output before doing anything that mutates the store.

Maintenance

These modify or destroy data. Confirm with the user, and make sure the peer using the store is stopped first — operating on a live store risks corruption.

To compact safely, prefer gc -o <new-file>: it collects into a fresh file and leaves the source unmodified (note: status levels above PERSISTED, e.g. ANNOUNCED, survive an in-place GC but not --output). Use migrate to copy into another (possibly non-empty) store, or to change the store's format version or encryption — see below.

For a combined snapshot and in-place collection, use gc --backup <snapshot-file>. The collected store keeps the original live path; the backup retains the pre-GC root and all old entries, including unreachable data. It uses a hard link, so the backup must have a new filename on the same filesystem with hard-link support. Existing backups are never overwritten. The old disk space remains allocated until the snapshot is removed. Treat the snapshot as read-only and use a separate copy for a backup on another filesystem. --backup and --output are mutually exclusive. A backup left by an interrupted command is not a confirmed snapshot; inspect the recovery state first. On Windows use the same NTFS volume. Live writes after cutover use the collected file, so they do not update the snapshot. The FFM backend releases mappings on close; MappedByteBuffer mappings may defer installation at the live filename. Etch follows its completion marker until a later open can finish installation.

repair is the offline salvage path for a dirty or damaged source. It holds an exclusive source lock, scans through physical EOF, and writes only canonical CAD3 values whose stored content hashes verify. A complete result requires both a fully persisted selected root and an exhaustive scan. If either condition fails, the command reports failure but may leave a valid partial destination; never replace the source with that output automatically.

Recovering an interrupted GC is what recover is for — reach for it before concluding a store is lost.

Encrypted Stores (Etch v3)

Etch format v3 supports encrypted stores — format spec in convex-core/docs/ETCHv3.md.

Opening an encrypted store: any etch subcommand takes --etch-key <alias> (keystore key alias or public-key prefix, password via --etch-keypass) or --etch-key-file <file> (raw or hex 32-byte master key; - reads stdin). With neither, the v3 header's public-key hint selects a keystore key automatically, or an interactive session prompts for a hex key.

Converting a store: etch migrate (and gc -o) accept destination options — --into-version (1, 2 or 3), --into-cipher (none, aes-256-ctr or chacha20), --into-key / --into-key-file / --into-keypass for the destination key, --into-encrypt-index (negatable) and --into-public-key-hint. Encryption options require version 3; without a destination key the resolved source key is reused.