datom

• **Upgrade everyone who shares a repo before anyone writes to it.** The compatibility above runs one way. After a single write from this version an older datom looks for the artifact list under a key that is no longer there, so datom_list() returns an empty frame and datom_summary() and datom_status() report zero -- **without an error**. datom_read() is unaffected: the data path never reads the manifest, so a collaborator who knows a table name and version can still read it. What they lose is the ability to see what the repo holds.
• A write that converts a repo now says so and names that consequence. datom_validate(fix = TRUE) converts the copy in storage too, which is worth knowing because it reads as a repair rather than as a format change.
• datom_list() gains a kind column, and its empty result gains current_data_sha (and version_count with include_versions = TRUE), so rbind() of two listings no longer fails when one is empty. datom_summary() gains set_count. datom_status()'s table count now counts tables rather than every artifact.
• Three return-value fields still named tables are **unchanged**: datom_status()$tables, datom_validate()$tables, and the per-table results from datom_sync(). Only the manifest key moved.

Before you upgrade: a write is refused when this build cannot account for the repo

Every manifest and every per-artifact metadata document now declares a

schema_version. Stamping it mints no new version for anything.

A write stops **at the door** -- before any hashing, any local file write and any

commit, so a refusal leaves nothing half-written -- when this build meets a repo

it cannot fully account for: the repo declares a minimum datom version above this

one, the manifest declares a format this build does not know, or any document

carries a top-level field this build cannot classify.

• **Reads and writes respond differently to the same evidence, on purpose.** A reader that meets a manifest it cannot use warns once and rebuilds the listing from storage rather than reporting an empty repo. A writer in the same position refuses. A reader that limps still answers the question it was asked; a writer that limps produces a file nobody agreed on.
• A too-new **per-artifact** document is refused at any role, because nothing can rebuild it.
• **All of this binds 0.1.1 and later.** 0.1.0 has no schema check, no field check and no version floor, and none can be added to a build that has shipped. If a 0.1.0 writer exists in your group, upgrading it is the only remedy.
• A field this build does not recognise is **preserved** rather than deleted, in per-artifact metadata, manifest entries, the manifest top level and version-history entries. So a document written by a newer datom survives a round trip through an older one.
• project.yaml now declares its own format number, separate from the repo's, so a manifest format bump can never refuse a config whose shape never changed. An unrecognised key in that file is still tolerated.

New: citable sets

A **set** is a named, versioned list of pointers at exact versions of other

artifacts, plus free-text labels. It stores no data of its own, and reading one

needs access to the set's own project only -- so a fifty-table product can be

citable by someone entitled to none of the tables. vignette("citable-sets")

walks the whole arc.

• **datom_write_set()** writes one. It needs a repo that declares mode: product and names its set in .datom/project.yaml; one repo owns one set. Re-writing identical content is a no-op: no commit, no version, no upload.
• **datom_get_set()** reads one back and returns references, not data, which is why the verb is get. Every member carries a $fetch(conn) link that pins the version it was read at -- a citation, not a subscription. A member that is itself a set comes back as a pointer, so read cost does not grow with depth.
• **datom_member()** declares one member. **datom_assemble_set()** starts an empty set and **datom_add_member()** adds one member at a time, to a new set or one read back, so a bad version is reported on the line that added it. A set in memory holds no connection: pass conn = to add a member by name, and write with datom_write_set(conn, x) or x |> datom_write_set(conn = conn).
• **On a product repo, datom_sync_manifest(conn, sources = ) and datom_sync(conn, manifest, sources = ) build and refresh the set** the way they onboard files elsewhere: map what the source projects hold, review it as a data frame, apply it. Tables and sets in a source are handled alike. The preview never proposes a removal and says which members it could not check. Unlike table sync, applying writes nothing: datom_write_set() saves the set. See ?datom_sync_manifest and ?datom_sync.
• **datom_parent(conn, table, x = )** takes the parent's version from the set you hold, resolving the name as datom_fetch_member() does, and returns a list ready for datom_write(parents = ). Calls with version = are unchanged.
• **datom_fetch_member()**, **datom_list_members()** and **datom_structure_members()** get at members: the data, a row per member per label, or a view grouped by label. Labels are the navigation axis -- a name is not a unique key, since one table at two versions is a legal pair.
• **datom_update_members()** and **datom_remove_members()** edit a set that already exists. Neither writes anything: they hand the edited set back and the write is yours. Repointing reports what moved before you write it, keeps each member's labels, and mints no version when nothing has moved. Edits made together produce one commit message naming all of them.
• A mode: product repo can commit your own code and renv.lock in the **same commit** as the set, with datom_write_set(include_paths = ), so checking out a set version gives you the data pointers plus what produced them. A product repo refuses to onboard files -- it builds its artifacts rather than importing them.
• datom_write_set() stops before writing anything when a member's recorded parents name an input that the set pins at a different version (the output was not derived again after its inputs moved), and when the set you pass belongs to another repo, by name or by project. See ?datom_write_set.
• datom_validate() understands both kinds. A set is checked for members whose pinned versions still exist and for a recorded payload hash, without descending into members that are themselves sets. fix = TRUE can restore a set's payload from git when storage has lost it.
• Per-artifact metadata now records kind, which is part of a version's identity, so a table and a set can never share a version string. **Every existing table mints one extra version at its next write** because of it; the stored data is not re-uploaded, since the content hash does not move.

New: committing your own content through datom

**datom_repo_commit()** and **datom_repo_push()** let a downstream package put

its code, lockfile or build state into the data repo without importing git2r.

paths = NULL means what git add . means, which is the **opposite** of what

datom's own writes do -- and that is the point: datom's own commits fire when

datom chose and must never sweep up work in progress, while these fire because

you asked. Commit is idempotent, push is convergent, and a no-op still pushes when

the branch is ahead. See ?datom_repo_commit for what it will and will not sweep

in.

Every version now records the commit that produced it

datom_history() gains a commit_sha column.

The value is worked out by datom, never supplied: from the commit it just made,

or recomputed from git history for a version that has none. Only the copy in

storage carries it -- your clone's copy is committed *inside* the commit that

would name it -- which is right, since the field exists for the reader who has no

clone.

**A version means content, not code**, and this is the part most likely to look

like a bug. Refactor a build script, re-run it, get identical data: no new version

is minted, and the recorded commit still points at an earlier one that does not

contain the code you are looking at. It names a commit that provably produces that

version, not every commit that could.

Smaller changes

• Every artifact's metadata records a **project** field: the name the writing repo declares in its own config, not the label on your connection. Two people can label the same repo differently, and a citation has to mean one thing. No existing artifact gains a version for it.
• A table's metadata **no longer records column_hashes**, the per-column digests added in 0.1.1. A per-column digest lets anyone who can read the metadata confirm a guess about a column's values. Nothing needs doing and no version moves; existing files lose the field the next time their table is written. See "What metadata reveals about your data" in vignette("design-version-shas") (#119).
• datom_example_data("vs") returns a vital-signs table for the same 48 subjects and visits as lb. The other four example tables are unchanged.
• New **datom_storage_read_json()** on the storage extension API, for reading a JSON document out of a project's namespace by relative key.
• A repo whose manifest cannot be read is listed by rebuilding the index from storage, with one warning naming the upgrade, rather than reported as empty.
• Identity hashing now selects fields by an explicit list rather than by exclusion, so a field this build has never heard of no longer folds into the hash and reports a change on content that did not move. Every existing hash is unchanged (#100).
• .datom_check_git_current() no longer compares against stale refs after a failed fetch (#104).