Artifacts and retention¶

Every expensive intermediate of a run is an artifact, keyed by the pydantic spec that produced it. A run asks the store before it computes, publishes what it made, and keeps in its bundle only what the store cannot give back. This page covers the store contract, how keys and names are derived, the publish and claim protocol, what the driver fetches and publishes per stage, the cache tiers, retention and purge, the mesh artifact, and the process boundary around gmsh. Artifacts and retention in the user guide covers configuration and the commands.

The store protocol¶

ArtifactStore is one storage tier addressed by (kind, key). Its contract is small enough for any backend to implement: an artifact exists exactly when its sidecar can be read, and put is first-wins and atomic (src/femforge/artifacts/store.py:28-71).

Failures raise ArtifactError, with ArtifactMissing for a key no tier holds and ArtifactCorrupt for files that disagree with their sidecar (src/femforge/artifacts/store.py:16-25).

LocalArtifactStore is the directory implementation, and femforge ships no other. Its root holds one directory per kind:

Kinds¶

A kind is a module-level ArtifactKind constant that callers pass by reference; no registry looks one up by name (src/femforge/artifacts/kind.py:49-51).

BuiltinKind is the only place the built-in kind names are spelled (src/femforge/artifacts/kind.py:19-29). The built-in kinds live beside the run identity in src/femforge/run/kinds.py:

json_record_kind declares a further kind whose artifact is one record.json holding a typed result, such as one point of a sweep; RecordKind.get_or_produce serves the stored value or computes and publishes it under a claim (src/femforge/artifacts/records.py).

Identity and names¶

artifact_key hashes the kind name, its producer version and the spec payload with every HashExclude() field removed, unless the kind overrides its key (src/femforge/artifacts/keys.py:97-105). The bundle kind overrides it with cache_key, so a bundle's store key and its run id agree (src/femforge/run/kinds.py:159-166). Config objects covers the run and STEP payloads.

An ArtifactKey is a str of 18 to 64 lowercase hex characters, typed by the spec model it was hashed from, so a mesh key cannot pass where a bundle key belongs (src/femforge/artifacts/keys.py:27-46). Its name is the <adjective>-<noun>-<hash8> directory name.

Every persisted key carries its directory name beside it. ArtifactRecord and ArtifactRef derive name from key in a validator and never read it from input (src/femforge/artifacts/sidecar.py:80-81, src/femforge/artifacts/sidecar.py:187-199), and RunManifest.mesh_name follows mesh_key the same way (src/femforge/run/manifest.py:117-118). A manifest's references therefore read straight to a store directory, and because the name is a pure function of the key, resolving a key never scans the store.

A name carries about 50 bits, so two keys can share one in principle. The store detects it: record returns None and logs an error when the sidecar holds a different key, and put raises (src/femforge/artifacts/local.py:143-150, src/femforge/artifacts/local.py:221-222). A collision is never served.

The sidecar records existence¶

artifact.json is the commit marker, so it is always written last (src/femforge/artifacts/sidecar.py:19-20). An ArtifactRecord lists every file with its size and SHA-256, the exact spec payload that was hashed, the producer's host and process id, and the tool versions (src/femforge/artifacts/sidecar.py:69-105). Tool versions are provenance and never enter a key (src/femforge/artifacts/sidecar.py:51-66).

A consumer records an ArtifactRef: the kind, the key, where it came from (local, remote or produced) and the consumer-relative files it materialized (src/femforge/artifacts/sidecar.py:23-33, src/femforge/artifacts/sidecar.py:176-199).

First-wins publish¶

LocalArtifactStore.put commits with one directory rename (src/femforge/artifacts/local.py:210-258):

1. When the source is a staging directory, prune it to the listed files; otherwise copy or link the listed files into a fresh staging directory.
2. Write the sidecar into the staging directory atomically.
3. Rename the staging directory onto the final path. os.replace fails when the target directory already exists, which is the first-wins signal (src/femforge/artifacts/local.py:36-38).
4. On that failure, or when the key was already committed, discard the copy and return False. The key is content-addressed, so the winner's copy is as good as the loser's.

A reader therefore sees a whole artifact or none, with no lock taken on the read path.

Heartbeat claims¶

A claim is a directory, because mkdir is atomic on NTFS, POSIX filesystems and SMB shares (src/femforge/artifacts/lock.py:1). DirLock writes an owner.json with the host, the process id, the start time and the last heartbeat (src/femforge/artifacts/lock.py:45-60).

try_acquire breaks a stale claim and retries once (src/femforge/artifacts/lock.py:103-118). A claim whose owner file is unreadable ages by its directory's modification time (src/femforge/artifacts/lock.py:84-91). keep_alive beats from a daemon thread for as long as its block runs, in a copy of the caller's context, so a failed beat's warning keeps the run id and the telemetry route (src/femforge/artifacts/lock.py:134-158). A holder never needs a timeout sized to its longest job: a live holder never looks stale, and a dead one goes stale in two minutes.

The driver claims runs with the same lock: it takes runs_dir/<run_id>.lock, keeps it alive while it solves, and a second process asking for the same run checks every 2 s for a reusable bundle (src/femforge/run/driver.py:54-55, src/femforge/run/driver.py:216-234). reap_runs settles a driver that died: a claim counts as silent when its directory is gone, its heartbeat is stale, or its owner ran on this host under a process id that no longer exists (src/femforge/run/reap.py:52-68).

What the driver fetches and publishes¶

The driver runs its pipeline as the Stage members in order, PUBLISH and CLEANUP last (src/femforge/run/stages.py:11-45). Three stages and the reuse check before them touch the artifact cache:

Every store-sourced input lands in manifest.artifacts as it arrives (src/femforge/run/driver.py:548-555), and manifest.cached_inputs() lists their files (src/femforge/run/manifest.py:130-132). That list is what lets retention drop the inputs freely, what rehydrate_bundle fetches back, and what prune_store_sourced removes (src/femforge/run/artifacts.py:145-169). The publish stage records manifest.published and writes the manifest before it commits, so the published manifest already names its own store entry, and it runs after the run's log context has closed run.log, so the logs class can be published and cleaned up too (src/femforge/run/driver.py:342-343).

BundlePolicy maps to retention classes: CORE publishes everything but fields and meshes, FIELDS adds the VTU output, and FULL publishes every class (src/femforge/run/artifacts.py:95-117). It also decides reuse: under FIELDS or FULL, a local bundle whose field output a cleanup removed no longer counts as reusable (src/femforge/run/driver.py:273-274).

The cache tiers¶

ArtifactCache puts a typed front door on one or two stores (src/femforge/artifacts/cache.py:60-88):

Claiming on the remote tier is what makes machines that share it produce each artifact once.

Retention internals¶

A bundle's files fall into the retention classes of RunFile; each member carries the bundle-relative patterns it covers (src/femforge/common/files.py:71-94). BUNDLE holds manifest.json and result.json and is protected: no policy field names it, and the planner never removes a file it matches (src/femforge/run/retention.py:338-339).

RetentionPolicy ties each field to its class with Map(RunFile.X) metadata, and rules() reads the pairs back through mapped_items, so no table beside the model lists the classes (src/femforge/run/retention.py:67-109). cached_inputs is no class: it escalates the rule for files the store can re-fetch, and the stricter rule wins. Cleanup members escalate in definition order, NEVER then ON_SUCCESS then ALWAYS, and ON_SUCCESS applies to status ok only (src/femforge/run/retention.py:20-45).

plan_cleanup resolves a policy against one bundle (src/femforge/run/retention.py:342-403):

1. Refuse a bundle that is a symlink or a junction.
2. When the run_dir rule applies, plan the whole directory and stop.
3. Glob each class's patterns, skipping links, protected files and paths an earlier class already claimed.
4. Escalate the rule of a store-sourced path by cached_inputs, and keep the paths whose rule applies to the run's status.

apply_cleanup never raises: a path it cannot remove lands in failed. It only reports a planned run directory, because the caller must write the manifest one last time first (src/femforge/run/retention.py:406-434). The driver's cleanup stage applies the plan and appends a CleanupRecord to the manifest; the directory itself goes after the stage's finish event (src/femforge/run/driver.py:433-498).

Purge¶

femforge purge and python -m femforge.run.purge apply a policy to bundles that already exist, as a dry run unless you pass --apply:

femforge purge --runs-dir runs --preset lean --dry-run
femforge purge --runs-dir runs --preset minimal --fields always --apply
femforge purge --runs-dir runs --preset lean --elmer-mesh always --only-status error abandoned --apply
python -m femforge.run.purge --runs-dir runs --preset lean

The parser adds one override flag per RetentionPolicy field, so a new file class gets its flag without a CLI change (src/femforge/run/purge.py:243-300). CLI lists every option. purge_runs walks the runs directory and leaves a directory alone for one of these reasons (src/femforge/run/purge.py:129-212):

Each pass that removes something appends a CleanupRecord with trigger purge (src/femforge/run/purge.py:88-126).

The mesh artifact¶

MeshSpec holds everything that decides the Elmer mesh files and their group names, and nothing that decides only the solve (src/femforge/meshing/spec.py:25-49):

Conditions, material properties, physics and metadata stay out, so configs that differ only there share one mesh (mesh_key, src/femforge/run/kinds.py:108-132).

The artifact keeps ElmerGrid's files, never the .msh, plus groups.json, a MeshGroupsFile of body and boundary group records and the element count (src/femforge/meshing/groups.py:156-258). rebind_groups turns the records back into groups against the current domain: materials by tag, tags by selector, and each body's name composed again with compose_group_name. Any mismatch raises MeshRebindError, so a condition never lands on the wrong boundary (src/femforge/run/artifacts.py:35-92). The driver then sets each group's id from mesh.names, because ElmerGrid renumbers the groups, boundary ids after body ids (src/femforge/run/driver.py:616-625, src/femforge/elmer/mesh_names.py).

The boundary groups come from body adjacency. partition_boundary_faces sorts every boundary face into an interface A__B, an external shell A__EXT or, for a single body on the symmetry axis, A__AXIS (src/femforge/meshing/boundary.py:98-129). The partition is total: once a model defines physical groups, gmsh saves only the elements that belong to one, so a face outside every group would vanish from the mesh. It is also sorted, so gmsh assigns the same ids on every run. A face that borders no known body is skipped, because it has no name to take (src/femforge/meshing/boundary.py:106-107).

The gmsh firewall¶

gmsh, the mesh backend implemented today, runs in the driver's process or behind a process boundary (src/femforge/meshing/backend.py:136-253):

WorkerMesher writes the job as JSON into a temporary directory and starts the worker on it (src/femforge/meshing/backend.py:219-253). The worker logs every record to stdout as one JSON line, meshes, and writes an outcome file, turning any exception into a failed outcome (src/femforge/meshing/worker.py:33-65). The parent relays each line into its own log with worker=mesh bound and strips FEMFORGE_TELEMETRY from the child's environment, so the child writes no telemetry of its own and its records reach the parent's terminal, run.log and telemetry through the relay (src/femforge/meshing/backend.py:227-243). A worker that exits without an outcome, or runs past timeout_s, raises MeshError with the tail of its log.

block_gmsh installs an import hook that makes every later import gmsh raise, and refuses when gmsh is already loaded (src/femforge/meshing/firewall.py:18-34). An entry point calls it to prove it never loads gmsh.

tests/run/test_gmsh_firewall.py::test_a_worker_driver_never_loads_gmsh guards the boundary. It runs tests/run/worker_driver.py in a fresh interpreter, which blocks gmsh and then solves through a WorkerMesher and a stand-in Elmer three times: uncached, producing a cached mesh, and reusing that mesh for a config with the magnetization flipped (tests/run/worker_driver.py:134-159). The test asserts three ok results, a written .msh, the installed gmsh version on the mesh record, and no gmsh among the driver process's modules (tests/run/test_gmsh_firewall.py:32-50).