Core Constraints
These are the rules barca was designed around. Some were written before the code existed. Each section gives the rule, whether 0.28.0 implements it, and what happens where it does not.
| Constraint | In 0.28.0 |
|---|---|
| Graphs are acyclic | Implemented |
| Python interpreter resolution | Implemented (no uv requirement) |
| Preflight consistency | Not needed between commands; not checked during a run |
| History is append-only | Rows are kept; no status flags. barca gc evicts result files, never rows |
| Freshness declarations | All three act under barca serve; a partitioned Manual node is not pinned yet |
| Caching follows provenance, not recency | Implemented |
| Asset identity | Implemented; no rename detection |
| User code is what runs | Implemented; no source snapshots |
Directed acyclic graphs only
Section titled “Directed acyclic graphs only”Rule. The graph has no cycles: no node depends on itself, directly or through other nodes. Iteration belongs inside one function.
Status. Implemented. Every command that reads the project checks the graph before anything runs:
$ barca list cyc.py{"code":2,"error":"DAG error: cycle detected in dependency graph","kind":"usage","remediation":"Fix the inputs between definitions, then run `barca list cyc.py` to check each node's inputs."}The exit code is 2. The same check rejects an input that names an unknown node, a sensor with inputs, and a task used as an input to an asset or a sensor.
Python interpreter resolution
Section titled “Python interpreter resolution”Rule. Steps run in the Python environment barca is installed in.
Status. Implemented. Barca uses the python (or python3) that sits beside the barca
executable, which is the virtualenv barca was installed into. If there is none, it uses
python3 from PATH. An early version of this page proposed requiring uv. That was never
built: barca does not need uv, does not check for it and does not manage an environment.
Python 3.12 or later is required.
Preflight consistency
Section titled “Preflight consistency”Rule (original). Before running a planned step, check that the function on disk still has the definition hash the plan was made with, and fail if it does not.
Status. The original rule assumed a stored plan that a later command executes. Barca has
none: barca get, barca run and every run started by barca serve parse the source and
plan again, so a plan cannot be stale between commands.
Within a run there is no check. The source is hashed when the run is planned, and the worker
imports the file when the step executes. If you edit a function while a run that uses it is
going, the step may run the new code and be recorded under the hash of the old code. Do not
edit pipeline files during a run; if you did, run the affected assets again with
--refresh <asset>.
History is append-only
Section titled “History is append-only”Rule. Barca does not delete results or history as part of normal operation.
Status. Partly implemented.
- Every step result adds a row to the
materializationstable and every run adds a row toruns. No barca command deletes rows. - Artifact files are kept. When a function’s code or inputs change, the new result is written
to a new file (
.barca/artifacts/{node}/{run_hash}{ext}) and the old file stays. - A result can be overwritten in place:
--refreshruns a step again under the same run hash, and a function that is not deterministic then writes different bytes to the same path. - The original design also called for a record of each definition and for status flags (stale, superseded, inactive). Those do not exist. A node that is removed from the source is no longer listed; its rows and files stay on disk.
Pruning
Section titled “Pruning”History rows are never pruned. Result files are: .barca/ has a size limit ([cache].max_size,
default 10% of the filesystem), and past it barca evicts files after every get and run, or
when you run barca gc (barca docs cache). An evicted result keeps its row, marked evicted.
You can also delete files under .barca/artifacts/ yourself. Barca handles a missing file:
when a step needs to read a cached result whose file is gone, the step that produced it runs
again (the JSON output gives the reason artifact_missing). A missing file that nothing needs
to read costs nothing. Deleting all of .barca/ removes the cache and the history.
Freshness declarations
Section titled “Freshness declarations”Rule. Every asset, sensor and task declares how it is kept up to date with freshness=:
Always (the default for assets and tasks), Manual (the default for sensors) or
Schedule("<cron>").
Status. All three act under barca serve (RFC-0008).
Schedule("<cron>")makesbarca servestart a run on each cron tick. The run’s targets are the scheduled node and everyAlwaysnode below it.Alwaysnodes are brought up to date at each tick of a scheduled node above them: an asset when it has no cached result for its current run hash, a task when its inputs changed since its last successful run. The walk down stops at a node with its ownSchedule. A node with no scheduled node above it is never run by a tick.Manualassets and tasks pin themselves. A tick never runs one, andbarca getorbarca runof something below it reads its latest successful result, even when stale, and says so on stderr. Only naming it in--refresh(or--refresh-all) recomputes it, also when it is the target ofbarca get; aManualtask named as the target ofbarca runruns. A never-computedManualupstream makesbarca getexit 2; under a tick, the nodes that read it are skipped asblocked.- Sensors are polled whenever a run needs them, whatever their freshness.
- A partitioned
Manualnode is not pinned yet: it is computed like anAlwaysnode. POST /get,POST /run/{task}andPOST /runcomputeManualupstreams as before.
Freshness is not part of the definition hash. Within a run, whether a function runs is decided
by the cache (next section) and by --refresh. See Scheduling.
Caching follows provenance, not recency
Section titled “Caching follows provenance, not recency”Rule. A result is valid if it was computed from the current code and the current inputs, however long ago. The most recent result is not special.
Status. Implemented. A result is stored under its run hash, which covers the function’s code, the helper code it reaches, and its inputs. If the code changes from version A to version B and back to A, the results computed for A are cache hits again:
$ barca get b prov.py -o value --json # a returns 1... "final_output":2 ... "steps_executed":2$ # edit a to return 5$ barca get b prov.py -o value --json... "final_output":6 ... "steps_executed":2$ # edit a back to return 1$ barca get b prov.py -o value --json... "final_output":2 ... "id":"prov.py:a" ... "status":"cached" ... "id":"prov.py:b" ... "status":"cached" ... "steps_executed":0This is why old artifact files are kept, until the cache size limit evicts them (see
Pruning). What the hash does not see is listed in
barca docs cache.
Asset identity
Section titled “Asset identity”Rule. A node’s identity is its explicit name= if it has one, otherwise the file path
relative to the project root plus the function name (pipeline.py:orders). Two nodes may not
share an identity.
Status. Implemented.
$ barca list dup.py{"code":2,"error":"DAG error: duplicate continuity key: 'same' defined in both 'dup.py' and 'dup.py'", ...}- A node with
name=keeps its cache and history when its file is moved or its function is renamed. - A node without
name=that is moved or renamed gets a new id. It is computed again, and the results and history under the old id stay on disk under the old id. - The original design mentioned suggesting a probable rename from source similarity. That does not exist.
User code is what runs
Section titled “User code is what runs”Rule. The worker imports your module from your project and calls your function, so imports, tracebacks and helper modules behave as they do outside barca.
Status. Implemented. The worker loads the pipeline file from source with whichever Python was resolved above. An early version of this page said barca also stores a snapshot of each function’s source for provenance. It does not: the metadata database holds hashes, not source.
What follows from these
Section titled “What follows from these”- Only
.pyfiles that import barca are read. Notebooks are not. - The graph is validated on every command.
- Helper code a function reaches is part of its hash, so changing a helper invalidates the functions that use it.
- Old results stay usable, and stay on disk, until you delete them.