Skip to content

Core Constraints

These are the rules barca was designed around. Some were written before the code existed. Each section gives the rule, whether 0.28.0 implements it, and what happens where it does not.

Constraint In 0.28.0
Graphs are acyclic Implemented
Python interpreter resolution Implemented (no uv requirement)
Preflight consistency Not needed between commands; not checked during a run
History is append-only Rows are kept; no status flags. barca gc evicts result files, never rows
Freshness declarations All three act under barca serve; a partitioned Manual node is not pinned yet
Caching follows provenance, not recency Implemented
Asset identity Implemented; no rename detection
User code is what runs Implemented; no source snapshots

Rule. The graph has no cycles: no node depends on itself, directly or through other nodes. Iteration belongs inside one function.

Status. Implemented. Every command that reads the project checks the graph before anything runs:

$ barca list cyc.py
{"code":2,"error":"DAG error: cycle detected in dependency graph","kind":"usage","remediation":"Fix the inputs between definitions, then run `barca list cyc.py` to check each node's inputs."}

The exit code is 2. The same check rejects an input that names an unknown node, a sensor with inputs, and a task used as an input to an asset or a sensor.

Rule. Steps run in the Python environment barca is installed in.

Status. Implemented. Barca uses the python (or python3) that sits beside the barca executable, which is the virtualenv barca was installed into. If there is none, it uses python3 from PATH. An early version of this page proposed requiring uv. That was never built: barca does not need uv, does not check for it and does not manage an environment. Python 3.12 or later is required.

Rule (original). Before running a planned step, check that the function on disk still has the definition hash the plan was made with, and fail if it does not.

Status. The original rule assumed a stored plan that a later command executes. Barca has none: barca get, barca run and every run started by barca serve parse the source and plan again, so a plan cannot be stale between commands.

Within a run there is no check. The source is hashed when the run is planned, and the worker imports the file when the step executes. If you edit a function while a run that uses it is going, the step may run the new code and be recorded under the hash of the old code. Do not edit pipeline files during a run; if you did, run the affected assets again with --refresh <asset>.

Rule. Barca does not delete results or history as part of normal operation.

Status. Partly implemented.

  • Every step result adds a row to the materializations table and every run adds a row to runs. No barca command deletes rows.
  • Artifact files are kept. When a function’s code or inputs change, the new result is written to a new file (.barca/artifacts/{node}/{run_hash}{ext}) and the old file stays.
  • A result can be overwritten in place: --refresh runs a step again under the same run hash, and a function that is not deterministic then writes different bytes to the same path.
  • The original design also called for a record of each definition and for status flags (stale, superseded, inactive). Those do not exist. A node that is removed from the source is no longer listed; its rows and files stay on disk.

History rows are never pruned. Result files are: .barca/ has a size limit ([cache].max_size, default 10% of the filesystem), and past it barca evicts files after every get and run, or when you run barca gc (barca docs cache). An evicted result keeps its row, marked evicted. You can also delete files under .barca/artifacts/ yourself. Barca handles a missing file: when a step needs to read a cached result whose file is gone, the step that produced it runs again (the JSON output gives the reason artifact_missing). A missing file that nothing needs to read costs nothing. Deleting all of .barca/ removes the cache and the history.

Rule. Every asset, sensor and task declares how it is kept up to date with freshness=: Always (the default for assets and tasks), Manual (the default for sensors) or Schedule("<cron>").

Status. All three act under barca serve (RFC-0008).

  • Schedule("<cron>") makes barca serve start a run on each cron tick. The run’s targets are the scheduled node and every Always node below it.
  • Always nodes are brought up to date at each tick of a scheduled node above them: an asset when it has no cached result for its current run hash, a task when its inputs changed since its last successful run. The walk down stops at a node with its own Schedule. A node with no scheduled node above it is never run by a tick.
  • Manual assets and tasks pin themselves. A tick never runs one, and barca get or barca run of something below it reads its latest successful result, even when stale, and says so on stderr. Only naming it in --refresh (or --refresh-all) recomputes it, also when it is the target of barca get; a Manual task named as the target of barca run runs. A never-computed Manual upstream makes barca get exit 2; under a tick, the nodes that read it are skipped as blocked.
  • Sensors are polled whenever a run needs them, whatever their freshness.
  • A partitioned Manual node is not pinned yet: it is computed like an Always node.
  • POST /get, POST /run/{task} and POST /run compute Manual upstreams as before.

Freshness is not part of the definition hash. Within a run, whether a function runs is decided by the cache (next section) and by --refresh. See Scheduling.

Rule. A result is valid if it was computed from the current code and the current inputs, however long ago. The most recent result is not special.

Status. Implemented. A result is stored under its run hash, which covers the function’s code, the helper code it reaches, and its inputs. If the code changes from version A to version B and back to A, the results computed for A are cache hits again:

$ barca get b prov.py -o value --json # a returns 1
... "final_output":2 ... "steps_executed":2
$ # edit a to return 5
$ barca get b prov.py -o value --json
... "final_output":6 ... "steps_executed":2
$ # edit a back to return 1
$ barca get b prov.py -o value --json
... "final_output":2 ... "id":"prov.py:a" ... "status":"cached" ... "id":"prov.py:b" ... "status":"cached" ... "steps_executed":0

This is why old artifact files are kept, until the cache size limit evicts them (see Pruning). What the hash does not see is listed in barca docs cache.

Rule. A node’s identity is its explicit name= if it has one, otherwise the file path relative to the project root plus the function name (pipeline.py:orders). Two nodes may not share an identity.

Status. Implemented.

$ barca list dup.py
{"code":2,"error":"DAG error: duplicate continuity key: 'same' defined in both 'dup.py' and 'dup.py'", ...}
  • A node with name= keeps its cache and history when its file is moved or its function is renamed.
  • A node without name= that is moved or renamed gets a new id. It is computed again, and the results and history under the old id stay on disk under the old id.
  • The original design mentioned suggesting a probable rename from source similarity. That does not exist.

Rule. The worker imports your module from your project and calls your function, so imports, tracebacks and helper modules behave as they do outside barca.

Status. Implemented. The worker loads the pipeline file from source with whichever Python was resolved above. An early version of this page said barca also stores a snapshot of each function’s source for provenance. It does not: the metadata database holds hashes, not source.

  • Only .py files that import barca are read. Notebooks are not.
  • The graph is validated on every command.
  • Helper code a function reaches is part of its hash, so changing a helper invalidates the functions that use it.
  • Old results stay usable, and stay on disk, until you delete them.