Error Handling

Use retries=, retry_backoff= and timeout_seconds= on @asset or @task when a step can fail for reasons that pass: a network call, a rate limit, a lock. The coordinator runs the retry loop; the function does not need one.

Option	Meaning	Default
retries=	Total attempts. retries=3 is one attempt and up to two more.	1 (no retry)
retry_backoff=	Base delay in seconds. The delay before attempt N is retry_backoff * (N - 1).	0
timeout_seconds=	Time limit for each attempt.	300

Example

This asset fails twice and succeeds on the third attempt. It appends a line to a file on each attempt so the attempts can be seen afterwards.

import os
import time
from pathlib import Path

from barca import asset, task


@asset(retries=3, retry_backoff=1.0)
def flaky() -> dict:
    log = Path("attempts.txt")
    n = len(log.read_text().splitlines()) + 1 if log.exists() else 1
    with log.open("a") as f:
        f.write(f"attempt {n} at {time.time():.1f} pid {os.getpid()}\n")
    if n < 3:
        raise RuntimeError(f"attempt {n} failed")
    return {"attempts": n}


@task(inputs={"data": flaky})
def notify(data: dict) -> dict:
    return {"notified_after_attempts": data["attempts"]}

barca run notify pipeline.py
cat attempts.txt

What barca does

[barca] 2/2 steps | done in 0.0s
Run 14f89b37ed6a | ran 'notify' in 3.210s (2 steps, 1 phase)

Value:
{
  "notified_after_attempts": 3
}

attempt 1 at 1792600016.6 pid 2036400
attempt 2 at 1792600017.6 pid 2036400
attempt 3 at 1792600019.6 pid 2036400

The second attempt starts one second after the first and the third two seconds after the second. All three ran in the same worker process (one process id): a step that raises an exception leaves its worker running, and the worker takes the next step or the retry. What the module holds in memory (a global, a cache, an open client) carries over from a failed attempt to the next. The run reports success and exits 0; the failed attempts are not shown in the summary.

When every attempt fails

@asset(retries=2)
def broken() -> dict:
    print("broken runs")
    raise ValueError("always fails")


@task(inputs={"data": broken})
def after_broken(data: dict) -> None:
    print(f"got {data}")

barca run after_broken pipeline.py

[pipeline.py:broken] broken runs
[pipeline.py:broken] broken runs
[barca] 0/2 steps | failed in 0.0s
Step 'pipeline.py:broken' failed: ValueError: always fails
  File "/home/you/project/pipeline.py", line 27, in broken
    raise ValueError("always fails")

Fix the error in 'pipeline.py:broken' (see the traceback) and re-run the same command. Steps that succeeded are cached and will not re-run.
[barca] run failed: step 'pipeline.py:broken' failed: ValueError: always fails; 1 downstream steps skipped (exit 1)

The command exits 1. after_broken does not run. With --json, stdout carries the failure and each step’s outcome:

{"status": "failed", "failed_node": "pipeline.py:broken", "error": "ValueError: always fails",
 "steps": [{"id": "pipeline.py:broken", "kind": "asset", "status": "failed", ...},
           {"id": "pipeline.py:after_broken", "kind": "task", "status": "skipped",
            "reason": "upstream_failed", "detail": "a step it depends on failed", ...}],
 "steps_executed": 1, ...}

Each step has its own retry setting. A retry of one step never re-runs a step that already succeeded, and a downstream step’s retries= does not apply while its upstream is failing, because the downstream step never starts.

Timeouts

An attempt that runs longer than timeout_seconds fails like any other error and is retried if attempts remain. With @asset(timeout_seconds=1) on a function that sleeps for five seconds, the command fails after about one second, and the remediation says to raise the limit:

Step 'pipeline.py:slow' failed: TimeoutError: Step 'pipeline.py:slow' exceeded timeout of 1s

'pipeline.py:slow' ran longer than its timeout_seconds= (300 when not set). Raise it on the decorator or make the step faster, then re-run the same command. Steps that succeeded are cached and will not re-run.

Limits

• A failed attempt may have had effects. A retry runs the whole function again. Make a retried step safe to repeat.
• An exception the function catches is not a failure. A body that catches everything and returns normally is recorded as a success and is not retried. Let the exception out of the function when you want a retry.
• Barca does not see a loop in the body. A hand-written retry loop works, but barca sees one attempt, and the attempts share one timeout_seconds.
• Branches of parallel() are not retried, whatever their task declares: Parallel Tasks.
• A failed asset is not cached. barca status shows it as never-run with failed under WHY, and the next barca get tries it again.
• sys.exit() in a step, with any code, fails the step (barca docs tasks).
• A failed attempt keeps its worker only when the interpreter is still sound. After an exception the worker goes on to its next step. It is replaced by a fresh one when the step called sys.exit(), ran out of memory (MemoryError), was still running at its timeout_seconds, failed while its module was imported, or crashed the interpreter.