Runners — beam-agents

Support claims are cheap to make and hard to check, so this page states a support level for each runner and the evidence in this repository behind it.

DirectRunner, Flink, and Dataflow are supported runners. Each is exercised by named tests on a stated schedule: DirectRunner and Flink on every pull request, Dataflow nightly against the real service. Spark is not supported. A weekly leg stands the Spark stack up, but a runner-level gap leaves every conformance cell a declared skip, so nothing on it actually executes.

The group is the supported set, and each row names what backs it. Spark sits outside the group and has no connector. A weekly leg runs against a real Spark stack, but a recorded Beam Spark portable-runner gap — no bundle checkpoint handler for the streaming ingest source — leaves every conformance cell a declared skip, so Spark is drawn as an unverified target rather than a fourth supported one.

What runs where today

Runner	Support	Exercised by	Tier
DirectRunner	Supported	The whole unit tier, plus the conformance matrix's DirectRunner leg	Offline, every PR
Flink	Supported	Mini-cluster in Docker; the conformance matrix's Flink leg and the effectively-once end-to-end gate	Docker-backed, every PR
Dataflow	Supported	`-m dataflow` tests against real Dataflow with FakeLLM over HTTP	Nightly
Spark	Unverified	A weekly leg (`docker/compose.spark.yaml`) stands up the stack, but every conformance cell is a declared skip — Beam's Spark portable runner registers no bundle checkpoint handler, which the streaming ingest source requires	Weekly; zero scenarios currently execute

DirectRunner, Flink, and Dataflow are the project's supported v1.0 runners, and the table above is what that word rests on. Spark stays best-effort, which on this page means unverified: a target the project intends to support, not one it claims to today.

What "supported" rests on

Flink. A per-pull-request job builds the SDK-harness image, brings the mini-cluster stack up, and runs two gates against it: the docker-backed effectively-once end-to-end semantics gate, and the adapter conformance matrix's Flink leg. Five of the seven canonical conformance scenarios run on that leg for every registered adapter. The other two are declared skips with recorded reasons, and neither is a Flink gap: the bundle-retry-cache cell needs an in-process commit-failure monkeypatch that cannot reach the harness container (real replay on Flink is covered instead by the restart-mid-suspension cell), and the TTL-expiry cell needs idle-partition watermark control the harness does not have (TTL garbage collection is runtime rather than adapter-visible behavior, so the DirectRunner leg keeps full adapter coverage).
Dataflow. A nightly job runs the dataflow-marked tests against the real service in the GCP project the repository's CI variables name: the pipeline --update state-compatibility gate, which launches the previous release and then replaces it in place with head (see docs/state-compat.md), and the Flex Template launch gate, which validates the packaging built from that same commit. Where those CI variables are unset the job reports a visible skip rather than a pass, so an unconfigured fork never reads as evidence.

What the runner has to provide

The requirements come from Beam's Python SDK realities rather than from preference:

Stateful DoFn support with KV input. RunAgent validates its input is KV-shaped at pipeline-construction time and raises ValueError otherwise.
Watermark and real-time timers. TTL_TIMER is event-time; HITL_TIMER is processing-time. A runner without both domains cannot provide memory GC and fail-closed approvals.
Bundle retry semantics. The atomic-commit and replay-cache guarantees are stated in terms of bundle retries; a runner that retries differently changes what those guarantees mean.

There is no MapState or OrderedListState in Beam Python user state, which is why bounded maps live inside protobuf blobs, and no portable async DoFn, which is why the runtime runs a bridge thread per DoFn instance.

Overhead is measured, not assumed

The project states a design budget for runtime overhead — p50 under 15 ms and p99 under 60 ms per activation, excluding model and tool time — and the repository carries the instrument that renders the verdict: an offline pyperf suite under benchmarks/ (make bench), gated against benchmark-baseline.toml by make bench-gate. The suite, its dimensions, and what each deliberately excludes are documented in docs/benchmarks.md. This site publishes no figures of its own — the gated baseline in the repository is the source of record.

Deploying

What has to be true of the container image before Dataflow workers can run an agent against a real model provider is documented in docs/deploying.md, alongside the quickstart's laptop → local Flink → Dataflow ladder.

Not yet implemented

This page carries the Partial badge even though three of its four runners are supported, because the badge describes the runner surface as a whole and two things in it are still missing:

No conformance scenario executes on Spark. The weekly leg exists and runs, but a runner-level gap (no bundle checkpoint handler in Beam's Spark portable runner) currently skips every cell. Treat "best-effort" as "unverified".
No production Flink deployment guide. docs/deploying.md covers the Dataflow image path; running against a production Flink cluster is exercised by tests, not documented as an operational procedure.

Testing tiers and CI — how the legs are split.
The effector — the piece that runs outside any runner.