Skip to content
beam-agents
GitHub

class

ShardKeys

beam_agents.keys.ShardKeys

ShardKeys(n: 'int', *, assignment: 'Assignment' = 'hash') -> 'None'

Docstring

Fan a hot logical key across n physical shards, upstream of RunAgent.

**Memory-free agents only.** Every shard gets its own MEMORY, CONTINUATION, LLM_CACHE, PENDING and SEQ cells, so an agent that carries state between events, depends on per-key ordering, or takes HITL approvals keyed by the logical entity is broken — silently — by sharding. The module docstring and docs/sharding.md state the contract in full; the runtime performs no detection.

Placement: the **events branch only**, after WithKeys(entity_key) and before any Flatten with the tool-results and approvals streams. Those streams already carry the physical shard key (the effector echoes ToolIntent.entity_key), so re-sharding them would either double-suffix the key or route a result to the wrong shard, where it finds no continuation and dead-letters as orphaned_result.

Consumes and produces PCollection[KV[bytes, AgentEnvelope]], rewriting the KV key and the envelope's own entity_key to the same physical key so the state layout and the envelope can never disagree. Input shape is validated at pipeline-construction time.

assignment:

- "hash" (default) — SHA-256 of the element payload modulo n. Pure, so a bundle retry reproduces the same physical keys, the same ``(key, seq) cache hits, and byte-identical intent_id``s. Its failure mode is skew: low-entropy payloads (many identical events) land on one shard and the fan-out collapses, so verify the spread before trusting n. - "round_robin" — an explicit opt-in for exactly that skew case, using a worker-local counter. **It forfeits deterministic shard assignment under bundle retries:** a retry's counter state differs, so the element lands on a different shard, mints different intent_ids (the effector's dedup no longer suppresses the duplicate side effect) and misses the replay cache (extra provider calls on every retry). Use it only for agents that emit no intents — or whose effects are idempotent independently of intent_id — and only where duplicate provider calls on a retry are an accepted cost.

src/beam_agents/keys.py:182

Members

1 declared on ShardKeys

Introspected from the class itself. A member with no docstring is shown as having none rather than described from its name.

  • expand

    method

    expand(self, pcoll: 'beam.pvalue.PCollection') -> 'beam.pvalue.PCollection'

    Fan each logical entity across shards physical keys.

    Raises ValueError at pipeline-construction time on non-KV input, naming the beam.WithKeys call the caller is missing. Safe for memory-free agents only — see docs/sharding.md.