Model client — beam-agents

This is the narrow waist between the runtime and whatever model provider you use. It is deliberately small: one Protocol with one coroutine method, two frozen value types, and a four-member exception taxonomy. Everything else — retry, backoff, circuit breaking, caching, tracing — lives above it in the model facade, so a provider client stays a transport and nothing more.

The four pieces

LlmRequest — a frozen, hashable value carrying the provider-neutral request material: model_id, messages, tools_schema, sampling_params. Those are exactly the four components compute_cache_key hashes, which is why the request type and the replay cache agree by construction rather than by convention. The activation-scoped key and seq are supplied separately, by the caller.
LlmResponse — the canonical response bytes plus their sha256 digest. Those bytes are exactly what the replay cache stores, so any client's response is cacheable without a re-serialization step that could change it.
LLMClient — a structural Protocol: async def complete(request) -> LlmResponse. There is no synchronous path, so the async bridge is the only invocation route into a provider.
ProviderError and its subclasses — RateLimitError (429, with an optional retry_after_ms), ServerError (5xx, carrying the status), and ProviderTimeout.

Why the error taxonomy is typed

Retry decisions are made by exception type, never by matching strings in a provider's message. String matching is the kind of code that works until a provider rewords an error, and then silently stops retrying something that should be retried — a failure that shows up as a mysterious drop in throughput rather than as a broken test. Typing the taxonomy makes "which failures are retryable" a property of the class hierarchy, and the facade's retry classification a table lookup.

The types also carry the fields the backoff needs: retry_after_ms on a rate limit is a lower bound the facade honors, rather than a hint it has to parse back out of prose.

Fake LLM — the in-process client the offline tier uses.
Model facade — what wraps a client into a resilient call.
Vertex AI provider — the one described client that does not exist yet.

Published verbatim from openspec/specs/model-client/spec.md — 4 requirements, 11 scenarios. Each scenario is the source a test is derived from and named after.

Purpose

TBD - created by archiving change add-fake-llm-provider. Update Purpose after archive.

Requirements

Requirement: LLM request value type

The system SHALL provide an LlmRequest frozen, hashable value type carrying the provider-neutral request material: model_id (str), messages, tools_schema, and sampling_params. These are the same four request components beam_agents.model.compute_cache_key hashes (the activation-scoped key/seq are supplied separately by the caller, not by the request). Instances MUST be immutable and MUST NOT carry provider connection details, credentials, or transport state.

Scenario: Request carries the four request-material components

WHEN an LlmRequest is constructed with model_id, messages, tools_schema, and sampling_params
THEN those four fields are readable back unchanged and no additional required field exists

Scenario: Request is immutable

WHEN code attempts to reassign any field of a constructed LlmRequest
THEN the attempt raises (frozen dataclass) and the instance is unchanged

Requirement: LLM response value type

The system SHALL provide an LlmResponse frozen value type wrapping the canonical provider response bytes and its response_digest (lowercase-hex or bytes sha256 of response). The response bytes are exactly the payload the replay cache stores, so a response produced by any LLMClient is directly cacheable without re-serialization.

Scenario: Response exposes cacheable bytes and digest

WHEN an LlmResponse is constructed from provider response bytes
THEN its response field returns those bytes unchanged and its response_digest is the sha256 of those bytes

Scenario: Response is immutable

WHEN code attempts to reassign a field of a constructed LlmResponse
THEN the attempt raises and the instance is unchanged

Requirement: Async LLMClient protocol

The system SHALL define an LLMClient typing Protocol with a single coroutine method async def complete(request: LlmRequest) -> LlmResponse. Every provider (FakeLLM now; anthropic, openai_compat, vertex, vllm later) SHALL be a structural subtype of this protocol. The protocol MUST be provider-neutral (no anthropic/openai-specific fields) and MUST NOT expose a synchronous call path, so the async bridge is the only invocation route.

Scenario: A conforming client structurally satisfies the protocol

WHEN a class defines async def complete(self, request: LlmRequest) -> LlmResponse
THEN an instance of it is accepted anywhere an LLMClient is annotated, with no explicit subclassing required

Scenario: complete is a coroutine

WHEN complete(request) is called
THEN it returns an awaitable that resolves to an LlmResponse, never a plain (already-computed) value

Requirement: Typed provider-error taxonomy for retry decisions

The system SHALL define a provider-error taxonomy the loop driver classifies for retry and backoff decisions. A ProviderError base collects the three retryable subclasses — RateLimitError (provider signalled HTTP 429, with an optional retry_after_ms), ServerError (provider signalled 5xx, carrying the numeric status), and ProviderTimeout (the provider did not respond within its deadline). The taxonomy SHALL additionally define one non-retryable typed error, ProviderRequestError (carrying the numeric status), for client-side failures — a non-429 4xx response or an undecodable success body — that a caller MUST NOT retry. ProviderRequestError SHALL NOT be a subclass of ProviderError, so an except ProviderError retry handler does not catch it and it propagates immediately (mirroring how CircuitOpenError/UnmatchedRequestError sit deliberately outside the retryable base). All are exceptions raised out of LLMClient.complete; none is returned as a value. The taxonomy MUST let a caller distinguish retryable transport failures from non-retryable client failures by type, without string-matching messages.

Scenario: Rate-limit error carries 429 semantics

WHEN a provider raises RateLimitError with retry_after_ms=1500
THEN it is an instance of ProviderError, exposes retry_after_ms == 1500, and is distinguishable by type from ServerError and ProviderTimeout

Scenario: Server error carries its status

WHEN a provider raises ServerError(status=503)
THEN it is an instance of ProviderError and exposes status == 503

Scenario: Timeout is its own type

WHEN a provider raises ProviderTimeout
THEN it is an instance of ProviderError and is neither a RateLimitError nor a ServerError

Scenario: Base type catches all retryable provider failures

WHEN any of RateLimitError, ServerError, or ProviderTimeout is raised
THEN a single except ProviderError handler catches it

Scenario: Non-retryable request error is outside the retryable base

WHEN a provider raises ProviderRequestError(status=400)
THEN it exposes status == 400, it is NOT an instance of ProviderError, and an except ProviderError retry handler does not catch it (so it propagates without retry)