Guides

Reliability

A metering SDK that can break your app or slow down a real call is worse than no metering at all. Every guarantee below is tested, not aspirational, and the failure modes are named, not hidden.

The guarantees

Never throws into host code. Only the SDK's own bookkeeping is wrapped in try/except. The real provider call and its real exceptions pass through untouched. Proven for the transport, the buffer, and every wrapper's success/error/stream paths.
Never adds latency. Capture is post-hoc from the response object the provider already returned. No network I/O on the metering path. The one deliberate exception: Bedrock's InvokeModel response body is read in-memory to extract usage, then replayed via a fresh in-memory stream so the host's own response["body"].read() still works. Extra in-memory work, never a network hop.
Fails open. If the Ingest API is unreachable, rate-limited, or returns an unexpected status, the call you made to the provider already succeeded or failed on its own terms: metering failure never retroactively affects it.

Governance preflight is the opt-in exception

These guarantees cover metering. If you opt into governance preflight (governed() / GovernanceClient.guard()), that step is deliberately on the request path: it makes one HTTP call to the Router before your provider call (2 s timeout by default), raises on a denial in enforce mode, and raises when the Router is unreachable unless you set the fail mode to open. A topicClassifier you supply also runs in-process before the request; a throwing classifier is ignored.

Delivery model

Events go into an in-memory ring buffer (10,000 by default) and flush every 2 seconds or 500 events, whichever happens first. Recording an event is a synchronous append. It never blocks a provider call.

Ring buffer, drop-oldest. On overflow, the buffer evicts the oldest event rather than rejecting the newest. Recent activity is what you're most likely to be debugging live. The evicted count is tracked as a running dropped_events counter and ships in the next batch header, so loss is visible on the coverage screen, never silent.
At-least-once delivery, exactly-once storage. A batch can be sent more than once (retry after a timeout the server actually received). The client doesn't try to guarantee exactly-once on the wire. Instead, every event carries a client-generated UUIDv7 as its ID, and storage deduplicates on that ID. A retried batch is safe: duplicates land on the same rows, not double-counted spend.

Honest failure modes

A stream that aborts before its usage frame

If a stream is aborted before its terminal usage frame arrives (or a provider response carries no usage at all), the event is recorded with usage_complete: false and every token field null. It is never estimated from partial deltas. A guessed number that looks confident is worse than an honest gap.

Serverless flush caveats

On a runtime that freezes the process between invocations, events recorded after the last flush()/waitUntil() call of an invocation are lost. There is no background timer left to catch them once serverless mode is on. See Frameworks → Serverless for the exact call pattern that avoids this.

On process exit, a best-effort flush runs capped at 2 seconds, enough for a normal shutdown, not a guarantee under a hard kill signal.