The `timings.py` module#

Summary#

Classes

TimingsCollector

Process-wide singleton. Thread-safe. No async is required.

Functions

get_collector

Return the collector.

Description#

Lightweight in-process latency collector for the gateway.

The attribute “the system feels slow” must be attributed to the right layer: HTTP endpoint, tool dispatch, or live Fluent backend call. Doing that with logs alone is hard because the user has to scroll a noisy file. This module keeps a fixed-size counter table per scope and exposes it through GET /v1/diagnostics/timings.

Design constraints#

Always-on, near-zero overhead. A timed block does one time.perf_counter() pair plus three dictionary updates under a single threading.Lock. No allocations per call are made beyond the key string the caller already has.
Bounded memory. Each scope (http/tool/ backend) keeps at most _MAX_KEYS distinct keys (1024). When the table is full, new keys are silently dropped. Stale-but-bounded is preferred over unbounded growth.
No external deps. Stdlib only. The collector is importable from any module without pulling in FastAPI, PyFluent, or asyncio.
Reset on demand. GET /v1/diagnostics/timings?reset=true clears the counters automatically so the user can take a fresh sample around a specific user action.

Records kept per (scope, key)#

count: Number of completed calls.
total_ms: Cumulative wall time.
min_ms / max_ms: Extremes.
last_ms: Most recent sample (helps catch cold-start outliers).
errors: Number of calls that left the timed block via an exception. The elapsed time is still recorded so a slow failure doesn’t disappear from the table.

Module detail#

timings.get_collector() → TimingsCollector#