Skip to content

High metric volume: system metrics are emitted per-process (per worker), not per host #2185

Description

@strawgate

Summary

instrument_system_metrics() can produce very high metric volume on multi-process deployments, because the host-scope system metrics are emitted once per process rather than once per host. On a many-worker server (e.g. Gunicorn/Uvicorn --workers, Celery prefork) running on a high-core machine, this multiplies into a large number of time series.

Where it comes from

The dominant term is the two per-core metrics, system.cpu.time and system.cpu.utilization. Their OTel callbacks iterate psutil.cpu_times(percpu=True), so each emits one time series per (core × CPU-state field):

per-core CPU series = 2 metrics × cores × cpu_state_fields

On a 96-core Linux host (~8 CPU-state fields kept) that is 2 × 96 × 8 ≈ 1,536 series — roughly 95% of everything base='full' emits. base='basic' avoids this entirely (its CPU metric is the single aggregate system.cpu.simple_utilization), but the per-process duplication below still applies to whatever is enabled.

The per-process multiplication

The system metrics are host-scope, but they are emitted by every process that has them enabled:

  • Each worker configures independently (preload_app=False, spawn): each worker process runs instrument_system_metrics() itself, so each gets its own MeterProvider + exporter. The in-process singleton (GLOBAL_CONFIG, and SystemMetricsInstrumentor no-op'ing on re-instrument) only dedupes within a process, not across processes.
  • Configure-then-fork (preload_app=True): the forked child keeps exporting too — OTel's PeriodicExportingMetricReader registers os.register_at_fork(after_in_child=...) and restarts its export thread in the child, and logfire re-stamps process.pid in its own fork hook.

Either way, N workers on a host produce ~N copies of the host-scope series. With the default 60s export interval that is series × N × 1440 points/day.

Suggestion

Host-scope metrics (system.cpu.*, system.memory.*, system.disk.*, system.network.*) only need one emitter per host; process.* metrics are legitimately per-process and should stay per worker. Some options, roughly in order of blast radius:

  1. Fork-child suppression of host-scope metrics. Extend the os.register_at_fork(after_in_child=...) hook logfire already registers so a forked child stops emitting host-scope system metrics (e.g. SystemMetricsInstrumentor().uninstrument() + no-op logfire's own system.* gauge callbacks), while keeping process.* per worker. Fixes the configure-then-fork case cleanly.
  2. Warn when already enabled in an ancestor. Set an inherited marker at instrument time and emit a warning (or no-op) if a descendant calls instrument_system_metrics() again — makes the duplication self-evident.
  3. Optional host-level single-writer (file lock / abstract socket) so only the first process per host emits host-scope metrics. This is the only thing that also covers the each-worker-configures-independently case; heavier, so opt-in.

Documentation could also steer users to enable system metrics once per host (a master-only hook or a sidecar) rather than in the per-worker app-import path.

Environment note

Confirmed against the current SDK: default metric export interval is 60s (inherited from OTel; logfire passes no export_interval_millis), and the per-core percpu=True behavior is in opentelemetry-instrumentation-system-metrics.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions