Bug description
The Prometheus exporter's internal topology refresh is throttled to at most once every 2 seconds (hardcoded in src/exporters/prometheus.rs, show_metrics()), but the timestamp used for that throttle (last_request) is updated unconditionally on every request, not only when a refresh actually happens:
if now - (*last_request) > Duration::from_secs(2) {
metric_generator.topology.refresh();
}
*last_request = now; // updated regardless of whether refresh() ranIf /metrics is scraped more frequently than every 2 seconds (e.g. a 1s Prometheus scrape_interval, or a tight polling loop), now - *last_request never exceeds 2s, so topology.refresh() effectively never runs again after the first successful call. All derived metrics (scaph_socket_energy_microjoules, scaph_socket_power_microwatts, scaph_process_power_consumption_microwatts, etc.) get stuck at whatever was captured on the last refresh that did occur, sometimes for well over a minute, producing a "staircase" pattern instead of a live per-scrape update.
This is easy to miss because a single scrape in isolation looks fine (a plausible, monotonically increasing counter value) — the problem only becomes visible when comparing consecutive scrapes and noticing long runs of bit-identical values.
To Reproduce
Run scaphandre with the prometheus exporter: scaphandre prometheus (tested both as a native binary and via the hubblo/scaphandre:1.0.2 Docker image).
Scrape /metrics repeatedly at an interval shorter than 2 seconds, e.g.:
while true; do curl -s localhost:8080/metrics | grep scaph_socket_energy_microjoules; sleep 1; done
Observe that scaph_socket_energy_microjoules (and other derived metrics) return the exact same value for many consecutive scrapes in a row, only updating sporadically — sometimes after 15-20+ seconds — rather than approximately every two seconds.
Expected behavior
Either:
The exporter should refresh on (approximately) every scrape regardless of interval, so metrics reflect current state each time /metrics is queried, or
If throttling is intentional (to avoid excessive sysfs/process-tree churn on very frequent scraping), this should be clearly documented, and ideally configurable via a CLI flag (there is currently no --step/-s equivalent for the prometheus exporter), so users scraping faster than the hardcoded 2s floor are aware their data will be coarser than their scrape interval implies.
At minimum, last_request should probably only be updated when a refresh actually executes, so that the throttle behaves as "at least one refresh every 2s" rather than "no refresh at all once requests arrive faster than every 2s."
Screenshots
N/A (metrics output only; see reproduction steps above for the observed raw output pattern).
Environment
Linux distribution version: Ubuntu 22.04
Scaphandre version: 1.0.2
Additional context: -
Bug description
The Prometheus exporter's internal topology refresh is throttled to at most once every 2 seconds (hardcoded in
src/exporters/prometheus.rs,show_metrics()), but the timestamp used for that throttle (last_request) is updated unconditionally on every request, not only when a refresh actually happens:If /metrics is scraped more frequently than every 2 seconds (e.g. a 1s Prometheus scrape_interval, or a tight polling loop), now - *last_request never exceeds 2s, so topology.refresh() effectively never runs again after the first successful call. All derived metrics (scaph_socket_energy_microjoules, scaph_socket_power_microwatts, scaph_process_power_consumption_microwatts, etc.) get stuck at whatever was captured on the last refresh that did occur, sometimes for well over a minute, producing a "staircase" pattern instead of a live per-scrape update.
This is easy to miss because a single scrape in isolation looks fine (a plausible, monotonically increasing counter value) — the problem only becomes visible when comparing consecutive scrapes and noticing long runs of bit-identical values.
To Reproduce
Run scaphandre with the prometheus exporter: scaphandre prometheus (tested both as a native binary and via the hubblo/scaphandre:1.0.2 Docker image).
Scrape /metrics repeatedly at an interval shorter than 2 seconds, e.g.:
while true; do curl -s localhost:8080/metrics | grep scaph_socket_energy_microjoules; sleep 1; done
Observe that scaph_socket_energy_microjoules (and other derived metrics) return the exact same value for many consecutive scrapes in a row, only updating sporadically — sometimes after 15-20+ seconds — rather than approximately every two seconds.
Expected behavior
Either:
The exporter should refresh on (approximately) every scrape regardless of interval, so metrics reflect current state each time /metrics is queried, or
If throttling is intentional (to avoid excessive sysfs/process-tree churn on very frequent scraping), this should be clearly documented, and ideally configurable via a CLI flag (there is currently no --step/-s equivalent for the prometheus exporter), so users scraping faster than the hardcoded 2s floor are aware their data will be coarser than their scrape interval implies.
At minimum, last_request should probably only be updated when a refresh actually executes, so that the throttle behaves as "at least one refresh every 2s" rather than "no refresh at all once requests arrive faster than every 2s."
Screenshots
N/A (metrics output only; see reproduction steps above for the observed raw output pattern).
Environment
Linux distribution version: Ubuntu 22.04
Scaphandre version: 1.0.2
Additional context: -