Skip to content

feat!: Global command-line output contract (ADR-T-010)#869

Merged
da2ce7 merged 13 commits into
torrust:developfrom
peer-cat:20260513_json_output
May 15, 2026
Merged

feat!: Global command-line output contract (ADR-T-010)#869
da2ce7 merged 13 commits into
torrust:developfrom
peer-cat:20260513_json_output

Conversation

@peer-cat
Copy link
Copy Markdown
Contributor

@peer-cat peer-cat commented May 14, 2026

Summary

Lands ADR-T-010 end to end: every shipped, documented, or operator-facing first-party Torrust Index command-line entrypoint now follows one JSON-only stream contract.

Stdout is reserved for machine-readable result data. Stderr carries machine-readable diagnostics and control records as JSON/NDJSON. Commands that emit stdout result data refuse direct terminal stdout, so automation cannot accidentally mix data streams with human-facing diagnostics.

Breaking for operators and scripts. Any automation that scraped previous plain-text output from helper binaries, root maintenance commands, parse_torrent, server logs, or container startup must switch to exit codes plus JSON/NDJSON parsing.

Why

ADR-T-009 introduced strict stream rules for container helper binaries, but the underlying problem was repository-wide: every CLI entrypoint had its own stdout/stderr behavior.

That made shell integration brittle:

  • helper JSON could be corrupted by plain diagnostics;
  • clap help/errors and Rust panic output could still leak raw text;
  • server and maintenance logs were not consistently machine-readable;
  • parse_torrent and maintenance tools mixed operator text with command results;
  • container startup used plain text and DEBUG=1 shell tracing instead of structured records;
  • command-reachable libraries could still print or exit outside the command boundary.

This PR promotes the helper contract into a global CLI contract and migrates the existing command surface to match it.

Highlights

  • ADR-T-010 becomes canonical. adr/010-global-command-line-output-contract.md now records the completed repository-wide contract, command classifications, shared schema, implementation summary, redaction policy, and guards.
  • Shared CLI infrastructure. torrust-index-cli-common now owns JSON clap help/version/usage handling, stderr control-plane records, baseline exit classes, JSON panic diagnostics, JSON tracing setup, TTY refusal, stdout JSON emission, command runners, locked stderr writes, and redaction helpers.
  • Helper binaries migrated. torrust-index-auth-keypair, torrust-index-config-probe, and torrust-index-health-check now share the same JSON control plane. Successful stdout payloads are single JSON objects with a top-level schema field; help, version, argv errors, TTY refusal, panic diagnostics, and tracing go to stderr as JSON.
  • Server logging boundary migrated. torrust-index now emits JSON tracing records on stderr. The configured logging threshold remains the default filter, while non-empty RUST_LOG takes precedence. Binary entrypoints return explicit ExitCode values instead of relying on Rust default termination.
  • Torrent helper commands migrated. parse_torrent is now a stdout-result command emitting schema, torrent, original_v1_info_hash, and input_byte_length. create_test_torrent is a no-stdout side-effect command with JSON stderr diagnostics.
  • Maintenance commands migrated. import_tracker_statistics, seeder, and upgrade now keep stdout empty and emit help, usage errors, status, diagnostics, tracing, and panic records as JSON/NDJSON on stderr.
  • Upgrade path hardened. The v1-to-v2 upgrader now propagates typed errors for setup, migrations, transfer validation, torrent decoding, missing fields, and timestamp conversion instead of relying on plain output, color formatting, unwraps, or panic-driven failures.
  • Command-reachable libraries cleaned up. Shutdown notices use structured tracing, mail template failures are returned to callers, terminal color formatting is removed from command paths, and parsing helpers leave reporting to their command callers.
  • Container entry script migrated. Startup validation failures, status notices, expected utility failures, jq failures, unexpected shell exits, and DEBUG=1 phase records now emit JSON/NDJSON on stderr. Helper stdout stays internal to command substitutions.
  • Regression guards added. Workspace Clippy now denies accidental raw print_stdout, print_stderr, and direct exit usage outside approved boundaries, and tests/cli_contract.rs keeps in-scope binaries returning ExitCode.

Breaking Changes

Surface Before After
CLI stream contract Per-command stdout/stderr behavior ADR-T-010 JSON-only contract across first-party entrypoints
Server logs Human-formatted tracing output JSON records on stderr
Helper stdout schemas Some payloads lacked explicit schema versioning Successful helper payloads include top-level schema
Helper help/errors/panics Plain clap or Rust panic text could reach stderr JSON control-plane records on stderr
parse_torrent Legacy plain-text CLI behavior Single JSON stdout result; refuses direct terminal stdout
create_test_torrent Plain status/diagnostic output Empty stdout; JSON/NDJSON stderr
import_tracker_statistics, seeder, upgrade Plain status/errors and panic/unwrap paths Empty stdout; JSON/NDJSON stderr; explicit exit codes
Container entry script Plain startup text and DEBUG=1 / set -x tracing JSON/NDJSON stderr records and explicit debug phase records
Command libraries Some direct printing/exiting/color formatting Structured diagnostics returned or logged through the command boundary

Operator Migration

Stdout-producing commands should be piped or redirected before inspection:

torrust-index-auth-keypair | jq .
torrust-index-config-probe | jq .
torrust-index-health-check http://127.0.0.1:3001/health_check | jq .
cargo run --quiet --bin parse_torrent -- "$fixture" | jq .'

Credits

Thank-you for @da2ce7 for writing the ADR 10, and starting the work.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 14, 2026

Codecov Report

❌ Patch coverage is 53.53626% with 519 lines in your changes missing coverage. Please review.
✅ Project coverage is 68.53%. Comparing base (70ba64c) to head (5a6dab0).

Files with missing lines Patch % Lines
packages/index-cli-common/src/lib.rs 51.85% 177 Missing and 5 partials ⚠️
src/console/commands/seeder/api.rs 0.00% 48 Missing ⚠️
..._0_0_to_v2_0_0/transferrers/torrent_transferrer.rs 62.76% 31 Missing and 4 partials ⚠️
src/console/commands/seeder/app.rs 0.00% 30 Missing ⚠️
...rc/console/cronjobs/tracker_statistics_importer.rs 18.75% 25 Missing and 1 partial ⚠️
src/bin/create_test_torrent.rs 76.71% 13 Missing and 4 partials ⚠️
src/main.rs 0.00% 17 Missing ⚠️
src/bin/parse_torrent.rs 80.28% 13 Missing and 1 partial ⚠️
...auth-keypair/src/bin/torrust-index-auth-keypair.rs 68.29% 13 Missing ⚠️
...0_0_to_v2_0_0/transferrers/category_transferrer.rs 69.04% 13 Missing ⚠️
... and 18 more
Additional details and impacted files
@@            Coverage Diff            @@
##           develop     #869    +/-   ##
=========================================
  Coverage    68.53%   68.53%            
=========================================
  Files          161      161            
  Lines        12394    13111   +717     
  Branches     12394    13111   +717     
=========================================
+ Hits          8494     8986   +492     
- Misses        3647     3850   +203     
- Partials       253      275    +22     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@peer-cat peer-cat force-pushed the 20260513_json_output branch 2 times, most recently from 09dbbe3 to 7309e62 Compare May 14, 2026 16:57
da2ce7 and others added 7 commits May 14, 2026 18:58
Add ADR-T-010 as the canonical repository-wide stdout/stderr
contract for first-party command-line entrypoints. This extracts
the helper-binary P8/P9 rules from ADR-T-009 into a global
JSON-only stream contract: stdout is reserved for result data,
stderr carries diagnostics/control records, and stdout-producing
commands refuse direct TTY output.

- Add the ADR-T-010 decision record and a conformance plan covering
  command classification, shared CLI infrastructure, logging,
  panic/help/usage handling, root binaries, helper binaries, the
  container entry script, docs, and tests.
- Update the changelog, README, ADR-T-007, and ADR-T-009 so the
  helper stdout/stderr rules now point at ADR-T-010 while ADR-T-009
  remains the container-infrastructure history.
- Refresh rustdoc and comments in the helper/config/JWT code paths
  to name ADR-T-010 instead of the older P8/P9 phase shorthand.

No runtime behaviour changes; this records the global contract and
the follow-up migration plan.
Add shared CLI output primitives to torrust-index-cli-common, including
control-plane record types, baseline exit-code classes, and diagnostic
redaction helpers for sensitive fields and database URLs.

Version helper stdout result schemas with top-level schema fields for the
auth-keypair, config-probe, and health-check contracts, and add coverage for
the new shared records and schema-bearing helper outputs.

Document the ADR-T-010 migration state for operators, including the helper
JSON stdout contract, TTY-refusal expectations, and legacy root maintenance
commands that still need follow-up migration.

BREAKING CHANGE: helper stdout JSON schemas now include top-level schema
fields, and first-party command-line output is governed by the ADR-T-010
JSON-only stdout/stderr contract.
Extend torrust-index-cli-common with the shared helper infrastructure for JSON
clap help/version/usage records, direct stderr control-plane emission, JSON-only
panic diagnostics, RUST_LOG/--debug tracing precedence, locked stderr writes, and
stdout/no-stdout command runners.

Wire auth-keypair, config-probe, and health-check through the shared helpers so
their help, version, argv errors, TTY refusal, and panic diagnostics are emitted
as stderr control-plane JSON while stdout remains reserved for successful result
JSON.

Update ADRs, operator docs, and the changelog for the helper rollout, and expand
cli-common coverage for parser wrapping, tracing filter resolution, and JSON line
emission.

BREAKING CHANGE: container helper help, version, argv errors, TTY refusal, and
panic diagnostics now emit JSON control-plane records on stderr instead of legacy
plain-text output paths.
Route root application logging through the shared ADR-T-010 JSON stderr
tracing setup, using the configured logging threshold as the default filter
while preserving non-empty RUST_LOG as the override.

Install the shared JSON panic hook and explicit CommandExit mappings at the
root server and maintenance-binary entrypoints, and convert the server task
boundary from panic/expect paths into JSON-logged failures with explicit exit
codes.

Update the changelog, README, container docs, and ADR-T-010 rollout plan to
document the server JSON stderr contract, the root ExitCode boundary state, and
the remaining legacy maintenance-command internals.

BREAKING CHANGE: the torrust-index server now emits application logs as JSON
records on stderr instead of human-formatted tracing output.
Move parse_torrent and create_test_torrent from hand-rolled argv handling and
plain-text diagnostics onto the shared ADR-T-010 CLI helpers for JSON clap
records, JSON panic reporting, and stdout/stderr command runners.

Make parse_torrent emit a schema-versioned JSON stdout result with the decoded
torrent, original v1 info hash, and input byte length, while routing failures
and terminal-stdout refusal through JSON stderr control-plane records.

Make create_test_torrent a no-stdout side-effect command that writes the torrent
file to the requested directory, reports the generated path through JSON stderr
tracing, and converts encode and file I/O failures into explicit diagnostics.

Remove stray parse_torrent utility print paths, add focused CLI contract tests,
and update the README, changelog, and ADR-T-010 rollout plan for the stage-five
root command migration.

BREAKING CHANGE: parse_torrent and create_test_torrent now follow the ADR-T-010
JSON stdout/stderr contracts instead of their legacy plain-text CLI behavior.
Move import_tracker_statistics, seeder, and upgrade onto the shared ADR-T-010
CLI boundary for JSON clap help/version/usage records, JSON panic reporting,
JSON stderr tracing, and explicit exit-code handling for no-stdout side-effect
commands.

Add an async no-stdout command runner in torrust-index-cli-common, wire the
maintenance binaries through it, and convert command-reachable seeder, tracker
statistics importer, and v1.0.0-to-v2.0.0 upgrade paths away from plain-text
prints, terminal color formatting, unwraps, and panic-driven failures.

Introduce typed upgrade errors for database setup, migrations, transfer
validation, torrent decoding, missing torrent fields, and timestamp conversion,
then propagate those failures through the upgrader and its tests.

Update the changelog, README, ADR-T-010 rollout plan, and upgrade guide to
document the migrated root maintenance command contract and remove the legacy
text-colorizer dependency.

BREAKING CHANGE: import_tracker_statistics, seeder, and upgrade now keep stdout
empty and emit help, usage errors, status, diagnostics, tracing, and panic
records as JSON/NDJSON on stderr instead of their previous plain-text output.
Route command-reachable server library diagnostics through the JSON logging
boundary instead of direct stream output. Shutdown grace-period notices now use
structured tracing, and mail template initialization and render failures now
return typed errors for caller-side JSON diagnostic reporting.

Document the stage-seven ADR-T-010 status across the README, container docs,
crate docs, changelog, and rollout plan.

BREAKING CHANGE: command-reachable shared libraries no longer print mailer or
shutdown diagnostics directly, and mail-template initialization failures are
reported through callers instead of exiting in the mailer library.
peer-cat added 4 commits May 14, 2026 19:16
Move the container entry script onto the ADR-T-010 no-stdout orchestration
contract. Startup validation failures, status notices, debug phase records,
utility failures, jq failures, and unexpected shell exits now emit JSON/NDJSON
control-plane records on stderr instead of plain text or shell trace output.

Add POSIX shell JSON emitters, checked utility wrappers, jq read/write helpers,
append helpers, phase tracking, and an unexpected-exit trap so controlled
failures capture stderr inside JSON fields while helper stdout stays internal to
command substitutions.

Update the entry-script host tests to parse stderr as JSON records, add shared
test assertions for the entry-script record shape, and document the stage-eight
container migration across the README, container docs, changelog, and ADR-T-010
rollout plan.

BREAKING CHANGE: the container entry script now emits JSON/NDJSON records on
stderr before su-exec and DEBUG=1 emits JSON phase diagnostics instead of
enabling set -x. Automation that scraped legacy startup text or shell tracing
must switch to exit codes and JSON stderr parsing.
Update the root maintenance and torrent-helper command docs to show the
current ADR-T-010 usage pattern: quiet cargo output, explicit argument
separators, NDJSON stderr capture, and jq inspection where applicable.

Refresh the v1.0.0-to-v2.0.0 upgrade guide and upgrader follow-up hint so
operators get the same stderr-capture command for upgrade and tracker
statistics import workflows.
Deny raw Rust stdout/stderr print macros and direct process exits at the
workspace Clippy boundary, with local exceptions for build-script protocol
output, developer examples, test diagnostics, and the shared CLI exit helper.

Add a cli_contract integration test that keeps in-scope binary main functions
returning ExitCode and rejects Result-based termination so future changes do not
reintroduce raw stderr output.

Update the ADR-T-010 rollout plan and changelog to record the stage-ten
conformance guards.
Promote ADR-T-010 into the canonical implemented command-line output contract.
Fold in the rollout plan's scope, command classifications, shared CLI
infrastructure summary, implementation status, and regression guards so the ADR
itself records the completed migration.

Refresh the changelog from stage-by-stage rollout notes into a completed
contract summary, and remove the now-obsolete standalone conformance plan.

Update ADR-T-009 to describe the helper and config-probe work as prerequisites
for the future entry-script rewrite.
Disable incremental compilation and dev/test debuginfo for the debug dependency
and nextest archive stages so the combined Cargo target/archive layer stays under
common builder storage limits.

Build the `torrust-index` debug runtime binary in a separate stage with those
profile overrides unset, then copy it into the debug image after the tested helper
artifacts from `test_debug`.
@da2ce7 da2ce7 marked this pull request as ready for review May 14, 2026 18:26
@da2ce7 da2ce7 requested review from Copilot, da2ce7 and josecelano May 14, 2026 18:26
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Lands ADR-T-010, replacing per-command stdout/stderr behavior with a global JSON-only CLI output contract across every first-party Torrust Index entrypoint. Helpers, server, maintenance commands, and the container entry script now emit machine-readable JSON/NDJSON on stderr while reserving stdout for typed result payloads. The upgrade path is refactored from panic/unwrap flows to typed UpgradeError propagation, and a regression test enforces ExitCode mains for in-scope binaries.

Changes:

  • Migrate all in-scope binaries (helpers, server, parse_torrent, create_test_torrent, import_tracker_statistics, seeder, upgrade) to JSON stdout/NDJSON stderr with explicit ExitCode returns.
  • Replace println!/eprintln!/unwrap/color-formatted output in command-reachable libraries (upgrade transferrers, shutdown signals, parse helpers) with structured tracing and typed errors.
  • Add CLI contract regression test plus operator-facing docs (upgrades/from_v1_0_0_to_v2_0_0/README.md, module docs) describing the new contract.

Reviewed changes

Copilot reviewed 73 out of 74 changed files in this pull request and generated no comments.

Show a summary per file
File Description
upgrades/from_v1_0_0_to_v2_0_0/README.md Update operator instructions for new -- argv separator and NDJSON stderr capture.
tests/upgrades/from_v1_0_0_to_v2_0_0/upgrader.rs Adapt test to new Result-returning upgrade() signature.
tests/upgrades/.../torrent_transferrer_tester.rs Handle now-fallible convert_timestamp_to_datetime via .expect.
tests/upgrades/.../sqlite_v1_0_0.rs Allow clippy::print_stdout for test scaffolding.
tests/e2e/.../torrent/{steps,contract}.rs, tests/common/contexts/torrent/fixtures.rs Allow clippy::print_stdout/print_stderr in e2e/test fixtures under new workspace lints.
tests/cli_contract.rs New regression test asserting in-scope binaries return ExitCode and not Result.
src/web/api/server/signals.rs Replace println! shutdown notice with structured info!; extract grace timeout constant.
src/utils/parse_torrent.rs Drop println!/eprintln! from decode_torrent/encode_torrent; leave reporting to callers.
src/upgrades/from_v1_0_0_to_v2_0_0/upgrader.rs Refactor run/upgrade to accept parsed Arguments and return Result<(), UpgradeError>; switch status output to tracing::info!.
src/upgrades/.../user_transferrer.rs Replace unwrap/assert!/println! with typed UpgradeError mapping and structured tracing.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@peer-cat
Copy link
Copy Markdown
Contributor Author

peer-cat commented May 14, 2026

ACK 5a6dab0

Comment thread packages/index-cli-common/src/lib.rs
Allow ADR-T-010 panic control-plane records to include string panic payloads
when debug diagnostics are enabled, while continuing to omit the payload field
for normal runs.

Wire the debug flag through the shared CLI runners and config probe, and add
coverage for the payload gate, JSON serialization, and string payload extraction.
Copy link
Copy Markdown
Contributor

@da2ce7 da2ce7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 5a6dab0

@da2ce7 da2ce7 merged commit 843aaff into torrust:develop May 15, 2026
18 of 19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants