feat(connectors): failure modes as a declared capability

[LGDiMaggio]

Summary

Failure-mode catalogs are now a first-class, declared connector capability. Before this PR, the runtime harvested diagnosis content by probing every connector for a private _failure_modes list — only the Generic CMMS demo connector exposed one, so diagnose_failure worked on the YAML demo and silently degraded to an empty catalog on SQL or Excel deployments. Now any substrate can feed the catalog by declaring Capability.READ_FAILURE_MODES and serving its public read_failure_modes(); the runtime discovers providers through the registry, exactly like every other capability.

The same plant data can now also declare which failure codes apply to which asset (failure_modes column/cell on the asset source), so diagnosis filters to the asset's declared modes instead of matching the whole catalog.

How it works

Substrate	Catalog source	Capability declared when
Generic CMMS	failure_modes.json in the data dir	the file exists (local mode only — REST never declares it until a REST fetch exists)
SQL	any table/view mapped with entity: FailureMode	the mapping is configured
Excel/CSV	optional failure_modes sheet schema	the sheet is configured

The harvest (_collect_failure_modes) fans out to all providers concurrently, dedups by code (first registration wins), and treats a provider that raises ConnectorError as contributing nothing — so a not-connected or failing source degrades diagnosis honestly instead of aborting the agent. Both consumers (the workflow FailureAnalyzer and the diagnose_failure tool) read this single source.

Multi-valued cells share one committed encoding across substrates: a semicolon-delimited string ("BEAR-WEAR-01;SEAL-LEAK-01"), implemented once in connectors/_entity_builders.py alongside the existing shared dict→entity builders.

Key decisions

• Conditional declaration: a connector declares the capability only when an actual source is configured, so capability discovery is a true signal of "has a catalog" — not "running in local mode". The Generic CMMS gate is snapshotted at construction/connect rather than stat-ing the file per capability access (capability and loaded catalog can never disagree mid-session).
• ConnectorError is the degradation contract: the harvest skips providers that raise it; anything else is a connector bug and surfaces loudly. The SQL connector was hardened to honor this — un-connected reads raise ConnectorError (was a raw AttributeError), non-transient driver errors are wrapped, and an invalid catalog row raises ConnectorSchemaError with row context instead of leaking a pydantic ValidationError that aborted agent.start().
• Async cascade: _build_domain_services and _tool_diagnose_failure became async with the harvest. The old "works on un-started agents" claim is gone — the public read method requires a connection, and the harvest degrades to an honest empty catalog when a provider isn't connected.
• No new MCP tool: READ_FAILURE_MODES is mapped to an explicit empty list in CAPABILITY_TO_TOOL per the existing gap-visibility convention (same as Calendar).
• Behavior preserved: the demo loads the same 10 modes and produces identical diagnoses (e2e-verified); the empty-catalog note ("No failure-mode data configured on any connector.") is locked by regression tests; an integration suite proves the CSV substrate and the JSON demo substrate yield the same catalog and the same diagnosis, field-for-field.

Migration note (third-party connectors): a connector that exposed _failure_modes without declaring the capability silently stops contributing — declare READ_FAILURE_MODES and implement async read_failure_modes(). Recorded in the CHANGELOG.

Known Residuals

Accepted from the multi-agent review (none introduced by this PR; recorded for follow-up):

• P2 — no timeout layer on connector reads (src/machina/agent/runtime.py:881): a hung DB connection blocks agent.start() and diagnose_failure until the driver gives up. Pre-existing property of every connector read path (asset loads included); fixing it only for the harvest would be inconsistent. Needs a cross-cutting timeout decision.
• P3 — outage vs. absence in the diagnosis note (src/machina/agent/runtime.py:2878): when all providers error, diagnosis reports "No failure-mode data configured" — same note as a genuinely unconfigured system. Distinguishing "configured but currently unavailable" changes the R9 note contract; deliberately deferred.
• P3 — SQL connection shared across threads without a lock (src/machina/connectors/sql/generic.py:_read_sync): concurrent diagnose calls fan asyncio.to_thread reads onto one pyodbc connection (threadsafety=1). Pre-existing for all SQL reads; the harvest raises the frequency.
• P3 — MCP failure-mode surface gap: no machina_diagnose_failure MCP tool exists, and the static machina://v1/failure-taxonomy resource diverges from connector catalogs. Follows the documented Calendar convention; pair the tool with the capability when the MCP diagnosis surface is built.

Post-Deploy Monitoring & Validation

Library change — no deployment infrastructure. For operators upgrading:

• Log signals: failure_mode_harvest_failed (WARNING, with connector= and error=) means a declared provider was skipped — expected for transient outages, a misconfiguration signal if persistent. loaded_failure_modes (DEBUG, count=) and domain_services_ready (failure_modes=) confirm the catalog loaded at start.
• Healthy signal: diagnose_failure returns probable_failures with no note; the demo agent reports 10 failure modes at start.
• Failure signal / rollback: diagnosis suddenly returning the empty-catalog note on a previously-working deployment → check the provider's source gate (file present? mapping configured?) before rolling back; the capability is intentionally undeclared when the source is absent.
• Validation: PYTHONPATH=src python -m pytest tests/integration/test_failure_modes_substrates.py proves substrate-agnostic harvest on the shipped sample data.

Test plan

• 2319 tests pass (pytest tests/), ruff and mypy (strict) clean.
• New coverage: capability-gated harvest semantics (dedup first-wins, no-provider start, empty/raising/buggy providers, single-source workflow↔diagnose), conditional declaration per substrate (incl. REST-mode never-declared and file-absent cases), asset↔code linkage and the shared encoding (incl. NULL→'None' regression), SQL ConnectorError contract (un-connected, invalid row), Excel connect-gating, watcher refresh reload + all-or-nothing atomicity, unknown-coercer validation, and CSV↔JSON substrate equivalence (catalog + diagnosis, field-level).
• Multi-agent review (10 reviewers): 10 findings fixed in 285d295, residuals listed above.

---