Skip to content

Latest commit

 

History

History
426 lines (325 loc) · 18.8 KB

File metadata and controls

426 lines (325 loc) · 18.8 KB

Remote Access — Reaching Devices Without Direct Connectivity

Secure Cartography assumes the host running it can open SSH (TCP/22) and SNMP (UDP/161) straight to every target. When that isn't true, there are two independent mechanisms, and they solve different halves of the problem:

SSH jump host SNMP proxy
Transport SSH direct-tcpip channel (TCP only) HTTP to a remote agent that does the SNMP locally
Reaches SSH collection SNMP collection
Used by Discovery engine — CLI crawl / device, and the GUI discovery run Device Poll dialog in the map viewer only
Config ~/.scng/jump_hosts.yaml ~/.scng/proxy_settings.json
Enabled by --jump-config, or the Connection panel field "Use SNMP Proxy" checkbox in the poll dialog
Credentials Resolved from the vault by name Community string sent per request
Server-side install None (any SSH bastion) snmp_proxy package must run on the remote host

They are not alternatives to each other. The jump host cannot carry SNMP, and the proxy is not wired into the discovery crawl. If you need both, see Running both on one host.


Option 1 — SSH via jump host (bastion)

What it does and does not reach

A jump host is a direct-tcpip channel opened on the bastion's SSH transport. That channel carries TCP only. SNMP is UDP and cannot traverse it. A device reached through a bastion is therefore collected over the SSH path alone, which means:

  • ✅ neighbors (CDP/LLDP), hostname, vendor, platform/version string
  • ❌ no sysObjectID, no ENTITY-MIB inventory (model/serial), no interface table, no ARP

Bastion-reached devices come back as thin records. Use the jump path for topology and for devices that are otherwise unreachable; run discovery from a host with direct SNMP reachability when you need full inventory.

Prerequisites

Requires Secure Cartography 2.5.6 or later. Jump-host config is accepted by 2.5.5, but a bug in the credential probe meant it never actually engaged — see the Appendix if you're pinned to an older release and need to know what you'd see.

Jump-host support comes from reachssh, which is a hard dependency of SC2 — sc2/scng/discovery/ssh/client.py is a thin re-export over it. Nothing extra to install.

The bastion's credential must exist in the SC2 vault as an SSH credential (vault.get_ssh_credential() is what the resolver calls):

sc2-creds add ssh bastion-lab --username netops --port 22 \
    --description "lab bastion"
# or with a key
sc2-creds add ssh bastion-lab --username netops --key-file ~/.ssh/id_lab

The credential is looked up lazily and cached once per jump host, not once per device.

Config file

Default path ~/.scng/jump_hosts.yaml. Two top-level blocks:

jump_hosts:
  lab-bastion:
    host: 192.0.2.10
    port: 22                     # optional, defaults to 22
    credential: bastion-lab      # name of an SSH credential in the vault
  dmz-jump:
    host: 198.51.100.5
    credential: bastion-dmz

proxy_rules:                                  # top to bottom, FIRST MATCH WINS
  - match: {devices: [lab-bastion]}           # the bastion itself, never via itself
    jump: direct
  - match: {devices: ["fw*-dmz"]}             # exception — must sit above the broad rule
    jump: dmz-jump
  - match: {platform: [arista]}
    jump: lab-bastion
  - match: {}                                 # catch-all
    jump: lab-bastion

jump_hosts — a registry of named bastions. host is required; port defaults to 22. credential is a vault credential name, not a secret. reachssh itself allows credential to be omitted (meaning "resolver default"), but SC2's resolver logs an error and fails the hop in that case, so always name it explicitly.

proxy_rules — an ordered list. Evaluated top to bottom, first match wins. Within a single rule, every key present must match (logical AND). A rule with an empty match: {} is an explicit catch-all. jump: direct short-circuits to a direct connection, and so does falling off the end of the list with no match.

This is route-map / ACL ordering, not implicit specificity ranking. A device that is both "in the lab" and "special" is resolved by position — put the narrow rule above the broad one.

Match vocabulary

Key Matches against Form
devices device name exact or fnmatch glob; scalar or list
name_regex device name re.search — anchor it yourself if you want anchoring
platform discovered vendor, lowercased (cisco, arista, juniper, …) scalar or list
site site_slug scalar or list
role role_slug scalar or list

Any other key is a config error and raises at load.

⚠️ site and role never match under SC2. SC2 has no CMDB, so device_context() in sc2/scng/discovery/jump.py leaves site_slug and role_slug unset. Only devices, name_regex, and platform are useful here. Naming conventions usually carry site information, so name_regex covers the site case:

  - match: {name_regex: "^eng-"}
    jump: lab-bastion

Note also that name falls back to the IP when no hostname is known yet — at seed time, before the device identifies itself, a devices: glob written against hostnames will not match. Match seeds on IP or use a catch-all.

Multi-hop

The config model accepts a chain (jump: [hop-a, hop-b]), but the SSH client implements single-hop today and raises an explicit error for more than one hop. Use one bastion per rule. direct cannot be combined with named hops.

Enabling it

CLI--jump-config on crawl, device, and test:

sc2-discover crawl 192.0.2.21 192.0.2.22 \
    --depth 3 \
    --domain local.lab \
    --jump-config ~/.scng/jump_hosts.yaml \
    -o ~/network_maps/lab \
    -v

If ~/.scng/jump_hosts.yaml exists, it is loaded even without the flag — the default path is checked at engine startup. Use --jump-config to point at an alternate file.

GUI — Connection panel → JUMP HOST CONFIG, with a browse button. The field prefills with ~/.scng/jump_hosts.yaml when that file exists, so an already-configured bastion is visible rather than silently applied. The hint line under the field tells you what the current setting will do:

Hint Meaning
Direct connections - no bastion field empty
⚠ File not found - discovery will fail to start path set but missing
Routes SSH via bastion (SNMP does not tunnel) loaded

Verifying

Run with -v and watch the startup line and the per-device lines:

Jump-host routing loaded from /home/you/.scng/jump_hosts.yaml (SSH only - SNMP does not tunnel)
...
Reached via jump host lab-bastion
Direct SSH connection (no jump host matched)

The resolver validates at construction: every jump name referenced by a rule must exist in jump_hosts. A typo fails loudly at startup rather than silently degrading a bastion-only device into a failing direct connect.

Failure modes

Symptom Cause
Jump-host configuration error: ... at startup, discovery never runs Malformed YAML, unknown match key, bad regex, or a rule referencing an undefined jump host. Deliberate — a silent fallback to direct would look like a working run that quietly skipped every bastion-only device.
Jump credential 'x' not found in vault The credential: name doesn't exist, or is not an SSH-type credential. sc2-creds list --type ssh.
Jump host needs credential 'x' but no vault is open Vault locked. Unlock before discovery.
Jump host has no 'credential' set credential: omitted from the jump_hosts entry.
Multi-hop jump chains are not yet supported A rule's jump is a list with more than one name.
Device reached, but record has no interfaces/ARP/serial Expected. SNMP didn't tunnel. Not a bug.
Everything connects directly despite config No rule matched. Add a catch-all match: {}, and remember site/role never match.
Resolver loads, logs "Jump-host routing active", then discovery finds 0 devices with no further output Fixed in 2.5.6. On 2.5.5 and earlier the SSH credential probe dialed the device directly instead of through the bastion, so every credential failed and discovery gave up before the collector ran. Upgrade — see Appendix.

Security note

reachssh uses paramiko.AutoAddPolicy() for both the bastion and the device connection — host keys are accepted on first sight and not verified. This matches how SC2's original SSH client behaved, but it means the jump path gives you reachability, not authentication of the bastion. Deploy accordingly.


Option 2 — SNMP proxy

What it does

A small FastAPI service (snmp_proxy, MIT, versioned separately at 2.0.0) runs on a host that does have SNMP reachability. The desktop client submits a poll job over HTTP, receives a ticket, and polls for status until results are ready.

┌─────────────┐         ┌─────────────┐         ┌─────────────┐
│   SC2 GUI   │  HTTP   │  Proxy host │  SNMP   │  lab devices│
│  (desktop)  │◄───────►│             │◄───────►│             │
└─────────────┘  :8899  └─────────────┘  :161   └─────────────┘

Scope: the proxy is consumed by PollWorker (sc2/ui/widgets/poll_worker.py), which is driven by the Device Poll dialog — right-click a node in the map viewer → Poll Device. It is not wired into the discovery crawl. Discovery still needs direct SNMP, or falls back to the SSH path (see Option 1).

Server setup

pip install -e ./snmp_proxy
# or: pip install fastapi uvicorn pysnmp pydantic
python -m snmp_proxy

Requires Python 3.10+ and pysnmp >= 6.0. Pure Python, so it runs on Linux, macOS, and Windows without net-snmp binaries.

On startup it prints a banner with the API key, bind address, and effective config.

Flag Default Notes
--host 0.0.0.0 Bind address
--port 8899
--max-concurrent 6 Semaphore limit; jobs queue past it
--retention-hours 1 How long completed jobs are kept
--get-timeout 10 Seconds, default SNMP GET
--walk-timeout 120 Seconds; raise for chassis with 500+ interfaces
--api-key (generated) Fixed key instead of a fresh UUID4
--log-level INFO

The API key matters more than it looks

By default the key is a fresh UUID4 generated on every startup and never stored. That is fine for an ad-hoc run, but it means every proxy restart silently invalidates the key saved in the GUI, and the next poll fails with:

Proxy authentication failed - check API key

For anything long-lived — a systemd unit, an NSSM service — pass --api-key with a stable value so client config survives restarts. /health is unauthenticated; every other endpoint requires the X-API-Key header.

Deployment recipes for systemd, NSSM, and Docker are in snmp_proxy/README.md, along with the full endpoint reference.

Client setup

Open any node's Device Poll dialog, then in Proxy Settings:

Field Value
Use SNMP Proxy check to enable
Proxy URL http://192.0.2.10:8899 — scheme and port required, trailing slash stripped
API Key the UUID from the proxy console

Click 🔗 Test Proxy — it hits /health unauthenticated and reports version and active/max job counts:

✓ Proxy OK - v2.0.0, 0/6 active jobs

Note that the test only proves the proxy is reachable. It does not validate the API key — a wrong key surfaces on the first real poll as a 401.

Settings persist to ~/.scng/proxy_settings.json:

{
  "enabled": true,
  "url": "http://192.0.2.10:8899",
  "api_key": "d0f0d9e6-36ae-489d-ba96-29da77e529cc"
}

⚠️ Unlike device credentials, the proxy API key is not vaulted — it sits in cleartext in that file. Treat it as a short-lived session token, restrict file permissions, and prefer a rotating key over a permanent one where the proxy host is shared.

TLS

Nothing in the proxy terminates TLS, and the client sends plain HTTP by default. If you front it with a reverse proxy and set an https:// URL, the client's urllib calls use Python's default certificate verification — the SSL-bypass context in the codebase applies only to the Recog/OUI data-file download, not to proxy traffic. A self-signed lab certificate will therefore fail with a certificate error.

Options, in order of preference:

  1. Keep the URL as http:// and forward the port over SSH instead: ssh -L 8899:127.0.0.1:8899 netops@192.0.2.10, then point the client at http://localhost:8899. Gets you an encrypted path with no cert work.
  2. Issue the proxy a certificate from a CA the client trusts.
  3. Point Python at a custom CA bundle for the SC2 process: export SSL_CERT_FILE=/path/to/lab-ca.pem.

There is no "skip verification" toggle in the UI; option 1 is the intended lab path.

Version support

The dialog's version selector offers v1 and v2c only. The proxy API and PollWorker both handle SNMPv3 — the v3_auth block is accepted with MD5/SHA/SHA224/SHA256/SHA384/ SHA512 auth and DES/3DES/AES/AES128/AES192/AES256 privacy — but the dialog does not expose those fields. Drive v3 polls against the API directly for now.

Failure modes

Symptom Cause
Cannot connect to proxy: <reason> URL wrong, service not running, or firewall between client and proxy host
Proxy authentication failed - check API key (401) Key mismatch — usually the proxy restarted and regenerated it
Proxy access denied - invalid API key (403) As above
Proxy health check failed Service answered but reported a non-ok status
Poll times out on a large chassis Raise --walk-timeout; watch progress counters (if:, arp:) in the status line
No SNMP response The proxy host, not your desktop, can't reach the device. Verify from the proxy host.
pysnmp import errors on the proxy host Needs pysnmp ≥ 6.0 — the API renamed getCmdget_cmd and UdpTransportTarget is now created via await ... .create()

Cancellation is wired end to end: ⛔ Cancel issues DELETE /jobs/{ticket} so the walk stops on the proxy rather than just being abandoned client-side.


Running both on one host

The two mechanisms compose well on the same bastion, and this is the configuration that gets you back to full-fidelity records in an isolated environment:

  1. Install and run snmp_proxy on the bastion, bound to localhost: python -m snmp_proxy --host 127.0.0.1 --api-key <stable-key>
  2. Forward it to your workstation: ssh -L 8899:127.0.0.1:8899 netops@192.0.2.10
  3. Point the Device Poll dialog at http://localhost:8899
  4. Point discovery at ~/.scng/jump_hosts.yaml with that same bastion

Discovery then builds topology over SSH through the bastion, and per-device SNMP enrichment — inventory, interfaces, ARP — comes back through the proxy on demand. The gap that remains is that crawl-time SNMP still does not run for bastion-reached devices; the proxy fills it node by node from the map viewer, not automatically.


Reference

Config file locations

Path Written by Contains
~/.scng/jump_hosts.yaml you, by hand Bastion registry and routing rules
~/.scng/proxy_settings.json GUI, on toggle and on poll Proxy URL, API key, enabled flag
vault sc2-creds Bastion SSH credentials, referenced by name

A note on CGNAT-addressed targets

Devices reached through a bastion frequently sit on carrier-grade NAT space (100.64.0.0/10). If a seed or a discovered neighbor address falls in that range, treat it as ambiguous rather than authoritative — the same address may exist in more than one place, and neighbor tables often carry it verbatim. Resolve the device's name to an address explicitly (a plain socket.gethostbyname() on the hostname is enough) and seed with that, or supply a --hosts-file mapping so name→IP resolution is deterministic before DNS is consulted.

Source map

Concern File
Jump config load, vault bridge, device match context sc2/scng/discovery/jump.py
Rule parsing and resolution reachssh/proxy.py (build_proxy_resolver, ProxyResolver)
direct-tcpip channel setup reachssh/client.py
Engine wiring, startup validation sc2/scng/discovery/engine.py
CLI flag sc2/scng/discovery/cli.py (--jump-config)
GUI field sc2/ui/widgets/connection_panel.py
Proxy client, ticket loop, cancellation sc2/ui/widgets/poll_worker.py
Proxy settings persistence and dialog sc2/ui/widgets/device_poll_dialog.py
Proxy service snmp_proxy/src/snmp_proxy/

Appendix — the 2.5.6 credential probe fix

Fixed in 2.5.6. This section is here for anyone running an older release, or reading engine.py and wondering why the probe carries a jump spec.

Before discovery collects anything, _get_working_credential() probes the device to pick a working credential. Through 2.5.5 the SSH branch of that probe built its own SSHClientConfig without setting jump=, so it always connected directly.

For a bastion-only device the consequence was a silent dead end: every SNMP credential timed out (SNMP does not tunnel), every SSH credential failed the direct probe, no credential was selected, and discover_device() returned a failed Device without ever calling _discover_via_ssh() — the only place the jump resolver is handed to SSHCollector. The resolver loaded, logged that routing was active, and was never consulted. Three swallowed exceptions on that path meant verbose mode said nothing at all about it.

2.5.6 routes the probe through the same resolver the collector uses, threads the device name down so devices: and name_regex: rules match identically in both places, and logs the probe, selection, and terminal failures. Verbose output on success:

  [discovery] Discovering eng-rtr-1 (192.0.2.21)
      [discovery] Probing 192.0.2.21 via jump host lab-bastion(192.0.2.10:22)
    [discovery] SSH credential 'lab-devices' works for 192.0.2.21
  [discovery] SSH fallback for eng-rtr-1 (192.0.2.21)
    [discovery] Reached via jump host lab-bastion(192.0.2.10:22)

and on failure:

      [discovery] SSH probe failed for 192.0.2.21 via lab-bastion(192.0.2.10:22)
                  as 'netops': AuthenticationException: Authentication failed.
    [discovery] No credential succeeded for 192.0.2.21 (tried 3: snmp-lab, lab-devices, bastion-lab)
  [discovery] FAILED eng-rtr-1 (192.0.2.21): no working SNMP or SSH credential
              - jump-host routing is active, check that a rule matches this device