Skip to content

Commit 91ae301

Browse files
mayankpande88mayankpande88
authored
docs: add a quickstart for running discovery without setup (#127)
* docs: add a quickstart for running discovery without setup The standalone subcommands are only useful if someone can find them. Shows the download, a sweep, and the pack sign plus inventory flow, with the safety notes that matter — scope is required, scanning looks like an attack to security tooling, and the rate is adjustable. Every command was run verbatim against the released rc.5 binary. The example MAC is synthetic; the first draft had my own router's. * docs: CLI reference for running discovery locally The README quickstart shows the shape; this is the reference someone actually works from — every flag with its default, real output for both commands, what each per-host status means, and a table of the failures people hit with what each usually is. The troubleshooting section is the part that took the longest to earn: pack key mismatch, version mismatch, missing MACs when targets are not on the same segment, and sshd logging 'Did not receive identification string' on a bare connect, which some fail2ban configs ban for and which then reads as a firewall problem. Also states plainly what it does not do — Windows, powered-off machines, Active Directory, and persisting anything — so nobody concludes those are broken rather than absent. Flags checked against the source: none undocumented, none invented.
1 parent af4cd25 commit 91ae301

3 files changed

Lines changed: 321 additions & 0 deletions

File tree

‎.gitignore‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,10 @@
11
# Binaries
22
bin/
3+
build/
4+
# `go build -o forager ./cmd` is the obvious thing to type when following
5+
# the CLI docs, and it drops a 60MB binary in the repo root.
6+
/forager
7+
/nudgebee-forager
38
*.exe
49
*.exe~
510
*.dll

‎README.md‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,69 @@ module by datasource ID, and returns responses. No inbound ports needed.
3636
See [docs/architecture.md](docs/architecture.md) for the full request
3737
flow and connection lifecycle.
3838

39+
## Try VM discovery without setting anything up
40+
41+
Discovery finds machines on a network and lists the packages installed on
42+
each, over SSH, with nothing installed on the machines themselves. You can
43+
run it straight from the binary — no account, no relay, no config file.
44+
45+
```bash
46+
# macOS (arm64); swap for your platform from the releases page
47+
curl -fsSL -o forager \
48+
https://github.com/nudgebee/forager/releases/download/v0.1.4-rc.5/nudgebee-forager-darwin-arm64
49+
chmod +x forager
50+
51+
# What is on this network?
52+
./forager sweep --cidr 192.168.1.0/24 --ports 22
53+
```
54+
55+
```json
56+
{
57+
"addresses_scanned": 254,
58+
"hosts": [
59+
{"ip": "192.168.1.50", "open_ports": [22], "mac": "aa:bb:cc:dd:ee:ff",
60+
"rdns": "web-01.lan", "sources": ["tcp", "arp"]}
61+
]
62+
}
63+
```
64+
65+
To list installed packages you need two things: a read-only SSH login on the
66+
target, and a signed content pack — the file that says which commands to run.
67+
Packs are signed so the agent never runs collection commands it cannot
68+
attribute, which is why there is no flag to skip verification.
69+
70+
```bash
71+
# Make a key and sign the example pack
72+
PRIV=$(./forager pack keygen) # public key is printed to stderr
73+
./forager pack sign docs/content-packs/linux-inventory-example.yaml --key "$PRIV"
74+
75+
# Collect from a host you can SSH into
76+
./forager inventory \
77+
--cidr 192.168.1.0/24 \
78+
--targets 192.168.1.50 \
79+
--user nudgebee-ro --key ~/.ssh/id_ed25519 \
80+
--pack docs/content-packs/linux-inventory-example.yaml \
81+
--pack-key <public key from keygen>
82+
```
83+
84+
Results go to stdout as JSON and logs to stderr, so output pipes into `jq`.
85+
`mac` appears only for hosts on the same network segment, and `rdns` only
86+
when reverse DNS resolves — a host missing either is normal, not an error.
87+
88+
Notes worth knowing:
89+
90+
- `--cidr` is required for both commands. It bounds what can be contacted, so
91+
a mistyped target is refused rather than reached.
92+
- Scanning a network looks like an attack to security tooling. On anything
93+
you do not own, tell whoever runs intrusion detection first.
94+
- `--rate-pps` defaults to 100 and can be lowered. Sweeps use ordinary TCP
95+
connections, not crafted packets.
96+
- On the target, `nudgebee-ro` needs only an SSH key and permission to run
97+
read-only commands. Nothing is written or changed.
98+
99+
Full flag reference, output shapes and troubleshooting are in
100+
[docs/cli.md](docs/cli.md). `./forager help` lists the commands.
101+
39102
## Install
40103

41104
### Linux
@@ -147,6 +210,9 @@ docker build -t forager . # local Docker image (with Oracle support)
147210

148211
## Documentation
149212

213+
- [Running discovery from the CLI](docs/cli.md) — sweep, inventory and
214+
content packs without a relay, with flags, output shapes and what the
215+
common failures mean.
150216
- [Architecture](docs/architecture.md) — overview and request flow.
151217
- [Configuration](docs/configuration.md) — config file, env vars, secret
152218
providers.

‎docs/cli.md‎

Lines changed: 250 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,250 @@
1+
# Running discovery from the command line
2+
3+
Discovery can be run directly from the binary, with no account, no relay and
4+
no config file. This is how to evaluate it, reproduce a customer's result on
5+
their own machine, or work on it from a clone of this repo.
6+
7+
These subcommands call the same code the agent calls, so what you see here is
8+
what the agent does.
9+
10+
## Getting the binary
11+
12+
Download from the [releases page](https://github.com/nudgebee/forager/releases)
13+
— pick the file matching your platform:
14+
15+
```bash
16+
curl -fsSL -o forager \
17+
https://github.com/nudgebee/forager/releases/download/v0.1.4-rc.5/nudgebee-forager-darwin-arm64
18+
chmod +x forager
19+
./forager --version
20+
```
21+
22+
Or build it:
23+
24+
```bash
25+
go build -o forager ./cmd
26+
```
27+
28+
Results print to stdout as JSON, logs to stderr, so output pipes into `jq`.
29+
30+
---
31+
32+
## sweep — what is on this network
33+
34+
Probes every address in a range and reports what answered. No credentials
35+
needed, and nothing is read from the machines themselves.
36+
37+
```bash
38+
./forager sweep --cidr 192.168.1.0/24 --ports 22
39+
```
40+
41+
| Flag | Default | |
42+
|---|---|---|
43+
| `--cidr` | *required* | IPv4 range to sweep |
44+
| `--ports` | `22` | Comma-separated ports to probe |
45+
| `--rate-pps` | `100` | Probes per second |
46+
| `--timeout-ms` | `1000` | Per-probe timeout |
47+
| `--exclude` | | Addresses or CIDRs to skip entirely |
48+
| `-v` | | Log progress to stderr |
49+
50+
```json
51+
{
52+
"cidrs": ["192.168.1.0/24"],
53+
"addresses_scanned": 254,
54+
"addresses_excluded": 0,
55+
"rate_pps": 100,
56+
"duration_seconds": 3.2,
57+
"hosts": [
58+
{"ip": "192.168.1.50", "open_ports": [22], "mac": "aa:bb:cc:dd:ee:ff",
59+
"rdns": "web-01.lan", "sources": ["tcp", "arp"]}
60+
]
61+
}
62+
```
63+
64+
`mac` appears only for hosts on the same network segment — anything reached
65+
through a router will not have one, and neither will the machine you are
66+
running from, since a host does not ARP for its own address. `rdns` appears
67+
only when reverse DNS resolves. Neither absence is an error.
68+
69+
`addresses_scanned` excludes the network and broadcast addresses, so a `/24`
70+
scans 254 rather than 256.
71+
72+
### Excluding things you must not touch
73+
74+
```bash
75+
./forager sweep --cidr 10.0.0.0/24 --exclude 10.0.0.5,10.0.0.100/30
76+
```
77+
78+
Exclusions are applied while building the address list, so an excluded host is
79+
never sent a packet at all.
80+
81+
---
82+
83+
## inventory — what is installed on those machines
84+
85+
Logs in over SSH and runs read-only commands. Needs two things sweep does not:
86+
an SSH login on the target, and a signed content pack.
87+
88+
```bash
89+
./forager inventory \
90+
--cidr 192.168.1.0/24 \
91+
--targets 192.168.1.50,192.168.1.51 \
92+
--user nudgebee-ro --key ~/.ssh/id_ed25519 \
93+
--pack ./linux-inventory-example.yaml \
94+
--pack-key '<base64 public key>'
95+
```
96+
97+
| Flag | Default | |
98+
|---|---|---|
99+
| `--cidr` | *required* | Scope the targets must fall within |
100+
| `--targets` | *required* | Comma-separated hosts |
101+
| `--pack` | *required* | Signed content pack |
102+
| `--pack-key` | *required* | Base64 Ed25519 public key the pack is signed with |
103+
| `--user` | `nudgebee-ro` | SSH username |
104+
| `--key` | | Path to SSH private key |
105+
| `--password-env` | | Env var holding the SSH password, instead of a key |
106+
| `--port` | `22` | SSH port |
107+
| `--concurrency` | `25` | Hosts inventoried in parallel |
108+
| `--known-hosts` | | known_hosts file; without it host keys are not verified |
109+
| `-v` | | Log progress to stderr |
110+
111+
`--cidr` is required here too. It bounds what can be contacted, so a mistyped
112+
target is refused rather than reached.
113+
114+
### 1. Create the read-only user on each target
115+
116+
```bash
117+
sudo useradd --system --create-home --shell /bin/sh nudgebee-ro
118+
sudo install -d -m700 -o nudgebee-ro -g nudgebee-ro /home/nudgebee-ro/.ssh
119+
echo '<your public key>' | sudo tee /home/nudgebee-ro/.ssh/authorized_keys
120+
sudo chmod 600 /home/nudgebee-ro/.ssh/authorized_keys
121+
sudo chown nudgebee-ro:nudgebee-ro /home/nudgebee-ro/.ssh/authorized_keys
122+
```
123+
124+
No sudo rights are needed. Every command the pack runs is read-only.
125+
126+
### 2. Sign a content pack
127+
128+
```bash
129+
PRIV=$(./forager pack keygen) # public key is printed to stderr
130+
./forager pack sign ./linux-inventory-example.yaml --key "$PRIV"
131+
```
132+
133+
The example pack lives at `docs/content-packs/linux-inventory-example.yaml`.
134+
135+
### 3. Run it
136+
137+
```json
138+
{
139+
"content_pack_version": 2,
140+
"targets": [
141+
{
142+
"host": "192.168.1.50",
143+
"status": "ok",
144+
"duration_seconds": 1.4,
145+
"facts": {"os_family": "debian", "os_id": "ubuntu",
146+
"os_major": "22", "arch": "x86_64"},
147+
"collectors": {
148+
"pkgs-dpkg": "acpid\t1:2.0.33-1ubuntu1\tamd64\tinstalled\n...",
149+
"machine-id": "ec2403e319a2f3f0ae53a05e3daf084b\n",
150+
"os-release": "NAME=\"Ubuntu\"\nID=ubuntu\n..."
151+
}
152+
},
153+
{"host": "192.168.1.51", "status": "ssh-auth-failed", "error": "..."}
154+
]
155+
}
156+
```
157+
158+
The right collectors run per OS family automatically — `dpkg-query` on
159+
Debian-like, `rpm -qa` on RHEL-like. You do not tell it what the target runs.
160+
161+
**A host that fails is a result, not an error.** Ask for ten targets and one
162+
refuses the connection, and you get nine inventories plus one entry saying
163+
why the tenth did not work. Per-host `status` is one of:
164+
165+
| Status | Means |
166+
|---|---|
167+
| `ok` | Collected |
168+
| `ssh-refused` | Port closed or filtered — usually a firewall |
169+
| `ssh-auth-failed` | Reached sshd, credentials rejected |
170+
| `timeout` | Reachable but did not finish in time |
171+
| `error` | Anything else, including a target outside `--cidr` |
172+
173+
Collector output is returned raw, exactly as the command printed it. Parsing
174+
happens server-side, which is why fixing a parser never requires touching a
175+
single machine.
176+
177+
---
178+
179+
## pack — managing content packs
180+
181+
A content pack is the file listing which commands to run. It is signed, and
182+
the signature is checked before anything executes, so the agent never runs
183+
collection commands it cannot attribute. There is deliberately no flag to skip
184+
verification — it would end up in a production config eventually.
185+
186+
```bash
187+
./forager pack keygen # new keypair
188+
./forager pack sign <file> --key <private> # sign, or re-sign, in place
189+
./forager pack verify <file> --key <public> # check without running anything
190+
```
191+
192+
`keygen` prints the private key to stdout and the public key to stderr, so
193+
`PRIV=$(./forager pack keygen)` captures the secret while leaving the public
194+
key visible.
195+
196+
For signing in CI, prefer `--key-env` over `--key`: a key on the command line
197+
lands in shell history and in the process list.
198+
199+
```bash
200+
./forager pack sign ./pack.yaml --key-env PACK_SIGNING_KEY
201+
```
202+
203+
`pack sign` also accepts `--out` to write elsewhere instead of overwriting.
204+
205+
---
206+
207+
## Before you scan someone else's network
208+
209+
Scanning looks like an attack to security tooling, because it is the same
210+
activity. On anything you do not own:
211+
212+
- Tell whoever runs intrusion detection, and get the machine you are running
213+
from allowlisted. Otherwise the first sweep becomes a security incident.
214+
- Agree which ranges are in scope and which must not be touched. Printers,
215+
industrial controllers and medical devices are the usual exclusions.
216+
- Lower `--rate-pps` if the network is monitored or fragile. Sweeps use
217+
ordinary TCP connections, not crafted packets, but volume is still visible.
218+
219+
One side effect worth knowing: a bare TCP connect to port 22 makes `sshd` log
220+
`Did not receive identification string`. It is harmless, but some `fail2ban`
221+
configurations act on it and will ban the scanning host — which then shows up
222+
as `ssh-refused` and looks like a firewall problem.
223+
224+
---
225+
226+
## When something does not work
227+
228+
| What you see | What it usually is |
229+
|---|---|
230+
| `pack signature verification failed` | The pack was signed with a different key than `--pack-key`. Re-sign it, or pass the matching key |
231+
| `loading content pack version N: no such file` | The pack file declares a different version than the one being requested. The version comes from inside the file |
232+
| Every host `ssh-refused` | A firewall between you and the targets, or sshd not listening on `--port` |
233+
| Every host `ssh-auth-failed` | Wrong `--user`, or the key is not in that user's `authorized_keys` |
234+
| Sweep finds nothing | Check `--ports` — the default is only 22. Also check that a firewall is not dropping the probes |
235+
| No `mac` on any host | The targets are not on the same network segment as the machine you are running from. Expected, not a fault |
236+
| `no ssh auth method provided` | Neither `--key` nor `--password-env` was given, or the password env var is empty |
237+
| Sweep is slower than expected | Dead addresses each cost a full `--timeout-ms`. Lower it, or raise `--rate-pps` if the network can take it |
238+
239+
---
240+
241+
## What this does not do
242+
243+
- **Windows.** Sweep will find Windows machines and report their open ports,
244+
but inventory is SSH-only and the content pack knows only Linux package
245+
managers. A Windows host comes back as `unsupported-os`.
246+
- **Machines that are switched off.** Nothing that looks at a network can see
247+
them. The agent can read a hypervisor to find those; the CLI cannot.
248+
- **Active Directory.** The agent supports it; the CLI does not expose it yet.
249+
- **Storing anything.** Results print and are gone. Persisting them, tracking
250+
changes over time and reporting coverage is what the full product does.

0 commit comments

Comments
 (0)