Skip to content

Commit 29f09fe

Browse files
author
Claudio Cesar de Oliveira Junior
committed
feat: harden dataplane security and move protocol layer into SDK
Implement secure session tokens, DID key agreement support, TLS/mTLS-ready dataplane transport, and SDK-extracted crypto/gRPC helpers with minimal automated security tests for the public demo flow. Made-with: Cursor
1 parent abf7e3d commit 29f09fe

30 files changed

Lines changed: 1791 additions & 114 deletions

‎.gitignore‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,4 +55,5 @@ terraform.tfvars
5555
*.html
5656

5757
examples/**/node_modules/
58-
examples/**/package-lock.json
58+
examples/**/package-lock.json
59+
examples/public-mesh-openai-demo/certs/

‎examples/public-mesh-openai-demo/.env.example‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,3 +5,18 @@ NATS_URL=nats.meshprotocol.dev:4222
55
NATS_TOKEN=your_token_here
66
REGISTRY_URL=https://api.meshprotocol.dev
77
OPENAI_API_KEY=sk-your-openai-key
8+
SESSION_TOKEN_SECRET=replace-with-strong-shared-secret
9+
DATAPLANE_BIND=0.0.0.0:50051
10+
DATAPLANE_PUBLIC_ENDPOINT=grpc://127.0.0.1:50051
11+
PROVIDER_ED25519_PRIVATE_KEY_BASE64=base64_pkcs8_private_key_for_provider
12+
CONSUMER_ED25519_PRIVATE_KEY_BASE64=base64_pkcs8_private_key_for_consumer
13+
ALLOW_LEGACY_DATAPLANE=0
14+
DATAPLANE_ALLOW_INSECURE=0
15+
DATAPLANE_TLS_CA_CERT_PATH=../certs/dev/ca.crt
16+
DATAPLANE_TLS_SERVER_CERT_PATH=../certs/dev/server.crt
17+
DATAPLANE_TLS_SERVER_KEY_PATH=../certs/dev/server.key
18+
DATAPLANE_TLS_REQUIRE_CLIENT_CERT=1
19+
DATAPLANE_TLS_CLIENT_CERT_PATH=../certs/dev/client.crt
20+
DATAPLANE_TLS_CLIENT_KEY_PATH=../certs/dev/client.key
21+
DATAPLANE_TLS_SERVER_NAME=localhost
22+
METRICS_PORT=9095
Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
# Security Rollout Guide (Data Plane)
2+
3+
## Current Mode
4+
5+
- Payload path migrated from `mesh.tasks.*` / `mesh.results.*` to DataPlane gRPC.
6+
- Handshake now enforces:
7+
- signed `session_token` (HS256),
8+
- Ed25519 consumer/provider signatures,
9+
- ephemeral X25519 key exchange.
10+
- DID Document now also publishes `key_agreement` (X25519) for provider/consumer.
11+
- Transfer/Result now use `AES-256-GCM` with per-message nonce and `sequence`.
12+
- Provider now exposes Prometheus metrics on `:METRICS_PORT/metrics`.
13+
- gRPC now supports TLS and optional mTLS (client cert validation).
14+
15+
## Rollout Strategy
16+
17+
1. Deploy provider with `dataplane_capability=v1-e2e` annotation (automatic in current provider script).
18+
2. Enable consumer strict mode (`ALLOW_LEGACY_DATAPLANE=0`) only after provider(s) are updated.
19+
3. Monitor provider metrics logs:
20+
- `handshakeFailed`
21+
- `decryptFailed`
22+
- `replayDetected`
23+
4. If handshake failures spike, temporarily set `ALLOW_LEGACY_DATAPLANE=1` on consumers during incident triage.
24+
5. After stabilization window, remove any legacy fallback and block providers that do not advertise `v1-e2e`.
25+
26+
## Local TLS/mTLS Quickstart
27+
28+
1. Generate dev certs:
29+
- `npm run certs:dev`
30+
2. Generate Ed25519 keys:
31+
- `npm run keys:provider`
32+
- `npm run keys:consumer`
33+
3. Copy `.env.example` to `.env` and fill:
34+
- `NATS_TOKEN`, `OPENAI_API_KEY`, `SESSION_TOKEN_SECRET`
35+
- `PROVIDER_ED25519_PRIVATE_KEY_BASE64`, `CONSUMER_ED25519_PRIVATE_KEY_BASE64`
36+
4. Start provider:
37+
- `npm run run:provider`
38+
5. Start consumer:
39+
- `npm run run:consumer -- "What is 17 * 23?"`
40+
6. Verify metrics:
41+
- `curl -s http://localhost:9095/metrics | rg "mesh_provider_dataplane_"`
42+
43+
## Operational Hardening
44+
45+
- Keep `SESSION_TOKEN_SECRET` rotated and out of source control.
46+
- Keep Ed25519 private keys outside repository and inject only via environment.
47+
- Enforce TLS transport for gRPC endpoints in non-local environments:
48+
- Set `DATAPLANE_ALLOW_INSECURE=0`
49+
- Set `DATAPLANE_TLS_SERVER_CERT_PATH` + `DATAPLANE_TLS_SERVER_KEY_PATH`
50+
- Set `DATAPLANE_TLS_CA_CERT_PATH`
51+
- Enable mTLS when both sides are ready:
52+
- Provider: `DATAPLANE_TLS_REQUIRE_CLIENT_CERT=1`
53+
- Consumer: `DATAPLANE_TLS_CLIENT_CERT_PATH` + `DATAPLANE_TLS_CLIENT_KEY_PATH`
54+
- Use `DATAPLANE_TLS_SERVER_NAME` on consumer when certificate CN/SAN does not match endpoint host.
55+
- Alert on any `replayDetected` event (treat as suspicious traffic).
56+
- Reject providers without `did_document` in registry metadata.
57+
58+
## Key Generation
59+
60+
- Generate consumer keypair:
61+
- `npm run keys:consumer`
62+
- Generate provider keypair:
63+
- `npm run keys:provider`
64+
- Copy `<ROLE>_ED25519_PRIVATE_KEY_BASE64` into `.env`.
65+
- Use `<ROLE>_ED25519_PUBLIC_KEY_BASE64` only for diagnostics/audits (public data).
66+
67+
## Prometheus Metrics
68+
69+
Main metrics:
70+
- `mesh_provider_dataplane_handshake_total{outcome,reason}`
71+
- `mesh_provider_dataplane_transfer_total{outcome,reason}`
72+
- `mesh_provider_dataplane_decrypt_failures_total{reason}`
73+
- `mesh_provider_dataplane_replay_total{reason}`
74+
- `mesh_provider_dataplane_phase_duration_seconds{phase}`
75+
76+
## Deploy Checklist (Production)
77+
78+
- `session_token` signing secret rotated and sourced from secret manager.
79+
- gRPC with `DATAPLANE_ALLOW_INSECURE=0`.
80+
- Valid server cert/key configured and signed by trusted CA.
81+
- mTLS enabled (`DATAPLANE_TLS_REQUIRE_CLIENT_CERT=1`) where client identity is required.
82+
- `DATAPLANE_TLS_SERVER_NAME` validated in consumer environment.
83+
- Ed25519 private keys injected by runtime secret manager (never in repo).
84+
- Alerts configured for:
85+
- handshake failures spike,
86+
- decrypt failures > baseline,
87+
- replay detections > 0.
88+
- Metrics endpoint restricted to trusted network or protected scrape path.

‎examples/public-mesh-openai-demo/consumer/index.js‎

Lines changed: 184 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -4,11 +4,27 @@
44
* Loads .env from parent (NATS_TOKEN, REGISTRY_URL, OPENAI_API_KEY).
55
*/
66

7-
import { connect } from "nats";
87
import { Agent, InMemory, Tool } from "@nebulaos/core";
98
import { OpenAI } from "@nebulaos/openai";
109
import { MeshClient } from "@meshprotocol/sdk";
1110
import { z } from "zod";
11+
import { createDataPlaneClient, parseGrpcEndpoint } from "../shared/dataplane.js";
12+
import {
13+
buildHandshakePayload,
14+
createX25519Ephemeral,
15+
decryptChunk,
16+
deriveSessionKey,
17+
encryptChunk,
18+
exportEd25519PublicKeyBase64,
19+
exportX25519PublicKeyBase64,
20+
exportX25519PublicKeyBytes,
21+
importX25519PublicKeyBase64,
22+
loadEd25519PrivateKeyFromBase64,
23+
publicKeyFromBase64,
24+
signEd25519,
25+
verifyEd25519,
26+
} from "../shared/security.js";
27+
import crypto from "node:crypto";
1228

1329
const DEFAULT_HOST = "nats.meshprotocol.dev";
1430
const DEFAULT_PORT = "4222";
@@ -39,10 +55,15 @@ const NATS_SERVER = getNatsServerUrl();
3955
const NATS_TOKEN = (process.env.NATS_TOKEN || "").trim();
4056
const REGISTRY_URL = process.env.REGISTRY_URL || "https://api.meshprotocol.dev";
4157
const CONSUMER_DID = "did:mesh:agent:public-demo-consumer";
42-
const TASK_PREFIX = "mesh.tasks.";
43-
const RESULT_PREFIX = "mesh.results.";
4458
const REQUEST_TIMEOUT_MS = 25_000;
4559
const RESULT_TIMEOUT_MS = 15_000;
60+
const CONSUMER_ED25519_PRIVATE_KEY_BASE64 = (process.env.CONSUMER_ED25519_PRIVATE_KEY_BASE64 || "").trim();
61+
const ALLOW_LEGACY_DATAPLANE = (process.env.ALLOW_LEGACY_DATAPLANE || "").trim() === "1";
62+
const DATAPLANE_ALLOW_INSECURE = (process.env.DATAPLANE_ALLOW_INSECURE || "").trim() === "1";
63+
const DATAPLANE_TLS_CA_CERT_PATH = (process.env.DATAPLANE_TLS_CA_CERT_PATH || "").trim();
64+
const DATAPLANE_TLS_CLIENT_CERT_PATH = (process.env.DATAPLANE_TLS_CLIENT_CERT_PATH || "").trim();
65+
const DATAPLANE_TLS_CLIENT_KEY_PATH = (process.env.DATAPLANE_TLS_CLIENT_KEY_PATH || "").trim();
66+
const DATAPLANE_TLS_SERVER_NAME = (process.env.DATAPLANE_TLS_SERVER_NAME || "").trim();
4667

4768
if (!process.env.OPENAI_API_KEY) {
4869
console.error("[Consumer] OPENAI_API_KEY is required (set in .env).");
@@ -52,14 +73,19 @@ if (!process.env.NATS_TOKEN) {
5273
console.error("[Consumer] NATS_TOKEN is required for public mesh (set in .env).");
5374
process.exit(1);
5475
}
76+
if (!CONSUMER_ED25519_PRIVATE_KEY_BASE64) {
77+
console.error("[Consumer] CONSUMER_ED25519_PRIVATE_KEY_BASE64 is required.");
78+
process.exit(1);
79+
}
5580

5681
async function main() {
5782
console.log("[Consumer] Connecting to PUBLIC mesh (meshprotocol.dev)...\n");
5883

59-
const natsForTasks = await connect({
60-
servers: NATS_SERVER,
61-
token: NATS_TOKEN || undefined,
62-
});
84+
const consumerPrivateKey = loadEd25519PrivateKeyFromBase64(CONSUMER_ED25519_PRIVATE_KEY_BASE64);
85+
const consumerPublicKey = crypto.createPublicKey(consumerPrivateKey);
86+
const consumerPublicKeyBase64 = exportEd25519PublicKeyBase64(consumerPublicKey);
87+
const consumerStaticKeyAgreement = createX25519Ephemeral();
88+
const consumerKeyAgreementPublicBase64 = exportX25519PublicKeyBase64(consumerStaticKeyAgreement.publicKey);
6389

6490
const mesh = new MeshClient({
6591
natsUrl: NATS_SERVER,
@@ -69,6 +95,43 @@ async function main() {
6995
natsAuth: NATS_TOKEN ? { token: NATS_TOKEN } : undefined,
7096
});
7197

98+
const consumerCard = {
99+
metadata: {
100+
id: CONSUMER_DID,
101+
name: "PublicDemoConsumer",
102+
version: "1.0.0",
103+
owner: "did:mesh:org:demo",
104+
did_document: {
105+
id: CONSUMER_DID,
106+
verification_method: [
107+
{
108+
id: `${CONSUMER_DID}#key-1`,
109+
type: "Ed25519VerificationKey2020",
110+
controller: CONSUMER_DID,
111+
public_key_base64: consumerPublicKeyBase64,
112+
},
113+
],
114+
key_agreement: [
115+
{
116+
id: `${CONSUMER_DID}#key-agreement-1`,
117+
type: "X25519KeyAgreementKey2020",
118+
controller: CONSUMER_DID,
119+
public_key_base64: consumerKeyAgreementPublicBase64,
120+
},
121+
],
122+
},
123+
},
124+
spec: {
125+
domains: { primary: ["demo", "consumer"] },
126+
capabilities: [{ id: "requester", description: "Consumer identity registration only" }],
127+
endpoints: {
128+
control_plane: { nats_subject: "mesh.agents.demo.consumer" },
129+
data_plane: { grpc: "grpc://127.0.0.1:0" },
130+
},
131+
},
132+
};
133+
await mesh.register(consumerCard, { status: "active" });
134+
72135
const requestMathFromMeshTool = new Tool({
73136
id: "request_math_from_mesh",
74137
description: "Sends a math/calculation question to a specialist on the mesh. Use this when the user asks for any math operation.",
@@ -88,41 +151,131 @@ async function main() {
88151
}
89152

90153
const sessionId = result.sessionId;
91-
const taskSubject = TASK_PREFIX + sessionId;
92-
const resultSubject = RESULT_PREFIX + sessionId;
93-
94-
await new Promise((r) => setTimeout(r, 300));
154+
const providerDid = result.parties.provider;
155+
const providerInfoResp = await fetch(`${REGISTRY_URL.replace(/\/$/, "")}/agents/${encodeURIComponent(providerDid)}`);
156+
if (!providerInfoResp.ok) {
157+
throw new Error(`Provider registry lookup failed: ${providerInfoResp.status}`);
158+
}
159+
const providerInfo = await providerInfoResp.json();
160+
const capability = providerInfo?.card?.metadata?.annotations?.dataplane_capability;
161+
if (capability !== "v1-e2e" && !ALLOW_LEGACY_DATAPLANE) {
162+
throw new Error("Provider does not advertise dataplane_capability=v1-e2e");
163+
}
164+
const providerGrpcEndpoint = parseGrpcEndpoint(providerInfo?.card?.spec?.endpoints?.data_plane?.grpc || "");
165+
const providerPubKeyB64 = providerInfo?.card?.metadata?.did_document?.verification_method?.[0]?.public_key_base64;
166+
const providerAgreementKeyB64 = providerInfo?.card?.metadata?.did_document?.key_agreement?.[0]?.public_key_base64;
167+
if (!providerPubKeyB64) {
168+
throw new Error("Provider DID document missing public key");
169+
}
170+
if (!providerAgreementKeyB64) {
171+
throw new Error("Provider DID document missing keyAgreement X25519 key");
172+
}
173+
const providerPublicKey = publicKeyFromBase64(providerPubKeyB64);
174+
const dpClient = createDataPlaneClient(providerGrpcEndpoint, {
175+
insecure: DATAPLANE_ALLOW_INSECURE,
176+
caCertPath: DATAPLANE_TLS_CA_CERT_PATH,
177+
clientCertPath: DATAPLANE_TLS_CLIENT_CERT_PATH,
178+
clientKeyPath: DATAPLANE_TLS_CLIENT_KEY_PATH,
179+
serverName: DATAPLANE_TLS_SERVER_NAME,
180+
});
181+
const consumerEph = createX25519Ephemeral();
182+
const consumerEphPub = exportX25519PublicKeyBytes(consumerEph.publicKey);
183+
// Ensures provider DID exposes a valid X25519 key for agreement.
184+
const providerAgreementKey = importX25519PublicKeyBase64(providerAgreementKeyB64);
185+
const didSharedSecret = crypto.diffieHellman({
186+
privateKey: consumerEph.privateKey,
187+
publicKey: providerAgreementKey,
188+
});
189+
if (!didSharedSecret || didSharedSecret.length === 0) {
190+
throw new Error("Unable to derive shared secret using provider DID key agreement");
191+
}
192+
const consumerSigPayload = buildHandshakePayload(sessionId, CONSUMER_DID, consumerEphPub);
193+
const consumerSig = signEd25519(consumerPrivateKey, consumerSigPayload);
95194

96-
natsForTasks.publish(
97-
taskSubject,
98-
new TextEncoder().encode(JSON.stringify({ description: question }))
195+
const handshakeResp = await new Promise((resolve, reject) => {
196+
dpClient.Handshake({
197+
session_id: sessionId,
198+
session_token: result.sessionToken,
199+
consumer_ephemeral_pub: consumerEphPub,
200+
consumer_did: CONSUMER_DID,
201+
consumer_did_signature: consumerSig,
202+
}, (err, response) => (err ? reject(err) : resolve(response)));
203+
});
204+
const providerSigPayload = buildHandshakePayload(
205+
sessionId,
206+
handshakeResp.provider_did,
207+
handshakeResp.provider_ephemeral_pub
99208
);
209+
const providerSigValid = verifyEd25519(
210+
providerPublicKey,
211+
providerSigPayload,
212+
Buffer.from(handshakeResp.provider_did_signature)
213+
);
214+
if (!providerSigValid) {
215+
throw new Error("Provider handshake signature invalid");
216+
}
217+
const sessionKey = deriveSessionKey({
218+
privateKey: consumerEph.privateKey,
219+
peerPublicKeyBytes: handshakeResp.provider_ephemeral_pub,
220+
sessionId,
221+
});
100222

101-
const answer = await new Promise((resolve, reject) => {
102-
const timeout = setTimeout(() => {
103-
sub.unsubscribe();
104-
reject(new Error("Timeout waiting for specialist response"));
105-
}, RESULT_TIMEOUT_MS);
106-
107-
const sub = natsForTasks.subscribe(resultSubject, {
108-
max: 1,
109-
callback: (err, msg) => {
110-
clearTimeout(timeout);
111-
sub.unsubscribe();
223+
await new Promise((resolve, reject) => {
224+
const call = dpClient.Transfer(
225+
{ "x-session-id": sessionId },
226+
(err, ack) => {
112227
if (err) {
113228
reject(err);
114229
return;
115230
}
116-
try {
117-
const data = JSON.parse(new TextDecoder().decode(msg.data));
118-
resolve(data.result ?? data.error ?? "(no response)");
119-
} catch {
120-
resolve("(invalid response)");
231+
if (!ack?.accepted) {
232+
reject(new Error(ack?.error_message || "provider rejected transfer"));
233+
return;
121234
}
122-
},
235+
resolve();
236+
}
237+
);
238+
const encrypted = encryptChunk({
239+
key: sessionKey,
240+
sequence: 1,
241+
payloadBuffer: Buffer.from(JSON.stringify({ description: question }), "utf8"),
123242
});
243+
call.write({
244+
ciphertext: encrypted.ciphertext,
245+
nonce: encrypted.nonce,
246+
sequence: encrypted.sequence,
247+
is_final: encrypted.is_final,
248+
algorithm: encrypted.algorithm,
249+
});
250+
call.end();
124251
});
125252

253+
const answer = await new Promise((resolve, reject) => {
254+
const timeout = setTimeout(() => {
255+
reject(new Error("Timeout waiting for specialist response"));
256+
}, RESULT_TIMEOUT_MS);
257+
const stream = dpClient.Result({ session_id: sessionId });
258+
stream.on("data", (chunk) => {
259+
clearTimeout(timeout);
260+
try {
261+
const plaintext = decryptChunk({ key: sessionKey, chunk });
262+
const data = JSON.parse(plaintext.toString("utf8"));
263+
resolve(data.result ?? data.error ?? "(no response)");
264+
} catch {
265+
resolve("(invalid response)");
266+
}
267+
});
268+
stream.on("error", (err) => {
269+
clearTimeout(timeout);
270+
reject(err);
271+
});
272+
stream.on("end", () => {
273+
clearTimeout(timeout);
274+
});
275+
});
276+
277+
dpClient.close?.();
278+
126279
return { success: true, result: answer };
127280
},
128281
});
@@ -151,7 +304,6 @@ async function main() {
151304

152305
console.log("[Consumer] Response:", exec.content ?? "(empty)");
153306

154-
await natsForTasks.close();
155307
await mesh.close();
156308
console.log("\n[Consumer] Done.");
157309
}

0 commit comments

Comments
 (0)