Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Required
AWS_REGION=us-east-1
AWS_ACCESS_KEY_ID=your-access-key
AWS_SECRET_ACCESS_KEY=your-secret-key
S3_BUCKET_NAME=nx-cloud
S3_ENDPOINT_URL=https://s3.amazonaws.com
NX_CACHE_ACCESS_TOKEN=your-secure-token

# Optional
PORT=3000

# Optional TLS/HTTPS — set BOTH to serve over HTTPS. Leave unset for plain HTTP.
# TLS_CERT_PATH=/path/to/tls.crt
# TLS_KEY_PATH=/path/to/tls.key
47 changes: 46 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,30 @@ S3_BUCKET_NAME=your-bucket-name
S3_ENDPOINT_URL=your-s3-endpoint-url
NX_CACHE_ACCESS_TOKEN=your-secure-token
PORT=3000 # Optional, defaults to 3000
TLS_CERT_PATH=/path/to/tls.crt # Optional, enables HTTPS (must be set with TLS_KEY_PATH)
TLS_KEY_PATH=/path/to/tls.key # Optional, enables HTTPS (must be set with TLS_CERT_PATH)
```

See [`.env.example`](.env.example) for a ready-to-copy template.

### HTTPS / TLS

By default the server listens over plain HTTP. To serve over HTTPS directly —
without putting a reverse proxy in front of it — set both `TLS_CERT_PATH` and
`TLS_KEY_PATH` to the PEM cert and key files (they must be set together; setting
only one exits with a configuration error). The files are read once at startup,
so rotating the certificate requires a restart. In Kubernetes, terminating TLS
at the Ingress is usually preferable; this option is for direct exposure or
mutual-TLS setups.

> **Certificate trust for Nx clients.** Nx's self-hosted cache client uses a
> native HTTP client that validates against the operating system trust store —
> it does **not** honor `NODE_EXTRA_CA_CERTS` or `SSL_CERT_FILE`. Use a
> certificate that is already trusted on the machines running Nx (a public CA
> such as Let's Encrypt, or your corporate CA), or install your CA into the
> system trust store. A bare self-signed certificate is rejected by Nx with
> `error sending request`.

## Installation

### Using Docker
Expand All @@ -58,6 +80,22 @@ docker run -p 3000:3000 \
ghcr.io/ikatsuba/nx-cache-server:latest
```

To serve over HTTPS, mount your PEM cert/key and point the TLS env vars at them:

```bash
docker run -p 3000:3000 \
-v /host/certs:/certs:ro \
-e TLS_CERT_PATH=/certs/tls.crt \
-e TLS_KEY_PATH=/certs/tls.key \
-e AWS_REGION=your-aws-region \
-e AWS_ACCESS_KEY_ID=your-access-key \
-e AWS_SECRET_ACCESS_KEY=your-secret-key \
-e S3_BUCKET_NAME=your-bucket-name \
-e S3_ENDPOINT_URL=your-s3-endpoint-url \
-e NX_CACHE_ACCESS_TOKEN=your-secure-token \
ghcr.io/ikatsuba/nx-cache-server:latest
```

### Using Helm (Kubernetes)

The chart is published as an OCI artifact to GHCR alongside the Docker image:
Expand Down Expand Up @@ -86,7 +124,14 @@ git clone <repository-url>
cd nx-cache-server
```

2. Start a local S3 emulator (no Docker required — uses
2. Copy the environment template and fill in your values (the `start` task reads
`.env`):

```bash
cp .env.example .env
```

3. Start a local S3 emulator (no Docker required — uses
[emulate.dev](https://emulate.dev)):

```bash
Expand Down
25 changes: 25 additions & 0 deletions charts/nx-cache-server/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,26 @@ keys in the Secret because the server reads them from env vars. To opt fully
out of static keys, fork the chart or set them to empty placeholders if your
S3 client picks up the IAM role from the metadata service.

## Serving over HTTPS

TLS is usually terminated at the Ingress. If you instead want the pod itself to
serve HTTPS, point `tls.secretName` at an existing `kubernetes.io/tls` Secret
(e.g. one issued by cert-manager) and enable TLS. The cert/key are mounted
read-only and the probes switch to the HTTPS scheme automatically.

```bash
helm install nx-cache oci://ghcr.io/ikatsuba/charts/nx-cache-server \
--set config.s3.endpointUrl=https://s3.amazonaws.com \
--set tls.enabled=true \
--set tls.secretName=nx-cache-tls
```

> **Certificate rotation.** The server reads the cert/key once at startup, so a
> renewed certificate is not picked up until the pod restarts. When cert-manager
> rotates the Secret, trigger a rollout (`kubectl rollout restart deployment/...`)
> or use a controller such as [Reloader](https://github.com/stakater/Reloader) to
> restart the pods automatically.

## Values

| Key | Default | Description |
Expand All @@ -78,6 +98,11 @@ S3 client picks up the IAM role from the metadata service.
| `secrets.nxCacheAccessToken` | `""` | Required if `existingSecret` is empty |
| `secrets.awsAccessKeyId` | `""` | Required if `existingSecret` is empty |
| `secrets.awsSecretAccessKey` | `""` | Required if `existingSecret` is empty |
| `tls.enabled` | `false` | Serve over HTTPS using a mounted cert/key |
| `tls.secretName` | `""` | Existing Secret holding the PEM cert/key. Required when `tls.enabled` |
| `tls.certKey` | `tls.crt` | Key in the Secret holding the PEM cert |
| `tls.keyKey` | `tls.key` | Key in the Secret holding the PEM key |
| `tls.mountPath` | `/etc/nx-cache-server/tls` | Mount path for the cert/key |
| `extraEnv` | `[]` | Extra env vars appended to the container |
| `resources` | `{}` | |
| `nodeSelector` | `{}` | |
Expand Down
24 changes: 24 additions & 0 deletions charts/nx-cache-server/templates/deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,12 @@ spec:
secretKeyRef:
name: {{ include "nx-cache-server.secretName" . }}
key: aws-secret-access-key
{{- if .Values.tls.enabled }}
- name: TLS_CERT_PATH
value: {{ printf "%s/%s" .Values.tls.mountPath .Values.tls.certKey | quote }}
- name: TLS_KEY_PATH
value: {{ printf "%s/%s" .Values.tls.mountPath .Values.tls.keyKey | quote }}
{{- end }}
{{- with .Values.extraEnv }}
{{- toYaml . | nindent 12 }}
{{- end }}
Expand All @@ -74,6 +80,9 @@ spec:
httpGet:
path: {{ .Values.probes.liveness.path }}
port: http
{{- if .Values.tls.enabled }}
scheme: HTTPS
{{- end }}
initialDelaySeconds: {{ .Values.probes.liveness.initialDelaySeconds }}
periodSeconds: {{ .Values.probes.liveness.periodSeconds }}
{{- end }}
Expand All @@ -82,13 +91,28 @@ spec:
httpGet:
path: {{ .Values.probes.readiness.path }}
port: http
{{- if .Values.tls.enabled }}
scheme: HTTPS
{{- end }}
initialDelaySeconds: {{ .Values.probes.readiness.initialDelaySeconds }}
periodSeconds: {{ .Values.probes.readiness.periodSeconds }}
{{- end }}
{{- with .Values.resources }}
resources:
{{- toYaml . | nindent 12 }}
{{- end }}
{{- if .Values.tls.enabled }}
volumeMounts:
- name: tls
mountPath: {{ .Values.tls.mountPath }}
readOnly: true
{{- end }}
{{- if .Values.tls.enabled }}
volumes:
- name: tls
secret:
secretName: {{ required "tls.secretName is required when tls.enabled is true" .Values.tls.secretName }}
{{- end }}
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
Expand Down
14 changes: 14 additions & 0 deletions charts/nx-cache-server/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,20 @@ secrets:
awsAccessKeyId: ""
awsSecretAccessKey: ""

# Optional TLS/HTTPS. When enabled the server reads the cert/key from a mounted
# Secret and serves HTTPS instead of HTTP. Disabled by default — TLS is commonly
# terminated at the Ingress instead.
tls:
enabled: false
# Name of an existing Secret (e.g. a kubernetes.io/tls Secret managed by
# cert-manager) holding the PEM cert and key. Required when enabled.
secretName: ""
# Keys within the Secret holding the PEM cert and key.
certKey: tls.crt
keyKey: tls.key
# Mount path for the cert/key inside the container.
mountPath: /etc/nx-cache-server/tls

# Extra environment variables appended to the container. Use either `value`
# or `valueFrom` per Kubernetes env semantics.
extraEnv: []
122 changes: 122 additions & 0 deletions e2e/https.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
import $ from '@david/dax';
import { assertEquals } from '@std/assert';
import { afterAll, beforeAll, describe, it } from '@std/testing/bdd';
import { join } from '@std/path/join';
import { startEmulator } from '../scripts/start-emulator.ts';

const CACHE_TOKEN = 'test-token';
const BUCKET = 'nx-cloud';
// Distinct from the Nx e2e emulator (4566) so the two suites never collide.
const EMULATE_PORT = 4567;
const CA = join(Deno.cwd(), 'src', 'fixtures', 'tls', 'ca.pem');
const CERT = join(Deno.cwd(), 'src', 'fixtures', 'tls', 'cert.pem');
const KEY = join(Deno.cwd(), 'src', 'fixtures', 'tls', 'key.pem');
const AUTH = `Authorization: Bearer ${CACHE_TOKEN}`;

function getFreePort(): number {
const listener = Deno.listen({ port: 0 });
const port = (listener.addr as Deno.NetAddr).port;
listener.close();
return port;
}

async function waitForHealth(
url: string,
client: Deno.HttpClient,
timeoutMs = 15000,
): Promise<void> {
const deadline = Date.now() + timeoutMs;
while (Date.now() < deadline) {
try {
const res = await fetch(`${url}/health`, { client });
await res.body?.cancel();
if (res.ok) return;
} catch {
// not yet listening
}
await new Promise((r) => setTimeout(r, 200));
}
throw new Error(`Cache server did not become ready at ${url}`);
}

// Drives the real `src/index.ts` process over HTTPS with an external client
// (curl), exercising the TLS startup wiring and a full PUT -> GET round-trip
// through the S3 emulator — the path Nx's native client cannot take with a
// self-signed cert.
describe('Remote Cache over HTTPS (curl)', () => {
let emulator: { url: string; close(): Promise<void> };
let server: Deno.ChildProcess;
let url: string;
let client: Deno.HttpClient;

beforeAll(async () => {
emulator = await startEmulator({ port: EMULATE_PORT, bucket: BUCKET });

const port = getFreePort();
url = `https://localhost:${port}`;
server = new Deno.Command(Deno.execPath(), {
args: [
'run',
'--allow-env',
'--allow-net',
'--allow-sys',
'--allow-read',
'src/index.ts',
],
env: {
PORT: String(port),
TLS_CERT_PATH: CERT,
TLS_KEY_PATH: KEY,
NX_CACHE_ACCESS_TOKEN: CACHE_TOKEN,
AWS_REGION: 'us-east-1',
AWS_ACCESS_KEY_ID: 'AKIAIOSFODNN7EXAMPLE',
AWS_SECRET_ACCESS_KEY: 'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY',
S3_BUCKET_NAME: BUCKET,
S3_ENDPOINT_URL: emulator.url,
},
stdout: 'inherit',
stderr: 'inherit',
}).spawn();

client = Deno.createHttpClient({ caCerts: [Deno.readTextFileSync(CA)] });
await waitForHealth(url, client);
});

afterAll(async () => {
try {
server.kill('SIGTERM');
await server.status;
} catch {
// already exited
}
client?.close();
await emulator.close();
});

it('uploads and downloads an artifact over HTTPS', async () => {
const hash = crypto.randomUUID();
const payload = `tls-artifact-${hash}`;
const tmp = await Deno.makeTempFile();
Deno.writeTextFileSync(tmp, payload);

try {
const putCode = await $`curl -sS -o /dev/null -w ${'%{http_code}'} \
-X PUT --cacert ${CA} -H ${AUTH} --data-binary ${`@${tmp}`} \
${`${url}/v1/cache/${hash}`}`.text();
assertEquals(putCode, '200');

const body = await $`curl -sS --cacert ${CA} -H ${AUTH} \
${`${url}/v1/cache/${hash}`}`.text();
assertEquals(body, payload);
} finally {
await Deno.remove(tmp);
}
});

it('rejects unauthenticated requests over HTTPS', async () => {
const hash = crypto.randomUUID();
const code = await $`curl -sS -o /dev/null -w ${'%{http_code}'} \
--cacert ${CA} ${`${url}/v1/cache/${hash}`}`.text();
assertEquals(code, '401');
});
});
19 changes: 19 additions & 0 deletions src/fixtures/tls/ca.pem
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
-----BEGIN CERTIFICATE-----
MIIDJzCCAg+gAwIBAgIUDoVRJ5x+/40PRH9wH0VhUoYjWjEwDQYJKoZIhvcNAQEL
BQAwIjEgMB4GA1UEAwwXbngtY2FjaGUtc2VydmVyLXRlc3QtY2EwIBcNMjYwNjAz
MjAyMDQyWhgPMjEyNjA1MTAyMDIwNDJaMCIxIDAeBgNVBAMMF254LWNhY2hlLXNl
cnZlci10ZXN0LWNhMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAxgRx
sZERsySWDrF7YCJZC28g7/2KuNLCVd7FeHQJCOhj3sU931sblrWwDPspL+oYL50z
3pBmswakm6qpwBmhYuexXwBPHZWI0Apl4YFBGHiLMHW6pPRvQcZ5KnGUgHdlgqrR
JgybaJgYcGEVUn50orKh1QyH+WpW17Kudp+eDuJOssy6eDh7On6bXoXkj1OF5XgP
ttIr6LBhXQSTeoMbATH2B1j+V1BkfyHx7QWvDNVFHKrXc15geajSG6qradVPpoY/
L2XyHgXBsyoAeNgecETV2rUV8uUhHVVKNpaI/1mLe3cRpEQ0soFYArFWMf0wmwZE
BYZkT9Nz/vU4Sf2nbQIDAQABo1MwUTAdBgNVHQ4EFgQUgjzLUP/Ubw8uDza6XF+o
PtIq6i8wHwYDVR0jBBgwFoAUgjzLUP/Ubw8uDza6XF+oPtIq6i8wDwYDVR0TAQH/
BAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAQEAs/qFIq5qdU8xsO967HIf9tUg/gLo
FQoMorM4fmPQUKLLv9//VT7gJWPzDT/3MNAhuPxYwQBAArrO0uM6nN8/Wshj1E+b
XkyJw7blT9spgl6LrMTTdbkCGPFvrjJADeO8/ZSLPiXTQva5e8HVcKP2msRzdVt3
+MTsqzOBRIGnGgv8ck0QpfBSvr2TVORndOK9qBIRndQ2w718yyzjpTTrHu4dh5mI
UVqLnaZKmF7sP2pTOqb3Z2amo3ZTLkWdMG19ivnDSxQoR40hhlrnMqBz+52hUKPq
oImgjNcSe8z2BWc4Gt4vlEuwsX1uq0Th8Rc8b9HQumgohOHpr5vgXPqeGQ==
-----END CERTIFICATE-----
20 changes: 20 additions & 0 deletions src/fixtures/tls/cert.pem
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
-----BEGIN CERTIFICATE-----
MIIDUzCCAjugAwIBAgIURbibTP7VS3kY7aR7BWSEUaRDXB8wDQYJKoZIhvcNAQEL
BQAwIjEgMB4GA1UEAwwXbngtY2FjaGUtc2VydmVyLXRlc3QtY2EwIBcNMjYwNjAz
MjAyMDQzWhgPMjEyNjA1MTAyMDIwNDNaMBQxEjAQBgNVBAMMCWxvY2FsaG9zdDCC
ASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALItZrrCsukf+3iKmlXjAUad
ob9qUf4Wg7EX3aenGyZ7yhfY9V19gBRy1jdzazvSsSDcK4uZtVBHe0VQwneZF1YI
kewgGSNJA7l7KKFGb4lwihtcQw7C+c35MXSExoEW0f0CN8O4kKNyvHknA3cTuBku
y7Hg7oFrMMUUzVZXDk8ikN3QXfQFR1DqOaAtkNC6xbwjybTM53wU3ikH15FVflgd
3FndcL9eLZOGeJQJudxNFkYHzLdCNrPl8AF9cSZTZBWCY/HptZBDs18v0JgbeycV
FuHBDSBlmRmowE17MGFV4koYDuiYzttFVdsVJR5OQk5dO+DROsi+uem1q45I2/UC
AwEAAaOBjDCBiTAaBgNVHREEEzARgglsb2NhbGhvc3SHBH8AAAEwCQYDVR0TBAIw
ADALBgNVHQ8EBAMCBaAwEwYDVR0lBAwwCgYIKwYBBQUHAwEwHQYDVR0OBBYEFG54
n0lwfvHJOx3OVQFQVhJmekXSMB8GA1UdIwQYMBaAFII8y1D/1G8PLg82ulxfqD7S
KuovMA0GCSqGSIb3DQEBCwUAA4IBAQBDWwo0/7GY9tvt6EZ3NNexIlrLoca/jUcN
VgqOY3F46wugDCpd8PyNNJ3fSdHWxWF7eMMUKr0LwloKaaTES6CLaHPeQNyjK9T7
MfrMvimqk7lu2uaXYeSrvARC6+AA8HqdlOwooAKqatjUqwTpPNu2vnC4XKcCxLkO
U6i2+bkZ1cZAFrt2oNo7WdJgrTiI1AjBWaWkObufvcA5uOSHyJ1PQTynSesso1jh
bm4XPmETCW+V+m6nvfMRtuyl3RkzsdPUvxS0iz4RU+abXx1fUfQ7sdj3k9U8q4UA
5HbLjoOysO2p8sOOE6xk2SFIsjvRYrhfJvvGXFp0y482QpOL0OVA
-----END CERTIFICATE-----
28 changes: 28 additions & 0 deletions src/fixtures/tls/key.pem
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
-----BEGIN PRIVATE KEY-----
MIIEvAIBADANBgkqhkiG9w0BAQEFAASCBKYwggSiAgEAAoIBAQCyLWa6wrLpH/t4
ippV4wFGnaG/alH+FoOxF92npxsme8oX2PVdfYAUctY3c2s70rEg3CuLmbVQR3tF
UMJ3mRdWCJHsIBkjSQO5eyihRm+JcIobXEMOwvnN+TF0hMaBFtH9AjfDuJCjcrx5
JwN3E7gZLsux4O6BazDFFM1WVw5PIpDd0F30BUdQ6jmgLZDQusW8I8m0zOd8FN4p
B9eRVX5YHdxZ3XC/Xi2ThniUCbncTRZGB8y3Qjaz5fABfXEmU2QVgmPx6bWQQ7Nf
L9CYG3snFRbhwQ0gZZkZqMBNezBhVeJKGA7omM7bRVXbFSUeTkJOXTvg0TrIvrnp
tauOSNv1AgMBAAECggEAL7HViPKrP5ovD3JSo+DzK3rwnDOSpShTsikITWN+8gs6
6q1J6O+6RD8FNqyvrMDWZ0xoX6MSlN29kq1ukOMrCv2aShHBiN93nsPn+VvIaYr1
fSNyU9Y/VjJ5WY5dMJryNP7N/JdAKHCDIQ8fb1WY9yy+m+vxDDWsRWUvHgiJrQbJ
dxnet6xP26qagN4xDbLWTfJmdOG+xEJenPSwUWOn73U7NHKPw2DpKnEQQOjdHpWn
IlRQBnjt/m3R9dhaQaqoZIsB10bO/r7h5Zu7lHwKpcCZnUuywKdL8PFBbmDCCQDz
R9la2m5KSgIUGVclVINZEWe+TowwxSnoDHiUpbkztQKBgQDxslcor8CndBABxUlB
XgAUrFXcjoYIoK9O2xMnj5zAAnPAlJ1gzw3nfbkpiepfH15JzUDQ4ypCqK7T3o02
pXCNc1KTdi1yzxIfO6QnsfrcSSBqEOlgAs9odvvQvJBsolVFanE73YYeI7tMKbtx
hiEFLfuCEG0yJYc0pprANW+h2wKBgQC8uMFXrtaWb6X9BmHwr0aqOI0zqEuEIlC5
JtQN4WnsarCuHGa+ldB2e+sgSO2FGBjJO4Ky/9fa9CWbiSfiN54Ia8PDqrMqYHIc
5QVrKygPmUdhl4SKc+bhB1kJbgIuDrj80tvyOyOqbFKvpkzV6CoBW7uqOYnh8nXF
HUWK/KdqbwKBgC3GVPODtb0BIjbzolga8XD44poO9BuWMx2AKDSJ4moLDNkgTF4V
WNDK4tKXKCgzzKu1j9z/6PUO9W1y7NO4qYRgnvcGnrzrssbviLtr3dLyloz//vMM
FTP3tOFnlQK+C+BFtIReBBSZN9PI45Mwj/hnKQcNDWE98klqdEyK0kWZAoGAGI04
qf2W8S9JOuevqi5DVTXSONw2JtSIB+5jPAMNUrJ2ldAgFb9LxX5Nkn+qVtxRrSxO
zpa2MEf+9IMdazKtHo9WgogckvDFJ7krwUt2BMF+JdYeahy19qSMEsv7AJ7ttgaA
krza4PsCrtNFv7ZacJ7MhVLC+2Adri0mtPQ1m4UCgYBEK/RigO0cSke5MGZ41ilP
01aqanXgTrQNmbQVKSRR3rkXPFezrUe4Qd4G+Q4lYHXo4dLqWNyklC5Jv2P+mcmM
PDUxarLUoGwgWGjFGEAIh+lqSDhjQNvwuijy4C2qTwVSjkh7/pniaa+YAHyzvvtX
FJRIX/WyuJC20ona1aP3/w==
-----END PRIVATE KEY-----
Loading
Loading