ESO-323: Adds Bitwarden and cross-platform e2e suites with failure artifact collection

Makefile

@@ -212,6 +212,7 @@ test-e2e: ## Run e2e tests against a cluster.
 		-count 1 -v -p 1 \
 		-tags e2e ./e2e \
 		-ginkgo.v \
+		-ginkgo.trace \
 		-ginkgo.show-node-events \
 		-ginkgo.label-filter=$(E2E_GINKGO_LABEL_FILTER)
 

README.md

@@ -129,6 +129,23 @@ kubectl apply -f https://raw.githubusercontent.com/<org>/external-secrets-operat
 
 More information can be found via the [Kubebuilder Documentation](https://book.kubebuilder.io/introduction.html)
 
+## Testing
+
+From the repo root you can run:
+
+| Target | Description |
+|--------|-------------|
+| `make test-unit` | Run unit tests (excluding test/e2e, test/apis, test/utils). |
+| `make test-apis` | Run API integration tests (Ginkgo tests in `test/apis` using envtest). |
+| `make test` | Run `test-apis` and `test-unit` (no cluster required). |
+| `make test-e2e` | Run end-to-end tests against a live cluster. |
+
+For e2e tests, including prerequisites and suite-specific commands (e.g. label filters for AWS, Bitwarden, cross-platform), see [test/e2e/README.md](test/e2e/README.md). Example:
+
+```sh
+make test-e2e E2E_GINKGO_LABEL_FILTER="<label-filter>"
+```
+
 ## Contributing
 We welcome contributions from the community! To contribute:
 

hack/govulncheck.sh

@@ -20,7 +20,11 @@ set -o errexit
 ## Below vulnerabilities are in the kubernetes package, which impacts the server and not the operator, which is the client.
 # - https://pkg.go.dev/vuln/GO-2025-3521 - Kubernetes GitRepo Volume Inadvertent Local Repository Access in k8s.io/kubernetes
 # - https://pkg.go.dev/vuln/GO-2025-3547 - Kubernetes kube-apiserver Vulnerable to Race Condition in k8s.io/kubernetes
-KNOWN_VULNS_PATTERN="GO-2025-3521|GO-2025-3547"
+#
+## Below vulnerabilities are in the go packages, which impacts the operator code and requires the fix to be available downstream.
+# - https://pkg.go.dev/vuln/GO-2026-4601 - Incorrect parsing of IPv6 host literals in net/url
+# - https://pkg.go.dev/vuln/GO-2026-4602 - FileInfo can escape from a Root in os
+KNOWN_VULNS_PATTERN="GO-2025-3521|GO-2025-3547|GO-2026-4601|GO-2026-4602"
 
 GOVULNCHECK_BIN="${1:-}"
 OUTPUT_DIR="${2:-}"

test/README.md

@@ -0,0 +1,22 @@
+# Test directory
+
+This directory holds the operator’s tests and shared test utilities.
+
+## Layout
+
+| Path | Purpose |
+|------|---------|
+| **e2e/** | End-to-end tests run against a live cluster. See [e2e/README.md](e2e/README.md) for suites, labels, and prerequisites. |
+| **apis/** | API integration tests (Ginkgo) run with envtest (no real cluster). See [apis/README.md](apis/README.md). |
+| **utils/** | Shared helpers for e2e (and related) tests; built only with the `e2e` build tag. |
+
+## Make targets (from repo root)
+
+| Command | What it runs |
+|---------|----------------|
+| `make test-unit` | Unit tests for all packages except `test/e2e`, `test/apis`, `test/utils`. |
+| `make test-apis` | API tests in `test/apis` (envtest + Ginkgo). |
+| `make test` | `make test-apis` and `make test-unit` (no cluster). |
+| `make test-e2e` | E2e tests in `test/e2e`; requires a cluster and optional label filter. |
+
+See the root [README.md](../README.md) Testing section and [e2e/README.md](e2e/README.md) for more detail.

test/apis/README.md

@@ -1 +1,24 @@
-Refer to the [openshift/api test suite](https://github.com/openshift/api/tree/master/tests) for more details.
+# API integration tests
+
+API tests exercise the operator’s APIs (CRDs, webhooks, controllers) using **envtest** — a local control plane (no real cluster required). They are implemented with [Ginkgo](https://onsi.github.io/ginkgo/).
+
+## Running
+
+From the repository root:
+
+```bash
+make test-apis
+```
+
+This installs envtest binaries if needed, sets `KUBEBUILDER_ASSETS`, and runs Ginkgo in `test/apis` with a 30-minute timeout. In OpenShift CI, when `OPENSHIFT_CI=true` and `ARTIFACT_DIR` is set, JUnit output and coverage are written to `ARTIFACT_DIR`.
+
+## Requirements
+
+- `make test-apis` pulls the correct Kubernetes version via `envtest`; ensure `make test-apis` (or `make test`) has been run at least once so envtest assets are present.
+
+## Relation to other tests
+
+- **Unit tests** (`make test-unit`): Pure Go tests excluding `test/e2e`, `test/apis`, and `test/utils`.
+- **E2e tests** (`make test-e2e`): Run against a real cluster; see [e2e/README.md](e2e/README.md).
+
+`make test` runs both unit and API tests (no cluster).

test/e2e/README.md

@@ -0,0 +1,146 @@
+# E2E Test Suites
+
+Run e2e tests from this repo’s root with `make test-e2e`. Use `E2E_GINKGO_LABEL_FILTER` to run a specific suite by label.
+
+---
+
+## Running tests
+
+From the repo root:
+
+```bash
+make test-e2e E2E_GINKGO_LABEL_FILTER="<label-filter>"
+```
+
+Default (if omitted): `E2E_GINKGO_LABEL_FILTER="Platform: isSubsetOf {AWS}"` (AWS-only).
+
+### Running the whole suite
+
+To run **all** e2e specs (every suite, regardless of label), pass an empty label filter:
+
+```bash
+make test-e2e E2E_GINKGO_LABEL_FILTER=""
+```
+
+Ensure pre-requisites for each suite you care about are in place (see [Suites by label](#suites-by-label)). Suites whose pre-requisites are not met will skip (e.g. missing `bitwarden-creds` or `aws-creds`). To run only a subset of suites, use a label filter (see [Running multiple suites](#running-multiple-suites)).
+
+### Failure artifacts (debugging)
+
+When a spec fails, the suite dumps logs and cluster state into **`<output-dir>/e2e-artifacts/failure-<timestamp>/`** so you can debug later. The output directory is **`ARTIFACT_DIR`** when set (e.g. in OpenShift CI); otherwise, when running locally via `make test-e2e`, it is **`_output`** at the repo root.
+
+Each failure dump includes:
+
+- **`pods/`** – Last 500 lines of logs per pod and `describe` (YAML) for operator, operand, and test namespaces.
+- **`events/`** – Recent events per namespace.
+- **`resources/`** – ExternalSecretsConfig (cluster), ClusterSecretStores, ExternalSecrets, and PushSecrets (YAML).
+
+JUnit and JSON reports are also written to the same output directory (see root README Testing section).
+
+---
+
+## Suites by label
+
+### Platform:AWS (AWS Secret Manager)
+
+| Item | Details |
+|------|--------|
+| **Label filter** | `"Platform: isSubsetOf {AWS}"` (default) |
+| **Pre-requisites** | Cluster on AWS (e.g. OpenShift on AWS); K8s secret `aws-creds` in namespace `kube-system` with keys `aws_access_key_id` and `aws_secret_access_key` (credentials for AWS Secrets Manager in `ap-south-1`). On OpenShift on AWS this secret is typically made available in `kube-system` by the platform. |
+| **Make command** | `make test-e2e` or `make test-e2e E2E_GINKGO_LABEL_FILTER="Platform: isSubsetOf {AWS}"` |
+
+---
+
+### Provider:Bitwarden (ESO CR–based Bitwarden)
+
+When this suite runs, it **enables Bitwarden in ExternalSecretsConfig** and **creates the TLS secret** (certificate for bitwarden-sdk-server) in the operand namespace. The default e2e cluster CR does not enable Bitwarden; enabling and the TLS secret are done only when tests with the Bitwarden label run. See `config/crd/bases/operator.openshift.io_externalsecretsconfigs.yaml` for `plugins.bitwardenSecretManagerProvider` (mode, secretRef).
+
+| Item | Details |
+|------|--------|
+| **Label filter** | `"Provider:Bitwarden"` |
+| **Pre-requisites** | ESO installed. Create the Bitwarden credentials secret **`bitwarden-creds`** in **`external-secrets-operator`** (see [Creating the Bitwarden credentials secret](#creating-the-bitwarden-credentials-secret)). The suite enables the plugin and creates the TLS secret. Optional env: `BITWARDEN_SDK_SERVER_URL` (default: `https://bitwarden-sdk-server.external-secrets.svc.cluster.local:9998`). |
+| **Make command** | `make test-e2e E2E_GINKGO_LABEL_FILTER="Provider:Bitwarden"` |
+
+---
+
+### API:Bitwarden (Direct HTTP to bitwarden-sdk-server)
+
+| Item | Details |
+|------|--------|
+| **Label filter** | `"API:Bitwarden"` |
+| **Pre-requisites** | bitwarden-sdk-server reachable (e.g. in-cluster). For Secrets API tests: create the Bitwarden credentials secret **`bitwarden-creds`** in **`external-secrets-operator`** (see [Creating the Bitwarden credentials secret](#creating-the-bitwarden-credentials-secret)). Optional env: `BITWARDEN_SDK_SERVER_URL` (default: `https://bitwarden-sdk-server.external-secrets.svc.cluster.local:9998`). |
+| **Make command** | `make test-e2e E2E_GINKGO_LABEL_FILTER="API:Bitwarden"` |
+
+#### Creating the Bitwarden credentials secret
+
+The Provider:Bitwarden and API:Bitwarden suites expect the secret to be named **`bitwarden-creds`** in namespace **`external-secrets-operator`**. The secret must have keys **`token`** (Bitwarden machine account access token), **`organization_id`**, and **`project_id`**.
+
+**From literal values:**
+
+```bash
+kubectl create secret generic bitwarden-creds \
+  --namespace=external-secrets-operator \
+  --from-literal=token="YOUR_BITWARDEN_ACCESS_TOKEN" \
+  --from-literal=organization_id="YOUR_ORGANIZATION_ID" \
+  --from-literal=project_id="YOUR_PROJECT_ID"
+```
+
+**From env vars (avoids secrets in shell history):**
+
+```bash
+export BITWARDEN_ACCESS_TOKEN="..."
+export BITWARDEN_ORGANIZATION_ID="..."
+export BITWARDEN_PROJECT_ID="..."
+kubectl create secret generic bitwarden-creds \
+  --namespace=external-secrets-operator \
+  --from-literal=token="${BITWARDEN_ACCESS_TOKEN}" \
+  --from-literal=organization_id="${BITWARDEN_ORGANIZATION_ID}" \
+  --from-literal=project_id="${BITWARDEN_PROJECT_ID}"
+```
+
+---
+
+### CrossPlatform:GCP-AWS (GCP cluster, AWS Secrets Manager)
+
+Cluster runs on **GCP**; the test uses a **ClusterSecretStore** backed by **AWS Secrets Manager**. You must create a Kubernetes secret that holds AWS credentials in a **fixed** name and namespace (see below).
+
+| Item | Details |
+|------|--------|
+| **Label filter** | `"CrossPlatform:GCP-AWS"` |
+| **Pre-requisites** | Cluster on GCP (or any non-AWS cluster). Create the AWS credentials secret with **name** `aws-creds` in **namespace** `kube-system` (see below). |
+| **Make command** | `make test-e2e E2E_GINKGO_LABEL_FILTER="CrossPlatform:GCP-AWS"` |
+
+#### Creating the AWS credentials secret
+
+The test expects the secret to be named **`aws-creds`** in namespace **`kube-system`**. The secret must have keys **`aws_access_key_id`** and **`aws_secret_access_key`**. The test uses AWS region `ap-south-1`.
+
+**From literal values:**
+
+```bash
+kubectl create secret generic aws-creds \
+  --namespace=kube-system \
+  --from-literal=aws_access_key_id="YOUR_AWS_ACCESS_KEY_ID" \
+  --from-literal=aws_secret_access_key="YOUR_AWS_SECRET_ACCESS_KEY"
+```
+
+**From env vars (avoids secrets in shell history):**
+
+```bash
+export AWS_ACCESS_KEY_ID="..."
+export AWS_SECRET_ACCESS_KEY="..."
+kubectl create secret generic aws-creds \
+  --namespace=kube-system \
+  --from-literal=aws_access_key_id="${AWS_ACCESS_KEY_ID}" \
+  --from-literal=aws_secret_access_key="${AWS_SECRET_ACCESS_KEY}"
+```
+
+---
+
+## Running multiple suites
+
+To run more than one label (e.g. Bitwarden provider and API):
+
+```bash
+make test-e2e E2E_GINKGO_LABEL_FILTER="Provider:Bitwarden || API:Bitwarden"
+```
+
+See [Ginkgo label documentation](https://onsi.github.io/ginkgo/#spec-labels) for label query syntax.

test/e2e/bitwarden_api_test.go

@@ -0,0 +1,146 @@
+//go:build e2e
+// +build e2e
+
+/*
+Copyright 2025.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+	http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package e2e
+
+import (
+	"context"
+	"fmt"
+	"strings"
+	"time"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+
+	"github.com/openshift/external-secrets-operator/test/utils"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"k8s.io/client-go/kubernetes"
+)
+
+const (
+	apiPath       = "/rest/api/1"
+	apiJobTimeout = 2 * time.Minute
+)
+
+// inClusterBaseURL is the Bitwarden SDK server URL as seen from inside the cluster (same as external-secrets controller). Respects BITWARDEN_SDK_SERVER_URL.
+var inClusterBaseURL = utils.GetBitwardenSDKServerURL()
+
+var _ = Describe("Bitwarden SDK Server API", Ordered, Label("API:Bitwarden", "Suite:Bitwarden"), func() {
+	var (
+		ctx       context.Context
+		clientset *kubernetes.Clientset
+		namespace string
+	)
+
+	BeforeAll(func() {
+		ctx = context.Background()
+		var err error
+		clientset = suiteClientset
+		Expect(clientset).NotTo(BeNil(), "suite clientset not initialized")
+
+		By("Ensuring Bitwarden operand is ready (enable plugin, wait for server)")
+		Expect(ensureBitwardenOperandReady(ctx)).To(Succeed())
+
+		credNamespace := utils.BitwardenCredSecretNamespace
+		_, err = clientset.CoreV1().Secrets(credNamespace).Get(ctx, utils.BitwardenCredSecretName, metav1.GetOptions{})
+		if err != nil {
+			Skip(fmt.Sprintf("Bitwarden credentials secret %s/%s required for API tests. See docs/e2e/README.md. Error: %v", credNamespace, utils.BitwardenCredSecretName, err))
+		}
+
+		// Run API test Jobs in the operand namespace (external-secrets) so they can reach bitwarden-sdk-server
+		// (same network as the controller). Copy the cred secret there so the Job can mount it.
+		namespace = utils.BitwardenOperandNamespace
+		By("Copying Bitwarden cred secret to " + namespace + " for API test Jobs")
+		Expect(utils.CopySecretToNamespace(ctx, clientset, utils.BitwardenCredSecretName, credNamespace, utils.BitwardenCredSecretName, namespace)).To(Succeed())
+	})
+
+	runAPIJob := func(jobName, script string) (int, string) {
+		code, logs, err := utils.RunBitwardenAPIJob(ctx, clientset, namespace, jobName, []string{"sh", "-c", script}, apiJobTimeout)
+		Expect(err).NotTo(HaveOccurred(), "job %s: %s", jobName, logs)
+		return code, logs
+	}
+
+	Context("Health", func() {
+		It("GET /ready should return 200 with body ready", func() {
+			script := fmt.Sprintf("code=$(curl -k -s -o /tmp/out -w '%%{http_code}' %s/ready) && body=$(cat /tmp/out) && [ \"$code\" = \"200\" ] && [ \"$body\" = \"ready\" ] || (echo \"code=$code body=$body\"; exit 1)", inClusterBaseURL)
+			code, logs := runAPIJob("api-ready-"+utils.GetRandomString(5), script)
+			Expect(code).To(Equal(0), "expected exit 0: %s", logs)
+		})
+
+		It("GET /live should return 200 with body live", func() {
+			script := fmt.Sprintf("code=$(curl -k -s -o /tmp/out -w '%%{http_code}' %s/live) && body=$(cat /tmp/out) && [ \"$code\" = \"200\" ] && [ \"$body\" = \"live\" ] || (echo \"code=$code body=$body\"; exit 1)", inClusterBaseURL)
+			code, logs := runAPIJob("api-live-"+utils.GetRandomString(5), script)
+			Expect(code).To(Equal(0), "expected exit 0: %s", logs)
+		})
+	})
+
+	Context("Auth", func() {
+		It("request without Warden-Access-Token should return 401", func() {
+			script := fmt.Sprintf("code=$(curl -k -s -o /dev/null -w '%%{http_code}' -X GET -H 'Content-Type: application/json' -d '{\"organizationId\":\"00000000-0000-0000-0000-000000000000\"}' %s%s/secrets) && [ \"$code\" = \"401\" ] || (echo \"code=$code\"; exit 1)", inClusterBaseURL, apiPath)
+			code, logs := runAPIJob("api-auth-no-token-"+utils.GetRandomString(5), script)
+			Expect(code).To(Equal(0), "expected exit 0: %s", logs)
+		})
+
+		It("request with invalid token should return 400", func() {
+			script := fmt.Sprintf("code=$(curl -k -s -o /dev/null -w '%%{http_code}' -X GET -H 'Content-Type: application/json' -H 'Warden-Access-Token: invalid-token' -d '{\"organizationId\":\"00000000-0000-0000-0000-000000000000\"}' %s%s/secrets) && [ \"$code\" = \"400\" ] || (echo \"code=$code\"; exit 1)", inClusterBaseURL, apiPath)
+			code, logs := runAPIJob("api-auth-invalid-token-"+utils.GetRandomString(5), script)
+			Expect(code).To(Equal(0), "expected exit 0: %s", logs)
+		})
+	})
+
+	Context("Secrets API", func() {
+		It("GET /rest/api/1/secrets (ListSecrets) should return 200 with data array", func() {
+			credPath := utils.BitwardenCredMountPath()
+			script := fmt.Sprintf("TOKEN=$(cat %s/token) && ORG=$(cat %s/organization_id) && code=$(curl -k -s -o /tmp/out -w '%%{http_code}' -X GET -H 'Content-Type: application/json' -H \"Warden-Access-Token: $TOKEN\" -d \"{\\\"organizationId\\\":\\\"$ORG\\\"}\" %s%s/secrets) && [ \"$code\" = \"200\" ] && grep -q '\"data\"' /tmp/out || (echo \"code=$code\"; cat /tmp/out; exit 1)", credPath, credPath, inClusterBaseURL, apiPath)
+			code, logs := runAPIJob("api-list-secrets-"+utils.GetRandomString(5), script)
+			Expect(code).To(Equal(0), "expected exit 0: %s", logs)
+		})
+
+		It("POST /rest/api/1/secret (CreateSecret) then GET and DELETE", func() {
+			credPath := utils.BitwardenCredMountPath()
+			// Create, extract id, Get, Update, Delete; each step must succeed (exit 0).
+			// Bitwarden Secrets Manager requires projectIds when creating a secret.
+			script := fmt.Sprintf(`
+TOKEN=$(cat %s/token) && ORG=$(cat %s/organization_id) && PROJECT=$(cat %s/project_id) && BASE=%s
+BODY="{\"key\":\"e2e-api-crud\",\"value\":\"v1\",\"note\":\"e2e\",\"organizationId\":\"$ORG\",\"projectIds\":[\"$PROJECT\"]}"
+code=$(curl -k -s -o /tmp/create.json -w '%%{http_code}' -X POST -H 'Content-Type: application/json' -H "Warden-Access-Token: $TOKEN" -d "$BODY" "$BASE%s/secret") && [ "$code" = "200" ] || (echo "create code=$code"; exit 1)
+id=$(grep -o '"id":"[^"]*"' /tmp/create.json | head -1 | cut -d'"' -f4) && [ -n "$id" ] || (echo "no id"; exit 1)
+code=$(curl -k -s -o /tmp/get.json -w '%%{http_code}' -X GET -H 'Content-Type: application/json' -H "Warden-Access-Token: $TOKEN" -d "{\"id\":\"$id\"}" "$BASE%s/secret") && [ "$code" = "200" ] || (echo "get code=$code"; exit 1)
+code=$(curl -k -s -o /dev/null -w '%%{http_code}' -X PUT -H 'Content-Type: application/json' -H "Warden-Access-Token: $TOKEN" -d "{\"id\":\"$id\",\"key\":\"e2e-api-crud\",\"value\":\"v2\",\"note\":\"updated\",\"organizationId\":\"$ORG\",\"projectIds\":[\"$PROJECT\"]}" "$BASE%s/secret") && [ "$code" = "200" ] || (echo "update code=$code"; exit 1)
+code=$(curl -k -s -o /dev/null -w '%%{http_code}' -X DELETE -H 'Content-Type: application/json' -H "Warden-Access-Token: $TOKEN" -d "{\"ids\":[\"$id\"]}" "$BASE%s/secret") && [ "$code" = "200" ] || (echo "delete code=$code"; exit 1)
+`, credPath, credPath, credPath, inClusterBaseURL, apiPath, apiPath, apiPath, apiPath)
+			code, logs := runAPIJob("api-crud-"+utils.GetRandomString(5), strings.TrimSpace(script))
+			Expect(code).To(Equal(0), "expected exit 0: %s", logs)
+		})
+
+		It("GET /rest/api/1/secret with non-existent ID should return 400", func() {
+			credPath := utils.BitwardenCredMountPath()
+			script := fmt.Sprintf("TOKEN=$(cat %s/token) && code=$(curl -k -s -o /dev/null -w '%%{http_code}' -X GET -H 'Content-Type: application/json' -H \"Warden-Access-Token: $TOKEN\" -d '{\"id\":\"00000000-0000-0000-0000-000000000000\"}' %s%s/secret) && [ \"$code\" = \"400\" ] || (echo \"code=$code\"; exit 1)", credPath, inClusterBaseURL, apiPath)
+			code, logs := runAPIJob("api-get-nonexistent-"+utils.GetRandomString(5), script)
+			Expect(code).To(Equal(0), "expected exit 0: %s", logs)
+		})
+
+		It("GET /rest/api/1/secrets-by-ids with empty ids should return 200 or 400", func() {
+			credPath := utils.BitwardenCredMountPath()
+			script := fmt.Sprintf("TOKEN=$(cat %s/token) && code=$(curl -k -s -o /dev/null -w '%%{http_code}' -X GET -H 'Content-Type: application/json' -H \"Warden-Access-Token: $TOKEN\" -d '{\"ids\":[]}' %s%s/secrets-by-ids) && ( [ \"$code\" = \"200\" ] || [ \"$code\" = \"400\" ] ) || (echo \"code=$code\"; exit 1)", credPath, inClusterBaseURL, apiPath)
+			code, logs := runAPIJob("api-secrets-by-ids-empty-"+utils.GetRandomString(5), script)
+			Expect(code).To(Equal(0), "expected exit 0: %s", logs)
+		})
+	})
+})