Skip to content

test(bdd): prepare named Helmfile environments #1101

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sbaum1994 merged 1 commit into from
Aug 23, 2026
Merged

test(bdd): prepare named Helmfile environments #1101

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions tests/bdd/PLAN.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,7 @@ refactor in every consumer; that is a feature.
|------|-------|
| `And I copy the file {string} to {string}` | Both paths are repo-relative. |
| `And I update yaml file {string} with keys:` (two-column table of dotted-path and value) | Path supports dotted notation and `[n]` indices (e.g. `global.imagePullSecrets[0].name`). Missing intermediate maps and missing list indices are upserted: writing `global.imagePullSecrets[0].name` against a file that has neither `global.imagePullSecrets` nor any list entry creates both. Existing scalars at intermediate positions cause the step to fail rather than silently overwrite a non-map. Value cells expand `${VAR}` from `os.Environ`. |
| `And I prepare Helmfile environment {string} for stack {string} from fixture {string} with values:` (two-column table of dotted-path and value) | Validates the stack and environment names, derives `deploy/stacks/<stack>/environments/<environment>.yaml` from the absolute repository root, copies the explicit fixture, and applies the visible values table with the same YAML update and `${VAR}` interpolation behavior. Supported stacks are `self-managed`, `observability`, and `nvcf-compute-plane`. The destination is ledger-backed. |
| `And I prepare self-managed secrets file {string} from template {string} using the current NGC registry credential` | The destination and template are explicit repo-relative paths with `${VAR}` interpolation. Replaces the template's registry credential placeholder with base64 of the current `$oauthtoken:<NGC_API_KEY>` credential and writes the destination with mode `0600`. The destination is ledger-backed, and secret material never enters Gherkin, command logs, or failure messages. |
| `And I substitute a block in file {string}:` (docstring) | The docstring contains an old block and replacement block separated by exactly one `---` line. `${VAR}` interpolation applies before an exact, ledger-backed replacement. Missing or malformed old blocks fail. |

Expand Down
48 changes: 48 additions & 0 deletions tests/bdd/dsl/helmfile_environment.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
/*
SPDX-FileCopyrightText: Copyright (c) NVIDIA CORPORATION & AFFILIATES. All rights reserved.
SPDX-License-Identifier: Apache-2.0

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package dsl

import (
"fmt"
"path/filepath"
"regexp"
)

var helmfileEnvironmentName = regexp.MustCompile(`^[a-z0-9](?:[a-z0-9-]*[a-z0-9])?$`)

var helmfileStacks = map[string]struct{}{
"nvcf-compute-plane": {},
"observability": {},
"self-managed": {},
}

// HelmfileEnvironmentPath validates a named stack environment and returns its
// destination beneath an absolute repository root. It never depends on the
// process working directory.
func HelmfileEnvironmentPath(repoRoot, stack, environment string) (string, error) {
if !filepath.IsAbs(repoRoot) {
return "", fmt.Errorf("repository root must be absolute")
}
if _, ok := helmfileStacks[stack]; !ok {
return "", fmt.Errorf("unsupported Helmfile stack %q", stack)
}
if !helmfileEnvironmentName.MatchString(environment) {
return "", fmt.Errorf("invalid Helmfile environment name %q", environment)
}
return filepath.Join(repoRoot, "deploy", "stacks", stack, "environments", environment+".yaml"), nil
}
65 changes: 65 additions & 0 deletions tests/bdd/dsl/helmfile_environment_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
/*
SPDX-FileCopyrightText: Copyright (c) NVIDIA CORPORATION & AFFILIATES. All rights reserved.
SPDX-License-Identifier: Apache-2.0

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package dsl

import (
"path/filepath"
"strings"
"testing"
)

func TestHelmfileEnvironmentPathSupportsKnownStacks(t *testing.T) {
repoRoot := t.TempDir()
for _, stack := range []string{"self-managed", "observability", "nvcf-compute-plane"} {
t.Run(stack, func(t *testing.T) {
got, err := HelmfileEnvironmentPath(repoRoot, stack, "local-bdd")
if err != nil {
t.Fatalf("environment path: %v", err)
}
want := filepath.Join(repoRoot, "deploy", "stacks", stack, "environments", "local-bdd.yaml")
if got != want {
t.Fatalf("path = %q, want %q", got, want)
}
})
}
}

func TestHelmfileEnvironmentPathRejectsInvalidInput(t *testing.T) {
repoRoot := t.TempDir()
tests := []struct {
name string
root string
stack string
environment string
want string
}{
{name: "relative root", root: "repo", stack: "self-managed", environment: "local", want: "repository root must be absolute"},
{name: "unknown stack", root: repoRoot, stack: "other", environment: "local", want: `unsupported Helmfile stack "other"`},
{name: "empty environment", root: repoRoot, stack: "self-managed", environment: "", want: `invalid Helmfile environment name ""`},
{name: "path traversal", root: repoRoot, stack: "self-managed", environment: "../local", want: `invalid Helmfile environment name "../local"`},
{name: "path separator", root: repoRoot, stack: "self-managed", environment: "team/local", want: `invalid Helmfile environment name "team/local"`},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
_, err := HelmfileEnvironmentPath(tc.root, tc.stack, tc.environment)
if err == nil || !strings.Contains(err.Error(), tc.want) {
t.Fatalf("err = %v, want containing %q", err, tc.want)
}
})
}
}
6 changes: 2 additions & 4 deletions tests/bdd/features/multi-cluster-eks-helmfile.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -132,8 +132,7 @@ Feature: Install a multi-cluster NVCF stack across two pre-provisioned EKS clust
# the agent config. The agent dials the bare-ELB service URLs
# (which DNS-resolve) and sends these hostnames as the HTTP Host
# header so the control-plane gateway HTTPRoutes match.
When I copy the file "deploy/stacks/self-managed/environments/base.yaml" to "deploy/stacks/self-managed/environments/eks-bdd-multi.yaml"
And I update yaml file "deploy/stacks/self-managed/environments/eks-bdd-multi.yaml" with keys:
When I prepare Helmfile environment "eks-bdd-multi" for stack "self-managed" from fixture "deploy/stacks/self-managed/environments/base.yaml" with values:
| global.helm.sources.repository | ${SAMPLE_NGC_ORG}/${SAMPLE_NGC_TEAM} |
| global.image.repository | ${SAMPLE_NGC_ORG}/${SAMPLE_NGC_TEAM} |
| global.imagePullSecrets[0].name | nvcr-pull-secret |
Expand Down Expand Up @@ -164,8 +163,7 @@ Feature: Install a multi-cluster NVCF stack across two pre-provisioned EKS clust
| openbao.migrations.issuerDiscovery.enabled | true |
Then yaml file "deploy/stacks/self-managed/environments/eks-bdd-multi.yaml" key "global.domain" should equal "${EKS_GATEWAY_DOMAIN}"

When I copy the file "deploy/stacks/nvcf-compute-plane/environments/base.yaml" to "deploy/stacks/nvcf-compute-plane/environments/eks-bdd-multi.yaml"
And I update yaml file "deploy/stacks/nvcf-compute-plane/environments/eks-bdd-multi.yaml" with keys:
When I prepare Helmfile environment "eks-bdd-multi" for stack "nvcf-compute-plane" from fixture "deploy/stacks/nvcf-compute-plane/environments/base.yaml" with values:
| global.helm.sources.repository | ${SAMPLE_NGC_ORG}/${SAMPLE_NGC_TEAM} |
| global.image.repository | ${SAMPLE_NGC_ORG}/${SAMPLE_NGC_TEAM} |
| global.imagePullSecrets[0].name | nvcr-pull-secret |
Expand Down
6 changes: 2 additions & 4 deletions tests/bdd/features/multi-cluster-helmfile.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,15 +37,13 @@ Feature: Install a local multi-cluster NVCF stack with Helmfile
# operator-specific registry values before the first Helmfile
# install. Later scenarios reuse that install instead of
# reinstalling with different secrets or URLs.
And I copy the file "tests/bdd/fixtures/self-managed-local-bdd-multi.yaml" to "deploy/stacks/self-managed/environments/local-bdd.yaml"
And I update yaml file "deploy/stacks/self-managed/environments/local-bdd.yaml" with keys:
And I prepare Helmfile environment "local-bdd" for stack "self-managed" from fixture "tests/bdd/fixtures/self-managed-local-bdd-multi.yaml" with values:
| global.imagePullSecrets[0].name | nvcr-pull-secret |
| global.helm.sources.repository | ${SAMPLE_NGC_ORG}/${SAMPLE_NGC_TEAM} |
| global.image.repository | ${SAMPLE_NGC_ORG}/${SAMPLE_NGC_TEAM} |
| api.env.NVCF_SIDECARS_LLM_ROUTER_CLIENT_IMAGE | nvcr.io/${SAMPLE_NGC_ORG}/${SAMPLE_NGC_TEAM}/stargate-client:0.2.0 |
| observability.profile | disabled |
And I copy the file "tests/bdd/fixtures/nvcf-compute-plane-local-bdd-multi.yaml" to "deploy/stacks/nvcf-compute-plane/environments/local-bdd.yaml"
And I update yaml file "deploy/stacks/nvcf-compute-plane/environments/local-bdd.yaml" with keys:
And I prepare Helmfile environment "local-bdd" for stack "nvcf-compute-plane" from fixture "tests/bdd/fixtures/nvcf-compute-plane-local-bdd-multi.yaml" with values:
| global.imagePullSecrets[0].name | nvcr-pull-secret |
| global.helm.sources.repository | ${SAMPLE_NGC_ORG}/${SAMPLE_NGC_TEAM} |
| global.image.repository | ${SAMPLE_NGC_ORG}/${SAMPLE_NGC_TEAM} |
Expand Down
6 changes: 2 additions & 4 deletions tests/bdd/features/multi-cluster-up.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,14 +38,12 @@ Feature: Bring up a local multi-cluster NVCF stack with the CLI
# on a fresh cluster the ServiceMonitor CRDs do not exist yet and
# the diff fails before anything installs. The Helmfile workflow
# (helmfile sync) has no diff phase and keeps the default profile.
And I copy the file "tests/bdd/fixtures/self-managed-local-bdd-multi.yaml" to "deploy/stacks/self-managed/environments/local.yaml"
And I update yaml file "deploy/stacks/self-managed/environments/local.yaml" with keys:
And I prepare Helmfile environment "local" for stack "self-managed" from fixture "tests/bdd/fixtures/self-managed-local-bdd-multi.yaml" with values:
| global.imagePullSecrets[0].name | nvcr-pull-secret |
| global.helm.sources.repository | ${SAMPLE_NGC_ORG}/${SAMPLE_NGC_TEAM} |
| global.image.repository | ${SAMPLE_NGC_ORG}/${SAMPLE_NGC_TEAM} |
| observability.profile | disabled |
And I copy the file "tests/bdd/fixtures/nvcf-compute-plane-local-bdd-multi.yaml" to "deploy/stacks/nvcf-compute-plane/environments/local.yaml"
And I update yaml file "deploy/stacks/nvcf-compute-plane/environments/local.yaml" with keys:
And I prepare Helmfile environment "local" for stack "nvcf-compute-plane" from fixture "tests/bdd/fixtures/nvcf-compute-plane-local-bdd-multi.yaml" with values:
| global.imagePullSecrets[0].name | nvcr-pull-secret |
| global.helm.sources.repository | ${SAMPLE_NGC_ORG}/${SAMPLE_NGC_TEAM} |
| global.image.repository | ${SAMPLE_NGC_ORG}/${SAMPLE_NGC_TEAM} |
Expand Down
9 changes: 3 additions & 6 deletions tests/bdd/features/observability-all.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,23 +20,20 @@ Feature: Install local Helmfile observability for both planes
bash -c 'set -eo pipefail; printf %s "$NGC_API_KEY" | helm registry login nvcr.io --username "\$oauthtoken" --password-stdin'
"""
# Configure the control-plane stack and its shared observability child.
And I copy the file "tests/bdd/fixtures/self-managed-local-bdd.yaml" to "deploy/stacks/self-managed/environments/local-bdd-observability-all.yaml"
And I update yaml file "deploy/stacks/self-managed/environments/local-bdd-observability-all.yaml" with keys:
And I prepare Helmfile environment "local-bdd-observability-all" for stack "self-managed" from fixture "tests/bdd/fixtures/self-managed-local-bdd.yaml" with values:
| global.imagePullSecrets[0].name | nvcr-pull-secret |
| global.helm.sources.repository | ${SAMPLE_NGC_ORG}/${SAMPLE_NGC_TEAM} |
| global.image.repository | ${SAMPLE_NGC_ORG}/${SAMPLE_NGC_TEAM} |
| observability.profile | all |
| functionAutoscaler.image.tag | 1.18.10 |
# Give the shared observability Helmfile the same named environment.
And I copy the file "tests/bdd/fixtures/self-managed-local-bdd.yaml" to "deploy/stacks/observability/environments/local-bdd-observability-all.yaml"
And I update yaml file "deploy/stacks/observability/environments/local-bdd-observability-all.yaml" with keys:
And I prepare Helmfile environment "local-bdd-observability-all" for stack "observability" from fixture "tests/bdd/fixtures/self-managed-local-bdd.yaml" with values:
| global.imagePullSecrets[0].name | nvcr-pull-secret |
| global.helm.sources.repository | ${SAMPLE_NGC_ORG}/${SAMPLE_NGC_TEAM} |
| global.image.repository | ${SAMPLE_NGC_ORG}/${SAMPLE_NGC_TEAM} |
| observability.profile | all |
# Configure NVCA to join the same cluster and enable its collector.
And I copy the file "tests/bdd/fixtures/nvcf-compute-plane-local-bdd.yaml" to "deploy/stacks/nvcf-compute-plane/environments/local-bdd-observability-all.yaml"
And I update yaml file "deploy/stacks/nvcf-compute-plane/environments/local-bdd-observability-all.yaml" with keys:
And I prepare Helmfile environment "local-bdd-observability-all" for stack "nvcf-compute-plane" from fixture "tests/bdd/fixtures/nvcf-compute-plane-local-bdd.yaml" with values:
| global.imagePullSecrets[0].name | nvcr-pull-secret |
| global.helm.sources.repository | ${SAMPLE_NGC_ORG}/${SAMPLE_NGC_TEAM} |
| global.image.repository | ${SAMPLE_NGC_ORG}/${SAMPLE_NGC_TEAM} |
Expand Down
9 changes: 3 additions & 6 deletions tests/bdd/features/observability-compute.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,23 +22,20 @@ Feature: Install local Helmfile observability with the compute profile
"""
# Install only control-plane prerequisites on ncp-local-cp. Shared
# observability is installed separately on the compute cluster below.
And I copy the file "tests/bdd/fixtures/self-managed-local-bdd-multi.yaml" to "deploy/stacks/self-managed/environments/local-bdd-observability-compute.yaml"
And I update yaml file "deploy/stacks/self-managed/environments/local-bdd-observability-compute.yaml" with keys:
And I prepare Helmfile environment "local-bdd-observability-compute" for stack "self-managed" from fixture "tests/bdd/fixtures/self-managed-local-bdd-multi.yaml" with values:
| global.imagePullSecrets[0].name | nvcr-pull-secret |
| global.helm.sources.repository | ${SAMPLE_NGC_ORG}/${SAMPLE_NGC_TEAM} |
| global.image.repository | ${SAMPLE_NGC_ORG}/${SAMPLE_NGC_TEAM} |
| addons.llm.enabled | false |
| observability.profile | disabled |
# Configure the shared observability stack for compute-plane monitors.
And I copy the file "tests/bdd/fixtures/self-managed-local-bdd-multi.yaml" to "deploy/stacks/observability/environments/local-bdd-observability-compute.yaml"
And I update yaml file "deploy/stacks/observability/environments/local-bdd-observability-compute.yaml" with keys:
And I prepare Helmfile environment "local-bdd-observability-compute" for stack "observability" from fixture "tests/bdd/fixtures/self-managed-local-bdd-multi.yaml" with values:
| global.imagePullSecrets[0].name | nvcr-pull-secret |
| global.helm.sources.repository | ${SAMPLE_NGC_ORG}/${SAMPLE_NGC_TEAM} |
| global.image.repository | ${SAMPLE_NGC_ORG}/${SAMPLE_NGC_TEAM} |
| observability.profile | compute |
# Configure NVCA to use the same compute observability profile.
And I copy the file "tests/bdd/fixtures/nvcf-compute-plane-local-bdd-multi.yaml" to "deploy/stacks/nvcf-compute-plane/environments/local-bdd-observability-compute.yaml"
And I update yaml file "deploy/stacks/nvcf-compute-plane/environments/local-bdd-observability-compute.yaml" with keys:
And I prepare Helmfile environment "local-bdd-observability-compute" for stack "nvcf-compute-plane" from fixture "tests/bdd/fixtures/nvcf-compute-plane-local-bdd-multi.yaml" with values:
| global.imagePullSecrets[0].name | nvcr-pull-secret |
| global.helm.sources.repository | ${SAMPLE_NGC_ORG}/${SAMPLE_NGC_TEAM} |
| global.image.repository | ${SAMPLE_NGC_ORG}/${SAMPLE_NGC_TEAM} |
Expand Down
6 changes: 2 additions & 4 deletions tests/bdd/features/observability-control.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,16 +17,14 @@ Feature: Install local Helmfile observability with the control profile
bash -c 'set -eo pipefail; printf %s "$NGC_API_KEY" | helm registry login nvcr.io --username "\$oauthtoken" --password-stdin'
"""
# Set the self-managed stack environment.
And I copy the file "tests/bdd/fixtures/self-managed-local-bdd.yaml" to "deploy/stacks/self-managed/environments/local-bdd-observability-control.yaml"
And I update yaml file "deploy/stacks/self-managed/environments/local-bdd-observability-control.yaml" with keys:
And I prepare Helmfile environment "local-bdd-observability-control" for stack "self-managed" from fixture "tests/bdd/fixtures/self-managed-local-bdd.yaml" with values:
| global.imagePullSecrets[0].name | nvcr-pull-secret |
| global.helm.sources.repository | ${SAMPLE_NGC_ORG}/${SAMPLE_NGC_TEAM} |
| global.image.repository | ${SAMPLE_NGC_ORG}/${SAMPLE_NGC_TEAM} |
| observability.profile | control |
| functionAutoscaler.image.tag | 1.18.10 |
# Set the shared observability stack environment.
And I copy the file "tests/bdd/fixtures/self-managed-local-bdd.yaml" to "deploy/stacks/observability/environments/local-bdd-observability-control.yaml"
And I update yaml file "deploy/stacks/observability/environments/local-bdd-observability-control.yaml" with keys:
And I prepare Helmfile environment "local-bdd-observability-control" for stack "observability" from fixture "tests/bdd/fixtures/self-managed-local-bdd.yaml" with values:
| global.imagePullSecrets[0].name | nvcr-pull-secret |
| global.helm.sources.repository | ${SAMPLE_NGC_ORG}/${SAMPLE_NGC_TEAM} |
| global.image.repository | ${SAMPLE_NGC_ORG}/${SAMPLE_NGC_TEAM} |
Expand Down
Loading
Loading