chart-test-swarm — Recommendations

open schema-missing low Schema or values configuration missing for scenario 'c3fail-deliberate'

Show details

Scenario: c3fail-deliberate

Detail:

Assert 'helm-status-deployed': jq: parse error: Invalid numeric literal at line 1, column 6

Run history: run-20260531-230537

open schema-missing low Schema or values configuration missing for scenario 'c3sig-slow'

Show details

Scenario: c3sig-slow

Detail:

Failure message: see examples/sample-product-chart/chart-test/reports/scenario-c3sig-slow-20260529-150454/artifacts/logs/cluster-up.log

Run history: run-20260531-230537

open schema-missing low Schema or values configuration missing for scenario 'ingress-controllers-contour-rate-limit'

Show details

Scenario: ingress-controllers-contour-rate-limit

Detail:

Assert 'smoke-script': exec: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/assertions/contour-rate-limit.sh (RELEASE=sample NAMESPACE=sample)
==> Waiting for envoy pod Ready (3m max)
error: no matching resources found

Run history: run-20260531-230537

open schema-missing low Schema or values configuration missing for scenario 'service-mesh-istio-service-mesh-sidecar-injection'

Show details

Scenario: service-mesh-istio-service-mesh-sidecar-injection

Detail:

Assert 'smoke-script': exec: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/assertions/istio-sidecar-injection.sh (RELEASE=sample NAMESPACE=sample)
==> Labeling namespace sample for istio injection
namespace/sample labeled
==> Restarting deployments to pick up sidecar injection
  Restarting deployment/sample
deployment.apps/sample restarted
  Waiting for rollout of deployment/sample to complete (3m max)
Waiting for deployment "sample" rollout to finish: 1 old replicas are pending termination...
Waiting for deployment "sample" rollout to finish: 1 old replicas are pending termination...
deployment "sample" successfully rolled out
PASS: all deployments rolled out
==> Waiting 5s for old pods to terminate
==> Verifying istio-proxy sidecar injected into every product pod
  Pod sample-cff994cb6-tjl8j (deploy/sample) containers (2): app istio-proxy
    ✓ istio-proxy sidecar present
PASS: every Running product pod has exactly 2 containers including istio-proxy
==> Probing product Service over the mesh from an in-mesh pod
HTTP response from in-mesh probe: 000
FAIL: expected HTTP 200 from in-mesh probe, got 000

Affected objects: Pod, Service

Run history: run-20260531-230537

open schema-missing medium Schema or values configuration missing for scenario 'labels-on'

Show details

Scenario: labels-on

Detail:

Failure message: result.yaml had parse errors; status from aggregate

open schema-missing medium Schema or values configuration missing for scenario 'labels-on'

Show details

Scenario: labels-on

Detail:

Failure message: helm install failed; see /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-bats-f17-3-containers-34814/scenario-labels-on-20260603-011404/artifacts/logs/product-install.log

Run history: run-bats-f17-3-containers-34814, run-bats-f17-3-single-7254

open schema-missing medium Schema or values configuration missing for scenario 'labels-on'

Show details

Scenario: labels-on

Detail:

Failure message: see /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-bats-f17-3-no-cloud-48006/scenario-labels-on-20260603-011932/artifacts/logs/cluster-up.log

Run history: run-bats-f17-3-no-cloud-48006

open schema-missing medium Schema or values configuration missing for scenario 'annotations-on'

Show details

Scenario: annotations-on

Detail:

Failure message: result.yaml had parse errors; status from aggregate

Run history: run-curated-live-20260602-234624-95262, run-full-bench-20260609-085346

open chart-fix high Add RBAC objects (ServiceAccount / Role / RoleBinding) for scenario 'rbac-on'

Show details

Scenario: rbac-on

Detail:

Assert 'rbac-objects': FAIL: expected RBAC objects but found 0

Run history: run-curated-live-20260602-234624-95262, run-full-bench-20260609-085346

open schema-missing high Schema or values configuration missing for scenario 'scheme-https-only'

Show details

Scenario: scheme-https-only

Detail:

Failure message: result.yaml had parse errors; status from aggregate

Run history: run-curated-live-20260602-234624-95262, run-full-bench-20260609-085346

open gap-probe low Honest gap: chart does not natively support scenario 'ingress-controllers-contour-basic-httpproxy'

Show details

Scenario: ingress-controllers-contour-basic-httpproxy

Detail:

Assert 'smoke-script': exec: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/assertions/contour-basic.sh (RELEASE=sample NAMESPACE=sample)
==> Waiting for envoy pod Ready (3m max)
pod/contour-envoy-vxvcj condition met
PASS: envoy pod Ready
==> Waiting for contour controller pod Ready (3m max)
pod/contour-contour-5847f7c7f9-vxwnb condition met
PASS: contour controller pod Ready
==> Waiting for product pod Ready (3m max)
pod/sample-69469cb4c6-wr6ns condition met
PASS: product pods Ready
==> Getting envoy pod IP (probe via pod IP)
envoy pod IP: 10.244.0.7
==> Verifying HTTPProxy exists and status is valid
HTTPProxy status: valid
PASS: HTTPProxy status is valid
==> Probing HTTP with Host header (expect 200) on envoy container port 8080
HTTP response (with Host): 200
PASS: HTTP 200 with Host header
==> Probing HTTP without Host header (expect 404) on envoy container port 8080
HTTP response (without Host): 404
PASS: HTTP 404 without Host header (envoy default)
PASS: Contour basic HTTPProxy integration verified
==> GAP-PROBE: Checking if chart natively emits an HTTPProxy CRD
GAP-PROBE: No HTTPProxy emitted by the chart template found
  The HTTPProxy 'sample-basic' was created by the raw_manifest fixture,
  not by a Helm-managed chart template. The chart has no contour
  HTTPProxy template — this is an honest gap (red cell).
FAIL: Chart does not natively emit Contour HTTPProxy CRD — honest gap

Affected objects: HTTPProxy

Run history: run-curated-live-20260602-234624-95262, run-full-bench-20260609-085346

open gap-probe low Honest gap: chart does not natively support scenario 'policy-opa-gatekeeper-required-labels'

Show details

Scenario: policy-opa-gatekeeper-required-labels

Detail:

Assert 'smoke-script': PASS: compliant Deployment accepted
==> Phase 3a: Non-compliant Ingress (missing app.kubernetes.io/name)
Error from server (Forbidden): error when creating "/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/fixtures/policy/test-ingress-noncompliant.yaml": admission webhook "validation.gatekeeper.sh" denied the request: [require-app-name-label] Missing required label(s): {"app.kubernetes.io/name"}
PASS: non-compliant Ingress rejected (missing required label)
==> Phase 3b: Compliant Ingress (has app.kubernetes.io/name)
ingress.networking.k8s.io/test-compliant-ingress created
PASS: compliant Ingress applied
NOTE: compliant Ingress may not yet have address (NGINX controller processes async)
==> Phase 4: Webhook failure mode — verify Fail policy + outage behavior
PASS: webhook gatekeeper-validating-webhook-configuration failurePolicy=Fail
Scaling gatekeeper-controller-manager to 0 in gatekeeper-system (original replicas: 3)...
deployment.apps/gatekeeper-controller-manager scaled
pod/gatekeeper-controller-manager-64bbbfbdbb-bqbvq condition met
pod/gatekeeper-controller-manager-64bbbfbdbb-fjpnx condition met
pod/gatekeeper-controller-manager-64bbbfbdbb-xn75d condition met
PASS: gatekeeper controller scaled to 0
Attempting kubectl apply with webhook unavailable...
Error from server (InternalError): error when creating "/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/fixtures/policy/test-deploy-compliant.yaml": Internal error occurred: failed calling webhook "validation.gatekeeper.sh": failed to call webhook: Post "https://gatekeeper-webhook-service.gatekeeper-system.svc:443/v1/admit?timeout=3s": dial tcp 10.96.91.237:443: connect: connection refused
PASS: admission correctly blocked during webhook outage (Fail policy)
Restoring gatekeeper-controller-manager to 3 in gatekeeper-system...
deployment.apps/gatekeeper-controller-manager scaled
Waiting for gatekeeper controller pods to be Ready again...
pod/gatekeeper-controller-manager-64bbbfbdbb-cbpzc condition met
pod/gatekeeper-controller-manager-64bbbfbdbb-g6j4d condition met
pod/gatekeeper-controller-manager-64bbbfbdbb-ttx4k condition met
PASS: gatekeeper controller pods Ready
Waiting for admission webhook to accept requests (180s max, may be slow on resource-constrained clusters)...
deployment.apps/test-compliant created (server dry run)
PASS: admission restored after webhook recovery (attempt 1)
PASS: OPA Gatekeeper required-labels + nginx-ingress cross-feature compose verified
==> Phase 5: GAP-PROBE — Does the chart's own Ingress pass the required-labels constraint?
Rendering chart Ingress with ingress.enabled=true for gap-probe dry-run...
GAP-PROBE: Chart Ingress does NOT have app.kubernetes.io/name label
  The chart's Ingress template uses selectorLabels (just 'app: <release>')
  instead of the full common labels (which include 'app.kubernetes.io/name').
  This is an honest gap — the Ingress would be denied in a cluster with
  the K8sRequiredLabels constraint active before chart installation.
Error from server (Forbidden): error when creating "STDIN": admission webhook "validation.gatekeeper.sh" denied the request: [require-app-name-label] Missing required label(s): {"app.kubernetes.io/name"}
GAP-PROBE confirmed: Chart Ingress denied by required-labels constraint (honest gap, red cell)
FAIL: Chart Ingress does not carry app.kubernetes.io/name label — honest gap (red cell)

Affected objects: Deployment, Ingress

Run history: run-curated-live-20260602-234624-95262, run-full-bench-20260609-085346

open schema-missing low Schema or values configuration missing for scenario 'service-mesh-linkerd-live'

Show details

Scenario: service-mesh-linkerd-live

Detail:

Failure message: result.yaml had parse errors; status from aggregate

Run history: run-curated-live-20260602-234624-95262

open schema-missing low Schema or values configuration missing for scenario 'imagepullsecrets-on'

Show details

Scenario: imagepullsecrets-on

Detail:

Failure message: result.yaml had parse errors; status from aggregate

Run history: run-full-bench-20260609-085346

open chart-fix low Add NetworkPolicy templates to chart for scenario 'network-policy-on'

Show details

Scenario: network-policy-on

Detail:

Assert 'network-policy': FAIL: expected NetworkPolicy but found 0

Affected objects: NetworkPolicy

Run history: run-full-bench-20260609-085346

open schema-missing low Schema or values configuration missing for scenario 'priority-class-on'

Show details

Scenario: priority-class-on

Detail:

Failure message: result.yaml had parse errors; status from aggregate

Run history: run-full-bench-20260609-085346

open schema-missing low Schema or values configuration missing for scenario 'resources-on'

Show details

Scenario: resources-on

Detail:

Failure message: result.yaml had parse errors; status from aggregate

Run history: run-full-bench-20260609-085346

open schema-missing low Schema or values configuration missing for scenario 'scheduling-affinity-on'

Show details

Scenario: scheduling-affinity-on

Detail:

Failure message: result.yaml had parse errors; status from aggregate

Run history: run-full-bench-20260609-085346

open schema-missing low Schema or values configuration missing for scenario 'scheduling-nodeselector-on'

Show details

Scenario: scheduling-nodeselector-on

Detail:

Failure message: result.yaml had parse errors; status from aggregate

Run history: run-full-bench-20260609-085346

open schema-missing low Schema or values configuration missing for scenario 'scheduling-tolerations-on'

Show details

Scenario: scheduling-tolerations-on

Detail:

Failure message: result.yaml had parse errors; status from aggregate

Run history: run-full-bench-20260609-085346

open schema-missing low Schema or values configuration missing for scenario 'scheduling-topology-spread-on'

Show details

Scenario: scheduling-topology-spread-on

Detail:

Failure message: result.yaml had parse errors; status from aggregate

Run history: run-full-bench-20260609-085346

open schema-missing high Schema or values configuration missing for scenario 'security-context-on'

Show details

Scenario: security-context-on

Detail:

Failure message: result.yaml had parse errors; status from aggregate

Run history: run-full-bench-20260609-085346

open chart-fix medium Fix chart capability gap: serviceaccount-annotations in scenario 'serviceaccount-annotations-on'

Show details

Scenario: serviceaccount-annotations-on

Detail:

Assert 'serviceaccount-annotations': FAIL: no ServiceAccount found in rendered output — chart lacks ServiceAccount template (documented gap)

Affected objects: Service, ServiceAccount

Run history: run-full-bench-20260609-085346

open schema-missing low Schema or values configuration missing for scenario 'service-mesh-istio-ingress-gateway-multi-host'

Show details

Scenario: service-mesh-istio-ingress-gateway-multi-host

Detail:

Failure message: helm install failed; see /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-service-mesh-istio-ingress-gateway-multi-host-20260609-102026/artifacts/logs/product-install.log

Run history: run-full-bench-20260609-085346

open schema-missing low Schema or values configuration missing for scenario 'service-mesh-istio-ingress-gateway-request-authentication'

Show details

Scenario: service-mesh-istio-ingress-gateway-request-authentication

Detail:

Failure message: helm install failed; see /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-service-mesh-istio-ingress-gateway-request-authentication-20260609-102606/artifacts/logs/product-install.log

Run history: run-full-bench-20260609-085346

open schema-missing low Schema or values configuration missing for scenario 'service-mesh-istio-service-mesh-cert-manager-tls'

Show details

Scenario: service-mesh-istio-service-mesh-cert-manager-tls

Detail:

Failure message: helm install failed; see /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-service-mesh-istio-service-mesh-cert-manager-tls-20260609-103144/artifacts/logs/product-install.log

Run history: run-full-bench-20260609-085346

open schema-missing low Schema or values configuration missing for scenario 'service-mesh-istio-service-mesh-peer-authentication'

Show details

Scenario: service-mesh-istio-service-mesh-peer-authentication

Detail:

Failure message: helm install failed; see /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-service-mesh-istio-service-mesh-peer-authentication-20260609-103742/artifacts/logs/product-install.log

Run history: run-full-bench-20260609-085346

open schema-missing low Schema or values configuration missing for scenario 'service-mesh-istio-service-mesh-sidecar-injection'

Show details

Scenario: service-mesh-istio-service-mesh-sidecar-injection

Detail:

Failure message: helm install failed; see /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-service-mesh-istio-service-mesh-sidecar-injection-20260609-104316/artifacts/logs/product-install.log

Run history: run-full-bench-20260609-085346

open schema-missing low Schema or values configuration missing for scenario 'service-mesh-istio-service-mesh-strict-mtls'

Show details

Scenario: service-mesh-istio-service-mesh-strict-mtls

Detail:

Failure message: helm install failed; see /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-service-mesh-istio-service-mesh-strict-mtls-20260609-104851/artifacts/logs/product-install.log

Run history: run-full-bench-20260609-085346

open schema-missing low Schema or values configuration missing for scenario 'service-mesh-istio-service-mesh-telemetry-v2'

Show details

Scenario: service-mesh-istio-service-mesh-telemetry-v2

Detail:

Failure message: helm install failed; see /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-service-mesh-istio-service-mesh-telemetry-v2-20260609-105426/artifacts/logs/product-install.log

Run history: run-full-bench-20260609-085346

open schema-missing low Schema or values configuration missing for scenario 'service-mesh-istio-sidecar-live'

Show details

Scenario: service-mesh-istio-sidecar-live

Detail:

Failure message: helm install failed; see /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-service-mesh-istio-sidecar-live-20260609-110001/artifacts/logs/product-install.log

Run history: run-full-bench-20260609-085346

open schema-missing low Schema or values configuration missing for scenario 'service-mesh-linkerd-basic-mesh'

Show details

Scenario: service-mesh-linkerd-basic-mesh

Detail:

Failure message: helm install failed; see /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-service-mesh-linkerd-basic-mesh-20260609-110535/artifacts/logs/product-install.log

Run history: run-full-bench-20260609-085346

open schema-missing low Schema or values configuration missing for scenario 'service-mesh-linkerd-live'

Show details

Scenario: service-mesh-linkerd-live

Detail:

Failure message: helm install failed; see /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-service-mesh-linkerd-live-20260609-111107/artifacts/logs/product-install.log

Run history: run-full-bench-20260609-085346

open schema-missing low Schema or values configuration missing for scenario 'service-mesh-linkerd-mtls-rotation'

Show details

Scenario: service-mesh-linkerd-mtls-rotation

Detail:

Failure message: helm install failed; see /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-service-mesh-linkerd-mtls-rotation-20260609-111638/artifacts/logs/product-install.log

Run history: run-full-bench-20260609-085346

open schema-missing low Schema or values configuration missing for scenario 'service-mesh-linkerd-multi-cluster-preview'

Show details

Scenario: service-mesh-linkerd-multi-cluster-preview

Detail:

Failure message: see /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-service-mesh-linkerd-multi-cluster-preview-20260609-112208/artifacts/logs/preinstall.log

Run history: run-full-bench-20260609-085346

open schema-missing low Schema or values configuration missing for scenario 'service-mesh-linkerd-service-profile'

Show details

Scenario: service-mesh-linkerd-service-profile

Detail:

Failure message: helm install failed; see /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-service-mesh-linkerd-service-profile-20260609-112237/artifacts/logs/product-install.log

Run history: run-full-bench-20260609-085346

open schema-missing low Schema or values configuration missing for scenario 'subchart-postgres-internal'

Show details

Scenario: subchart-postgres-internal

Detail:

Failure message: helm install failed; see /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-subchart-postgres-internal-20260609-112806/artifacts/logs/product-install.log

Run history: run-full-bench-20260609-085346