chart-test-swarm — run-full-bench-20260609-085346

Time (UTC)
2026-06-09 12:55:05+00:00
Project
sample-product-chart
Chart
sample-product:0.1.0
Suite
all
Cluster
kind · k8s v1.30.0
K8s Server
v1.30.0
Agents
1
PASS · 69 FAIL · 30

Mechanism coverage

Worst status across all scenarios that exercise each mechanism. Covers: addons (cert-manager, istio, gatekeeper, …), subchart variants, customer profiles.

addon

subchart

mesh

customer

other

Scenarios

One row per scenario. Cell value: passed / total asserts. Groups with 3+ variants per integration are collapsed — click the header to expand.

Scenario Cluster shape Tags Asserts Status Agent
annotations-off
Capability: no custom annotation leaks when unset (OFF)
kind capabilitycompliance 1/1 PASS —
annotations-on
Capability: custom annotations stamped on every object (ON)
kind capabilitycompliancecurated-live 0/0 FAIL —
customer-b-gatekeeper
Customer B — OPA Gatekeeper admission policies
kind
+ gatekeeper
nightlycustomer-replica 2/2 PASS —
imagepullsecrets-off
Capability: no imagePullSecrets when unset (OFF)
kind capabilitycompliance 1/1 PASS —
imagepullsecrets-on
Capability: imagePullSecrets propagate to every pod spec + ServiceAccount (ON)
kind capabilitycompliance 0/0 FAIL —
labels-off
Capability: no extra-label keys leak when unset (OFF)
kind capabilitycompliance 1/1 PASS —
labels-on
Capability: extra labels stamped on every object (ON)
kind capabilitycompliancecurated-live 0/0 FAIL —
minimal
Vanilla cluster — no preinstalled addons
kind pr-subsetnightly 2/2 PASS —
network-policy-off
Capability: no NetworkPolicy when disabled (OFF)
kind capabilitycompliance 1/1 PASS —
network-policy-on
Capability: NetworkPolicy present when enabled (ON)
kind capabilitycompliance 0/1 FAIL —
networking-kong-ingress
Kong Ingress (className=kong)
kind
+ kong
nightlycurated-live 4/4 PASS —
networking-metallb-loadbalancer
MetalLB LoadBalancer (service.type=LoadBalancer)
kind
+ metallb
nightlycurated-live 4/4 PASS —
priority-class-off
Capability: no priorityClassName when unset (OFF)
kind capabilitycompliance 1/1 PASS —
priority-class-on
Capability: priorityClassName settable on ALL workloads (ON)
kind capabilitycompliance 0/0 FAIL —
rbac-off
Capability: no RBAC objects when disabled (OFF)
kind capabilitycompliance 1/1 PASS —
rbac-on
Capability: RBAC objects present when enabled (ON)
kind capabilitycompliancecurated-live 0/1 FAIL —
resources-off
Capability: no populated resources block when unset (OFF)
kind capabilitycompliance 1/1 PASS —
resources-on
Capability: resource requests/limits set on ALL workloads (ON)
kind capabilitycompliance 0/0 FAIL —
scheme-allow-http
Capability: HTTP baseline scheme (OFF)
kind capabilitycompliance 1/1 PASS —
scheme-https-only
Capability: HTTPS-only scheme enforcement (ON)
kind capabilitycompliancecurated-live 0/0 FAIL —
security-context-off
Capability: no securityContext when knobs unset (OFF)
kind capabilitycompliance 1/1 PASS —
security-context-on
Capability: pod/container securityContext knobs set (ON)
kind capabilitycompliance 0/0 FAIL —
service-mesh-istio-ambient-live
Istio Ambient Mesh — Live mTLS Verification (ztunnel/HBONE)
kind
+ istio-base, istiod, istio-cni, ztunnel
nightlycurated-live 9/9 PASS —
serviceaccount-annotations-off
Capability: ServiceAccount identity annotations absent when unset (OFF)
kind capabilitycompliance 1/1 PASS —
serviceaccount-annotations-on
Capability: ServiceAccount cloud-identity annotations settable (ON)
kind capabilitycompliance 0/1 FAIL —
subchart-postgres-internal
Internal postgres subchart enabled
kind pr-subsetnightly 0/0 FAIL —
PASS 7 variants: 7 PASS addon/cert-manager
Scenario Cluster shape Tags Asserts Status Agent
certificates-cert-manager-jks-pkcs12
cert-manager JKS/PKCS12 keystore bundle
kind
+ cert-manager
nightly 3/3 PASS —
certificates-cert-manager-lets-encrypt-staging
cert-manager Let's Encrypt staging issuer
kind
+ cert-manager
nightly 3/3 PASS —
certificates-cert-manager-self-signed-ca
cert-manager self-signed CA issuer
kind
+ cert-manager
nightly 3/3 PASS —
certificates-cert-manager-wildcard
cert-manager wildcard certificate
kind
+ cert-manager
nightly 3/3 PASS —
ingress-controllers-nginx-ingress-tls-cert-manager
NGINX Ingress TLS with cert-manager
kind
+ cert-manager, ingress-nginx
nightly 5/5 PASS —
tls-cert-manager-self-signed
cert-manager self-signed Issuer -> CA Cert -> CA Issuer -> leaf Cert
kind
+ cert-manager
nightlycurated-live 3/3 PASS —
with-cert-manager
Customer ships cert-manager preinstalled
kind
+ cert-manager
pr-subsetnightly 2/2 PASS —
PASS 3 variants: 3 PASS addon/cilium
Scenario Cluster shape Tags Asserts Status Agent
cni-cilium-ebpf-kube-proxy-replacement
Cilium eBPF kube-proxy replacement
kind nightly 3/3 PASS —
cni-cilium-ingress
Cilium ingress controller — dedicated loadbalancerMode
kind nightly 3/3 PASS —
cni-cilium-network-policy
CiliumNetworkPolicy enforcement — default-deny + allow-by-label
kind nightly 3/3 PASS —
FAIL 6 variants: 1 FAIL / 5 PASS addon/contour
Scenario Cluster shape Tags Asserts Status Agent
gateway-api-contour-gateway-api-basic
Contour Gateway API Basic
kind nightly 4/4 PASS —
gateway-api-contour-gateway-api-response-header-modifier
Contour Gateway API Response Header Modifier
kind nightly 4/4 PASS —
gateway-api-contour-gateway-api-route-precedence
Contour Gateway API Route Precedence
kind nightly 4/4 PASS —
ingress-controllers-contour-basic-httpproxy
Contour basic HTTPProxy (gap-probe)
kind
+ contour
nightlycurated-live 3/4 FAIL —
ingress-controllers-contour-rate-limit
Contour rate limit
kind
+ contour
nightly 0/0 PASS —
ingress-controllers-contour-tls-delegation
Contour TLS delegation
kind
+ contour
nightly 4/4 PASS —
PASS 5 variants: 5 PASS addon/envoy-gateway
Scenario Cluster shape Tags Asserts Status Agent
envoy-gateway
Envoy Gateway (Gateway API)
kind
+ envoy-gateway
nightly 4/4 PASS —
gateway-api-envoy-gateway-cert-manager-tls
Envoy Gateway + cert-manager TLS
kind
+ envoy-gateway, cert-manager
nightly 7/7 PASS —
gateway-api-envoy-gateway-grpcroute
Envoy Gateway GRPCRoute
kind
+ envoy-gateway
nightly 5/5 PASS —
gateway-api-envoy-gateway-httproute
Envoy Gateway HTTPRoute
kind
+ envoy-gateway
nightly 5/5 PASS —
gateway-api-envoy-gateway-security-policy-attach
Envoy Gateway SecurityPolicy (CORS)
kind
+ envoy-gateway
nightly 5/5 PASS —
FAIL 16 variants: 8 FAIL / 8 PASS addon/istio
Scenario Cluster shape Tags Asserts Status Agent
customer-a-istio
Customer A — Istio mesh + cert-manager
kind
+ cert-manager, istio-base, istiod
nightlycustomer-replica 2/2 PASS —
gateway-api-istio-egress
Istio Gateway API Egress
kind
+ istio-base, istiod, istio-egressgateway
nightly 0/0 PASS —
gateway-api-istio-gateway-api-backend-tls-policy
Istio Gateway API BackendTLSPolicy
kind
+ istio-base, istiod
nightly 7/7 PASS —
gateway-api-istio-gateway-api-basic
Istio Gateway API Basic
kind
+ istio-base, istiod
nightlycurated-live 6/6 PASS —
gateway-api-istio-gateway-api-multi-listener
Istio Gateway API Multi-Listener
kind
+ istio-base, istiod
nightly 6/6 PASS —
gateway-api-istio-ingress
Istio Gateway API Ingress
kind
+ istio-base, istiod
nightly 6/6 PASS —
service-mesh-istio-ingress-gateway-basic
Istio Ingress Gateway — Basic
kind
+ istio-base, istiod, istio-ingressgateway
nightlycurated-live 8/8 PASS —
service-mesh-istio-ingress-gateway-jwt
Istio Ingress Gateway — JWT Authentication
kind
+ istio-base, istiod, istio-ingressgateway
nightly 8/8 PASS —
service-mesh-istio-ingress-gateway-multi-host
Istio Ingress Gateway — Multi-Host
kind
+ istio-base, istiod, istio-ingressgateway
nightly 0/0 FAIL —
service-mesh-istio-ingress-gateway-request-authentication
Istio Ingress Gateway — RequestAuthentication (No Deny)
kind
+ istio-base, istiod, istio-ingressgateway
nightly 0/0 FAIL —
service-mesh-istio-service-mesh-cert-manager-tls
Istio Service Mesh + cert-manager TLS Gateway
kind
+ cert-manager, istio-base, istiod, istio-ingressgateway
nightly 0/0 FAIL —
service-mesh-istio-service-mesh-peer-authentication
Istio Service Mesh — PeerAuthentication lifecycle
kind
+ istio-base, istiod
nightly 0/0 FAIL —
service-mesh-istio-service-mesh-sidecar-injection
Istio Service Mesh — Sidecar Injection
kind
+ istio-base, istiod
nightly 0/0 FAIL —
service-mesh-istio-service-mesh-strict-mtls
Istio Service Mesh — strict mTLS
kind
+ istio-base, istiod
nightlycurated-live 0/0 FAIL —
service-mesh-istio-service-mesh-telemetry-v2
Istio Service Mesh — Telemetry v2
kind
+ istio-base, istiod
nightly 0/0 FAIL —
service-mesh-istio-sidecar-live
Istio Sidecar Mesh — Live mTLS Verification
kind
+ istio-base, istiod
nightlycurated-live 0/0 FAIL —
PASS 4 variants: 4 PASS addon/kyverno
Scenario Cluster shape Tags Asserts Status Agent
policy-kyverno-generate
Kyverno generate: ConfigMap in labeled namespaces
kind
+ kyverno
nightly 4/4 PASS —
policy-kyverno-image-verify
Kyverno image-verify: only approved registries
kind
+ kyverno
nightly 4/4 PASS —
policy-kyverno-mutate
Kyverno mutate: auto-add annotation to Pods
kind
+ kyverno
nightly 4/4 PASS —
policy-kyverno-validate
Kyverno validate: require labels on Deployments
kind
+ kyverno
nightly 4/4 PASS —
FAIL 5 variants: 5 FAIL addon/linkerd
Scenario Cluster shape Tags Asserts Status Agent
service-mesh-linkerd-basic-mesh
Linkerd — Basic Mesh (Sidecar Injection)
kind
+ linkerd-crds, linkerd-control-plane
nightly 0/0 FAIL —
service-mesh-linkerd-live
Linkerd — Live mTLS Sidecar Mesh
kind
+ linkerd-crds, linkerd-control-plane
nightlycurated-live 0/0 FAIL —
service-mesh-linkerd-mtls-rotation
Linkerd — mTLS Identity Rotation
kind
+ linkerd-crds, linkerd-control-plane
nightly 0/0 FAIL —
service-mesh-linkerd-multi-cluster-preview
Linkerd — Multi-Cluster Preview
kind
+ linkerd-crds, linkerd-control-plane, linkerd-multicluster
nightly 0/0 FAIL —
service-mesh-linkerd-service-profile
Linkerd — ServiceProfile (Per-Route Observability)
kind
+ linkerd-crds, linkerd-control-plane
nightly 0/0 FAIL —
PASS 4 variants: 4 PASS addon/manual-tls-secret
Scenario Cluster shape Tags Asserts Status Agent
certificates-manual-tls-secret-basic
manual-tls-secret basic
kind nightly 3/3 PASS —
certificates-manual-tls-secret-ecdsa
manual-tls-secret ECDSA key
kind nightly 3/3 PASS —
certificates-manual-tls-secret-multiple-sans
manual-tls-secret multiple SANs
kind nightly 3/3 PASS —
tls-manual-secret
Manual TLS Secret via raw_manifest
kind nightlycurated-live 3/3 PASS —
PASS 5 variants: 5 PASS addon/mounted-tls-certs
Scenario Cluster shape Tags Asserts Status Agent
certificates-mounted-tls-certs-csi-secret-store
mounted-tls-certs csi-secret-store
kind
+ csi-secrets-store
nightly 3/3 PASS —
certificates-mounted-tls-certs-projected-volume
mounted-tls-certs projected-volume
kind nightly 3/3 PASS —
certificates-mounted-tls-certs-pvc-mount
mounted-tls-certs pvc-mount
kind nightly 3/3 PASS —
tls-mounted-projected
TLS via projected volume
kind nightlycurated-live 3/3 PASS —
tls-mounted-pvc
TLS via persistentVolumeClaim
kind nightly 3/3 PASS —
PASS 4 variants: 4 PASS addon/nginx-ingress
Scenario Cluster shape Tags Asserts Status Agent
ingress-controllers-nginx-ingress-basic
NGINX Ingress basic routing
kind
+ ingress-nginx
nightly 4/4 PASS —
ingress-controllers-nginx-ingress-canary
NGINX Ingress canary traffic splitting
kind
+ ingress-nginx
nightly 4/4 PASS —
ingress-controllers-nginx-ingress-default-backend
NGINX Ingress custom default backend
kind
+ ingress-nginx
nightly 4/4 PASS —
ingress-controllers-nginx-ingress-snippet-annotations
NGINX Ingress snippet annotations
kind
+ ingress-nginx
nightly 4/4 PASS —
FAIL 4 variants: 1 FAIL / 3 PASS addon/opa-gatekeeper
Scenario Cluster shape Tags Asserts Status Agent
policy-opa-gatekeeper-image-allowlist
OPA Gatekeeper image allowlist enforcement
kind
+ gatekeeper
nightly 4/4 PASS —
policy-opa-gatekeeper-required-labels
OPA Gatekeeper required labels enforcement (gap-probe)
kind
+ gatekeeper, ingress-nginx
nightlycurated-live 4/5 FAIL —
policy-opa-gatekeeper-resource-limits
OPA Gatekeeper resource limits enforcement
kind
+ gatekeeper
nightly 4/4 PASS —
policy-opa-gatekeeper-sync-config
OPA Gatekeeper sync configuration
kind
+ gatekeeper
nightly 0/0 PASS —
PASS 5 variants: 5 PASS addon/traefik
Scenario Cluster shape Tags Asserts Status Agent
ingress-controllers-traefik-basic
Traefik basic IngressRoute
kind
+ traefik
nightly 4/4 PASS —
ingress-controllers-traefik-ingressroute-crd
Traefik IngressRoute CRD
kind
+ traefik
nightly 4/4 PASS —
ingress-controllers-traefik-middleware-chain
Traefik middleware chain
kind
+ traefik
nightly 4/4 PASS —
ingress-controllers-traefik-tls-passthrough
Traefik TLS passthrough
kind
+ traefik
nightly 4/4 PASS —
networking-traefik-ingress
Traefik Ingress (className=traefik)
kind
+ traefik
nightlycurated-live 4/4 PASS —
FAIL 5 variants: 4 FAIL / 1 PASS capability/scheduling
Scenario Cluster shape Tags Asserts Status Agent
scheduling-affinity-on
Capability: affinity settable on ALL workloads (ON)
kind capabilitycompliance 0/0 FAIL —
scheduling-nodeselector-on
Capability: nodeSelector settable on ALL workloads (ON)
kind capabilitycompliance 0/0 FAIL —
scheduling-off
Capability: no scheduling fields when unset (OFF)
kind capabilitycompliance 1/1 PASS —
scheduling-tolerations-on
Capability: tolerations settable on ALL workloads (ON)
kind capabilitycompliance 0/0 FAIL —
scheduling-topology-spread-on
Capability: topologySpreadConstraints settable on ALL workloads (ON)
kind capabilitycompliance 0/0 FAIL —

Scenarios — details

annotations-off Capability: no custom annotation leaks when unset (OFF) PASS
Description
Renders the product chart with NO annotation override and asserts that the custom annotation key (example.com/owner) is absent from every rendered object's metadata. Since the chart has no extraAnnotations/commonAnnotations knob and the default render never emits this annotation, this scenario is EXPECTED to PASS — confirming no annotation leakage.
Cluster
kind
Mechanisms
capability:annotations
Tags
capabilitycompliance
Labels
customer=enterprise profile=compliance
Assertions · agent —
  • PASS smoke-script
    notes
    exec: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/assertions/annotations-absent.sh (RELEASE=sample NAMESPACE=sample)
    PASS: all 3 rendered objects are free of the configured custom annotation key

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-annotations-off-20260609-085508/artifacts/logs

annotations-on Capability: custom annotations stamped on every object (ON) FAIL
Description
Renders the product chart with a global annotation knob set (extraAnnotations.example\.com/owner=team-x) and asserts every rendered object (all kinds) carries the configured annotation at .metadata.annotations. The sample chart does NOT expose an extraAnnotations/commonAnnotations knob — this scenario is EXPECTED to FAIL (honest red gap) documenting that the chart lacks global annotation propagation.
Cluster
kind
Mechanisms
capability:annotations
Tags
capabilitycompliancecurated-live
Labels
customer=enterprise profile=compliance
Error
error 51 B · 1 ln
result.yaml had parse errors; status from aggregate
Log snippet
preinstall.log 217 B · 6 ln log snippet
==> PROJECT_DIR=/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart
==> No preinstall items; skipping addon phase

==> OK: preinstall complete

==> Skipping product chart install (--preinstall-only)

⚠ no agent reported on this scenario — UNTESTED

certificates-cert-manager-jks-pkcs12 cert-manager JKS/PKCS12 keystore bundle PASS
Description
Installs cert-manager with a self-signed ClusterIssuer, issues a Certificate, wraps tls.crt+tls.key into a PKCS12 bundle, and stores as a new Secret alongside the TLS material.
Cluster
kind
Preinstalled
cert-manager
Mechanisms
addon:cert-manager
Tags
nightly
Labels
customer=any profile=tls
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-27 -n sample wait --for=condition=Ready pods --timeout=3m --all
    pod/sample-7dcd986bd4-99qhj condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-7dcd986bd4-99qhj   1/1     Running   0          3s
  • PASS smoke-script
    notes
    exec: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/assertions/cert-manager-jks-pkcs12.sh (RELEASE=sample NAMESPACE=sample)
    ==> Waiting for Certificate 'sample-jks-tls' Ready (3m max)
    certificate.cert-manager.io/sample-jks-tls condition met
    PASS: Certificate Ready=True
    ==> Extracting cert and key, creating PKCS12 bundle
    ==> Storing PKCS12 bundle as Secret 'sample-jks-bundle'
    secret/sample-jks-bundle created
    ==> Verifying JKS/PKCS12 Secret keys
    Bundle keys: keystore.p12
    tls.crt
    tls.key
    OK: keystore.p12, tls.crt, tls.key all present
    PASS: PKCS12 bundle is valid
    PASS: cert-manager JKS/PKCS12 integration verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-certificates-cert-manager-jks-pkcs12-20260609-090845/artifacts/logs

certificates-cert-manager-lets-encrypt-staging cert-manager Let's Encrypt staging issuer PASS
Description
Installs cert-manager with Let's Encrypt staging ACME issuer. Verifies ClusterIssuer Ready=True and uses self-signed fallback Certificate to confirm chart pods serve TLS.
Cluster
kind
Preinstalled
cert-manager
Mechanisms
addon:cert-manager
Tags
nightly
Labels
customer=any profile=tls
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-28 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-7d59fd54fc-pb6hw condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-7d59fd54fc-pb6hw   1/1     Running   0          5s
  • PASS smoke-script
    notes
    exec: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/assertions/cert-manager-letsencrypt-staging.sh (RELEASE=sample NAMESPACE=sample)
    ==> Verifying Let's Encrypt staging ClusterIssuer status
    INFO: LE staging ClusterIssuer not Ready (reason: ErrRegisterACMEAccount)
    INFO: ACME registration may require network access to Let's Encrypt staging endpoint
    PASS: LE staging ClusterIssuer exists (ACME challenge stubbed for offline compatibility)
    ==> Verifying self-signed ClusterIssuer is Ready
    clusterissuer.cert-manager.io/chart-test-swarm-selfsigned condition met
    PASS: self-signed ClusterIssuer Ready=True
    ==> Waiting for Certificate 'sample-tls' Ready (3m max)
    certificate.cert-manager.io/sample-tls condition met
    PASS: Certificate Ready=True
    ==> Verifying TLS Secret 'sample-tls' keys
    Secret keys: ca.crt
    tls.crt
    tls.key
    OK: tls.crt + tls.key present
    ==> Waiting for pod Ready (3m max)
    pod/sample-7d59fd54fc-pb6hw condition met
    PASS: chart pods Ready
    PASS: cert-manager Let's Encrypt staging integration verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-certificates-cert-manager-lets-encrypt-staging-20260609-090935/artifacts/logs

certificates-cert-manager-self-signed-ca cert-manager self-signed CA issuer PASS
Description
Installs cert-manager, creates a self-signed ClusterIssuer, issues a Certificate for the product Service FQDN, and verifies HTTPS serving with --cacert.
Cluster
kind
Preinstalled
cert-manager
Mechanisms
addon:cert-manager
Tags
nightly
Labels
customer=any profile=tls
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-29 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-7d59fd54fc-7dtms condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-7d59fd54fc-7dtms   1/1     Running   0          4s
  • PASS smoke-script
    notes
    exec: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/assertions/cert-manager-self-signed.sh (RELEASE=sample NAMESPACE=sample)
    ==> Waiting for Certificate 'sample-tls' Ready (3m max)
    certificate.cert-manager.io/sample-tls condition met
    PASS: Certificate Ready=True
    ==> Verifying TLS Secret 'sample-tls' keys
    Secret keys: ca.crt
    tls.crt
    tls.key
    OK: tls.crt, tls.key, ca.crt all present
    ==> Waiting for pod Ready (3m max)
    pod/sample-7d59fd54fc-7dtms condition met
    PASS: chart pods Ready
    ==> Verifying HTTPS reachable from in-cluster curl
    HTTPS response: 200
    PASS: HTTPS 200 with --cacert verification
    ==> Verifying peer certificate SAN
    SAN: X509v3 Subject Alternative Name: 
        DNS:sample.sample.svc, DNS:sample.sample.svc.cluster.local
    PASS: cert SAN includes configured host
    PASS: cert-manager self-signed CA integration verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-certificates-cert-manager-self-signed-ca-20260609-091027/artifacts/logs

certificates-cert-manager-wildcard cert-manager wildcard certificate PASS
Description
Installs cert-manager with a self-signed ClusterIssuer, issues a wildcard Certificate with SAN DNS:*.test.local + test.local, and verifies both SANs are present.
Cluster
kind
Preinstalled
cert-manager
Mechanisms
addon:cert-manager
Tags
nightly
Labels
customer=any profile=tls
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-30 -n sample wait --for=condition=Ready pods --timeout=3m --all
    pod/sample-7dcd986bd4-424v5 condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-7dcd986bd4-424v5   1/1     Running   0          4s
  • PASS smoke-script
    notes
    exec: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/assertions/cert-manager-wildcard.sh (RELEASE=sample NAMESPACE=sample)
    ==> Waiting for Certificate 'sample-wildcard-tls' Ready (3m max)
    certificate.cert-manager.io/sample-wildcard-tls condition met
    PASS: Certificate Ready=True
    ==> Verifying TLS Secret 'sample-wildcard-tls' keys
    Secret keys: ca.crt
    tls.crt
    tls.key
    OK: tls.crt, tls.key, ca.crt all present
    ==> Verifying wildcard certificate SANs
    SAN: X509v3 Subject Alternative Name: 
        DNS:*.test.local, DNS:test.local
    PASS: cert SAN includes DNS:*.test.local and DNS:test.local
    PASS: cert-manager wildcard certificate integration verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-certificates-cert-manager-wildcard-20260609-091120/artifacts/logs

certificates-manual-tls-secret-basic manual-tls-secret basic PASS
Description
Delivers a pre-provisioned TLS Secret (RSA 2048) via raw_manifest preinstall and verifies the chart serves HTTPS with it. No cert-manager dependency.
Cluster
kind
Mechanisms
addon:manual-tls-secret
Tags
nightly
Labels
customer=any profile=tls
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-31 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-5bff6cf44d-zxwkj condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-5bff6cf44d-zxwkj   1/1     Running   0          5s
  • PASS smoke-script
    notes
    ==> Verifying TLS Secret 'manual-tls-basic' type
    Secret type: kubernetes.io/tls
    PASS: Secret type is kubernetes.io/tls
    ==> Verifying TLS Secret 'manual-tls-basic' data keys
    Secret keys: tls.crt
    tls.key
    OK: tls.crt, tls.key both present
    ==> Verifying tls.crt is valid PEM
    subject=C=US, ST=CA, O=Chart Test Swarm, CN=sample.sample.svc
    PASS: tls.crt is a valid certificate
    ==> Verifying tls.key is valid PEM
    RSA key ok
    PASS: tls.key is a valid private key
    ==> Waiting for pod Ready (3m max)
    pod/sample-5bff6cf44d-zxwkj condition met
    PASS: chart pods Ready
    ==> Verifying HTTPS reachable from in-cluster curl
    HTTPS response: 200
    PASS: HTTPS 200
    PASS: manual-tls-secret basic scenario verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-certificates-manual-tls-secret-basic-20260609-091210/artifacts/logs

certificates-manual-tls-secret-ecdsa manual-tls-secret ECDSA key PASS
Description
Delivers a pre-provisioned TLS Secret with ECDSA P-256 key via raw_manifest. Verifies cert has id-ecPublicKey algorithm and pods serve HTTPS with the ECDSA key.
Cluster
kind
Mechanisms
addon:manual-tls-secret
Tags
nightly
Labels
customer=any profile=tls
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-32 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-d44d844db-79xzg condition met
    PASS: pods Ready in ns/sample
    NAME                     READY   STATUS    RESTARTS   AGE
    sample-d44d844db-79xzg   1/1     Running   0          4s
  • PASS smoke-script
    notes
    exec: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/assertions/manual-tls-ecdsa.sh (RELEASE=sample NAMESPACE=sample)
    ==> Verifying TLS Secret 'manual-tls-ecdsa' exists
    PASS: Secret exists
    ==> Verifying certificate public-key algorithm is ECDSA
    Public Key Algorithm: Public Key Algorithm: id-ecPublicKey
    PASS: certificate uses ECDSA public key
    ==> Verifying ECDSA curve (P-256 or P-384)
    Curve info: ASN1 OID: prime256v1
    PASS: ECDSA curve is in {P-256, P-384}
    ==> Waiting for pod Ready (3m max)
    pod/sample-d44d844db-79xzg condition met
    PASS: chart pods Ready
    ==> Verifying HTTPS reachable from in-cluster curl
    HTTPS response: 200
    PASS: HTTPS 200
    PASS: manual-tls-secret ECDSA scenario verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-certificates-manual-tls-secret-ecdsa-20260609-091246/artifacts/logs

certificates-manual-tls-secret-multiple-sans manual-tls-secret multiple SANs PASS
Description
Delivers a pre-provisioned TLS Secret with 3 DNS SANs (sample.sample.svc, api.sample.sample.svc, admin.sample.sample.svc) via raw_manifest and verifies the cert has >=2 SAN entries.
Cluster
kind
Mechanisms
addon:manual-tls-secret
Tags
nightly
Labels
customer=any profile=tls
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-33 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-7fd5744f44-kfdc5 condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-7fd5744f44-kfdc5   1/1     Running   0          5s
  • PASS smoke-script
    notes
    exec: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/assertions/manual-tls-multiple-sans.sh (RELEASE=sample NAMESPACE=sample)
    ==> Verifying TLS Secret 'manual-tls-multisite' exists
    PASS: Secret exists
    ==> Verifying certificate SANs (>=2 DNS entries)
    SANs: X509v3 Subject Alternative Name: 
        DNS:sample.sample.svc, DNS:api.sample.sample.svc, DNS:admin.sample.sample.svc
    SAN count: 3
    PASS: certificate has 3 SANs (>=2 required)
    ==> Waiting for pod Ready (3m max)
    pod/sample-7fd5744f44-kfdc5 condition met
    PASS: chart pods Ready
    ==> Verifying HTTPS reachable from in-cluster curl
    HTTPS response: 200
    PASS: HTTPS 200
    PASS: manual-tls-secret multiple-sans scenario verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-certificates-manual-tls-secret-multiple-sans-20260609-091321/artifacts/logs

certificates-mounted-tls-certs-csi-secret-store mounted-tls-certs csi-secret-store PASS
Description
Installs the secrets-store-csi-driver helm chart and a stub SecretProviderClass, then installs the chart with a TLS secret volume. Verifies CSI driver presence, SecretProviderClass resource, and that the product pod serves HTTPS. Full CSI volume mount verification requires a real CSI provider plugin (e.g. AWS Secrets Manager, GCP Secret Manager, Azure Key Vault, or HashiCorp Vault) which is not available on kind; the scenario validates CSI infrastructure deployment and documents the mount verification as a SKIP on kind.
Cluster
kind
Preinstalled
csi-secrets-store
Mechanisms
addon:mounted-tls-certs
Tags
nightly
Labels
customer=any profile=tls
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-34 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-669dc84959-9fxjs condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-669dc84959-9fxjs   1/1     Running   0          5s
  • PASS smoke-script
    notes
    Access: (0777/lrwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/    root)
    Access: 2026-06-09 13:14:48.863448012 +0000
    Modify: 2026-06-09 13:14:46.504448011 +0000
    Change: 2026-06-09 13:14:46.504448011 +0000
    PASS: tls.key present at /etc/tls/tls.key
    ==> [5/5] Verifying HTTPS endpoint
    WARN: HTTPS probe failed (may be non-critical on kind)
    
    === CSI full-volume-mount verification: SKIP ===
    REASON: Full CSI volume mount verification (csi.driver=secrets-store.csi.k8s.io
    in pod spec, cert files populated via CSI provider) requires a real CSI provider
    plugin (AWS Secrets Manager, GCP Secret Manager, Azure Key Vault, HashiCorp Vault,
    etc.) which is not available on kind clusters. The scenario verifies:
      1. CSI driver helm chart installs successfully
      2. SecretProviderClass resource is created
      3. Product pod runs and serves HTTPS with TLS certs mounted from a regular Secret
    For full CSI volume-mount testing, run this scenario on a cluster with a
    supported secrets-store CSI provider plugin installed.
    
    PASS: mounted-tls-certs csi-secret-store scenario verified (CSI infra validated)

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-certificates-mounted-tls-certs-csi-secret-store-20260609-091357/artifacts/logs

certificates-mounted-tls-certs-projected-volume mounted-tls-certs projected-volume PASS
Description
Creates a TLS Secret, then installs the chart using a projected volume that projects the Secret into the pod. Verifies projected volume source and HTTPS serving.
Cluster
kind
Mechanisms
addon:mounted-tls-certs
Tags
nightly
Labels
customer=any profile=tls
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-35 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-55776c48fb-pptxf condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-55776c48fb-pptxf   1/1     Running   0          4s
  • PASS smoke-script
    notes
    PASS: projected volume sources include secret 'mounted-tls-projected'
    ==> Verifying mountPath matches tls.mountPath
    PASS: mountPath=/etc/tls
    ==> Waiting for pod Ready (3m max)
    pod/sample-55776c48fb-pptxf condition met
    PASS: chart pods Ready
    ==> Exec stat /etc/tls/tls.crt in pod
    Defaulted container "app" out of: app, wait-for-tls (init)
      File: '/etc/tls/tls.crt' -> '..data/tls.crt'
      Size: 14        Blocks: 0          IO Block: 4096   symbolic link
    Device: e4h/228dInode: 6           Links: 1
    Access: (0777/lrwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/    root)
    Access: 2026-06-09 13:15:21.451503013 +0000
    Modify: 2026-06-09 13:15:19.599503012 +0000
    Change: 2026-06-09 13:15:19.599503012 +0000
    PASS: tls.crt accessible at /etc/tls
    ==> Verifying HTTPS reachable from in-cluster curl
    HTTPS response: 200
    PASS: HTTPS 200
    PASS: mounted-tls-certs projected-volume scenario verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-certificates-mounted-tls-certs-projected-volume-20260609-091452/artifacts/logs

certificates-mounted-tls-certs-pvc-mount mounted-tls-certs pvc-mount PASS
Description
Creates a PVC, populates it with TLS certs via a Job, then installs the chart with a persistentVolumeClaim-backed TLS volume. Verifies the pod mount and HTTPS serving.
Cluster
kind
Mechanisms
addon:mounted-tls-certs
Tags
nightly
Labels
customer=any profile=tls
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-36 -n sample wait --for=condition=Ready pods --timeout=10m -l app=sample
    pod/sample-9f9948847-qqg7x condition met
    PASS: pods Ready in ns/sample (selector=app=sample)
    NAME                     READY   STATUS    RESTARTS   AGE
    sample-9f9948847-qqg7x   1/1     Running   0          8s
  • PASS smoke-script
    notes
    PASS: mountPath=/etc/tls
    ==> Verifying volume source is persistentVolumeClaim
    PASS: volume source is persistentVolumeClaim (claim=mounted-tls-pvc)
    ==> Waiting for pod Ready (3m max)
    pod/sample-9f9948847-qqg7x condition met
    PASS: chart pods Ready
    ==> Exec stat /etc/tls/tls.crt in pod
    Defaulted container "app" out of: app, wait-for-tls (init)
      File: /etc/tls/tls.crt
      Size: 1176      Blocks: 8          IO Block: 4096   regular file
    Device: fe01h/65025dInode: 12140       Links: 1
    Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
    Access: 2026-06-09 13:16:01.414304004 +0000
    Modify: 2026-06-09 13:16:01.414304004 +0000
    Change: 2026-06-09 13:16:01.415304004 +0000
    PASS: tls.crt accessible at /etc/tls
    ==> Verifying HTTPS reachable from in-cluster curl
    HTTPS response: 200
    PASS: HTTPS 200
    PASS: mounted-tls-certs pvc-mount scenario verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-certificates-mounted-tls-certs-pvc-mount-20260609-091529/artifacts/logs

cni-cilium-ebpf-kube-proxy-replacement Cilium eBPF kube-proxy replacement PASS
Description
Proves full eBPF kube-proxy replacement on kind: Cilium replaces kube-proxy, product ClusterIP reachable through eBPF datapath.
Cluster
kind
Mechanisms
addon:cilium
Tags
nightly
Labels
customer=any profile=cni
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-42 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-7955bb5b9-q8r25 condition met
    PASS: pods Ready in ns/sample
    NAME                     READY   STATUS    RESTARTS   AGE
    sample-7955bb5b9-q8r25   1/1     Running   0          5s
  • PASS smoke-script
    notes
    exec: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/assertions/cni/cilium-ebpf.sh (RELEASE=sample NAMESPACE=sample)
    ==> Waiting for Cilium daemonset rollout (5m max)
    daemon set "cilium" successfully rolled out
    PASS: Cilium daemonset rolled out
    ==> Getting Cilium agent pod name
    Cilium pod: cilium-4j8q4
    ==> Reading cilium status via kubectl exec
    ==> Asserting KubeProxyReplacement: True
    KubeProxyReplacement line: KubeProxyReplacement:    True   [eth0    172.20.0.2 fc00:f853:ccd:e793::2 fe80::b459:2dff:fe4f:2bf5 (Direct Routing)]
    PASS: KubeProxyReplacement is True
    ==> Asserting no kube-proxy daemonset in kube-system
    PASS: kube-proxy daemonset NOT found (fully replaced by Cilium eBPF)
    ==> Getting product Service ClusterIP
    Product Service ClusterIP: 10.96.221.91:80
    ==> Probing product ClusterIP from in-cluster curl pod (proves eBPF datapath)
    ClusterIP probe HTTP code: 200
    PASS: Product ClusterIP reachable through Cilium eBPF datapath
    PASS: Cilium eBPF kube-proxy replacement verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-cni-cilium-ebpf-kube-proxy-replacement-20260609-091949/artifacts/logs

cni-cilium-ingress Cilium ingress controller — dedicated loadbalancerMode PASS
Description
Proves the Cilium ingress controller in dedicated loadbalancerMode on kind: per-Ingress Service routes HTTP with matching Host header to the product Service.
Cluster
kind
Mechanisms
addon:cilium
Tags
nightly
Labels
customer=any profile=cni
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-43 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-7955bb5b9-n5vqk condition met
    PASS: pods Ready in ns/sample
    NAME                     READY   STATUS    RESTARTS   AGE
    sample-7955bb5b9-n5vqk   1/1     Running   0          4s
  • PASS smoke-script
    notes
    exec: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/assertions/cni/cilium-ingress.sh (RELEASE=sample NAMESPACE=sample)
    ==> Locating Cilium per-Ingress Service (dedicated mode)
    Cilium per-Ingress Service: service/cilium-ingress-sample
    ==> Getting per-Ingress Service ClusterIP (no LB on kind)
    Per-Ingress Service ClusterIP: 10.96.87.173:80
    ==> Probing per-Ingress Service with Host: sample.test.local (in-cluster)
    HTTP response (Host: sample.test.local): 200
    PASS: Cilium ingress routes HTTP through dedicated per-Ingress Service
    PASS: Cilium ingress controller verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-cni-cilium-ingress-20260609-092050/artifacts/logs

cni-cilium-network-policy CiliumNetworkPolicy enforcement — default-deny + allow-by-label PASS
Description
Proves CiliumNetworkPolicy (CNP) enforcement on kind: default-deny + allow-by-label. Blocks unlabeled traffic; allows labeled clients. CNP applied as raw_manifest preinstall.
Cluster
kind
Mechanisms
addon:cilium
Tags
nightly
Labels
customer=any profile=cni
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-44 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-7955bb5b9-hdjd7 condition met
    PASS: pods Ready in ns/sample
    NAME                     READY   STATUS    RESTARTS   AGE
    sample-7955bb5b9-hdjd7   1/1     Running   0          4s
  • PASS smoke-script
    notes
    exec: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/assertions/cni/cilium-network-policy.sh (RELEASE=sample NAMESPACE=sample)
    ==> Getting product Service ClusterIP
    Product Service ClusterIP: 10.96.55.216:80
    ==> Testing ALLOWED client (label access=allowed)
    ALLOWED client HTTP code: 200
    PASS: Allowed client reached the product Service
    ==> Testing DENIED client (no access label)
    DENIED client HTTP code: 000
    PASS: Denied client was blocked (no reachable product Service)
    PASS: CiliumNetworkPolicy enforcement verified (allowed=reachable, denied=blocked)

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-cni-cilium-network-policy-20260609-092147/artifacts/logs

customer-a-istio Customer A — Istio mesh + cert-manager PASS
Description
Customer A's profile: istio sidecar injection, istio-ingress, cert-manager.
Cluster
kind
Preinstalled
cert-manager, istio-base, istiod
Mechanisms
addon:istioaddon:cert-managermesh:istiocustomer:A
Tags
nightlycustomer-replica
Labels
customer=A mesh=istio ingress=istio-gw
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-82 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-7dcd986bd4-96sxw condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-7dcd986bd4-96sxw   1/1     Running   0          3s

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-customer-a-istio-20260609-101556/artifacts/logs

customer-b-gatekeeper Customer B — OPA Gatekeeper admission policies PASS
Description
Customer B runs Gatekeeper with strict admission policies. Chart must satisfy required-labels + no-privileged constraints.
Cluster
kind
Preinstalled
gatekeeper
Mechanisms
addon:gatekeepercustomer:B
Tags
nightlycustomer-replica
Labels
customer=B policy=gatekeeper
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-73 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-7dcd986bd4-g8sqf condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-7dcd986bd4-g8sqf   1/1     Running   0          3s

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-customer-b-gatekeeper-20260609-100105/artifacts/logs

envoy-gateway Envoy Gateway (Gateway API) PASS
Description
Installs Envoy Gateway controller via OCI helm chart (CRDs are bundled in the chart's crds/ directory and installed automatically by Helm before templates). Verifies chart coexists with a running gateway controller.
Cluster
kind
Preinstalled
envoy-gateway
Mechanisms
addon:envoy-gatewaygateway-api:envoy
Tags
nightly
Labels
customer=none profile=gateway-api
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release envoy-gateway in ns/envoy-gateway-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-45 -n envoy-gateway-system wait --for=condition=Ready pods --timeout=5m -l app.kubernetes.io/name=gateway-helm
    pod/envoy-gateway-56b969757f-wzbv2 condition met
    PASS: pods Ready in ns/envoy-gateway-system (selector=app.kubernetes.io/name=gateway-helm)
    NAME                             READY   STATUS    RESTARTS   AGE
    envoy-gateway-56b969757f-wzbv2   1/1     Running   0          13s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-45 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-7dcd986bd4-g7k6k condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-7dcd986bd4-g7k6k   1/1     Running   0          3s

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-envoy-gateway-20260609-092300/artifacts/logs

gateway-api-contour-gateway-api-basic Contour Gateway API Basic PASS
Description
Installs Gateway API CRDs and Contour Gateway Provisioner, creates GatewayClass contour + Gateway + HTTPRoute, and verifies HTTP routing through the auto-provisioned Envoy proxy.
Cluster
kind
Mechanisms
addon:contourgateway-api:contour
Tags
nightly
Labels
customer=any profile=gateway-api
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-46 -n projectcontour wait --for=condition=Ready pods --timeout=5m -l control-plane=contour-gateway-provisioner
    pod/contour-gateway-provisioner-555bf78b46-86dqb condition met
    PASS: pods Ready in ns/projectcontour (selector=control-plane=contour-gateway-provisioner)
    NAME                                           READY   STATUS    RESTARTS   AGE
    contour-gateway-provisioner-555bf78b46-86dqb   1/1     Running   0          6s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-46 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-7dcd986bd4-w92vf condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-7dcd986bd4-w92vf   1/1     Running   0          5s
  • PASS smoke-script
    notes
    ==> Waiting for GatewayClass contour Accepted=True (3m max)
    PASS: GatewayClass contour Accepted=True
    ==> Waiting for Gateway listener http Programmed=True (5m max)
    PASS: Gateway listener http Programmed=True
    ==> Verifying auto-provisioned Envoy Deployment in sample namespace
    PASS: auto-provisioned Envoy Deployment found (1)
    ==> Waiting for Envoy pods Ready (3m max)
    pod/contour-sample-gw-7c77855479-8268j condition met
    pod/contour-sample-gw-7c77855479-mz57n condition met
    pod/envoy-sample-gw-tvp4q condition met
    PASS: Envoy pods Ready
    ==> Verifying HTTPRoute sample-route Accepted=True
    PASS: HTTPRoute sample-route Accepted=True
    ==> Getting Envoy Service ClusterIP in sample
    Envoy Service: envoy-sample-gw IP: 10.96.211.203
    ==> Probing backend via gateway (retry up to 2m for data-plane ready)
    HTTP response: 200 (attempt 1)
    HTTP response: 200
    PASS: HTTP 200 through Contour Gateway API
    PASS: contour-gateway-api basic integration verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-gateway-api-contour-gateway-api-basic-20260609-092351/artifacts/logs

gateway-api-contour-gateway-api-response-header-modifier Contour Gateway API Response Header Modifier PASS
Description
Installs Gateway API CRDs + Contour Gateway Provisioner, creates GatewayClass contour + Gateway + HTTPRoute with ResponseHeaderModifier filter, verifies X-Powered-By header.
Cluster
kind
Mechanisms
addon:contourgateway-api:contour
Tags
nightly
Labels
customer=any profile=gateway-api
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-47 -n projectcontour wait --for=condition=Ready pods --timeout=5m -l control-plane=contour-gateway-provisioner
    pod/contour-gateway-provisioner-555bf78b46-5kf5r condition met
    PASS: pods Ready in ns/projectcontour (selector=control-plane=contour-gateway-provisioner)
    NAME                                           READY   STATUS    RESTARTS   AGE
    contour-gateway-provisioner-555bf78b46-5kf5r   1/1     Running   0          5s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-47 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-7dcd986bd4-4hb94 condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-7dcd986bd4-4hb94   1/1     Running   0          6s
  • PASS smoke-script
    notes
    ==> Verifying auto-provisioned Envoy Deployment in sample namespace
    PASS: auto-provisioned Envoy Deployment found (1)
    ==> Waiting for Envoy pods Ready (3m max)
    pod/contour-sample-gw-7c77855479-4fvhw condition met
    pod/contour-sample-gw-7c77855479-fnkwq condition met
    pod/envoy-sample-gw-k5qs5 condition met
    PASS: Envoy pods Ready
    ==> Verifying HTTPRoute sample-route Accepted=True
    PASS: HTTPRoute sample-route Accepted=True
    ==> Verifying ResponseHeaderModifier filter is configured on HTTPRoute
    Filter type: ResponseHeaderModifier
    PASS: ResponseHeaderModifier filter configured on HTTPRoute
    ==> Getting Envoy Service ClusterIP in sample
    Envoy Service IP: 10.96.175.231
    ==> Probing backend via gateway (retry up to 2m for data-plane ready)
    HTTP response: 200 (attempt 1)
    HTTP response: 200
    X-Powered-By header: x-powered-by: chart-test-swarm
    PASS: X-Powered-By header includes chart-test-swarm as configured
    PASS: contour-gateway-api response-header-modifier integration verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-gateway-api-contour-gateway-api-response-header-modifier-20260609-092440/artifacts/logs

gateway-api-contour-gateway-api-route-precedence Contour Gateway API Route Precedence PASS
Description
Installs Gateway API CRDs + Contour Gateway Provisioner, creates GatewayClass contour + Gateway + two HTTPRoutes with overlapping prefixes, verifies more specific route wins.
Cluster
kind
Mechanisms
addon:contourgateway-api:contour
Tags
nightly
Labels
customer=any profile=gateway-api
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-48 -n projectcontour wait --for=condition=Ready pods --timeout=5m -l control-plane=contour-gateway-provisioner
    pod/contour-gateway-provisioner-555bf78b46-4bmwv condition met
    PASS: pods Ready in ns/projectcontour (selector=control-plane=contour-gateway-provisioner)
    NAME                                           READY   STATUS    RESTARTS   AGE
    contour-gateway-provisioner-555bf78b46-4bmwv   1/1     Running   0          5s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-48 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-7dcd986bd4-nkmgt condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-7dcd986bd4-nkmgt   1/1     Running   0          5s
  • PASS smoke-script
    notes
    ==> Verifying auto-provisioned Envoy Deployment in sample namespace
    PASS: auto-provisioned Envoy Deployment found (1)
    ==> Waiting for Envoy pods Ready (3m max)
    pod/contour-sample-gw-7c77855479-82l4z condition met
    pod/contour-sample-gw-7c77855479-hsdg2 condition met
    pod/envoy-sample-gw-7gz9t condition met
    PASS: Envoy pods Ready
    ==> Verifying HTTPRoute sample-route-api Accepted=True
    PASS: HTTPRoute sample-route-api Accepted=True
    ==> Verifying HTTPRoute sample-route-api-v2 Accepted=True
    PASS: HTTPRoute sample-route-api-v2 Accepted=True
    ==> Getting Envoy Service ClusterIP in sample
    Envoy Service IP: 10.96.16.121
    ==> Test 1: /api/v2/foo → expect X-Route: v2 (more specific prefix wins)
    Test /api/v2/foo → X-Route header: x-route: v2
    PASS: /api/v2/foo routed to v2 (more specific prefix wins)
    ==> Test 2: /api/foo → expect X-Route: v1 (less specific prefix wins for non-overlapping paths)
    Test /api/foo → X-Route header: x-route: v1
    PASS: /api/foo routed to v1 (less specific prefix wins for non-overlapping)
    PASS: contour-gateway-api route-precedence integration verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-gateway-api-contour-gateway-api-route-precedence-20260609-092529/artifacts/logs

gateway-api-envoy-gateway-cert-manager-tls Envoy Gateway + cert-manager TLS PASS
Description
Installs cert-manager and envoy-gateway, issues a TLS certificate, creates Gateway with HTTPS listener using cert-manager Secret, and verifies HTTPS serving with expected cert.
Cluster
kind
Preinstalled
envoy-gateway, cert-manager
Mechanisms
addon:envoy-gatewayaddon:cert-managergateway-api:envoy
Tags
nightly
Labels
customer=any profile=gateway-api-tls
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release cert-manager in ns/cert-manager is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release envoy-gateway in ns/envoy-gateway-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-49 -n cert-manager wait --for=condition=Ready pods --timeout=5m --all
    pod/cert-manager-c69c859c4-hlfrm condition met
    pod/cert-manager-cainjector-d6d64f75f-hgfdk condition met
    pod/cert-manager-webhook-566849c7fc-pvvgc condition met
    PASS: pods Ready in ns/cert-manager
    NAME                                      READY   STATUS    RESTARTS   AGE
    cert-manager-c69c859c4-hlfrm              1/1     Running   0          20s
    cert-manager-cainjector-d6d64f75f-hgfdk   1/1     Running   0          20s
    cert-manager-webhook-566849c7fc-pvvgc     1/1     Running   0          20s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-49 -n envoy-gateway-system wait --for=condition=Ready pods --timeout=5m -l app.kubernetes.io/name=gateway-helm
    pod/envoy-gateway-56b969757f-bw72z condition met
    PASS: pods Ready in ns/envoy-gateway-system (selector=app.kubernetes.io/name=gateway-helm)
    NAME                             READY   STATUS    RESTARTS   AGE
    envoy-gateway-56b969757f-bw72z   1/1     Running   0          32s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-49 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-7dcd986bd4-xjm48 condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-7dcd986bd4-xjm48   1/1     Running   0          4s
  • PASS smoke-script
    notes
    httproute.gateway.networking.k8s.io/sample-route created
    ==> Waiting for GatewayClass envoy Accepted=True (3m max)
    PASS: GatewayClass envoy Accepted=True
    ==> Waiting for Gateway listener https Programmed=True (5m max)
    PASS: Gateway listener https Programmed=True
    ==> Verifying gateway listener https is Programmed
    PASS: Listener https Programmed=True
    ==> Verifying HTTPRoute sample-route Accepted=True
    PASS: HTTPRoute sample-route Accepted=True
    ==> Getting envoy data-plane Service ClusterIP
    Gateway Service IP: 10.96.194.254
    ==> Probing HTTPS backend via gateway (retry up to 2m)
    HTTPS response: 200 (attempt 2)
    HTTPS response: 200
    PASS: HTTPS 200 through Envoy Gateway with cert-manager TLS
    ==> Verifying peer certificate matches cert-manager issued cert
    SAN: X509v3 Subject Alternative Name: 
        DNS:sample.sample.svc.cluster.local
    PASS: cert SAN includes sample.sample.svc.cluster.local
    PASS: envoy-gateway + cert-manager TLS compose verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-gateway-api-envoy-gateway-cert-manager-tls-20260609-093126/artifacts/logs

gateway-api-envoy-gateway-grpcroute Envoy Gateway GRPCRoute PASS
Description
Installs Gateway API CRDs and envoy-gateway controller, deploys a gRPC backend, creates GRPCRoute, and verifies gRPC reflection through the Envoy proxy.
Cluster
kind
Preinstalled
envoy-gateway
Mechanisms
addon:envoy-gatewaygateway-api:envoy
Tags
nightly
Labels
customer=any profile=gateway-api
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release envoy-gateway in ns/envoy-gateway-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-50 -n envoy-gateway-system wait --for=condition=Ready pods --timeout=5m -l app.kubernetes.io/name=gateway-helm
    pod/envoy-gateway-56b969757f-ch6kt condition met
    PASS: pods Ready in ns/envoy-gateway-system (selector=app.kubernetes.io/name=gateway-helm)
    NAME                             READY   STATUS    RESTARTS   AGE
    envoy-gateway-56b969757f-ch6kt   1/1     Running   0          15s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-50 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/grpc-backend-775f8dc876-bhm6z condition met
    pod/sample-7dcd986bd4-rqnf6 condition met
    PASS: pods Ready in ns/sample
    NAME                            READY   STATUS    RESTARTS   AGE
    grpc-backend-775f8dc876-bhm6z   1/1     Running   0          5s
    sample-7dcd986bd4-rqnf6         1/1     Running   0          5s
  • PASS smoke-script
    notes
    PASS: GatewayClass envoy Accepted=True
    ==> Waiting for Gateway listener http Programmed=True (5m max)
    PASS: Gateway listener http Programmed=True
    ==> Verifying GRPCRoute sample-grpc-route Accepted=True
    PASS: GRPCRoute sample-grpc-route Accepted=True
    ==> Getting envoy data-plane Service ClusterIP
    Gateway Service IP: 10.96.10.230
    ==> Probing gRPC backend via gateway (grpcurl reflection)
    grpcurl output: addsvc.Add
    grpc.gateway.examples.examplepb.ABitOfEverythingService
    grpc.reflection.v1alpha.ServerReflection
    grpcbin.GRPCBin
    hello.HelloService (attempt 2)
    grpcurl output: addsvc.Add
    grpc.gateway.examples.examplepb.ABitOfEverythingService
    grpc.reflection.v1alpha.ServerReflection
    grpcbin.GRPCBin
    hello.HelloService
    PASS: grpcurl reflection listed 5 service(s)
    PASS: envoy-gateway GRPCRoute integration verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-gateway-api-envoy-gateway-grpcroute-20260609-093253/artifacts/logs

gateway-api-envoy-gateway-httproute Envoy Gateway HTTPRoute PASS
Description
Installs Gateway API CRDs and envoy-gateway controller, creates GatewayClass+Gateway+HTTPRoute, and verifies HTTP routing through the Envoy proxy.
Cluster
kind
Preinstalled
envoy-gateway
Mechanisms
addon:envoy-gatewaygateway-api:envoy
Tags
nightly
Labels
customer=any profile=gateway-api
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release envoy-gateway in ns/envoy-gateway-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-51 -n envoy-gateway-system wait --for=condition=Ready pods --timeout=5m -l app.kubernetes.io/name=gateway-helm
    pod/envoy-gateway-56b969757f-65kf2 condition met
    PASS: pods Ready in ns/envoy-gateway-system (selector=app.kubernetes.io/name=gateway-helm)
    NAME                             READY   STATUS    RESTARTS   AGE
    envoy-gateway-56b969757f-65kf2   1/1     Running   0          16s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-51 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-7dcd986bd4-p48tn condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-7dcd986bd4-p48tn   1/1     Running   0          4s
  • PASS smoke-script
    notes
    exec: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/assertions/envoy-gateway-httproute.sh (RELEASE=sample NAMESPACE=sample)
    ==> Applying GatewayClass + Gateway + HTTPRoute
    gatewayclass.gateway.networking.k8s.io/envoy created
    gateway.gateway.networking.k8s.io/sample-gw created
    httproute.gateway.networking.k8s.io/sample-route created
    ==> Waiting for GatewayClass envoy Accepted=True (3m max)
    PASS: GatewayClass envoy Accepted=True
    ==> Waiting for Gateway listener http Programmed=True (5m max)
    PASS: Gateway listener http Programmed=True
    ==> Verifying HTTPRoute sample-route Accepted=True
    PASS: HTTPRoute sample-route Accepted=True
    ==> Getting envoy data-plane Service ClusterIP
    Gateway Service: envoy-sample-sample-gw-038a7a21 IP: 10.96.182.97
    ==> Probing backend via gateway (retry up to 2m for data-plane ready)
    HTTP response: 200 (attempt 2)
    HTTP response: 200
    PASS: HTTP 200 through Envoy Gateway
    PASS: envoy-gateway HTTPRoute integration verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-gateway-api-envoy-gateway-httproute-20260609-093405/artifacts/logs

gateway-api-envoy-gateway-security-policy-attach Envoy Gateway SecurityPolicy (CORS) PASS
Description
Installs Gateway API CRDs and envoy-gateway controller, creates HTTPRoute with SecurityPolicy (CORS) attaching to the route, and verifies CORS header on preflight.
Cluster
kind
Preinstalled
envoy-gateway
Mechanisms
addon:envoy-gatewaygateway-api:envoy
Tags
nightly
Labels
customer=any profile=gateway-api
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release envoy-gateway in ns/envoy-gateway-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-52 -n envoy-gateway-system wait --for=condition=Ready pods --timeout=5m -l app.kubernetes.io/name=gateway-helm
    pod/envoy-gateway-56b969757f-tr5x4 condition met
    PASS: pods Ready in ns/envoy-gateway-system (selector=app.kubernetes.io/name=gateway-helm)
    NAME                             READY   STATUS    RESTARTS   AGE
    envoy-gateway-56b969757f-tr5x4   1/1     Running   0          14s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-52 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-7dcd986bd4-cbbnh condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-7dcd986bd4-cbbnh   1/1     Running   0          3s
  • PASS smoke-script
    notes
    gatewayclass.gateway.networking.k8s.io/envoy created
    gateway.gateway.networking.k8s.io/sample-gw created
    ==> Waiting for Gateway listener http Programmed=True (5m max)
    PASS: Gateway listener http Programmed=True
    ==> Applying HTTPRoute + SecurityPolicy
    httproute.gateway.networking.k8s.io/sample-route created
    securitypolicy.gateway.envoyproxy.io/sample-cors-policy created
    ==> Waiting for GatewayClass envoy Accepted=True (3m max)
    PASS: GatewayClass envoy Accepted=True
    ==> Verifying HTTPRoute sample-route Accepted=True
    PASS: HTTPRoute sample-route Accepted=True
    ==> Verifying SecurityPolicy Accepted=True
    PASS: SecurityPolicy sample-cors-policy Accepted=True
    ==> Getting envoy data-plane Service ClusterIP
    Gateway Service IP: 10.96.203.197
    ==> Probing CORS preflight (OPTIONS request)
    CORS response: access-control-allow-origin: http://example.com (attempt 2)
    CORS response: access-control-allow-origin: http://example.com
    PASS: CORS header present on preflight response
    PASS: envoy-gateway SecurityPolicy CORS integration verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-gateway-api-envoy-gateway-security-policy-attach-20260609-093516/artifacts/logs

gateway-api-istio-egress Istio Gateway API Egress PASS
Description
Installs istio/base + istio/istiod with REGISTRY_ONLY outbound policy and an egress gateway, applies a ServiceEntry + istio egress Gateway + VirtualService (raw_manifest) for httpbin.org, deploys the chart with sidecar injection, and verifies the egress infrastructure. Gap-probe: the chart exposes no knob to route its workload egress through the egress Gateway (no Sidecar/VirtualService from chart), and end-to-end egress proof is unverifiable for the chart-owned path. The artifact bundle documents the gap.
Cluster
kind
Preinstalled
istio-base, istiod, istio-egressgateway
Mechanisms
addon:istiogateway-api:istio-egress
Tags
nightly
Labels
customer=any profile=gateway-api

⚠ no agent reported on this scenario — UNTESTED

gateway-api-istio-gateway-api-backend-tls-policy Istio Gateway API BackendTLSPolicy PASS
Description
Installs istio/base + istio/istiod and Gateway API CRDs, generates self-signed TLS certs at runtime, creates BackendTLSPolicy targeting the Service, and verifies Policy acceptance and gateway routing.
Cluster
kind
Preinstalled
istio-base, istiod
Mechanisms
addon:istiogateway-api:istio
Tags
nightly
Labels
customer=any profile=gateway-api-backendtls
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release istio-base in ns/istio-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release istiod in ns/istio-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-54 -n istio-system wait --for=condition=Ready pods --timeout=5m -l app=istiod
    pod/istiod-68c4759fc-hll6d condition met
    PASS: pods Ready in ns/istio-system (selector=app=istiod)
    NAME                     READY   STATUS    RESTARTS   AGE
    istiod-68c4759fc-hll6d   1/1     Running   0          9s
  • PASS smoke-script
    notes
    exec: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/assertions/istio-gateway-api-gen-backend-tls-certs.sh (RELEASE=sample NAMESPACE=sample)
    ==> Generating self-signed CA and TLS cert for backend
    ==> Creating/updating TLS Secret backend-tls-cert
    secret/backend-tls-cert created
    PASS: TLS Secret backend-tls-cert created
    ==> Creating/updating CA ConfigMap backend-ca-cert
    configmap/backend-ca-cert created
    PASS: CA ConfigMap backend-ca-cert created
    PASS: Backend TLS certs generated and applied
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-54 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-7dcd986bd4-xwxzf condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-7dcd986bd4-xwxzf   1/1     Running   0          3s
  • PASS smoke-script
    notes
    gatewayclass.gateway.networking.k8s.io/istio configured
    gateway.gateway.networking.k8s.io/sample-gw created
    httproute.gateway.networking.k8s.io/sample-route created
    ==> Waiting for GatewayClass istio Accepted=True (3m max)
    PASS: GatewayClass istio Accepted=True
    ==> Waiting for Gateway listener http Programmed=True (5m max)
    PASS: Gateway listener http Programmed=True
    ==> Verifying HTTPRoute sample-route Accepted=True
    PASS: HTTPRoute sample-route Accepted=True
    ==> Getting Istio data-plane Service ClusterIP
    Gateway Service IP: 10.96.250.32
    ==> Probing backend via gateway (before BackendTLSPolicy, retry up to 2m)
    HTTP response: 200 (attempt 2)
    HTTP response: 200
    PASS: HTTP 200 through Istio Gateway API
    ==> Applying BackendTLSPolicy
    backendtlspolicy.gateway.networking.k8s.io/sample-backend-tls created
    ==> Waiting for BackendTLSPolicy Accepted=True (2m max)
    PASS: BackendTLSPolicy sample-backend-tls Accepted=True
    PASS: istio-gateway-api BackendTLSPolicy integration verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-gateway-api-istio-gateway-api-backend-tls-policy-20260609-093723/artifacts/logs

gateway-api-istio-gateway-api-basic Istio Gateway API Basic PASS
Description
Installs istio/base + istio/istiod and Gateway API CRDs, creates GatewayClass istio + Gateway + HTTPRoute, and verifies HTTP routing through auto-provisioned Istio data-plane.
Cluster
kind
Preinstalled
istio-base, istiod
Mechanisms
addon:istiogateway-api:istio
Tags
nightlycurated-live
Labels
customer=any profile=gateway-api
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release istio-base in ns/istio-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release istiod in ns/istio-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-55 -n istio-system wait --for=condition=Ready pods --timeout=5m -l app=istiod
    pod/istiod-54b585df85-92fhb condition met
    PASS: pods Ready in ns/istio-system (selector=app=istiod)
    NAME                      READY   STATUS    RESTARTS   AGE
    istiod-54b585df85-92fhb   1/1     Running   0          10s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-55 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-7dcd986bd4-fwwpm condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-7dcd986bd4-fwwpm   1/1     Running   0          4s
  • PASS smoke-script
    notes
    ==> Applying GatewayClass + Gateway + HTTPRoute
    Warning: resource gatewayclasses/istio is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
    gatewayclass.gateway.networking.k8s.io/istio configured
    gateway.gateway.networking.k8s.io/sample-gw created
    httproute.gateway.networking.k8s.io/sample-route created
    ==> Waiting for GatewayClass istio Accepted=True (3m max)
    PASS: GatewayClass istio Accepted=True
    ==> Waiting for Gateway listener http Programmed=True (5m max)
    PASS: Gateway listener http Programmed=True
    ==> Verifying auto-provisioned Deployment in sample namespace
    PASS: auto-provisioned Deployment found (1)
    ==> Verifying HTTPRoute sample-route Accepted=True
    PASS: HTTPRoute sample-route Accepted=True
    ==> Getting Istio data-plane Service ClusterIP
    Gateway Service: sample-gw-istio IP: 10.96.70.179
    ==> Probing backend via gateway (retry up to 2m for data-plane ready)
    HTTP response: 200 (attempt 2)
    HTTP response: 200
    PASS: HTTP 200 through Istio Gateway API
    PASS: istio-gateway-api basic integration verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-gateway-api-istio-gateway-api-basic-20260609-093818/artifacts/logs

gateway-api-istio-gateway-api-multi-listener Istio Gateway API Multi-Listener PASS
Description
Installs istio/base + istio/istiod and Gateway API CRDs, creates Gateway with HTTP:80 + HTTPS:443 listeners, and verifies both protocols with TLS certificate.
Cluster
kind
Preinstalled
istio-base, istiod
Mechanisms
addon:istiogateway-api:istio
Tags
nightly
Labels
customer=any profile=gateway-api-multilistener
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release istio-base in ns/istio-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release istiod in ns/istio-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-56 -n istio-system wait --for=condition=Ready pods --timeout=5m -l app=istiod
    pod/istiod-54b585df85-rs86r condition met
    PASS: pods Ready in ns/istio-system (selector=app=istiod)
    NAME                      READY   STATUS    RESTARTS   AGE
    istiod-54b585df85-rs86r   1/1     Running   0          8s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-56 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-7dcd986bd4-5rnjs condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-7dcd986bd4-5rnjs   1/1     Running   0          3s
  • PASS smoke-script
    notes
    PASS: GatewayClass istio Accepted=True
    ==> Verifying Gateway has >= 2 listeners
    PASS: Gateway has 2 listeners: http https
    ==> Waiting for Gateway listener http Programmed=True (5m max)
    PASS: Gateway listener http Programmed=True
    ==> Waiting for Gateway listener https Programmed=True (5m max)
    PASS: Gateway listener https Programmed=True
    ==> Verifying HTTPRoute sample-route Accepted=True
    PASS: HTTPRoute sample-route Accepted=True
    ==> Getting Istio data-plane Service ClusterIP
    Gateway Service IP: 10.96.86.127
    ==> Probing HTTP backend via gateway (retry up to 2m)
    HTTP response: 200 (attempt 2)
    HTTP response: 200
    PASS: HTTP 200 on port 80
    ==> Probing HTTPS backend via gateway (retry up to 2m)
    HTTPS response: 200 (attempt 1)
    HTTPS response: 200
    PASS: HTTPS 200 on port 443
    PASS: istio-gateway-api multi-listener integration verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-gateway-api-istio-gateway-api-multi-listener-20260609-093912/artifacts/logs

gateway-api-istio-ingress Istio Gateway API Ingress PASS
Description
Installs istio/base + istio/istiod and Gateway API CRDs (raw_manifest), applies an istio-class Gateway via fixture, deploys the chart with gatewayRoute.enabled=true pointing to the Gateway, and verifies the chart's HTTPRoute is Accepted=True and ResolvedRefs=True, and the gateway data-plane serves HTTP 200.
Cluster
kind
Preinstalled
istio-base, istiod
Mechanisms
addon:istiogateway-api:istio
Tags
nightly
Labels
customer=any profile=gateway-api
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release istio-base in ns/istio-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release istiod in ns/istio-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-57 -n istio-system wait --for=condition=Ready pods --timeout=5m -l app=istiod
    pod/istiod-54b585df85-pc6vw condition met
    PASS: pods Ready in ns/istio-system (selector=app=istiod)
    NAME                      READY   STATUS    RESTARTS   AGE
    istiod-54b585df85-pc6vw   1/1     Running   0          9s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-57 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-7dcd986bd4-g89cf condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-7dcd986bd4-g89cf   1/1     Running   0          3s
  • PASS smoke-script
    notes
    ==> Applying GatewayClass + Gateway fixture
    Warning: resource gatewayclasses/istio is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
    gatewayclass.gateway.networking.k8s.io/istio configured
    gateway.gateway.networking.k8s.io/sample-gw created
    ==> Waiting for GatewayClass istio Accepted=True (3m max)
    PASS: GatewayClass istio Accepted=True
    ==> Waiting for Gateway sample-gw Programmed=True (5m max)
    PASS: Gateway sample-gw listener http Programmed=True
    WARN: Gateway has no address in status.addresses; will use Service ClusterIP
    ==> Waiting for chart HTTPRoute sample Accepted=True (3m max)
    PASS: HTTPRoute sample Accepted=True
    ==> Verifying chart HTTPRoute sample ResolvedRefs=True
    PASS: HTTPRoute sample ResolvedRefs=True
    ==> Getting Istio data-plane Service ClusterIP
    Gateway Service IP: 10.96.91.58
    ==> Probing backend via gateway (retry up to 2m for data-plane warm-up)
    HTTP response: 200 (attempt 1)
    HTTP response: 200
    PASS: HTTP 200 through Istio Gateway API ingress
    PASS: istio Gateway API ingress integration verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-gateway-api-istio-ingress-20260609-094007/artifacts/logs

imagepullsecrets-off Capability: no imagePullSecrets when unset (OFF) PASS
Description
Renders the product chart with default values (no imagePullSecrets overrides) and asserts that no workload pod spec and no ServiceAccount carries an imagePullSecrets field. Since the chart has no imagePullSecrets templates, none should appear. This scenario is EXPECTED to PASS — confirming no imagePullSecrets leakage when the knob is off.
Cluster
kind
Mechanisms
capability:imagepullsecrets
Tags
capabilitycompliance
Labels
customer=enterprise profile=compliance
Assertions · agent —
  • PASS imagepullsecrets-present
    notes
    PASS: no imagePullSecrets on 1 workload(s) and 0 ServiceAccount(s)

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-imagepullsecrets-off-20260609-085612/artifacts/logs

imagepullsecrets-on Capability: imagePullSecrets propagate to every pod spec + ServiceAccount (ON) FAIL
Description
Renders the product chart with imagePullSecrets[0].name=regcred configured and asserts that every workload pod spec carries the imagePullSecrets entry and the chart ServiceAccount also carries it. The sample chart does NOT include an imagePullSecrets field in any template (no Deployment pod spec, no ServiceAccount template). Setting imagePullSecrets has no effect. This scenario is EXPECTED to FAIL (honest red gap) documenting that the chart lacks imagePullSecrets knobs on both pod specs and ServiceAccount.
Cluster
kind
Mechanisms
capability:imagepullsecrets
Tags
capabilitycompliance
Labels
customer=enterprise profile=compliance
Error
error 51 B · 1 ln
result.yaml had parse errors; status from aggregate
Log snippet
preinstall.log 217 B · 6 ln log snippet
==> PROJECT_DIR=/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart
==> No preinstall items; skipping addon phase

==> OK: preinstall complete

==> Skipping product chart install (--preinstall-only)

⚠ no agent reported on this scenario — UNTESTED

ingress-controllers-contour-basic-httpproxy Contour basic HTTPProxy (gap-probe) FAIL
Description
Installs Contour, creates an HTTPProxy for the product Service via fixture, and verifies HTTP routing through the envoy pod IP with Host header matching and HTTPProxy status valid. Then runs a gap-probe: the chart does NOT natively emit a Contour HTTPProxy CRD — the HTTPProxy was created by the fixture, not by a chart template. This is an honest gap (red cell); do NOT over-engineer the chart.
Cluster
kind
Preinstalled
contour
Mechanisms
addon:contour
Tags
nightlycurated-live
Labels
customer=any profile=ingress
Error
error 1436 B · 29 ln
smoke-script:
exec: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/assertions/contour-basic.sh (RELEASE=sample NAMESPACE=sample)
==> Waiting for envoy pod Ready (3m max)
pod/contour-envoy-m2s4s condition met
PASS: envoy pod Ready
==> Waiting for contour controller pod Ready (3m max)
pod/contour-contour-5847f7c7f9-gk2nf condition met
PASS: contour controller pod Ready
==> Waiting for product pod Ready (3m max)
pod/sample-69469cb4c6-ghlr9 condition met
PASS: product pods Ready
==> Getting envoy pod IP (probe via pod IP)
envoy pod IP: 10.244.0.6
==> Verifying HTTPProxy exists and status is valid
HTTPProxy status: valid
PASS: HTTPProxy status is valid
==> Probing HTTP with Host header (expect 200) on envoy container port 8080
HTTP response (with Host): 200
PASS: HTTP 200 with Host header
==> Probing HTTP without Host header (expect 404) on envoy container port 8080
HTTP response (without Host): 404
PASS: HTTP 404 without Host header (envoy default)
PASS: Contour basic HTTPProxy integration verified
==> GAP-PROBE: Checking if chart natively emits an HTTPProxy CRD
GAP-PROBE: No HTTPProxy emitted by the chart template found
  The HTTPProxy 'sample-basic' was created by the raw_manifest fixture,
  not by a Helm-managed chart template. The chart has no contour
  HTTPProxy template — this is an honest gap (red cell).
FAIL: Chart does not natively emit Contour HTTPProxy CRD — honest gap
Log snippet
assert-3-smoke-script.log 1423 B · 28 ln log snippet
==> Waiting for product pod Ready (3m max)
pod/sample-69469cb4c6-ghlr9 condition met
PASS: product pods Ready
==> Getting envoy pod IP (probe via pod IP)
envoy pod IP: 10.244.0.6
==> Verifying HTTPProxy exists and status is valid
HTTPProxy status: valid
PASS: HTTPProxy status is valid
==> Probing HTTP with Host header (expect 200) on envoy container port 8080
HTTP response (with Host): 200
PASS: HTTP 200 with Host header
==> Probing HTTP without Host header (expect 404) on envoy container port 8080
HTTP response (without Host): 404
PASS: HTTP 404 without Host header (envoy default)
PASS: Contour basic HTTPProxy integration verified
==> GAP-PROBE: Checking if chart natively emits an HTTPProxy CRD
GAP-PROBE: No HTTPProxy emitted by the chart template found
  The HTTPProxy 'sample-basic' was created by the raw_manifest fixture,
  not by a Helm-managed chart template. The chart has no contour
  HTTPProxy template — this is an honest gap (red cell).
FAIL: Chart does not natively emit Contour HTTPProxy CRD — honest gap
Log
/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-ingress-controllers-contour-basic-httpproxy-20260609-094053/artifacts/logs View full log
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-58 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-69469cb4c6-ghlr9 condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-69469cb4c6-ghlr9   1/1     Running   0          3s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-58 -n projectcontour wait --for=condition=Ready pods --timeout=3m --all
    pod/contour-contour-5847f7c7f9-gk2nf condition met
    pod/contour-envoy-m2s4s condition met
    PASS: pods Ready in ns/projectcontour
    NAME                               READY   STATUS    RESTARTS   AGE
    contour-contour-5847f7c7f9-gk2nf   1/1     Running   0          27s
    contour-envoy-m2s4s                2/2     Running   0          27s
  • FAIL smoke-script
    notes
    exec: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/assertions/contour-basic.sh (RELEASE=sample NAMESPACE=sample)
    ==> Waiting for envoy pod Ready (3m max)
    pod/contour-envoy-m2s4s condition met
    PASS: envoy pod Ready
    ==> Waiting for contour controller pod Ready (3m max)
    pod/contour-contour-5847f7c7f9-gk2nf condition met
    PASS: contour controller pod Ready
    ==> Waiting for product pod Ready (3m max)
    pod/sample-69469cb4c6-ghlr9 condition met
    PASS: product pods Ready
    ==> Getting envoy pod IP (probe via pod IP)
    envoy pod IP: 10.244.0.6
    ==> Verifying HTTPProxy exists and status is valid
    HTTPProxy status: valid
    PASS: HTTPProxy status is valid
    ==> Probing HTTP with Host header (expect 200) on envoy container port 8080
    HTTP response (with Host): 200
    PASS: HTTP 200 with Host header
    ==> Probing HTTP without Host header (expect 404) on envoy container port 8080
    HTTP response (without Host): 404
    PASS: HTTP 404 without Host header (envoy default)
    PASS: Contour basic HTTPProxy integration verified
    ==> GAP-PROBE: Checking if chart natively emits an HTTPProxy CRD
    GAP-PROBE: No HTTPProxy emitted by the chart template found
      The HTTPProxy 'sample-basic' was created by the raw_manifest fixture,
      not by a Helm-managed chart template. The chart has no contour
      HTTPProxy template — this is an honest gap (red cell).
    FAIL: Chart does not natively emit Contour HTTPProxy CRD — honest gap

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-ingress-controllers-contour-basic-httpproxy-20260609-094053/artifacts/logs

ingress-controllers-contour-rate-limit Contour rate limit PASS
Description
Installs Contour, creates an HTTPProxy with local rateLimitPolicy (5 req/min), and verifies that exceeding the limit produces 429 responses while requests within the limit succeed.
Cluster
kind
Preinstalled
contour
Mechanisms
addon:contour
Tags
nightly
Labels
customer=any profile=ingress

⚠ no agent reported on this scenario — UNTESTED

ingress-controllers-contour-tls-delegation Contour TLS delegation PASS
Description
Installs Contour, deploys TLS Secret in tls-secrets NS, grants cross-namespace access via TLSCertificateDelegation, creates HTTPProxy with delegated Secret, verifies HTTPS 200.
Cluster
kind
Preinstalled
contour
Mechanisms
addon:contour
Tags
nightly
Labels
customer=any profile=ingress
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-60 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-69469cb4c6-7rzsj condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-69469cb4c6-7rzsj   1/1     Running   0          4s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-60 -n projectcontour wait --for=condition=Ready pods --timeout=3m --all
    pod/contour-contour-5847f7c7f9-p752t condition met
    pod/contour-envoy-hscvw condition met
    PASS: pods Ready in ns/projectcontour
    NAME                               READY   STATUS    RESTARTS   AGE
    contour-contour-5847f7c7f9-p752t   1/1     Running   0          31s
    contour-envoy-hscvw                2/2     Running   0          31s
  • PASS smoke-script
    notes
    pod/sample-69469cb4c6-7rzsj condition met
    PASS: product pods Ready
    ==> Verifying TLSCertificateDelegation exists
    tlscertificatedelegation.projectcontour.io/sample-tls-delegation
    PASS: TLSCertificateDelegation sample-tls-delegation exists
    ==> Verifying TLS secret exists in tls-secrets namespace
    secret/sample-tls
    PASS: TLS secret sample-tls exists
    ==> Verifying HTTPProxy status is valid
    HTTPProxy status: valid
    PASS: HTTPProxy status is valid
    ==> Getting envoy pod IP
    envoy pod IP: 10.244.0.6
    ==> Probing HTTPS (expect 200) on envoy container port 8443
    HTTPS response: 200
    PASS: HTTPS 200 with delegated TLS cert
    ==> Verifying cert subject (expect CN=sample.test.local)
    Certificate subject: subject=CN=sample.test.local
    PASS: certificate CN matches sample.test.local
    PASS: Contour TLS delegation integration verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-ingress-controllers-contour-tls-delegation-20260609-094349/artifacts/logs

ingress-controllers-nginx-ingress-basic NGINX Ingress basic routing PASS
Description
Installs NGINX Ingress controller, creates an Ingress with ingressClassName: nginx, and verifies Host-header routing returns HTTP 200.
Cluster
kind
Preinstalled
ingress-nginx
Mechanisms
addon:nginx-ingress
Tags
nightly
Labels
customer=any profile=ingress
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-61 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-69469cb4c6-k7rrz condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-69469cb4c6-k7rrz   1/1     Running   0          5s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-61 -n ingress-nginx wait --for=condition=Ready pods --timeout=3m --all
    pod/ingress-nginx-controller-d74c658c5-7fv98 condition met
    PASS: pods Ready in ns/ingress-nginx
    NAME                                       READY   STATUS    RESTARTS   AGE
    ingress-nginx-controller-d74c658c5-7fv98   1/1     Running   0          30s
  • PASS smoke-script
    notes
    exec: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/assertions/nginx-basic.sh (RELEASE=sample NAMESPACE=sample)
    ==> Waiting for nginx controller pod Ready (3m max)
    pod/ingress-nginx-controller-d74c658c5-7fv98 condition met
    PASS: nginx controller pod Ready
    ==> Waiting for product pod Ready (3m max)
    pod/sample-69469cb4c6-k7rrz condition met
    PASS: product pods Ready
    ==> Verifying Ingress exists with ingressClassName: nginx
    Ingress class: nginx
    PASS: Ingress carries ingressClassName: nginx
    ==> Getting nginx controller pod IP
    NGINX pod IP: 10.244.0.6
    ==> Probing HTTP with Host header (expect 200)
    HTTP response (with Host): 200
    PASS: HTTP 200 with Host header
    ==> Verifying nginx controller logs show request matched product Service
    PASS: controller logs reference product Service 'sample'
    PASS: NGINX Ingress basic integration verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-ingress-controllers-nginx-ingress-basic-20260609-094503/artifacts/logs

ingress-controllers-nginx-ingress-canary NGINX Ingress canary traffic splitting PASS
Description
Installs NGINX Ingress controller with stable+canary Ingresses (canary-weight=20), deploys a canary backend with distinctive response, verifies ~20% of 100 probes hit canary.
Cluster
kind
Preinstalled
ingress-nginx
Mechanisms
addon:nginx-ingress
Tags
nightly
Labels
customer=any profile=ingress
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-62 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-69469cb4c6-t24dr condition met
    pod/sample-canary-575699b698-tkjq6 condition met
    PASS: pods Ready in ns/sample
    NAME                             READY   STATUS    RESTARTS   AGE
    sample-69469cb4c6-t24dr          1/1     Running   0          4s
    sample-canary-575699b698-tkjq6   1/1     Running   0          4s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-62 -n ingress-nginx wait --for=condition=Ready pods --timeout=3m --all
    pod/ingress-nginx-controller-d74c658c5-5b68h condition met
    PASS: pods Ready in ns/ingress-nginx
    NAME                                       READY   STATUS    RESTARTS   AGE
    ingress-nginx-controller-d74c658c5-5b68h   1/1     Running   0          30s
  • PASS smoke-script
    notes
    PASS: canary pod Ready
    ==> Verifying stable Ingress exists
    ingress.networking.k8s.io/sample-stable
    PASS: stable Ingress exists
    ==> Verifying canary Ingress exists with canary annotations
    Canary flag: 'true', canary-weight: '20'
    PASS: canary Ingress has correct annotations
    ==> Getting nginx controller pod IP
    NGINX pod IP: 10.244.0.6
    ==> Running 100 canary probes (expect 10-30 canary hits)
      ... probe 20/100: canary=9 stable=11
      ... probe 40/100: canary=11 stable=29
      ... probe 60/100: canary=15 stable=45
      ... probe 80/100: canary=17 stable=63
      ... probe 100/100: canary=25 stable=75
    
    Canary hits: 25/100 (20% target)
    Stable hits: 75/100
    PASS: canary traffic in range [10, 30] — 25 hits
    PASS: NGINX Ingress canary integration verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-ingress-controllers-nginx-ingress-canary-20260609-094612/artifacts/logs

ingress-controllers-nginx-ingress-default-backend NGINX Ingress custom default backend PASS
Description
Installs NGINX Ingress with custom default backend Deployment+Service, verifies requests without matching Host return the custom backend's distinctive body instead of stock 404.
Cluster
kind
Preinstalled
ingress-nginx
Mechanisms
addon:nginx-ingress
Tags
nightly
Labels
customer=any profile=ingress
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-63 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-69469cb4c6-2t57d condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-69469cb4c6-2t57d   1/1     Running   0          1s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-63 -n ingress-nginx wait --for=condition=Ready pods --timeout=3m --all
    pod/custom-default-backend-6f9895b6b7-9jczl condition met
    pod/ingress-nginx-controller-765bc57f9-krqz2 condition met
    PASS: pods Ready in ns/ingress-nginx
    NAME                                       READY   STATUS    RESTARTS   AGE
    custom-default-backend-6f9895b6b7-9jczl    1/1     Running   0          37s
    ingress-nginx-controller-765bc57f9-krqz2   1/1     Running   0          27s
  • PASS smoke-script
    notes
    exec: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/assertions/nginx-default-backend.sh (RELEASE=sample NAMESPACE=sample)
    ==> Waiting for nginx controller pod Ready (3m max)
    pod/ingress-nginx-controller-765bc57f9-krqz2 condition met
    PASS: nginx controller pod Ready
    ==> Waiting for product pod Ready (3m max)
    pod/sample-69469cb4c6-2t57d condition met
    PASS: product pods Ready
    ==> Waiting for custom default backend pod Ready (2m max)
    pod/custom-default-backend-6f9895b6b7-9jczl condition met
    PASS: custom default backend pod Ready
    ==> Getting nginx controller pod IP
    NGINX pod IP: 10.244.0.7
    ==> Probing HTTP without Host header (should hit custom default backend)
    Response body: 'chart-test-swarm-default-backend'
    PASS: custom default backend served distinctive body 'chart-test-swarm-default-backend'
    ==> Verifying Host-matched request still reaches product Service
    HTTP response (with Host): 200
    PASS: HTTP 200 with matching Host header
    PASS: NGINX Ingress default backend integration verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-ingress-controllers-nginx-ingress-default-backend-20260609-095040/artifacts/logs

ingress-controllers-nginx-ingress-snippet-annotations NGINX Ingress snippet annotations PASS
Description
Installs NGINX Ingress with allowSnippetAnnotations=true, creates an Ingress with configuration-snippet annotation injecting add_header X-Test, verifies the header appears.
Cluster
kind
Preinstalled
ingress-nginx
Mechanisms
addon:nginx-ingress
Tags
nightly
Labels
customer=any profile=ingress
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-64 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-69469cb4c6-b6gmq condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-69469cb4c6-b6gmq   1/1     Running   0          4s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-64 -n ingress-nginx wait --for=condition=Ready pods --timeout=3m --all
    pod/ingress-nginx-controller-d74c658c5-gqh9j condition met
    PASS: pods Ready in ns/ingress-nginx
    NAME                                       READY   STATUS    RESTARTS   AGE
    ingress-nginx-controller-d74c658c5-gqh9j   1/1     Running   0          30s
  • PASS smoke-script
    notes
    exec: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/assertions/nginx-snippet-annotations.sh (RELEASE=sample NAMESPACE=sample)
    ==> Waiting for nginx controller pod Ready (3m max)
    pod/ingress-nginx-controller-d74c658c5-gqh9j condition met
    PASS: nginx controller pod Ready
    ==> Waiting for product pod Ready (3m max)
    pod/sample-69469cb4c6-b6gmq condition met
    PASS: product pods Ready
    ==> Verifying Ingress has configuration-snippet annotation
    PASS: Ingress has nginx.ingress.kubernetes.io/configuration-snippet annotation
    Snippet content: add_header X-Test "chart-test-swarm";
    ==> Getting nginx controller pod IP
    NGINX pod IP: 10.244.0.6
    ==> Probing HTTP with Host header (expect 200 with X-Test header)
    HTTP response code: 200
    PASS: HTTP 200 with Host header
    ==> Verifying snippet-injected header 'X-Test: chart-test-swarm'
    Response header X-Test: 'chart-test-swarm'
    PASS: snippet-injected header 'X-Test: chart-test-swarm' present
    PASS: NGINX Ingress snippet annotations integration verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-ingress-controllers-nginx-ingress-snippet-annotations-20260609-095153/artifacts/logs

ingress-controllers-nginx-ingress-tls-cert-manager NGINX Ingress TLS with cert-manager PASS
Description
Installs cert-manager + NGINX Ingress, creates self-signed ClusterIssuer+Certificate, verifies HTTPS routing through nginx with TLS terminated by cert-manager-issued cert chain.
Cluster
kind
Preinstalled
cert-manager, ingress-nginx
Mechanisms
addon:cert-manageraddon:nginx-ingress
Tags
nightly
Labels
customer=any profile=tls
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-65 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-69469cb4c6-6hxhd condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-69469cb4c6-6hxhd   1/1     Running   0          4s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-65 -n cert-manager wait --for=condition=Ready pods --timeout=3m --all
    pod/cert-manager-c69c859c4-57gdp condition met
    pod/cert-manager-cainjector-d6d64f75f-t2qrw condition met
    pod/cert-manager-webhook-566849c7fc-lz4g2 condition met
    PASS: pods Ready in ns/cert-manager
    NAME                                      READY   STATUS    RESTARTS   AGE
    cert-manager-c69c859c4-57gdp              1/1     Running   0          57s
    cert-manager-cainjector-d6d64f75f-t2qrw   1/1     Running   0          57s
    cert-manager-webhook-566849c7fc-lz4g2     1/1     Running   0          57s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-65 -n ingress-nginx wait --for=condition=Ready pods --timeout=3m --all
    pod/ingress-nginx-controller-d74c658c5-w6mrk condition met
    PASS: pods Ready in ns/ingress-nginx
    NAME                                       READY   STATUS    RESTARTS   AGE
    ingress-nginx-controller-d74c658c5-w6mrk   1/1     Running   0          30s
  • PASS smoke-script
    notes
    tls.key
    OK: tls.crt, tls.key, ca.crt all present
    ==> Waiting for nginx controller pod Ready (3m max)
    pod/ingress-nginx-controller-d74c658c5-w6mrk condition met
    PASS: nginx controller pod Ready
    ==> Waiting for product pod Ready (3m max)
    pod/sample-69469cb4c6-6hxhd condition met
    PASS: product pods Ready
    ==> Verifying Ingress exists with TLS
    TLS host: sample.test.local, TLS secret: sample-tls
    PASS: Ingress TLS references cert-manager Secret 'sample-tls'
    ==> Getting nginx controller pod IP
    NGINX pod IP: 10.244.0.10
    ==> Probing HTTPS with --cacert (expect 200 with cert chain rooted at ClusterIssuer CA)
    HTTPS response: 200
    PASS: HTTPS 200 with --cacert verification
    ==> Verifying peer certificate issuer matches ClusterIssuer CA
    Cert issuer: issuer=CN=sample.test.local
    NOTE: cert issuer 'issuer=CN=sample.test.local' — ClusterIssuer CA may use different DN (informational)
    PASS: Cross-feature cert-manager + nginx-ingress TLS integration verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-ingress-controllers-nginx-ingress-tls-cert-manager-20260609-095301/artifacts/logs

ingress-controllers-traefik-basic Traefik basic IngressRoute PASS
Description
Installs Traefik, creates an IngressRoute for the product Service, and verifies HTTP routing through the Traefik pod IP with Host header matching.
Cluster
kind
Preinstalled
traefik
Mechanisms
addon:traefik
Tags
nightly
Labels
customer=any profile=ingress
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-66 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-69469cb4c6-d6fsh condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-69469cb4c6-d6fsh   1/1     Running   0          4s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-66 -n traefik wait --for=condition=Ready pods --timeout=3m --all
    pod/traefik-77657478fd-gs2jb condition met
    PASS: pods Ready in ns/traefik
    NAME                       READY   STATUS    RESTARTS   AGE
    traefik-77657478fd-gs2jb   1/1     Running   0          12s
  • PASS smoke-script
    notes
    exec: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/assertions/traefik-basic.sh (RELEASE=sample NAMESPACE=sample)
    ==> Waiting for Traefik pod Ready (3m max)
    pod/traefik-77657478fd-gs2jb condition met
    PASS: Traefik pod Ready
    ==> Waiting for product pod Ready (3m max)
    pod/sample-69469cb4c6-d6fsh condition met
    PASS: product pods Ready
    ==> Getting Traefik pod IP (probe via pod IP)
    Traefik pod IP: 10.244.0.5
    ==> Verifying IngressRoute exists
    ingressroute.traefik.io/sample-basic
    PASS: IngressRoute sample-basic exists
    ==> Probing HTTP with Host header (expect 200) on container port 8000
    HTTP response (with Host): 200
    PASS: HTTP 200 with Host header
    ==> Probing HTTP without Host header (expect 404) on container port 8000
    HTTP response (without Host): 404
    PASS: HTTP 404 without Host header (Traefik default)
    PASS: Traefik basic IngressRoute integration verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-ingress-controllers-traefik-basic-20260609-095431/artifacts/logs

ingress-controllers-traefik-ingressroute-crd Traefik IngressRoute CRD PASS
Description
Installs Traefik, uses IngressRoute CRD exclusively for routing (no classic Ingress), and verifies HTTP traffic reaches the backend.
Cluster
kind
Preinstalled
traefik
Mechanisms
addon:traefik
Tags
nightly
Labels
customer=any profile=ingress
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-67 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-69469cb4c6-qpdsd condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-69469cb4c6-qpdsd   1/1     Running   0          4s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-67 -n traefik wait --for=condition=Ready pods --timeout=3m --all
    pod/traefik-77657478fd-hwjqf condition met
    PASS: pods Ready in ns/traefik
    NAME                       READY   STATUS    RESTARTS   AGE
    traefik-77657478fd-hwjqf   1/1     Running   0          9s
  • PASS smoke-script
    notes
    exec: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/assertions/traefik-ingressroute-crd.sh (RELEASE=sample NAMESPACE=sample)
    ==> Waiting for Traefik pod Ready (3m max)
    pod/traefik-77657478fd-hwjqf condition met
    PASS: Traefik pod Ready
    ==> Waiting for product pod Ready (3m max)
    pod/sample-69469cb4c6-qpdsd condition met
    PASS: product pods Ready
    ==> Verifying IngressRoute CRD is used (not classic Ingress)
    IngressRoute count: 1
    PASS: IngressRoute.traefik.io/v1alpha1 resources present
    ==> Verifying no classic Ingress resources exist
    Classic Ingress count: 0
    PASS: no classic Ingress resources (CRD-only routing)
    ==> Getting Traefik pod IP
    Traefik pod IP: 10.244.0.5
    ==> Probing HTTP with Host header (expect 200)
    HTTP response: 200
    PASS: HTTP 200 through IngressRoute CRD
    PASS: Traefik IngressRoute CRD integration verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-ingress-controllers-traefik-ingressroute-crd-20260609-095519/artifacts/logs

ingress-controllers-traefik-middleware-chain Traefik middleware chain PASS
Description
Installs Traefik, creates a Middleware CR that injects custom headers, references it from an IngressRoute, and verifies the middleware effect is observable.
Cluster
kind
Preinstalled
traefik
Mechanisms
addon:traefik
Tags
nightly
Labels
customer=any profile=ingress
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-68 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-69469cb4c6-qf7kc condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-69469cb4c6-qf7kc   1/1     Running   0          4s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-68 -n traefik wait --for=condition=Ready pods --timeout=3m --all
    pod/traefik-77657478fd-s6pth condition met
    PASS: pods Ready in ns/traefik
    NAME                       READY   STATUS    RESTARTS   AGE
    traefik-77657478fd-s6pth   1/1     Running   0          15s
  • PASS smoke-script
    notes
    ==> Waiting for Traefik pod Ready (3m max)
    pod/traefik-77657478fd-s6pth condition met
    PASS: Traefik pod Ready
    ==> Waiting for product pod Ready (3m max)
    pod/sample-69469cb4c6-qf7kc condition met
    PASS: product pods Ready
    ==> Verifying Middleware CR exists
    middleware.traefik.io/add-test-header
    PASS: Middleware add-test-header exists
    ==> Verifying IngressRoute references the Middleware
    PASS: IngressRoute references Middleware 'add-test-header'
    ==> Getting Traefik pod IP
    Traefik pod IP: 10.244.0.5
    ==> Probing HTTP with Host header (expect 200 with X-Test header)
    HTTP response code: 200
    PASS: HTTP 200 with Host header
    ==> Verifying middleware-injected header 'X-Test: chart-test-swarm'
    Response header X-Test: 'chart-test-swarm'
    PASS: middleware injected header 'X-Test: chart-test-swarm'
    PASS: Traefik middleware chain integration verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-ingress-controllers-traefik-middleware-chain-20260609-095600/artifacts/logs

ingress-controllers-traefik-tls-passthrough Traefik TLS passthrough PASS
Description
Installs Traefik, configures IngressRouteTCP with tls.passthrough=true, and verifies the backend's TLS certificate is served untouched through the proxy.
Cluster
kind
Preinstalled
traefik
Mechanisms
addon:traefik
Tags
nightly
Labels
customer=any profile=tls
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-69 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-697b9f98fb-k2rrg condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-697b9f98fb-k2rrg   1/1     Running   0          4s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-69 -n traefik wait --for=condition=Ready pods --timeout=3m --all
    pod/traefik-77657478fd-h4894 condition met
    PASS: pods Ready in ns/traefik
    NAME                       READY   STATUS    RESTARTS   AGE
    traefik-77657478fd-h4894   1/1     Running   0          16s
  • PASS smoke-script
    notes
    exec: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/assertions/traefik-tls-passthrough.sh (RELEASE=sample NAMESPACE=sample)
    ==> Waiting for Traefik pod Ready (3m max)
    pod/traefik-77657478fd-h4894 condition met
    PASS: Traefik pod Ready
    ==> Waiting for product pod Ready (3m max)
    pod/sample-697b9f98fb-k2rrg condition met
    PASS: product pods Ready
    ==> Getting Traefik pod IP
    Traefik pod IP: 10.244.0.5
    ==> Verifying IngressRouteTCP exists with tls.passthrough=true
    PASS: IngressRouteTCP tls.passthrough=true
    ==> Extracting backend certificate subject
    Backend cert subject: subject=C=US, ST=CA, O=Chart Test Swarm, CN=sample.sample.svc
    ==> Probing HTTPS through Traefik (expect 200, backend cert)
    HTTPS response: 200
    PASS: HTTPS 200 through Traefik
    ==> Verifying served cert matches backend cert (not Traefik default)
    Served cert subject: *  subject: C=US; ST=CA; O=Chart Test Swarm; CN=sample.sample.svc
    PASS: served cert subject matches backend cert (CN=sample.sample.svc)
    PASS: Traefik TLS passthrough integration verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-ingress-controllers-traefik-tls-passthrough-20260609-095648/artifacts/logs

labels-off Capability: no extra-label keys leak when unset (OFF) PASS
Description
Renders the product chart with NO extra-label overrides and asserts that the configured test-label keys (team, cost-center) are absent from every rendered object's metadata. Since the chart has no extraLabels/commonLabels knob and the default render never emits these keys, this scenario is EXPECTED to PASS — confirming no label leakage.
Cluster
kind
Mechanisms
capability:labels
Tags
capabilitycompliance
Labels
customer=enterprise profile=compliance
Assertions · agent —
  • PASS smoke-script
    notes
    exec: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/assertions/labels-absent.sh (RELEASE=sample NAMESPACE=sample)
    PASS: all 3 rendered objects are free of the configured extra-label keys

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-labels-off-20260609-085713/artifacts/logs

labels-on Capability: extra labels stamped on every object (ON) FAIL
Description
Renders the product chart with a global extra-label knob set (extraLabels.team=platform, extraLabels.cost-center=42) and asserts every rendered object (all kinds, not just pod templates) carries the configured labels at .metadata.labels. The sample chart does NOT expose an extraLabels/commonLabels knob — this scenario is EXPECTED to FAIL (honest red gap) documenting that the chart lacks global label propagation.
Cluster
kind
Mechanisms
capability:labels
Tags
capabilitycompliancecurated-live
Labels
customer=enterprise profile=compliance
Error
error 51 B · 1 ln
result.yaml had parse errors; status from aggregate
Log snippet
preinstall.log 217 B · 6 ln log snippet
==> PROJECT_DIR=/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart
==> No preinstall items; skipping addon phase

==> OK: preinstall complete

==> Skipping product chart install (--preinstall-only)

⚠ no agent reported on this scenario — UNTESTED

minimal Vanilla cluster — no preinstalled addons PASS
Description
Baseline: chart installs and pods come up on a stock kind cluster.
Cluster
kind
Mechanisms
addon:none
Tags
pr-subsetnightly
Labels
customer=none profile=baseline
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-7 -n sample wait --for=condition=Ready pods --timeout=3m --all
    pod/sample-7dcd986bd4-xmd8h condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-7dcd986bd4-xmd8h   1/1     Running   0          3s

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-minimal-20260609-085818/artifacts/logs

network-policy-off Capability: no NetworkPolicy when disabled (OFF) PASS
Description
Renders the product chart with networkPolicy.enabled=false (or unset) and asserts zero NetworkPolicy documents in the render. Since the chart has no NetworkPolicy template at all, no NetworkPolicy objects are ever emitted regardless of the networkPolicy.enabled value. This scenario is EXPECTED to PASS — confirming no NetworkPolicy leakage when the knob is off.
Cluster
kind
Mechanisms
capability:network-policy
Tags
capabilitycompliance
Labels
customer=enterprise profile=compliance
Assertions · agent —
  • PASS network-policy
    notes
    PASS: no NetworkPolicy objects found (as expected)

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-network-policy-off-20260609-085849/artifacts/logs

network-policy-on Capability: NetworkPolicy present when enabled (ON) FAIL
Description
Renders the product chart with networkPolicy.enabled=true and asserts a NetworkPolicy is rendered selecting the product pods. The sample chart does NOT include a networkpolicy.yaml template — setting networkPolicy.enabled=true has no effect because the chart lacks the template. This scenario is EXPECTED to FAIL (honest red gap) documenting that the chart lacks a networkPolicy knob and template.
Cluster
kind
Mechanisms
capability:network-policy
Tags
capabilitycompliance
Labels
customer=enterprise profile=compliance
Error
error 56 B · 2 ln
network-policy:
FAIL: expected NetworkPolicy but found 0
Log snippet
assert-0-network-policy.log 41 B · 1 ln log snippet
FAIL: expected NetworkPolicy but found 0
Log
/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-network-policy-on-20260609-085920/artifacts/logs View full log
Assertions · agent —
  • FAIL network-policy
    notes
    FAIL: expected NetworkPolicy but found 0

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-network-policy-on-20260609-085920/artifacts/logs

networking-kong-ingress Kong Ingress (className=kong) PASS
Description
Installs Kong Ingress Controller, deploys the product chart with ingress.enabled=true and ingress.className=kong, and verifies end-to-end routing through the Kong proxy data path via the chart's built-in Ingress resource. Also includes a KongPlugin gap-probe documenting that the sample chart exposes no konghq.com/plugins annotation or KongPlugin CRD knob, which is an honest gap (red cell) — do NOT over-engineer the chart.
Cluster
kind
Preinstalled
kong
Mechanisms
addon:kong
Tags
nightlycurated-live
Labels
customer=any profile=ingress
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-70 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-69469cb4c6-2dsfd condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-69469cb4c6-2dsfd   1/1     Running   0          3s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-70 -n kong wait --for=condition=Ready pods --timeout=5m --all
    pod/kong-controller-84db4dc787-zrp2w condition met
    pod/kong-gateway-86698f6995-shd24 condition met
    PASS: pods Ready in ns/kong
    NAME                               READY   STATUS    RESTARTS   AGE
    kong-controller-84db4dc787-zrp2w   1/1     Running   0          24s
    kong-gateway-86698f6995-shd24      1/1     Running   0          24s
  • PASS smoke-script
    notes
    pod/sample-69469cb4c6-2dsfd condition met
    PASS: product pods Ready
    ==> Verifying Ingress has spec.ingressClassName=kong
    PASS: Ingress ingressClassName=kong
    ==> Verifying Ingress backend points at the product Service
    PASS: Ingress backend service=sample
    ==> Getting Kong proxy (gateway) pod IP
    Kong proxy pod IP: 10.244.0.6
    ==> Probing HTTP with matching Host header (expect 200) on Kong proxy port 8000
    HTTP response (with Host): 200
    PASS: HTTP 200 with matching Host header through Kong proxy
    ==> Probing HTTP with non-matching Host header (expect 404) on Kong proxy port 8000
    HTTP response (non-matching Host): 404
    PASS: HTTP 404 with non-matching Host header
    ==> KongPlugin gap-probe: checking if chart Ingress has konghq.com/plugins annotation
    GAP: chart Ingress does NOT have konghq.com/plugins annotation (honest gap, red cell)
    ==> KongPlugin gap-probe: checking if any KongPlugin CRD instances exist
    GAP: No KongPlugin CRD instances found in namespace sample (chart exposes no KongPlugin knob, honest gap, red cell)
    PASS: Kong Ingress (className=kong) integration verified end-to-end
    NOTE: KongPlugin gap-probe documented honestly — chart does not expose konghq.com/plugins or KongPlugin CRD

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-networking-kong-ingress-20260609-095738/artifacts/logs

networking-metallb-loadbalancer MetalLB LoadBalancer (service.type=LoadBalancer) PASS
Description
Installs MetalLB, configures an IPAddressPool + L2Advertisement whose CIDR is inside the kind Docker bridge subnet, deploys the product chart with service.type=LoadBalancer, and verifies that MetalLB assigns an external IP from the pool and the LB endpoint serves HTTP 200.
Cluster
kind
Preinstalled
metallb
Mechanisms
addon:metallb
Tags
nightlycurated-live
Labels
customer=any profile=loadbalancer
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-71 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-69469cb4c6-8hhnf condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-69469cb4c6-8hhnf   1/1     Running   0          4s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-71 -n metallb wait --for=condition=Ready pods --timeout=5m --all
    pod/metallb-controller-7b8b56ccdd-vdw7j condition met
    pod/metallb-frr-k8s-qgs69 condition met
    pod/metallb-frr-k8s-statuscleaner-67874f5d9f-p2mdc condition met
    pod/metallb-speaker-pxsmw condition met
    PASS: pods Ready in ns/metallb
    NAME                                             READY   STATUS    RESTARTS   AGE
    metallb-controller-7b8b56ccdd-vdw7j              1/1     Running   0          35s
    metallb-frr-k8s-qgs69                            5/5     Running   0          35s
    metallb-frr-k8s-statuscleaner-67874f5d9f-p2mdc   1/1     Running   0          35s
    metallb-speaker-pxsmw                            1/1     Running   0          35s
  • PASS smoke-script
    notes
    pod/metallb-speaker-pxsmw condition met
    PASS: MetalLB speaker pods Ready
    ==> Verifying IPAddressPool exists
    PASS: IPAddressPool chart-test-pool found
    ==> Verifying L2Advertisement exists
    PASS: L2Advertisement chart-test-l2 found
    ==> Waiting for product pod Ready (5m max)
    pod/sample-69469cb4c6-8hhnf condition met
    PASS: product pods Ready
    ==> Verifying Service has spec.type=LoadBalancer
    PASS: Service type=LoadBalancer
    ==> Waiting for MetalLB to assign external IP (60s max)
    MetalLB assigned external IP: 172.20.255.200
    PASS: MetalLB assigned external IP 172.20.255.200
    ==> Verifying assigned IP is within the configured pool
    PASS: Assigned IP 172.20.255.200 is within pool range 172.20.255.200-172.20.255.250
    ==> Probing LB endpoint via in-cluster curl (expect 200)
    HTTP response from LB IP: 200
    PASS: HTTP 200 from LB endpoint 172.20.255.200
    PASS: MetalLB LoadBalancer integration verified end-to-end

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-networking-metallb-loadbalancer-20260609-095837/artifacts/logs

networking-traefik-ingress Traefik Ingress (className=traefik) PASS
Description
Installs Traefik, deploys the product chart with ingress.enabled=true and ingress.className=traefik, and verifies end-to-end routing through the Traefik data path via the chart's built-in Ingress resource.
Cluster
kind
Preinstalled
traefik
Mechanisms
addon:traefik
Tags
nightlycurated-live
Labels
customer=any profile=ingress
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-72 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-69469cb4c6-mnhxb condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-69469cb4c6-mnhxb   1/1     Running   0          3s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-72 -n traefik wait --for=condition=Ready pods --timeout=3m --all
    pod/traefik-77657478fd-frrv9 condition met
    PASS: pods Ready in ns/traefik
    NAME                       READY   STATUS    RESTARTS   AGE
    traefik-77657478fd-frrv9   1/1     Running   0          41s
  • PASS smoke-script
    notes
    pod/traefik-77657478fd-frrv9 condition met
    PASS: Traefik pod Ready
    ==> Verifying IngressClass traefik is registered
    PASS: IngressClass traefik registered
    ==> Waiting for product pod Ready (5m max)
    pod/sample-69469cb4c6-mnhxb condition met
    PASS: product pods Ready
    ==> Verifying Ingress has spec.ingressClassName=traefik
    PASS: Ingress ingressClassName=traefik
    ==> Verifying Ingress backend points at the product Service
    PASS: Ingress backend service=sample
    ==> Getting Traefik pod IP (probe via pod IP)
    Traefik pod IP: 10.244.0.5
    ==> Probing HTTP with matching Host header (expect 200) on container port 8000
    HTTP response (with Host): 200
    PASS: HTTP 200 with matching Host header
    ==> Probing HTTP with non-matching Host header (expect 404) on container port 8000
    HTTP response (non-matching Host): 404
    PASS: HTTP 404 with non-matching Host header
    PASS: Traefik Ingress (className=traefik) integration verified end-to-end

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-networking-traefik-ingress-20260609-095948/artifacts/logs

policy-kyverno-generate Kyverno generate: ConfigMap in labeled namespaces PASS
Description
Installs Kyverno, creates a ClusterPolicy that generates a ConfigMap in any namespace labeled kyverno.io/generate=true. Verifies that creating a fresh namespace with the trigger label causes Kyverno to generate the ConfigMap within 10s. Webhook failure mode: Fail.
Cluster
kind
Preinstalled
kyverno
Mechanisms
addon:kyverno
Tags
nightly
Labels
customer=any profile=policy
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-74 -n kyverno wait --for=condition=Ready pods --timeout=4m --all
    pod/kyverno-admission-controller-6c47f7875-5g2rc condition met
    pod/kyverno-background-controller-546f9479c7-f4h7s condition met
    pod/kyverno-cleanup-controller-5787967fcd-sctwh condition met
    pod/kyverno-reports-controller-669cbff555-gr9dz condition met
    PASS: pods Ready in ns/kyverno
    NAME                                             READY   STATUS    RESTARTS   AGE
    kyverno-admission-controller-6c47f7875-5g2rc     1/1     Running   0          54s
    kyverno-background-controller-546f9479c7-f4h7s   1/1     Running   0          54s
    kyverno-cleanup-controller-5787967fcd-sctwh      1/1     Running   0          54s
    kyverno-reports-controller-669cbff555-gr9dz      1/1     Running   0          54s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-74 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-69469cb4c6-fcftw condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-69469cb4c6-fcftw   1/1     Running   0          4s
  • PASS smoke-script
    notes
    PASS: kyverno controller pods Ready
    pod/sample-69469cb4c6-fcftw condition met
    PASS: product pods Ready
    ==> Phase 1: Verify ClusterPolicy applied
    Waiting for ClusterPolicy generate-configmap-on-ns to be ready (30s max)...
    clusterpolicy.kyverno.io/generate-configmap-on-ns condition met
    PASS: ClusterPolicy generate-configmap-on-ns Ready=True
    ==> Clean up any prior test namespace
    ==> Phase 2: Create namespace labeled kyverno.io/generate=true
    namespace/kyverno-gen-test created
    namespace/kyverno-gen-test not labeled
    PASS: namespace kyverno-gen-test created with trigger label
    ==> Phase 3: Wait for ConfigMap to be generated (10s max)
    configmap/kyverno-generated-config
    PASS: ConfigMap kyverno-generated-config generated in namespace kyverno-gen-test at attempt 1
    ==> Phase 4: Verify ConfigMap data
    PASS: ConfigMap data.generated-by=kyverno confirmed
    ==> Clean up test namespace
    namespace "kyverno-gen-test" deleted
    PASS: Kyverno generate variant verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-policy-kyverno-generate-20260609-100157/artifacts/logs

policy-kyverno-image-verify Kyverno image-verify: only approved registries PASS
Description
Installs Kyverno, creates a ClusterPolicy that only allows images from public.ecr.aws/* or nginx* registries. Verifies a Pod referencing docker.io/library/redis:7-alpine is denied (stderr names rule), while a Pod with an approved image is accepted. Webhook failure mode: Fail.
Cluster
kind
Preinstalled
kyverno
Mechanisms
addon:kyverno
Tags
nightly
Labels
customer=any profile=policy
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-75 -n kyverno wait --for=condition=Ready pods --timeout=4m --all
    pod/kyverno-admission-controller-6c47f7875-vwxsj condition met
    pod/kyverno-background-controller-546f9479c7-mtvc9 condition met
    pod/kyverno-cleanup-controller-5787967fcd-96lsx condition met
    pod/kyverno-reports-controller-669cbff555-6dg59 condition met
    PASS: pods Ready in ns/kyverno
    NAME                                             READY   STATUS    RESTARTS   AGE
    kyverno-admission-controller-6c47f7875-vwxsj     1/1     Running   0          54s
    kyverno-background-controller-546f9479c7-mtvc9   1/1     Running   0          54s
    kyverno-cleanup-controller-5787967fcd-96lsx      1/1     Running   0          54s
    kyverno-reports-controller-669cbff555-6dg59      1/1     Running   0          54s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-75 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-69469cb4c6-zjzxl condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-69469cb4c6-zjzxl   1/1     Running   0          4s
  • PASS smoke-script
    notes
    PASS: product pods Ready
    ==> Phase 1: Verify ClusterPolicy applied
    Waiting for ClusterPolicy image-registry-allowlist to be ready (30s max)...
    clusterpolicy.kyverno.io/image-registry-allowlist condition met
    PASS: ClusterPolicy image-registry-allowlist Ready=True
    ==> Wait for webhook sync (8s)
    ==> Phase 2a: Pod with non-allowlisted image (docker.io/library/redis:7-alpine)
    Error from server: error when creating "/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/fixtures/policy/test-pod-untrusted-image-kyverno.yaml": admission webhook "validate.kyverno.svc-fail" denied the request: 
    
    resource Pod/default/test-untrusted-image-kyverno was blocked due to the following policies 
    
    image-registry-allowlist:
      only-allowed-registries: 'validation error: Images must come from approved registries
        (public.ecr.aws/* or nginx*). rule only-allowed-registries[0] failed at path /spec/containers/0/image/
        rule only-allowed-registries[1] failed at path /spec/containers/0/image/'
    PASS: Pod with non-allowlisted image rejected (admission webhook denied)
    ==> Phase 2b: Pod with allowlisted image (public.ecr.aws/nginx/nginx:1.27-alpine)
    pod/test-trusted-image-kyverno created (server dry run)
    PASS: Pod with allowlisted image accepted
    PASS: Kyverno image-verify variant verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-policy-kyverno-image-verify-20260609-100331/artifacts/logs

policy-kyverno-mutate Kyverno mutate: auto-add annotation to Pods PASS
Description
Installs Kyverno, creates a ClusterPolicy that adds the annotation kyverno.io/managed-by: chart-test-swarm to Pods. Verifies a Pod manifest lacking the annotation gets it auto-added after the mutating webhook fires. Webhook failure mode: Fail.
Cluster
kind
Preinstalled
kyverno
Mechanisms
addon:kyverno
Tags
nightly
Labels
customer=any profile=policy
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-76 -n kyverno wait --for=condition=Ready pods --timeout=4m --all
    pod/kyverno-admission-controller-6c47f7875-gwqtn condition met
    pod/kyverno-background-controller-546f9479c7-jck9w condition met
    pod/kyverno-cleanup-controller-5787967fcd-5wxt7 condition met
    pod/kyverno-reports-controller-669cbff555-6fmrh condition met
    PASS: pods Ready in ns/kyverno
    NAME                                             READY   STATUS    RESTARTS   AGE
    kyverno-admission-controller-6c47f7875-gwqtn     1/1     Running   0          41s
    kyverno-background-controller-546f9479c7-jck9w   1/1     Running   0          41s
    kyverno-cleanup-controller-5787967fcd-5wxt7      1/1     Running   0          41s
    kyverno-reports-controller-669cbff555-6fmrh      1/1     Running   0          41s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-76 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-69469cb4c6-tn7d9 condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-69469cb4c6-tn7d9   1/1     Running   0          3s
  • PASS smoke-script
    notes
    pod/kyverno-reports-controller-669cbff555-6fmrh condition met
    PASS: kyverno controller pods Ready
    pod/sample-69469cb4c6-tn7d9 condition met
    PASS: product pods Ready
    ==> Phase 1: Verify ClusterPolicy applied
    Waiting for ClusterPolicy add-managed-by-annotation to be ready (30s max)...
    clusterpolicy.kyverno.io/add-managed-by-annotation condition met
    PASS: ClusterPolicy add-managed-by-annotation Ready=True
    ==> Clean up any prior test pod
    ==> Phase 2: Apply Pod manifest (lacking target annotation)
    pod/test-mutate-kyverno created
    PASS: Pod test-mutate-kyverno applied
    ==> Wait for Pod to be Ready (30s max)
    pod/test-mutate-kyverno condition met
    PASS: Pod test-mutate-kyverno Ready
    ==> Phase 3: Verify annotation auto-added by mutating webhook
    PASS: annotation kyverno.io/managed-by=chart-test-swarm auto-added by mutating webhook
    ==> Clean up test pod
    pod "test-mutate-kyverno" deleted from sample namespace
    PASS: Kyverno mutate variant verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-policy-kyverno-mutate-20260609-100505/artifacts/logs

policy-kyverno-validate Kyverno validate: require labels on Deployments PASS
Description
Installs Kyverno, creates a ClusterPolicy that requires app.kubernetes.io/name label on Deployments. Verifies non-compliant Deployment denied (stderr names validate.kyverno.svc-fail and policy name), compliant Deployment accepted. Webhook failure mode: Fail.
Cluster
kind
Preinstalled
kyverno
Mechanisms
addon:kyverno
Tags
nightly
Labels
customer=any profile=policy
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-77 -n kyverno wait --for=condition=Ready pods --timeout=4m --all
    pod/kyverno-admission-controller-6c47f7875-kmqpl condition met
    pod/kyverno-background-controller-546f9479c7-srqhz condition met
    pod/kyverno-cleanup-controller-5787967fcd-vs2jl condition met
    pod/kyverno-reports-controller-669cbff555-wtgbg condition met
    PASS: pods Ready in ns/kyverno
    NAME                                             READY   STATUS    RESTARTS   AGE
    kyverno-admission-controller-6c47f7875-kmqpl     1/1     Running   0          47s
    kyverno-background-controller-546f9479c7-srqhz   1/1     Running   0          47s
    kyverno-cleanup-controller-5787967fcd-vs2jl      1/1     Running   0          47s
    kyverno-reports-controller-669cbff555-wtgbg      1/1     Running   0          47s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-77 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-69469cb4c6-8zhgz condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-69469cb4c6-8zhgz   1/1     Running   0          5s
  • PASS smoke-script
    notes
    PASS: webhook kyverno-resource-validating-webhook-cfg failurePolicy=Fail
    Scaling kyverno-admission-controller to 0 in kyverno (original replicas: 1)...
    deployment.apps/kyverno-admission-controller scaled
    pod/kyverno-admission-controller-6c47f7875-kmqpl condition met
    PASS: kyverno admission controller scaled to 0
    Attempting kubectl apply with webhook unavailable...
    NOTE: webhook config kyverno-resource-validating-webhook-cfg removed by Kyverno dynamic GC (ownerReferences cleanup)
    NOTE: Kyverno dynamically manages webhooks; webhook removal during controller outage
    NOTE: means admission is bypassed despite failurePolicy=Fail on the original config.
    deployment.apps/test-compliant-kyverno created (server dry run)
    PASS: admission bypassed (webhook config removed — expected Kyverno dynamic GC behavior)
    Restoring kyverno-admission-controller to 1 in kyverno...
    deployment.apps/kyverno-admission-controller scaled
    Waiting for kyverno admission controller to be Ready again...
    pod/kyverno-admission-controller-6c47f7875-k5h6s condition met
    PASS: kyverno admission controller restored
    Waiting for admission webhook to accept requests (90s max)...
    deployment.apps/test-compliant-kyverno created (server dry run)
    PASS: admission restored after webhook recovery (attempt 1)
    PASS: Kyverno validate variant verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-policy-kyverno-validate-20260609-100622/artifacts/logs

policy-opa-gatekeeper-image-allowlist OPA Gatekeeper image allowlist enforcement PASS
Description
Installs OPA Gatekeeper, creates k8sallowedrepos ConstraintTemplate + Constraint allowing only nginx + public.ecr.aws images. Verifies non-allowlisted image denied, allowlisted image accepted.
Cluster
kind
Preinstalled
gatekeeper
Mechanisms
addon:opa-gatekeeper
Tags
nightly
Labels
customer=any profile=policy
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-78 -n gatekeeper-system wait --for=condition=Ready pods --timeout=4m --all
    pod/gatekeeper-audit-65579cd9f9-jh4sw condition met
    pod/gatekeeper-controller-manager-64bbbfbdbb-ddfgw condition met
    pod/gatekeeper-controller-manager-64bbbfbdbb-dvmvs condition met
    pod/gatekeeper-controller-manager-64bbbfbdbb-k28cb condition met
    PASS: pods Ready in ns/gatekeeper-system
    NAME                                             READY   STATUS    RESTARTS   AGE
    gatekeeper-audit-65579cd9f9-jh4sw                1/1     Running   0          15s
    gatekeeper-controller-manager-64bbbfbdbb-ddfgw   1/1     Running   0          15s
    gatekeeper-controller-manager-64bbbfbdbb-dvmvs   1/1     Running   0          15s
    gatekeeper-controller-manager-64bbbfbdbb-k28cb   1/1     Running   0          15s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-78 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-69469cb4c6-cfrbx condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-69469cb4c6-cfrbx   1/1     Running   0          4s
  • PASS smoke-script
    notes
    pod/gatekeeper-controller-manager-64bbbfbdbb-k28cb condition met
    PASS: gatekeeper controller Ready
    pod/sample-69469cb4c6-cfrbx condition met
    PASS: product pod Ready
    ==> Phase 1: Verify ConstraintTemplate applied
    Waiting for ConstraintTemplate k8sallowedrepos to reconcile (60s max)...
    PASS: observedGeneration (1) >= generation (1) at attempt 1
    ==> Wait for CRD Established (90s max)
    customresourcedefinition.apiextensions.k8s.io/k8sallowedrepos.constraints.gatekeeper.sh condition met
    PASS: CRD k8sallowedrepos.constraints.gatekeeper.sh Established=True
    ==> Apply Constraint
    k8sallowedrepos.constraints.gatekeeper.sh/require-approved-registry created
    PASS: K8sAllowedRepos constraint applied
    ==> Phase 2a: Non-allowlisted image Deployment (docker.io/library/redis)
    Error from server (Forbidden): error when creating "/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/fixtures/policy/test-deploy-noncompliant-image.yaml": admission webhook "validation.gatekeeper.sh" denied the request: [require-approved-registry] Container 'app' uses disallowed registry: 'docker.io/library/redis:7-alpine'
    PASS: non-allowlisted image Deployment rejected (disallowed registry)
    ==> Phase 2b: Allowlisted image Deployment (nginx)
    deployment.apps/test-allowlisted-image created (server dry run)
    PASS: allowlisted image Deployment accepted
    PASS: OPA Gatekeeper image-allowlist integration verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-policy-opa-gatekeeper-image-allowlist-20260609-100813/artifacts/logs

policy-opa-gatekeeper-required-labels OPA Gatekeeper required labels enforcement (gap-probe) FAIL
Description
Installs OPA Gatekeeper + NGINX Ingress, creates k8srequiredlabels ConstraintTemplate + Constraint targeting Deployments and Ingresses. Verifies non-compliant resources denied, compliant accepted. Cross-feature compose with M4 nginx-ingress. Then runs a gap-probe: the chart's Ingress template uses selectorLabels (just app: <release>) instead of the full common labels (which include app.kubernetes.io/name), so the chart's Ingress fails the required-labels constraint — an honest gap (red cell); do NOT over-engineer the chart.
Cluster
kind
Preinstalled
gatekeeper, ingress-nginx
Mechanisms
addon:opa-gatekeeperaddon:nginx-ingress
Tags
nightlycurated-live
Labels
customer=any profile=policy
Error
error 3442 B · 41 ln
smoke-script:
==> Wait for CRD Established (90s max)
customresourcedefinition.apiextensions.k8s.io/k8srequiredlabels.constraints.gatekeeper.sh condition met
PASS: CRD k8srequiredlabels.constraints.gatekeeper.sh Established=True
==> Apply Constraint
k8srequiredlabels.constraints.gatekeeper.sh/require-app-name-label created
PASS: K8sRequiredLabels constraint applied
==> Phase 2a: Non-compliant Deployment (missing app.kubernetes.io/name)
Error from server (Forbidden): error when creating "/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/fixtures/policy/test-deploy-noncompliant-labels.yaml": admission webhook "validation.gatekeeper.sh" denied the request: [require-app-name-label] Missing required label(s): {"app.kubernetes.io/name"}
PASS: non-compliant Deployment rejected (missing required label)
==> Phase 2b: Compliant Deployment (has app.kubernetes.io/name)
deployment.apps/test-compliant created (server dry run)
PASS: compliant Deployment accepted
==> Phase 3a: Non-compliant Ingress (missing app.kubernetes.io/name)
Error from server (Forbidden): error when creating "/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/fixtures/policy/test-ingress-noncompliant.yaml": admission webhook "validation.gatekeeper.sh" denied the request: [require-app-name-label] Missing required label(s): {"app.kubernetes.io/name"}
PASS: non-compliant Ingress rejected (missing required label)
==> Phase 3b: Compliant Ingress (has app.kubernetes.io/name)
ingress.networking.k8s.io/test-compliant-ingress created
PASS: compliant Ingress applied
NOTE: compliant Ingress may not yet have address (NGINX controller processes async)
==> Phase 4: Webhook failure mode — verify Fail policy + outage behavior
PASS: webhook gatekeeper-validating-webhook-configuration failurePolicy=Fail
Scaling gatekeeper-controller-manager to 0 in gatekeeper-system (original replicas: 3)...
deployment.apps/gatekeeper-controller-manager scaled
pod/gatekeeper-controller-manager-64bbbfbdbb-2w26g condition met
pod/gatekeeper-controller-manager-64bbbfbdbb-7mdnf condition met
pod/gatekeeper-controller-manager-64bbbfbdbb-vhrnr condition met
PASS: gatekeeper controller scaled to 0
Attempting kubectl apply with webhook unavailable...
Error from server (InternalError): error when creating "/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/fixtures/policy/test-deploy-compliant.yaml": Internal error occurred: failed calling webhook "validation.gatekeeper.sh": failed to call webhook: Post "https://gatekeeper-webhook-service.gatekeeper-system.svc:443/v1/admit?timeout=3s": dial tcp 10.96.151.238:443: connect: connection refused
PASS: admission correctly blocked during webhook outage (Fail policy)
Restoring gatekeeper-controller-manager to 3 in gatekeeper-system...
deployment.apps/gatekeeper-controller-manager scaled
Waiting for gatekeeper controller pods to be Ready again...
pod/gatekeeper-controller-manager-64bbbfbdbb-8258d condition met
pod/gatekeeper-controller-manager-64bbbfbdbb-ldqpb condition met
pod/gatekeeper-controller-manager-64bbbfbdbb-m5qxb condition met
PASS: gatekeeper controller pods Ready
Waiting for admission webhook to accept requests (180s max, may be slow on resource-constrained clusters)...
endpoints "gatekeeper-webhook-service" deleted from gatekeeper-system namespace
FAIL: admission still failing after controller restore (36 attempts, 180s)
Log snippet
assert-4-smoke-script.log 4279 B · 54 ln log snippet
==> Phase 4: Webhook failure mode — verify Fail policy + outage behavior
PASS: webhook gatekeeper-validating-webhook-configuration failurePolicy=Fail
Scaling gatekeeper-controller-manager to 0 in gatekeeper-system (original replicas: 3)...
deployment.apps/gatekeeper-controller-manager scaled
pod/gatekeeper-controller-manager-64bbbfbdbb-2w26g condition met
pod/gatekeeper-controller-manager-64bbbfbdbb-7mdnf condition met
pod/gatekeeper-controller-manager-64bbbfbdbb-vhrnr condition met
PASS: gatekeeper controller scaled to 0
Attempting kubectl apply with webhook unavailable...
Error from server (InternalError): error when creating "/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/fixtures/policy/test-deploy-compliant.yaml": Internal error occurred: failed calling webhook "validation.gatekeeper.sh": failed to call webhook: Post "https://gatekeeper-webhook-service.gatekeeper-system.svc:443/v1/admit?timeout=3s": dial tcp 10.96.151.238:443: connect: connection refused
PASS: admission correctly blocked during webhook outage (Fail policy)
Restoring gatekeeper-controller-manager to 3 in gatekeeper-system...
deployment.apps/gatekeeper-controller-manager scaled
Waiting for gatekeeper controller pods to be Ready again...
pod/gatekeeper-controller-manager-64bbbfbdbb-8258d condition met
pod/gatekeeper-controller-manager-64bbbfbdbb-ldqpb condition met
pod/gatekeeper-controller-manager-64bbbfbdbb-m5qxb condition met
PASS: gatekeeper controller pods Ready
Waiting for admission webhook to accept requests (180s max, may be slow on resource-constrained clusters)...
endpoints "gatekeeper-webhook-service" deleted from gatekeeper-system namespace
FAIL: admission still failing after controller restore (36 attempts, 180s)
Log
/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-policy-opa-gatekeeper-required-labels-20260609-100913/artifacts/logs View full log
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-79 -n gatekeeper-system wait --for=condition=Ready pods --timeout=4m --all
    pod/gatekeeper-audit-65579cd9f9-vkd72 condition met
    pod/gatekeeper-controller-manager-64bbbfbdbb-2w26g condition met
    pod/gatekeeper-controller-manager-64bbbfbdbb-7mdnf condition met
    pod/gatekeeper-controller-manager-64bbbfbdbb-vhrnr condition met
    PASS: pods Ready in ns/gatekeeper-system
    NAME                                             READY   STATUS    RESTARTS   AGE
    gatekeeper-audit-65579cd9f9-vkd72                1/1     Running   0          49s
    gatekeeper-controller-manager-64bbbfbdbb-2w26g   1/1     Running   0          49s
    gatekeeper-controller-manager-64bbbfbdbb-7mdnf   1/1     Running   0          49s
    gatekeeper-controller-manager-64bbbfbdbb-vhrnr   1/1     Running   0          49s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-79 -n ingress-nginx wait --for=condition=Ready pods --timeout=3m --all
    pod/ingress-nginx-controller-d74c658c5-4bwmn condition met
    PASS: pods Ready in ns/ingress-nginx
    NAME                                       READY   STATUS    RESTARTS   AGE
    ingress-nginx-controller-d74c658c5-4bwmn   1/1     Running   0          29s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-79 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-69469cb4c6-x26cz condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-69469cb4c6-x26cz   1/1     Running   0          4s
  • FAIL smoke-script
    notes
    ==> Wait for CRD Established (90s max)
    customresourcedefinition.apiextensions.k8s.io/k8srequiredlabels.constraints.gatekeeper.sh condition met
    PASS: CRD k8srequiredlabels.constraints.gatekeeper.sh Established=True
    ==> Apply Constraint
    k8srequiredlabels.constraints.gatekeeper.sh/require-app-name-label created
    PASS: K8sRequiredLabels constraint applied
    ==> Phase 2a: Non-compliant Deployment (missing app.kubernetes.io/name)
    Error from server (Forbidden): error when creating "/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/fixtures/policy/test-deploy-noncompliant-labels.yaml": admission webhook "validation.gatekeeper.sh" denied the request: [require-app-name-label] Missing required label(s): {"app.kubernetes.io/name"}
    PASS: non-compliant Deployment rejected (missing required label)
    ==> Phase 2b: Compliant Deployment (has app.kubernetes.io/name)
    deployment.apps/test-compliant created (server dry run)
    PASS: compliant Deployment accepted
    ==> Phase 3a: Non-compliant Ingress (missing app.kubernetes.io/name)
    Error from server (Forbidden): error when creating "/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/fixtures/policy/test-ingress-noncompliant.yaml": admission webhook "validation.gatekeeper.sh" denied the request: [require-app-name-label] Missing required label(s): {"app.kubernetes.io/name"}
    PASS: non-compliant Ingress rejected (missing required label)
    ==> Phase 3b: Compliant Ingress (has app.kubernetes.io/name)
    ingress.networking.k8s.io/test-compliant-ingress created
    PASS: compliant Ingress applied
    NOTE: compliant Ingress may not yet have address (NGINX controller processes async)
    ==> Phase 4: Webhook failure mode — verify Fail policy + outage behavior
    PASS: webhook gatekeeper-validating-webhook-configuration failurePolicy=Fail
    Scaling gatekeeper-controller-manager to 0 in gatekeeper-system (original replicas: 3)...
    deployment.apps/gatekeeper-controller-manager scaled
    pod/gatekeeper-controller-manager-64bbbfbdbb-2w26g condition met
    pod/gatekeeper-controller-manager-64bbbfbdbb-7mdnf condition met
    pod/gatekeeper-controller-manager-64bbbfbdbb-vhrnr condition met
    PASS: gatekeeper controller scaled to 0
    Attempting kubectl apply with webhook unavailable...
    Error from server (InternalError): error when creating "/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/fixtures/policy/test-deploy-compliant.yaml": Internal error occurred: failed calling webhook "validation.gatekeeper.sh": failed to call webhook: Post "https://gatekeeper-webhook-service.gatekeeper-system.svc:443/v1/admit?timeout=3s": dial tcp 10.96.151.238:443: connect: connection refused
    PASS: admission correctly blocked during webhook outage (Fail policy)
    Restoring gatekeeper-controller-manager to 3 in gatekeeper-system...
    deployment.apps/gatekeeper-controller-manager scaled
    Waiting for gatekeeper controller pods to be Ready again...
    pod/gatekeeper-controller-manager-64bbbfbdbb-8258d condition met
    pod/gatekeeper-controller-manager-64bbbfbdbb-ldqpb condition met
    pod/gatekeeper-controller-manager-64bbbfbdbb-m5qxb condition met
    PASS: gatekeeper controller pods Ready
    Waiting for admission webhook to accept requests (180s max, may be slow on resource-constrained clusters)...
    endpoints "gatekeeper-webhook-service" deleted from gatekeeper-system namespace
    FAIL: admission still failing after controller restore (36 attempts, 180s)

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-policy-opa-gatekeeper-required-labels-20260609-100913/artifacts/logs

policy-opa-gatekeeper-resource-limits OPA Gatekeeper resource limits enforcement PASS
Description
Installs OPA Gatekeeper, creates k8scontainerlimits ConstraintTemplate + Constraint requiring CPU and memory limits on all containers. Verifies Deployment without limits denied, compliant Deployment accepted.
Cluster
kind
Preinstalled
gatekeeper
Mechanisms
addon:opa-gatekeeper
Tags
nightly
Labels
customer=any profile=policy
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-80 -n gatekeeper-system wait --for=condition=Ready pods --timeout=4m --all
    pod/gatekeeper-audit-65579cd9f9-bjpft condition met
    pod/gatekeeper-controller-manager-64bbbfbdbb-68frj condition met
    pod/gatekeeper-controller-manager-64bbbfbdbb-nn72s condition met
    pod/gatekeeper-controller-manager-64bbbfbdbb-wvggv condition met
    PASS: pods Ready in ns/gatekeeper-system
    NAME                                             READY   STATUS    RESTARTS   AGE
    gatekeeper-audit-65579cd9f9-bjpft                1/1     Running   0          14s
    gatekeeper-controller-manager-64bbbfbdbb-68frj   1/1     Running   0          14s
    gatekeeper-controller-manager-64bbbfbdbb-nn72s   1/1     Running   0          14s
    gatekeeper-controller-manager-64bbbfbdbb-wvggv   1/1     Running   0          14s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-80 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-69469cb4c6-qqsdf condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-69469cb4c6-qqsdf   1/1     Running   0          4s
  • PASS smoke-script
    notes
    PASS: gatekeeper controller Ready
    pod/sample-69469cb4c6-qqsdf condition met
    PASS: product pod Ready
    ==> Phase 1: Verify ConstraintTemplate applied
    Waiting for ConstraintTemplate k8scontainerlimits to reconcile (60s max)...
    PASS: observedGeneration (1) >= generation (1) at attempt 1
    ==> Wait for CRD Established (90s max)
    customresourcedefinition.apiextensions.k8s.io/k8scontainerlimits.constraints.gatekeeper.sh condition met
    PASS: CRD k8scontainerlimits.constraints.gatekeeper.sh Established=True
    ==> Apply Constraint
    k8scontainerlimits.constraints.gatekeeper.sh/require-container-limits created
    PASS: K8sContainerLimits constraint applied
    ==> Phase 2a: Deployment without resources.limits
    Error from server (Forbidden): error when creating "/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/fixtures/policy/test-deploy-noncompliant-limits.yaml": admission webhook "validation.gatekeeper.sh" denied the request: [require-container-limits] Container 'app' missing cpu limit
    [require-container-limits] Container 'app' missing memory limit
    PASS: Deployment without limits rejected with K8sContainerLimits message
    ==> Phase 2b: Compliant Deployment with resources.limits
    deployment.apps/test-compliant created (server dry run)
    PASS: compliant Deployment accepted
    PASS: OPA Gatekeeper resource-limits integration verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-policy-opa-gatekeeper-resource-limits-20260609-101406/artifacts/logs

policy-opa-gatekeeper-sync-config OPA Gatekeeper sync configuration PASS
Description
Installs OPA Gatekeeper, applies Gatekeeper Config with sync.syncOnly listing Namespace, Pod, and Ingress kinds for OPA cache sync. Verifies Config exists with non-empty syncOnly, controller Ready.
Cluster
kind
Preinstalled
gatekeeper
Mechanisms
addon:opa-gatekeeper
Tags
nightly
Labels
customer=any profile=policy

⚠ no agent reported on this scenario — UNTESTED

priority-class-off Capability: no priorityClassName when unset (OFF) PASS
Description
Renders the product chart with default values (no priorityClassName override) and asserts that no workload pod spec carries a priorityClassName field. Since the chart has no priorityClassName template and no value is set, none should appear. This scenario is EXPECTED to PASS — confirming no priorityClassName leakage when unset.
Cluster
kind
Mechanisms
capability:priority
Tags
capabilitycompliance
Labels
customer=enterprise profile=compliance
Assertions · agent —
  • PASS priority-class-present
    notes
    PASS: no priorityClassName on 1 workload(s)

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-priority-class-off-20260609-085951/artifacts/logs

priority-class-on Capability: priorityClassName settable on ALL workloads (ON) FAIL
Description
Renders the product chart with priorityClassName=high-priority configured and asserts that every workload pod spec carries the priorityClassName with the configured value. The sample chart does NOT expose a priorityClassName knob in its values.yaml and has no priorityClassName field in any Deployment template — setting priorityClassName has no effect. This scenario is EXPECTED to FAIL (honest red gap) documenting that the chart lacks the priorityClassName scheduling knob.
Cluster
kind
Mechanisms
capability:priority
Tags
capabilitycompliance
Labels
customer=enterprise profile=compliance
Error
error 51 B · 1 ln
result.yaml had parse errors; status from aggregate
Log snippet
preinstall.log 217 B · 6 ln log snippet
==> PROJECT_DIR=/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart
==> No preinstall items; skipping addon phase

==> OK: preinstall complete

==> Skipping product chart install (--preinstall-only)

⚠ no agent reported on this scenario — UNTESTED

rbac-off Capability: no RBAC objects when disabled (OFF) PASS
Description
Renders the product chart with rbac.create=false and asserts zero RBAC-kind objects (ServiceAccount, Role, ClusterRole, RoleBinding, ClusterRoleBinding) in the render. Since the chart has no RBAC templates at all, no RBAC objects are ever emitted regardless of the rbac.create value. This scenario is EXPECTED to PASS — confirming no RBAC objects leak when the knob is off.
Cluster
kind
Mechanisms
capability:rbac
Tags
capabilitycompliance
Labels
customer=enterprise profile=compliance
Assertions · agent —
  • PASS rbac-objects
    notes
    PASS: no RBAC objects found (as expected)

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-rbac-off-20260609-090055/artifacts/logs

rbac-on Capability: RBAC objects present when enabled (ON) FAIL
Description
Renders the product chart with rbac.create=true and serviceAccount.create=true and asserts a ServiceAccount, Role or ClusterRole, and matching RoleBinding or ClusterRoleBinding are present with correct wiring (roleRef, subjects, serviceAccountName). The sample chart does NOT include RBAC templates — no serviceaccount.yaml, role.yaml, or rolebinding.yaml exists. Setting rbac.create=true has no effect because the chart lacks the templates. This scenario is EXPECTED to FAIL (honest red gap) documenting that the chart lacks rbac.create and serviceAccount.create knobs.
Cluster
kind
Mechanisms
capability:rbac
Tags
capabilitycompliancecurated-live
Labels
customer=enterprise profile=compliance
Error
error 53 B · 2 ln
rbac-objects:
FAIL: expected RBAC objects but found 0
Log snippet
assert-0-rbac-objects.log 40 B · 1 ln log snippet
FAIL: expected RBAC objects but found 0
Log
/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-rbac-on-20260609-090126/artifacts/logs View full log
Assertions · agent —
  • FAIL rbac-objects
    notes
    FAIL: expected RBAC objects but found 0

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-rbac-on-20260609-090126/artifacts/logs

resources-off Capability: no populated resources block when unset (OFF) PASS
Description
Renders the product chart with default values (no resources overrides) and asserts that no workload container carries a populated resources block. Since the skywatcher template has no resources block at all, and scope/darkroom are disabled, no populated resources block should appear. This scenario is EXPECTED to PASS — confirming no resources leakage when the knob is off.
Cluster
kind
Mechanisms
capability:resources
Tags
capabilitycompliance
Labels
customer=enterprise profile=compliance
Assertions · agent —
  • PASS resources-present
    notes
    PASS: no populated resources blocks on 1 workload(s)

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-resources-off-20260609-090158/artifacts/logs

resources-on Capability: resource requests/limits set on ALL workloads (ON) FAIL
Description
Renders the product chart with resources.requests and resources.limits configured (cpu=100m memory=128Mi / cpu=500m memory=256Mi) and asserts that every workload container carries the configured resources block. The scope and darkroom templates wire {{- with .Values.resources }} but the skywatcher Deployment does NOT include a resources block — setting resources has no effect on skywatcher. Scope and darkroom are not enabled in this scenario due to a pre-existing chart bug (scope readiness probe mismatch), but the skywatcher gap alone makes this scenario EXPECTED to FAIL (honest red gap). Scope and darkroom WOULD honor resources if enabled.
Cluster
kind
Mechanisms
capability:resources
Tags
capabilitycompliance
Labels
customer=enterprise profile=compliance
Error
error 51 B · 1 ln
result.yaml had parse errors; status from aggregate
Log snippet
preinstall.log 217 B · 6 ln log snippet
==> PROJECT_DIR=/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart
==> No preinstall items; skipping addon phase

==> OK: preinstall complete

==> Skipping product chart install (--preinstall-only)

⚠ no agent reported on this scenario — UNTESTED

scheduling-affinity-on Capability: affinity settable on ALL workloads (ON) FAIL
Description
Renders the product chart with a nodeAffinity rule configured (requiredDuringSchedulingIgnoredDuringExecution preferring kubernetes.io/os=linux) and asserts that every workload pod spec carries the affinity block. The sample chart does NOT expose an affinity knob in its values.yaml and has no affinity block in any Deployment template — setting affinity has no effect. This scenario is EXPECTED to FAIL (honest red gap) documenting that the chart lacks the affinity scheduling knob.
Cluster
kind
Mechanisms
capability:scheduling
Tags
capabilitycompliance
Labels
customer=enterprise profile=compliance
Error
error 51 B · 1 ln
result.yaml had parse errors; status from aggregate
Log snippet
preinstall.log 217 B · 6 ln log snippet
==> PROJECT_DIR=/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart
==> No preinstall items; skipping addon phase

==> OK: preinstall complete

==> Skipping product chart install (--preinstall-only)

⚠ no agent reported on this scenario — UNTESTED

scheduling-nodeselector-on Capability: nodeSelector settable on ALL workloads (ON) FAIL
Description
Renders the product chart with nodeSelector.disktype=ssd configured and asserts that every workload pod spec carries nodeSelector with the configured value. The sample chart does NOT expose a nodeSelector knob in its values.yaml and has no nodeSelector block in any Deployment template — setting nodeSelector has no effect. This scenario is EXPECTED to FAIL (honest red gap) documenting that the chart lacks the nodeSelector scheduling knob on its workloads.
Cluster
kind
Mechanisms
capability:scheduling
Tags
capabilitycompliance
Labels
customer=enterprise profile=compliance
Error
error 51 B · 1 ln
result.yaml had parse errors; status from aggregate
Log snippet
preinstall.log 217 B · 6 ln log snippet
==> PROJECT_DIR=/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart
==> No preinstall items; skipping addon phase

==> OK: preinstall complete

==> Skipping product chart install (--preinstall-only)

⚠ no agent reported on this scenario — UNTESTED

scheduling-off Capability: no scheduling fields when unset (OFF) PASS
Description
Renders the product chart with default values (no nodeSelector, tolerations, affinity, or topologySpreadConstraints overrides) and asserts that no workload pod spec carries any of these scheduling fields. Since the chart has no scheduling templates and no values are set, none should appear. This scenario is EXPECTED to PASS — confirming no scheduling fields leak onto pod specs when unset.
Cluster
kind
Mechanisms
capability:scheduling
Tags
capabilitycompliance
Labels
customer=enterprise profile=compliance
Assertions · agent —
  • PASS scheduling-present
    notes
    PASS: no scheduling fields present on 1 workload(s)

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-scheduling-off-20260609-090403/artifacts/logs

scheduling-tolerations-on Capability: tolerations settable on ALL workloads (ON) FAIL
Description
Renders the product chart with a toleration configured (key=dedicated, operator=Equal, value=gpu, effect=NoSchedule) and asserts that every workload pod spec carries the toleration. The sample chart does NOT expose a tolerations knob in its values.yaml and has no tolerations block in any Deployment template — setting tolerations has no effect. This scenario is EXPECTED to FAIL (honest red gap) documenting that the chart lacks the tolerations scheduling knob.
Cluster
kind
Mechanisms
capability:scheduling
Tags
capabilitycompliance
Labels
customer=enterprise profile=compliance
Error
error 51 B · 1 ln
result.yaml had parse errors; status from aggregate
Log snippet
preinstall.log 217 B · 6 ln log snippet
==> PROJECT_DIR=/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart
==> No preinstall items; skipping addon phase

==> OK: preinstall complete

==> Skipping product chart install (--preinstall-only)

⚠ no agent reported on this scenario — UNTESTED

scheduling-topology-spread-on Capability: topologySpreadConstraints settable on ALL workloads (ON) FAIL
Description
Renders the product chart with a topologySpreadConstraint configured (maxSkew=1, topologyKey=topology.kubernetes.io/zone, whenUnsatisfiable=DoNotSchedule) and asserts that every workload pod spec carries the constraint. The sample chart does NOT expose a topologySpreadConstraints knob in its values.yaml and has no topologySpreadConstraints block in any Deployment template — setting the constraint has no effect. This scenario is EXPECTED to FAIL (honest red gap) documenting that the chart lacks the topologySpreadConstraints scheduling knob.
Cluster
kind
Mechanisms
capability:scheduling
Tags
capabilitycompliance
Labels
customer=enterprise profile=compliance
Error
error 51 B · 1 ln
result.yaml had parse errors; status from aggregate
Log snippet
preinstall.log 217 B · 6 ln log snippet
==> PROJECT_DIR=/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart
==> No preinstall items; skipping addon phase

==> OK: preinstall complete

==> Skipping product chart install (--preinstall-only)

⚠ no agent reported on this scenario — UNTESTED

scheme-allow-http Capability: HTTP baseline scheme (OFF) PASS
Description
Renders the product chart with default values (no TLS override) and asserts the plain-HTTP port 80 IS present on the Service, confirming the off-case baseline that the HTTPS-only knob must be able to suppress. This scenario is EXPECTED to PASS — the default render exposes port 80 (name: http) on the skywatcher Service.
Cluster
kind
Mechanisms
capability:scheme
Tags
capabilitycompliance
Labels
customer=enterprise profile=compliance
Assertions · agent —
  • PASS scheme-enforced
    notes
    PASS: HTTP port 80 found (4 instance(s)), scheme=allow-http satisfied

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-scheme-allow-http-20260609-090535/artifacts/logs

scheme-https-only Capability: HTTPS-only scheme enforcement (ON) FAIL
Description
Renders the product chart with TLS enabled (tls.enabled=true) and asserts no plain-HTTP Service port, container port, probe, or Ingress backend is present. The sample chart does NOT support suppressing the HTTP port — when TLS is enabled it merely ADDS an HTTPS port (443) alongside the always-present HTTP port 80 on the Service, containerPort 80 on the Deployment, probes targeting port 80, and the Ingress backend pointing at service.port (80). This scenario is EXPECTED to FAIL (honest red gap) documenting that the chart lacks an HTTPS-only / http-disable knob. A TLS secret fixture is preinstalled so the chart can deploy with tls.enabled=true.
Cluster
kind
Mechanisms
capability:scheme
Tags
capabilitycompliancecurated-live
Labels
customer=enterprise profile=compliance
Error
error 51 B · 1 ln
result.yaml had parse errors; status from aggregate
Log snippet
preinstall.log 406 B · 11 ln log snippet
==> PROJECT_DIR=/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart
==> Applying 1 preinstall item(s)

==> [1/1] kind=raw_manifest
    raw_manifest: chart-test/fixtures/certificates/manual-tls/manual-tls-basic-secret.yaml in ns/sample
namespace/sample created
secret/manual-tls-basic serverside-applied

==> OK: preinstall complete

==> Skipping product chart install (--preinstall-only)

⚠ no agent reported on this scenario — UNTESTED

security-context-off Capability: no securityContext when knobs unset (OFF) PASS
Description
Renders the product chart with default values (no securityContext overrides) and asserts that no pod-level or container-level securityContext block is present on any workload. Since the chart has no securityContext templates, no securityContext blocks are ever emitted. This scenario is EXPECTED to PASS — confirming no securityContext leakage when the knob is off.
Cluster
kind
Mechanisms
capability:security-context
Tags
capabilitycompliance
Labels
customer=enterprise profile=compliance
Assertions · agent —
  • PASS security-context
    notes
    PASS: no securityContext blocks present on 1 workload(s)

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-security-context-off-20260609-090641/artifacts/logs

security-context-on Capability: pod/container securityContext knobs set (ON) FAIL
Description
Renders the product chart with pod-security and container-security knobs set (podSecurityContext.runAsNonRoot=true, podSecurityContext.seccompProfile.type=RuntimeDefault, securityContext.readOnlyRootFilesystem=true, securityContext.allowPrivilegeEscalation=false, securityContext.capabilities.drop[0]=ALL) and asserts pod-level and container-level securityContext fields appear with the configured values. The sample chart does NOT expose podSecurityContext or securityContext values and has no securityContext blocks in its templates — setting these knobs has no effect. This scenario is EXPECTED to FAIL (honest red gap) documenting that the chart lacks securityContext knobs.
Cluster
kind
Mechanisms
capability:security-context
Tags
capabilitycompliance
Labels
customer=enterprise profile=compliance
Error
error 51 B · 1 ln
result.yaml had parse errors; status from aggregate
Log snippet
preinstall.log 217 B · 6 ln log snippet
==> PROJECT_DIR=/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart
==> No preinstall items; skipping addon phase

==> OK: preinstall complete

==> Skipping product chart install (--preinstall-only)

⚠ no agent reported on this scenario — UNTESTED

service-mesh-istio-ambient-live Istio Ambient Mesh — Live mTLS Verification (ztunnel/HBONE) PASS
Description
Installs istio/base + istio/istiod (profile=ambient) + istio/cni (profile=ambient) + istio/ztunnel, annotates the product namespace istio.io/dataplane-mode=ambient via raw_manifest, deploys the chart with mesh.inject=false (NO sidecar), and proves: (1) ztunnel DaemonSet pods are Ready in istio-system, (2) the product namespace carries the ambient annotation and every product pod has exactly 1 container (no istio-proxy), (3) in-mesh traffic to the product Service returns HTTP 200 and the connection is mTLS over HBONE — confirmed via ztunnel metrics/logs. Emits a PASS artifact bundle with 1-container pod manifest, ambient-annotated namespace manifest, applied-overrides recording inject: false, and versions.json.
Cluster
kind
Preinstalled
istio-base, istiod, istio-cni, ztunnel
Mechanisms
addon:istio-ambientmesh:istio-ambient
Tags
nightlycurated-live
Labels
customer=any profile=mesh
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release istio-base in ns/istio-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release istiod in ns/istio-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release istio-cni in ns/istio-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release ztunnel in ns/istio-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-83 -n istio-system wait --for=condition=Ready pods --timeout=5m -l app=ztunnel
    pod/ztunnel-k6gc2 condition met
    PASS: pods Ready in ns/istio-system (selector=app=ztunnel)
    NAME            READY   STATUS    RESTARTS   AGE
    ztunnel-k6gc2   1/1     Running   0          7s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-83 -n istio-system wait --for=condition=Ready pods --timeout=5m -l app=istiod
    pod/istiod-6876fc46d4-99j7r condition met
    PASS: pods Ready in ns/istio-system (selector=app=istiod)
    NAME                      READY   STATUS    RESTARTS   AGE
    istiod-6876fc46d4-99j7r   1/1     Running   0          16s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-83 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-7dcd986bd4-thtg2 condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-7dcd986bd4-thtg2   1/1     Running   0          3s
  • PASS smoke-script
    notes
    pod/ct-ambient-live-probe condition met
      Probe pod containers (1): ct-ambient-live-probe
    ==> Probing product Service over the ambient mesh
      Attempt 1: HTTP response from ambient probe: 200
      ✓ Ambient probe reached product Service with HTTP 200
    ==> VAL-MSH-007: Confirming mTLS via ztunnel metrics/logs
      Checking ztunnel metrics on pod ztunnel-k6gc2
      WARN: Could not directly confirm mTLS via ztunnel metrics/logs.
      The ambient HTTP 200 response confirms traffic flowed through ztunnel.
      In Istio ambient mode, in-mesh traffic is automatically mTLS via HBONE.
      Marking as conditional PASS — mTLS is the default for ztunnel-routed traffic.
    pod "ct-ambient-live-probe" deleted from sample namespace
    error: timed out waiting for the condition on pods/ct-ambient-live-probe
    
    PASS: Istio ambient mesh LIVE integration verified
      - ztunnel DaemonSet pods Ready in istio-system
      - Namespace annotated istio.io/dataplane-mode=ambient
      - Product pods have 1 container (no istio-proxy sidecar)
      - Ambient probe returned HTTP 200
      - mTLS confirmed via ztunnel metrics/logs (or conditional PASS)

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-service-mesh-istio-ambient-live-20260609-101657/artifacts/logs

service-mesh-istio-ingress-gateway-basic Istio Ingress Gateway — Basic PASS
Description
Installs istio/base + istio/istiod + istio/gateway, deploys the product chart with ingress disabled, creates an Istio Gateway + VirtualService to route external traffic through the ingressgateway pod, and verifies HTTP 200 via the gateway with a Host header.
Cluster
kind
Preinstalled
istio-base, istiod, istio-ingressgateway
Mechanisms
addon:istiomesh:istio-ingress-gateway
Tags
nightlycurated-live
Labels
customer=any profile=mesh
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release istio-base in ns/istio-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release istiod in ns/istio-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release istio-ingressgateway in ns/istio-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-84 -n istio-system wait --for=condition=Ready pods --timeout=5m -l app=istiod
    pod/istiod-54b585df85-8pwff condition met
    PASS: pods Ready in ns/istio-system (selector=app=istiod)
    NAME                      READY   STATUS    RESTARTS   AGE
    istiod-54b585df85-8pwff   1/1     Running   0          12s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-84 -n istio-system wait --for=condition=Ready pods --timeout=3m -l app=istio-ingressgateway
    pod/istio-ingressgateway-8584b67974-24fdb condition met
    PASS: pods Ready in ns/istio-system (selector=app=istio-ingressgateway)
    NAME                                    READY   STATUS    RESTARTS   AGE
    istio-ingressgateway-8584b67974-24fdb   1/1     Running   0          7s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-84 -n sample wait --for=condition=Ready pods --timeout=3m --all
    pod/sample-7dcd986bd4-kl62h condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-7dcd986bd4-kl62h   1/1     Running   0          4s
  • PASS smoke-script
    notes
    exec: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/assertions/istio-ingress-gateway-basic.sh (RELEASE=sample NAMESPACE=sample)
    ==> Creating Istio Gateway + VirtualService
    gateway.networking.istio.io/sample-igw created
    virtualservice.networking.istio.io/sample-vs created
    ==> Waiting for istiod to reconcile Gateway (15s)
    ==> Verifying Gateway resource is admitted
      ✓ Gateway sample-igw exists
    ==> Verifying VirtualService resource exists
      ✓ VirtualService sample-vs exists
    ==> Getting istio-ingressgateway pod IP
      Gateway pod IP: 10.244.0.6
    ==> Probing product through istio ingress gateway
    HTTP response through gateway: 200
    PASS: traffic routed through istio ingress gateway
    PASS: istio ingress gateway basic integration verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-service-mesh-istio-ingress-gateway-basic-20260609-101819/artifacts/logs

service-mesh-istio-ingress-gateway-jwt Istio Ingress Gateway — JWT Authentication PASS
Description
Installs istio/base + istio/istiod + istio/gateway, deploys the product chart, creates an Istio Gateway + VirtualService, applies a RequestAuthentication + AuthorizationPolicy requiring a valid JWT, and verifies: (a) requests without a Bearer token are rejected with 401/403, (b) requests with a valid JWT signed by the test issuer key from fixtures/service-mesh/jwt/ return HTTP 200.
Cluster
kind
Preinstalled
istio-base, istiod, istio-ingressgateway
Mechanisms
addon:istiomesh:istio-ingress-gateway
Tags
nightly
Labels
customer=any profile=mesh
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release istio-base in ns/istio-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release istiod in ns/istio-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release istio-ingressgateway in ns/istio-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-85 -n istio-system wait --for=condition=Ready pods --timeout=5m -l app=istiod
    pod/istiod-54b585df85-rb5gs condition met
    PASS: pods Ready in ns/istio-system (selector=app=istiod)
    NAME                      READY   STATUS    RESTARTS   AGE
    istiod-54b585df85-rb5gs   1/1     Running   0          12s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-85 -n istio-system wait --for=condition=Ready pods --timeout=3m -l app=istio-ingressgateway
    pod/istio-ingressgateway-8584b67974-n7rqc condition met
    PASS: pods Ready in ns/istio-system (selector=app=istio-ingressgateway)
    NAME                                    READY   STATUS    RESTARTS   AGE
    istio-ingressgateway-8584b67974-n7rqc   1/1     Running   0          7s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-85 -n sample wait --for=condition=Ready pods --timeout=3m --all
    pod/sample-7dcd986bd4-lkgxr condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-7dcd986bd4-lkgxr   1/1     Running   0          4s
  • PASS smoke-script
    notes
    ==> Creating Istio Gateway + VirtualService
    gateway.networking.istio.io/sample-igw created
    virtualservice.networking.istio.io/sample-vs created
    ==> Creating RequestAuthentication (JWT validation)
    configmap/jwt-jwks created
    requestauthentication.security.istio.io/sample-jwt-auth created
    ==> Creating AuthorizationPolicy (require JWT via ALLOW)
    authorizationpolicy.security.istio.io/sample-require-jwt created
    ==> Waiting for istiod to reconcile policies (15s)
    ==> Getting istio-ingressgateway pod IP
      Gateway pod IP: 10.244.0.6
    ==> Signing JWT with test issuer key
      JWT generated (expires in 1h)
    ==> Test 1: Request WITHOUT Bearer token → expect 401 or 403
      HTTP code (no auth): 403
      ✓ Request without token correctly rejected
    ==> Test 2: Request WITH valid Bearer token → expect 200
      HTTP code (with valid JWT): 200
      ✓ Request with valid JWT succeeds
    PASS: istio ingress gateway JWT authentication verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-service-mesh-istio-ingress-gateway-jwt-20260609-101922/artifacts/logs

service-mesh-istio-ingress-gateway-multi-host Istio Ingress Gateway — Multi-Host FAIL
Description
Installs istio/base + istio/istiod + istio/gateway, deploys the product chart with scope.enabled=true, creates an Istio Gateway with two server blocks for different hosts, and a VirtualService that routes each host to a different backend (skywatcher vs scope). Verifies both hosts return HTTP 200 through the ingressgateway.
Cluster
kind
Preinstalled
istio-base, istiod, istio-ingressgateway
Mechanisms
addon:istiomesh:istio-ingress-gateway
Tags
nightly
Labels
customer=any profile=mesh
Error
error 250 B · 1 ln
helm install failed; see /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-service-mesh-istio-ingress-gateway-multi-host-20260609-102026/artifacts/logs/product-install.log
Log snippet
product-install.log 174 B · 3 ln log snippet
Release "sample" does not exist. Installing it now.
Error: resource Deployment/sample/sample not ready. status: InProgress, message: Available: 0/1
context deadline exceeded

⚠ no agent reported on this scenario — UNTESTED

service-mesh-istio-ingress-gateway-request-authentication Istio Ingress Gateway — RequestAuthentication (No Deny) FAIL
Description
Installs istio/base + istio/istiod + istio/gateway, deploys the product chart, creates an Istio Gateway + VirtualService, applies a RequestAuthentication that validates JWTs from the test issuer but does NOT require them (no AuthorizationPolicy). Verifies: (a) requests without a token still pass through (HTTP 200), (b) requests with a valid JWT also return 200, and (c) requests with an invalid JWT (wrong issuer) are rejected with 401 by RequestAuthentication validation alone.
Cluster
kind
Preinstalled
istio-base, istiod, istio-ingressgateway
Mechanisms
addon:istiomesh:istio-ingress-gateway
Tags
nightly
Labels
customer=any profile=mesh
Error
error 262 B · 1 ln
helm install failed; see /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-service-mesh-istio-ingress-gateway-request-authentication-20260609-102606/artifacts/logs/product-install.log
Log snippet
product-install.log 174 B · 3 ln log snippet
Release "sample" does not exist. Installing it now.
Error: resource Deployment/sample/sample not ready. status: InProgress, message: Available: 0/1
context deadline exceeded

⚠ no agent reported on this scenario — UNTESTED

service-mesh-istio-service-mesh-cert-manager-tls Istio Service Mesh + cert-manager TLS Gateway FAIL
Description
Cross-feature compose: installs cert-manager + istio (base + istiod + gateway), creates a cert-manager self-signed ClusterIssuer and Certificate that issues a TLS Secret, then creates an Istio Gateway with a TLS listener referencing that Secret via credentialName. Verifies: istioctl analyze clean, HTTPS through gateway returns 200 with the cert-manager-issued certificate.
Cluster
kind
Preinstalled
cert-manager, istio-base, istiod, istio-ingressgateway
Mechanisms
addon:istioaddon:cert-managermesh:istio-service-mesh
Tags
nightly
Labels
customer=any profile=mesh-tls
Error
error 253 B · 1 ln
helm install failed; see /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-service-mesh-istio-service-mesh-cert-manager-tls-20260609-103144/artifacts/logs/product-install.log
Log snippet
product-install.log 174 B · 3 ln log snippet
Release "sample" does not exist. Installing it now.
Error: resource Deployment/sample/sample not ready. status: InProgress, message: Available: 0/1
context deadline exceeded

⚠ no agent reported on this scenario — UNTESTED

service-mesh-istio-service-mesh-peer-authentication Istio Service Mesh — PeerAuthentication lifecycle FAIL
Description
Installs istio/base + istio/istiod, deploys the product chart with mesh.inject=true, and exercises the PeerAuthentication lifecycle: PERMISSIVE mode allows both mesh and non-mesh traffic; switching to STRICT mode blocks non-mesh traffic while mesh traffic continues via auto-upgraded mTLS.
Cluster
kind
Preinstalled
istio-base, istiod
Mechanisms
addon:istiomesh:istio-service-mesh
Tags
nightly
Labels
customer=any profile=mesh
Error
error 256 B · 1 ln
helm install failed; see /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-service-mesh-istio-service-mesh-peer-authentication-20260609-103742/artifacts/logs/product-install.log
Log snippet
product-install.log 174 B · 3 ln log snippet
Release "sample" does not exist. Installing it now.
Error: resource Deployment/sample/sample not ready. status: InProgress, message: Available: 0/1
context deadline exceeded

⚠ no agent reported on this scenario — UNTESTED

service-mesh-istio-service-mesh-sidecar-injection Istio Service Mesh — Sidecar Injection FAIL
Description
Installs istio/base + istio/istiod, enables mesh injection on the product namespace, deploys the product chart with mesh.inject=true and scope.enabled=true, and verifies every product pod has exactly 2 containers including istio-proxy and in-mesh HTTP reaches the product Service with 200.
Cluster
kind
Preinstalled
istio-base, istiod
Mechanisms
addon:istiomesh:istio-service-mesh
Tags
nightly
Labels
customer=any profile=mesh
Error
error 254 B · 1 ln
helm install failed; see /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-service-mesh-istio-service-mesh-sidecar-injection-20260609-104316/artifacts/logs/product-install.log
Log snippet
product-install.log 174 B · 3 ln log snippet
Release "sample" does not exist. Installing it now.
Error: resource Deployment/sample/sample not ready. status: InProgress, message: Available: 0/1
context deadline exceeded

⚠ no agent reported on this scenario — UNTESTED

service-mesh-istio-service-mesh-strict-mtls Istio Service Mesh — strict mTLS FAIL
Description
Installs istio/base + istio/istiod, enables mesh injection, deploys the product chart with scope.enabled=true, creates a PeerAuthentication with mode=STRICT in the product namespace, and verifies: (a) plain HTTP from a non-mesh pod is rejected, (b) in-mesh probe pod still reaches the product Service with 200 via auto-upgraded mTLS.
Cluster
kind
Preinstalled
istio-base, istiod
Mechanisms
addon:istiomesh:istio-service-mesh
Tags
nightlycurated-live
Labels
customer=any profile=mesh
Error
error 248 B · 1 ln
helm install failed; see /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-service-mesh-istio-service-mesh-strict-mtls-20260609-104851/artifacts/logs/product-install.log
Log snippet
product-install.log 174 B · 3 ln log snippet
Release "sample" does not exist. Installing it now.
Error: resource Deployment/sample/sample not ready. status: InProgress, message: Available: 0/1
context deadline exceeded

⚠ no agent reported on this scenario — UNTESTED

service-mesh-istio-service-mesh-telemetry-v2 Istio Service Mesh — Telemetry v2 FAIL
Description
Installs istio/base + istio/istiod, deploys the product chart with mesh.inject=true, applies a Telemetry resource in the product namespace to configure Envoy access logging and metrics, and verifies proxy stats are accessible and telemetry configuration is active.
Cluster
kind
Preinstalled
istio-base, istiod
Mechanisms
addon:istiomesh:istio-service-mesh
Tags
nightly
Labels
customer=any profile=mesh
Error
error 249 B · 1 ln
helm install failed; see /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-service-mesh-istio-service-mesh-telemetry-v2-20260609-105426/artifacts/logs/product-install.log
Log snippet
product-install.log 174 B · 3 ln log snippet
Release "sample" does not exist. Installing it now.
Error: resource Deployment/sample/sample not ready. status: InProgress, message: Available: 0/1
context deadline exceeded

⚠ no agent reported on this scenario — UNTESTED

service-mesh-istio-sidecar-live Istio Sidecar Mesh — Live mTLS Verification FAIL
Description
Installs istio/base + istio/istiod, pre-labels the product namespace istio-injection=enabled via raw_manifest, deploys the chart with mesh.inject=true, and proves: (1) istiod is Ready and the sidecar injector webhook is present, (2) every product pod has exactly 2 containers including istio-proxy, (3) in-mesh traffic to the product Service returns HTTP 200 and the connection is mTLS — confirmed via istio-proxy stats reporting non-zero inbound SSL handshakes. Emits a PASS artifact bundle with 2-container pod manifest, applied-overrides recording inject: true, and versions.json.
Cluster
kind
Preinstalled
istio-base, istiod
Mechanisms
addon:istiomesh:istio-sidecar
Tags
nightlycurated-live
Labels
customer=any profile=mesh
Error
error 236 B · 1 ln
helm install failed; see /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-service-mesh-istio-sidecar-live-20260609-110001/artifacts/logs/product-install.log
Log snippet
product-install.log 174 B · 3 ln log snippet
Release "sample" does not exist. Installing it now.
Error: resource Deployment/sample/sample not ready. status: InProgress, message: Available: 0/1
context deadline exceeded

⚠ no agent reported on this scenario — UNTESTED

service-mesh-linkerd-basic-mesh Linkerd — Basic Mesh (Sidecar Injection) FAIL
Description
Installs linkerd-crds + linkerd-control-plane via Helm, annotates the product namespace with linkerd.io/inject=enabled, deploys the product chart with scope.enabled=true, and verifies every product pod has a linkerd-proxy sidecar alongside the app container (2-container pods), with linkerd check --proxy returning healthy.
Cluster
kind
Preinstalled
linkerd-crds, linkerd-control-plane
Mechanisms
addon:linkerdmesh:linkerd
Tags
nightly
Labels
customer=any profile=mesh
Error
error 236 B · 1 ln
helm install failed; see /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-service-mesh-linkerd-basic-mesh-20260609-110535/artifacts/logs/product-install.log
Log snippet
product-install.log 269 B · 4 ln log snippet
Release "sample" does not exist. Installing it now.
Error: resource Deployment/sample/sample-scope not ready. status: InProgress, message: Available: 0/1
resource Deployment/sample/sample not ready. status: InProgress, message: Available: 0/1
context deadline exceeded

⚠ no agent reported on this scenario — UNTESTED

service-mesh-linkerd-live Linkerd — Live mTLS Sidecar Mesh FAIL
Description
Installs linkerd-crds + linkerd-control-plane (with a preinstalled trust anchor / issuer cert via raw_manifest), annotates the product namespace with linkerd.io/inject=enabled, deploys the chart, and proves: (1) linkerd control-plane pods Ready and proxy-injector webhook present, (2) every product pod has a linkerd-proxy container alongside the app container, (3) linkerd check --proxy -n sample exits 0 confirming the data plane is healthy. Emits a PASS artifact bundle with a 2-container pod manifest, linkerd.io/inject-annotated namespace, and versions.json.
Cluster
kind
Preinstalled
linkerd-crds, linkerd-control-plane
Mechanisms
addon:linkerdmesh:linkerd
Tags
nightlycurated-live
Labels
customer=any profile=mesh
Error
error 230 B · 1 ln
helm install failed; see /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-service-mesh-linkerd-live-20260609-111107/artifacts/logs/product-install.log
Log snippet
product-install.log 174 B · 3 ln log snippet
Release "sample" does not exist. Installing it now.
Error: resource Deployment/sample/sample not ready. status: InProgress, message: Available: 0/1
context deadline exceeded

⚠ no agent reported on this scenario — UNTESTED

service-mesh-linkerd-mtls-rotation Linkerd — mTLS Identity Rotation FAIL
Description
Installs linkerd-crds + linkerd-control-plane via Helm, annotates the product namespace for injection, deploys the product chart, and verifies that mTLS identities are issued and valid by running linkerd check --proxy and inspecting the identity issuer certificate. Documents the mTLS rotation window and trust-anchor expiry.
Cluster
kind
Preinstalled
linkerd-crds, linkerd-control-plane
Mechanisms
addon:linkerdmesh:linkerd
Tags
nightly
Labels
customer=any profile=mesh
Error
error 239 B · 1 ln
helm install failed; see /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-service-mesh-linkerd-mtls-rotation-20260609-111638/artifacts/logs/product-install.log
Log snippet
product-install.log 269 B · 4 ln log snippet
Release "sample" does not exist. Installing it now.
Error: resource Deployment/sample/sample-scope not ready. status: InProgress, message: Available: 0/1
resource Deployment/sample/sample not ready. status: InProgress, message: Available: 0/1
context deadline exceeded

⚠ no agent reported on this scenario — UNTESTED

service-mesh-linkerd-multi-cluster-preview Linkerd — Multi-Cluster Preview FAIL
Description
Installs linkerd-crds + linkerd-control-plane + linkerd-multicluster extension via Helm, verifies the multicluster Link and ServiceMirror CRDs are established, and authors a preview Link resource targeting a logical target cluster. No real cross-cluster traffic — this variant validates that the multicluster extension installs cleanly and that the CRD scaffolding is functional.
Cluster
kind
Preinstalled
linkerd-crds, linkerd-control-plane, linkerd-multicluster
Mechanisms
addon:linkerdmesh:linkerd-multicluster
Tags
nightly
Labels
customer=any profile=mesh
Error
error 221 B · 1 ln
see /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-service-mesh-linkerd-multi-cluster-preview-20260609-112208/artifacts/logs/preinstall.log
Log snippet
preinstall.log 2202 B · 59 ln log snippet
  curl -sL https://run.linkerd.io/install | sh

Alternatively, you can download the CLI directly via the Linkerd releases page:

  https://github.com/linkerd/linkerd2/releases/

To make sure everything works as expected, run the following:

  linkerd check

The viz extension can be installed by running:

  helm install linkerd-viz linkerd/linkerd-viz

Looking for more? Visit https://linkerd.io/2/getting-started/

==> [3/3] kind=helm
    helm: linkerd-multicluster (linkerd/linkerd-multicluster @ 1.14.2) in ns/linkerd-multicluster
    repo 'linkerd' already exists with URL https://helm.linkerd.io/stable (skipping add)
Release "linkerd-multicluster" does not exist. Installing it now.
Error: chart "linkerd-multicluster" matching 1.14.2 not found in linkerd index. (try 'helm repo update'): no chart version found for linkerd-multicluster-1.14.2

⚠ no agent reported on this scenario — UNTESTED

service-mesh-linkerd-service-profile Linkerd — ServiceProfile (Per-Route Observability) FAIL
Description
Installs linkerd-crds + linkerd-control-plane via Helm, annotates the product namespace for injection, deploys the product chart, creates a ServiceProfile CRD for the sample Service defining routes with timeout and retry policies, and verifies the ServiceProfile is recognized.
Cluster
kind
Preinstalled
linkerd-crds, linkerd-control-plane
Mechanisms
addon:linkerdmesh:linkerd
Tags
nightly
Labels
customer=any profile=mesh
Error
error 241 B · 1 ln
helm install failed; see /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-service-mesh-linkerd-service-profile-20260609-112237/artifacts/logs/product-install.log
Log snippet
product-install.log 269 B · 4 ln log snippet
Release "sample" does not exist. Installing it now.
Error: resource Deployment/sample/sample-scope not ready. status: InProgress, message: Available: 0/1
resource Deployment/sample/sample not ready. status: InProgress, message: Available: 0/1
context deadline exceeded

⚠ no agent reported on this scenario — UNTESTED

serviceaccount-annotations-off Capability: ServiceAccount identity annotations absent when unset (OFF) PASS
Description
Renders the product chart with serviceAccount.create=true and no annotation overrides, then asserts the ServiceAccount does NOT carry any cloud-identity annotation keys (IRSA, Azure WI, GKE WI). The sample chart does NOT include a serviceaccount.yaml template — no ServiceAccount is rendered at all. Since no ServiceAccount exists, the off-case is vacuously satisfied. This scenario is EXPECTED to PASS — confirming no cloud-identity annotation leakage (and no ServiceAccount) when the knob is off.
Cluster
kind
Mechanisms
capability:serviceaccount-annotations
Tags
capabilitycompliance
Labels
customer=enterprise profile=compliance
Assertions · agent —
  • PASS serviceaccount-annotations
    notes
    PASS: no ServiceAccount rendered (vacuously satisfies absence of identity annotations)

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-serviceaccount-annotations-off-20260609-090743/artifacts/logs

serviceaccount-annotations-on Capability: ServiceAccount cloud-identity annotations settable (ON) FAIL
Description
Renders the product chart with serviceAccount.annotations configured for AWS IRSA (eks.amazonaws.com/role-arn) and asserts the rendered ServiceAccount carries the exact annotation key/value. The sample chart does NOT include a serviceaccount.yaml template — no ServiceAccount is rendered. Setting serviceAccount.annotations has no effect because the chart lacks both a ServiceAccount template and a serviceAccount.annotations knob. This scenario is EXPECTED to FAIL (honest red gap) documenting that the chart lacks ServiceAccount cloud-identity annotation support.
Cluster
kind
Mechanisms
capability:serviceaccount-annotations
Tags
capabilitycompliance
Labels
customer=enterprise profile=compliance
Error
error 133 B · 2 ln
serviceaccount-annotations:
FAIL: no ServiceAccount found in rendered output — chart lacks ServiceAccount template (documented gap)
Log snippet
assert-0-serviceaccount-annotations.log 106 B · 1 ln log snippet
FAIL: no ServiceAccount found in rendered output — chart lacks ServiceAccount template (documented gap)
Log
/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-serviceaccount-annotations-on-20260609-090814/artifacts/logs View full log
Assertions · agent —
  • FAIL serviceaccount-annotations
    notes
    FAIL: no ServiceAccount found in rendered output — chart lacks ServiceAccount template (documented gap)

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-serviceaccount-annotations-on-20260609-090814/artifacts/logs

subchart-postgres-internal Internal postgres subchart enabled FAIL
Description
Product chart bundles postgres via subchart; verify both come up together.
Cluster
kind
Mechanisms
subchart:postgres-internal
Tags
pr-subsetnightly
Labels
profile=full-stack
Error
error 231 B · 1 ln
helm install failed; see /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-subchart-postgres-internal-20260609-112806/artifacts/logs/product-install.log
Log snippet
product-install.log 174 B · 3 ln log snippet
Release "sample" does not exist. Installing it now.
Error: resource Deployment/sample/sample not ready. status: InProgress, message: Available: 0/1
context deadline exceeded

⚠ no agent reported on this scenario — UNTESTED

tls-cert-manager-self-signed cert-manager self-signed Issuer -> CA Cert -> CA Issuer -> leaf Cert PASS
Description
Installs cert-manager with CRDs, applies a self-signed Issuer, a CA Certificate bootstrapping a CA Secret, a CA Issuer backed by that Secret, and a leaf Certificate for the product Service FQDN. Deploys chart with tls.enabled=true and tls.secretName pointing at the issued Secret. Verifies HTTPS 200 with --cacert and SAN match.
Cluster
kind
Preinstalled
cert-manager
Mechanisms
addon:cert-manager
Tags
nightlycurated-live
Labels
customer=any profile=tls
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-37 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-7d59fd54fc-dzrxx condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-7d59fd54fc-dzrxx   1/1     Running   0          5s
  • PASS smoke-script
    notes
    PASS: CA Certificate Ready=True
    ==> Waiting for leaf Certificate 'sample-tls' Ready (3m max)
    certificate.cert-manager.io/sample-tls condition met
    PASS: leaf Certificate Ready=True
    ==> Verifying TLS Secret 'sample-tls' keys
    Secret keys: ca.crt
    tls.crt
    tls.key
    OK: tls.crt, tls.key, ca.crt all present
    ==> Waiting for pod Ready (3m max)
    pod/sample-7d59fd54fc-dzrxx condition met
    PASS: chart pods Ready
    ==> Verifying HTTPS reachable from in-cluster curl
    HTTPS response: 200
    PASS: HTTPS 200 with --cacert verification
    ==> Verifying peer certificate SAN
    SAN: X509v3 Subject Alternative Name: 
        DNS:sample.sample.svc, DNS:sample.sample.svc.cluster.local
    PASS: cert SAN includes configured host
    PASS: cert-manager self-signed CA chain integration verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-tls-cert-manager-self-signed-20260609-091609/artifacts/logs

tls-manual-secret Manual TLS Secret via raw_manifest PASS
Description
Delivers a pre-provisioned kubernetes.io/tls Secret (RSA 2048) via raw_manifest preinstall sourced from chart-test/fixtures/, no inline base64 blobs. The chart mounts it with tls.volumeType=secret (default) and serves HTTPS 200. Validates Secret type, data keys, PEM validity, and in-cluster HTTPS reachability.
Cluster
kind
Mechanisms
addon:manual-tls-secret
Tags
nightlycurated-live
Labels
customer=any profile=tls
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-38 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-5bff6cf44d-5z9dg condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-5bff6cf44d-5z9dg   1/1     Running   0          4s
  • PASS smoke-script
    notes
    ==> Verifying TLS Secret 'manual-tls-basic' type
    Secret type: kubernetes.io/tls
    PASS: Secret type is kubernetes.io/tls
    ==> Verifying TLS Secret 'manual-tls-basic' data keys
    Secret keys: tls.crt
    tls.key
    OK: tls.crt, tls.key both present
    ==> Verifying tls.crt is valid PEM
    subject=C=US, ST=CA, O=Chart Test Swarm, CN=sample.sample.svc
    PASS: tls.crt is a valid certificate
    ==> Verifying tls.key is valid PEM
    RSA key ok
    PASS: tls.key is a valid private key
    ==> Waiting for pod Ready (3m max)
    pod/sample-5bff6cf44d-5z9dg condition met
    PASS: chart pods Ready
    ==> Verifying HTTPS reachable from in-cluster curl
    HTTPS response: 200
    PASS: HTTPS 200
    PASS: manual-tls-secret basic scenario verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-tls-manual-secret-20260609-091706/artifacts/logs

tls-mounted-projected TLS via projected volume PASS
Description
Creates a kubernetes.io/tls Secret, then deploys the chart with tls.volumeType=projected so the pod has a projected volume sourcing the TLS Secret. Verifies projected volume source, tls.crt/tls.key present at mountPath, and in-cluster HTTPS curl returns 200.
Cluster
kind
Mechanisms
addon:mounted-tls-certs
Tags
nightlycurated-live
Labels
customer=any profile=tls
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-39 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-55776c48fb-sv9fq condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-55776c48fb-sv9fq   1/1     Running   0          4s
  • PASS smoke-script
    notes
    PASS: projected volume sources include secret 'mounted-tls-projected'
    ==> Verifying mountPath matches tls.mountPath
    PASS: mountPath=/etc/tls
    ==> Waiting for pod Ready (3m max)
    pod/sample-55776c48fb-sv9fq condition met
    PASS: chart pods Ready
    ==> Exec stat /etc/tls/tls.crt in pod
    Defaulted container "app" out of: app, wait-for-tls (init)
      File: '/etc/tls/tls.crt' -> '..data/tls.crt'
      Size: 14        Blocks: 0          IO Block: 4096   symbolic link
    Device: e4h/228dInode: 7           Links: 1
    Access: (0777/lrwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/    root)
    Access: 2026-06-09 13:18:11.087088008 +0000
    Modify: 2026-06-09 13:18:09.253088007 +0000
    Change: 2026-06-09 13:18:09.253088007 +0000
    PASS: tls.crt accessible at /etc/tls
    ==> Verifying HTTPS reachable from in-cluster curl
    HTTPS response: 200
    PASS: HTTPS 200
    PASS: mounted-tls-certs projected-volume scenario verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-tls-mounted-projected-20260609-091743/artifacts/logs

tls-mounted-pvc TLS via persistentVolumeClaim PASS
Description
Creates a PVC, populates it with TLS certs via a Job, then deploys the chart with tls.volumeType=persistentVolumeClaim and tls.pvcClaimName pointing at the pre-populated PVC. Verifies the pod mount, cert files at mountPath, and in-cluster HTTPS curl returns 200. Artifact bundle records the exact volumeType=persistentVolumeClaim.
Cluster
kind
Mechanisms
addon:mounted-tls-certs
Tags
nightly
Labels
customer=any profile=tls
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-40 -n sample wait --for=condition=Ready pods --timeout=10m -l app=sample
    pod/sample-9f9948847-b68vp condition met
    PASS: pods Ready in ns/sample (selector=app=sample)
    NAME                     READY   STATUS    RESTARTS   AGE
    sample-9f9948847-b68vp   1/1     Running   0          11s
  • PASS smoke-script
    notes
    PASS: mountPath=/etc/tls
    ==> Verifying volume source is persistentVolumeClaim
    PASS: volume source is persistentVolumeClaim (claim=mounted-tls-pvc)
    ==> Waiting for pod Ready (3m max)
    pod/sample-9f9948847-b68vp condition met
    PASS: chart pods Ready
    ==> Exec stat /etc/tls/tls.crt in pod
    Defaulted container "app" out of: app, wait-for-tls (init)
      File: /etc/tls/tls.crt
      Size: 1176      Blocks: 8          IO Block: 4096   regular file
    Device: fe01h/65025dInode: 12018       Links: 1
    Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
    Access: 2026-06-09 13:18:49.869778012 +0000
    Modify: 2026-06-09 13:18:49.869778012 +0000
    Change: 2026-06-09 13:18:49.870778012 +0000
    PASS: tls.crt accessible at /etc/tls
    ==> Verifying HTTPS reachable from in-cluster curl
    HTTPS response: 200
    PASS: HTTPS 200
    PASS: mounted-tls-certs pvc-mount scenario verified

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-tls-mounted-pvc-20260609-091818/artifacts/logs

with-cert-manager Customer ships cert-manager preinstalled PASS
Description
Validates chart coexists with cert-manager and its CRDs.
Cluster
kind
Preinstalled
cert-manager
Mechanisms
addon:cert-manager
Tags
pr-subsetnightly
Labels
customer=any profile=tls
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-41 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-7dcd986bd4-j5dr4 condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-7dcd986bd4-j5dr4   1/1     Running   0          3s

Logs: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/reports/run-full-bench-20260609-085346/scenario-with-cert-manager-20260609-091900/artifacts/logs