chart-test-swarm — run-curated-live-20260602-234624-95262

Time (UTC)
2026-06-03 03:46:25+00:00
Project
sample-product-chart
Chart
sample-product:0.1.0
Suite
curated-live
Cluster
kind · k8s v1.30.0
K8s Server
v1.30.0
Agents
1
FAIL · 7 PASS · 11

Mechanism coverage

Worst status across all scenarios that exercise each mechanism. Covers: addons (cert-manager, istio, gatekeeper, …), subchart variants, customer profiles.

addon

mesh

other

Scenarios

One row per scenario. Cell value: passed / total asserts. Groups with 3+ variants per integration are collapsed — click the header to expand.

Scenario Cluster shape Tags Asserts Status Agent
annotations-on
Capability: custom annotations stamped on every object (ON)
kind capabilitycompliancecurated-live 0/0 FAIL —
ingress-controllers-contour-basic-httpproxy
Contour basic HTTPProxy (gap-probe)
kind
+ contour
nightlycurated-live 3/4 FAIL —
labels-on
Capability: extra labels stamped on every object (ON)
kind capabilitycompliancecurated-live 0/0 FAIL —
networking-kong-ingress
Kong Ingress (className=kong)
kind
+ kong
nightlycurated-live 4/4 PASS —
networking-metallb-loadbalancer
MetalLB LoadBalancer (service.type=LoadBalancer)
kind
+ metallb
nightlycurated-live 4/4 PASS —
networking-traefik-ingress
Traefik Ingress (className=traefik)
kind
+ traefik
nightlycurated-live 4/4 PASS —
policy-opa-gatekeeper-required-labels
OPA Gatekeeper required labels enforcement (gap-probe)
kind
+ gatekeeper, ingress-nginx
nightlycurated-live 4/5 FAIL —
rbac-on
Capability: RBAC objects present when enabled (ON)
kind capabilitycompliancecurated-live 0/1 FAIL —
scheme-https-only
Capability: HTTPS-only scheme enforcement (ON)
kind capabilitycompliancecurated-live 0/0 FAIL —
service-mesh-istio-ambient-live
Istio Ambient Mesh — Live mTLS Verification (ztunnel/HBONE)
kind
+ istio-base, istiod, istio-cni, ztunnel
nightlycurated-live 9/9 PASS —
service-mesh-linkerd-live
Linkerd — Live mTLS Sidecar Mesh
kind
+ linkerd-crds, linkerd-control-plane
nightlycurated-live 0/0 FAIL —
tls-cert-manager-self-signed
cert-manager self-signed Issuer -> CA Cert -> CA Issuer -> leaf Cert
kind
+ cert-manager
nightlycurated-live 3/3 PASS —
tls-manual-secret
Manual TLS Secret via raw_manifest
kind nightlycurated-live 3/3 PASS —
tls-mounted-projected
TLS via projected volume
kind nightlycurated-live 3/3 PASS —
PASS 4 variants: 4 PASS addon/istio
Scenario Cluster shape Tags Asserts Status Agent
gateway-api-istio-gateway-api-basic
Istio Gateway API Basic
kind
+ istio-base, istiod
nightlycurated-live 6/6 PASS —
service-mesh-istio-ingress-gateway-basic
Istio Ingress Gateway — Basic
kind
+ istio-base, istiod, istio-ingressgateway
nightlycurated-live 8/8 PASS —
service-mesh-istio-service-mesh-strict-mtls
Istio Service Mesh — strict mTLS
kind
+ istio-base, istiod
nightlycurated-live 6/6 PASS —
service-mesh-istio-sidecar-live
Istio Sidecar Mesh — Live mTLS Verification
kind
+ istio-base, istiod
nightlycurated-live 6/6 PASS —

Scenarios — details

annotations-on Capability: custom annotations stamped on every object (ON) FAIL
Description
Renders the product chart with a global annotation knob set (extraAnnotations.example\.com/owner=team-x) and asserts every rendered object (all kinds) carries the configured annotation at .metadata.annotations. The sample chart does NOT expose an extraAnnotations/commonAnnotations knob — this scenario is EXPECTED to FAIL (honest red gap) documenting that the chart lacks global annotation propagation.
Cluster
kind
Mechanisms
capability:annotations
Tags
capabilitycompliancecurated-live
Labels
customer=enterprise profile=compliance
Error
error 51 B · 1 ln
result.yaml had parse errors; status from aggregate
Log snippet
preinstall.log 217 B · 6 ln log snippet
==> PROJECT_DIR=/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart
==> No preinstall items; skipping addon phase

==> OK: preinstall complete

==> Skipping product chart install (--preinstall-only)

⚠ no agent reported on this scenario — UNTESTED

gateway-api-istio-gateway-api-basic Istio Gateway API Basic PASS
Description
Installs istio/base + istio/istiod and Gateway API CRDs, creates GatewayClass istio + Gateway + HTTPRoute, and verifies HTTP routing through auto-provisioned Istio data-plane.
Cluster
kind
Preinstalled
istio-base, istiod
Mechanisms
addon:istiogateway-api:istio
Tags
nightlycurated-live
Labels
customer=any profile=gateway-api
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release istio-base in ns/istio-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release istiod in ns/istio-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-8 -n istio-system wait --for=condition=Ready pods --timeout=5m -l app=istiod
    pod/istiod-54b585df85-7znr8 condition met
    PASS: pods Ready in ns/istio-system (selector=app=istiod)
    NAME                      READY   STATUS    RESTARTS   AGE
    istiod-54b585df85-7znr8   1/1     Running   0          10s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-8 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-7dcd986bd4-gn9ch condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-7dcd986bd4-gn9ch   1/1     Running   0          3s
  • PASS smoke-script
    notes
    ==> Applying GatewayClass + Gateway + HTTPRoute
    Warning: resource gatewayclasses/istio is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
    gatewayclass.gateway.networking.k8s.io/istio configured
    gateway.gateway.networking.k8s.io/sample-gw created
    httproute.gateway.networking.k8s.io/sample-route created
    ==> Waiting for GatewayClass istio Accepted=True (3m max)
    PASS: GatewayClass istio Accepted=True
    ==> Waiting for Gateway listener http Programmed=True (5m max)
    PASS: Gateway listener http Programmed=True
    ==> Verifying auto-provisioned Deployment in sample namespace
    PASS: auto-provisioned Deployment found (1)
    ==> Verifying HTTPRoute sample-route Accepted=True
    PASS: HTTPRoute sample-route Accepted=True
    ==> Getting Istio data-plane Service ClusterIP
    Gateway Service: sample-gw-istio IP: 10.96.31.85
    ==> Probing backend via gateway (retry up to 2m for data-plane ready)
    HTTP response: 200 (attempt 2)
    HTTP response: 200
    PASS: HTTP 200 through Istio Gateway API
    PASS: istio-gateway-api basic integration verified

Logs: examples/sample-product-chart/chart-test/reports/run-curated-live-20260602-234624-95262/scenario-gateway-api-istio-gateway-api-basic-20260602-235040/artifacts/logs

ingress-controllers-contour-basic-httpproxy Contour basic HTTPProxy (gap-probe) FAIL
Description
Installs Contour, creates an HTTPProxy for the product Service via fixture, and verifies HTTP routing through the envoy pod IP with Host header matching and HTTPProxy status valid. Then runs a gap-probe: the chart does NOT natively emit a Contour HTTPProxy CRD — the HTTPProxy was created by the fixture, not by a chart template. This is an honest gap (red cell); do NOT over-engineer the chart.
Cluster
kind
Preinstalled
contour
Mechanisms
addon:contour
Tags
nightlycurated-live
Labels
customer=any profile=ingress
Error
error 1436 B · 29 ln
smoke-script:
exec: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/assertions/contour-basic.sh (RELEASE=sample NAMESPACE=sample)
==> Waiting for envoy pod Ready (3m max)
pod/contour-envoy-vxvcj condition met
PASS: envoy pod Ready
==> Waiting for contour controller pod Ready (3m max)
pod/contour-contour-5847f7c7f9-vxwnb condition met
PASS: contour controller pod Ready
==> Waiting for product pod Ready (3m max)
pod/sample-69469cb4c6-wr6ns condition met
PASS: product pods Ready
==> Getting envoy pod IP (probe via pod IP)
envoy pod IP: 10.244.0.7
==> Verifying HTTPProxy exists and status is valid
HTTPProxy status: valid
PASS: HTTPProxy status is valid
==> Probing HTTP with Host header (expect 200) on envoy container port 8080
HTTP response (with Host): 200
PASS: HTTP 200 with Host header
==> Probing HTTP without Host header (expect 404) on envoy container port 8080
HTTP response (without Host): 404
PASS: HTTP 404 without Host header (envoy default)
PASS: Contour basic HTTPProxy integration verified
==> GAP-PROBE: Checking if chart natively emits an HTTPProxy CRD
GAP-PROBE: No HTTPProxy emitted by the chart template found
  The HTTPProxy 'sample-basic' was created by the raw_manifest fixture,
  not by a Helm-managed chart template. The chart has no contour
  HTTPProxy template — this is an honest gap (red cell).
FAIL: Chart does not natively emit Contour HTTPProxy CRD — honest gap
Log snippet
assert-3-smoke-script.log 1423 B · 28 ln log snippet
==> Waiting for product pod Ready (3m max)
pod/sample-69469cb4c6-wr6ns condition met
PASS: product pods Ready
==> Getting envoy pod IP (probe via pod IP)
envoy pod IP: 10.244.0.7
==> Verifying HTTPProxy exists and status is valid
HTTPProxy status: valid
PASS: HTTPProxy status is valid
==> Probing HTTP with Host header (expect 200) on envoy container port 8080
HTTP response (with Host): 200
PASS: HTTP 200 with Host header
==> Probing HTTP without Host header (expect 404) on envoy container port 8080
HTTP response (without Host): 404
PASS: HTTP 404 without Host header (envoy default)
PASS: Contour basic HTTPProxy integration verified
==> GAP-PROBE: Checking if chart natively emits an HTTPProxy CRD
GAP-PROBE: No HTTPProxy emitted by the chart template found
  The HTTPProxy 'sample-basic' was created by the raw_manifest fixture,
  not by a Helm-managed chart template. The chart has no contour
  HTTPProxy template — this is an honest gap (red cell).
FAIL: Chart does not natively emit Contour HTTPProxy CRD — honest gap
Log
examples/sample-product-chart/chart-test/reports/run-curated-live-20260602-234624-95262/scenario-ingress-controllers-contour-basic-httpproxy-20260602-235135/artifacts/logs View full log
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-9 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-69469cb4c6-wr6ns condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-69469cb4c6-wr6ns   1/1     Running   0          3s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-9 -n projectcontour wait --for=condition=Ready pods --timeout=3m --all
    pod/contour-contour-5847f7c7f9-vxwnb condition met
    pod/contour-envoy-vxvcj condition met
    PASS: pods Ready in ns/projectcontour
    NAME                               READY   STATUS    RESTARTS   AGE
    contour-contour-5847f7c7f9-vxwnb   1/1     Running   0          28s
    contour-envoy-vxvcj                2/2     Running   0          28s
  • FAIL smoke-script
    notes
    exec: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/assertions/contour-basic.sh (RELEASE=sample NAMESPACE=sample)
    ==> Waiting for envoy pod Ready (3m max)
    pod/contour-envoy-vxvcj condition met
    PASS: envoy pod Ready
    ==> Waiting for contour controller pod Ready (3m max)
    pod/contour-contour-5847f7c7f9-vxwnb condition met
    PASS: contour controller pod Ready
    ==> Waiting for product pod Ready (3m max)
    pod/sample-69469cb4c6-wr6ns condition met
    PASS: product pods Ready
    ==> Getting envoy pod IP (probe via pod IP)
    envoy pod IP: 10.244.0.7
    ==> Verifying HTTPProxy exists and status is valid
    HTTPProxy status: valid
    PASS: HTTPProxy status is valid
    ==> Probing HTTP with Host header (expect 200) on envoy container port 8080
    HTTP response (with Host): 200
    PASS: HTTP 200 with Host header
    ==> Probing HTTP without Host header (expect 404) on envoy container port 8080
    HTTP response (without Host): 404
    PASS: HTTP 404 without Host header (envoy default)
    PASS: Contour basic HTTPProxy integration verified
    ==> GAP-PROBE: Checking if chart natively emits an HTTPProxy CRD
    GAP-PROBE: No HTTPProxy emitted by the chart template found
      The HTTPProxy 'sample-basic' was created by the raw_manifest fixture,
      not by a Helm-managed chart template. The chart has no contour
      HTTPProxy template — this is an honest gap (red cell).
    FAIL: Chart does not natively emit Contour HTTPProxy CRD — honest gap

Logs: examples/sample-product-chart/chart-test/reports/run-curated-live-20260602-234624-95262/scenario-ingress-controllers-contour-basic-httpproxy-20260602-235135/artifacts/logs

labels-on Capability: extra labels stamped on every object (ON) FAIL
Description
Renders the product chart with a global extra-label knob set (extraLabels.team=platform, extraLabels.cost-center=42) and asserts every rendered object (all kinds, not just pod templates) carries the configured labels at .metadata.labels. The sample chart does NOT expose an extraLabels/commonLabels knob — this scenario is EXPECTED to FAIL (honest red gap) documenting that the chart lacks global label propagation.
Cluster
kind
Mechanisms
capability:labels
Tags
capabilitycompliancecurated-live
Labels
customer=enterprise profile=compliance
Error
error 51 B · 1 ln
result.yaml had parse errors; status from aggregate
Log snippet
preinstall.log 217 B · 6 ln log snippet
==> PROJECT_DIR=/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart
==> No preinstall items; skipping addon phase

==> OK: preinstall complete

==> Skipping product chart install (--preinstall-only)

⚠ no agent reported on this scenario — UNTESTED

networking-kong-ingress Kong Ingress (className=kong) PASS
Description
Installs Kong Ingress Controller, deploys the product chart with ingress.enabled=true and ingress.className=kong, and verifies end-to-end routing through the Kong proxy data path via the chart's built-in Ingress resource. Also includes a KongPlugin gap-probe documenting that the sample chart exposes no konghq.com/plugins annotation or KongPlugin CRD knob, which is an honest gap (red cell) — do NOT over-engineer the chart.
Cluster
kind
Preinstalled
kong
Mechanisms
addon:kong
Tags
nightlycurated-live
Labels
customer=any profile=ingress
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-10 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-69469cb4c6-jcbbz condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-69469cb4c6-jcbbz   1/1     Running   0          2s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-10 -n kong wait --for=condition=Ready pods --timeout=5m --all
    pod/kong-controller-84db4dc787-xlsn7 condition met
    pod/kong-gateway-86698f6995-h8cmq condition met
    PASS: pods Ready in ns/kong
    NAME                               READY   STATUS    RESTARTS   AGE
    kong-controller-84db4dc787-xlsn7   1/1     Running   0          24s
    kong-gateway-86698f6995-h8cmq      1/1     Running   0          24s
  • PASS smoke-script
    notes
    pod/sample-69469cb4c6-jcbbz condition met
    PASS: product pods Ready
    ==> Verifying Ingress has spec.ingressClassName=kong
    PASS: Ingress ingressClassName=kong
    ==> Verifying Ingress backend points at the product Service
    PASS: Ingress backend service=sample
    ==> Getting Kong proxy (gateway) pod IP
    Kong proxy pod IP: 10.244.0.6
    ==> Probing HTTP with matching Host header (expect 200) on Kong proxy port 8000
    HTTP response (with Host): 200
    PASS: HTTP 200 with matching Host header through Kong proxy
    ==> Probing HTTP with non-matching Host header (expect 404) on Kong proxy port 8000
    HTTP response (non-matching Host): 404
    PASS: HTTP 404 with non-matching Host header
    ==> KongPlugin gap-probe: checking if chart Ingress has konghq.com/plugins annotation
    GAP: chart Ingress does NOT have konghq.com/plugins annotation (honest gap, red cell)
    ==> KongPlugin gap-probe: checking if any KongPlugin CRD instances exist
    GAP: No KongPlugin CRD instances found in namespace sample (chart exposes no KongPlugin knob, honest gap, red cell)
    PASS: Kong Ingress (className=kong) integration verified end-to-end
    NOTE: KongPlugin gap-probe documented honestly — chart does not expose konghq.com/plugins or KongPlugin CRD

Logs: examples/sample-product-chart/chart-test/reports/run-curated-live-20260602-234624-95262/scenario-networking-kong-ingress-20260602-235245/artifacts/logs

networking-metallb-loadbalancer MetalLB LoadBalancer (service.type=LoadBalancer) PASS
Description
Installs MetalLB, configures an IPAddressPool + L2Advertisement whose CIDR is inside the kind Docker bridge subnet, deploys the product chart with service.type=LoadBalancer, and verifies that MetalLB assigns an external IP from the pool and the LB endpoint serves HTTP 200.
Cluster
kind
Preinstalled
metallb
Mechanisms
addon:metallb
Tags
nightlycurated-live
Labels
customer=any profile=loadbalancer
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-11 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-69469cb4c6-pg8x4 condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-69469cb4c6-pg8x4   1/1     Running   0          2s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-11 -n metallb wait --for=condition=Ready pods --timeout=5m --all
    pod/metallb-controller-7b8b56ccdd-pp5ht condition met
    pod/metallb-frr-k8s-5czl4 condition met
    pod/metallb-frr-k8s-statuscleaner-67874f5d9f-xdbf9 condition met
    pod/metallb-speaker-dblpw condition met
    PASS: pods Ready in ns/metallb
    NAME                                             READY   STATUS    RESTARTS   AGE
    metallb-controller-7b8b56ccdd-pp5ht              1/1     Running   0          32s
    metallb-frr-k8s-5czl4                            5/5     Running   0          32s
    metallb-frr-k8s-statuscleaner-67874f5d9f-xdbf9   1/1     Running   0          32s
    metallb-speaker-dblpw                            1/1     Running   0          32s
  • PASS smoke-script
    notes
    pod/metallb-speaker-dblpw condition met
    PASS: MetalLB speaker pods Ready
    ==> Verifying IPAddressPool exists
    PASS: IPAddressPool chart-test-pool found
    ==> Verifying L2Advertisement exists
    PASS: L2Advertisement chart-test-l2 found
    ==> Waiting for product pod Ready (5m max)
    pod/sample-69469cb4c6-pg8x4 condition met
    PASS: product pods Ready
    ==> Verifying Service has spec.type=LoadBalancer
    PASS: Service type=LoadBalancer
    ==> Waiting for MetalLB to assign external IP (60s max)
    MetalLB assigned external IP: 172.20.255.200
    PASS: MetalLB assigned external IP 172.20.255.200
    ==> Verifying assigned IP is within the configured pool
    PASS: Assigned IP 172.20.255.200 is within pool range 172.20.255.200-172.20.255.250
    ==> Probing LB endpoint via in-cluster curl (expect 200)
    HTTP response from LB IP: 200
    PASS: HTTP 200 from LB endpoint 172.20.255.200
    PASS: MetalLB LoadBalancer integration verified end-to-end

Logs: examples/sample-product-chart/chart-test/reports/run-curated-live-20260602-234624-95262/scenario-networking-metallb-loadbalancer-20260602-235344/artifacts/logs

networking-traefik-ingress Traefik Ingress (className=traefik) PASS
Description
Installs Traefik, deploys the product chart with ingress.enabled=true and ingress.className=traefik, and verifies end-to-end routing through the Traefik data path via the chart's built-in Ingress resource.
Cluster
kind
Preinstalled
traefik
Mechanisms
addon:traefik
Tags
nightlycurated-live
Labels
customer=any profile=ingress
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-12 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-69469cb4c6-bmfgw condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-69469cb4c6-bmfgw   1/1     Running   0          3s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-12 -n traefik wait --for=condition=Ready pods --timeout=3m --all
    pod/traefik-77657478fd-k72h5 condition met
    PASS: pods Ready in ns/traefik
    NAME                       READY   STATUS    RESTARTS   AGE
    traefik-77657478fd-k72h5   1/1     Running   0          14s
  • PASS smoke-script
    notes
    pod/traefik-77657478fd-k72h5 condition met
    PASS: Traefik pod Ready
    ==> Verifying IngressClass traefik is registered
    PASS: IngressClass traefik registered
    ==> Waiting for product pod Ready (5m max)
    pod/sample-69469cb4c6-bmfgw condition met
    PASS: product pods Ready
    ==> Verifying Ingress has spec.ingressClassName=traefik
    PASS: Ingress ingressClassName=traefik
    ==> Verifying Ingress backend points at the product Service
    PASS: Ingress backend service=sample
    ==> Getting Traefik pod IP (probe via pod IP)
    Traefik pod IP: 10.244.0.5
    ==> Probing HTTP with matching Host header (expect 200) on container port 8000
    HTTP response (with Host): 200
    PASS: HTTP 200 with matching Host header
    ==> Probing HTTP with non-matching Host header (expect 404) on container port 8000
    HTTP response (non-matching Host): 404
    PASS: HTTP 404 with non-matching Host header
    PASS: Traefik Ingress (className=traefik) integration verified end-to-end

Logs: examples/sample-product-chart/chart-test/reports/run-curated-live-20260602-234624-95262/scenario-networking-traefik-ingress-20260602-235449/artifacts/logs

policy-opa-gatekeeper-required-labels OPA Gatekeeper required labels enforcement (gap-probe) FAIL
Description
Installs OPA Gatekeeper + NGINX Ingress, creates k8srequiredlabels ConstraintTemplate + Constraint targeting Deployments and Ingresses. Verifies non-compliant resources denied, compliant accepted. Cross-feature compose with M4 nginx-ingress. Then runs a gap-probe: the chart's Ingress template uses selectorLabels (just app: <release>) instead of the full common labels (which include app.kubernetes.io/name), so the chart's Ingress fails the required-labels constraint — an honest gap (red cell); do NOT over-engineer the chart.
Cluster
kind
Preinstalled
gatekeeper, ingress-nginx
Mechanisms
addon:opa-gatekeeperaddon:nginx-ingress
Tags
nightlycurated-live
Labels
customer=any profile=policy
Error
error 3465 B · 41 ln
smoke-script:
PASS: compliant Deployment accepted
==> Phase 3a: Non-compliant Ingress (missing app.kubernetes.io/name)
Error from server (Forbidden): error when creating "/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/fixtures/policy/test-ingress-noncompliant.yaml": admission webhook "validation.gatekeeper.sh" denied the request: [require-app-name-label] Missing required label(s): {"app.kubernetes.io/name"}
PASS: non-compliant Ingress rejected (missing required label)
==> Phase 3b: Compliant Ingress (has app.kubernetes.io/name)
ingress.networking.k8s.io/test-compliant-ingress created
PASS: compliant Ingress applied
NOTE: compliant Ingress may not yet have address (NGINX controller processes async)
==> Phase 4: Webhook failure mode — verify Fail policy + outage behavior
PASS: webhook gatekeeper-validating-webhook-configuration failurePolicy=Fail
Scaling gatekeeper-controller-manager to 0 in gatekeeper-system (original replicas: 3)...
deployment.apps/gatekeeper-controller-manager scaled
pod/gatekeeper-controller-manager-64bbbfbdbb-bqbvq condition met
pod/gatekeeper-controller-manager-64bbbfbdbb-fjpnx condition met
pod/gatekeeper-controller-manager-64bbbfbdbb-xn75d condition met
PASS: gatekeeper controller scaled to 0
Attempting kubectl apply with webhook unavailable...
Error from server (InternalError): error when creating "/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/fixtures/policy/test-deploy-compliant.yaml": Internal error occurred: failed calling webhook "validation.gatekeeper.sh": failed to call webhook: Post "https://gatekeeper-webhook-service.gatekeeper-system.svc:443/v1/admit?timeout=3s": dial tcp 10.96.91.237:443: connect: connection refused
PASS: admission correctly blocked during webhook outage (Fail policy)
Restoring gatekeeper-controller-manager to 3 in gatekeeper-system...
deployment.apps/gatekeeper-controller-manager scaled
Waiting for gatekeeper controller pods to be Ready again...
pod/gatekeeper-controller-manager-64bbbfbdbb-cbpzc condition met
pod/gatekeeper-controller-manager-64bbbfbdbb-g6j4d condition met
pod/gatekeeper-controller-manager-64bbbfbdbb-ttx4k condition met
PASS: gatekeeper controller pods Ready
Waiting for admission webhook to accept requests (180s max, may be slow on resource-constrained clusters)...
deployment.apps/test-compliant created (server dry run)
PASS: admission restored after webhook recovery (attempt 1)
PASS: OPA Gatekeeper required-labels + nginx-ingress cross-feature compose verified
==> Phase 5: GAP-PROBE — Does the chart's own Ingress pass the required-labels constraint?
Rendering chart Ingress with ingress.enabled=true for gap-probe dry-run...
GAP-PROBE: Chart Ingress does NOT have app.kubernetes.io/name label
  The chart's Ingress template uses selectorLabels (just 'app: <release>')
  instead of the full common labels (which include 'app.kubernetes.io/name').
  This is an honest gap — the Ingress would be denied in a cluster with
  the K8sRequiredLabels constraint active before chart installation.
Error from server (Forbidden): error when creating "STDIN": admission webhook "validation.gatekeeper.sh" denied the request: [require-app-name-label] Missing required label(s): {"app.kubernetes.io/name"}
GAP-PROBE confirmed: Chart Ingress denied by required-labels constraint (honest gap, red cell)
FAIL: Chart Ingress does not carry app.kubernetes.io/name label — honest gap (red cell)
Log snippet
assert-4-smoke-script.log 5244 B · 65 ln log snippet
Restoring gatekeeper-controller-manager to 3 in gatekeeper-system...
deployment.apps/gatekeeper-controller-manager scaled
Waiting for gatekeeper controller pods to be Ready again...
pod/gatekeeper-controller-manager-64bbbfbdbb-cbpzc condition met
pod/gatekeeper-controller-manager-64bbbfbdbb-g6j4d condition met
pod/gatekeeper-controller-manager-64bbbfbdbb-ttx4k condition met
PASS: gatekeeper controller pods Ready
Waiting for admission webhook to accept requests (180s max, may be slow on resource-constrained clusters)...
deployment.apps/test-compliant created (server dry run)
PASS: admission restored after webhook recovery (attempt 1)
PASS: OPA Gatekeeper required-labels + nginx-ingress cross-feature compose verified
==> Phase 5: GAP-PROBE — Does the chart's own Ingress pass the required-labels constraint?
Rendering chart Ingress with ingress.enabled=true for gap-probe dry-run...
GAP-PROBE: Chart Ingress does NOT have app.kubernetes.io/name label
  The chart's Ingress template uses selectorLabels (just 'app: <release>')
  instead of the full common labels (which include 'app.kubernetes.io/name').
  This is an honest gap — the Ingress would be denied in a cluster with
  the K8sRequiredLabels constraint active before chart installation.
Error from server (Forbidden): error when creating "STDIN": admission webhook "validation.gatekeeper.sh" denied the request: [require-app-name-label] Missing required label(s): {"app.kubernetes.io/name"}
GAP-PROBE confirmed: Chart Ingress denied by required-labels constraint (honest gap, red cell)
FAIL: Chart Ingress does not carry app.kubernetes.io/name label — honest gap (red cell)
Log
examples/sample-product-chart/chart-test/reports/run-curated-live-20260602-234624-95262/scenario-policy-opa-gatekeeper-required-labels-20260602-235537/artifacts/logs View full log
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-13 -n gatekeeper-system wait --for=condition=Ready pods --timeout=4m --all
    pod/gatekeeper-audit-65579cd9f9-79bl7 condition met
    pod/gatekeeper-controller-manager-64bbbfbdbb-bqbvq condition met
    pod/gatekeeper-controller-manager-64bbbfbdbb-fjpnx condition met
    pod/gatekeeper-controller-manager-64bbbfbdbb-xn75d condition met
    PASS: pods Ready in ns/gatekeeper-system
    NAME                                             READY   STATUS    RESTARTS   AGE
    gatekeeper-audit-65579cd9f9-79bl7                1/1     Running   0          44s
    gatekeeper-controller-manager-64bbbfbdbb-bqbvq   1/1     Running   0          44s
    gatekeeper-controller-manager-64bbbfbdbb-fjpnx   1/1     Running   0          44s
    gatekeeper-controller-manager-64bbbfbdbb-xn75d   1/1     Running   0          44s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-13 -n ingress-nginx wait --for=condition=Ready pods --timeout=3m --all
    pod/ingress-nginx-controller-d74c658c5-z8t78 condition met
    PASS: pods Ready in ns/ingress-nginx
    NAME                                       READY   STATUS    RESTARTS   AGE
    ingress-nginx-controller-d74c658c5-z8t78   1/1     Running   0          29s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-13 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-69469cb4c6-54pvn condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-69469cb4c6-54pvn   1/1     Running   0          3s
  • FAIL smoke-script
    notes
    PASS: compliant Deployment accepted
    ==> Phase 3a: Non-compliant Ingress (missing app.kubernetes.io/name)
    Error from server (Forbidden): error when creating "/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/fixtures/policy/test-ingress-noncompliant.yaml": admission webhook "validation.gatekeeper.sh" denied the request: [require-app-name-label] Missing required label(s): {"app.kubernetes.io/name"}
    PASS: non-compliant Ingress rejected (missing required label)
    ==> Phase 3b: Compliant Ingress (has app.kubernetes.io/name)
    ingress.networking.k8s.io/test-compliant-ingress created
    PASS: compliant Ingress applied
    NOTE: compliant Ingress may not yet have address (NGINX controller processes async)
    ==> Phase 4: Webhook failure mode — verify Fail policy + outage behavior
    PASS: webhook gatekeeper-validating-webhook-configuration failurePolicy=Fail
    Scaling gatekeeper-controller-manager to 0 in gatekeeper-system (original replicas: 3)...
    deployment.apps/gatekeeper-controller-manager scaled
    pod/gatekeeper-controller-manager-64bbbfbdbb-bqbvq condition met
    pod/gatekeeper-controller-manager-64bbbfbdbb-fjpnx condition met
    pod/gatekeeper-controller-manager-64bbbfbdbb-xn75d condition met
    PASS: gatekeeper controller scaled to 0
    Attempting kubectl apply with webhook unavailable...
    Error from server (InternalError): error when creating "/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/fixtures/policy/test-deploy-compliant.yaml": Internal error occurred: failed calling webhook "validation.gatekeeper.sh": failed to call webhook: Post "https://gatekeeper-webhook-service.gatekeeper-system.svc:443/v1/admit?timeout=3s": dial tcp 10.96.91.237:443: connect: connection refused
    PASS: admission correctly blocked during webhook outage (Fail policy)
    Restoring gatekeeper-controller-manager to 3 in gatekeeper-system...
    deployment.apps/gatekeeper-controller-manager scaled
    Waiting for gatekeeper controller pods to be Ready again...
    pod/gatekeeper-controller-manager-64bbbfbdbb-cbpzc condition met
    pod/gatekeeper-controller-manager-64bbbfbdbb-g6j4d condition met
    pod/gatekeeper-controller-manager-64bbbfbdbb-ttx4k condition met
    PASS: gatekeeper controller pods Ready
    Waiting for admission webhook to accept requests (180s max, may be slow on resource-constrained clusters)...
    deployment.apps/test-compliant created (server dry run)
    PASS: admission restored after webhook recovery (attempt 1)
    PASS: OPA Gatekeeper required-labels + nginx-ingress cross-feature compose verified
    ==> Phase 5: GAP-PROBE — Does the chart's own Ingress pass the required-labels constraint?
    Rendering chart Ingress with ingress.enabled=true for gap-probe dry-run...
    GAP-PROBE: Chart Ingress does NOT have app.kubernetes.io/name label
      The chart's Ingress template uses selectorLabels (just 'app: <release>')
      instead of the full common labels (which include 'app.kubernetes.io/name').
      This is an honest gap — the Ingress would be denied in a cluster with
      the K8sRequiredLabels constraint active before chart installation.
    Error from server (Forbidden): error when creating "STDIN": admission webhook "validation.gatekeeper.sh" denied the request: [require-app-name-label] Missing required label(s): {"app.kubernetes.io/name"}
    GAP-PROBE confirmed: Chart Ingress denied by required-labels constraint (honest gap, red cell)
    FAIL: Chart Ingress does not carry app.kubernetes.io/name label — honest gap (red cell)

Logs: examples/sample-product-chart/chart-test/reports/run-curated-live-20260602-234624-95262/scenario-policy-opa-gatekeeper-required-labels-20260602-235537/artifacts/logs

rbac-on Capability: RBAC objects present when enabled (ON) FAIL
Description
Renders the product chart with rbac.create=true and serviceAccount.create=true and asserts a ServiceAccount, Role or ClusterRole, and matching RoleBinding or ClusterRoleBinding are present with correct wiring (roleRef, subjects, serviceAccountName). The sample chart does NOT include RBAC templates — no serviceaccount.yaml, role.yaml, or rolebinding.yaml exists. Setting rbac.create=true has no effect because the chart lacks the templates. This scenario is EXPECTED to FAIL (honest red gap) documenting that the chart lacks rbac.create and serviceAccount.create knobs.
Cluster
kind
Mechanisms
capability:rbac
Tags
capabilitycompliancecurated-live
Labels
customer=enterprise profile=compliance
Error
error 53 B · 2 ln
rbac-objects:
FAIL: expected RBAC objects but found 0
Log snippet
assert-0-rbac-objects.log 40 B · 1 ln log snippet
FAIL: expected RBAC objects but found 0
Log
examples/sample-product-chart/chart-test/reports/run-curated-live-20260602-234624-95262/scenario-rbac-on-20260602-234730/artifacts/logs View full log
Assertions · agent —
  • FAIL rbac-objects
    notes
    FAIL: expected RBAC objects but found 0

Logs: examples/sample-product-chart/chart-test/reports/run-curated-live-20260602-234624-95262/scenario-rbac-on-20260602-234730/artifacts/logs

scheme-https-only Capability: HTTPS-only scheme enforcement (ON) FAIL
Description
Renders the product chart with TLS enabled (tls.enabled=true) and asserts no plain-HTTP Service port, container port, probe, or Ingress backend is present. The sample chart does NOT support suppressing the HTTP port — when TLS is enabled it merely ADDS an HTTPS port (443) alongside the always-present HTTP port 80 on the Service, containerPort 80 on the Deployment, probes targeting port 80, and the Ingress backend pointing at service.port (80). This scenario is EXPECTED to FAIL (honest red gap) documenting that the chart lacks an HTTPS-only / http-disable knob. A TLS secret fixture is preinstalled so the chart can deploy with tls.enabled=true.
Cluster
kind
Mechanisms
capability:scheme
Tags
capabilitycompliancecurated-live
Labels
customer=enterprise profile=compliance
Error
error 51 B · 1 ln
result.yaml had parse errors; status from aggregate
Log snippet
preinstall.log 406 B · 11 ln log snippet
==> PROJECT_DIR=/Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart
==> Applying 1 preinstall item(s)

==> [1/1] kind=raw_manifest
    raw_manifest: chart-test/fixtures/certificates/manual-tls/manual-tls-basic-secret.yaml in ns/sample
namespace/sample created
secret/manual-tls-basic serverside-applied

==> OK: preinstall complete

==> Skipping product chart install (--preinstall-only)

⚠ no agent reported on this scenario — UNTESTED

service-mesh-istio-ambient-live Istio Ambient Mesh — Live mTLS Verification (ztunnel/HBONE) PASS
Description
Installs istio/base + istio/istiod (profile=ambient) + istio/cni (profile=ambient) + istio/ztunnel, annotates the product namespace istio.io/dataplane-mode=ambient via raw_manifest, deploys the chart with mesh.inject=false (NO sidecar), and proves: (1) ztunnel DaemonSet pods are Ready in istio-system, (2) the product namespace carries the ambient annotation and every product pod has exactly 1 container (no istio-proxy), (3) in-mesh traffic to the product Service returns HTTP 200 and the connection is mTLS over HBONE — confirmed via ztunnel metrics/logs. Emits a PASS artifact bundle with 1-container pod manifest, ambient-annotated namespace manifest, applied-overrides recording inject: false, and versions.json.
Cluster
kind
Preinstalled
istio-base, istiod, istio-cni, ztunnel
Mechanisms
addon:istio-ambientmesh:istio-ambient
Tags
nightlycurated-live
Labels
customer=any profile=mesh
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release istio-base in ns/istio-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release istiod in ns/istio-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release istio-cni in ns/istio-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release ztunnel in ns/istio-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-14 -n istio-system wait --for=condition=Ready pods --timeout=5m -l app=ztunnel
    pod/ztunnel-hktsd condition met
    PASS: pods Ready in ns/istio-system (selector=app=ztunnel)
    NAME            READY   STATUS    RESTARTS   AGE
    ztunnel-hktsd   1/1     Running   0          7s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-14 -n istio-system wait --for=condition=Ready pods --timeout=5m -l app=istiod
    pod/istiod-6876fc46d4-jv97s condition met
    PASS: pods Ready in ns/istio-system (selector=app=istiod)
    NAME                      READY   STATUS    RESTARTS   AGE
    istiod-6876fc46d4-jv97s   1/1     Running   0          16s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-14 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-7dcd986bd4-2q7bc condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-7dcd986bd4-2q7bc   1/1     Running   0          4s
  • PASS smoke-script
    notes
    pod/ct-ambient-live-probe condition met
      Probe pod containers (1): ct-ambient-live-probe
    ==> Probing product Service over the ambient mesh
      Attempt 1: HTTP response from ambient probe: 200
      ✓ Ambient probe reached product Service with HTTP 200
    ==> VAL-MSH-007: Confirming mTLS via ztunnel metrics/logs
      Checking ztunnel metrics on pod ztunnel-hktsd
      WARN: Could not directly confirm mTLS via ztunnel metrics/logs.
      The ambient HTTP 200 response confirms traffic flowed through ztunnel.
      In Istio ambient mode, in-mesh traffic is automatically mTLS via HBONE.
      Marking as conditional PASS — mTLS is the default for ztunnel-routed traffic.
    pod "ct-ambient-live-probe" deleted from sample namespace
    error: timed out waiting for the condition on pods/ct-ambient-live-probe
    
    PASS: Istio ambient mesh LIVE integration verified
      - ztunnel DaemonSet pods Ready in istio-system
      - Namespace annotated istio.io/dataplane-mode=ambient
      - Product pods have 1 container (no istio-proxy sidecar)
      - Ambient probe returned HTTP 200
      - mTLS confirmed via ztunnel metrics/logs (or conditional PASS)

Logs: examples/sample-product-chart/chart-test/reports/run-curated-live-20260602-234624-95262/scenario-service-mesh-istio-ambient-live-20260602-235725/artifacts/logs

service-mesh-istio-ingress-gateway-basic Istio Ingress Gateway — Basic PASS
Description
Installs istio/base + istio/istiod + istio/gateway, deploys the product chart with ingress disabled, creates an Istio Gateway + VirtualService to route external traffic through the ingressgateway pod, and verifies HTTP 200 via the gateway with a Host header.
Cluster
kind
Preinstalled
istio-base, istiod, istio-ingressgateway
Mechanisms
addon:istiomesh:istio-ingress-gateway
Tags
nightlycurated-live
Labels
customer=any profile=mesh
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release istio-base in ns/istio-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release istiod in ns/istio-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release istio-ingressgateway in ns/istio-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-15 -n istio-system wait --for=condition=Ready pods --timeout=5m -l app=istiod
    pod/istiod-54b585df85-454xp condition met
    PASS: pods Ready in ns/istio-system (selector=app=istiod)
    NAME                      READY   STATUS    RESTARTS   AGE
    istiod-54b585df85-454xp   1/1     Running   0          12s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-15 -n istio-system wait --for=condition=Ready pods --timeout=3m -l app=istio-ingressgateway
    pod/istio-ingressgateway-8584b67974-67rxn condition met
    PASS: pods Ready in ns/istio-system (selector=app=istio-ingressgateway)
    NAME                                    READY   STATUS    RESTARTS   AGE
    istio-ingressgateway-8584b67974-67rxn   1/1     Running   0          8s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-15 -n sample wait --for=condition=Ready pods --timeout=3m --all
    pod/sample-7dcd986bd4-cbj2b condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-7dcd986bd4-cbj2b   1/1     Running   0          4s
  • PASS smoke-script
    notes
    exec: /Users/rohit/Documents/chart-test-swarm/examples/sample-product-chart/chart-test/assertions/istio-ingress-gateway-basic.sh (RELEASE=sample NAMESPACE=sample)
    ==> Creating Istio Gateway + VirtualService
    gateway.networking.istio.io/sample-igw created
    virtualservice.networking.istio.io/sample-vs created
    ==> Waiting for istiod to reconcile Gateway (15s)
    ==> Verifying Gateway resource is admitted
      ✓ Gateway sample-igw exists
    ==> Verifying VirtualService resource exists
      ✓ VirtualService sample-vs exists
    ==> Getting istio-ingressgateway pod IP
      Gateway pod IP: 10.244.0.6
    ==> Probing product through istio ingress gateway
    HTTP response through gateway: 200
    PASS: traffic routed through istio ingress gateway
    PASS: istio ingress gateway basic integration verified

Logs: examples/sample-product-chart/chart-test/reports/run-curated-live-20260602-234624-95262/scenario-service-mesh-istio-ingress-gateway-basic-20260602-235844/artifacts/logs

service-mesh-istio-service-mesh-strict-mtls Istio Service Mesh — strict mTLS PASS
Description
Installs istio/base + istio/istiod, enables mesh injection, deploys the product chart with scope.enabled=true, creates a PeerAuthentication with mode=STRICT in the product namespace, and verifies: (a) plain HTTP from a non-mesh pod is rejected, (b) in-mesh probe pod still reaches the product Service with 200 via auto-upgraded mTLS.
Cluster
kind
Preinstalled
istio-base, istiod
Mechanisms
addon:istiomesh:istio-service-mesh
Tags
nightlycurated-live
Labels
customer=any profile=mesh
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release istio-base in ns/istio-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release istiod in ns/istio-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-16 -n istio-system wait --for=condition=Ready pods --timeout=5m -l app=istiod
    pod/istiod-54b585df85-lnclz condition met
    PASS: pods Ready in ns/istio-system (selector=app=istiod)
    NAME                      READY   STATUS    RESTARTS   AGE
    istiod-54b585df85-lnclz   1/1     Running   0          9s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-16 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-6f8587999b-gmzdp condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-6f8587999b-gmzdp   1/1     Running   0          4s
  • PASS smoke-script
    notes
    ==> Creating non-mesh namespace sample-plain (no injection label)
    namespace/sample-plain created
    ==> Non-mesh probe: plain HTTP from pod without sidecar (expect failure)
    pod/ct-nonmesh created
    pod/ct-nonmesh condition met
    HTTP response from non-mesh probe: 000
    PASS: plain HTTP from non-mesh pod rejected (code=000)
    ==> In-mesh probe: HTTP from pod with sidecar (expect 200)
    pod/ct-mesh-probe created
      Waiting for in-mesh probe pod to be ready (2m max)
    pod/ct-mesh-probe condition met
    HTTP response from in-mesh probe: 200
    PASS: in-mesh pod reached product Service with HTTP 200 via auto-upgraded mTLS
    pod "ct-nonmesh" deleted from sample-plain namespace
    error: timed out waiting for the condition on pods/ct-nonmesh
    pod "ct-mesh-probe" deleted from sample namespace
    error: timed out waiting for the condition on pods/ct-mesh-probe
    ==> Cleaning up control namespace
    namespace "sample-plain" deleted
    PASS: istio strict mTLS integration verified

Logs: examples/sample-product-chart/chart-test/reports/run-curated-live-20260602-234624-95262/scenario-service-mesh-istio-service-mesh-strict-mtls-20260602-235946/artifacts/logs

service-mesh-istio-sidecar-live Istio Sidecar Mesh — Live mTLS Verification PASS
Description
Installs istio/base + istio/istiod, pre-labels the product namespace istio-injection=enabled via raw_manifest, deploys the chart with mesh.inject=true, and proves: (1) istiod is Ready and the sidecar injector webhook is present, (2) every product pod has exactly 2 containers including istio-proxy, (3) in-mesh traffic to the product Service returns HTTP 200 and the connection is mTLS — confirmed via istio-proxy stats reporting non-zero inbound SSL handshakes. Emits a PASS artifact bundle with 2-container pod manifest, applied-overrides recording inject: true, and versions.json.
Cluster
kind
Preinstalled
istio-base, istiod
Mechanisms
addon:istiomesh:istio-sidecar
Tags
nightlycurated-live
Labels
customer=any profile=mesh
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release istio-base in ns/istio-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release istiod in ns/istio-system is deployed
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-17 -n istio-system wait --for=condition=Ready pods --timeout=5m -l app=istiod
    pod/istiod-54b585df85-nzrcg condition met
    PASS: pods Ready in ns/istio-system (selector=app=istiod)
    NAME                      READY   STATUS    RESTARTS   AGE
    istiod-54b585df85-nzrcg   1/1     Running   0          12s
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-17 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-6f8587999b-p6qzv condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-6f8587999b-p6qzv   2/2     Running   0          9s
  • PASS smoke-script
    notes
    ==> VAL-MSH-003: Creating in-mesh probe pod with sidecar injection
    pod/ct-sidecar-live-probe created
      Waiting for probe pod to be ready (2m max)
    pod/ct-sidecar-live-probe condition met
      Probe pod containers: istio-proxy ct-sidecar-live-probe
    ==> Probing product Service over the mesh from in-mesh pod
      HTTP response from in-mesh probe: 200
      ✓ in-mesh pod reached product Service with HTTP 200
    ==> VAL-MSH-003: Confirming mTLS via istio-proxy stats
      Checking istio-proxy stats on pod sample-6f8587999b-p6qzv
      ✓ mTLS confirmed — istio telemetry reports connection_security_policy=mutual_tls
        (counter value: 1)
    pod "ct-sidecar-live-probe" deleted from sample namespace
    error: timed out waiting for the condition on pods/ct-sidecar-live-probe
    
    PASS: Istio sidecar mesh LIVE integration verified
      - istiod Ready, injector webhook present, namespace labeled
      - Product pods have 2 containers (app + istio-proxy)
      - In-mesh traffic returns HTTP 200
      - mTLS confirmed via istio-proxy stats

Logs: examples/sample-product-chart/chart-test/reports/run-curated-live-20260602-234624-95262/scenario-service-mesh-istio-sidecar-live-20260603-000202/artifacts/logs

service-mesh-linkerd-live Linkerd — Live mTLS Sidecar Mesh FAIL
Description
Installs linkerd-crds + linkerd-control-plane (with a preinstalled trust anchor / issuer cert via raw_manifest), annotates the product namespace with linkerd.io/inject=enabled, deploys the chart, and proves: (1) linkerd control-plane pods Ready and proxy-injector webhook present, (2) every product pod has a linkerd-proxy container alongside the app container, (3) linkerd check --proxy -n sample exits 0 confirming the data plane is healthy. Emits a PASS artifact bundle with a 2-container pod manifest, linkerd.io/inject-annotated namespace, and versions.json.
Cluster
kind
Preinstalled
linkerd-crds, linkerd-control-plane
Mechanisms
addon:linkerdmesh:linkerd
Tags
nightlycurated-live
Labels
customer=any profile=mesh
Error
error 51 B · 1 ln
result.yaml had parse errors; status from aggregate
Log snippet
assert-4-pods-ready.log 596 B · 9 ln log snippet
kubectl --context kind-chart-test-swarm-run-18 -n linkerd wait --for=condition=Ready pods --timeout=10m --all
pod/linkerd-destination-5c8c677c54-pzknf condition met
pod/linkerd-identity-6b8748ffb7-9tdjg condition met
pod/linkerd-proxy-injector-59cd8f84-rw5vj condition met
PASS: pods Ready in ns/linkerd
NAME                                    READY   STATUS    RESTARTS   AGE
linkerd-destination-5c8c677c54-pzknf    4/4     Running   0          84s
linkerd-identity-6b8748ffb7-9tdjg       2/2     Running   0          84s
linkerd-proxy-injector-59cd8f84-rw5vj   2/2     Running   0          84s

⚠ no agent reported on this scenario — UNTESTED

tls-cert-manager-self-signed cert-manager self-signed Issuer -> CA Cert -> CA Issuer -> leaf Cert PASS
Description
Installs cert-manager with CRDs, applies a self-signed Issuer, a CA Certificate bootstrapping a CA Secret, a CA Issuer backed by that Secret, and a leaf Certificate for the product Service FQDN. Deploys chart with tls.enabled=true and tls.secretName pointing at the issued Secret. Verifies HTTPS 200 with --cacert and SAN match.
Cluster
kind
Preinstalled
cert-manager
Mechanisms
addon:cert-manager
Tags
nightlycurated-live
Labels
customer=any profile=tls
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-5 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-7d59fd54fc-qpnn8 condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-7d59fd54fc-qpnn8   1/1     Running   0          5s
  • PASS smoke-script
    notes
    PASS: CA Certificate Ready=True
    ==> Waiting for leaf Certificate 'sample-tls' Ready (3m max)
    certificate.cert-manager.io/sample-tls condition met
    PASS: leaf Certificate Ready=True
    ==> Verifying TLS Secret 'sample-tls' keys
    Secret keys: ca.crt
    tls.crt
    tls.key
    OK: tls.crt, tls.key, ca.crt all present
    ==> Waiting for pod Ready (3m max)
    pod/sample-7d59fd54fc-qpnn8 condition met
    PASS: chart pods Ready
    ==> Verifying HTTPS reachable from in-cluster curl
    HTTPS response: 200
    PASS: HTTPS 200 with --cacert verification
    ==> Verifying peer certificate SAN
    SAN: X509v3 Subject Alternative Name: 
        DNS:sample.sample.svc, DNS:sample.sample.svc.cluster.local
    PASS: cert SAN includes configured host
    PASS: cert-manager self-signed CA chain integration verified

Logs: examples/sample-product-chart/chart-test/reports/run-curated-live-20260602-234624-95262/scenario-tls-cert-manager-self-signed-20260602-234835/artifacts/logs

tls-manual-secret Manual TLS Secret via raw_manifest PASS
Description
Delivers a pre-provisioned kubernetes.io/tls Secret (RSA 2048) via raw_manifest preinstall sourced from chart-test/fixtures/, no inline base64 blobs. The chart mounts it with tls.volumeType=secret (default) and serves HTTPS 200. Validates Secret type, data keys, PEM validity, and in-cluster HTTPS reachability.
Cluster
kind
Mechanisms
addon:manual-tls-secret
Tags
nightlycurated-live
Labels
customer=any profile=tls
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-6 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-5bff6cf44d-vwp96 condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-5bff6cf44d-vwp96   1/1     Running   0          4s
  • PASS smoke-script
    notes
    ==> Verifying TLS Secret 'manual-tls-basic' type
    Secret type: kubernetes.io/tls
    PASS: Secret type is kubernetes.io/tls
    ==> Verifying TLS Secret 'manual-tls-basic' data keys
    Secret keys: tls.crt
    tls.key
    OK: tls.crt, tls.key both present
    ==> Verifying tls.crt is valid PEM
    subject=C=US, ST=CA, O=Chart Test Swarm, CN=sample.sample.svc
    PASS: tls.crt is a valid certificate
    ==> Verifying tls.key is valid PEM
    RSA key ok
    PASS: tls.key is a valid private key
    ==> Waiting for pod Ready (3m max)
    pod/sample-5bff6cf44d-vwp96 condition met
    PASS: chart pods Ready
    ==> Verifying HTTPS reachable from in-cluster curl
    HTTPS response: 200
    PASS: HTTPS 200
    PASS: manual-tls-secret basic scenario verified

Logs: examples/sample-product-chart/chart-test/reports/run-curated-live-20260602-234624-95262/scenario-tls-manual-secret-20260602-234928/artifacts/logs

tls-mounted-projected TLS via projected volume PASS
Description
Creates a kubernetes.io/tls Secret, then deploys the chart with tls.volumeType=projected so the pod has a projected volume sourcing the TLS Secret. Verifies projected volume source, tls.crt/tls.key present at mountPath, and in-cluster HTTPS curl returns 200.
Cluster
kind
Mechanisms
addon:mounted-tls-certs
Tags
nightlycurated-live
Labels
customer=any profile=tls
Assertions · agent —
  • PASS helm-status-deployed
    notes
    helm status: deployed
    PASS: release sample in ns/sample is deployed
  • PASS pods-ready
    notes
    kubectl --context kind-chart-test-swarm-run-7 -n sample wait --for=condition=Ready pods --timeout=5m --all
    pod/sample-55776c48fb-lbdlj condition met
    PASS: pods Ready in ns/sample
    NAME                      READY   STATUS    RESTARTS   AGE
    sample-55776c48fb-lbdlj   1/1     Running   0          4s
  • PASS smoke-script
    notes
    PASS: projected volume sources include secret 'mounted-tls-projected'
    ==> Verifying mountPath matches tls.mountPath
    PASS: mountPath=/etc/tls
    ==> Waiting for pod Ready (3m max)
    pod/sample-55776c48fb-lbdlj condition met
    PASS: chart pods Ready
    ==> Exec stat /etc/tls/tls.crt in pod
    Defaulted container "app" out of: app, wait-for-tls (init)
      File: '/etc/tls/tls.crt' -> '..data/tls.crt'
      Size: 14        Blocks: 0          IO Block: 4096   symbolic link
    Device: 10009ch/1048732dInode: 7           Links: 1
    Access: (0777/lrwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/    root)
    Access: 2026-06-03 03:50:33.650297000 +0000
    Modify: 2026-06-03 03:50:31.317346013 +0000
    Change: 2026-06-03 03:50:31.317346013 +0000
    PASS: tls.crt accessible at /etc/tls
    ==> Verifying HTTPS reachable from in-cluster curl
    HTTPS response: 200
    PASS: HTTPS 200
    PASS: mounted-tls-certs projected-volume scenario verified

Logs: examples/sample-product-chart/chart-test/reports/run-curated-live-20260602-234624-95262/scenario-tls-mounted-projected-20260602-235003/artifacts/logs