Documentation menu

Wildcard TLS certificates

Serve browser-trusted HTTPS from your klimax clusters — a real Let's Encrypt certificate for *.lab.example.dev, valid on private 172.30.x.y addresses, with no custom CA to install and no certificate warnings to click through.

This is the companion to Hostnames with ExternalDNS. Read that first — it sets up the DNS zone this guide issues a certificate for.

Why this is possible at all

A certificate authority will happily issue for a name that resolves to a private address. What it will not do is reach that address to verify you control the name. So the usual HTTP-01 challenge, where the CA connects to your service on port 80, cannot work: Let's Encrypt has no route to 172.30.1.7.

DNS-01 sidesteps this entirely. Instead of proving control by serving a file, you prove it by writing a TXT record into the zone. The CA queries public DNS; nothing connects to your Mac. Since you already delegated a zone and hold API credentials for it, you have everything the challenge needs.

Self-signed / mkcertLet's Encrypt via DNS-01
Browser trustafter installing a CA in every trust storeimmediate, everywhere
Non-browser clients (Go, JVM, curl)only after per-runtime trust configout of the box
Works on a colleague's machinenoyes
Wildcard across all lab servicespossibleyes, one certificate
Renewalmanualscriptable or automated
🔑

The second row matters more than it looks. A JVM or Go client rejecting your self-signed CA is a classic afternoon lost to trust-store debugging — and it shows up exactly when you are demoing OIDC, mTLS or webhooks.

Prerequisites

  • A delegated DNS zone and cloud credentials — see ExternalDNS, Steps 1–2.
  • A running klimax cluster.
  • lego (brew install lego) for the one-shot path, or helm for the automated path.
$ export GCP_PROJECT=my-project
$ export DNS_ZONE_NAME=lab.example.dev
$ export DNS_ZONE_LITERAL=lab-example-dev
$ export CREDS_FILE="$HOME/.klimax/external-dns-credentials.json"

You can reuse the ExternalDNS service account — roles/dns.admin already covers writing the _acme-challenge TXT records.

Option A — one-shot certificate with lego

Best when you want a certificate now and do not need in-cluster automation.

$ GCE_PROJECT="${GCP_PROJECT}" \
  GCE_SERVICE_ACCOUNT_FILE="${CREDS_FILE}" \
  lego --email you@example.com \
       --dns gcloud \
       -d "*.${DNS_ZONE_NAME}" \
       -a run

lego writes the TXT record, waits for propagation, completes the challenge and cleans up after itself. The certificate and key land in .lego/certificates/.

📐

Request -d "*.${DNS_ZONE_NAME}" and -d "${DNS_ZONE_NAME}" if you want the apex covered too. A wildcard matches one label only: *.lab.example.dev covers keycloak.lab.example.dev but neither lab.example.dev itself nor a.b.lab.example.dev.

Load it into the cluster as a TLS secret — one certificate is valid in all of them:

$ for CTX in dev staging prod; do
    kubectl --context "$CTX" -n kong create secret tls "wildcard-${DNS_ZONE_LITERAL}" \
      --cert=".lego/certificates/_.${DNS_ZONE_NAME}.crt" \
      --key=".lego/certificates/_.${DNS_ZONE_NAME}.key"
  done

Renewing

Let's Encrypt certificates last 90 days. Re-run with renew and refresh the secrets:

$ GCE_PROJECT="${GCP_PROJECT}" GCE_SERVICE_ACCOUNT_FILE="${CREDS_FILE}" \
  lego --email you@example.com --dns gcloud -d "*.${DNS_ZONE_NAME}" renew

$ kubectl --context "${CTX}" -n kong create secret tls "wildcard-${DNS_ZONE_LITERAL}" \
    --cert=".lego/certificates/_.${DNS_ZONE_NAME}.crt" \
    --key=".lego/certificates/_.${DNS_ZONE_NAME}.key" \
    --dry-run=client -o yaml | kubectl --context "${CTX}" apply -f -

For labs that outlive a certificate, Option B removes this chore.

Option B — automated renewal with cert-manager

Best for long-lived clusters, or when you would rather not think about expiry.

$ helm repo add jetstack https://charts.jetstack.io
$ helm repo update
$ helm --kube-context "${CTX}" upgrade -i cert-manager jetstack/cert-manager \
    -n cert-manager --create-namespace --set crds.enabled=true

$ kubectl --context "${CTX}" -n cert-manager create secret generic clouddns-credentials \
    --from-file=credentials.json="${CREDS_FILE}"

Create a ClusterIssuer using the DNS-01 solver:

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-dns
spec:
  acme:
    email: you@example.com
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-dns-account-key
    solvers:
      - dns01:
          cloudDNS:
            project: my-project
            serviceAccountSecretRef:
              name: clouddns-credentials
              key: credentials.json

Then request the wildcard, and cert-manager keeps it renewed:

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: wildcard-lab
  namespace: kong
spec:
  secretName: wildcard-lab-example-dev
  issuerRef:
    name: letsencrypt-dns
    kind: ClusterIssuer
  dnsNames:
    - "*.lab.example.dev"
    - "lab.example.dev"
$ kubectl --context "${CTX}" -n kong get certificate wildcard-lab -w

Use the staging ACME endpoint while iterating. Let's Encrypt enforces rate limits — notably 5 duplicate certificates per week — and a misconfigured solver burns through them fast. Swap server: for https://acme-staging-v02.api.letsencrypt.org/directory until issuance succeeds, then switch back. Staging certificates are untrusted by design; that is expected.

Using the certificate

Reference the secret from whatever terminates TLS. For a Kubernetes Ingress:

spec:
  tls:
    - hosts:
        - "keycloak.lab.example.dev"
      secretName: wildcard-lab-example-dev

For a Kong Gateway data plane, mount it as a secret volume and point the listener at it:

# values-gw-dp.yaml
proxy:
  tls:
    enabled: true
env:
  ssl_cert: /etc/secrets/wildcard/tls.crt
  ssl_cert_key: /etc/secrets/wildcard/tls.key
secretVolumes:
  - wildcard-lab-example-dev

Then, from your Mac, with no trust-store changes at all:

$ curl -sS https://keycloak.lab.example.dev/

Troubleshooting

SymptomLikely cause
Challenge times outTXT record not visible publicly. Check dig +short TXT _acme-challenge.lab.example.dev; usually incomplete NS delegation
rateLimited from ACMEToo many issuance attempts — switch to the staging endpoint and wait out the window
Issued, but the browser still warnsThe proxy is still serving the old certificate — restart or reload it so it re-reads the secret
Works in curl, fails in a JVM or Go clientAlmost always a leftover self-signed certificate still being served, not a trust problem. Verify with openssl s_client -connect keycloak.lab.example.dev:443
cert-manager stuck Pendingkubectl describe challenge -A shows the solver error; usually the credentials secret or project: is wrong
Wildcard rejected for a.b.lab.example.devA wildcard covers one label — request a deeper wildcard or a dedicated name