Keycloak OIDC Zero‑Trust Guide (Envoy Gateway + cert-manager + .NET)¶

TODO (next phase) - 🔒 Add Envoy JWT validation (Gateway-level token verification)

This page documents the setup we built: - Keycloak on Kubernetes (official image via KeycloakX Helm chart) - Exposed to the internet via Envoy Gateway (Gateway API) - HTTPS termination with cert-manager - A realm + client for zero-trust access to your API - .NET JWT validation (issuer, signature, lifetime, audience)

0) Prerequisites¶

You should already have: - A working Envoy Gateway Gateway (public) with a public IP - cert-manager installed and a ClusterIssuer (e.g., letsencrypt) - DNS A-record for keycloak.hershkowitz.co.il pointing to the gateway public IP - Kubernetes access via kubectl and Helm installed

Quick checks:

kubectl get gateway -A
kubectl get svc -A | grep -i envoy
kubectl get certificate -A

1) Install Keycloak (NO Bitnami)¶

We intentionally avoid Bitnami images/charts.

1.1 Create namespace¶

kubectl create namespace keycloak

1.2 Add KeycloakX Helm repo and install¶

helm repo add codecentric https://codecentric.github.io/helm-charts
helm repo update

Install using your values file:

helm install keycloak codecentric/keycloakx -n keycloak -f k8s/keycloak/values.yaml

Upgrade later with:

helm upgrade keycloak codecentric/keycloakx -n keycloak -f k8s/keycloak/values.yaml

1.3 Minimal `values.yaml` (what matters)¶

Key points: - Keycloak runs HTTP inside the cluster - Envoy Gateway terminates HTTPS - We keep Keycloak under /auth

Example:

image:
  repository: quay.io/keycloak/keycloak
  tag: 26.4.5

extraEnv: |
  - name: KC_HTTP_ENABLED
    value: "true"
  - name: KC_PROXY
    value: "edge"
  - name: KC_HOSTNAME
    value: "keycloak.hershkowitz.co.il"
  - name: KC_HTTP_RELATIVE_PATH
    value: "/auth"

  # Database (existing Postgres in infra namespace)
  - name: KC_DB
    value: postgres
  - name: KC_DB_URL_HOST
    value: pg-main-postgresql.infra.svc.cluster.local
  - name: KC_DB_URL_PORT
    value: "5432"
  - name: KC_DB_URL_DATABASE
    value: keycloak

extraEnvFrom: |
  - secretRef:
      name: keycloak-secrets

1.4 The Keycloak secrets¶

We used a secret like keycloak-secrets in the keycloak namespace:

kubectl -n keycloak get secrets
kubectl -n keycloak describe secret keycloak-secrets

Expected keys: - KEYCLOAK_ADMIN - KEYCLOAK_ADMIN_PASSWORD - KC_DB_USERNAME - KC_DB_PASSWORD

✅ KC_DB_USERNAME / KC_DB_PASSWORD must match a Postgres role that can access the Keycloak database.

2) Verify Keycloak service inside the cluster¶

List services:

kubectl -n keycloak get svc

Typical service name for KeycloakX: - keycloak-keycloakx-http

Test from inside cluster:

kubectl -n keycloak run tmp-curl --rm -it --image=curlimages/curl --restart=Never --   curl -I http://keycloak-keycloakx-http.keycloak.svc.cluster.local/auth/

Expected: a 302 redirect (Keycloak sends /auth/ → /auth/admin/), meaning it’s alive.

3) Expose Keycloak publicly with Envoy Gateway (HTTPS + redirect)¶

We use Gateway API HTTPRoute resources.

3.1 HTTPS route to Keycloak (forward traffic)¶

Create/maintain: k8s/keycloak/keycloak-httproute.yaml

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: keycloak
  namespace: keycloak
spec:
  parentRefs:
    - name: public-gw
      namespace: default
  hostnames:
    - keycloak.hershkowitz.co.il
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /
      backendRefs:
        - name: keycloak-keycloakx-http
          port: 80

Apply + check status:

kubectl apply -f k8s/keycloak/keycloak-httproute.yaml
kubectl -n keycloak get httproute keycloak -o yaml | sed -n '/status:/,$p'

You want: - Accepted=True - ResolvedRefs=True

3.2 HTTP → HTTPS redirect (same file, second object)¶

Add this second object in the same file (so you still have one file):

---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: keycloak-http-redirect
  namespace: keycloak
spec:
  parentRefs:
    - name: public-gw
      namespace: default
      sectionName: http        # IMPORTANT: attach only to port 80 listener
  hostnames:
    - keycloak.hershkowitz.co.il
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /
      filters:
        - type: RequestRedirect
          requestRedirect:
            scheme: https
            statusCode: 301

Apply and verify redirect:

kubectl apply -f k8s/keycloak/keycloak-httproute.yaml
curl -I http://keycloak.hershkowitz.co.il/auth/ | egrep -i 'http/|location:'

4) TLS with cert-manager¶

You have one SAN certificate secret used by Envoy:

kubectl -n default get certificate hershkowitz-co-il -o yaml

The cert SAN list must include keycloak.hershkowitz.co.il.

Example:

spec:
  secretName: hershkowitz-co-il-tls
  dnsNames:
    - hershkowitz.co.il
    - www.hershkowitz.co.il
    - search.hershkowitz.co.il
    - seq.hershkowitz.co.il
    - grafana.hershkowitz.co.il
    - keycloak.hershkowitz.co.il

If you add a new DNS name but the domain doesn’t resolve publicly yet, Let’s Encrypt will fail. Fix DNS first.

5) Ensure Keycloak generates HTTPS URLs (avoid Mixed Content)¶

We validated the OIDC discovery document is HTTPS:

curl -s https://keycloak.hershkowitz.co.il/auth/realms/hershkowitz/.well-known/openid-configuration | grep '"issuer"'

Expected:

"issuer":"https://keycloak.hershkowitz.co.il/auth/realms/hershkowitz"

If you ever see HTTP here, Keycloak will produce mixed content in the admin console.

6) Configure Keycloak for Zero Trust¶

6.1 Create realm¶

In Keycloak Admin UI: - Create realm: hershkowitz

6.2 Create confidential client: `search-api`¶

In realm hershkowitz: - Clients → Create client - Client ID: search-api - Client authentication: ON (confidential) - Service accounts enabled: ON (required for client_credentials) - Save - Credentials tab → copy client secret

6.3 Add Audience mapper (critical)¶

We require tokens to be intended for the API (audience = search-api).

In Keycloak: - Clients → search-api → Mappers (or Client scopes → Mappers) - Add Mapper: - Type: Audience - Name: audience-search-api - Included Client Audience: search-api - Add to access token: ✅ - Save

7) Get a token (client_credentials)¶

curl -sS -X POST "https://keycloak.hershkowitz.co.il/auth/realms/hershkowitz/protocol/openid-connect/token"   -H "Content-Type: application/x-www-form-urlencoded"   -d "grant_type=client_credentials"   -d "client_id=search-api"   -d "client_secret=YOUR_SECRET_HERE" | head -c 400 && echo

If you get: - unauthorized_client / “Client not enabled to retrieve service account”
Enable Service accounts on the client.

8) Configure the Search API (.NET) to validate JWT (Zero Trust)¶

8.1 Add NuGet package¶

dotnet add package Microsoft.AspNetCore.Authentication.JwtBearer

8.2 Program.cs (full working setup)¶

This matches what we want: issuer + signature + lifetime + audience.

using Microsoft.AspNetCore.Authentication.JwtBearer;
using Microsoft.IdentityModel.Tokens;

var authority = "https://keycloak.hershkowitz.co.il/auth/realms/hershkowitz";

builder.Services
    .AddAuthentication(JwtBearerDefaults.AuthenticationScheme)
    .AddJwtBearer(options =>
    {
        options.Authority = authority;

        // We are running Keycloak on HTTPS publicly
        options.RequireHttpsMetadata = true;

        options.TokenValidationParameters = new TokenValidationParameters
        {
            // Issuer must match exactly
            ValidateIssuer = true,
            ValidIssuer = authority,

            // Zero-trust: token must be intended for THIS API
            ValidateAudience = true,
            ValidAudience = "search-api",

            // Expired tokens must be rejected
            ValidateLifetime = true
        };
    });

builder.Services.AddAuthorization();

Middleware order (important):

app.UseAuthentication();
app.UseAuthorization();

8.3 Protect endpoints¶

Controllers:

using Microsoft.AspNetCore.Authorization;

[Authorize]
[ApiController]
public class SearchController : ControllerBase
{
    [HttpGet("/search")]
    public IActionResult Search() => Ok("authenticated");
}

Minimal APIs:

app.MapGet("/search", () => "authenticated").RequireAuthorization();

Health endpoint public:

app.MapGet("/health", () => "ok").AllowAnonymous();

9) Verify “Zero Trust” behavior¶

Expected results:

Request	Expected
No Authorization header	`401 Unauthorized`
Invalid token	`401 Unauthorized`
Expired token	`401 Unauthorized`
Token with wrong audience	`401 Unauthorized`
Valid token	`200 OK`

10) TODO (Next Phase): Envoy JWT validation¶

Right now the API validates JWTs.

Next phase is to enforce JWT validation at Envoy Gateway: - Requests without valid JWT never reach your services - Gateway becomes a strict policy enforcement point

TODO - 🔒 Add Envoy JWT validation (Gateway-level token verification)