Skip to content

Route k8s traffic through Gateway API (Traefik) - #2758

Draft
hahn-kev-bot wants to merge 5 commits into
developfrom
claude/k8s-ingress-gateway-migration-d97e14
Draft

hahn-kev-bot wants to merge 5 commits into
developfrom
claude/k8s-ingress-gateway-migration-d97e14

Conversation

@hahn-kev-bot

@hahn-kev-bot hahn-kev-bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

prepping to migrate from ingress to gateway


AI summary

This is the first of two PRs moving routing off ingress-nginx and onto Gateway API. It adds HTTPRoutes next to the existing Ingress, and switches local-dev and CI to Traefik.

Routes

  • base/http-routes.yaml adds lexbox-web and lexbox-hosts:
    • lexbox-web sends /api, /hg, /security.txt and /.well-known to lexbox, /v1/traces to the otel port, and everything else to ui.
    • lexbox-hosts sends the hg, resumable and admin hosts to lexbox.
  • Each environment sets its hostnames as a list in http-routes.patch.yaml. That replaces patching Ingress rules by position and the duplicated rule blocks in prod and develop. The route hostnames match the current Ingress hosts exactly in every environment.
  • In the clusters, parentRefs point at a placeholder for the LTOps shared Gateway (ltops-shared-gateway / ltops-gateway, marked TODO(LTOps)). Until we have the real name, the routes attach to nothing and the Ingress keeps serving traffic.

Old Ingress

  • The Ingress moved unchanged to a legacy-ingress/ kustomize component, which develop, staging and prod include. Their rendered Ingress is identical to develop.
  • A follow-up PR deletes that component and the ingress-config*.yaml patches once each cluster has been cut over.

local-dev / CI

  • The vendored ingress-nginx v1.8.0 is replaced by Traefik v3.7.14 (chart 41.7.1), rendered from local-dev/traefik/values.yaml. Re-render it with task k8s:render-traefik.
  • Traefik settings:
    • Gateway provider only.
    • readTimeout: 0, so long hg pushes aren't cut off at Traefik's 60s default.
    • Trusts forwarded headers, which replaces nginx's use-forwarded-headers.
  • local-dev/gateway.yaml defines a GatewayClass and a Gateway with HTTP and HTTPS listeners and no redirect. The HTTPS listener uses an explicit self-signed Certificate instead of the Ingress annotation.
  • The local-dev patch that deleted the OAuth certificates now only matches oauth-*; before, it deleted every Certificate.
  • Gateway API CRDs (v1.6.2) are now a cluster prerequisite, installed by task setup-k8s and the setup-k8s action. Existing local clusters need to re-run task setup-k8s once.
  • In CI, the setup-k8s input ingress-controller-port is renamed to gateway-port. Port-forwards and log collection now use service/traefik and the app.kubernetes.io/name=traefik label.

Before the clusters can switch (needed from LTOps)

  • The shared Gateway's name, namespace and sectionName, with allowedRoutes permitting languagedepot.
  • How TLS is handled: LTOps' own certs, or ours. If ours, we add a ReferenceGrant.
  • Traefik entrypoint readTimeout: 0 (or a high value), and forwardedHeaders.trustedIPs set to the Cloudflare ranges.
  • A plan for the DNS/LB cutover for each environment.

Test plan

  • kubectl kustomize renders local-dev, gha, develop, staging and production.
  • The develop, staging and prod renders match develop except for the two added HTTPRoutes, and the Ingress is unchanged.
  • kubeconform -strict passes on all 153 rendered resources.
  • GHA integration tests pass on kind with Traefik (setup-k8s).
  • fw-lite e2e passes, including the MSAL HTTPS flow through the service/traefik 443 port-forward.
  • tilt up on Docker Desktop, after re-running task setup-k8s: https://localhost works, and hg clone and push work.

🤖 Generated with Claude Code

Add HTTPRoutes for the main site and the hg/resumable/admin hosts, with
per-environment hostname patches. develop/staging/prod keep the old
ingress-nginx Ingress (moved to the legacy-ingress component) until each
cluster is cut over to the LTOps shared Gateway.

local-dev and CI now run Traefik (vendored helm render) with a local
Gateway instead of the vendored ingress-nginx. Gateway API CRDs are
installed as a cluster prerequisite in `task setup-k8s` and the
setup-k8s action.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added 💻 FW Lite issues related to the fw lite application, not miniLcm or crdt related 📦 Lexbox issues related to any server side code, fw-headless included labels Oct 15, 2026
@argos-ci

argos-ci Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Argos notifications ↗︎

Build Status Details Updated (UTC)
default (Inspect) ✅ No changes detected - Oct 16, 2026, 8:14 AM
e2e (Inspect) ✅ No changes detected - Oct 16, 2026, 8:22 AM

hahn-kev and others added 2 commits October 16, 2026 11:36
kind-action's default node image is k8s 1.29, which rejects the Gateway
API v1.6 CRDs (CEL isIP(), ValidatingAdmissionPolicy v1). Pin kind
v0.33.0 with a k8s 1.34 node image and matching kubectl.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Its default kind is v0.33.0, so drop the explicit version input. Keep the
k8s 1.34 node image and kubectl pins.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hahn-kev and others added 2 commits October 16, 2026 12:57
kubectl port-forward exits when a forwarded stream is reset. The 120MB
upload in MediaFileTests.UploadReplacementFile_TooLarge_ThrowsError gets
a 413, the connection is reset, and every later integration test failed
with connection refused until the job timed out.

kind now maps localhost:6579/6580 to fixed traefik NodePorts, so kubectl
is no longer in the request path. The setup-k8s gateway-port input is
gone since the ports are fixed by deployment/gha/kind.yaml.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FwHeadless rejects uploads over Kestrel's 30 MB body limit before reading
them. ingress-nginx buffered the whole upload, so clients always got the
413. Traefik streams the body, so lexbox finished the request mid-upload
and the client saw a connection reset instead
(UploadReplacementFile_TooLarge_ThrowsError).

When the proxied response is a 413, read and discard the rest of the
upload, bounded to 256 MiB and 60s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

💻 FW Lite issues related to the fw lite application, not miniLcm or crdt related 📦 Lexbox issues related to any server side code, fw-headless included

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants