Putting Traefik in front of a host that is not in the cluster

A Service without a selector plus a hand-written EndpointSlice points Traefik at any address, and the existing wildcard certificate and DNS keep working. The risky part is the cutover order, because Flux prunes less than you think.

I moved a web dashboard off the cluster and onto a Raspberry Pi, but I wanted its address to stay the same: same hostname, same wildcard certificate, same Traefik in front. Traefik can do that without help from the app. It only needs a Service that points at the Pi.

A Service with no selector

A Service normally finds its pods through a label selector. Leave the selector out and Kubernetes creates no endpoints at all, so you write them yourself:

apiVersion: v1
kind: Service
metadata:
  name: dashboard
spec:
  ports:
    - name: http
      port: 9119
      targetPort: 9119
---
apiVersion: discovery.k8s.io/v1
kind: EndpointSlice
metadata:
  name: dashboard-1
  labels:
    kubernetes.io/service-name: dashboard
addressType: IPv4
ports:
  - name: http
    port: 9119
    protocol: TCP
endpoints:
  - addresses: ["<address of the Pi>"]

The label kubernetes.io/service-name is what ties the slice to the Service, and the port name has to match. The IngressRoute is then the same one you’d write for a pod, and tls: {} picks up the wildcard certificate from Traefik’s default TLS store:

apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: dashboard
spec:
  entryPoints: [websecure]
  routes:
    - match: Host(`dashboard.example.org`)
      kind: Rule
      services:
        - name: dashboard
          port: 9119
  tls: {}

DNS already points at Traefik’s address, so nothing changes there. The Pi has to accept connections from the cluster nodes, and since Traefik is now a proxy in front of the app, the app needs to know about it. Mine refused to listen beyond loopback without an authentication provider, and wanted the proxy’s address listed as trusted before it would believe the forwarded headers.

The order of the cutover

I had written the plan as one pull request that removed the old Deployment and added the new Service. The review caught what that would do: the pod would go away before I had copied its data out, and Flux would prune the volume claim along with it.

The order that worked: stop the old pod without deleting anything (a Flux skip annotation and scale to 0), copy the data off the volume with a throwaway pod that mounts it, check the checksums, and only then merge the change that swaps the backend. The volume claim and its secret stayed in that first change and went in a second, later one. Don’t copy a live SQLite directory out of a running pod; stopping it first is the only way to get a consistent copy.

The annotation also stops pruning

kustomize.toolkit.fluxcd.io/reconcile=disabled on the Deployment did what I wanted during the cutover. What I forgot is that it also exempts the object from pruning. After the change merged, the old Deployment was gone from Git and still in the cluster, and I deleted it by hand.