I couldn't run Talos in CI, so I tested what I could

I wanted real-cluster tests for an Ansible collection that drives Talos. On my machine neither Docker nor QEMU would give me a node in maintenance mode, so I tested the rendering instead: fixtures, talosctl validate, determinism and negative cases. What it caught, and the five CI runs it took to go green.

Checked against Molecule 26, Talos 1.13 and 1.14.

The collection I’m writing configures Talos nodes: it renders a machine config, asks each node what would change, applies, bootstraps etcd once. The obvious test is a real cluster that gets built, changed and rebuilt in a loop. I wanted that first, so I started by trying to get one on my workstation.

What I tried

Docker works here, and talosctl cluster create docker starts a cluster. It always injects a config, though, so there is never a node in maintenance mode, and maintenance mode is exactly the state the role has to handle. The one node I let run stalled at “Waiting for api certificates, config to be ready” and was still there four minutes later. QEMU clusters would need root networking, and /dev/kvm being present on this machine didn’t change that.

I dropped the cluster scenario instead of committing one I couldn’t make pass. A test I never saw go green isn’t evidence of anything.

What I built instead

The part of the role that can be wrong without any node is the config it sends. So the render scenario builds a fixture inventory for a Talos 1.14 and a 1.13 cluster, makes a throwaway PKI with talosctl gen secrets, renders every node’s config and runs talosctl validate on each. Then it checks content and determinism (render twice, same bytes) and a handful of negative cases: bad input has to fail, with the right message. A second scenario covers the tool installer.

I mutation-tested one of the negative assertions by breaking the expected text on purpose. The scenario went red, so it really was testing something, and I put the text back.

The fixtures earned their keep immediately. Two roles that load the cluster in one play (cluster, bootstrap, upgrade) were appending to the node list instead of starting from an empty one, so every node appeared twice on the second load. Writing the fixture exposed it. The verify step now loads twice and checks the count.

And talosctl validate rejected my own fixture: on 1.14 a machine.kubelet block next to a typed KubeletConfig is invalid. That’s the same rule that bit me with node labels, found without touching a node.

Molecule oddities

In Molecule 26 scenarios live in extensions/molecule/<name>. molecule init scenario may create a stray molecule/ directory, and that one takes precedence and hides yours, which looks like “my scenario doesn’t exist”. With the default driver and options.managed: false you need no create or destroy playbooks; inventory and group vars go under provisioner.inventory.

Five runs to get CI green

The pipeline needed five attempts, and each was a different thing about the runner:

• ubuntu-latest maps to a node 16 image on my runner, so the workflow pins a container image. The runner notes cover that.
• actions/checkout needs Node, which a Python image doesn’t have, so the job clones with git and the job token. And inside a job GITHUB_SERVER_URL is the in-cluster address over plain http, so the clone URL has to be built from its scheme.
• ansible-core doesn’t bundle kubernetes.core; a requirements.yml installs it.
• ansible-lint finds the project’s own collection through a link in a git-ignored .cache directory, which a fresh checkout never creates when you run it offline. CI runs it online.

One I couldn’t solve: from the runner’s network a connection to the Helm download host hangs, while GitHub and the Kubernetes download host answer, and the same host works from my LAN. DNS and Pi-hole aren’t the cause. I found no matching firewall rule. The installer now has a 60 second timeout and three tries, and CI skips Helm.

Where it stands

Lint and both scenarios are green. At that point nothing in CI ran against a real node, so cluster behaviour was still checked by hand. I’ve since added scenarios against a throwaway cluster on Proxmox, which CI runs too. What the render approach covers is everything up to the moment bytes go to a node, and it caught a real bug in the role before any node saw it.