Talos 1.14 config documents, and why the node version decides which ones you may send
From 1.14 Talos moves kubelet, labels, taints and the service range out of the v1alpha1 config into typed documents, and rejects the old fields next to them. A node on an older version rejects the new documents. What that does to patches, fixtures and upgrade order.
I render machine configs with a newer talosctl than some of my nodes run, and
that gap bit me three different ways before I understood the rule: the node’s
version decides which config shape it accepts.
Typed documents replace fields
Since 1.14, talosctl gen config renders extra documents next to the old
v1alpha1 one: KubeNodeConfig, KubeletConfig, KubeNetworkConfig,
DiscoveryServiceConfig and others. The settings that moved there are rejected
if you also set them the old way:
machine.kubeletnext to aKubeletConfigcluster.allowSchedulingOnControlPlanesmachine.nodeLabelsandmachine.nodeTaints
The error is “already set in v1alpha1 config”. talosctl validate shows it
without touching a node, which is how I found that my own test fixture was invalid.
The replacements, in the cases I needed:
apiVersion: v1alpha1
kind: KubeNodeConfig
taints:
node-role.kubernetes.io/control-plane:
$patch: delete # lets workloads run on control planes
labels:
node.kubernetes.io/exclude-from-external-load-balancers:
$patch: deleteKubelet server certificate bootstrapping moved into KubeletConfig.config.
Older nodes reject newer documents
The same rule works in reverse. A node booted from an older ISO rejects documents
a newer talosctl renders. It also gets the newer client’s default Kubernetes
version, which the older node may not support at all. Boot an ISO of the version
you mean to run, or pass --talos-version and --kubernetes-version explicitly
when rendering.
What it means for upgrades
Upgrade the nodes first while your config is still rendered for the old version, then raise the version and apply the new config. Applying the new shape first fails on every node that hasn’t been upgraded yet.
If one tool serves both an old and a new cluster, as mine does, keep the “config
version” a separate setting from the client version. A patch that works on one
cluster, such as deleting a label through machine.nodeLabels, has to be written
the other way on the next.