ansible-galaxy install can override the collections you pinned

The ansible pip package already ships its collections. Installing them again into the project directory puts newer copies first on the collections path, silently replacing the versions you froze.

The ansible package on PyPI contains more than ansible-core. It bundles a curated set of collections (kubernetes.core, community.general, ansible.posix and many more) at fixed versions. If you pin ansible==<version> in a requirements file, you’ve pinned those collections as well.

I hadn’t thought that through, and it means a very common repository layout is quietly wrong.

How the pins get replaced

Plenty of projects carry a collections/requirements.yml, and on a new machine the reflex is to run:

ansible-galaxy collection install -r collections/requirements.yml -p ./collections

If ansible.cfg puts ./collections on the collections path (which is the reason the directory exists), it gets searched before the collections bundled in site-packages. ansible-galaxy installs whatever satisfies the requirement, and without an exact pin that’s the newest release. From then on the project runs a different collection version from the one the pip pin implies, and nothing tells you.

For me, kubernetes.core 6.6.0 landed in ./collections on top of the 6.3.0 that the pinned ansible release shipped. Nothing broke that day, which is what makes this version of the bug dangerous. It sits there until some behaviour changes between minor releases.

To see which copy actually wins:

ansible-galaxy collection list kubernetes.core

That lists every installed copy by path, in search order.

Decide which file is the source

In that repository, requirements.yml had never been meant as an install source. It was a record, generated by a script that froze the environment, and treating that record as an input is how the shadowing started. You have to pick one of two setups:

  • Pip is the source. Pin ansible==<version>, don’t install collections separately, and if you keep a collections file, treat it as documentation only and say so at the top of it.
  • Galaxy is the source. Pin exact versions (version: "==6.3.0") in requirements.yml, and install ansible-core instead of ansible so there isn’t a second copy that can disagree.

Doing both is how you end up with two copies and a precedence rule nobody remembers.

Other traps from the same move

I ran into this while moving an Ansible control node from a VM onto WSL on my workstation. That move turned up a few more problems, and they aren’t specific to WSL.

apt_repository appends; it doesn’t reconcile. When I added an entry for a URL that already had one with a different signed-by keyring, it left both lines in place, and apt then refused to read the sources at all: E:Conflicting values set for option Signed-By. For a repository you own end to end, write the .list file with copy, which replaces whatever was there.

ansible_env.HOME is a fact, and lookup('env', 'HOME') isn’t. In a play with gather_facts: false, and that includes ad-hoc ansible -m debug, the fact is undefined. On a local-connection control node the controller is the target, so the lookup gives you the same answer without needing facts.

Python’s write_text normalises line endings without telling you. I round-tripped an inventory file with CRLF endings through read_text / write_text to change one value, and all 266 of its lines came back rewritten. For surgical edits use read_bytes / write_bytes, and compare git diff --stat with the size of the change you meant to make.

One thing was WSL-specific. WSL regenerates /etc/resolv.conf on every restart unless /etc/wsl.conf sets generateResolvConf=false, so a role that manages resolv.conf seems to work and then gets undone at the next boot.