Resetting a Debian host: apt history is your baseline, and unmount before you rm

How I took a Raspberry Pi back to a fresh-install state without reflashing it: reconstruct what was added from /var/log/apt/history.log, unmount NFS before touching the mountpoint, and reboot to clear leftover Docker networking.

I wanted a Raspberry Pi back to how it was after the image was written: no Docker, no NFS mount, none of the packages my Ansible roles had added. Reflashing would have been the honest way, but the Pi was reachable, had no running containers, and I wanted to keep its network setup and SSH access. The catch is that “as it was after install” isn’t written down anywhere.

Where the baseline comes from

apt-mark showmanual is the obvious start, and it’s a poor one. It lists everything that was ever marked manual, and that includes the packages the image shipped with, so I’d have been guessing which of a couple of hundred names were mine.

/var/log/apt/history.log (and the rotated history.log.1.gz and friends) is better. Each entry has a Start-Date, the exact command line, and the packages it installed. Mine showed the Ansible package role’s single apt-get install with its ten packages, the guest agent, apt-transport-https, one apt install lsof I typed by hand, and finally the Docker install. Those were the packages to purge, and everything else stayed.

zcat -f /var/log/apt/history.log* | grep -E 'Start-Date|Commandline|^Install:'

The limit is worth stating. The oldest log started at the first Ansible run, not at first boot, so anything installed before that is invisible to this method. I treated “not in the log” as “came with the image”. That’s an assumption, and it held here, but on a host with a long history it might not.

Removal was apt-get purge for the listed packages, then apt-get autoremove --purge for the dependencies they had pulled in.

Unmount first, then look at the mountpoint

The Pi mounted a directory from the NAS over NFS. Deleting anything under a live mountpoint deletes it on the NAS, so the order matters: take the line out of /etc/fstab, run systemctl daemon-reload, unmount, and confirm with mount that it is really gone before any rm.

Then look at the directory, because my rmdir failed. Underneath the mount there was a small folder from before the mount existed. Anything written to a directory before something is mounted over it is hidden by the mount and comes back into view when you unmount. It was leftover local data from my own early setup, and only after looking at it did I delete it.

Leftovers that survive the purge

Removing the Docker packages doesn’t remove the running daemon’s state. After the purge the Pi still had a docker0 interface and a network-namespace mount under /run/docker. Both were in memory, not on disk, and a reboot cleared them. I rebooted, then checked for what I cared about: no docker0, no NFS mounts, no failed units, and SSH still answering.

I had deliberately left SSH and its root login settings alone. A reset that locks you out of the machine you’re resetting is only half done.