ansible: codify workstation SSH keys after a fleet-wide lockout - #606
Merged
Merged
Conversation
2.15 set PasswordAuthentication no everywhere on 09-20. On 09-23 the Windows laptop could not log in anywhere: it had never been given a key, and password auth had been carrying it silently. ansible/files/ held one public key - the break-glass one - and no mechanism for a workstation key existed. That entry examines what the change does to sshd in detail and never asks who was authenticating with a password. Before removing an authentication method, enumerate who is actually using it. Recovery was the break-glass key, which worked on .160, .42 and .21 - so 1.2's 'test it' checkbox, open since August, is answered by a real lockout rather than a drill. Key auth is unaffected by PasswordAuthentication no, which is why that path survived the change that caused the outage. admin-keys.yml replaces the ad-hoc recovery push. It asserts at least one key exists and that each file matches an SSH public-key shape, because the recovery command ran with an unset variable, handed authorized_key an empty string, and got SUCCESS on thirteen hosts with changed: false. Regex verified against the real keys and against empty, garbage, a bare key type, and a private key pasted by mistake. Excludes xu3-1 deliberately (2.16): a key granting NOPASSWD sudo on a host running OpenSSL 1.0.2g. Records that the recovery push already put one there and must be removed. Also records opi-zero2w-1 down with No route to host - the second Zero 2W in four days to silently miss a fleet-wide change, and the DNS tertiary.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
2.15 set PasswordAuthentication no everywhere on 09-20. On 09-23 the Windows laptop could not log in anywhere: it had never been given a key, and password auth had been carrying it silently. ansible/files/ held one public key - the break-glass one - and no mechanism for a workstation key existed.
That entry examines what the change does to sshd in detail and never asks who was authenticating with a password. Before removing an authentication method, enumerate who is actually using it.
Recovery was the break-glass key, which worked on .160, .42 and .21 - so 1.2's 'test it' checkbox, open since August, is answered by a real lockout rather than a drill. Key auth is unaffected by PasswordAuthentication no, which is why that path survived the change that caused the outage.
admin-keys.yml replaces the ad-hoc recovery push. It asserts at least one key exists and that each file matches an SSH public-key shape, because the recovery command ran with an unset variable, handed authorized_key an empty string, and got SUCCESS on thirteen hosts with changed: false. Regex verified against the real keys and against empty, garbage, a bare key type, and a private key pasted by mistake.
Excludes xu3-1 deliberately (2.16): a key granting NOPASSWD sudo on a host running OpenSSL 1.0.2g. Records that the recovery push already put one there and must be removed. Also records opi-zero2w-1 down with No route to host - the second Zero 2W in four days to silently miss a fleet-wide change, and the DNS tertiary.