SSH Key Sprawl on Linux: Find and Fix Forgotten Keys

A production server gets rebuilt from an old image. A contractor finishes their work and moves on. CI/CD pipelines get decommissioned. Months later, the same SSH public keys are still in authorized_keys, quietly trusted by root or a service account nobody owns any more. That is SSH key sprawl on Linux in a nutshell. It builds up slowly through small access decisions that were never reversed. For attackers, those forgotten keys are not clutter. They are silent backdoors and a direct path around every identity control you think is protecting your fleet.

Why SSH Key Sprawl Gets Out of Control

SSH keys are easy to generate and have no expiry by default. That combination is what causes the problem. Someone creates a key pair for a quick task, adds the public key to authorized_keys, and moves on. The key stays there. Repeat that across dozens of engineers, contractors, and automated jobs over several years and you end up with hundreds of keys on your servers that nobody has reviewed.

A Linux server running for three years can easily accumulate 40 to 80 authorized keys. Others belong to engineers who left the company. Others came from CI/CD jobs that were replaced. Some cannot be traced to anyone at all. Each one grants passwordless root access for whoever holds the matching private key.

According to SANS Institute SSH key security guidance, attackers who get a private key gain repeatable access without triggering credential alerts. Unlike passwords, key compromise leaves almost no evidence in standard auth logs.

SSH Key Sprawl Linux: Audit Every Key Right Now

Managing SSH key sprawl on Linux starts with a full audit. Find every authorized_keys file on the system:

# Find all authorized_keys files
find /home /root /etc -name "authorized_keys" 2>/dev/null

# Count keys per user
for f in $(find /home /root -name "authorized_keys" 2>/dev/null); do
    count=$(grep -c "^ssh" "$f" 2>/dev/null || echo 0)
    echo "$count keys: $f"
done

Pull the fingerprints to cross-reference against your team’s known keys:

# List fingerprints with comments (shows who the key belongs to)
ssh-keygen -l -f /root/.ssh/authorized_keys

# Show last login per key from auth logs
grep "Accepted publickey" /var/log/auth.log | awk '{print $9, $11}' | sort | uniq -c | sort -rn

The comment field in each key usually shows the user and hostname it was generated on, like alice@laptop. Keys with no comment or a generic placeholder are the ones to investigate first. Anything that has not appeared in the last 30 days is a candidate for removal.

Also check service accounts:

for user in $(awk -F: '$3 >= 1000 {print $1}' /etc/passwd); do
    home=$(eval echo ~$user)
    if [ -f "$home/.ssh/authorized_keys" ]; then
        count=$(wc -l < "$home/.ssh/authorized_keys")
        echo "$user: $count keys"
    fi
done

Remove Stale Keys Safely

Back up the file first. Then remove keys one at a time and verify access still works before touching the next one.

# Back up first
cp /root/.ssh/authorized_keys /root/.ssh/authorized_keys.bak.$(date +%F)

# Remove a specific key by its comment field
grep -v "alice@oldlaptop" /root/.ssh/authorized_keys > /tmp/ak.tmp
mv /tmp/ak.tmp /root/.ssh/authorized_keys
chmod 600 /root/.ssh/authorized_keys

# Verify remaining keys
cat /root/.ssh/authorized_keys

Test that users who should still have access can connect before deleting the backup. Skipping that test is how you lock yourself out at the worst possible moment.

Stop Sprawl From Rebuilding

Cleaning up once is not enough if the habits that created the sprawl are still in place. Three changes make a real difference.

Switch to SSH Certificates

Static keys live forever unless someone removes them. SSH certificates have a built-in expiry. When a certificate expires, access ends automatically. Set up your own CA:

# Create an SSH CA key pair (store the private key offline)
ssh-keygen -t ed25519 -f /etc/ssh/ssh_ca -C "ssh-ca"

# Sign a user's public key with a 30-day certificate
ssh-keygen -s /etc/ssh/ssh_ca -I "alice" -n "root" -V +30d alice_key.pub

# Tell sshd to trust the CA
echo "TrustedUserCAKeys /etc/ssh/ssh_ca.pub" >> /etc/ssh/sshd_config
systemctl reload sshd

Contractors get certificates that expire when their engagement ends. No manual cleanup needed.

Weekly Audit Cron

cat > /etc/cron.weekly/ssh-key-audit << 'EOF'
#!/bin/bash
echo "=== SSH Key Audit: $(hostname) ===" > /tmp/ssh-audit.txt
find /home /root -name "authorized_keys" 2>/dev/null | while read f; do
    echo "" >> /tmp/ssh-audit.txt
    echo "File: $f" >> /tmp/ssh-audit.txt
    ssh-keygen -l -f "$f" 2>/dev/null >> /tmp/ssh-audit.txt
done
mail -s "Weekly SSH Audit - $(hostname)" root < /tmp/ssh-audit.txt
EOF
chmod +x /etc/cron.weekly/ssh-key-audit

Manage Keys Through Ansible

If you use Ansible, manage authorized_keys through it. When someone leaves, remove them from the playbook and run it. Every server updates at once:

- name: Manage authorized SSH keys
  ansible.posix.authorized_key:
    user: root
    key: "{{ lookup('file', 'keys/alice.pub') }}"
    state: present   # change to absent to remove

This makes your key state version-controlled, auditable, and consistent across the whole fleet.

Why This Matters More Right Now

SSH key sprawl on Linux is a bigger risk than usual given this month's kernel vulnerabilities. CVE-2026-46333 (ssh-keysign-pwn) specifically targets ssh-keysign to steal SSH host private keys. If an attacker exploits that flaw and then finds 60 stale authorized keys on your servers, they have a large set of persistent access paths to explore. Clean up the keys first. Then patch the kernel. Our full guide on CVE-2026-46333 and the LinuxSecurity.com SSH sprawl coverage both go deeper on the attack paths these forgotten keys enable. And the Linux server hardening checklist covers the full access control baseline that keeps stale keys from being the weakest link.

Conclusion

The audit takes 10 minutes. Removing stale keys takes another 20. Switching to certificates or Ansible-managed keys prevents the problem coming back. None of this needs downtime or specialist tools. It just needs treating authorized_keys with the same discipline you apply to any other access control list. Start today and run the audit command on every server you manage. You will almost certainly find keys you forgot were there.