Locking Down SSH with Public Key Authentication

Introduction

If you're running a VPS, locking down SSH access is a must. Using passwords alone isn't safe anymore. Let's walk through how to set up SSH key-based authentication on Linux, Windows, and Mac — and sprinkle in some fun facts, pro tips, and gotchas to watch for.

Step 1: Create Your SSH Key Pair

Open your terminal and run:

ssh-keygen -t rsa -b 4096 -C "[email protected]"
  • It will ask: "Enter file in which to save the key" — just hit enter to accept the default location (~/.ssh/id_rsa).
  • Enter a passphrase if you want extra security (optional but recommended).

This creates two files: - id_rsa — your private key (keep it SECRET!) - id_rsa.pub — your public key (share this freely)

Step 2: Installing SSH if You Don't Have It

  • Debian/Ubuntu:
sudo apt update
sudo apt install openssh-client
  • Arch Linux:
sudo pacman -S openssh

Step 3: Upload Your Public Key to the Server

If password authentication is still enabled, you can simply run:

ssh-copy-id user@your_vps_ip

This will: - Copy your public key to ~/.ssh/authorized_keys on the server. - Set the correct permissions automatically.

When it works, you'll get a confirmation like:

Number of key(s) added: 1
Now try logging into the machine...

Step 4: Configure the SSH Server to Disable Password Authentication

Edit your server's SSH config file:

sudo nano /etc/ssh/sshd_config

Make sure you have:

PasswordAuthentication no
ChallengeResponseAuthentication no
UsePAM no
PubkeyAuthentication yes

Note: If changing /etc/ssh/sshd_config doesn't seem to work, check /etc/ssh/sshd_config.d/50-cloud-init.conf or other .conf files included by your main config. Cloud providers sometimes override your settings!

Restart SSH after changes:

sudo systemctl restart sshd

Step 5: Testing Your Setup

Before closing your current SSH session, open a new terminal window and try to connect:

ssh user@your_vps_ip

If everything is set up correctly, it will log you in without asking for a password. Instead, it uses your private key.

✅ On success, you'll see your normal login greeting — no password prompt.

If you try from a machine without your key, it should now fail with a Permission denied (publickey) error.

For example, from an Arch Linux WSL machine without the right keys, you'll get:

Permission denied (publickey).

Which is exactly what you want — a locked down server!

Step 6: Backup Your SSH Keys

Back up your .ssh folder somewhere safe:

cp -r ~/.ssh /path/to/your/backup/location

You can then restore it on another machine if needed.

Pro Tip: Always back up your private keys before locking down the server, so you don't lock yourself out.

Why SSH Keys Are Safer Than Passwords

  • Passwords can be brute-forced.
  • Private keys are extremely hard to guess, especially when encrypted.
  • Even if someone knows your username, without your private key, they're out of luck.

Key-based auth is millions of times safer than relying on a password alone.

Common Pitfall: Why ssh-copy-id Might Fail After Disabling PasswordAuth

If you already disabled password login (PasswordAuthentication no), ssh-copy-id won't work anymore because it needs a password to transfer the key.

Solutions: - Add keys manually if you still have SSH access. - Or pre-load your keys before turning off password authentication.

Manual upload:

cat ~/.ssh/id_rsa.pub | ssh user@your_vps_ip "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys"

Cool SSH Tricks and Techniques ✨

  • Connect with a specific identity file:

Use a custom private key when connecting instead of the default one:

ssh -i ~/.ssh/my_other_key user@your_vps_ip
  • Use a config file for multiple hosts (~/.ssh/config):

Simplify your SSH commands by setting up shortcuts:

Host myserver
    HostName your_vps_ip
    User user
    IdentityFile ~/.ssh/id_rsa

Then you can just connect by typing:

ssh myserver
  • SSH Agent Forwarding (advanced):

Forward your SSH credentials when hopping between servers:

ssh -A user@your_vps_ip

This lets you SSH from your VPS to another machine using your local keys, without copying your private key around.

  • Port forwarding over SSH:

Tunnel a port securely through your SSH connection:

ssh -L 8080:localhost:80 user@your_vps_ip

This forwards your VPS's port 80 (web server) to your local machine's port 8080.

Useful for accessing internal websites, databases, or admin panels securely!

Final Thoughts

Setting up SSH keys isn't just "Geek stuff" — it's basic security hygiene today. Plus, it's pretty satisfying to zip right into your server without ever typing a password.

🔒 Lock it down, back it up, and sleep better knowing your server is secure!