How to Secure Your SSH Server: The Definitive Hardening Guide

Securing Your SSH Server: Best Practices with Practical Insights

SSH is how most of us manage Linux servers remotely, and it works beautifully — until your server starts showing up in automated scans. If you’ve ever grepped through /var/log/auth.log and found thousands of failed login attempts from random IPs around the world, you already know what I’m talking about. The default SSH configuration that ships with most Linux distributions is perfectly functional, but it’s wide open for attack.

Hardening SSH isn’t a single fix — it’s a set of layers you stack on top of each other. Each one raises the cost of an attack. Combine enough of them, and your server stops being an easy target. This guide walks through every layer worth applying, with working configuration examples for modern Linux systems.

Note: Security is a continuous process of managing risk, not eliminating it. These steps will stop the vast majority of automated threats and make targeted attacks significantly harder.

---

1. Enforce Key-Based Authentication

If you do nothing else in this guide, do this. Cryptographic key pairs are fundamentally stronger than passwords — an Ed25519 private key is practically impossible to brute-force, whereas a weak or reused password can be compromised in seconds. The switch from passwords to keys is the single biggest improvement you can make.

Practical Steps:

1. Generate a modern key pair on your local machine: Ed25519 is the recommended algorithm today — it’s faster and more secure than legacy RSA. Run this on your local workstation, not the server:

   ssh-keygen -t ed25519 -C "your_email@example.com"
   

   When prompted for a passphrase, set a strong one. It protects your private key if your machine is ever compromised.

2. Copy the public key to your remote server: The ssh-copy-id utility handles appending your public key to the server’s authorized keys file cleanly:

   ssh-copy-id -i ~/.ssh/id_ed25519.pub username@<server_ip>
   

3. Verify the connection: Before you touch any configuration files, confirm that key-based login is actually working:

   ssh username@<server_ip>
   

   If you connected without being asked for a password, you’re ready for the next step.

4. Set correct file permissions on the server: SSH will silently reject your keys if the permissions are too loose. These exact modes are required:

   chmod 700 ~/.ssh
   chmod 600 ~/.ssh/authorized_keys
   

---

2. Disable SSH Password Authentication

Once keys are confirmed working, turn off password authentication completely. There’s no reason to keep it enabled — it only exists as a fallback that attackers can exploit.

Configuration:

On modern Linux distributions, the cleanest approach is to place your changes in /etc/ssh/sshd_config.d/ rather than editing the main config file directly. Create a dedicated hardening file:

PasswordAuthentication no
PermitEmptyPasswords no
PubkeyAuthentication yes
KbdInteractiveAuthentication no

(On older OpenSSH versions prior to 8.x, the directive is ChallengeResponseAuthentication instead of KbdInteractiveAuthentication.)

A note on PAM:

You’ll likely see UsePAM yes in /etc/ssh/sshd_config. Leave it there — PAM handles session setup, environment variables, and user limits, none of which are password authentication. Keeping it enabled while setting the above directives is perfectly safe.

Apply the changes by restarting SSH:

# On Debian/Ubuntu:
sudo systemctl restart ssh

# On RHEL/CentOS/Rocky Linux:
sudo systemctl restart sshd

[!WARNING] Keep your current SSH session open and test the connection in a separate terminal before closing anything. A misconfiguration here can lock you out permanently. Test first, then close.

---

3. Disallow Root Login

The root account exists on every Linux system, which means an attacker already knows one valid username. All they need is the password or key. By blocking direct root login over SSH, you force anyone who needs root access to log in as a regular user first and then escalate with sudo — adding a meaningful extra step.

Configuration:

Add this to your SSH hardening file:

PermitRootLogin no

What if you need root SSH access for automation?

Some backup agents and configuration management tools require it. In that case, restrict root to key-based authentication only — no passwords ever:

PermitRootLogin prohibit-password

Restart SSH to apply:

sudo systemctl restart ssh

---

4. Change the Default SSH Port

Moving SSH off port 22 won’t stop a determined attacker who specifically scans your IP, but it filters out an enormous amount of noise from internet-wide scanners that only probe the default port. Most automated brute-force bots never bother with non-standard ports. It’s a small change that meaningfully reduces log noise and attack surface.

Steps:

1. Pick a high-numbered port (anywhere between 1024 and 65535) and add it to your config:

   Port 5823
   

2. Open the new port in your firewall before restarting SSH — this is the step people forget, and it locks them out:

   Using UFW (Ubuntu/Debian):

   sudo ufw allow 5823/tcp
   sudo ufw reload
   

   Using Firewalld (RHEL/Rocky Linux):

   sudo firewall-cmd --add-port=5823/tcp --permanent
   sudo firewall-cmd --reload
   

3. On SELinux-enforced systems (Rocky Linux, RHEL): SELinux blocks SSH from binding to unlisted ports by default. Allow the new port explicitly:

   sudo semanage port -a -t ssh_port_t -p tcp 5823
   

4. Restart SSH:

   sudo systemctl restart ssh
   

5. Connect using the new port:

   ssh -p 5823 username@<server_ip>
   

---

5. Implement Two-Factor Authentication (2FA)

For servers that handle sensitive data or are accessible from the public internet, adding a second authentication factor is worth the extra setup time. The most common approach pairs your SSH key with a time-based one-time password (TOTP) using Google Authenticator’s PAM module — though any TOTP app works.

Setup:

1. Install the Google Authenticator PAM module:

   # On Debian/Ubuntu:
   sudo apt update && sudo apt install libpam-google-authenticator -y
   
   # On RHEL/Rocky Linux:
   sudo dnf install epel-release -y
   sudo dnf install google-authenticator -y
   

2. Run the initialization tool as the user who will be logging in:

   google-authenticator
   

   Walk through the prompts:

   • Scan the QR code with your authenticator app (Google Authenticator, Authy, Aegis, or any TOTP app).
   • Save the emergency scratch codes somewhere safe — you’ll need them if you lose your phone.
   • Answer yes to time-based tokens, rate limiting, and disallowing token reuse.

3. Configure PAM to require the authenticator: Edit /etc/pam.d/sshd and add this line at the end:

   auth required pam_google_authenticator.so nullok
   

   The nullok flag lets users who haven’t set up 2FA yet still log in. Remove it once everyone has enrolled to make 2FA mandatory.

4. Update the SSH daemon to enable keyboard-interactive auth: In your hardening config file:

   KbdInteractiveAuthentication yes
   

   (Use ChallengeResponseAuthentication yes on older OpenSSH versions.)

5. Require both key AND authenticator code: Without this, SSH might accept either factor alone. To require both:

   AuthenticationMethods publickey,keyboard-interactive
   

6. Restart SSH:

   sudo systemctl restart ssh
   

---

6. Limit Access by User, Group, or IP Address

The principle of least privilege applies to SSH access too. If only three people need to log into a server, there’s no reason to allow SSH for every account on the system. Lock it down explicitly.

Restricting Users & Groups:

Add AllowUsers or AllowGroups to your SSH config:

AllowUsers alice bob
AllowGroups sysadmins

Restricting Access by IP:

If your team connects from a known IP range or through a VPN, you can limit SSH exposure significantly.

1. Bind SSH to a private network interface: If the server is on a VPN like WireGuard or Tailscale, configure SSH to listen only on that private IP — not the public-facing interface:

   ListenAddress 10.0.0.5
   

2. Use firewall rules to whitelist source IPs:

   sudo ufw allow from 203.0.113.50 to any port 5823 proto tcp
   

---

7. Harden Cryptographic Ciphers & Algorithms

Out-of-the-box SSH enables a range of legacy algorithms for backward compatibility. Several of these — including SHA-1-based MACs, 3DES, and Arcfour — are cryptographically weak and should not be in use on any server you care about.

You can audit your current configuration with the ssh-audit tool from the command line, or check it online at ssh-audit.com.

To enforce a clean, modern algorithm set, add the following to your hardening config:

# Restrict host key types to Ed25519 only
HostKeyAlgorithms ssh-ed25519,ssh-ed25519-cert-v01@openssh.com

# Modern key exchange algorithms
KexAlgorithms curve25519-sha256,curve25519-sha256@libssh.org,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512

# Strong symmetric ciphers only
Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com

# Authenticated encryption MACs
MACs hmac-sha2-512-etm@openssh.com

After adding these, run ssh-audit against your server again — you should see nothing flagged as weak.

---

8. Configure Connection and Session Timeouts

An idle SSH session left open on an unlocked laptop is a real attack vector. Configuring the server to close unresponsive connections limits this risk, and throttling concurrent connection attempts slows down any brute-force tools that slip through your other defenses.

# Ping the client every 5 minutes to check if it's still there
ClientAliveInterval 300

# Close the session after 2 missed responses (roughly 10 minutes of silence)
ClientAliveCountMax 2

# Limit authentication attempts per connection
MaxAuthTries 3

# Throttle concurrent unauthenticated connections
# After 10 simultaneous attempts, start randomly rejecting at 30% rate, hard-stop at 30 connections
MaxStartups 10:30:30

---

9. Set Up Log Monitoring and Fail2Ban

Everything above reduces the attack surface, but it doesn’t stop someone from trying. Fail2ban watches your SSH logs in real time and automatically bans IP addresses that repeatedly fail authentication — turning persistent attacks into a self-defeating exercise.

Steps:

1. Install Fail2ban:

   sudo apt install fail2ban -y
   

2. Create a local configuration file: Never edit jail.conf directly — updates will overwrite your changes. Copy it to .local first:

   sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
   

   Then configure the SSH jail in /etc/fail2ban/jail.local:

   [sshd]
   enabled = true
   port = 5823
   logpath = %(sshd_log)s
   backend = %(sshd_backend)s
   maxretry = 3
   bantime = 1d
   findtime = 10m
   

   This bans any IP for 24 hours after 3 failed attempts within a 10-minute window. Adjust the values to match your threat tolerance.

3. Start and enable Fail2ban:

   sudo systemctl restart fail2ban
   sudo systemctl enable fail2ban
   

Watching the logs:

Check SSH activity directly through journald on modern systems:

sudo journalctl -u ssh -f
# or on RHEL-based systems:
sudo journalctl -u sshd -f

Or tail the auth log file directly:

# Debian/Ubuntu:
sudo tail -f /var/log/auth.log

# RHEL/Rocky Linux:
sudo tail -f /var/log/secure

---

Summary Checklist

Here’s a quick reference for everything covered above:

• SSH key-based authentication with Ed25519
• Password authentication disabled (PasswordAuthentication no)
• Root login disabled (PermitRootLogin no)
• SSH moved to a non-standard port
• Multi-Factor Authentication via PAM (for high-security environments)
• User and IP access restrictions in place
• Weak ciphers and algorithms removed
• Idle timeout and connection limits configured
• Fail2ban running and monitoring SSH logs

Every one of these layers adds friction for an attacker. Together, they make your server an unappealing target compared to the thousands of unprotected machines on the same internet. Put in the work once, and you’ll rarely think about it again.

Stay Vigilant, Stay Secure.