How to Secure Your SSH Server: The Definitive Hardening Guide

Securing Your SSH Server: Best Practices with Practical Insights

Stand up a fresh VPS with a public IP and time how long it takes for the first unsolicited SSH login attempt to land. On a typical cloud provider’s address range it is usually under an hour, often under ten minutes. Nobody found your server. Somebody is scanning the entire IPv4 space on port 22 continuously, and your address came up in the rotation.

That is the baseline condition, and it explains why the default sshd_config shipped by most distributions is the wrong starting point. It is not insecure by accident — it is permissive on purpose, because it has to work on a laptop on a private LAN, a build box behind a bastion, and a machine sitting naked on the internet. Only one of those is your situation.

Hardening SSH is not one fix. It is a stack of layers, each of which raises the cost of an attack a little, and several of which are worthless on their own. Moving off port 22 stops nothing if passwords are still enabled. Disabling passwords is undermined by a helpdesk that will reset a key on request. What follows is every layer worth applying, with working configuration for current OpenSSH, and — more usefully — what each one costs you.

Note: None of this eliminates risk; it moves you out of the population that automated tooling can harvest at scale. Against an attacker who has specifically decided to get into your server, these measures buy time and generate evidence. That is still the whole game — most compromises are opportunistic, and opportunistic attackers move on to the next address.

---

1. Enforce Key-Based Authentication

If you do nothing else here, do this. The argument is not that keys are “stronger” in some abstract sense — it is that a password is a secret small enough to be guessed remotely, and an Ed25519 private key is not. Brute-forcing a 256-bit curve key over the network is not a slow attack; it is not an attack. Meanwhile the password on that account is probably reused somewhere that has already been breached, and the credential-stuffing bots have the list.

The cost is real and worth stating: you are trading a secret you can remember for a file you can lose. Lose the laptop holding the only copy of your private key and you are locked out of your own server, with no reset flow to fall back on. Before you go any further, decide where your second key lives — a hardware token, an offline backup, a second admin account with its own key. “I’ll sort that out later” is how people end up rebuilding a server from backups over a lost key.

Practical Steps:

1. Generate a modern key pair on your local machine: Ed25519 is the default choice on any OpenSSH from 6.5 onwards: small keys, fast signatures, no parameter choices to get wrong. RSA is still fine at 4096 bits if you have to interoperate with something old, but you have to pick the size, and plenty of legacy tooling will happily hand you 1024. Generate this on your workstation, never on the server — the private key should not exist on the machine it authenticates to:

   ssh-keygen -t ed25519 -C "your_email@example.com"
   

   Set a passphrase. An unencrypted private key is a file that grants root-equivalent access to anyone who copies it — including malware running as your own user, which does not need to escalate privileges to read ~/.ssh/. Use ssh-agent so you type the passphrase once per session rather than once per connection; that is the difference between a control people keep and one they disable in week two.

2. Copy the public key to your remote server: ssh-copy-id appends the key and fixes the directory permissions in one step, which is why it is worth using over a hand-rolled cat >> authorized_keys. This is the last time you will need password authentication:

   ssh-copy-id -i ~/.ssh/id_ed25519.pub username@<server_ip>
   

3. Verify the connection: Before you touch any configuration file, confirm key login actually works:

   ssh username@<server_ip>
   

   If you got in without a password prompt, you are ready. If you were prompted, stop — you are about to disable the only method that works. Re-run with -vvv and read the client output; nine times out of ten it names the file it tried and the reason the server rejected it.

4. Set correct file permissions on the server: This is the classic silent failure. If ~/.ssh or authorized_keys is group- or world-writable, sshd ignores the key entirely and falls through to the next auth method — no error to the client, just a password prompt that makes it look as though the key was never copied. The rejection is logged server-side; the client is told nothing useful. The home directory itself matters too, which is why a chmod 777 ~ during some unrelated troubleshooting session breaks SSH login hours later:

   chmod 700 ~/.ssh
   chmod 600 ~/.ssh/authorized_keys
   

---

2. Disable SSH Password Authentication

Keys working is not the same as passwords disabled. Until you turn the fallback off, every one of those thousands of automated attempts is still a live lottery ticket against every account on the box — including the service accounts nobody remembers creating, with the passwords nobody ever changed. Leaving it on buys you nothing, because you are no longer using it.

Configuration:

Put your changes in a file under /etc/ssh/sshd_config.d/ rather than editing the main config. Two reasons: a package upgrade that ships a new sshd_config will not silently revert your hardening, and a single 99-hardening.conf is far easier to audit than a diff against a 130-line default. One caveat worth checking first — the drop-in only works if the main config actually contains an Include /etc/ssh/sshd_config.d/*.conf line, and it must appear near the top, because OpenSSH takes the first value it sees for most keywords. An Include at the bottom of the file will be quietly overridden by anything set above it:

PasswordAuthentication no
PermitEmptyPasswords no
PubkeyAuthentication yes
KbdInteractiveAuthentication no

(On older OpenSSH versions prior to 8.x, the directive is ChallengeResponseAuthentication instead of KbdInteractiveAuthentication.)

Name the file so it sorts last — 99-hardening.conf. Cloud images matter here: Ubuntu’s cloud-init writes 50-cloud-init.conf containing PasswordAuthentication yes, and since first-match wins, a file named 10-hardening.conf would take precedence while a badly named one would not. Verify rather than assume, with sudo sshd -T | grep -Ei 'passwordauth|permitroot|pubkeyauth' — that prints the daemon’s effective configuration after every include and override, which is the only view that tells you the truth.

A note on PAM:

You’ll likely see UsePAM yes in /etc/ssh/sshd_config. Leave it. PAM is doing session setup, pam_limits, environment, and login records — none of that is password authentication, and turning it off to “be safe” gives you sessions with missing ulimits and no wtmp entry, which you will notice only when an investigation needs the login history. It is also the hook 2FA plugs into later in this guide.

Apply the changes by restarting SSH:

# On Debian/Ubuntu:
sudo systemctl restart ssh

# On RHEL/CentOS/Rocky Linux:
sudo systemctl restart sshd

[!WARNING] Keep your current session open and open a second terminal to test. An existing SSH session survives a daemon restart, so the connection you are typing in proves nothing about whether the next one will work. Run sudo sshd -t before every restart — it parses the config and refuses on error, which catches most lockouts before they happen. If you have console access through your provider’s dashboard, confirm it works now, while you still have a shell to fall back on.

---

3. Disallow Root Login

Every Linux system has a root account, so an attacker starts every attempt already holding half the credential. Blocking direct root login forces a two-stage path: authenticate as a named user, then escalate with sudo.

The real benefit is not the extra step — it is attribution. sudo writes who ran what, and when. On a box where three admins all log in as root, the audit trail says “root deleted the database” and stops there. That distinction is the whole reason auditors care about this setting, and it is why the answer is still no even on a single-admin server that you might one day hand over to someone else.

Configuration:

Add this to your SSH hardening file:

PermitRootLogin no

Before you set this, make sure your unprivileged account is actually in sudo (Debian/Ubuntu) or wheel (RHEL family) and that you have tested sudo -v in a live session. Locking root out of SSH on a machine where your user cannot escalate is the second most common way to lose access to a server.

What if you need root SSH access for automation?

Some backup agents and configuration management tools genuinely need it — Borg pulling a full filesystem, or an Ansible bootstrap running before any account exists. Restrict root to keys only:

PermitRootLogin prohibit-password

Better still, constrain what that key can do. A command="/usr/bin/borg serve --append-only" prefix with restrict in front of the key in root’s authorized_keys means a stolen backup key gets an append-only repository handler rather than an interactive root shell. Most automation needs one command, not a login.

Restart SSH to apply:

sudo systemctl restart ssh

---

4. Change the Default SSH Port

Be clear about what this buys: nothing, against anyone who runs nmap -p- against your address. Masscan sweeps the entire IPv4 range on a single port in minutes; sweeping all 65,535 ports on one host takes seconds. Port 5823 is not a secret.

What it does is drop the volume of untargeted attempts by roughly two orders of magnitude, and that is an operational win rather than a security one. When auth.log is 40,000 lines of bot traffic a day, nobody reads it, and the one interesting entry — a successful login from an IP nobody recognises — sits in the middle of the noise unnoticed. Cut the noise and the log becomes something you can actually alert on.

The trade-off is friction that lands on humans, not attackers: every team member’s ~/.ssh/config needs the port, every runbook and CI job needs updating, and the person debugging a connection at 2 a.m. six months from now will try port 22 first and conclude the host is down.

Steps:

1. Pick a high-numbered port (anywhere between 1024 and 65535) and add it to your config. Stay above 32768 or check /proc/sys/net/ipv4/ip_local_port_range first — pick a port inside the ephemeral range and an outbound connection can occasionally grab it before sshd binds, giving you a daemon that fails to start after a reboot for no apparent reason:

   Port 5823
   

   On Ubuntu 22.10 and later, including 24.04, SSH is socket-activated by default. systemd owns the listening socket, so the Port directive in sshd_config is ignored and your change appears to do nothing. Override the socket instead — sudo systemctl edit ssh.socket, clear ListenStream= with an empty assignment, then set your port — or disable socket activation with sudo systemctl disable --now ssh.socket && sudo systemctl enable --now ssh.service. Either way, sudo ss -tlnp | grep ssh is the check that tells you which port is genuinely bound.

2. Open the new port in your firewall before restarting SSH — this is the step people forget, and it locks them out:

   Using UFW (Ubuntu/Debian):

   sudo ufw allow 5823/tcp
   sudo ufw reload
   

   Using Firewalld (RHEL/Rocky Linux):

   sudo firewall-cmd --add-port=5823/tcp --permanent
   sudo firewall-cmd --reload
   

3. On SELinux-enforced systems (Rocky Linux, RHEL): SELinux only permits sshd to bind ports labelled ssh_port_t. Skip this and the daemon fails to start with a permission error that has nothing to do with your firewall, which is a confusing twenty minutes if you are not expecting it. semanage lives in policycoreutils-python-utils and is often not installed on a minimal image:

   sudo semanage port -a -t ssh_port_t -p tcp 5823
   

4. Restart SSH:

   sudo systemctl restart ssh
   

5. Connect using the new port:

   ssh -p 5823 username@<server_ip>
   

---

5. Implement Two-Factor Authentication (2FA)

Worth being honest about the threat model here. If you have already disabled passwords, TOTP is not protecting you from remote guessing — that was already impossible. It protects you from one specific scenario: someone has your private key file and its passphrase, typically because they compromised your workstation. That is a narrower risk than most guides imply, but it is also the risk that actually materialises, because workstations are far softer targets than servers.

The cost is availability. A phone that is lost, wiped, or simply out of battery is now between you and your infrastructure, and TOTP additionally depends on the server’s clock being roughly correct — let NTP drift by a couple of minutes on a machine with a bad RTC and every code is rejected with no explanation beyond “access denied”. Print the scratch codes. Store them somewhere that is not the phone.

The setup below pairs your key with a time-based one-time password using the Google Authenticator PAM module; any TOTP app generates compatible codes.

Setup:

1. Install the Google Authenticator PAM module:

   # On Debian/Ubuntu:
   sudo apt update && sudo apt install libpam-google-authenticator -y
   
   # On RHEL/Rocky Linux:
   sudo dnf install epel-release -y
   sudo dnf install google-authenticator -y
   

2. Run the initialisation tool as the user who will be logging in:

   google-authenticator
   

   Walk through the prompts:

   • Scan the QR code with your authenticator app (Google Authenticator, Authy, Aegis, or any TOTP app).
   • Save the emergency scratch codes somewhere safe — you’ll need them if you lose your phone.
   • Answer yes to time-based tokens, rate limiting, and disallowing token reuse.

3. Configure PAM to require the authenticator: Edit /etc/pam.d/sshd and add this line at the end:

   auth required pam_google_authenticator.so nullok
   

   nullok lets users who have not enrolled log in without a code. That is the setting that stops you locking out the accounts you forgot about — and it is also the setting that makes your 2FA optional in practice. Leave it in place during rollout, then remove it, and understand that removing it will break any account without a ~/.google_authenticator file, including service accounts. Check first: sudo find /home -maxdepth 2 -name .google_authenticator.

   Alternatively, if you want per-user control instead of an all-or-nothing switch, AuthenticationMethods can be scoped inside a Match User or Match Group block so 2FA applies to human admins while an automation account authenticates by key alone.

4. Update the SSH daemon to enable keyboard-interactive auth: In your hardening config file:

   KbdInteractiveAuthentication yes
   

   (Use ChallengeResponseAuthentication yes on older OpenSSH versions.)

5. Require both key AND authenticator code: This is the step that decides whether you built two-factor authentication or two independent single-factor paths. Without it, sshd accepts whichever method succeeds first, so an attacker with a key never sees a code prompt — and the setup looks fully working from the admin’s side, because the admin has both. Comma-separated means “all of these, in order”:

   AuthenticationMethods publickey,keyboard-interactive
   

6. Restart SSH:

   sudo systemctl restart ssh
   

---

6. Limit Access by User, Group, or IP Address

By default, any account with a valid shell can log in over SSH. On a web server that includes whatever the package manager created — and while www-data normally has /usr/sbin/nologin, plenty of application installers set a real shell because their upgrade script needed one. AllowUsers turns SSH access from a property of every account into an explicit list, which means adding a service account no longer silently adds a login path.

The trade-off is that the list is now a thing that has to be maintained, and it fails closed: onboard someone, forget the config, and they cannot get in. That is the correct failure direction, but expect the support ticket.

Restricting Users & Groups:

Prefer AllowGroups for anything with more than two admins — group membership is managed with usermod, which does not require editing and reloading the daemon config every time the team changes. Note that AllowUsers and AllowGroups are OR’d together if both are present, which surprises people who expect them to intersect:

AllowUsers alice bob
AllowGroups sysadmins

Restricting Access by IP:

This is the layer that actually removes the attack surface rather than hardening it. Everything above assumes an attacker can reach the daemon; network restriction means they cannot, and a brute-force attempt against a port that never answers is not an attack you have to defend against.

1. Bind SSH to a private network interface: If the server sits on a WireGuard or Tailscale network, have SSH listen only on that address. Two cautions. ListenAddress binds at daemon start, so if sshd comes up before the VPN interface exists — a real possibility after a reboot — the bind fails and the service does not start; add the appropriate After= ordering or set net.ipv4.ip_nonlocal_bind=1. And you have now made the VPN a single point of failure for all administrative access, so your console fallback stops being optional:

   ListenAddress 10.0.0.5
   

2. Use firewall rules to allow only known source IPs:

   sudo ufw allow from 203.0.113.50 to any port 5823 proto tcp
   

   Workable when the team has static addresses, painful when they do not — residential connections change on reconnect, and mobile networks give you a different address every hour. If half your admins are on dynamic IPs this rule becomes a weekly chore, and the usual outcome is somebody widening it to a /16 “temporarily”. A VPN solves the same problem properly. Also add the equivalent IPv6 rule or check that IPv6 is genuinely disabled: a v4-only allow rule leaves port 5823 wide open on the AAAA address that most cloud providers now assign by default, and ufw status will show a tidy, restrictive-looking ruleset the whole time.

---

7. Harden Cryptographic Ciphers & Algorithms

Set expectations first: this is the lowest-value section in the guide for most people. Nobody’s server gets breached through a SHA-1 MAC. Current OpenSSH already disables the genuinely broken algorithms — Arcfour and CBC-mode ciphers went out of the defaults years ago, and ssh-rsa with SHA-1 signatures was disabled by default in OpenSSH 8.8. If you are on a recent distribution, the defaults are already fine.

Where it does matter is compliance evidence and old servers. A CIS benchmark or a client’s vulnerability scanner will flag the permitted algorithm list, and a box that has been upgraded in place since 2016 may well be carrying configuration that predates all those default changes.

Audit before you change anything, with ssh-audit from the command line or via ssh-audit.com — you may find there is nothing to do.

To enforce a modern algorithm set, add the following to your hardening config:

# Restrict host key types to Ed25519 only
HostKeyAlgorithms ssh-ed25519,ssh-ed25519-cert-v01@openssh.com

# Modern key exchange algorithms
KexAlgorithms curve25519-sha256,curve25519-sha256@libssh.org,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512

# Strong symmetric ciphers only
Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com

# Authenticated encryption MACs
MACs hmac-sha2-512-etm@openssh.com

Re-run ssh-audit afterwards; nothing should come back flagged.

Understand what you have just done, though. Restricting HostKeyAlgorithms to Ed25519 means any client that cannot do Ed25519 — old embedded devices, some Java-based automation using dated JSch or Paramiko builds, certain backup appliances — can no longer connect at all, and the error they report is usually an unhelpful “no matching host key type”. Restricting MACs to a single ETM algorithm is similarly absolute. Roll this out on one host, connect from every client that matters, and only then push it everywhere. This is also the section most likely to be the reason a CI job started failing on a Tuesday for no reason anyone can trace.

---

8. Configure Connection and Session Timeouts

Two different problems share this section. Idle sessions are a physical-access risk — a root shell left open on an unlocked laptop in a co-working space is the entire attack. Connection throttling is a resource problem: unauthenticated connections consume a sshd process each, and a few thousand of them will exhaust memory on a small VPS long before anyone guesses a key.

The trade-off with ClientAliveInterval is that it kills long-running interactive work. A rsync you started before lunch is fine — it generates traffic — but a shell parked at a prompt while you read documentation is not, and neither is an SSH tunnel sitting idle between requests. If you rely on either, use tmux so a dropped session costs you nothing, rather than raising the timeout until it no longer does anything.

Note also that MaxAuthTries counts offered keys, not typed passwords. Load six identities into your agent and the client will offer all of them in turn; with MaxAuthTries 3 you are disconnected before it reaches the right one, and the error reads like a rejected key rather than a tripped limit. IdentitiesOnly yes in your client config fixes it.

# Ping the client every 5 minutes to check if it's still there
ClientAliveInterval 300

# Close the session after 2 missed responses (roughly 10 minutes of silence)
ClientAliveCountMax 2

# Limit authentication attempts per connection
MaxAuthTries 3

# Throttle concurrent unauthenticated connections
# After 10 simultaneous attempts, start randomly rejecting at 30% rate, hard-stop at 30 connections
MaxStartups 10:30:30

---

9. Set Up Log Monitoring and Fail2Ban

Fail2ban’s real contribution, once passwords are off, is log hygiene rather than security — it stops the same three botnets filling your disk. Treat it as the thing that keeps auth.log readable, and the security benefit as a bonus.

Two things it will not do. It cannot ban a distributed attack, because a botnet with 40,000 addresses never trips a per-IP counter. And it introduces a denial-of-service path of its own: an attacker who can generate failed attempts appearing to come from your office IP can get your entire team banned. Put your own ranges in ignoreip before this becomes a Monday morning discovery.

Steps:

1. Install Fail2ban:

   sudo apt install fail2ban -y
   

2. Create a local configuration file: Never edit jail.conf — a package update overwrites it and your jails revert without warning. Cleaner still is a small /etc/fail2ban/jail.d/sshd.local containing only your overrides, rather than a full copy you then have to reconcile against upstream changes:

   sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
   

   Then configure the SSH jail in /etc/fail2ban/jail.local:

   [sshd]
   enabled = true
   port = 5823
   logpath = %(sshd_log)s
   backend = %(sshd_backend)s
   maxretry = 3
   bantime = 1d
   findtime = 10m
   

   Three failures in ten minutes earns a 24-hour ban. Note the port line: get this wrong after moving SSH off 22 and fail2ban happily parses the log, matches the failures, and writes firewall rules for a port nothing is listening on. The jail reports as active. Nothing is blocked. sudo fail2ban-client status sshd is the check — if Currently banned stays at zero on an internet-facing host while Total failed climbs, the jail is misconfigured, not effective.

3. Start and enable Fail2ban:

   sudo systemctl restart fail2ban
   sudo systemctl enable fail2ban
   

Watching the logs:

Failed logins are the boring half. The line worth alerting on is Accepted publickey for from a source address that is not in your expected set — a successful login you cannot account for is the only SSH log entry that ever needs a human at 3 a.m. Forward these off the host, too; an attacker with root can edit local logs, and on a fully hardened box the logs are frequently the only evidence you will have.

Check SSH activity directly through journald on modern systems:

sudo journalctl -u ssh -f
# or on RHEL-based systems:
sudo journalctl -u sshd -f

Or tail the auth log file directly:

# Debian/Ubuntu:
sudo tail -f /var/log/auth.log

# RHEL/Rocky Linux:
sudo tail -f /var/log/secure

---

Summary Checklist

Not all of these are equal. The first two do most of the work; items four and seven are largely hygiene. If you only have twenty minutes, do keys, disable passwords, and restrict the source addresses — the rest can wait for a quieter afternoon.

• SSH key-based authentication with Ed25519
• Password authentication disabled (PasswordAuthentication no)
• Root login disabled (PermitRootLogin no)
• SSH moved to a non-standard port
• Multi-Factor Authentication via PAM (for high-security environments)
• User and IP access restrictions in place
• Weak ciphers and algorithms removed
• Idle timeout and connection limits configured
• Fail2ban running and monitoring SSH logs

One habit is worth more than any single setting on that list: after every change, run sudo sshd -T and read the effective configuration rather than trusting the file you edited. Includes, Match blocks, cloud-init drop-ins and first-match-wins ordering conspire to make sshd_config say one thing while the daemon does another, and a hardening directive that is being silently overridden looks exactly like one that is working.

Then re-check it in six months. Configuration drifts — a distribution upgrade rewrites a file, someone re-enables passwords to onboard a contractor and never turns them back off, a departed admin’s key is still sitting in authorized_keys. That last one is the most common finding in real audits, and no amount of cipher tuning compensates for it.

Stay Vigilant, Stay Secure.