How to Harden SSH Access on Your Linux Server: A Practical Security Checklist

In this article

1. 1. Switch to SSH Key Authentication and Disable Passwords
2. Generating an ED25519 Key Pair
3. Deploying the Public Key to Your Server
4. Disabling Password Authentication in OpenSSH
5. 2. Disable Direct Root Logins
6. Creating a Dedicated Administrative User
7. Updating the Configuration
8. 3. Restrict Session Lifespans and Authentication Attempts
9. 4. Restrict SSH Access to Specific Users or Groups
10. 5. Deploy Automated Intrusion Prevention with Fail2ban
11. Installing and Configuring Fail2ban
12. 6. Limit Network Exposure with Firewalls
13. Using UFW (Uncomplicated Firewall)
14. 7. Test and Apply Your Configuration Safely
15. Summary Security Checklist
16. Frequently Asked Questions
17. Does changing the default SSH port (Port 22) improve security?
18. What should I do if I lose my SSH private key?
19. Can I use RSA keys instead of ED25519?
20. How do I secure the SSH private key stored on my local machine?

Every server connected to the public internet is subject to constant scanning. Within minutes of provisioning a virtual private server or opening port 22 on a home gateway, automated botnets begin testing common usernames and passwords against your Secure Shell (SSH) service. If your server relies on default configurations or weak credentials, it is only a matter of time before an automated script gains access.

Securing SSH access is one of the most critical foundational tasks for system administrators, developers, and self-hosters. Rather than relying on a single defensive layer, securing remote access requires a defense-in-depth strategy. This guide walks through practical, effective steps to harden your SSH daemon, prevent unauthorized login attempts, and reduce the attack surface of your Linux systems.

1. Switch to SSH Key Authentication and Disable Passwords

Password-based authentication is inherently vulnerable to brute-force attacks, dictionary attacks, and credential stuffing. Replacing password logins with cryptographic SSH keys eliminates these vectors entirely.

Generating an ED25519 Key Pair

Modern SSH implementations support several key types. ED25519 is currently recommended for most environments because it offers high security, excellent performance, and immunity to many side-channel attacks compared to older RSA keys.

On your local client machine (not the server), open a terminal and generate a new key pair:

ssh-keygen -t ed25519 -C "user@localdevice"

You will be prompted to choose a file location and enter a passphrase. Always secure your private key with a strong passphrase. The passphrase encrypts the private key on your local disk, ensuring that even if your device is compromised or stolen, your server keys cannot be used immediately.

Deploying the Public Key to Your Server

Transfer your newly created public key to the remote server using the automated utility:

ssh-copy-id user@your-server-ip

Once the key is copied, verify that you can log in without entering your remote system account password:

ssh user@your-server-ip

Disabling Password Authentication in OpenSSH

Confirming key authentication works is vital before completing this step. Once verified, open the main SSH daemon configuration file on your server:

sudo nano /etc/ssh/sshd_config

Locate the following directives (or add them if they are missing) and set their values accordingly:

• PasswordAuthentication no — Disables password prompt logins for all users.
• PubkeyAuthentication yes — Enables SSH public key authentication.
• KbdInteractiveAuthentication no — Disables keyboard-interactive authentication to prevent secondary password challenges.

2. Disable Direct Root Logins

The root user exists on every standard Linux installation. Because the username is universal, attackers direct a massive portion of automated brute-force traffic toward the root account. Disabling direct root login forces administrators to authenticate as an unprivileged user first, creating an auditable log trail and adding another layer of authentication.

Creating a Dedicated Administrative User

If you currently log in directly as root, create a separate user account with administrative (sudo) permissions before altering your SSH settings:

sudo adduser adminuser

Add the user to the appropriate administrative group (usually sudo on Debian/Ubuntu or wheel on RHEL/Fedora):

sudo usermod -aG sudo adminuser

Ensure you copy your SSH public key to this new user’s ~/.ssh/authorized_keys file and verify access before proceeding.

Updating the Configuration

Inside /etc/ssh/sshd_config, modify the root login setting:

PermitRootLogin no

Setting this directive ensures that even if an attacker acquires root credentials through an unexpected vector, they cannot authenticate via SSH directly.

3. Restrict Session Lifespans and Authentication Attempts

Limiting login attempts and timing out idle sessions prevents attackers from maintaining open sockets indefinitely or running continuous brute-force scripts against single connections.

Add or modify the following settings in /etc/ssh/sshd_config:

• MaxAuthTries 3 — Limits the number of failed authentication attempts permitted per connection. If an attacker exceeds this number, the server drops the connection.
• ClientAliveInterval 300 — Instructs the server to send a keepalive message to the client every 300 seconds (5 minutes) if no data has been transmitted.
• ClientAliveCountMax 2 — Drops the connection if two consecutive keepalive probes go unanswered, automatically terminating stale or abandoned sessions after 10 minutes of inactivity.
• LoginGraceTime 30 — Reduces the time window a user has to complete authentication from the default 120 seconds to 30 seconds, freeing up resources faster during automated port-scanning events.

4. Restrict SSH Access to Specific Users or Groups

By default, OpenSSH allows any valid system user with a shell to attempt an SSH login. If your server hosts multiple service accounts or system users, you should explicitly restrict which identities are permitted over remote connections.

To limit SSH access to specific users, append the AllowUsers directive to your configuration:

AllowUsers adminuser developer1

Alternatively, you can restrict access by user group:

AllowGroups sshaccess

Any user account not explicitly listed in these directives will be denied access at the protocol layer, even if they possess valid credentials on the local system.

5. Deploy Automated Intrusion Prevention with Fail2ban

Even after disabling password authentication, brute-force bots will continue to spam your SSH port, consuming system resources and filling authentication logs. Fail2ban monitors log files for repeated authentication failures and dynamically updates host firewall rules to temporarily or permanently block offending IP addresses.

Installing and Configuring Fail2ban

On Debian or Ubuntu systems, install Fail2ban using the package manager:

sudo apt update && sudo apt install fail2ban -y

Copy the default configuration file to create a local override file:

sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

Open /etc/fail2ban/jail.local and configure the SSH protection jail section:

[sshd]
enabled = true
port = ssh
filter = sshd
logpath = /var/log/auth.log
maxretry = 3
findtime = 10m
bantime = 1h

Under this configuration, if an IP address records three failed authentication attempts within a 10-minute window (findtime), Fail2ban blocks traffic from that IP address for one hour (bantime). Enable and start the service to activate enforcement:

sudo systemctl enable --now fail2ban

6. Limit Network Exposure with Firewalls

Exposing port 22 directly to the open internet should be avoided whenever possible. Placing network-level barriers around SSH drastically reduces exposure.

Using UFW (Uncomplicated Firewall)

If your server uses a static IP address for management, configure your firewall to allow SSH traffic exclusively from your trusted IP:

sudo ufw allow from 192.0.2.50 to any port 22 proto tcp

If you operate on a dynamic IP connection, consider placing your administrative access behind a self-hosted VPN (such as WireGuard) or an overlay network (such as Tailscale). When configured this way, you can bind SSH to listen exclusively on your private network interface, rendering port 22 completely invisible to public internet scans.

7. Test and Apply Your Configuration Safely

A simple syntax error in /etc/ssh/sshd_config can prevent the OpenSSH daemon from starting, potentially locking you out of a remote server. Always validate the syntax before applying changes.

Run the built-in configuration test utility:

sudo sshd -t

If the command returns no output, your configuration syntax is valid. Restart the SSH service to apply the changes:

sudo systemctl restart ssh

Crucial step: Do not close your existing SSH terminal session immediately after restarting the service. Keep your current session open and open a new terminal window on your client machine to test logging in under the new rules. If something goes wrong, you can fix the configuration using your active terminal without needing out-of-band recovery tools.

Summary Security Checklist

• Authentication: SSH keys enabled (ED25519), password authentication disabled, keyboard-interactive auth disabled.
• Account Control: Direct root login blocked, access restricted to explicit administrative users or groups.
• Timeouts & Retries: Max authentication retries capped, idle client timeouts enforced.
• Network Security: Fail2ban active, network firewall or VPN access restricting exposed ports.

Frequently Asked Questions

Does changing the default SSH port (Port 22) improve security?

Changing the SSH port to a non-standard port (e.g., 22222) reduces noise in your system logs by evading low-effort, automated script scans targeted specifically at port 22. However, this is security through obscurity. It does not replace core hardening measures like key-based authentication and proper firewall rules, as port scanners will eventually discover open high-numbered ports.

What should I do if I lose my SSH private key?

If you disable password authentication and lose your private key, you will be unable to log in via standard SSH. To regain access, you must use an out-of-band management tool provided by your hosting vendor, such as a web-based serial console, VNC access, or recovery mode. Once inside, you can mount your drive and insert a new public key into your account’s authorized_keys file.

Can I use RSA keys instead of ED25519?

Yes, but you should avoid legacy 1024-bit or 2048-bit RSA keys. If compatibility requirements demand RSA keys, ensure you generate keys with a minimum length of 409 Large key sizes (4096-bit RSA) provide adequate security, but ED25519 keys offer equivalent or superior security with significantly smaller key sizes and better computational efficiency.

How do I secure the SSH private key stored on my local machine?

Always protect local private keys with a strong passphrase during generation. Additionally, restrict file permissions on your local ~/.ssh directory so that only your user account can read or write to them (typically mode 700 for the folder and mode 600 for private key files).