Deploying a Secure Prosody XMPP Server on Docker with SSL & Firewall

Why XMPP for Secure Communications?

Signal encrypts your message contents. It also knows, to the second, who spoke to whom. That distinction is the whole argument for self-hosting: end-to-end encryption protects the payload, but the social graph, the timing, and the retention policy belong to whoever runs the server. On Slack, Teams or WhatsApp, that is not you.

XMPP has been around since 1999 — old enough that people assume it died, standardised enough that it did not. It is the only widely deployed open federation protocol for real-time messaging where you can run the server yourself, on your own hardware, and still talk to the outside world. Run your own and you own the account database, the routing, the archive, and the retention window. Nobody reads it unless you let them.

The honest cost: you are now the operator. Certificate expiry is your outage. Spam from open federation is your problem. Nobody else is patching this at 3 a.m. If your threat model is “I want a group chat that works”, use Signal and stop reading. If it is “no third party should hold my organisation’s metadata”, keep going.

Why Prosody?

ejabberd is the one you pick if you are running a telco-scale deployment across a cluster and have Erlang people on staff. MongooseIM is the same bet with a commercial support contract attached. Openfire is Java, which means a JVM to tune and a heap to size.

Prosody is Lua. A modest deployment idles at tens of megabytes of RAM, and the entire config file is a readable Lua table you can diff in Git. That matters more than the benchmark numbers: you can actually hold the running configuration in your head, and a configuration nobody understands is a configuration nobody audits.

The module system is where the security argument lives. Prosody ships almost nothing enabled by default and you add what you need, so the attack surface is the list you wrote rather than whatever the distribution maintainer decided was reasonable. The trade-off is real and worth stating — the features you would expect to be present are not. No message archive until you enable mam. No group chat until you enable muc. Mobile clients drain batteries until you enable csi_simple. Expect to spend an evening working out which modules you actually needed, usually by way of a user telling you their history vanished.

Why Docker for This?

Containerising Prosody buys three things. The process runs isolated from the host, so an application-level compromise does not immediately own the box. The deployment is reproducible — the compose file plus the mounted config directory is the server, and you can rebuild it on different hardware in minutes. And upgrades become a pull and a restart rather than a fight with distribution package versions.

What it costs you: certificate paths and file ownership become fiddly in ways they are not on a bare-metal install. Certbot renews into a host directory; the container reads a bind mount; if the UID inside the container cannot read privkey.pem, Prosody starts, logs a warning, and serves without the certificate you thought you deployed. That failure is quiet. Check the log after every renewal cycle for the first few months, or you will discover it when a client refuses to connect.

Infrastructure Requirements

XMPP connections are long-lived and mostly idle, so the binding constraint is rarely CPU. It is memory per connected session and, once you enable message archiving, disk. Size for the archive, not the chat:

User Capacity	CPU	RAM	Storage
Small (up to 100 users)	1 vCPU	1 GB	10 GB SSD
Medium (up to 500 users)	2 vCPU	2 GB	20 GB SSD
Large (1000+ users)	4 vCPU	4 GB	50 GB SSD

These assume text messaging with mam on. File transfer changes the arithmetic entirely — HTTP upload stores attachments on your disk until an expiry job removes them, and a hundred users sharing photographs will fill 10 GB faster than a year of chat logs will. If you enable uploads, set a size cap and an expiry policy on day one rather than after the disk fills and Prosody stops accepting writes.

Step-by-Step Deployment

1. Pick Your OS

Debian 12 (Bookworm) or newer. The reasoning is boring and correct: long support window, security updates you can apply unattended, and a minimal install that ships almost nothing you have to then remove. Since the application runs in a container, the host distribution is doing one job — running Docker and renewing certificates — and you want it to be the least interesting part of the stack.

2. Harden the Firewall First

Do this before Docker is installed, not after. Once the daemon is running it writes its own rules into the DOCKER iptables chain, which is consulted before UFW’s — so a published port can be reachable from the internet even though ufw status swears it is blocked. Firewalling first means you at least know what the baseline was.

apt update && apt upgrade -y
apt install ufw
ufw default deny incoming
ufw default allow outgoing
ufw allow ssh  # Ideally restrict this to your management IP
ufw allow 5222/tcp  # Client-to-server connections
ufw allow 5269/tcp  # Server-to-server federation
ufw allow 80,443/tcp  # Required for Let's Encrypt certificate issuance
ufw enable

Default-deny inbound means nothing reaches the host unless it is on that list. Two notes on the list itself. Port 5269 is server-to-server federation — if this instance only ever talks to its own users, leave it closed and you have removed the entire open-federation spam problem in one line. And ports 80/443 are only needed for certificate issuance and, later, HTTP file upload; if you use DNS-01 challenges instead of --standalone, port 80 never needs to be open at all.

Verify from outside the host, not from the host. nmap from another machine tells you what the internet sees; ufw status tells you what UFW believes, and thanks to the Docker chain those are not always the same thing.

3. Install Docker

Debian’s own packages are enough here, and they keep you inside the distribution’s security update stream:

apt install -y docker.io docker-compose
systemctl enable --now docker

4. Write the Docker Compose File

The rule for this file: anything you would be upset to lose lives on the host, not in the container’s writable layer. That means the account database (/var/lib/prosody), the configuration (/etc/prosody) and the certificates. Everything else should be disposable.

version: '3.8'
services:
  prosody:
    image: prosody/prosody:latest
    container_name: prosody
    restart: unless-stopped
    ports:
      - "5222:5222"
      - "5269:5269"
    volumes:
      - ./data:/var/lib/prosody
      - ./config:/etc/prosody
      - ./certs:/etc/letsencrypt:ro
    environment:
      - XMPP_DOMAIN=example.com
    security_opt:
      - no-new-privileges:true

no-new-privileges stops any process in the container gaining privileges through a setuid binary — cheap, and it costs nothing here because Prosody never needs to escalate.

Two things worth changing before this goes anywhere real. image: prosody/prosody:latest means your next docker-compose pull silently changes the software version, which is fine for a lab and unhelpful when you are trying to work out what broke; pin a version tag and update deliberately. And the certificate mount is :ro, which is correct — but the certificates it points at are renewed by certbot on the host, so the container has to be restarted afterwards to reload them. That restart is the entire reason for the cron line further down.

5. Get Your TLS Certificate

XMPP without TLS is a plaintext protocol carrying credentials. Let’s Encrypt removes any excuse:

apt install certbot
certbot certonly --standalone -d example.com

Then point Prosody at those certificates in config/prosody.cfg.lua:

ssl = {
    key = "/etc/letsencrypt/live/example.com/privkey.pem";
    certificate = "/etc/letsencrypt/live/example.com/fullchain.pem";
    ciphers = "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384";
}

An explicit cipher list stops a client negotiating something weak. It also ages badly: a list written today will, in three years, be the thing forcing an obsolete suite on a server whose defaults had long since improved. Hand-maintained cipher strings are how servers end up stuck on TLS configurations nobody has reviewed since the day they were pasted in. Setting a protocol floor and letting the library choose the suite — the approach in the hardening section below — is the version that stays correct without maintenance.

One detail the certbot certonly --standalone command hides: it binds port 80 itself, so it will fail if anything else is listening there. On renewal that is usually a reverse proxy you added later, and the failure mode is a certificate that quietly stops renewing while everything appears to work — right up to the expiry date, when every client disconnects at once.

6. Start the Server

docker-compose up -d

Then read the logs properly, because Prosody starts successfully in several states you do not want. A certificate it cannot read produces a warning, not a failure. A module name it does not recognise produces a warning, not a failure. Both leave you with a running server that is missing the thing you just configured:

docker logs -f prosody

Managing Users

Adding Accounts

With allow_registration = false set (see below), accounts only exist because you made them. prosodyctl runs inside the container:

docker exec -it prosody prosodyctl adduser user@example.com

Day-to-Day Admin

• Remove a user: docker exec -it prosody prosodyctl deluser user@example.com
• Change a password: docker exec -it prosody prosodyctl passwd user@example.com

Manual provisioning is the trade-off you accepted when you closed registration. It is fine for a team of twenty and miserable at two hundred, which is the point where you move authentication to LDAP or PostgreSQL and stop treating account creation as an SSH task. Note also that deluser removes the account, not necessarily every trace of it — archived messages in other users’ mam stores are their data, on their side of the conversation, and no server-side deletion reaches them.

Choosing a Client

This is where most XMPP deployments actually fail. The server is fine; the client experience is uneven, and users who find messaging unpleasant go back to WhatsApp regardless of your data sovereignty argument. Pick clients that support OMEMO — the Signal-protocol-derived end-to-end encryption scheme for XMPP — and check they support it in group chats, which is where support has historically been patchiest.

• Desktop (Windows/macOS/Linux): Gajim or Dino are both solid choices
• Android: Conversations is the go-to
• iOS: Monal is actively maintained and works well

Connect with the full Jabber ID (user@example.com) and confirm OMEMO is on before anything sensitive goes over the wire. Two habits worth building into whatever onboarding note you write for users. First, OMEMO is per-conversation and per-device — adding a new phone means the old devices must trust its key, and until they do, messages sent to it are simply not readable. Second, encrypted history does not sync to a device that was not in the conversation, so “I set up my laptop and my chats are empty” is expected behaviour rather than a fault. If you skip explaining that, you will explain it individually, repeatedly, over the following month.

Keeping It Running

Auto-Renew Certificates

Let’s Encrypt certificates last 90 days, and an expired certificate on an XMPP server is a total outage — every client disconnects, and federated servers refuse to talk to you. Automate the renewal:

crontab -e

Add this line:

0 0 * * * certbot renew --quiet && docker restart prosody

The --quiet flag is the risk in that line. It suppresses output, cron discards it, and a renewal that has been failing for weeks looks exactly like one that has been succeeding. Point the job’s output somewhere you will see it, or run an external check that alerts on the certificate’s remaining validity — the only signal that actually reflects reality. Monitoring the cron job tells you it ran; monitoring the expiry date tells you it worked.

Staying Up to Date

Pull new images regularly for security fixes — and back up ./data and ./config before you do, because a pull that changes major version can migrate the account store in ways that are not trivially reversible:

docker-compose pull && docker-compose up -d

Watch Your Logs

Two patterns are worth knowing by sight. Repeated authentication failures against one account are a brute-force attempt; repeated failures spread thinly across many accounts are password spraying, and they are much easier to miss because no single account looks interesting. The other is a burst of inbound s2s connections from domains you have never federated with — that is spam infrastructure finding you, usually within days of your DNS records going live.

Tailing logs by hand does not scale past the first fortnight. Ship them somewhere that can alert:

docker logs --tail 100 prosody

Advanced Configuration

Scaling

Two levers, in order of how far they get you. Adding CPU and RAM handles more concurrent sessions and is the right answer for a long time — XMPP is cheap per connection, and a single modest instance serves far more users than people expect.

The one that matters structurally is moving storage from Prosody’s default flat files to PostgreSQL. Flat-file storage is fine until the archive grows, at which point search and expiry queries get slow, backups mean copying thousands of small files, and there is no sane way to run two Prosody instances against the same data. Postgres fixes all three. It also adds a database to run, back up, and patch — you have traded a simple thing that stops scaling for a complex thing that does not, which is the usual bargain and worth making only when you can name the limit you are hitting.

Useful Modules

These four are close to mandatory in practice. Without smacks, a phone that switches from Wi-Fi to mobile data loses messages sent during the gap and nobody is told. Without csi_simple, presence noise wakes mobile clients constantly and users report the app as a battery hog. mam is what makes history exist at all across devices — and is also the module that makes your server a retention liability, so decide the archive expiry policy at the same time you enable it:

modules_enabled = {
    "muc";        -- Multi-user chat rooms
    "mam";        -- Message archive management (local message history)
    "csi_simple"; -- Client state indication (better mobile battery life)
    "smacks";     -- Stream management (prevents message loss on flaky connections)
}

Mandate Encryption and Shut Down Spam

OMEMO is a client decision, and any user can turn it off. The server’s job is the layer underneath: guaranteeing that no connection, client or federated, is ever carried in the clear regardless of what the client chose. These four lines do most of the work:

-- Force TLS on every connection; refuse anything unencrypted
c2s_require_encryption = true   -- clients must use TLS
s2s_require_encryption = true   -- federated servers must use TLS
s2s_secure_auth = true          -- verify remote server certificates (blocks spoofed federation)

-- Modern TLS floor — drop the old cipher-list approach and let Prosody negotiate securely
ssl = {
    protocol = "tlsv1_2+";      -- TLS 1.2 and 1.3 only; no 1.0/1.1
}

-- Kill open registration: the #1 source of XMPP spam accounts
allow_registration = false      -- create users manually with prosodyctl instead

s2s_secure_auth = true is the one with a visible cost. It requires remote servers to present a certificate that actually validates, and a portion of the federated XMPP network — small servers, hobbyist deployments, anything with a self-signed certificate — will simply stop being reachable. That is the correct trade for most organisations, but expect at least one user to report that they can no longer message a contact on some obscure domain, and understand that this is the setting doing its job rather than a fault.

Open federation is the other half of the problem. Any server on the planet can route messages to your users, which is exactly the property that makes XMPP worth running and exactly the property spammers exploit. XMPP spam (“spim”) is not theoretical — a new domain typically starts receiving it within weeks. Two defences matter:

• mod_firewall — Prosody’s scriptable rule engine, and the single most effective anti-abuse module available. It can rate-limit, drop messages from senders not in a user’s roster, and match known spam patterns before anything reaches an inbox. The rule most worth writing first is also the bluntest: silently discard messages from strangers on domains you have never federated with. Note the failure mode you are accepting — a legitimate first contact from a new domain gets dropped with no bounce, and neither party learns why.
• Blocklists — subscribe to a community-maintained bad-server list (via mod_firewall or mod_s2s_blacklist) so abusive instances are defederated automatically instead of one user report at a time. The cost is that you have delegated part of your federation policy to whoever maintains that list.

If the instance is private, stop trying to filter and use an explicit federation allowlist instead. A list of the handful of domains you actually talk to is trivially auditable and ends the spam problem outright, at the price of a manual change every time that list needs to grow.

Admin Interface

The Prosody Web Admin module gives you a browser-based panel. If you enable it, put it behind an Nginx reverse proxy with basic auth and IP allowlisting, and never bind it to a public interface.

The better question is whether you need it at all. Everything it does, prosodyctl does over SSH — a channel you have already hardened, already log, and already control access to. A web admin panel is a second authentication surface guarding the same account database, added for convenience. On a server whose entire justification is minimising who can reach your data, that is a poor trade unless someone genuinely needs to administer it without a shell.

Wrapping Up

An afternoon gets this running. What determines whether it is still running in a year is smaller and less interesting: whether certificate renewal is monitored by expiry date rather than by cron exit status, whether the archive has a retention policy, and whether someone other than you can rebuild it from the compose file and a backup.

Be clear-eyed about what you have bought. Self-hosting does not make messages more encrypted than Signal’s — OMEMO and the Signal protocol are close relatives. What it changes is who holds the metadata: who talked to whom, when, how often, and for how long that record survives. That is the one thing no third-party platform will ever give you, and it is the entire reason to accept being the person responsible at 3 a.m.