How to Set Up Your Own Mail Server on a VPS

To set up a mail server on a VPS you need three things: a mail transfer agent (MTA) like Postfix for sending, an IMAP server like Dovecot for receiving, and the correct DNS records so other servers trust your domain. The full process takes 60 to 90 minutes the first time.

This guide walks you through every step on Ubuntu/Debian, with each decision explained so you understand what you're doing and can adapt it to your setup.

Before You Start: Requirements and Warnings

Running your own mail server gives you full control, but also full responsibility for deliverability, reputation, and maintenance. Check these points before proceeding:

• Clean IP — verify your VPS IP isn't on any blacklists using MXToolbox. Many providers assign IPs with a spam history.
• Port 25 unblocked — some providers block port 25 by default. Contact support to enable it before installing anything.
• PTR (rDNS) — the reverse DNS record for your IP must resolve to your server's hostname. Configure this in your VPS provider's panel.
• Your own domain — you need DNS panel access to create MX, SPF, DKIM, and DMARC records.

Installing Postfix and Dovecot

Update the system and install the main packages:

apt update && apt upgrade -y
apt install -y postfix dovecot-core dovecot-imapd dovecot-pop3d \
  mailutils certbot

During the Postfix installation wizard, choose Internet Site and enter your domain name (e.g. yourdomain.com).

Set the Server Hostname

The server hostname must match the MX record you'll create shortly:

hostnamectl set-hostname mail.yourdomain.com

Make sure /etc/hosts includes the local resolution:

127.0.0.1   localhost
YOUR_VPS_IP   mail.yourdomain.com mail

Configure DNS Records

This step has the biggest impact on deliverability. Add these records in your domain's DNS panel:

You'll generate the DKIM record after installing the signing software, so it's not listed here yet.

Configure Postfix for Sending and Receiving

Edit the main Postfix configuration file:

nano /etc/postfix/main.cf

Replace or adjust the key lines:

myhostname = mail.yourdomain.com
mydomain = yourdomain.com
myorigin = $mydomain
inet_interfaces = all
mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain
home_mailbox = Maildir/
smtpd_tls_cert_file = /etc/letsencrypt/live/mail.yourdomain.com/fullchain.pem
smtpd_tls_key_file  = /etc/letsencrypt/live/mail.yourdomain.com/privkey.pem
smtpd_use_tls = yes
smtpd_tls_security_level = may
smtp_tls_security_level = may

Before restarting Postfix, get a TLS certificate with Certbot:

certbot certonly --standalone -d mail.yourdomain.com

Then restart the service:

systemctl restart postfix

Configure Dovecot for IMAP

Dovecot handles IMAP connections from your email clients (Outlook, Thunderbird, etc.). Edit the auth config:

nano /etc/dovecot/conf.d/10-auth.conf

Make sure these lines read:

disable_plaintext_auth = yes
auth_mechanisms = plain login

Set the mailbox path to match Postfix:

nano /etc/dovecot/conf.d/10-mail.conf

mail_location = maildir:~/Maildir

Point Dovecot to the TLS certificate:

nano /etc/dovecot/conf.d/10-ssl.conf

ssl = required
ssl_cert = </etc/letsencrypt/live/mail.yourdomain.com/fullchain.pem
ssl_key  = </etc/letsencrypt/live/mail.yourdomain.com/privkey.pem

Restart Dovecot:

systemctl restart dovecot

Create Email Accounts and Test

In a basic system-user–based mail server, any Linux user with a valid password can receive email. Create a user:

adduser john

To test sending from the command line:

echo "Mail test" | mail -s "Test" [email protected]

Check the log to confirm it went out without errors:

tail -f /var/log/mail.log

To verify receiving, configure an IMAP client with these settings:

• IMAP server: mail.yourdomain.com, port 993, SSL/TLS
• SMTP server: mail.yourdomain.com, port 587, STARTTLS
• Username: the Linux username (e.g. john)
• Password: the Linux user's password

For a broader look at VPS options and how to pick the right plan for hosting services like this, check out our complete VPS server guide.

If you'd rather focus on your business and delegate the technical administration, elenlace.com offers managed VPS plans with expert support for mail server setup and beyond.

Key Takeaways

• Postfix handles outgoing mail (MTA) and Dovecot handles incoming IMAP — you need both.
• DNS records (MX, SPF, DKIM, DMARC) matter as much as the software; without them your emails will land in spam.
• Check that port 25 is open and your VPS IP is clean before installing anything.
• A TLS certificate (Let's Encrypt) is required for email clients to accept the connection.
• In a basic setup, Linux system users with valid passwords serve as email accounts.

FAQ

Is running your own mail server complicated to maintain?

More so than hosting a website, yes. You need to manage your IP reputation, renew certificates, update software, and monitor logs. For a business with many accounts or high email volume, a managed service like Google Workspace or Zoho Mail alongside your VPS web server is often the more practical choice.

Why are my emails going to spam?

The most common causes are: IP on a blacklist, missing SPF or DKIM records, no rDNS configured, or an inconsistent server hostname. Check all of these in MXToolbox before looking for another cause.

What's the difference between Postfix and Exim?

Both are solid MTAs. Postfix is more common on modern Ubuntu/Debian servers and has a clearer modular configuration. Exim is the default MTA on cPanel/WHM. If you have no prior preference, Postfix is the better-documented and more widely supported option in 2025.

Do I need to open firewall ports?

Yes. Open ports 25 (server-to-server SMTP), 587 (authenticated client SMTP), 993 (IMAPS), and 465 (SMTPS) in your firewall. With UFW: ufw allow 25,587,993,465/tcp. Port 25 also needs to be enabled by your VPS provider.

Prefer it done for you? El Enlace handles hosting and professional web development.

Compare providers

Other providers and guides worth comparing:

• GoDaddy
• HostGator México
• Hostinger