Why Emails Go to Spam: A Guide to SPF, DKIM and DMARC

"Why do emails go to spam?" The most common and fundamental answer is that receiving servers (Gmail, Outlook, Yahoo, etc.) cannot verify your sender identity. The SMTP protocol, which forms the foundation of email infrastructure, lacks built-in authentication mechanisms; anyone can try to send a message on behalf of any email address. To close this vulnerability and block spammers, major email providers now require SPF, DKIM, and DMARC DNS records. If these records are missing or misconfigured, your messages will land in the spam folder or be rejected outright by the server, no matter how clean and legitimate they are.

In this post, we will explore the steps you need to take as a server administrator to maximize email deliverability, the technical details of DNS configurations, and common mistakes. We will also see how you can automate these processes when setting up a modern email server.

Why is Email Authentication So Important?

In the past, email servers evaluated incoming messages purely by checking if the IP address was on a blacklist and if the content contained spam keywords. However, with the rise of phishing attacks, the focus has shifted to authentication.

Major providers want to know if the sending domain actually authorized that server, in order to protect their users. If your domain is yourcompany.com, Gmail's servers ask three questions:

1. Is this IP address authorized to send emails on behalf of yourcompany.com? (SPF)
2. Has this message been altered in transit, and did it really originate from the yourcompany.com server? (DKIM)
3. If either of the first two checks fails, what should I do with this message? (DMARC)

When this triad is not fully established, your reputation drops, and your emails go to spam.

What is SPF (Sender Policy Framework) and How to Set It Up?

SPF is a TXT record held in DNS that specifies which IP addresses or servers are allowed to send emails on behalf of your domain. When receiving a message, the destination server looks at the Envelope From address and queries that domain's SPF record. If the sending IP address is not in this list, the SPF check fails.

A simple SPF record looks like this:

v=spf1 mx a ip4:192.0.2.50 ~all

What this record means:

• v=spf1: This is an SPF record.
• mx: Servers listed in the domain's MX records can send email.
• a: The IP address in the domain's A record can send email.
• ip4:192.0.2.50: This specific IPv4 address can send email.
• ~all: (Soft fail) Accept messages from unlisted IPs but mark them as suspicious (spam). If you use -all (Hard fail), messages from unlisted IPs are rejected outright.

Common SPF Mistakes

• Multiple SPF Records: A domain can only have one TXT record starting with v=spf1. If you use Google Workspace and also send notifications from your web server, you must combine them into a single record (v=spf1 include:_spf.google.com ip4:192.0.2.50 ~all) instead of creating two separate ones.
• Exceeding the DNS Lookup Limit: When the SPF mechanism points to other domains using directives like include, a, or mx, background DNS queries run. The total number of these queries cannot exceed 10. If it does, the SPF validation returns a permanent error (PermError).

Signing Messages with DKIM (DomainKeys Identified Mail)

SPF only verifies the IP address. However, if you are hosted on a shared server or your messages pass through other servers (forwarding), SPF alone is insufficient. This is where DKIM comes into play.

DKIM adds a cryptographic digital signature to the headers and body of your emails. This system works on the principle of asymmetric encryption:

1. A private key is generated on your server (for example, in Postfix).
2. The public key, which is the pair to this private key, is published as a TXT record in DNS.
3. Every email leaving your server is signed with the private key.
4. The receiving server fetches the public key from DNS and verifies the signature. If the signature is valid, it proves that the message was not altered in transit and genuinely originated from your server.

A DKIM DNS record usually looks like this (example selector: default):

default._domainkey.yourcompany.com IN TXT "v=DKIM1; k=rsa; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBC..."

Because DKIM configuration requires key management on the server side, setting it up using traditional methods can be cumbersome. Rotating keys periodically (e.g., once a year) is recommended for security.

DMARC (Domain-based Message Authentication, Reporting, and Conformance)

DMARC is the ultimate authority over SPF and DKIM. It gives the receiving server the following instruction: "If one or both of the SPF and DKIM checks fail, what should I do with this message?"

Additionally, DMARC performs another critical check called "Alignment". Malicious actors can use a valid domain they own as the Envelope From, while writing [email protected] in the "From" header of the email. In this case, even if SPF and DKIM pass, the user sees a fake sender. DMARC requires the Envelope From and the "From" header address to match.

A basic DMARC record is added as a TXT record to _dmarc.yourcompany.com:

v=DMARC1; p=quarantine; rua=mailto:[email protected];

• p=none: Monitoring mode only. Failing messages are delivered normally, but reports are sent to you. This mode is recommended for initial setup.
• p=quarantine: Send failing messages to the spam folder.
• p=reject: Reject failing messages at the server gate. This is the most secure mode, but you should not switch to it until you are 100% sure of your SPF and DKIM configuration.
• rua: The email address where receiving servers will send daily XML reports. By reviewing these reports, you can see who is using your domain.

Why is a PTR (Reverse DNS) Record Critical?

In DNS records, we usually point a domain name to an IP address (A record). Reverse DNS (PTR) translates an IP address back into a domain name. Receiving servers query the PTR record of the IP address connecting to them and expect the returned hostname to match the HELO/EHLO name provided during the connection.

If your server lacks a PTR record or it is left as a default cloud provider name (like ec2-198-51-100-2.compute.amazonaws.com), many strict email providers will instantly send your messages to spam or reject them.

You do not set the PTR record from your domain's DNS panel (Cloudflare, Route 53, etc.). It is assigned to that IP address from the control panel of the data center or cloud provider where you rent the server (Hetzner, DigitalOcean, etc.).

Improving Email Deliverability with bipanel

Hosting email on your own server requires managing complex components like spam filters, virus scanning, and queue management alongside the protocols mentioned above. With traditional hosting panels, you might have to configure these settings manually or buy extra software.

bipanel provides a fully-fledged email infrastructure for the accounts you host on your own dedicated server or VPS (Ubuntu, Debian, AlmaLinux, Rocky Linux, or RHEL). Built with Postfix, Dovecot, and Rspamd, this infrastructure comes with the following features by default to improve deliverability:

Automatic SPF, DKIM, and DMARC Management

When you add a domain or manage a DNS zone in bipanel, the system automatically generates the most appropriate SPF, DKIM, and DMARC records for you. DKIM keys are securely generated on the server side and added to your DNS zone. If you host your DNS with an external provider like Cloudflare, bipanel's cloud sync feature pushes these records to the provider instantly.

Sender Identity and Delivery Protection

The quality of outgoing emails directly affects your server's IP reputation. bipanel performs SPF, DKIM, and DMARC checks on outgoing messages and protects sender identity. Additionally:

• Hourly Sending Limits: By setting hourly email sending limits per account or per domain, you prevent your server from turning into a spam hub in the event of a password leak.
• Two-Way Filtering with Rspamd: Not only incoming but also outgoing messages pass through the spam filter and ClamAV virus scanner. Suspicious messages are stopped before they leave.
• Delivery Tracking and Queue Management: You can see exactly where your message went, why it bounced, or why it is waiting in the queue directly from the panel. System admins receive alerts for high bounce rates.

SMTP Relay (Smarthost) and Per-Domain Routing

If the IP block of the data center where your server is located has a poor reputation (common with some cheap VPS providers), you might want to route your emails through professional SMTP providers like Amazon SES, SendGrid, or Mailgun instead of sending them directly from your server.

The bipanel Community edition includes server-wide SMTP relay (smarthost) support. The Pro edition takes this a step further, allowing you to define a Per-Domain SMTP Relay. This way, while emails for some customers on the server go out through the public network, you can route emails for mission-critical customers through a highly reputable provider assigned specifically to them.

(Note: Docker and Railway deployment modes have no mail server, authoritative DNS server, or FTP. To use email services, you need to install bipanel directly on a clean Linux server.)

Conclusion and Next Steps

Preventing your emails from going to spam is not a matter of luck; it is a matter of proper configuration. You must authorize your IP addresses with SPF, sign your messages with DKIM, define your policies with DMARC, and ensure your PTR record is accurate. Monitoring your server's outbound traffic is also vital to stop compromised accounts from sending spam.

Managing all these processes manually can take hours and increases the risk of making mistakes. If you want to move to a modern, secure, and performant infrastructure, you can install bipanel on a clean Ubuntu 24.04 LTS (or your preferred supported operating system) with a single command.

To try the Community edition, which is completely free for up to 3 hosting accounts, visit the Download bipanel page. For more accounts, incremental remote backups, multi-level resellers, and the AI assistant, you can explore our Pro plans on the Pricing page.