Index

SPF

TL;DR

The Sender Policy Framework (SPF) is a method aimed at reducing spam and fraud for email. SPF allows domain owners to publish a policy about which sender are allowed to send email for that domain. Receivers use SPF as one of the methods for spam detection.

The SPF policy is published as a DNS record by the owner of the domain.

SPF is a basic security mechanism for email. It's easy to set up but it does have some flaws and shortcomings. Additional mechanisms such as DKIM and DMARC complement SPF to allow for better security.

What SPF aims to solve

The goal of SPF is to reduce the amount of spam and phishing emails. SPF allows receivers to detect forged sender addresses in email, which is common for spam, phishing and other fraudulent emails. SPF is defined in RFC7208, which was finalized in 2014.

How email works

To understand how SPF works and why it is needed, you must first have a basic understanding of how email works. If you are already familiar with SMTP, you can skip this section.

Email is transmitted over the internet using the Simple Mail Transfer Protocol (SMTP). SMTP is designed around the principles of paper mail. Like with paper mail, SMTP features the concept of an envelope. The envelope contains the address of the intended receiver and the address of the sender to which the mail can be returned if the mail is undeliverable. Inside the envelope is the letter itself, just like on a formal letter it contains the address of the sender and the receiver. Once the email is delivered to the intended receiver (your mailbox) the envelope is discarded.

Sticking with the paper letter analogy: the postman (called the Mail Transfer Agent (MTA) in SMTP) uses the address on the envelope, and the receiver (your email program) uses the address on the actual letter.

The problem is that (just like with paper mail) anyone can write any sender address on the envelope or the letter. Before SPF, there used to be no way for the MTA (the postman) to determine if the letter actually came from the sender address.

Paper mail Electronic Mail (email)
Postal service SMTP
Envelope SMTP envelope
Postman Mail Transfer Agent (MTA)
Mailbox Email software

What SPF does

With SPF the receiving MTA checks if the computer that sent the email was allowed to use the domain name in the address on the SMTP envelope.

If if turns out the sender was not authorized to use the address, the MTA may mark that email as spam or refuse to deliver it to the receiver. The exact action taken by the receiver upon SPF failure can vary. Most receivers use the SPF result as one of the factors in detecting spam. The receiver may still reject an email that passes SPF for other reasons. Vice-versa the receiver may also still accept email that fail SPF.

The SPF record

A domain owner can publish the SPF policy through the Domain Name System (DNS) by adding a DNS record, which we'll call the SPF record.

The SPF record is a DNS record of type TXT that is placed in the (sub) domain that the SPF policy is to be applied to. The SPF record contains the list of senders that are allowed to send email for the domain the SPF record is placed under. It also contains a policy on how to process email from a sender that does not match the list.

The SPF syntax is as follows:

[modifier]=[value] {prefix}[mechanism]{:value}

A SPF record can have multiple modifiers and mechanisms, they are separated by whitespace. A mechanism has an optional prefix and value. Modifiers always have a value.

A SPF record must start with the v (version) modifier with value spf1. So, a SPF record always starts with the text v=spf1. This allows MTAs to distinguish the SPF TXT record from other TXT records.

A DNS query on a (sub)domain should result in a maximum of 1 TXT record starting with v=spf1. If more than 1 record is found it is not guaranteed which record will be used by the MTA.

Modifiers

Modifiers are used to instruct the receiver on how to use this SPF record. Modifiers always have a value, the modifier van value are separated by an equal sign (=).

Modifier Value Effect
v version The SPF version that this record should be used for. This modifier is mandatory and should always be at the beginning of the record. Currently, the only supported value is spf1
redirect domain name Use the SPF record found at the given domain name for this domain. Setting a redirect value will cause the MTA to ignore any other mechanisms in this record.
exp domain name Use the txt type DNS record found at the given domain as a human-friendly error text if the email fails SPF validation. Rarely used.

Read more about the SPF redirect modifier

Prefix

The mechanism prefix is optional and instructs the receiver what to do with the sender if it matches the mechanism. If not prefix is set, the + (pass) prefix is assumed.

Prefix Effect
+ A sender matching this mechanism should pass SPF validation
- A sender matching this mechanism should fail SPF validation
~ A sender matching this mechanism should soft-fail SPF validation
? The sender matching this mechanism should be treated as neutral

Mechanism

The mechanism that is used to match the sender IP address. Most mechanisms have optional values.

The mechanism and value are separated by a colon (:), if no value is used the colon should be omitted.

Mechanism Value Effect
mx domain name (optional) Match any sender that the DNS mx record resolves to for given domain. If no domain is specified, the current domain is used
a domain name (optional) Match any sender that the DNS A or AAAA record resolves to for given domain. If no domain is specified, the current domain is used
ip4 address Match any sender that matches this IP address, the value may be one IP address, or an IP range in CIDR notation
ip6 address Match any sender that matches this IPv6 address, the value may be one IPv6 address, or an IP range in CIDR notation
ptr domain name (optional) Match any sender whose reverse-DNS lookup results in this domain name. If no domain is specified, the current domain is used
include domain name Match any sender found in the SPF record found in the specified domain name.
exists macro Expand the given macro (see section 'marcos' below) perform a DNS query on the resulting value. If the query yields a result the sender matches the mechanism.
all Match any server

The SPF record is evaluated from left to right, the first mechanism that matches is used. It is therefore common to end the SPF record with a -all or ~all mechanism to reject any sender that does not match any of the preceding mechanisms.

If no prefix is set, the default prefix + is used, so:

v=spf1 mx a -all

Is the same as:

v=spf1 +mx +a -all

Fail vs Soft-fail

The difference between the fail (-) and softfail (~) prefixes are often poorly understood. The softfail prefix is used for situations where the domain owner is not ready to commit to using a fail prefix.

Most receivers will still accept email from senders that match a softfail mechanism, but it may apply stricter spam detection or display a warning to the receiver about this.

When first setting up SPF, it is recommended to use softfail, so you don't accidentally block senders that you forgot to add to the SPF record. If you look at any SPF implementation guide (such as that of Gmail of Mailchimp) you'll notice that they always recommend softfail, because advising a fail prefix may cause email to be blocked unwittingly.

With an email monitoring service (such as offered by Mailhardener) you can create an inventory of senders that you want to allow to send email for your domain. Once you are confident you have added all legitimate senders to the SPF record, you can switch to -all mechanism to block any unauthorised senders.

Examples

Allow email from this domain's mail exchange (MX), reject all other senders:

v=spf1 mx -all

Allow email from this domain's mail exchange or any server with an IP address in range 10.0.0.1 - 10.0.0.255 and reject all other senders:

v=spf1 mx ip4:10.0.0.1/24 -all

Allow email from this a sender whose IP address matches an address found in the domain DNS A record, or any server that matches the SPF record found at _spf.google.com. Soft-fail all others.

v=spf1 a include:_spf.google.com ~all

Deny any email for this domain (no email is expected to be send by this domain):

v=spf1 -all

You can use our SPF validator to inspect the SPF settings for your domain.

Issues

SPF was one the early attempts at reducing spam and fraud. Since its introduction it became apparent that it has some flaws and shortcomings. You should still use SPF, but you shouldn't rely on it solely. We recommend using SPF in combination with DKIM for a better protection against fraud.

SPF can be bypassed

We explained that an email has a sender address on both the envelope and the message itself. The address on the envelope and the letter itself can be different, even from different domains. The MTA (postman) uses the address on the envelope, but your email program discards the envelope and displays the sender address that is on the letter.

SPF is checked by the MTA, not your email program. Thus SPF is only applied on the address on the envelope. A fraudulent sender can simply use a sender address of a domain that they control on the envelope and use a different domain as the sender address on the letter. Because they control the domain on the envelope sender address, they can publish an SPF record allowing themselves to send email for that domain.

Some email services (most notably Gmail) will display a notification in their web interface if the domain in the sender address differs between the envelope and the letter. It will add via <server> behind the sender address.

Receiving MTAs may also apply stricter spam checking on emails where the address on the envelope and the letter differ.

SPF can break forwarding

SPF can break certain forms of email forwarding. We must be clear here that this only affects automated forwarding, not manually forwarded emails by users.

On an email client, if you forward an email it will be wrapped in a new envelope. This is because your email client has already discarded the original envelope. The receiver will see the address of the forwarder as the sender, it is clear that the message inside is forwarded by that person. SPF will not affect this.

However, some mail services offer automated forwarding, where email is forwarded while retaining the original envelope, and thus the sender address. This may be used when a domain name is no longer in use and all email must be forwarded to a new domain name. This scenario is common for businesses that re-brand to a new name. The original sender address is retained so that the receiver sees the name of the real sender, and can use the reply button to answer the email. The issue here is that this server is now sending email for a different domain (the one in the sender address on the envelope). This server is most likely not allowed to send email for this domain by the SPF policy.

That is why it is recommended to complement SPF with DKIM, as DKIM signatures are stored in the email content (the letter) and not the envelope. So a DKIM signature will remain intact when the email is forwarded.

The sender can be spoofed

A sending MTA may conceal its real IP address with a different IP address. This is known as spoofing. By spoofing its IP address a sender may trick the receiver into believing that the email came from a legitimate sender.

SPF on shared services

If you use a common mail service (such as Office365 or Gmail) for your domain, you'll have to add that service to your SPF record (using the include mechanism). The problem is that this would allow any other user of that service to send email using your domain name. In practice all large email providers won't allow a users logged in as domain_a.com to send email for domain_b.com, but that is something you'll need to trust the service for.

In other words: by adding a shared email service to your SPF, you must trust this service to never let any of their other customers send email using your domain name.

Recommendations

Despite the shortcomings described above, it is still recommended to use SPF, but is also recommended to use additional mechanisms such as DKIM and DMARC.

Even if a domain is not used to send email, it is recommended to set up a SPF record with a single -all mechanism to indicate to receivers that they should not accept email from this domain.

When first setting up SPF, you should apply softfail (indicated by ~all) and use a monitoring service such as our own to make an inventory of all your legitimate senders and check if they pass SPF validation the receiving MTAs. Once you are confident that all senders are added to your SPF policy, you can switch to a stricter fail (-all) setting to start blocking unauthorized senders.

Further reading

Tools