Overview

5 common mistakes with SPF

At Mailhardener, we analyse many domains for email security on a daily basis. This puts us in a unique position to list the most commonly made mistakes.

Here are the 5 most common mistakes with SPF.

Update: we now have an SPF validator tool available that tests every aspect of your SPF record, including the 5 mistakes described in this article.

Mistake 1: Multiple SPF records

A domain can have no more than one SPF record. A DNS query of type TXT should result in 1 TXT record that starts with v=spf1.

If a DNS query results in more than one SPF record, there will be no guarantee which one will be used. As a result: it may happen that sometimes your mail is not delivered, depending on which record is used by the receiving email service. This situation can be really tricky to track down.

If a domain is using SPF, any new SPF term should be added to (merged into) the existing SPF record, rather than adding another SPF record.

The Mailhardener SPF validator can be used to verify that there is no more than one SPF record for a domain.

Alternatively, the dig command line tool can also be used:

dig example.com TXT +short

The query should not yield more than one DNS resource record that starts with v=spf1

You can use the Mailhardener SPF merge guide to learn how to merge SPF records.

Mistake 2: Quoted TXT record

The content of a DNS TXT type record is often displayed with double quotes, but those quotes are not (or should not be) part of the actual DNS record content.

The quotes are there for display purposes only. Since a TXT type DNS record is allowed to contain leading/trailing whitespace characters, the quotes help to distinguish the start and end of the record content.

Hence, it is common for email services to show setup instruction with quotes in the examples, but you should not include those quotes in the actual DNS record.

The SPF standard states that the SPF record must begin with v=spf1, so if your record starts with "v=spf1 (note the quote) it will not be recognized.

To make matters worse, some DNS service providers require you to add the quotes when pasting, and some don't. This leads to confusion.

Mailhardener tip:

When creating SPF (or any other email hardening related DNS record), first try to paste the record contents without the quotes. If the DNS service provider does not accept the value without quotes, only then add the quotes.

Once the record is created, use the free Mailhardener SPF validator to verify your SPF record is correctly set up.

You can also verify your SPF record with the dig command line tool. But please note that:

dig example1.com TXT
> example1.com. 21599 IN TXT "v=spf1 mx -all"      <-- correct

dig example2.com TXT
> example2.com. 21599 IN TXT "\"v=spf1 mx -all\""  <-- wrong!

In the example above, the SPF record of the first example is correct (the quotes you see there are added by dig for readability). The second result is incorrect, here there are quotes in the TXT record value itself (escaped with a leading backslash).

Mistake 3: Permissive all mechanism

An SPF record is constructed with one or more mechanisms to identify which email services are allowed to send email on behalf of the domain. If you want to learn exactly how an SPF record is constructed, we recommend reading our SPF article in our knowledge base, or consult rfc7208.

An SPF record is interpreted from left to right, the all mechanism will match all senders that did not match the preceding mechanisms. Therefore, you should place the all mechanism at the end of the SPF record, and use it with the ~ (softfail) or - (fail) prefix. Do note that if no prefix is set, the + (pass) is used by default.

It is easy to make mistakes in the SPF record, which will cause unintended permissions. Here are some examples:

v=spf1 mx +all   <-- allows any host to send email from the domain (that's bad!)
v=spf1 mx all    <-- no prefix, so +all is used    
v=spf1 mx        <-- no all mechanism, ?all is assumed
v=spf1 mx ?all   <-- neutral prefix, which effectively disables SPF (opt-out)

In the first 2 examples above the result is that every host on the internet is allowed to send email on behalf of the domain, that is probably not what was intended. In the third and fourth example a neutral stance (?all) is used, which if DMARC is used results in SPF not being used for hosts that do not match earlier rules, forcing the receiver to only use DKIM for authentication.

It is recommended that an SPF record always ends with ~all (softfail) or -all (fail). Where -all should only be used in specific situations (see: Why Mailhardener recommends SPF softfail over fail).

When a new SPF record is created, it is typical to start with v=spf1 ~all, and then add services or hosts that are allowed to send email on behalf of the domain.

With the SPF record set up, a DMARC aggregation service such as Mailhardener is typically used to verify if all legitimate email is passing SPF inspections for the domain.

Mistake 4: The use of the ptr mechanism

The SPF ptr mechanism owes its name to the PTR (pointer) DNS record type, which is used for reverse DNS. With a reverse DNS lookup, an IP address, rather than a domain, is queried. The result is (or should be) the hostname(s) belonging to the IP.

However, a PTR record can point to any arbitrary domain, even if that domain is not owned by the IP address owner. Thus, it becomes trivial for a malicious actor to have their host pass SPF for a domain that uses the ptr mechanism in its SPF policy.

So, when the original SPF standard from 2006 (rfc4408) was replaced in 2014 with rfc7208, the section on the ptr mechanism (section 5.5) was changed to discourage its use.

5.5.  "ptr" (do not use)

This mechanism tests whether the DNS reverse-mapping for <ip> exists
and correctly points to a domain name within a particular domain.
This mechanism SHOULD NOT be published.

In most cases, the ptr mechanism can be replaced with the a mechanism.

Mistake 5: Use of the SPF record type

When SPF was originally standardized in rfc4408, a new DNS record type named SPF was also specified. To accommodate rapid adoption of SPF, rfc4408 also allowed the use of the common TXT type record, in case the SPF type record wasn't supported by the involved systems. The official recommendation (back then) was to publish both types.

But this dual record solution caused more issues than it was intended to solve, most notably situations where an administrator would accidentally update one type but not the other. Adoption of the SPF type record didn't pick up as hoped. So, when the original SPF standard was replaced in 2014 by rfc7208, it was decided that the use of the SPF record type would be dropped in future versions.

To quote rfc7208 section 3.1:

Many alternatives were considered to resolve this issue, but ultimately
the working group concluded that significant migration to the SPF RR 
type in the foreseeable future was very unlikely and that the best 
solution for resolving this interoperability issue was to drop support 
for the SPF RR type from SPF version 1.*

So, although the SPF type DNS record is not officially deprecated just yet, it shouldn't be relied on.

Since it is always required to publish SPF as a TXT type anyway, Mailhardener recommends only using the TXT type record to prevent accidental mismatches between the two.

Bonus mistake: Exceeding the DNS lookup limit

While technically it is not a mistake, but certainly an issue that can negatively impact email deliverability is an SPF policy that requires more than 10 DNS lookups to evaluate.

The SPF standard limits the number of allowed DNS lookups for a full evaluation to 10. If a receiver needs to perform more than additional DNS queries (after fetching the SPF policy itself), it may be rejected during SPF inspection with a permerror state. This can prevent the email from being delivered to the recipient inbox, and it can harm the domain reputation.

We have published an article dedicated to this issue here: SPF lookup limit explained.

To check the number of DNS lookups your policy requires you can use our free SPF validator tool.

Conclusion

As you can see, there are quite a few subtle configuration mistakes that can be made with SPF. We hope this article helps you to catch those mistakes in time.

When using SPF settings, we recommend using DMARC monitoring with a service such as Mailhardener, to catch problems before your customers do.

When making changes to an SPF DNS record, always verify that the record is correct.

Further reading

See also

With Mailhardener you can configure, validate and monitor your domain for all aspects of email security. Mailhardener is free to use for a single domain.
Sign up now