Mailhardener recently started displaying a warning in the Mailhardener dashboard, and SPF validator if a domain uses an SPF 'hard' fail catch-all term (
We do this to warn domain administrators for the often unintended consequences of using the SPF 'hard' fail in the
Following this change, we got many questions from our community.
Understandable, because there are many resources online which claim that
-all should be used as part of good email hardening.
Some of our customers also mentioned that some well-known companies and government organizations are (still) using SPF 'hard' fail.
This causes understandable doubt in our push to recommend softfail over 'hard' fail.
In this article we'll try to give a concise and clear explanation on this much debated, sometimes even controversial topic. Our goal is to make our readers understand why the SPF 'hard' fail should be used with caution. We'll use the various relevant RFCs, as well as explaining several design flaws in SPF to explain why SPF 'hard' fail can cause deliverability issues. Finally, we'll explain why 'hard' fail does not add any security benefits in a modern email infrastructure, and which methods domain administrators should focus on instead.
-all) may cause DKIM to be ignored (rfc7208 section 8.4). This can cause relayed email to become undeliverable, even if the email is legitimate and fully DKIM and DMARC aligned.
Which leads Mailhardener to give the following advice:
~all) and combining it with DMARC. This gives softfail the same security properties as 'hard' fail, but without sacrificing deliverability.
-all) can still be used.
-all), Mailhardener displays a warning, as per rfc7489 section 10.1
The M3AAWG (Messaging, Malware and Mobile Anti-Abuse Working Group) also gives this advice in their Email Authentication Best Practices.
Note: The official name of the SPF
-prefix, and its corresponding evaluation result is "fail".
To differentiate fail from softfail, fail is often called "hard fail". Though 'hard' fail is technically incorrect, we may use this term in this article for readability.
In order to understand the problem, you should first have a basic understanding of how SPF works. You can read a longer version of how SPF works in our SPF knowledge base.
SPF is the Sender Policy Framework, a framework that allows a domain to publish a policy about which senders are designated to send email on behalf of the domain.
When an email is sent, the sending SMTP service identifies the sender domain in the
rfc5321.MailFrom address (read more about the different addresses).
The receiving SMTP service will then match the sending host's IP-address against the SPF policy of the sender domain, and apply the corresponding policy.
An SPF record consists of one or more terms, each term has a mechanism and optionally a prefix.
The prefix is the policy the receiver should apply to a sender whose IP matches the term.
If no prefix is defined in the term, the
+ (pass) prefix is assumed, so for example: term
mx is the same as
There are 4 prefixes (policies) to be used:
||pass||Email from a host matching the term will pass SPF evaluation. This is the default prefix if no prefix is specified.|
||neutral||The SPF result for hosts matching this term will not be used, the receiver will only rely on other mechanisms (like DKIM) instead.|
||softfail||Email originating from a host that matches the term will not pass SPF evaluation, but will still be tested for other mechanisms (like DKIM).|
||fail||Email originating from a host that matches the term will be rejected, no further mechanism (like DKIM) may be used.|
SPF policies are evaluated from left-to-right, and the first term that matches the sender's IP-address is applied.
This is why in practice all SPF records end with a 'catch-all' term, using the
all mechanism, which will be applied to all senders that do not match preceding terms.
all mechanism, domain administrators can choose between using
~all (softfail), or
Let's dig a bit deeper in the difference between softfail (
~) and fail (
Over the years, different RFCs have been written on the subject, defining and redefining how SMTP services must treat a fail, versus a softfail.
The latest version of the SPF standard is rfc7208, published in 2014. The 'hard' fail is described in section 8.4 or rfc7208.
A "fail" result is an explicit statement that the client is not authorized to use the domain in the given identity. Disposition of SPF fail messages is a matter of local policy.
As you can see, rfc7208 does not mandate how a fail result should be handled by a receiver. However, the most common outcome is that the receiver simply rejects the email on the SMTP level (also known as a 'bounce').
According to the same section 8.4 in rfc7208:
If the checking software chooses to reject the mail during the SMTP transaction, then it SHOULD use an SMTP reply code of 550
An SPF softfail is different:
A "softfail" result ought to be treated as somewhere between "fail" and "neutral"/"none". The ADMD believes the host is not authorized but is not willing to make a strong policy statement. Receiving software SHOULD NOT reject the message based solely on this result, but MAY subject the message to closer scrutiny than normal.
So, an email from a sender that matches a softfail (
~) term will be accepted on the SMTP level for further evaluation, though it will have failed SPF.
An email with an SPF softfail result may still be inspected for other authentication methods, such as DKIM.
It was soon discovered that SPF has a major design flaw (amongst other problems), which allows an attacker to trivially bypass SPF, and still send email on behalf of a domain that they are not authorized to.
As explained earlier, SPF uses the
rfc5321.MailFrom address to determine the 'sender domain', which is where the SPF policy is fetched from.
rfc5321.MailFrom address is not visible to the receiver, who only sees the
rfc5322.From address as the sender.
The flaw here is that the 'sender domain' and the domain used in the sender's email address do not have to match.
An attacker can simply use
firstname.lastname@example.org as the
rfc5321.MailFrom address, and
email@example.com as the
Because the hacker controls the
hacker.com domain, it will have an SPF policy allowing the host used for the attack to send email on behalf of
The email will be delivered, and to the receiver it will still look like the email came from
This bypass was solved with the introduction for DMARC, which mandates that the domains used in
rfc5322.From must match for the email to be considered SPF aligned.
Besides adding the alignment requirement, which solves the bypass vulnerability, DMARC also simplifies how a receiver should evaluate the authenticity of an email.
DMARC requires an email to be SPF aligned, or DKIM aligned to be considered DMARC aligned.
An email which is DMARC aligned will be accepted by the receiver.
If an email is not DMARC aligned (not SPF aligned, and not DKIM aligned) the receiver will use the policy value from the DMARC record (
reject) to decide how to process the email.
This greatly simplified the processing terms for the receiver, as anything that is not an SPF pass (
+), is not SPF aligned, and thus will not result in DMARC alignment.
So SPF 'hard' fail (
-), softfail (
~) and even neutral (
?) have the exact same effect when using DMARC.
|SPF policy||No DMARC||With DMARC|
||likely accept||use DKIM|
||likely reject||use DKIM|
||always reject||use DKIM*|
*with SPF 'hard' fail, the email may already be rejected on the SMTP level. This may prevent DMARC and DKIM evaluation from happening.
Besides the bypass vulnerability, which was solved with DMARC, SPF has many more design problems. One of them is that a relaying SMTP service (sometimes called a 'forwarder') will break SPF. This is known as the relaying problem.
Note: Most documentation and RFCs use the term 'forwarding' and 'relaying' interchangeably. Because 'forwarding' may be confused with the common 'forward' feature found in email clients, we will use the term 'relaying' in this article.
A relaying SMTP service relays (forwards) all incoming email to a different domain.
Relaying services are common for organizations that have multiple domains.
For example: all email from
old-comany-name.com may automatically be relayed to
The problem here is that from the receiver's perspective the email (which could originally have been SPF aligned) is now being sent by the relaying service.
Because the relaying SMTP service has an IP address that will likely not match the SPF policy, the receiver will apply the
all term policy, causing the email to no longer be considered SPF aligned.
An email originating from a designated (authorized) sender may not pass SPF if it is relayed.
It is important to understand here that relaying happens on the receiver's side of the email, you cannot control this.
The relaying problem cannot be solved in the SPF policy.
(Well, maybe except for allowing the entire internet with
+all, but don't do that!).
This was one of the reasons why DKIM was introduced, as DKIM is (or should be) resilient to relaying.
Furthermore, with DMARC you need only one aligned factor (SPF or DKIM) for the email to be considered DMARC aligned and be accepted. So with DMARC an authentic email that is no longer SPF aligned due to forwarding can still be delivered based on DKIM alignment.
This is how DMARC can improve deliverability, as it allows email to pass based on DKIM, even if SPF is broken due to forwarding.
So, we just learned that an SPF 'hard' fail rejects email messages from an unauthorized sender on the SMTP level. Great! Right?
Well no, not exactly.
As we have also learned, an SPF 'hard' fail (
-all) is performed on the SMTP level.
If the email is bounced on the SMTP level due to an SPF 'hard' fail, the receiver may not perform further evaluation of the email, ignoring DKIM and DMARC.
An email that fails SPF due to relaying, may be hard rejected before any DKIM or DMARC evaluation takes place!
With SPF 'hard' fail (
-all), a fully authentic email with an aligned DKIM signature could still be rejected if it was relayed.
And this is why domain administrators should be very much aware that using SPF with a 'hard' fail
-all term may cause fully authenticated email to become undeliverable.
The DMARC standard even warns for this in rfc7489 section 10.1
Some receiver architectures might implement SPF in advance of any DMARC operations. This means that a "-" prefix on a sender's SPF mechanism, such as "-all", could cause that rejection to go into effect early in handling, causing message rejection before any DMARC processing takes place. Operators choosing to use "-all" should be aware of this.
And because 'Operators choosing to use "-all" should be aware of this.' Mailhardener decided to add the warning message in the Mailhardener Dashboard for domains using
This is also why it is recommended to use SPF softfail.
Let's start with emphasizing that SPF has many design flaws, making it insecure regardless of the applied SPF policy. Do not rely on SPF alone for protection against unauthorized use of a domain. It is better to focus on DKIM and DMARC instead.
SPF without DMARC can be trivially bypassed, and should never be relied on from a security perspective. SPF with DMARC improves the security aspect, but can't be relied on from the deliverability aspect due to the relaying/forwarding issue with SPF.
As we have now learned, DMARC changes the behavior of how receivers are to treat the SPF evaluation results.
Simply said: any SPF result which is not an aligned pass, is a failure.
So SPF 'hard' fail (
-), softfail (
~) and even neutral (
?) have the exact same effect when using DMARC.
So, when DMARC is used (which all domains should) 'hard' fail will not make your domain less susceptible to unauthorized use. Because with DMARC, from a security perspective SPF 'hard' fail is identical to softfail. But from a deliverability perspective, SPF 'hard' fail is worse than softfail.
Yes, you should.
SPF is flawed, but not deprecated. It is still the de facto basic authentication method for email deliverability. There are still legacy or otherwise badly configured email systems that do not support DKIM and/or DMARC, so you need SPF for those.
We do not have any indication that a domain using SPF 'hard' fail over softfail affects domain reputation.
Obviously every SMTP software implementations may choose to deviate from the standards, and make their own implementation of a reputation system which checks for the SPF policy. However, the deliverability issues with SPF 'hard' fail will vastly outweigh the potential reputation benefits of obscure spam detection systems.
For Mailhardener customers, the use of
-allwill not affect the domain rating in your Mailhardener Dashboard, but it will display a warning to make sure you are aware of the potential problems with using
We encourage domain administrators to adopt DMARC, not only does it solve many of the security issues with SPF, it will also positively affect deliverability, thus domain reputation. Remember that the whole point of email hardening is to provide the receiver with as much information as possible to make informed decisions on whether your email is spam or not. If the sender domain uses DMARC, the receiver will have a high confidence in the email being legitimate, thus counting the email towards the domain reputation (that is, unless it is spam).
SPF 'hard' fail has unintended consequences for domains sending email, but it is still usable for domains that shouldn't be sending email.
These may, for example, be parked domains or relaying domains that should only accept inbound email, not outbound.
For domains that do not send email, a 'block all' SPF term consisting of
v=spf1 -all is useful.
Combined with a DMARC policy with
p=reject, it should prevent any email from the domain to be deliverable.
See also: hardening unused domains.
Due to the many problems that SPF has, for a reliable email infrastructure it is always recommended to adopt DMARC, and focus on DKIM instead.
Achieving DKIM alignment with all the designated senders allows a domain to deploy a strict (
reject) DMARC policy.
Compared to SPF, DKIM with DMARC is a much more robust and reliable method of preventing email impersonation
Remember that you need either SPF or DKIM alignment to achieve DMARC alignment, so achieving 100% DKIM pretty much guarantees DMARC alignment, and the domain will no longer be relying on SPF.
Many popular email platforms (Mailchimp for example) no longer even bother with SPF alignment. This sometimes confuses first-time DMARC adopters, seeing a 0% SPF alignment rate for these services. But this is perfectly fine since they use DKIM for all emails. The DMARC pass rate will still be 100%, even though they consistently fail SPF alignment.
Again, an SPF 'hard' fail term may prevent a receiver from evaluating the DKIM signature, which means that even a fully legitimate DKIM aligned email may become undeliverable.
As you have now learned, this is why SPF 'hard' fail should be avoided.
The screenshot above is taken from a domain that uses SPF softfail (
~), had they used the SPF 'hard' fail, deliverability would have been negatively affected.
So, the gist of our email hardening guide is:
rejectpolicy to prevent unauthorized email (impersonation) from being deliverable.
Looking at domains of well-known technology companies, we can see that most domains such as
oracle.com all use SPF softfail (
However, some other domains such as
facebook.com use a 'hard' fail policy.
Amongst government organizations this also differs. For example:
cia.gov both use softfail, whereas
fbi.gov uses a 'hard' fail policy.
As you can see, some prominent domains (still) do use
Organizations may choose to use
-all, either intentionally (accepting the risk of undeliverable email), or unintentionally (the domain administrator does not understand the consequences of using SPF 'hard' fail).
Luckily for you, you have already found and read this article, so you can now make your own informed decision on whether to use softfail, or fail.
Measuring the impact of using SPF 'hard' fail vs softfail is challenging.
As explained, when using SPF 'hard' fail, the email may be rejected before any other processing such as DMARC is performed. The result is that you may never receive a DMARC report for an email that was hard rejected due to a 'hard' fail SPF policy.
Depending on the configuration of both your sending systems and the receiving system, you may or may not receive a NDR (non-delivery report / bounce report) upon a hard rejection.
So, you may have deliverability problems, but just never been able to detect and measure them.
How severe the deliverability of your email is impacted by SPF 'hard' fail primarily depends on the receiver side. Since this depends on a domain's market, target audience (for example: business vs consumer), and region, we can't really give a number on deliverability with SPF 'hard' fail vs softfail.
The SPF 'hard' fail policy is an artifact from a time before DKIM and DMARC.
As we have now learned, SPF 'hard' fail can cause undeliverable emails, even if that email was authentic and fully DKIM aligned. SPF softfail does not have this problem, and when combined with DMARC, it has the exact same security properties as SPF 'hard' fail.
You should now understand why Mailhardener displays a warning when 'hard' fail (
-all) is used, and recommends using SPF softfail (
There is one exception though: for domains that are not supposed to send email, the SPF 'hard' fail (
-all) can, and should be used.
We hope this article has given you a better understanding of why the SPF 'hard' fail should be avoided. You should now also have a better understanding of the many flaws of SPF, and why you should focus on DKIM and DMARC instead. By adopting DMARC and DKIM, you lower your dependency on SPF, which should result in a robust email infrastructure.
If you have any questions or suggestions on this article, feel free to contact us at firstname.lastname@example.org.