Received-SPF: pass (gmail.com: domain of clinfoot@myplace.example designates 192.168.97.84 as permitted sender)

So. SPF is now working at Gmail. Don't miss the next exciting instalment - will it make any difference?

Category: GMail
Technorati: GMail

1. Ian Young21/09/2004 15:46:36
Homepage: http://www.iay.org.uk/blog/

Will it make a difference? Probably not until they start using SPF to filter incoming mail... which a quick test indicates they don't do yet:

Received-SPF: fail (gmail.com: domain of xxx@xxx.org.uk does not designate 82.41.xxx.xxx as permitted sender)

(xxx's included to protect the guilty)

2. Chris Linfoot21/09/2004 16:02:48

Will it make a difference? - Tongue was in cheek when I wrote that.

It will make no difference at all. Ever.

They can never block or even filter on it because so many sites will never publish SPF. If most sites can't even manage PTR records for their mail hosts, we can be absolutely confident that SPF will pass them by.

3. Jason Roysdon22/03/2005 19:56:29
Homepage: http://jason.roysdon.net/

Should they enable filtering, it protects me from forged emails coming from my domain. I don't care about other domains sending me, SPF is about protecting me. It'll also get you a lower (negative) score if you have SPF set up for your domain and the mail gets a PASS when emailing me, but I don't count on it (other than to deny if you violate a domain's SPF policy).

TrackBack From jason.roysdon.net22/03/2005 19:58:05

Sender Policy Framework / Sender Permitted From

I just created an SPF ? record for jason.roysdon.net (not to be confused with the routing protocol, OSPF ?). I used this guide to figure it out.

5. Mats01/10/2005 02:19:29

I get the following in the header when I try to send one of my newsletters to Gmail:

Received-SPF: neutral (gmail.com: 194.xx.xxx.xx is neither permitted nor denied by best guess record for domain of webmaster@ayianapa.nu)

What does this mean? Will my newsletters be stopped by Gmails SPAM filter if the reciver do not have me in their reciver list?

6. Chris Linfoot03/10/2005 08:51:32

It means that you have not publised an SPF record in DNS for ayianapa.nu.

It does not mean that Gmail will stop your email from being delivered. At least as much spam has valid SPF as non-spam. SPF is not an anti-spam feature (it is anti-forgery - it should stop people forging your domain in spam that they send).

If you want to set up an SPF record, go here.

http://spf.pobox.com

7. Steven Maesslein08/04/2006 21:40:42

The best thing about this is that GMail's verification of SPF is flawed. Several of my domains do have SPF records, I'm sending mail through designated relays, and GMail still manages to say:

Received-SPF: neutral (gmail.com: aaa.bbb.ccc.ddd is neither permitted nor denied by best guess record for domain of user@domain.tld)

8. Steven Maesslein08/04/2006 22:47:43

Actually, I take that previous comment back. My SPF records were malformed so it's to be expected that GMail wouldn't make any sense of them.

/me looks sheepish...

Please feel free to take both submissions of mine down.

9. Bill11/05/2007 15:16:49
Homepage: http://www.billrowell.com/

We recently put in SPF records for our domain so 3 additional SMTP servers could deliver mail for us. The DNS records look good and openspf.org reports on the records we have being good as well. What's odd is that GMail still marks our messages as spam with the same Received-SPF: neutral message that Steven reported earlier. Does GMail cache SPF records in their system for some period of time?

10. Chris Linfoot11/05/2007 15:43:11

SPF is just a DNS TXT record, so it will be cached according to the settings you specified in your DNS zone. The fact that Google is not seeing your revised SPF suggests that their cached answer to the TXT record lookup has not expired. OpenSPF either had no cached answer when you tested, or else does an recursive lookup to find the most current answer.

In any case, Gmail does not appear to use SPF pass, neutral, hard and soft fail etc in its determination of what is and what is not spam, so even if/when Google starts to see your updated SPF record and the header changes to Received-SPF: pass, you may still see your emails flagged as spam.

Are you sending from a dynamic IP, or one that is widely blacklisted for some other reason?

11. Bill11/05/2007 15:50:57
Homepage: http://www.billrowell.com/

We're sending from static IP's which are not blacklisted.

12. Chris Linfoot11/05/2007 18:02:58

In that case, I can't guess why Gmail thinks your email is spam. Want to send a sample to my Gmail address and I'll look out for it in the spam folder?

13. Bill11/05/2007 21:27:09
Homepage: http://www.billrowell.com/

Yeah, sure I can do that. I had set up a "test" gmail account to do just this the other day to test my theory. I'll blast it out in a little bit and you can look for it. Should be some random email from "Bill" and the sender's email address should be no-reply@list.yumdrop.com.

14. Bill11/05/2007 21:32:48
Homepage: http://www.billrowell.com/

Chris,

Email is sent. I'm going to assume it ended up in your spam box.

If you have any clue as to why it might still be ending up there, I'd grateful to know. Thanks again!

-Bill

15. Chris Linfoot12/05/2007 16:26:49

OK I can see that in my Gmail spam folder and there are several reasons why it landed there.

1. Your SPF appears to be well formed but will always fail (well, will never pass anyway) for the host that delivered your email to me. Your SPF record lists a number of host names and for a pass, the IP that delivers your email must have a PTR that matches the SPF record. Yours doesn't. The PTR is in a completely different domain to that named in the SPF record, so the SPF check will always fail. It matters not that your SMTP client, on connecting to the target server, appears to HELO/EHLO with a name that matches your SPF. HELO is easily forged and is therefore always ignored - although...

2. The fact that your HELO phrase is a qualified hostname that does not match the PTR of the IP address at that host looks pretty spammy.

3. No X-Mailer header. Often a spam indicator. Most spam scoring systems add a number of points for an absent X-Mailer.

4. Text/plain and HTML versions do not match. Different encoding is used for each (quoted printable and base64 respectively). If you are going to have a plain text version, then it really should have some meaningful content and not just an invitation to click a link and see an HTML version.

5. Base64 encoding used unnecessarily to encode content that can be completely represented in other, more efficient ways. Your HTML part probably does not need to be encoded at all, but if you do need to encode it, then quoted printable would be the way to go as it is more efficient for data which is primarily ASCII and it would also match the plain text part which already uses quoted printable. Base64 used to encode ASCII content is also a very common spam feature.

In summary, I'm not surprised that Gmail thinks your email is spam and I have demonstrated why your SPF implementation is broken.

I'll be happy to help some more if you need to test any further ideas...

16. Bill Rowell13/05/2007 14:36:15
Homepage: http://www.billrowell.com/

All of that makes sense. I'm a little confused about the first part though about the IP that sends the mail must have a PTR that matches the SPF record. There should be A host records in our DNS (set up by our hosting provider) that list those servers (list.yumdrop.com, list2.yumdrop.com, list3.yumdrop.com) as PTR or A records. What were you noticing that didn't match up? Thanks for taking the time to explain this stuff. I really appreciate it.

17. Chris Linfoot13/05/2007 20:17:28

Yes, you have A records so that those names point to the correct IP addresses.

What is needed is the reverse - PTR records so that the IP addresses point back to the right names. Currently they don't - that is forward and reverse DNS lookups on your hosts do not match.

It is the reverse lookup that is important for SPF the way you have it implemented (using qualified names, not addresses).

The receiving system will see your IP connecting and will look up its name so that this can be compared with the SPF policy.

Because the reverse lookup points to a host name like host111.192-168-101.oem.net * and not one like list.yumdrop.com, the SPF check fails (or, more accurately, does not pass - since your published SPF policy is, I assume deliberately, ambiguous).

* (and please note I am deliberately not using a real example)

18. Bill Rowell14/05/2007 00:57:49
Homepage: http://www.billrowell.com/

Ok, that makes much more sense to me. Thanks. I'll have to look into getting that fixed. I think I need to take care of that with our ISP. I'll get on that tomorrow. Thanks again Chris!

19. Bill21/05/2007 14:21:06
Homepage: http://www.billrowell.com/

Chris,

I took care of all the items you mentioned. Unfortunately, mail is still ending up in the spam directory. It looks like it passes the SPF test, though as you mentioned, that might not affect the email getting into the spam bin. Would you mind letting me sending you another test and taking a look at the email again? Thanks.

-Bill

20. Chris Linfoot21/05/2007 16:07:14

Go ahead.

21. Bill21/05/2007 16:34:18
Homepage: http://www.billrowell.com/

Ok, its sent. Should look similar to the first test I sent.

22. Chris Linfoot22/05/2007 12:13:40

Briefly:

- SPF now working - for all the good it'll do which isn't much
- still flagged as spam by Gmail

Why?

Not sure but you could try these:

- Plain text part still contains "click the link to view the email" and not a readable plain text version. This will increase spam score.

- HELO given by your sending MTA does not match that MTA's resolved name. RFC only says that HELO should be a valid primary domain name and yours is, so this is RFC correct, however it would probably help if it did say HELO with the same primary domain name as DNS finds when doing a reverse lookup of the IP.

And if addressing these two items still does not fix it, then I am all out of ideas.

Sorry.

23. Bill22/05/2007 16:08:40
Homepage: http://www.billrowell.com/

Ok, I made sure that both versions of the email are quoted printable, so hopefully that first issue is taken care of. As for the DNS, we use these servers to email out newsletters for several different domains so there are reverse dns records for multiple domains. list.frightcatalog.com probably is listed first thus comes back first and doesn't match. I'm not aware of an easy way around that.

24. Bill23/05/2007 15:25:07
Homepage: http://www.billrowell.com/

So you were right about the HELO thing. We have 3 SMTP servers and the two outside our firewall work ok. The one behind the firewall doesn't. I think its because the machine that's sending the emails through the SMTP server is on the same local network and that's the IP that it sees (hence the 192.168.0.1 Unknown business).

Unable to post a comment? Please read this for a possible explanation...