What's the most broken feature in Domino SMTP?Verify connecting hostname in DNS:
You may recall a poll I did here a while back in which I asked "Should the whitelist trump Verify connecting hostname in DNS?"
The result of that poll was that 67% said yes, it should. Less than 10% said no. The rest were undecided.
It now seems that IBM may still need a little help in understanding why this needs to be addressed, so I'll wade in again briefly.
- Currently, Verify connecting hostname in DNS: trumps whitelists, both DNS and local. Thus, if a host which has no PTR and which we have explicitly chosen to trust via whitelisting connects to deliver email, we will reject it anyway with a 554 permanent failure.
- Notwithstanding best practice which, yes, is to have a PTR record for all mail servers, many do not and never will have. Often this is due to a misguided sense of security by obscurity - if they can't resolve my name, they won't know who I am - but far more often it is because the administrators of these systems have no control over DNS for them. Typically these are companies (often large companies) in the far east and Asia and it is increasingly common for western businesses to have strategic links with such companies and to need working email links with them.
- DNS is fragile. As a test to see whether it would be feasible to enable Verify connecting hostname in DNS: here despite this problem, I analysed the logs looking for inbound SMTP activity coming from hosts with no PTR. I found not one but two significant groups of trusted remote systems failing the PTR test:
- Systems with no PTR but whose email we still want (see point 2 above).
- Systems with valid PTR records with short TTL.
Yes, occasionally (and not that rarely) the Domino server will simply fail to spot a remote system's PTR, even if it has one. This may be a timing issue, a problem with the network stack in the host OS or a problem with the name servers or recursion. However, the net effect is the same. Email from trusted hosts with valid PTR records would still be rejected occasionally due to the pre-emption of whitelisting by PTR lookups.
- And even if the ultimate view is that the PTR check should win, the error message given in the bounce is not helpful to a remote sender or administrator trying to debug the problem. Consider two scenarios:
External sender sends mail from a system with no PTR. Bounce message is "554 Mail from sender@envelope rejected for policy reasons." Helpful?
Local sender sends mail to a system with no PTR. Mail is accepted but bounces after delivery due to, for example, no such user. This message will go dead in the remote system's queue, but if anyone looks at it they'll see "554 Mail from rejected for policy reasons." - the rejection text merely repeats the envelope sender which is null in the case of a bounce.
Now look what you get when you enable "Verify connecting hostname in DNS:"
- An end to all spam from China and Korea.
- A very significant reduction in spam from many other sources.
It is just so tempting to enable this feature but we can't because we'd suffer too much collateral damage.
If you are a Domino administrator and you have a support contract with IBM, please ask them to fix this bug*.
* Is it a bug or an enhancement request? It is a potentially very useful feature, rendered unusable by one simple logical flaw. You be the judge. I just want a fix.
1. Stoomaroo31/01/2007 20:47:12
Chris,
submitted my request for it to be a fix - however, I think the logical flaw (don't know - maybe someone at IBM has a diverging opinion on this?) is more critical in the sense that it affects Domino's ability to be used as a perimeter MX-type device.
I have a Domino/Exchange internal mail infrastructure but our MX system is a SendMail infrastructure...for the simply reason that I have the option to play with settings/flaws that you write about here.
It would be fun to see Domino have this sort of flexibility. However, until then...I'll stick to it as an internal (non-perimeter) system.
stoomaroo
2. Chris Linfoot31/01/2007 20:59:39
In fact, that was to have been the thrust of my next post on the subject.
Unless IBM is prepared to tackle issues like this, the Domino server itself will rapidly become unsuitable for use as a gateway server and we'll all have to deploy Sendmail at the border.
This makes the earlier good work by IBM in providing whitelist and blacklist features redundant, and that would be a shame.
3. Bart Severein01/02/2007 05:55:59
That is exactly what I am going to do (in the form of a Ironport). Well, in a test first of course. I must add that my spamproblem is basically that most of the mail (bad and good) comes from Korea and China, and a lot from mailservers with a dynamic IP. It's very hard to filter without some collatoral damage.
4. Stoomaroo01/02/2007 21:12:05
@2 - Chris, as much as I hate to admit it perhaps a nod from Microsoft's approach would be good here.
Their Exchange 2007 has been rebuilt for servers fulfilling different roles in the infrastructure. How different they actually end up, and how well they work in these roles has yet to be seen. A Domino SMTP perimeter flavour anyone?
However, this may be as moot a point as asking for a Domino HTTP flavour, or LDAP flavour or...whatever.
Ho-hum, oh well.
Stew
5. Chris Linfoot02/02/2007 08:49:02
Domino's core strength has always been its ability to do everything, and to do most of those things well. By contrast, one of the manifold weaknesses of Exchange is the fact that you need so many different flavours just to build a simple email system.
I'd rather not see Domino start to slide down that particular slippery slope.
6. Eric Parsons02/02/2007 14:33:51
Homepage: http://startingblockcomputing.com
Sorry, I disagree on this one. As you point out, having a PTR record, and valid, consistent DNS available is the key here. That said, set up a dual system, and have your trusted connections connect to a non-published "white listed heaven" if you will allow such terms.
I can't believe that there aren't other means to stop the spam from the countries mentioned.
7. Chris Linfoot02/02/2007 15:02:29
@6 
So even though we have an IP in our whitelist, the fact that it has no PTR (or at least as commonly Domino thinks it has no PTR when in fact it has) should still mean we reject it?
Really?
8. Bill02/02/2007 20:33:49
It's not a 'bug' in that it is working as designed, it's just a bad design which should be addressed sooner rather than later. The whiltelist should win. That's it's purpose.
9. Budi Febrianto03/02/2007 04:43:42
Homepage: http://indomino.blogspot.com
I give up using domino anti spam technique months ago, just not reliable enough.
On that time, I never use verify hostname technique, because there are sooo many bad admin out there. Too many legitimate emails being rejected, just too many.
I swicth to third party solution and solved the problem. But still I doesn't check the hostname 
10. Chris Miller04/02/2007 05:40:59
Homepage: http://www.IdoNotes.com
This is something that should be modified. Whitelisting was an afterthought and I can see while it was entered as a new feature, they didnt go back and clean up the earlier 'anti-spam' coding in Domino. An enhancement request is where this is heade since it is working as they designed it unfortunately
11. Chris Miller04/02/2007 05:42:01
Homepage: http://www.IdoNotes.com
This is something that should be modified. Whitelisting was an afterthought and I can see while it was entered as a new feature, they didnt go back and clean up the earlier 'anti-spam' coding in Domino. An enhancement request is where this is heade since it is working as they designed it unfortunately
12. Eric Parsons05/10/2007 01:11:23
Homepage: http://startingblockcomputing.com
@7 Whoops... swing and a miss there. Two systems -- one is a white listed only server that only allows mail from known senders (sender IP's). The other, you apply all the smart, untrusted factors like Domain Verification, IP Verification, etc.
13. Chris Linfoot05/10/2007 11:59:11
@12 - Now describe MX records for that Heath-Robinson lash-up
Unable to post a comment? Please read this for a possible explanation...
- 
