Chris-Linfoot.net

• Use aggressive DNSBLs (SORBS composite zone et al), kill lots of spam and suffer the inevitable false positives that result
  
• Use weaker though still very worthwhile DNSBLs (Spamhaus et al) and watch lots of spam stroll right past the DNSBLs, but with almost no false positives

Given this choice, I too would have to opt for the latter option and may well have concluded by now that DNSBLs were a waste of time.

Well, gentle reader, I am here to tell you that there is a third way. Having used a whitelist now for more or less a year, I know from direct experience how effective DNSBLs can be when balanced by that whitelist. We still block over 95% of spam at source and reports of false positives remain too infrequent to be statistically significant.

Whitelisting, both DNS and local, is natively supported in ND7 and now would be a good time to start planning your whitelist. If you get it right, then from day one of ND7 you can switch to the use of strong DNSBLs, safe in the knowledge that you will reject no email from sites that you have chosen to trust.

But how to start?

You must start with your logs of course. This is made easier if you use event monitoring to record occurrences of "SMTP Server: mail.example.com (192.168.0.1) disconnected. x message[s] received" (where x>0).

Analyse these and you will quickly build up a picture of which sites are sending significant amounts of email. Most spam comes from IPs that are only used for spam for a short while and these IPs individually will show up in your logs with far lower frequency than IPs being used for real email. Now, take the IP addresses that deliver say 80% of your email (pareto works for email as for so many other things) and use the Sam Spade whois tool to find out about them. You could also user Senderbase and OpenRBL to verify the size and spamminess of network blocks used to send you email.

You should now be able to identify a smallish number of networks mainly ranging in size from perhaps /28 or /29 up to /16 (but rarely bigger than /24) that you can put in a local whitelist. We have a total of 386 networks whitelisted of which the smallest are /32 (very few) and the largest /16. Most are /24 or smaller.

When the time comes to implement ND7, populate your local whitelist with the networks that you have taken the time to identify in advance and switch on some well chosen, strong DNSBLs. Then watch the spam stop.

DNSBLs to consider when the time comes

• DSBL
  
• SORBS composite zone
  
• Spamcop
  
• Spamhaus (SBL-XBL zone)
  
• Choose appropriate blackholes.us zones (perhaps china.blackholes.us)
  
• Consider ORDB as a backstop against open relay spam though that is not the huge problem it once was
  
  
• ... and perhaps spend some time making a local list of unwelcome networks

Finally, if there is sufficient demand/interest, I may consider publishing a starter whitelist here.

Category: Domino 7
Technorati: Domino 7

1. Nathan T. Freeman03/11/2004 12:15:24

Consider this the first instance of demand for your starter whitelist :)

2. Chris LeRoy03/11/2004 13:41:19
Homepage: http://www.brainbent.com

I would welcome a user driven whitelist, based possibly on mail that the internal user sends out (minus possible postmaster rejections or out of office messages, assuming that these would be spam responses). The assumption is, based on an established relationship from the internal user, that the sending individual would then be deemed as 'acceptable'. While this doesn't protect against spoofing or inappropriate content, that is where other rules should further refine harmful or inappropriate messages.

FWIW, I haven't found host based whitelists to be as effective as they once were. I see spam that is being routed through hosts that were considered legitimate. AOL is also seeing the same trend --> ( http://www.circleid.com/article/794_0_1_0_C/ ). Of course, I have had instances of spam from AOL's own published servers, too... However, the vast majority of our external customers are making contact from their homes or small businesses, which could lend to the different train of thought.

3. Chris Linfoot03/11/2004 14:01:58

That does not square with what we see here. I will publish a graph shortly showing the top three DNSBLs here. This clearly shows DSBL, SORBS and our local list combining to deliver 83% of our DNSBL hits. DNSBL hits in turn, as I have said, kill more than 95% of our spam at source.

This is net after whitelisting the absence of which would have meant that email from some ISPs' mail cores would have been bounced.

Of course we do see spam from sources that are whitelisted at the DNSBL/WL level and content filtering weeds out most of them but the amount of spam thus routed has been overstated. I see maybe a handful of samples every month from the likes of AOL and Hotmail both of which organisations, whitelisted here, handle abuse reports pretty effectively.

For the sake of balance, we do have one rogue ISP in Sweden whose mail core is relaying spam on behalf of one or more trojaned customers (though as that spam is entirely in cyrillic character sets it is trivially easy to filter), but the volume is very small.

Our DSBL stats speak for themselves - DSBL lists only abusable relays and proxies which are almost without exception end user devices that have been compromised by malware.

4. Chris LeRoy03/11/2004 16:07:12
Homepage: http://www.brainbent.com

Please don't get me wrong. Our use of SORBS and Spamhaus still produce awesome results (somewhere between 70% and 80% on avg.). It's the false positive/negative potential that I am looking at, and minimizing those false positives through the combination of a whitelisted sender that has a dynamic IP or less than crediulous (sp?) ISP. This would be an added layer of relief to the admin and protection to the end user that we serve, preserving potentially time sensitive email by offloading some of the administrative tasks to automated means. This added option opens up the ability to use more aggressive blackholes lists and more aggressive methods of DNSBL's.

Is it foolproof? No. The only technique that is foolproof is not accepting mail in the first place. But the nature of our business and architecture, which may significantly differ from yours, while on the same platform (Domino), requires some additional options in the toolbox to work with. I would hope that IBM would look at some additional, yet optional, features to assist more administrators with the antispam effort. I would love to lose my dedicated antispam servers if more features were offered in Domino.

Does that make more sense?

5. Chris Linfoot03/11/2004 16:14:48

You have a large number of external parties sending you email from dial/dynamic IPs?

1. Whitelist them (if their IPs don't change often)
2. Educate them - they should not be operating an MTA on the end of a DHCP allocated IP

6. Chris LeRoy03/11/2004 17:50:30
Homepage: http://www.brainbent.com

We, as an organization, are not going to tell our customers how to run their business. Our customers subscribe to our company's faculities for the recreational and social services we provide, so the attitude is that we are at their control. Whether I agree with what those customers are doing (which I don't) or not, it's not affecting their bottom line if we reject their mail. But it stands to impact ours.

I guess we can agree to disagree.

7. Chris Linfoot03/11/2004 20:09:45

Sounds like we are almost in agreement about the main point here which is this - for many shops (yours evidently included) the opportunity to use a whitelist in future will be greatly beneficial.

Even if you whitelist every Cable/DSL IP in the US and Canada, you can still use SORBS and DSBL and get some of the benefit.

8. Gerco Wolfswinkel04/11/2004 09:34:04
Homepage: http://www.wolfswinkel.net

"spamminess" - what a beautiful word, hadn't heard that one yet

But otherwise, it's a good idea to start compiling a whitelist in advance. An example from you would be of help, so I second Nathan's request. tia!

9. Gerco Wolfswinkel04/11/2004 10:46:34
Homepage: http://www.wolfswinkel.net

One question, though.. The helptext says:

"Enter IP addresses or host names of the systems to whitelist. Enclose IP addresses with square brackets; for example: [127.0.0.1]. IP ranges and masks are supported. Wildcards can be used except within ranges. "

I'd like to use *.domain.tld for a couple of well known customers (easier, and works even when they change isps), and the helptext seems to indicate that wildcards can be used. Any experiences with that?

10. Chris Linfoot04/11/2004 11:00:17

You can just use domain.tld, no need for a wildcard. This will whitelist hosts with PTR records in domain.tld and all subdomains.

11. g04/11/2004 16:21:22
Homepage: http://www.wolfswinkel.net

Thanks Chris - we'll start compiling a whitelist soon.

Unable to post a comment? Please read this for a possible explanation...