Chris Linfoot | Who do you trust?

Wednesday, 7. April 2004

Who do you trust?

Pay attention. Time for today's sermon.

Regular readers will already have realised that there is no single solution to spam. Rather, as the war rages on around us, we seek out and deploy additional weapons -- some tactical, others strategic. Most are subject to counterattack and so the cycle of measures and countermeasures seems almost inevitably set to continue.

Since version 6, Domino shops have become accustomed to using DNSBLs (though without a balancing whitelist, which Domino currently lacks, the use of particularly aggressive lists may be unwise). DNSBLs aim to kill spam at source by denying email from hosts which become listed according to various criteria (who they are, where they are, how they are used). DNSBLs remain a vital part of the armoury -- but we need more.

More recently, we have seen the emergence of a new type of content filtering which seemed initially to be very promising. Bayesian filtering uses a purely mathematical approach by weighing the statistical likelihood that individual words in a message may be spam, then compounding a score for an entire email to decide how probable it is that the message in hand is spam. But now we see evidence that this is vulnerable to attack -- it is possible to poison Bayesian scores by loading a spam with obviously spammy words, and adding many more that are not (see Nathan's comment on my story here).

Then came SPF and caller ID, intended to verify that a connecting SMTP client attempting to deliver mail from user@example.com was actually permitted to deliver on behalf of example.com. Sadly, these are similar not simply because of how they work, but also in how widely they have thus far been deployed (which is to no significant extent whatsoever).

And now, those nice people who brought you the Spamhaus Block List, ROKSO and the XBL enter the fray with their "bus lane for trusted mail servers".

The problem we have all been facing is that all IPs are essentially created equal. And even with IPv4 there are 4 billion of them (with IPv6 the issue is unimaginably larger). Most should of course never even attempt to send email (well, send it direct-to-MX anyway). But how to know, in this network of peers, which are mail senders and which are not?

In this context, the bus lane analogy very nearly says all that needs to be said, but I will elaborate briefly. What is proposed is that a new TLD is created called .mail. Users will register domains in .mail under very strict guidelines. DNS for all .mail names will be controlled by a single body or regulator. So that regulator of .mail essentially becomes a trusted third party.

And in future, if our own MTA sees an inbound connection from a host named *.mail, it will bypass all further checking (DNSBLs, local block lists, Bayesian filtering and so on) and just deliver the message. Like the man said - a bus lane for trusted mail servers.

The crunch -- will it work? My friends, I have no idea. But I will say this:

When did Spamhaus last let you down?
If your answer is the same as mine (and I can't believe it isn't), then we already have a trusted third party. Trusted, that is, to say what mail should be denied. Why not extend the remit to include trusting them to say what mail we should ~~accept~~ prioritise?

So. Where next? Well we'll never even get started if the .mail TLD never sees the light of day, so this should be our focus for now. By now, you will have guessed my opinion. If you think the application for the .mail TLD should be granted, say so here by sending an email to stld-rfp-mail (squiggly thing) icann (you know what) org.

(The archive of postings to stld-rfp-mail is here.)

Category: Fixing SMTP
Technorati: Fixing SMTP

Comments :

1. Jerry Carter08/04/2004 01:13:13
Homepage: http://datatribesoftwerks.com

Two possible foils (questions mostly):

1) What will prevent someone spoofing a *.mail host name?

2) If there is a centrally managed authentication body, wouldn't it be simple for spammers and their virus writing chums to DDOS the .mail DNS off the net?

If the above questions are at all not founded in faulty logic (You're the expert, Chris, not me! ), then my following comment is valid: we'll never be free of spam until there is a fundemental change in the way email is authenticated, formulated, and transmitted. SMTP has to go, Authentication has to be regulated and required, and you almost can't transmit it over TCPIP.

2. Chris Linfoot08/04/2004 08:18:31

1 - Go ahead. Spoof away. The test is not whether my host says is is mx1.example.com.mail it is whether rDNS for my IP (controlled by the .mail sTLD, remember) actually is mx1.example.com.mail. So - impossible to spoof. Next question.

2 - Spamhaus is being DDoSsed all the time. They hardly notice any more. Why? Just look at the number and location of name servers serving up spamhaus.org DNS. They are massively distributed across many networks and many countries.

In practical terms, it would probably be more difficult to DDOS the .mail name servers than to DDOS the Internet root servers (last I recall there were only about 19 of those in the world).

With regard to your last comment, I see your point but disagree fundamentally. Yes, SMTP is far from ideal. Yes, some form of sender authentication is needed (that is what SPF and Caller ID attempt to do).

But SMTP is what we have and is now so massively deployed and such a core part of the Internet that replacing it is logistically unfeasible. So an incremental approach to patching the flaws is needed. And remember, while SMTP is flawed, it is so by design -- it does exactly what it set out to do. It is just that it has been misapplied but I reiterate -- it is too late to start again.

Here's an equation for you:

SMTP + SPF/Called ID + .mail = Basic Transport + Sender Authentication + Regulation

No it isn't very pretty and there are technically more elegant solutions, but none which build on the infrastructure already deployed today.

3. Gerco Wolfswinkel08/04/2004 10:30:12
Homepage: http://www.wolfswinkel.net

There's a point in the proposal that kind of worries me:

"Abusive registrations will be minimized for two reasons:
1) The high per name-year fee."

They don't mention any price here, but a high entrance fee is kind of prohibitive, don't you think? If the charge is too high, the hobbyists are probably not going to join in. That could mean no 'grassroots' acceptance of this otherwise good idea .

Opinions?

4. Chris Linfoot08/04/2004 10:55:23

Yes, it is expensive but is not intended for use by hobbyists but by businesses which need to be able to rely on mail delivery.

Remember, the fact that a connecting host in future does not resolve in the .mail TLD is not itself grounds for rejecting the mail. It just means you need to check it rigorously (DNSBLs, Bayesian filters and so on). Where a connecting host does resolve in .mail, save yourself the trouble and overhead of DNSBL checks and Bayesian filtering and just accept the email.

Not sure grass roots acceptance is necessary. Acceptance by the (non-spamming) business community is.

RTFAQ here - http://www.spamhaus.org/tld/faq.lasso

5. Scott Wolfersberger09/04/2004 13:41:23
Homepage: http://www.cse-consulting.com

Chris,
These are the same comments I put on the ICANN website, but I wanted your response so I'm posting them here as well.

Although I agree that spam has become one of the biggest problems in the
IT and business communities today, I do not agree that charging legitimate
email domain owners $2000 to continue to use the email system is
reasonable.
In your scenario, you have people, eventually, completely ignoring mail
from standard .com/.net/.org addresses. In that case I would have to pay
the $2000 to continue to communicate with other businesses on the web. I
find it completely backwards to impose a fine (and that's what it really is)
on the people that use the email system responsibly and for good business
reasons. Perhaps this proposal should be re-assessed to see if the right
people are being punished for the spam problem.

Thank you,
Scott Wolfersberger
www.cse-consulting.com

6. Chris Linfoot09/04/2004 14:56:08

First up, the use of .mail is intended and expected to be optional. Many senders, my own company included, regularly send large volumes of email to many different systems and have never had issues with our mail being blocked or tagged as spam. But some businesses that rely on electronic communications to interact with customers do have serious issues with this (e.g. low cost airlines that use email to inform passengers about flights they have booked). .mail could be a real benefit to them.

Secondly, I personally would like to see the price lowered. But the economics of the issue are complex.

It starts with the facts that:

1 - SMTP is the wrong technology to have deployed for busines to business email precisely because it works peer to peer with no regulation or validation. Remember, there were such mechanisms around before SMTP became the prevalent standard (UUCP, x.400) and these were not widely taken up precisely because of the burden of administration of trust relationships and the consequent cost.

2 - It is far too late to fix it by starting again, so a means must be found to superimpose regulation and validation on the existing infrastructure without breaking it.

Because we cannot change the transport mechanism, which will continue to be SMTP, the idea of regulating it by creating the Anti-Spam Community Registry, controlling all DNS for mail senders in a special .mail domain, seems a pragmatic solution.

This body will have to run the .mail name servers and directly administer all DNS in the domain. It will also handle all abuse email, so if someone is operating a host named mx1.spammer.biz.mail and someone sends an abuse report to abuse[at]spammer.biz.mail, it will be the Anti-Spam Community Registry that has to respond to and deal with the complaint.

And all of this will cost money. The Anti-Spam Community Registry will be a not for profit organisation, so the fees charged are evidently expected to cover their costs.

Analogy - We pay through taxes for a police force to protect us from people who would murder us or steal our property. This payment is not punishing the general public for the crime problem, it is a general recognition that eternal vigilance is the price of liberty (and eternal vigilance costs money).

Trouble is, we have become accustomed to using email apparently for free*, so any attempt to retrofit regulation and recover the costs of that regulation is bound to be unpopular - why pay for something tomorrow that is free today?

But consider this: The alternative to regulation is anarchy which is what we have now. And it is getting exponentially worse.

Will .mail even work at all? Well, if it had been proposed by anyone other than Spamhaus I would have ignored it completely. Certainly, none of the other current applications for sTLDs seems to have been well thought out or to offer any real innovation. But Spamhaus has a long and well documented track record of getting these things right and I think they should be allowed to run with .mail even if ultimately it doesn't work out.

* In truth, the days of free email have already ended. How much do we all spend on anti-virus and content filtering software, additional storage and administration? How much is lost in wasted network resources and wasted users' time? How much is lost in business disruption when real email is incorrectly tagged as spam? The costs of a regulated .mail "backbone" must surely be weighed against the concomitant benefit of a reduction in all of these other costs.

All this said, are the proposed costs for .mail domains too high? I think they might be and this is certainly something I would wish to see probed in some detail by ICANN on granting the .mail sTLD - but grant it they should.

And like I said, if you don't need to use the .mail backbone, well don't. It isn't compulsory and no-one else can steal your .mail domain if you don't register it.

7. Mike McCann09/04/2004 18:27:39
Homepage: http://www.buykearney.com

Chris,

Although, when I first read the article on MSNBC.MSN.com http://www.msnbc.msn.com/id/4694684/ I was excited, after learning more and reading various comments I tend to agree with Scott W. and Gerco W. in regards to cost.

Yes, many companies can afford it...but I agree with Kevin R. that small businesses cannot afford to, or simply will not, pay the high fee which I understand is $2,000.
Can you expand on the cost analysis and how that is being proposed as I have seen mention of yearly and one-time fees and I am confused.

Using myself as an example, the vast majority of my industry...real estate...converses with clients daily who would never have a .mail e-mail address. So the .com/.net problem would still be there wouldn't it? I think most companies will have this problem except for internal e-mails.

Will I be able to assign an unlimited number of e-mails to those in my database? If so, will there be a cost and how do I keep the spam man from getting an e-mail from me or someone else and then doing the same thing with a legit .mail address?

One thing I do know is the number of junk e-mails and now viruses I receive are a major pain in the neck and I am now having to change e-mail names every few months due to the clutter as well as losing quality e-mails that get buried in the junk.

Why can't someone come up with a white hat virus that attacks the bad guy viruses and spam man? Thanks for the opportunity to join in.

Mike McCann

8. Chris Linfoot10/04/2004 11:05:39

Thanks for posting.

This needs a longer response than I have time for now. So I will try and post again next week after the Easter break. May not be till Wednesday or Thursday.

TrackBack From Chris-Linfoot.net14/04/2004 12:11:44

Meltdown and .mail

I hope this new post will address some of the concerns that have been raised here.

Unable to post a comment? Please read this for a possible explanation...