Chris-Linfoot.net

############################################################
### FATAL THREAD 13/14 [   nsmtp:  0930:  07e0]
### FP=0x0d19ed7c, PC=0x602715ac, SP=0x0d19ecd4
### stkbase=0d1a0000, total stksize=262144, used stksize=4908
### EAX=0x00000001, EBX=0x00000000, ECX=0x00000000, EDX=0x00000002
### ESI=0x00000000, EDI=0x0000005c, CS=0x0000001b, SS=0x00000023
### DS=0x00000023, ES=0x00000023, FS=0x00000038, GS=0x00000000 Flags=0x00010202
Exception code: c0000005 (ACCESS_VIOLATION)
############################################################
@[ 1] 0x602715ac nnotes.TNEFProcessNote+156 (0,0,0,cd11834)
@[ 2] 0x60219e3c nnotes.CIMsgImport::Import+428 (cd11834,a400070,5c,3)
@[ 3] 0x00412d76 nSMTP.CSMTPProtocol::SubmitMessage+502 (cd11834,d19fb36,2,cd12014)
@[ 4] 0x00411229 nSMTP.CSMTPProtocol::CommandDATA+1417 (2014,2,cd12014,60e6a140)
@[ 5] 0x004123e4 nSMTP.CSMTPProtocol::StateConnected+1076 (cd12014,a400000,a400050,a400048)
@[ 6] 0x004126dd nSMTP.CSMTPProtocol::Run+429 (cd12014,54bda58,43ae28,54bda58)
@[ 7] 0x0042099e nSMTP.CBaseTask::StateMachine+398 (a400048,54bda58,43ae28,54bda74)
@[ 8] 0x00402fc3 nSMTP.CSMTPSrv::OnConnect+211 (43ae28,54bda74,b800001,54bda58)
@[ 9] 0x004167d6 nSMTP.CIServ::ServerTaskProtocolMachine+262 (43ae28,54bda58,3,22ad434)
@[10] 0x0041b007 nSMTP.CIServ::ServerTaskIOCP+1127 (43ae28,0,60103820,0)
@[11] 0x0041b93d nSMTP._ServerThread@4+29 (0)
 [12] 0x7c57b396 KERNEL32
Invalid stack frame detected: Unable to read process memory for frame

Looks like some further work is required.

Update: It seems to be spam or a virus that is causing the TNEF crashes.

Looking at the Domino server log (Mail Routing Events) we see, in virtually every case, an IP we have not seen before connecting, an obviously bogus sender envelope (SMTP MAIL FROM) and a valid local recipient envelope (SMTP RCPT TO), then nothing as it is at this point that the system crashes - after the DATA phase has completed but before the SMTP connection has been closed and the message written, itemised, into mail.box.

Following the restart, there is never any later attempt to deliver an email with the same sending IP, sender and recipient envelopes. In other words, this crash is just extreme greylisting.

Sadly, because there's never a later attempt to deliver, we never see a copy of a message that has caused the crash, but I suspect either that some spamware is forging TNEF characteristics and getting it wrong, or that this is deliberate use of malformed TNEF as a malware attack vector.

I guess we'll never know for sure but the Domino TNEF converter needs to be made a little more robust to survive this kind of abuse.

Category: Domino 7
Technorati: IBM Lotus Domino TNEF 7.0.2

1. Peter von Stöckel11/10/2006 11:09:59
Homepage: http://www.bananahome.com/

What a relief! I'm not the only one having problems with the TNEF conversion.

http://www.bananahome.com/users/bananahome/blog.nsf/d6plinks/THOR-6UEJXG

Let's get our hopes up for 7.0.3!

2. Chris Linfoot11/10/2006 11:27:00

It is interesting that every Internet originated email received after TNEF conversion was turned on had an additional header inserted by Domino (the MIMETrack itemize process):

X-TNEFEvaluated: 1

This was regardless of whether the message was TNEF or not. It appears that the TNEF conversion runs on all MIME messages regardless of whether TNEF is present and I am fairly certain that the crashes I have seen happened while the SMTP server was itemising a non-TNEF message in every case.

3. Kendall11/10/2006 13:14:23

We also had the crashes, repeated, last night. (Yay autorestart!) Thanks for posting this -- great to see confirmation that I was reading our NSD right. BTW, were you using the "break SMIME" setting for TNEF? (We were.)

That new Notes item is weird; I hadn't noticed that, thanks for mentioning it. (In Notes, I see TNEFEvaluated, without X-). I would've hoped Domino would just check for a TNEF attachment (I hope that's all that's evaluated) but anyway, why record that item for each doc; it seems like overkill. I suppose I should've been suspicious of a new feature that was off by default and only accessible via notes.ini, though.

Re. when it's crashed, I had thought it was crashing on files WITH TNEFs (most of our e-mails, of which many were processed, had no TNEFs). But I don't know--didn't log extra SMTP stuff to see. It's really weird if it's crashing on files WITHOUT them....

Anyway, thanks again!

4. Chris Whisonant11/10/2006 14:55:29
Homepage: http://cwhisonant.blogspot.com

I'm glad I implemented Julian's TNEF mail-in DB! :) Good to know that I should test this some more before enabling it. Thanks for the heads up.

5. Chris Linfoot11/10/2006 15:22:56

@3: Not using break S/MIME. You should never break S/MIME. There is a reason why people sign their emails...

6. Peter von Stöckel11/10/2006 16:32:22
Homepage: http://www.bananahome.com/

@3: Not using break S/MIME here either, and for the same reason as Chris didn't.

7. Ben Rose11/10/2006 16:32:26
Homepage: http://www.jaffacake.net

No issues so far, maybe my spam filter is stopping them all - it auto-purges malformed messages.

8. Chris Linfoot11/10/2006 16:55:18

@7: That should make you pretty safe if my hypothesis is correct.

And of course there is at least one known exploit against Exchange using malformed TNEF so that may be what we are seeing...

http://chris-linfoot.net/d6plinks/CWLT-6KXBUN

9. Keith Brooks12/10/2006 04:00:42
Homepage: http://kbmsg.blogspot.com

I saw this more and more since going to FP1, I thought it would help, instead my server(mail hub) crashes daily and sometimes immediately after itself.
I know its an email and its hard to nail down the problem, but if 702 won't fix it do I need to revert back to 701?
Man this coexistence stuff sucks.
Of course my developers are sending test emails as well with the java problem now patched in 702 so I guess I will uopgrade the server and hope the magic dust does its bit.

10. Gary Cousins12/10/2006 07:39:37
Homepage: http://www.thirdforce.com

I enabled it (TNEFEnableConversion=1 in the notes.ini) on our mail server and the server crashed within a few hours so I raised a PMR. According to tech support it's a known issue: SPR DPOS6PVLFC - TNEF causing SMTP crash.

A hotfix is due in the next few weeks.

11. paul lyons13/10/2006 18:33:43

mea culpa -- sorry about that. i fixed this in 6.5.6, 7.0.3, and 8.0 and i'm expediting the hot fix process for 7.0.2. --pwl

12. Chris Linfoot13/10/2006 20:34:04

Paul - Thanks. I do appreciate the feedback.

13. Gary Cousins18/10/2006 15:30:45

Lotus sent me the hotfix yesterday (10mb), I installed this morning and all seems well.

14. Gary Cousins20/10/2006 07:38:25

I spoke too soon. Domino crashed after a day and half with a nSMTP error and something to do with cmime. It might not be related but then again...

15. Gerco Wolfswinkel03/11/2006 19:52:50
Homepage: http://www.domino-weblog.nl

We received the hotfix today, size is 9320 KB, not counting the accompanying document. Will be installing later.

16. Gary Cousins20/11/2006 14:51:32

I just installed the second hotfix (HF101) today. So far so good...

17. Henning Heinz11/01/2007 14:42:23

You lucky ones. I asked for the hotfix doing my very best but at the end support said they only give it away if I send them another mass of debug files ( and that although I provided 4 nsmtp nsd files).
At this point I gave up and decided to keep it disabled. I am happy that this works for you.

18. Gary Cousins18/01/2007 07:33:29

Sorry for the late response on hf101, I've been away. The hf didn't work out, testing caused a huge amount of distruption to my mail system so I dropped the PMR. I removed the notes.ini variables.

Unable to post a comment? Please read this for a possible explanation...