Subject: Re: Transient fetchmail corruption problem under Windows 2000
From: Jason Tishler <[email-masked]>
To: "Gerrit P. Haase" <[email-masked]>
Cc: Cygwin <[email-masked]>, Olaf Foellinger <[email-masked]>
Date: Tue, 7 Aug 2001 16:10:42 -0400

Gerrit,

[This message was received corrupted...  I have attached it for perusal
by the curious.]

On Tue, Aug 07, 2001 at 09:26:03PM +0100, Gerrit P. Haase wrote:
> What are the types of the binary attachments?

Actually, after further investigation it appears that even plain text
attachments are affected too.

> Is it various or are there only some files corrupted or are all 
> attachment destroyed?
            ^^

Note the null character above, this is part of the corruption.

If I send the same message (which includes an attachment) to myself
repeatedly, most of the time to it is corrupted when fetched via POP by
fetchmail.

> Sorry, I don't know enough to help you out of this misere since i 
> never used procmail, but io have some more questions.

Thanks for the sympathy and yes, this is misery!

> Jason, do you use also use ssmtp as Olaf mentioned?

Yes.

> Anyway, since it only happens on NT5 and not on NT4 as you reported,
> it seems not to be a problem with Fetchmail.

I'm not so sure.  Maybe Windows 2000 tickles a buffer overrun problem
that does occur under NT 4.0?

> Since Fetchmail does 
> nothing else but retrieve and send out to a smtp server i could not 
> think of big problems with this. Or is it an option to pipe directly 
> from fetchmail to procmail?

I have fetchmail configured to pipe directly into procmail.  Note that
this is a very typical configuration and IIRC, fetchmail attempts to
pipe into procmail when it can't find a SMTP listener.

> Are the files stored to disk between retrieving and sending out?

I don't know.

> Does Procmail store to disk before delivering?

I don't know.

> Are your mounts of binary type?

Yes.

> Please report if you track down the source of the problems, maybe a 
> problem with the pipes?

After further experimentation, I believe that I have ruled out procmail
as a potential culprit.  I replaced procmail with a shell script that
just cats stdin to a file.  Even without procmail in the picture, I'm
still getting corrupted messages.  Hence, the problem is with fetchmail
(or Cygwin itself).

> Olaf could you please report exactely what you do to send out a 
> corrupt message? Is it a known issue? I never heard before of 
> problems with mailing through ssmtp on win2k.
> I will try tomorrow to send some mail with ssmtp and binaries 
> attached.

I have not had any problems sending email (without or with attachments)
with ssmtp under Windows 2000.

Thanks,
Jason

-- 
Jason Tishler
Director, Software Engineering       Phone: 732.264.8770 x235
Dot Hill Systems Corp.               Fax:   732.264.8798
82 Bethany Road, Suite 7             Email: [email-masked]
Hazlet, NJ 07730 USA                 WWW:   http://www.dothill.com

Subject: Re: Transient fetchmail corruption problem under Windows 2000
From: "Gerrit P. Haase" <[email-masked]>
To: Cygwin <[email-masked]>
Cc: Olaf Foellinger <[email-masked]>, Jason Tishler <[email-masked]>
Date: Tue, 7 Aug 2001 21:26:03 +0100

Am 7 Aug 2001, um 15:39 hat Olaf Foellinger geschrieben:

> On Tue, Aug 07, 2001 at 11:31:10AM -0400, Jason Tishler wrote:
> > [I normally would not post such a vague bug report but since this
> > problem significantly impacts my mail delivery I hope that I will be
> > allowed some latitude...]
> > 
> > I am using fetchmail/procmail/mutt to retrieve, process, and read my
> > email.  I recently changed from using a Windows NT 4.0 SP5 to a
> > Windows 2000 SP1 machine.  Shortly thereafter, I started to notice
> > that some of the emails that I received with binary attachments were
> > corrupted.

What are the types of the binary attachments?
Is it various or are there only some files corrupted or are all 
attachment destroyed?

> > After sending the same message to myself repeatedly and examining
> > the raw message in an mbox file, I began to see the following
> > pattern. The corrupted messages are always larger than they should
> > be and there are chunks of garbage characters in them.  Sometimes
> > the MIME separators are repeated as if fetchmail shuttered when
> > writing to down the pipe to procmail (this is probably just a
> > special case of the garbage characters).
> > 
> > Not all messages get corrupted.  A little more half were corrupted
> > -- the others were fine.  IA little more half were corrupted
> > -- the others were fine.  Iwere corrupted
> > -- the others were fine.  I Irs were fine.  Iupted.  A little more half were corrupted
> > -- the others were fine.  Ited.  A little more half were corrupted
> > -- the others were fine.  I.  A little more half were corrupted
> > -- the others were fine.  Iupted.  A little more half were corrupted
> > -- the others were fine.  Ine.  I> > -- the others were fine.  Iupted.  A little more half were corrupted
> > -- the others were fine.  Ied.  A little more half were corrupted
> > -- the others were fine.  IA little more half were corrupted
> > -- the others were fine.  I I more half were corrupted
> > -- the others were fine.  I -- the others were fine.  Ire corrupted
> > -- the others were fine.  Irrupted
> > -- the others were fine.  I were corrupted
> > -- the others were fine.  Imore half were corrupted
> > -- the others were fine.  Ittle more half were corrupted
> > -- the others were fine.  I A little more half were corrupted
> > -- the others were fine.  I>> food.what()
I a> food.__class__.__bases__ = (Eggs, Spam)
>>> food.what()
I a> food.__class__.__bases__ = (Eggs, Spam)
>>> food.what()
I a
>>> food.__class__.__bases__ = (Eggs, Spam)
>>> food.what()
I a>> food.__class__.__bases__ = (Eggs, Spam)
>>> food.what()
I a aases__ = (Eggs, Spam)
>>> food.what()
I a
>>> food.__class__.__bases__ = (Eggs, Spam)
>>> food.what()
I aw°#øwÿÿÿÿÀÂF¡üwXG¤ÅFîð"øwÿÿÿÿàÁFà¥üw|ÁFq#øwXGG`ôGüÃFEûw°#øwÿÿÿÿÃF¡üwXGÜÅFîÄÁFÒ"øwGH[G ÁFˆGTÂFEûwð"øwÿÿÿÿdÂFà¥üwØ
GP[G˜ÂFôÂüøwüÂFÒ"øwGXôGœÂF.xøwÃFtÂFÃFôÄF`ôG¤ÅFoøwŸ´t%¯GèyøwxÂFôÂF€ÂFštøwØôGšÛw1202661000ÂF×ÌÏFEûwXxøw\ôÄFÐÄFpwøwn[GÈÄF€ÅFS-1-5-21-839522115-1957994488-1202660629-1000°#øwÿÿÿÿüØ
G€ÅFEGISTRYHINETWARE\CLASSES\WinNTNamespace\CLSIDNE\SOFTWARE\CLASSES\WinNTNamespace\¼ÅFÁwØÁwðÃFLÅFÜÄF0ÄF~/øwÁw¼ÄFØÁwÆw@Æw¼høw	ÆwÁwÁwØÁwDÄFøÄFøÄFÁw„ÄF~/øwÁwäÄFØÁwÁwÅF×jøwÁwäÄF¼ÅF*	ÈÆw	[email-masked]|ÎFüÆwœÄFÌÏFæÆwæÆwXÅFñ}âwæÆwêÆw;~âwæÆwèÆwÈÿÅw±háw¼Rèw0¢üÿÿÿ¤ÅFÚ ÍFùzâwÿÿÿÿ{ÁwÀtÁw Æw´zÁw¬Ž75?hÒ¨šÀO»Ï¢usÁw¨ñÅw ÆwPÏFTempryy Internet FilessBÆF@€DÈF;Ÿ¨ÊFŒ¡¥w@ÆFèËG¨ÊF[¢¥wQüwÿÿÿÿlÆFŒÆFG0
H@G”ÆFEq#øwxGGEØ!GlÆF$ÈFEûw°#øwÿÿÿÿ4ÈF¡üwxG
¨ÆFˆG\ÇFEûwð"øwxGxGà¥üwÇFXÈFìÆF`ÈFGøw€0ÈFÄ¡Ûw€ÔÎF¸ÆF|ÆFPGe°²Gì²GøÉFààüw)Aøw\ÇF(	$¤ÎF68ð+øwpáüwx>øw^>øwØ!GˆÇwøëýð)³wÇFœ³G„ÊF™qøwìýŒ³GCSwœÈF6 Ôw6T9G68¤ÎFqøwCžÈF6xÛwøëýàýàýÈÊFEûw€>øwÿÿÿÿØÊFØÊFð+øwpáüw|@øwT@øw¤ÎF$ÇFEq#øw0ÇFq#øwØGG¸×GÇFÀÈFEûw°#øwÿÿÿÿÐÈF¡üwØG ˆÇF
s destroyed?

> > After sending the same message to myself repeatedly and examining
> > the raw message in an mbox file, I began to see the following
> > pattern. The corrupted messages are always larger than they should
> > be and there are chunks of garbage characters in them.  Sometimes
> > the MIME separators are repeated as if fetchmail shuttered when
> > writing to down the pipe to procmail (this is probably just a
> > special case of the garbage characters).
> > 
> > Not all messages get corrupted.  A little more half were corrupted
> > -- the others were fine.  I also POP-ed directly into the mail
> > server and verified that the messages were not corrupt and only
> > became so when I used fetchmail to retrieve them.
> > 
> > I tried various versions of fetchmail (5.8.12, 5.8.14, 5.8.16) and
> > Cygwin (1.3.2, my CVS build, latest snapshot).  All exhibit the same
> > transient binary attachment corruption problem.
> > 
> > When I repeat the same experiments on my old Windows NT 4.0 SP5
> > machine, I have not received any corrupted email (yet).
> > 
> > Is anyone using fetchmail under Windows 2000?  If so, are you
> > experience any problems?
> 
> I don't use fetchmail, but with mutt and ssmtp all my sent binary
> attachments get corrupted on W2000SP1. Don't know why, didn't try it
> under NT.

Sorry, I don't know enough to help you out of this misere since i 
never used procmail, but io have some more questions.
Jason, do you use also use ssmtp as Olaf mentioned?
Anyway, since it only happens on NT5 and not on NT4 as you reported,
it seems not to be a problem with Fetchmail. Since Fetchmail does 
nothing else but retrieve and send out to a smtp server i could not 
think of big problems with this. Or is it an option to pipe directly 
from fetchmail to procmail?
Are the files stored to disk between retrieving and sending out?
Does Procmail store to disk before delivering?
Are your mounts of binary type?

Please report if you track down the source of the problems, maybe a 
problem with the pipes?

Olaf could you please report exactely what you do to send out a 
corrupt message? Is it a known issue? I never heard before of 
problems with mailing through ssmtp on win2k.
I will try tomorrow to send some mail with ssmtp and binaries 
attached.

Gerrit

-- 
=^..^=

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/