Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sources.redhat.com/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sources.redhat.com/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com
Date: Mon, 05 May 2003 13:58:33 -0400
From: Jason Tishler <jason AT tishler DOT net>
Subject: vsftpd corruption problem (Re: proftpd (and vsftpd too))
In-reply-to: <20030422153812.GR11137@cygbert.vinschen.de>
To: cygwin AT cygwin DOT com
Mail-followup-to: cygwin AT cygwin DOT com
Message-id: <20030505175833.GI556@tishler.net>
MIME-version: 1.0
Content-type: text/plain; charset=us-ascii
Content-transfer-encoding: 7BIT
Content-disposition: inline
User-Agent: Mutt/1.4i
References: <20030109164943 DOT GA3100 AT redhat DOT com>
 <20030109182038 DOT G21839 AT cygbert DOT vinschen DOT de>
 <20030113143459 DOT GE1012 AT tishler DOT net>
 <176380637637 DOT 20030421225541 AT familiehaase DOT de>
 <20030421222353 DOT GF11137 AT cygbert DOT vinschen DOT de>
 <9326520504 DOT 20030422071811 AT familiehaase DOT de>
 <20030422115156 DOT GB1696 AT tishler DOT net>
 <20030422121232 DOT GK11137 AT cygbert DOT vinschen DOT de>
 <20030422144225 DOT GA552 AT tishler DOT net> <20030422153812 DOT GR11137 AT cygbert DOT vinschen DOT de>

On Tue, Apr 22, 2003 at 05:38:12PM +0200, Corinna Vinschen wrote:
> On Tue, Apr 22, 2003 at 10:42:25AM -0400, Jason Tishler wrote:
> >     1. vsftpd.conf must be in binary mode on a binary mount
> >     2. file transfers from/to text mounts can cause "corruption"
> >     3. ASCII transfer mode does not seem to work properly
> > [...]
> > I'm willing to contribute vsftpd with the above caveats and the
> > intention to solve the binary/text issues as time permits.  Is this
> > acceptable?
> 
> As a "test" release, sure!

I may have to renege on the above offer or at least add a *big* caveat
to the README.  Over the last few months, we have transferred tens of
thousands (possibly hundreds of thousands) of "normal" sized (i.e.,
<=100K) files with vsftpd put tunneled through ssh without any problems.
Unfortunately, I have recently discovered a corruption problem when
transferring large files (i.e., >=200MB) with vsftpd get tunneled
through ssh.

So far, I have been able to only trigger the problem on 4-way, NT 4.0
boxes with get when tunneled through ssh on a "slowish" (e.g., ~600
KB/s) network connection.  I cannot reproduce the problem on 1-way
systems, 2000, without ssh, put, or fast network connections (e.g., same
switch).

By constructing a specially formating file, I was able to determine that
after about 150 MB is transferred, a chunk of the file is erroneously
resent.

The special file contains lines with the following format:

    LLLLLLL:12345678901234567890123456789012345678901

where LLLLLLL is the line number zero padded to 7 digits.

The corruption looks like the following:

    3327913:12345678901234567890123456789012345678901
    3327914:12345678901234567890123456789012345678901
    3327915:12345678901234567890123456789012345678901
    3327916:12345678901234567890123456789012345678901
    3327917:12345678901234567890123456789012345678901
    33271234567890123456789012345678901
    3326608:12345678901234567890123456789012345678901
    3326609:12345678901234567890123456789012345678901
    3326610:12345678901234567890123456789012345678901
    3326611:12345678901234567890123456789012345678901
    3326612:12345678901234567890123456789012345678901

So in this case, after starting to transfer line 3327918, for some
reason vsftpd started to transfer data from line 3326607 and onward
again.  But, the file is otherwise not corrupt.  BTW, I have noticed a
200 MB file will be exactly 64 KB larger and a 300 MB file will be
exactly 128 KB larger.

Any ideas on how to debug this further will be greatly appreciate.

BTW, since proftpd CVS works under Cygwin now, should I just give up on
vsftpd and save my head banging for other more productive activities?

Thanks,
Jason

-- 
PGP/GPG Key: http://www.tishler.net/jason/pubkey.asc or key servers
Fingerprint: 7A73 1405 7F2B E669 C19D  8784 1AFD E4CC ECF4 8EF6

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/