X-Recipient: archive-cygwin AT delorie DOT com
X-Spam-Check-By: sourceware.org
Date: Fri, 27 Nov 2009 15:33:53 +0100
From: Corinna Vinschen <corinna-cygwin AT cygwin DOT com>
To: cygwin AT cygwin DOT com
Subject: Re: Problem [1.7]: Inconsistent and wrong results from e.g. ls and  md5sum
Message-ID: <20091127143353.GX29173@calimero.vinschen.de>
Reply-To: cygwin AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
References: <4B0F8C59 DOT 2090809 AT bonhard DOT uklinux DOT net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <4B0F8C59.2090809@bonhard.uklinux.net>
User-Agent: Mutt/1.5.20 (2009-06-14)
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
Precedence: bulk
List-Id: <cygwin.cygwin.com>
List-Unsubscribe: <mailto:cygwin-unsubscribe-archive-cygwin=delorie DOT com AT cygwin DOT com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com

On Nov 27 08:22, Fergus wrote:
> I am having ongoing problems with file processing using [1.7] with
> W7 on FAT32. [...]
> $ find . -type f | xargs md5sum | grep d41d8cd9
> d41d8cd98f00b204e9800998ecf8427e
> [...]
> 2) If I use ls to see whether, here too, there are any files of size
> 0 wrongly identified, I get
> 
> $ ls -alR | sed '/^d/d' | grep " 0 "
> /bin/ls: ./tetex/libkpathsea3/..: Not a directory
> $ ls -alR | sed '/^d/d' | grep " 0 "
> /bin/ls: ./GNOME/GConf2/GConf2-devel/.: Unknown error 3221225768
> [...]

IT'S ALL MY FAULT.  

That's one of those embarrassing bugs which take painful revenge by
being really hard to debug.

No wonder I could never reproduce this problem since it only manifests
on FAT or FAT32, not on NTFS.  And I dislike FAT32 a lot, so I usually
never test on it.

What happens is this:  On filesystems which don't support security
descriptors stat() has to read the first few bytes of a file to figure
out if it should set the execute bit.  This is primarily for the sake
of shell scripts.

At one point I found that the method used in stat() to read the file
is not thread-safe.  It moves the file position and moves it back
afterwards.  If the file is read at this point in time, shit happens.

So I made this functionality thread-safe.  Just wrong.  The problem was
that I missed to set specific flags when opening the file for reading
so that the file is accidentally read asynchronously.  So what?  Well,
the function didn't take this into account and buffer in which the file
content is saved is on the stack.  If the function happens to return and
the next function is called, and *then* the read function does its job,
the stack content of the new function gets overwritten in quite amusing
and ambiguous ways...

Thanks for the report.  I'm going to upload a -67 release today so that
you can test it ASAP.

You will notice that find and ls -R got slower on your FAT32 drives.
That the result of reading synchronously now.  I'm sorry, but either
it's fast, *or* it's correct, never both.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple