delorie.com/archives/browse.cgi   search  
Mail Archives: cygwin/2008/12/22/19:49:37

X-Recipient: archive-cygwin AT delorie DOT com
X-SWARE-Spam-Status: No, hits=0.8 required=5.0 tests=AWL,BAYES_00,J_CHICKENPOX_83,KAM_MX,URI_HEX
X-Spam-Check-By: sourceware.org
From: Allan Schrum <aschrum AT ecdeliverysystems DOT com>
To: "cygwin AT cygwin DOT com" <cygwin AT cygwin DOT com>
Date: Mon, 22 Dec 2008 19:48:04 -0500
Subject: RE: [1.7] Pipes intermittently lose data on Cygwin 1.7
Message-ID: <9BE596E8BDDC3443BF23B9678D03CC2928DF6E97@ECDS-CLT-MX1.ecdeliverysystems.com>
References: <494CB77F DOT 4040403 AT i12 DOT com> <494D561E DOT 9060607 AT i12 DOT com> <20081222173111 DOT GD27364 AT ednor DOT casa DOT cgf DOT cx> <494FDD16 DOT 4030903 AT i12 DOT com> <20081222184701 DOT GB23447 AT ednor DOT casa DOT cgf DOT cx> <494FF338 DOT 4020900 AT i12 DOT com>
In-Reply-To: <494FF338.4020900@i12.com>
MIME-Version: 1.0
X-IsSubscribed: yes
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id: <cygwin.cygwin.com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com
X-MIME-Autoconverted: from quoted-printable to 8bit by delorie.com id mBN0nZPY026633

> 
> >> Not yet. The bug is still present in the current snapshot
> >> cygwin-inst-20081220.tar.bz2. Thanks for trying to fix it and for
> >> responding to my post.
> 
> 
> On 081222 10:47, Christopher Faylor wrote:
> > Too bad.  Since I can't duplicate the problem it will be difficult to
> > fix it.
> 
> Have you tried running the examples I provided in my original post
> (with foo = ~ 5MB text file) on a DOS shell (cmd.exe)?
> 
> I suggest running each example (in a DOS shell) at least 10 times,
> comparing the file sizes of foo and bar each time. On my system, size
> of bar < size of foo roughly 25-75% of the time.
> 
> 
> > It would be helpful if you provided more information like the
> cygcheck
> > information requested in http://cygwin.com/problems.html.
> 
> Please see attached the output cygcheck -s -v -r > cygcheck.out. I ran
> cygcheck on the current snapshot cygwin-inst-20081220.tar.bz2.
> 
> It would also
> > be interesting to see the actual file sizes of foo and bar from your
> > example.  If they always vary in a consistent way that would be a
> clue.
> 
> File sizes from latest test I ran (Example 1 of my original post)
> 
> 	tr \32 \0 < foo | tr \0 \32 > bar
> 
> on current snapshot cygwin-inst-20081220.tar.bz:
> 
> 	foo = 5,138,895 bytes
> 	bar1 (runs 1-4, 6-7, 9-10) = 5,132,288 bytes
> 	bar2 (runs 5 and 8) = 5,136,384 bytes
> 
> As you can see, there is some consistency. When I ran the above test 10
> consecutive times, there were only two types of output. On 8/10 runs,
> bar was missing the same 6,607 (5,138,895 - 5,132,288) byte chunk. On
> the remaining 2/10 runs, bar was missing the same 2,511 byte chunk.
> 
> As mentioned in my original post, the missing chunk is always at the
> end (or beginning in the tac example) of bar. It's as if the pipe exits
> too early, before all the data can make it through.
> 
> I'm attaching foo, bar1, and bar2 from the above example (compressed
> with 7zip).
> 
> Greetings,
> Lawrence

A couple of observations that might trigger a clear thought. The numbers 5,132,288 and 5,138,895 are multiples of 4096. Assuming a standard 4096-byte block, these sizes of "bar" are files of size 1253 and 1254 blocks. The file "foo" only contains 1254 blocks with a remainder of 2511 bytes. Since the last block, or sometimes the last two blocks, are not copied, this might imply some type of race condition that only appears on Lawrence's machine. Christopher's machine might be faster and not subject to this race condition.

Lawrence: what is the hardware that your are running this test upon? What other processes are running at the same time (e.g. CPU usage, I/O usage, etc.)?

It would appear the leading "tr" writes the file "foo" to the pipe, and closes the pipe. This may cause the trailing "tr" to get a signal that it does not handle well before it can complete the read from the pipe. If the hardware is too slow, or occupied, then perhaps it happens because of this rather than on other hardware which might be faster (or perhaps the reverse).

Christopher: could you run the example in a mode where your computer is heavily loaded with other tasks?

Only a suggestion.

Regards,

-Allan

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/


- Raw text -


  webmaster     delorie software   privacy  
  Copyright © 2019   by DJ Delorie     Updated Jul 2019