Date: Wed, 25 Jul 2007 10:19:49 -0400
Message-Id: <200707251419.l6PEJnYK011631@linode.hodain.net>
To: cygwin AT cygwin DOT com, ebb9 AT byu DOT net
In-reply-to: <46A64F0F.20706@byu.net> (message from Eric Blake on Tue, 24 Jul 	2007 13:12:15 -0600)
Subject: Re: 1.5.24: incorrect default behavior of dd in popen context on  text-mounted filesystem
From: hsw AT hodain DOT net
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: 7bit
References: <200707241548 DOT l6OFmXAE008571 AT linode DOT hodain DOT net> <46A64F0F DOT 20706 AT byu DOT net>
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
Precedence: bulk
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com

Eric Blake writes:
> According to Hugh Secker-Walker on 7/24/2007 9:48 AM:
> > Firstly, a thank you to Eric Blake for his quick response to the
> > dd problem we uncovered yesterday.
> ...
> Why not also thank cgf, who uploaded a fixed gzip yesterday?
> Because of his fix, this should now work:
> 
> popen("gzip > popenbug.out.gz", "w")

Yes, in my rush to try out the modified dd I did not notice that cgf
had quietly fixed the binary issue in gzip.  So, thank you to cgf for
so quickly fixing the original problem.


> Did you not read my statement that:
> 
> [io]f= unspecified - no change to existing mode of std{in,out}
> 
> I did not change that behavior, nor did I claim to change it in
> 6.9-4.  I am still open to the idea of changing it.  For 6.9-4, I
> only changed the behavior for when of= but not oflag= was specified.

I read your statements, albeit quickly and with an eye to a
consistency of specs.  I'm trying to resolve the meaning of three
different specifications about the default behavior of programs
(e.g. dd, gzip) using a text-mounted filesystem with CYGWIN=binmode
env variable:

1) In the Cygwin User's Guide, page 33:

    c. Pipes and non-file devices are opened in binary mode, except if
    the CYGWIN environment variable contains nobinmode.

	Warning!
	In b20.1 of 12/98, a file will be opened in binary mode if any
	of the following conditions hold:
	1. binary mode is specified in the open call
	2. the filename is a MS-DOS filename
	3. the file resides on a binary mounted partition
	4. CYGWIN contains binmode
	5. the file is not a disk file

    d. When redirecting, the Cygwin shells uses rules (a-e) [sic]. For
    these shells the relevant value of CYGWIN is that at the time the
    shell was launched and not that at the time the program is
    executed.

My reading of this says that I should expect dd to use binary mode on
its input and output files.  And I should expect that stdin and stdout
from shell-launched programs will be in binmode, so that
popen("gzip|dd>file", "w") will use binmode.  Please explain if my
interpretation is incorrect.

2) In http://cygwin.com/ml/cygwin/2007-07/msg00610.html Eric wrote:

    [io]f= unspecified - no change to existing mode of std{in,out}

My understanding is that the "existing mode" of stdin/stdout will be
binary (given what the User's Guide says), so it appeared to me that
dd was actively changing stdout back to text....

3) In http://cygwin.com/ml/cygwin/2007-07/msg00610.html Eric wrote:

    ....  Look for coreutils 6.9-4 coming soon to a mirror near you,
    with dd once again defaulting to full binary operation.

which I took to be further confirmation of dd using binary mode unless
otherwise specified.  I understand now that you meant this comment to
apply only to files that dd opens, but I took this as a more blanket
statement and further confirmation that dd does whatever in needs to
do to implement the User's Guide spec.


So I'm trying to understand the state of things.  AFAICT, the spec in
the User's Guide must not be being honored by popen().  Is that the
case?  Otherwise, why would dd's stdout in popen("gzip|dd>file", "w")
suffer text-mode modification?  And why could gzip be patched to fix
things?  And why can Eric "be open" to the idea of changing dd's
behavior in this case?  

I don't know much about Cygwin internals.  Is there a bug in popen()?
Or, is it the case that each executable is responsible for ensuring
that it honors the shell redirecting specifications in the User's
Guide?  Or is my reading of the User's Guide incorrect?

Thanks again for your responsiveness.

-Hugh

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/