From: cgf AT cygnus DOT com (Christopher G. Faylor)
Subject: Re: When is \n converted to \r\n? and why?
1 Nov 1998 00:45:26 GMT
Message-ID: <71gav6$rrh$1@cronkite.cygnus.com>
References: <19981030020029 DOT 5367 DOT rocketmail DOT cygnus DOT gnu-win32 AT send102 DOT yahoomail DOT com>
X-Newsreader: trn 4.0-test63 (15 March 1998)

In article <19981030020029 DOT 5367 DOT rocketmail DOT cygnus DOT gnu-win32 AT send102 DOT yahoomail DOT com>,
Earnie Boyd <earnie_boyd AT yahoo DOT com> wrote:
>---Urban Widmark <urban AT svenskatest DOT se> wrote:
>>
>> (using cygwin32 19.3 & 19.1?)
>> 
>> If I create a small example file:
>> 
>> $ echo a > xx
>> $ ls -l xx
>> -rw-r--r--   1 544      everyone        3 Oct 28 15:43 xx
>> 
>> it will contain "a\r\n" since I use the default (non-binary mounts)
>> 
>> But if I do:
>> $ echo a | wc -c
>>       2
>> 
>> I get only 2 chars ... ok, so the translation is done when writing to
>> disk. Then something like this will fail:
>
>Actually, the translation is done at the time of read.  The \r is
>removed when processing in text mode and the read would return 2 even
>though 3 was actully read from disk.

No.  In this case the stdout of the echo command was a pipe.  The
shell in use has put the pipe into binmode which means that it will
not add a \r before the \n.  IMO, pipes should always be opened
in binmode.

>> $ echo a | tr -d '\r' > yy
>> $ ls -l yy
>> -rw-r--r--   1 544      everyone        3 Oct 28 15:48 yy
>
>Well, another improperly ported utility.  `tr' should open it's input
>and output in binary mode; obviously from this example it isn't.

Right.  It should be calling setmode(1, O_BINARY).  When tr is invoked
it inherits an fd.  It has no idea if that fd is associated with a file
in a directory that is mounted "binary" or not by this time.  The only
thing it can do is call setmode() on its stdout fd.

>>It's also strange that writing to a pipe is different from writing to
>>disk, that seems very non-unix'ish.  I think it would be better if
>>conversion could be toggled by changing an environment variable, it
>>should not depend on the destination of my output.
>
>Well, you are at the mercy of the filesystem and text mode processing
>will prefix a \r to every \n.  This is only a problem if the programs
>are not properly ported.

The "filesystem" has nothing to do with this.  The filesystem is under
NT control.  It is cygwin which is writing the \r stuff.  What gets
written depends on how the directory you are writing to is mounted,
on the setting of the CYGWIN binmode option, and on options to the
open and setmode functions.

>>An environment variable would be better than a registry entry, since
>>it's hard to write "non-conversion" shell scripts that can be run on
>>other machines if they depend on a registry entry ...  Also some
>>scripts may need the feature turned on, others off.
>
>Actually, I'd be in favor of not having either if that meant getting
>properly ported code.  I wonder if we could get this resolved prior to
>the b20 release?

If you're asking for something other than CYGWIN=binmode then I'm afraid
this won't be happening.
-- 
cgf AT cygnus DOT com
http://www.cygnus.com/