Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sources.redhat.com/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sources.redhat.com/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com
Date: Thu, 29 Aug 2002 20:48:15 -0400
From: Christopher Faylor <cgf AT redhat DOT com>
To: cygwin AT cygwin DOT com
Subject: Re: Bug? Mixed CR/LF and LF line endings from different programs
Message-ID: <20020830004815.GA25716@redhat.com>
Reply-To: cygwin AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
References: <20020829203820 DOT GB23580 AT redhat DOT com> <20020830001556 DOT 9D66C8B1F AT bellmann DOT research DOT canon DOT com DOT au>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20020830001556.9D66C8B1F@bellmann.research.canon.com.au>
User-Agent: Mutt/1.3.23.1i

On Fri, Aug 30, 2002 at 10:19:38AM +1000, luke DOT kendall AT cisra DOT canon DOT com DOT au wrote:
>On 29 Aug, Christopher Faylor wrote:
>>  On Thu, Aug 29, 2002 at 09:21:10AM -0700, Shankar Unni wrote:
>>  >Christopher Faylor wrote:
>>  >>awk and sed open their standard input in textmode.  This is by design.
>>  >
>>  >Don't they open their stdout in textmode, then?  Otherwise they should
>>  >have been "fixed up back" to \r\n when they wrote the lines out, no?
>>  
>>  I think you can draw your own conclusions on what is happening pretty
>>  easily.
>
>Yep.  I don't understand why, though.

awk and sed both do input text and, on output, default to the mode
derived from the mount table.  So, \r\n is changed to \n internally and
is changed to whatever is appropriate on output.  That means that a '$'
will match eol in sed but may still output a \r\n.

>matching - depending on the mount type - DOS or Unix, seems correct to
>me.  Yet awk and sed don't do it.  This suggests that they're not
>opening the output file in text mode,

They're not opening the output file in any mode.  The mount type of the
disk prevails.  Forcing them to write \r\n line endings is exactly the
wrong thing to do.  The default for cygwin should be binary out.
Actually they probably should both be using "automode" (input text,
output binary) but both packages probably predate that.

>which seems to contradict the Cygwin FAQ:
>
>    It is rather easy for the porter to fix the source code by supplying
>    the appropriate file processing mode switches to the open/fopen
>    functions. Treat all text files as text and treat all binary files
>    as binary. 
>
>Since awk and sed work on streams, I doubt that they'd be doing lseeks,

That section of the FAQ is out-of-date.  lseeks are not a problem.

>so in fact I can't imagine why they're not opening *input* files in text
>mode, too.  They *are* text processing tools, after all.

Huh?  I already said that they are opening input files in text mode.

>>  I just wanted to make sure that people understand that the behavior is
>>  not a random event.  It comes up from time to time here and I thought
>>  that it bears repeating that both are working the way they are designed
>>  to work.
>
>I understand that it's not random, but now I'm at a loss to understand
>why that behaviour was chosen.
>
>So, what's the recommended way for using Cygwin for any sort of text
>processing?

Mount everything in binmode if you want to only see \n line endings.

>If you work in Unix mode this wouldn't happen, but then *all* the files
>you produce won't be acceptable to lots of native applications, since
>they won't have the native line ending.

And, that is tough for native applications.  There is no magic bullet
here.

>If I write a program that filters input to output converting all line
>endings to the native style, that would "solve" this problem, but it
>means that every script will have to be altered, replacing almost every
>occurrence of ">" by "| sanitise >", which really is out of the
>question.

Oh well.

>Ah, hang on, I've just been poking through the user guide and found the
>CYGWIN=nobinmode option, that makes everything work as I would have
>expected.
>
>Whew!

Actually, if it does that's a bug.  nobinmode only works on things for
which no mount info can be derived, like pipes.

cgf

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/