Subject: RE: B20.1: CR-LF translation not always handled properly
From: "Jon Leichter" <[email-masked]>
To: "Bernard Dautrevaux" <[email-masked]>
Cc: <[email-masked]>
Date: Fri, 22 Oct 1999 12:32:05 -0700

Bernard,

Thanks for your comments. This is EXACTLY what I have been talking about.

Thanks for pointing out that 'cat' is doing the right thing. I forgot the fact
that 'cat' is SUPPOSED to use binary mode.

I'm glad that you've been able to confirm my dilemma. I still have a problem
that remains. Although I can manually use a tool to suppress ^M characters
(like grep), there is no automated general solution (that I can think of) to
change the various 'configure' and 'libtool' scripts in the projects that I
build. I can manually edit these files, but I think that is the WRONG move.

Like I've said... for the time being, I've been using Mingw32 support provided
by Cygwin. I'd prefer to use the standalone Mingw32 package, but until these
textmode/binmode translations are fixed, that just won't be possible.

To summarize, you and I both believe that the following tools are 'broken':

- bash, sh, ash (command substitution)
- sed

There are probably others, but I like I said, I haven't done an extensive job
of auditing the other tools.

In the case of 'cat', perhaps a switch could be added to the tool to
specifically use textmode when specified. This would be useful to add to
scripts where you 'knew' you were processing text files.

Jon Leichter
[email-masked]

> -----Original Message-----
> From: [email-masked]
> [mailto:[email-masked]]On Behalf Of Bernard
> Dautrevaux
> Sent: Friday, October 22, 1999 12:09 PM
> To: 'Jon Leichter'; [email-masked]
> Subject: RE: B20.1: CR-LF translation not always handled properly
>
>
> > -----Original Message-----
> > From: Jon Leichter [mailto:[email-masked]]
> > Sent: Friday, October 22, 1999 1:34 PM
> > To: [email-masked]
> > Subject: RE: B20.1: CR-LF translation not always handled properly
> >
> >
>
> 	<skipped>
>
> > - At the command prompt, type the following:
> >
> >   $ xx=`gcc -print-prog-name=ld`
> >   $ echo $xx | od -x
> >
> > This step ought to clearly show (by looking at the hex dump)
> > that the xx
> > variable has been set to the value "ld^M", which is completely wrong.
>
> I agree, `` should read in text mode
>
> >
> > Now, try this:
> >
> >   $ echo $xx | cat | od -x
> >
> > Notice no change.
>
> This is correct. By definition 'cat' was expected to correctly handle binary
> data, so it _must_ read the pipe in binary mode. No problem here; it's
> execatly what it should be.
>
> >
> > Now try this:
> >
> >   $ echo $xx | grep ".*" | od -x
> >
> > Notice that the ^M has (correctly) been stripped.
>
>
> This is correct also; grep is handling textual data, so correctly set its
> input pipe to read in text mode.
>
> >
> > Not only should 'cat' have done the right thing, but the
> > orignal command
> > substitution should never have ended up with the ^M character.
>
> 'cat' is ok, but bash (or ash, depending on what you use as /bin/sh) _must_
> use text mode when reading the pipe used for command substitution. This
> thread was already followed seevral times. I don't think theer is a problem
> here with cygwin1.dll :-), but rather with bash/ash command substitution
> ;-<.
>
> >
> > The Mingw32 tools produce ^M characters in its output, and there isn't
> > anything that can be done about that. It is the
> > responsibility of the Cygwin
> > tools (especially the ones that process text) to make sure
> > that stdin and
> > stdout have been opened in textmode, and this simply does not
> > seem to be
> > happening with all of the tools.
>
>
> And this is not only the case of mingw32. Using a Windows box mean we are
> interested in accessing Windows resources. You'll get the same problems if
> some Windows application generate a list of items and I try to handle it by:
>
>     for i in `cat windows-generated-file.txt`; do
> 	etc...
>
> 'cat' is correct in handling 'windows-generated-file.txt' in binary mode as
> he has no way to know its text; but the shell _knows_ it's text so it must
> read it as text.
>
> >
> > Someone please try these tests and let me know if you see the
> > same behavior
> > that I have described.
>
> I've seen this regularly and have found no other solution as filtering
> through some '^M'-suppressing program (although I've not think at grep for
> this purpose).
>
>
>
> Hope this helps clarify what the problem is; I think cygwin1.dll is OK, but
> 'sh' is not.
>
> Regards,
>
> 		Bernard
>
> --------------------------------------------
> Bernard Dautrevaux
> Microprocess Ingéniérie
> 97 bis, rue de Colombes
> 92400 COURBEVOIE
> FRANCE
> Tel:	+33 (0) 1 47 68 80 80
> Fax:	+33 (0) 1 47 88 97 85
> e-mail:	[email-masked]
> 		[email-masked]
> --------------------------------------------
>
> --
> Want to unsubscribe from this list?
> Send a message to [email-masked]
>

--
Want to unsubscribe from this list?
Send a message to [email-masked]