X-Recipient: archive-cygwin AT delorie DOT com
X-SWARE-Spam-Status: No, hits=-0.8 required=5.0	tests=AWL,BAYES_00,DKIM_SIGNED,DKIM_VALID,FREEMAIL_FROM,KAM_VIAGRA1,RCVD_IN_DNSWL_LOW,RFC_ABUSE_POST,T_TO_NO_BRKTS_FREEMAIL
X-Spam-Check-By: sourceware.org
MIME-Version: 1.0
In-Reply-To: <4DD36619.1010401@hima.com>
References: <4DD36619 DOT 1010401 AT hima DOT com>
Date: Tue, 24 May 2011 06:20:03 +0200
Message-ID: <BANLkTino_ZBUjCzbrmLezRJqjz2Y+04DCQ@mail.gmail.com>
Subject: Re: 1.7.9: Problem with line endings of Perl output redirected to a file with textmode mounting
From: Reini Urban <rurban AT x-ray DOT at>
To: cygwin AT cygwin DOT com
Cc: pp <perl5-porters AT perl DOT org>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
X-IsSubscribed: yes
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
Precedence: bulk
List-Id: <cygwin.cygwin.com>
List-Unsubscribe: <mailto:cygwin-unsubscribe-archive-cygwin=delorie DOT com AT cygwin DOT com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com
Note-from-DJ: This may be spam

2011/5/18 Sven Severus:
> let me report a strange behaviour with Cygwin Perl (I'm using cygwin1.dll
> 1.7.9-1, full installation 2 weeks ago).
>
> File foo.h is an ordinary text file, all lines are terminated with DOS
> style line endings <cr> <lf> (hex: 0d 0a).
> It is located in a directory with textmode mounting in cygwin.
> One <cr> <lf> sequence of foo.h is split by a 4096 byte boundary within
> the file: "od -c -Ax foo.h" shows a <cr> (=3D'\r') at byte offset 4095
> (0xfff)
> and a <lf> (=3D'\n') at offset 4096 (0x1000):
> ...
> 000ff0 =A0 / =A0 / =A0 / =A0 / =A0 / =A0 / =A0\r =A0\n =A0 / =A0 / =A0 X =
=A0 X =A0 X =A0 X =A0 X =A0\r
> 001000 =A0\n =A0 / =A0 / =A0\r =A0\n =A0 / =A0 / =A0\r =A0\n
> 001009
>
> Now I issued the command "perl -pe 's/12345/54321/' foo.h >foomod.h"
> to produce foomod.h, located in the same directory as foo.h, thus with
> textmode mounting too.
> When I examined the result, I noticed that foomod.h was one byte bigger
> then foo.h. I expected identical size, and "od -c -Ax foomod.h" reports:
> ...
> 000ff0 =A0 / =A0 / =A0 / =A0 / =A0 / =A0 / =A0\r =A0\n =A0 / =A0 / =A0 X =
=A0 X =A0 X =A0 X =A0 X =A0\r
> 001000 =A0\r =A0\n =A0 / =A0 / =A0\r =A0\n =A0 / =A0 / =A0\r =A0\n
> 00100a
>
> Ups! The original <cr> <lf> sequence starting at offset 4095 (0xfff)
> became a three character sequence <cr> <cr> <lf>! The <cr> is duplicated!
>
> In other files created by Perl with output redirection I observed this
> behaviour with every <cr> <lf> line ending, that is split by a 4096 byte
> boundary (even multiple times in one output file). Line endings not
> split by a 4096 byte boundary do not show this behaviour.
>
> The behaviour does not occur, when the destination file is located
> in a directory with binmode mounting. It does not occur either, when
> I use sed instead of Perl ("sed -e 's/12345/54321/' foo.h >foomod.h"),
> so I think the problem is specific to Cygwin Perl, not to Cygwin in
> general.
>
> I this a bug of the output buffering mechanism of Cygwin Perl?
> Or do I anything wrong?
> Any answer is highly appreciated. Thanks in advance.

Yes, this looks like a PerlIO buffering bug for MSWin32 and cygwin.
The last char of the buffer is not stored when checking the first char
of the new buffer.
I think first we have to provide a sample test case to perl core.
--=20
Reini Urban
http://phpwiki.org/=A0 =A0 =A0 =A0 =A0=A0 http://murbreak.at/

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple