X-Recipient:	archive-cygwin AT delorie DOT com
DomainKey-Signature:	a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:mime-version:in-reply-to:references:from:date
	:message-id:subject:to:content-type; q=dns; s=default; b=UWacXeJ
	9zs8q0xz94FkC1TS7le0BRlRINyI7eDUrJq0Y/dlgvLe3N3tREr8aUoKaxcIZvtz
	rMUgokshYLIk0NpVAZ7R4/30DF2ZBq4YgvKBZh5B24s6cV6ZTt2JsI+IKsdydKGi
	5Uj2Yil09mBlxJPu8eEIFXDLsIB37rzAUImw=
DKIM-Signature:	v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:mime-version:in-reply-to:references:from:date
	:message-id:subject:to:content-type; s=default; bh=H7I1XGVrPnv0E
	AoxPiaB06Zn+hk=; b=dNCsVWVzC0T8qR8ycqTTsFmPOem3RKYecjJJ8zYK0ag5g
	FRmlCnMbxfhNRat5ZgcKxs57EanGcXcLLCu+bKE859S8z5QIfMeHE9gsuh+oOvXI
	dmJlmXHaynWWKTni+/W7MoXd/qocOiHQOGfhcBIJo/UaAgZsehSeRP3vPsBH/U=
Mailing-List:	contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id:	<cygwin.cygwin.com>
List-Subscribe:	<mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive:	<http://sourceware.org/ml/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender:	cygwin-owner AT cygwin DOT com
Mail-Followup-To:	cygwin AT cygwin DOT com
Delivered-To:	mailing list cygwin AT cygwin DOT com
Authentication-Results:	sourceware.org; auth=none
X-Virus-Found:	No
X-Spam-SWARE-Status:	No, score=-0.0 required=5.0 tests=AWL,BAYES_50,FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,SPF_PASS autolearn=ham version=3.3.2 spammy=kitchen, viable, portability, eating
X-HELO:	mail-it0-f52.google.com
X-Google-DKIM-Signature:	v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=Z0AooWgfAoyGHZFzuWiUNxs1xZ8XxZuJ2uqAoQOTWXM=; b=cnxobFSkKXno4ccy2QQiqreEXOp6oWJIkbTvC5RaqywZvArZzx4ivZCZrqJwxuTqIQ Oqu1m3hoYNASoAGzKcjUDabSzDVK73WLuAwOwMRpzuSYZX0vx/glP9lHeU1I3VYeqdPs buBpAFG+0YxYG9HRs+BAA+YY7kwBZK5Xf7Op80rhjUcb/XZJfxqoI1ePEzfUa5pupZ0k cOcXKjMzqngblBALynf4il4kEmkXCctcdC91dBD/UeKQj6RcFs2xPfuaonQ9vuUUT9ir M6qszl1NiZzBvuMdKSUKSEJimdlBVp1DdjJF/StO16WF3rpWb+zuMxLmih5fhLVd3FXP OkVg==
X-Gm-Message-State:	AE9vXwPN6F/+7yw+U6yLpaiv56m32rpv5rZsJ4GBRa2C0tTHsQEw1j5X6SKIjuNzwC4sNF1ANHQ626HGfi5fRA==
X-Received:	by 10.107.53.213 with SMTP id k82mr1865091ioo.133.1473118628643; Mon, 05 Sep 2016 16:37:08 -0700 (PDT)
MIME-Version:	1.0
In-Reply-To:	<CAD8GWsuUSSTWCtETNZG4KmpHkrGtm4Dy6vgdjOcRHtTTFX2oMA@mail.gmail.com>
References:	<CAPTiy3PW0+bk20d30-J=sLr=PPnFOeU0Fb495hh1bBVtbtPrzw AT mail DOT gmail DOT com> <93be816b-952c-20cd-575e-940cdf4fbbd1 AT redhat DOT com> <e49f6018-a054-85c1-7856-cd81e4f54fa2 AT gmail DOT com> <1de2efdc-b26c-4914-580c-1a640d0a46fd AT redhat DOT com> <7314ffba-a927-f565-e38e-7454d6c2ef0f AT gmail DOT com> <CAPTiy3PjvXPZMisP_haTFjJ9tHQzu40qdcTo3M7xW6P-NDnoDQ AT mail DOT gmail DOT com> <CAD8GWsuUSSTWCtETNZG4KmpHkrGtm4Dy6vgdjOcRHtTTFX2oMA AT mail DOT gmail DOT com>
From:	Gene Pavlovsky <gene DOT pavlovsky AT gmail DOT com>
Date:	Tue, 6 Sep 2016 02:37:08 +0300
Message-ID:	<CAPTiy3PH-t0H87=t_SQ9m2AiZ=zdYs+MGemsQR5CxWp3Gd-Tdw@mail.gmail.com>
Subject:	Re: Script broken after updating bash to 4.3.46-7?
To:	cygwin AT cygwin DOT com
X-IsSubscribed:	yes

Lee, you have a point.

Adding mysql.sh that runs `mysql.exe | d2u`, or using a cygwin version
of mysql looks like the most viable solution at the moment.
Or staying with the older version of bash, in which `read` still
handled `\r\n` line endings...
My point is OK I'll fix that script, who knows what other scripts
might be broken now? And each one might need an investigation before I
find out which program's windows version needs a wrapper script etc.

--Gene

On 4 September 2016 at 20:37, Lee <ler762 AT gmail DOT com> wrote:
> On 9/4/16, Gene Pavlovsky wrote:
>> This issue never affected me personally, but it sounds like a serious
>> bug and I'm as glad as anybody that it's finally fixed.
>> However, having existing scripts suddenly breaking is not great. I'd
>> like to remind that I'm not the author of the script in question,
>> [AutoMySQLBackup](http://sourceforge.net/projects/automysqlbackup/).
>> If I put "| d2u" there I'll have to remember it and do it every time
>> automysqlbackup is updated ...
>
> Right - updating something like automysqlbackup to add "| d2u" after
> every dos program call is a non-starter.  But would putting the dos
> program inside a script that converted dos -> unix line endings work?
> eg - have a mysql script that comes before the windows version of
> mysql in your path that does something like
> /path/to/windows/version/of/mysql $@ | dos2unix
>
>> - or create and maintain a Cygwin package for this script.
>
> It seems to me that it isn't the script that's broken - it's the whole
> idea of having a dos program transparently integrate into an
> environment that expects unix line endings that's broken.  So until
> there's a bash option that automatically translates '\r\n' line
> endings into '\n' line endings you're stuck doing some kind of
> work-around or using a cygwin version of mysql.
>
> Lee
>
>
>> And who knows how many other scripts might be broken,
>> I just didn't find it yet?
>>
>> Having a `read`-specific shell option telling read to treat `\r\n`
>> (and only `\r\n`, not \r followed by something else) same as `\n`
>> would be bad things to have? For me, this kind of option would be more
>> useful than the `igncr` crutch
>> Let me say it another way - in OOP programming, one of good practices
>> is Single Responsibility Principle - a class should be responsible for
>> only one feature/function, and that feature/function should be totally
>> encapsulated in that class. Similar to that, an option should be
>> responsible for one behavior. With this change to `read`, the `igncr`
>> shell option is starting to look like a kitchen sink... split it into
>> separate options, please!
>> I think making UNIX scripts work on Cygwin with no or minimum
>> modifications (or bug-hunting) should be one of high priorities, no?
>> If some scripts erroneously have CRLF line endings, it's easy to find
>> and `d2u` them, rather than using the `igncr` crutch, but with the
>> recent change to `read`, countless scripts might be broken in a
>> non-obvious way. Fixing them would require finding out they're broken,
>> in the first place. Imagine if I didn't set up my cron to e-mail me
>> the cron jobs output? My backup script would just stop working
>> silently, and some time later when I needed a recent backup, I would
>> find out there aren't any. There might be something else lurking that
>> I haven't found yet. Once a script, broken by this change to `read`,
>> is found, it must be checked thoroughly to find out where exactly is
>> the problem, where to put '| d2u', or maybe 'set -o igncr'. These
>> fixes must also be applied anytime a 3rd party script is updated.
>> Quite a lot of work!
>> Hope you will consider my point.
>> Regards,
>> Gene.
>>
>> On 30 August 2016 at 23:57, cyg Simple <cygsimple AT gmail DOT com> wrote:
>>> On 8/30/2016 1:38 PM, Eric Blake wrote:
>>>> On 08/30/2016 12:04 PM, cyg Simple wrote:
>>>>> On 8/29/2016 2:30 PM, Eric Blake wrote:
>>>>>>
>>>>>> Simplest fix:
>>>>>>
>>>>>> read ... < <(mysql ... | dos2unix)
>>>>>>
>>>>>
>>>>> This will break when the data returned by mysql is supposed to contain
>>>>> \r.
>>>>>
>>>>>> There. Now you aren't feeding \r to read in the first place.
>>>>>>
>>>>>
>>>>> But you might want to feed \r to read.  It isn't a fix, it is a
>>>>> potential work around dependent on the data set results.  If a read
>>>>> that
>>>>> is supposed to be reading binary data doesn't pass all of the data to
>>>>> the routine then it is broken.
>>>>
>>>> Now we're talking past each other.
>>>>
>>>> That's what the recent bash fixed.  'read' in bash 3.2.42-4 was broken -
>>>> it corrupted binary data, with no recourse, by eating \r (and worse, by
>>>> sometimes eating the byte after \r).  'read' in bash 3.2.46-7 is fixed -
>>>> by default it is strictly binary (all bytes are read as-is, including
>>>> \r), but can also be switched to text mode (using 'igncr', all \r are
>>>> ignored).  If you want to preserve mid-line \r but treat line endings of
>>>> \r\n as a single byte, then leave binary mode on and strip the line
>>>> endings via a separate tool like d2u (note, however, that it is very
>>>> rare to have data where mid-line \r is important but line-ending \r\n
>>>> should be treated as plain \n).
>>>>
>>>> I strongly think that using igncr is a crutch, and you normally
>>>> shouldn't use it; particularly not if you want to be portable to other
>>>> platforms.  Instead, massaging your data through d2u is a great way to
>>>> be portable.  But sometimes the ease of ignoring ALL \r is easier than
>>>> worrying about portability, so I keep the 'igncr' code in Cygwin.
>>>>
>>>> And it is only because the OP tried using 'igncr' in the first place
>>>> (whether or not it was actually needed) that we have now flushed out the
>>>> existence of a latent bug in the 'igncr' implementation that interacts
>>>> weirdly with $()\n in PS1.  On that front, I'm still hoping to find time
>>>> to debug and/or for someone to post a patch.  But whether PS1 behaves
>>>> weirdly under 'igncr' is orthogonal to my suggestion above - using
>>>> 'mysql|d2u' is a great way to avoid the need to worry about 'igncr'.
>>>>
>>>
>>> Thank you for the retort Eric.  Happy to know that it is fixed which in
>>> the back of my mind I knew already.  I can imagine data such as full
>>> message email or a small document data containing \r\n as valid data in
>>> the database field and if you use a line ending conversion utility you
>>> might loose that data.
>>>
>>> --
>>> cyg Simple
>>>
>>> --
>>> Problem reports:       http://cygwin.com/problems.html
>>> FAQ:                   http://cygwin.com/faq/
>>> Documentation:         http://cygwin.com/docs.html
>>> Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
>>>
>>
>> --
>> Problem reports:       http://cygwin.com/problems.html
>> FAQ:                   http://cygwin.com/faq/
>> Documentation:         http://cygwin.com/docs.html
>> Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
>>
>>
>
> --
> Problem reports:       http://cygwin.com/problems.html
> FAQ:                   http://cygwin.com/faq/
> Documentation:         http://cygwin.com/docs.html
> Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
>

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

- Raw text -