X-Recipient: archive-cygwin AT delorie DOT com
DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:date:from:reply-to:message-id:to:subject
	:in-reply-to:references:mime-version:content-type
	:content-transfer-encoding; q=dns; s=default; b=ZRDFXtpLbiRcIbv9
	+CtdOqLRYlHNrriZWmjhBw2F1dR+YIr/7TXU2q0M02bWh7snqCW88MRzXXrayzKo
	8idfuR5xjj1K91KFaW5rpUUZZCl6KlTYPr0WPYR//QCvkrpeC4WSHDYRIxFsRs88
	zo0jul5JHUX7PxUrjIqVUaeSNMM=
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:date:from:reply-to:message-id:to:subject
	:in-reply-to:references:mime-version:content-type
	:content-transfer-encoding; s=default; bh=WwWaE85NgKpwyS2j5qvHCt
	jVG6c=; b=H9mScJXDIjuP1exwLnvChImT3LEk7w37tkzu3DwmZiusgzGulrW045
	kyDhSj7lI1QtMXago0phfJar1eRm/u1O8tgNSD97C6oqJ7fAWz/iy9Lr5NTSdM7o
	Qid4cH+67DV4NAb8eyVNgOshixhuDV0VWRXG8ZHyJ0DFEHHjaHgxA=
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id: <cygwin.cygwin.com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com
Authentication-Results: sourceware.org; auth=none
X-Virus-Found: No
X-Spam-SWARE-Status: No, score=3.8 required=5.0 tests=AWL,BAYES_50,FREEMAIL_FROM,KAM_THEBAT,LIKELY_SPAM_SUBJECT,RCVD_IN_DNSWL_LOW,SPF_PASS autolearn=no version=3.3.2 spammy=German, german, ronald, Ronald
X-HELO: forward1j.cmail.yandex.net
Authentication-Results: smtp2p.mail.yandex.net; dkim=pass header.i=@yandex.ru
X-Yandex-Suid-Status: 1 0,1 0
Date: Wed, 24 May 2017 13:03:59 +0300
From: Andrey Repin <anrdaemon AT yandex DOT ru>
Reply-To: cygwin AT cygwin DOT com
Message-ID: <1221683706.20170524130359@yandex.ru>
To: Ronald Fischer <ynnor AT mm DOT st>, cygwin AT cygwin DOT com
Subject: Re: Bug: grep behaves incorrectly under the locale C.UTF-8, if a file contains  Umlaut characters
In-Reply-To: <1495612367.2760331.986814392.79C77EB2@webmail.messagingengine.com>
References: <1495612367 DOT 2760331 DOT 986814392 DOT 79C77EB2 AT webmail DOT messagingengine DOT com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
X-IsSubscribed: yes

Greetings, Ronald Fischer!

> I have a file X which contains ASCII text, but also in some lines German
> umlaut characters. The file is classified as:

>      $ file X
>      X: ISO-8859 text, with CRLF line terminators

> If I grep the file using, say,

>      $ grep  .  X  >Y

> (i.e. select every non-empty line and write the result to Y), this works
> fine, if LANG is set to one of: UTF-8, C, C.de_DE, C.en_EN, en_EN,
> de_DE.

> However, if LANG is set to C.UTF-8, two things happen:

> - grep classifies the file as binary file and produces the error message
> "Binary file X matches" 

This is an intended behavior, upstream decision since mid-2015, I recall.

> - Both the grepped lines (i.e. in our example the non-empty lines) AND
> the error message end up in the standard output (i.e. in file Y).

> IMO, there are several problems with this:

> 1. It's hard to see, why an umlaut character makes the file X binary
> under encoding C.UTF-8, but not under encoding UTF-8 or C.en_EN

> 2. If grep classifies a file as binary, I think the desired behaviour
> would be to NOT produce any output, unless the -a flag has been
> supplied.

> 3. If grep writes a message "Binary file ... matches", this message
> should go to stderr, not stdout. The stdout is supposed to contain only
> a subset of the input lines.
>  Ronald


-- 
With best regards,
Andrey Repin
Wednesday, May 24, 2017 13:02:39

Sorry for my terrible english...


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple