X-Recipient: archive-cygwin AT delorie DOT com DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:content-type:mime-version:subject:from :in-reply-to:date:content-transfer-encoding:message-id :references:to; q=dns; s=default; b=nXn7UBq6kSBgZ1BxSGQME5qy82uy SoX1UWebQ+Fuvs1/uK+W7DfM8Ev5lF7SpCHdRuIDjpiR/Ix3o82kdWTsVLfXem6t jvy+2VfBmWUMqQPDpqkp9G21PVnzHaUfKBQny/JWb9ETpf+E36y+Aq7+ACUnN7KZ occ+PFuyV66noKA= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:content-type:mime-version:subject:from :in-reply-to:date:content-transfer-encoding:message-id :references:to; s=default; bh=y2WHt1xLOAgvUyuQTCx+oxGfADQ=; b=wW xV91+NYKlRgWw4q2gzp68OcTDBTdiErrLdZPALUIAeQKlSXNb26d2dOmZpC888V4 c+ydWBeYmVhVLcM+GsjKQJaWPuzWpuE0eTQvZPvG7fBxufBaa0wZ0dw3MlA95XZ4 iiAW/0kl+D7UvwAED4S+7r5VPxNikGPVBCCS2eQqo= Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner AT cygwin DOT com Mail-Followup-To: cygwin AT cygwin DOT com Delivered-To: mailing list cygwin AT cygwin DOT com Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=0.1 required=5.0 tests=AWL,BAYES_50,T_RP_MATCHES_RCVD autolearn=ham version=3.3.2 X-HELO: etr-usa.com Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: grep treating my text files as binary! From: Warren Young In-Reply-To: <549C5A6B.2000509@towo.net> Date: Fri, 26 Dec 2014 19:00:38 -0700 Message-Id: <27CE6A0A-9845-4A1C-A0F8-C0236B95A1E3@etr-usa.com> References: <549B4258 DOT 5050509 AT redhat DOT com> <549C5A6B DOT 2000509 AT towo DOT net> To: The Cygwin Mailing List X-IsSubscribed: yes Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by delorie.com id sBR212uq005437 On Dec 25, 2014, at 11:41 AM, Thomas Wolff wrote: > In any case the argument is quite artificial since the new behaviour > hits many files that are in fact text files. Please define the term “text file” in a way that allows a C programmer to write a program that automatically does the correct thing for all members of the class “text file” without involving locales, or an equivalent mechanism. Just because you, the human, can use your superior pattern matching skills to see that a file is a “text file” doesn’t mean that a relatively dumb program like grep(1) can. You can’t expect someone to build an AI system into grep(1) just to get automatic locale detection. If grep runs into a byte sequence that makes it think it is not legal for your current locale, it must treat the file as raw bytes, unless you give it -a. If you don’t like this behavior, say “alias grep=grep -a” in your ~/.bashrc, and forget the change ever happened. It’ll be on you when some non-text file gets treated as text and grep spams your terminal with binary garbage, though. This isn’t really a Cygwin problem. It just happens to affect it more than other *ix systems because there are two sets of rules on the same system, and they may conflict. But, if I go and copy a UTF-16LE file to a Linux box, I’m not going to complain to the grep bug list when grep doesn’t automatically do the right thing with it while $LANG contains UTF-8. Ultimately, the proper solution is to use UTF-8 on all systems you use Cygwin on. Many Unicode-aware native Windows programs that deal with text files can cope with UTF-8. If you have one of those that demands UTF-16LE, iconv(1) can make the conversion for you. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple