X-Recipient:	archive-cygwin AT delorie DOT com
DomainKey-Signature:	a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:reply-to:subject:references:from:to:message-id
	:date:mime-version:in-reply-to:content-type
	:content-transfer-encoding; q=dns; s=default; b=N2lDegF1mqz5tKci
	Yl5xnhMHhAAXfjOKYOiQZpInkSzGSchly7TCwvqZ+oxglCUhem5W9cwGKwS5Jnl8
	L30G4+Ptf8jw4RaFTAayq+TNYs0o3IDnroVqINmU5XBeHk+Owg9DSN7o2Na2tou5
	u4KWlGaTyjYE+wRIN0HP+7zPwoE=
DKIM-Signature:	v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:reply-to:subject:references:from:to:message-id
	:date:mime-version:in-reply-to:content-type
	:content-transfer-encoding; s=default; bh=bEPGMFWyJVuYdLnJGcwPwZ
	Afonw=; b=WyfraioI4xJgM+23ULy4sgrcL6SU8tRViAqRUa+izeAe4L5CdY6AwG
	QlZ6GFuvLIHl+qSdMNLO1ZiJjDjpD0KS5j0WeBTbLN27kUusVH2CqVJB+UEHfdnz
	HZTknz2NVvBKbGha7lXl3zm54OexAwuBdxVqXlteipnumju1x8cCw=
Mailing-List:	contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id:	<cygwin.cygwin.com>
List-Subscribe:	<mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive:	<http://sourceware.org/ml/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender:	cygwin-owner AT cygwin DOT com
Mail-Followup-To:	cygwin AT cygwin DOT com
Delivered-To:	mailing list cygwin AT cygwin DOT com
Authentication-Results:	sourceware.org; auth=none
X-Virus-Found:	No
X-Spam-SWARE-Status:	Yes, score=6.4 required=5.0 tests=AWL,BAYES_50,BODY_8BITS,GARBLED_BODY,KAM_LAZY_DOMAIN_SECURITY,LIKELY_SPAM_SUBJECT,RCVD_IN_DNSWL_LOW autolearn=no version=3.3.2 spammy=iso88591, iso-8859-1, 2.6.0, unreadable
X-HELO:	smtp-out-no.shaw.ca
X-Authority-Analysis:	v=2.2 cv=Qb8khYTv c=1 sm=1 tr=0 a=WqCeCkldcEjBO3QZneQsCg==:117 a=WqCeCkldcEjBO3QZneQsCg==:17 a=IkcTkHD0fZMA:10 a=3YTwEyP03cqNC2KE9nAA:9 a=QEXdDO2ut3YA:10
Reply-To:	Brian DOT Inglis AT SystematicSw DOT ab DOT ca
Subject:	Re: Cygwin 2.6.0: unreadable UTF-8 in Windows console
References:	<123291584 DOT 20161001051347 AT vanav DOT org> <f4712f19-ef37-2040-1cda-3e352f09c8cd AT SystematicSw DOT ab DOT ca>
From:	Brian Inglis <Brian DOT Inglis AT SystematicSw DOT ab DOT ca>
To:	cygwin AT cygwin DOT com
Message-ID:	<d1da6d0e-380b-ec15-7fac-89747f04dc30@SystematicSw.ab.ca>
Date:	Fri, 30 Sep 2016 23:15:02 -0600
User-Agent:	Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0
MIME-Version:	1.0
In-Reply-To:	<f4712f19-ef37-2040-1cda-3e352f09c8cd@SystematicSw.ab.ca>
X-CMAE-Envelope:	MS4wfH238VGQbtM5zNc2p1pkYJO4FbNHxkDRSRPeLbjuHATf5QmdXgWfsQQE32zUkPVoZj29CgKv8eYgwBibqHUbmt05FWR7wD6X9SX5WFvpLDz8og8cClYf U74uwQES3vyEIBazKScAwY+EgOIvMJr/POXp6PwMsreBJ1fiRkTG0YWiC+o1zvJT+l0eKzQzayVnKA==
X-IsSubscribed:	yes
Note-from-DJ:	This may be spam

On 2016-09-30 22:34, Brian Inglis wrote:
> On 2016-09-30 20:13, Ivan Vanyushkin wrote:
>> Something has changed in version 2.6.0, and now UTF-8 text can't be displayed in Windows console (cmd).
>> 1. Create a file "test.txt" with non-ASCII text in UTF-8 encoding.
>> 2. Run "cmd".
>> 3. Run:
>> C:\Cygwin\bin\cat test.txt
>> â–’â–’â–’â–’â–’â–’â–’â–’â–’â–’â–’â–’â–’â–’â–’â–’ â–’â–’â–’â–’â–’â–’â–’â–’â–’â–’â–’â–’â–’â–’ â–’â–’â–’â–’ â–’â–’â–’â–’â–’â–’ 8000 â–’â–’. â–’â–’â–’â–’ â–’â–’â–’â–’â–’â–’â–’â–’â–’â–’â–’â–’â–’â–’â–’â–’â–’â–’â–’â–’â–’â–’ â–’â–’â–’â–’â–’â–’â–’â–’â–’â–’.
>> Non-ASCII text is not readable. Older Cygwin 2.5.2 has no such issue.
>> C:\Cygwin\bin\uname -a
>> CYGWIN_NT-10.0 PCName 2.6.0(0.304/5/3) 2016-08-31 14:32 x86_64 Cygwin
>> C:\Cygwin\bin\locale
>> LANG=
>> LC_CTYPE="C.UTF-8"
>> LC_NUMERIC="C.UTF-8"
>> LC_TIME="C.UTF-8"
>> LC_COLLATE="C.UTF-8"
>> LC_MONETARY="C.UTF-8"
>> LC_MESSAGES="C.UTF-8"
>> LC_ALL=
>> Same issue with any other commands like "grep", or with utilities built and run under Cygwin 2.6.0.
>> Same issue in other Windows consoles, like ConEmu or FAR Manager.
>> If I change Windows console encoding to UTF-8 (run: "chcp 65001"), file can be correctly displayed natively
>> (run: "type test.txt"), but Cygwin "cat" still has the same issue.
>> How should I display UTF-8 now?
>
> No problems here - same setup.
> Don't have files containing UTF-8 specials handy, but do have with Latin1 (ISO-8859-1) specials,
> convertable to UTF-8.
> Stripped common ASCII-only lines from output below.
> Default email encoding is Unicode (hopefully UTF-8) not Western (presumably Latin1), so should render accurately.
>
> $ uname -srvmo
> CYGWIN_NT-10.0 2.6.0(0.304/5/3) 2016-08-31 14:32 x86_64 Cygwin
> $ locale
> LANG=C.UTF-8
> LC_CTYPE="C.UTF-8"
> LC_NUMERIC="C.UTF-8"
> LC_TIME="C.UTF-8"
> LC_COLLATE="C.UTF-8"
> LC_MONETARY="C.UTF-8"
> LC_MESSAGES="C.UTF-8"
> LC_ALL=C.UTF-8
> $ egrep -a 'Deg|LF' latin1.txt # -a needed to override binary assumption - garbled characters
> DegN='â–’N'
> DegW='â–’W'
> Y2LF='%sâ–’%s %s %s'
> Y2LLF='|â–’%.0s|'
> LF='|â–’'.YFP.'|'
> $ iconv -f iso-8859-1 -t utf-8 latin1.txt | egrep 'Deg|LF' # good utf-8 characters
> DegN='°N'
> DegW='°W'
> Y2LF='%s±%s %s %s'
> Y2LLF='|±%.0s|'
> LF='|±'.YFP.'|'

Sorry - this was mintty - you used cmd!
Saw similar problems you had until I set LC_ALL=C.UTF-8 (and LANG for consistency, but doesn't really matter) and chcp 65001.
Then type and Cygwin commands produce the same output.
Without CP65001 (and a Unicode console font mapping most characters - I use DejaVu Sans Mono everywhere I can) there may be no valid encoding for UTF-8 special characters in your default console CP (437 for US, 850 for non-US, others for localized versions).
Unfortunately then less displays spaces as squares, so you may have to set PAGER=more for readability.

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

- Raw text -