X-Recipient: archive-cygwin AT delorie DOT com DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:date:from:message-id:to:subject:in-reply-to :references:mime-version:content-type:content-transfer-encoding; q=dns; s=default; b=BYGviIDW4eTPs5ClQdxLxKdYdL6I2MRJf/Ie/Ni6oc5 TTPdB36BwC6RC4W/ny3uNLTgB1PYn5IGZ8theyQQMrjlzGYlJUh2+XURNclGaFbz I8yzgFNvU4lF2XBVjQDDJD+qUlYIB226MQCp7Mry0zb0p4NxQh6RBgyw27ceEz0c = DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:date:from:message-id:to:subject:in-reply-to :references:mime-version:content-type:content-transfer-encoding; s=default; bh=HQqy02phfscgxC6miFrn6vEMLoE=; b=tUKBKxfnpc2K8Xw61 zcrXq4D8LEwxh6pKYAeCn1nMZNbSRgLVgpOlGXP8QG+u7e4W1MaHv5HXevnhChIa RkxUCm/plPqtgnf4+p72KWkYYxCIrVg9W0r7pYsi418cZ6mSiumc4E21jZYOJGuZ ZETUaq0/tMS6V1j8s28YPQeK+w= Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner AT cygwin DOT com Mail-Followup-To: cygwin AT cygwin DOT com Delivered-To: mailing list cygwin AT cygwin DOT com Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: =?ISO-8859-1?Q?Yes, score=6.5 required=5.0 tests=AWL,BAYES_50,BODY_8BITS,GARBLED_BODY,LIKELY_SPAM_SUBJECT,SPF_PASS autolearn=no version=3.3.2 spammy==d0=b8=d1, unreadable, lc_messages, LC_MESSAGES?= X-HELO: Marvin.c0d.org Authentication-Results: marvin.c0d.org (amavisd-new); dkim=pass (1024-bit key) reason="pass (just generated, assumed good)" header.d=vanav.org Date: Sun, 2 Oct 2016 09:26:50 +0300 From: Ivan Vanyushkin Message-ID: <1025056909.20161002092650@vanav.org> To: cygwin AT cygwin DOT com Subject: Re: Cygwin 2.6.0: unreadable UTF-8 in Windows console In-Reply-To: References: <123291584 DOT 20161001051347 AT vanav DOT org>

MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-IsSubscribed: yes Note-from-DJ: This may be spam "set LANG=C.UTF-8" has fixed the issue on Cygwin 2.6.0. But documentation says [1], that "The default locale in the absence of the aforementioned locale environment variables is "C.UTF-8"." Seems this is broken in Cygwin 2.6.0. "chcp 65001", console font or console charset doesn't matter here. This is bad, because now I can't share compiled binaries to anyone, because users will have no LANG in environment variable, and any non-ACSII text will not be readable. For example, list running Windows services: sc query | grep -i "running" - will not work for not-English Windows, because output in console will not be readable. Watch Windows log: tail -f C:\Windows\Logs\SomeLog.log - will be not readable if there are some non-English file names. I think locale should remain default UTF-8, as in Cygwin 2.5.2. This is expected by both applications and users. Tests: // Run Windows console. cmd C:\Cygwin_2.6.0\bin\echo ±5°> utf-8.txt C:\Cygwin_2.6.0\bin\od -t x1z utf-8.txt 0000000 c2 b1 35 c2 b0 0d 0a >..5....< // We have UTF-8 now in "utf-8.txt" file. C:\Cygwin_2.6.0\bin\locale LANG= LC_CTYPE="C.UTF-8" LC_NUMERIC="C.UTF-8" LC_TIME="C.UTF-8" LC_COLLATE="C.UTF-8" LC_MONETARY="C.UTF-8" LC_MESSAGES="C.UTF-8" LC_ALL= C:\Cygwin_2.6.0\bin\cat utf-8.txt ▒▒5▒▒ C:\Cygwin_2.6.0\bin\env LANG=C.UTF-8 C:\Cygwin_2.6.0\bin\cat utf-8.txt ±5° // Fixed! But what is default locale then? C:\Cygwin_2.6.0\bin\env LANG=C.CP1251 C:\Cygwin_2.6.0\bin\cat utf-8.txt В±5В° C:\Cygwin_2.6.0\bin\env LANG=C.CP866 C:\Cygwin_2.6.0\bin\cat utf-8.txt ┬▒5┬░ C:\Cygwin_2.6.0\bin\env LANG=C.ISO8859-1 C:\Cygwin_2.6.0\bin\cat utf-8.txt Â±5Â° // Doesn't match. I have no idea what is default locale in Cygwin 2.6.0. // Let's try console native encoding echo ±5°> cp866.txt C:\Cygwin_2.6.0\bin\od -t x1z cp866.txt 0000000 2b 35 f8 0d 0a >+5...< type cp866.txt +5° C:\Cygwin_2.6.0\bin\cat cp866.txt +5▒ // Bad. Cygwin 2.6.0 can't display even non-UTF-8. // Try filenames: ls -al lrwxrwxrwx 1 vanav ▒▒▒▒▒▒▒▒▒▒▒▒▒▒ 33 Sep 17 07:43 ''$'\320\234\320\276\320\270'' '$'\320\2 64\320\276\320\272\321\203\320\274\320\265\320\275\321\202\321\213' -> /cygdrive/c/Users/Vanav/Documents // Bad. C:\Cygwin_2.6.0\bin\env LANG=C.UTF-8 C:\Cygwin_2.6.0\bin\ls -al lrwxrwxrwx 1 vanav система 33 Sep 17 07:43 'Мои документы' -> /cygdrive/c/Users/Vanav/Documents // Good. // Now try previous Cygwin 2.5.2. C:\Cygwin_2.5.2\bin\locale LANG= LC_CTYPE="C.UTF-8" LC_NUMERIC="C.UTF-8" LC_TIME="C.UTF-8" LC_COLLATE="C.UTF-8" LC_MONETARY="C.UTF-8" LC_MESSAGES="C.UTF-8" LC_ALL= C:\Cygwin_2.5.2\bin\cat utf-8.txt ±5° // Good. C:\Cygwin_2.5.2\bin\env LANG=C.UTF-8 C:\Cygwin_2.5.2\bin\cat utf-8.txt ±5° // Good. [1] https://cygwin.com/cygwin-ug-net/setup-locale.html Saturday, October 1, 2016, 8:15:02 AM, you wrote: > On 2016-09-30 22:34, Brian Inglis wrote: > Sorry - this was mintty - you used cmd! > Saw similar problems you had until I set LC_ALL=C.UTF-8 (and LANG > for consistency, but doesn't really matter) and chcp 65001. > Then type and Cygwin commands produce the same output. > Without CP65001 (and a Unicode console font mapping most characters > - I use DejaVu Sans Mono everywhere I can) there may be no valid > encoding for UTF-8 special characters in your default console CP > (437 for US, 850 for non-US, others for localized versions). > Unfortunately then less displays spaces as squares, so you may have > to set PAGER=more for readability. -- Best regards, Ivan mailto:vanav AT vanav DOT org -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple