DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:mime-version:in-reply-to:references:from:date
	:message-id:subject:to:content-type; q=dns; s=default; b=mcCNKks
	HkEnXLgsuzgY1qatLhl+jz+CbtSssQcoFFp9mzXcAc8GRPuiskPvlYjeoW76J7G3
	fUZ7xhP45XNSnPwvbtU/iInALdYG7zLVJjcx37S8CW38jmUlSdMr6bOZhJkmp5SB
	fRbAoZ/QAxvwsskT6jGWE/v54gsoUGg+iIGI=
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
MIME-Version: 1.0
In-Reply-To: <9d3b73ff-f596-51a2-909a-30a767e3e9b3@gmail.com>
References: <626a3c06-e9f2-1932-f1f3-47ddb2051215 AT gmail DOT com> <9d3b73ff-f596-51a2-909a-30a767e3e9b3 AT gmail DOT com>
From: Doug Henderson <djndnbvg AT gmail DOT com>
Date: Tue, 12 Dec 2017 09:40:47 -0700
Message-ID: <CAJ1FpuM_omZszrpUj9TtjgrQbvpUqKb6AUnGOzZfgmSLiMb1Tg@mail.gmail.com>
Subject: Re: Need help with multibyte UTF-8 characters
To: cygwin <cygwin AT cygwin DOT com>
Content-Type: text/plain; charset="UTF-8"

On 11 December 2017 at 16:36, Thomas Taylor wrote:
> Thank you for your advice on setting my locale to en_US.UTF-8.
> Unfortunately, Cygwin still seems to have trouble displaying some three-byte
> UTF-8 encoded characters correctly.  For example, see the following snippet
> from a "sed" file.  This file attempts to convert XML-encoded filenames to
> UTF-8.  As you can see, it converts one- and two-byte encodings correctly,
> but fails on some three-byte encodings (the en dash, the em dash, and the
> ellipsis, all of which are displayed as a filled-in rectangle):
>

Your sed script works for me. I copy/pasted your sample script into
"cvt_script.sed" and also into "cvt_input.txt". My sed command looks
like: "sed --file=cvt_script.sed < cvt_input.txt > cvt_output.txt". It
correctly translates all the encoded utf-8 strings.

Your display may appear different if you are using different fonts in
mintty or the windows console. I am using Lucinda Console, 10pt and
Consolas 16, respectively. They display different glyphs for the
non-breaking space, but are otherwise identical. In mintty, I have
LANG and all the LC_* variables set to en_CA.UTF-8, and in the windows
console, to en_US.UTF-8.

I am running Win 10 and cygwin setup was last updated a couple or
three days ago.

Check the output of the "locale" command. All variables should have
the same value.

Is your cygwin installation up to date, or fairly close to current?
What wiindows version are you using?

HTH,
Doug


-- 
Doug Henderson, Calgary, Alberta, Canada - from gmail.com

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple