DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:subject:references:to:from:message-id:date
	:mime-version:in-reply-to:content-type
	:content-transfer-encoding; q=dns; s=default; b=ilTgRlb9lqua8QqA
	nRXaSU9aYEXUQrPUTqI0p60nmRxRQVgdM1TkvELtPvJFNTbYtOD5t1xgxhASVDqU
	mbDzBgMv60cMbgkRjqWo4/eYjN/8sQV36fk8P+M5ZZqskF7q47bxXN7ApkGwRODb
	lt75uxTK7M/BP2StoY8XmqtdCwk=
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Subject: Re: Need help with multibyte UTF-8 characters
References: <626a3c06-e9f2-1932-f1f3-47ddb2051215 AT gmail DOT com>
To: cygwin AT cygwin DOT com
From: Thomas Taylor <tayloth AT gmail DOT com>
Message-ID: <9d3b73ff-f596-51a2-909a-30a767e3e9b3@gmail.com>
Date: Mon, 11 Dec 2017 18:36:09 -0500
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0
MIME-Version: 1.0
In-Reply-To: <626a3c06-e9f2-1932-f1f3-47ddb2051215@gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit

Thank you for your advice on setting my locale to en_US.UTF-8.  
Unfortunately, Cygwin still seems to have trouble displaying some 
three-byte UTF-8 encoded characters correctly.  For example, see the 
following snippet from a "sed" file.  This file attempts to convert 
XML-encoded filenames to UTF-8.  As you can see, it converts one- and 
two-byte encodings correctly, but fails on some three-byte encodings 
(the en dash, the em dash, and the ellipsis, all of which are displayed 
as a filled-in rectangle):

# Match longest strings first

# Three-byte encodings:

# En dash
s/%[Ee]2%80%93/–/g

# Em dash
s/%[Ee]2%80%94/—/g

# Horizontal ellipsis
s/%[Ee]2%80%[Aa]6/…/g

# Less-than-or-equal sign
s/%[Ee]2%89%[Aa]4/≤/g

# Euro symbol
s/%[Ee]2%82%[Aa][Cc]/€/g

# Two-byte encodings:

# Non-break space
#s/%[Cc]2%[Aa]0/⎵/g

# Lowercase a with acute accent
s/%[Cc]3%[Aa]1/á/g

# Lowercase a with umlaut (a.k.a. diaeresis)
s/%[Cc]3%[Aa]4/ä/g

# Lowercase e with acute accent
s/%[Cc]3%[Aa]9/é/g

# Lowercase i with acute accent
s/%[Cc]3%[Aa]D/í/g

# Lowercase o with acute accent
s/%[Cc]3%[Bb]3/ó/g


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple