Mail Archives: cygwin/2009/03/19/13:13:47
On Mar 19 10:33, David Rothenberger wrote:
> On 3/19/2009 6:09 AM, Corinna Vinschen wrote:
> > If you've set $LANG to, say, "en_US.UTF-8", Cygwin would use the UTF-8
> > charset *iff* the application switched the codepage by calling something
> > along the lines of `setlocale(LC_ALL, "");'.
> > An application which does not call setlocale (which means, it's not
> > native language aware anyway) would still use the default ANSI codepage.
>
> First, please forgive my ignorance about LC_ALL, LANG, etc.
>
> I ran into an issue yesterday where I was trying to "du -sh" a directory
> that contained files whose names included UTF characters, I think.
> Without CYGWIN=codepage:utf8, this failed. It worked fine when I added
> CYGWIN=codepage:utf8.
Yes, sure. As described in the User's Guide. That's exactly what bugs
me right now. To get UTF-8 support you have to set LANG or LC_ALL or
whatever, *and* CYGWIN=codepage:utf8.
I *think* we can get rid of the codepage setting in favor of the
$LANG/$LC_foo setting, but we couldn't support both, ANSI and OEM
codepages anymore in this case. In the long run I'm looking into not
using the ANSI/OEM codepages at all, though, but instead have real, full
locale support. But that's a dream of the future.
> So my question is, will this work if codepage is dropped and I set LANG
> to en_US.UTF-8? Is there anything in the Cygwin DLL itself that uses
> codepage that might be valuable to enable even for applications that
> aren't native language aware and don't call setlocale()?
Not exactly. However, assuming you have a file using characters which
are not in your current ANSI codeset, then you could only manipulate
that file when setting LANG="xx_YY.UTF-8", and only in applications
which call setlocale().
In contrast to UNIX systems, we have the problem that the underlying
filesystems are using the UTF-16 charset for filenames. So we must
convert from the used singlebyte or multibyte charset to wide character.
Other systems don't care, the filename is just a byte stream. On
Windows, you always have a conversion step which requires to know the
multibyte character set. There's no way to convert a wide character
string into a multibyte string without knowing that charset.
Of course, what we could do is to call setlocale from within Cygwin so
we always have a base for the conversion, whether or not the application
calls it again. In theory this should not affect applications which
don't call setlocale since these applications are like other OSes; they
handle the filename as a simple bytestream.
The problem: I'm not really sure calling setlocale in Cygwin is a good
idea. Maybe there's some downside I just don't see right now.
Corinna
--
Corinna Vinschen Please, send mails regarding Cygwin to
Cygwin Project Co-Leader cygwin AT cygwin DOT com
Red Hat
--
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Problem reports: http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ: http://cygwin.com/faq/
- Raw text -