X-Recipient:	archive-cygwin AT delorie DOT com
DomainKey-Signature:	a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:date:from:to:subject:message-id:reply-to
	:references:mime-version:content-type:in-reply-to; q=dns; s=
	default; b=ZaXpBkLBsebpNr2bzlRbpm2Hm3tpieN7K8soAwJFEqWoRDu+pHFl6
	ouIzvY6aIpK8mohhPNrK66OTBmgE4d6RlCibDC+wKD91vXCdY/cwE5dDqMI//UFE
	UW0RxqjpM86+rW09Ks7wJs1F2RaKnf2+shwTNGgn5NXf9dpE6cL2P4=
DKIM-Signature:	v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:date:from:to:subject:message-id:reply-to
	:references:mime-version:content-type:in-reply-to; s=default;
	bh=G0e1fgSpGkj2o6lytmuPgN2LOyw=; b=yl6QpHM2PUOo7+Gq6LamxEmUVKMP
	t1Kt6r+9zS1Pxyqd9SKgNIJLGRTP38GnaLvkJSWoqadxjkh3vsu0S2DTbS+qHEYa
	6Dkcwb59BpNFxerMWZanlwWhPhi8YuIIdXLrTcvyqDaRNOHjOHsG2NG76HC2sykH
	XNyCuLEnf56M5Ys=
Mailing-List:	contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id:	<cygwin.cygwin.com>
List-Subscribe:	<mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive:	<http://sourceware.org/ml/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender:	cygwin-owner AT cygwin DOT com
Mail-Followup-To:	cygwin AT cygwin DOT com
Delivered-To:	mailing list cygwin AT cygwin DOT com
Authentication-Results:	sourceware.org; auth=none
X-Virus-Found:	No
X-Spam-SWARE-Status:	No, score=-101.9 required=5.0 tests=AWL,BAYES_00,GOOD_FROM_CORINNA_CYGWIN,KAM_LAZY_DOMAIN_SECURITY,RCVD_IN_DNSWL_LOW,SPF_HELO_PASS autolearn=ham version=3.3.2 spammy=incredible, poorly, H*c:application
X-HELO:	drew.franken.de
Date:	Tue, 6 Jun 2017 17:57:39 +0200
From:	Corinna Vinschen <corinna-cygwin AT cygwin DOT com>
To:	cygwin AT cygwin DOT com
Subject:	Re: Bug? wcsxfrm causing memory corruption
Message-ID:	<20170606155739.GC23208@calimero.vinschen.de>
Reply-To:	cygwin AT cygwin DOT com
Mail-Followup-To:	cygwin AT cygwin DOT com
References:	<CAOTD34aCROSAQojYvV4rjwiWOfiALFP+P2wODoMV1dcaOhKPFQ AT mail DOT gmail DOT com> <20170521042352 DOT GA4045 AT dimstar DOT local DOT net> <CAOTD34buK86Xdd-XV5DGtfD_xyMYT++-O6y+gpg7YC4YNLXLsQ AT mail DOT gmail DOT com>
MIME-Version:	1.0
In-Reply-To:	<CAOTD34buK86Xdd-XV5DGtfD_xyMYT++-O6y+gpg7YC4YNLXLsQ@mail.gmail.com>
User-Agent:	Mutt/1.8.0 (2017-02-23)

--xo44VMWPx7vlQ2+2
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Hi Erik,

On May 22 14:50, Erik Bray wrote:
> On Sun, May 21, 2017 at 6:23 AM, Duncan Roe wrote:
> > On Wed, May 10, 2017 at 11:30:46AM +0200, Erik Bray wrote:
> >> [...]
> >> The attached test demonstrates the bug.  Given an output buffer of N
> >> wide characters, wcsxfrm will cause bytes beyond the destination size
> >> to be reversed. I believe it might actually be a bug in the underlying
> >> LCMapStringW workhorse (this is on Windows 10; have not tested other
> >> versions).
> >>
> >> According to its docs [1], the cchDest argument (size of the
> >> destination buffer) is treated as a *byte* count when using
> >> LCMAP_SORTKEY.  However, for the purposes of applying the
> >> LCMAP_BYTEREV transformation it seems to be treating the output size
> >> (in bytes) as character count.  So in the example I give, where the
> >> output sort key is 7 bytes (including the null terminator), it swaps
> >> *14* bytes--the bytes including the sort key as well as the next 7
> >> adjacent bytes.  This is obviously a problem if the destination buffer
> >> is allocated out of some larger memory pool.
> >>
> >> This definitely has to be a bug, right?  Or at least very poorly
> >> documented on MS's part.  A workaround would either be to not use
> >> LCMAP_BYTEREV and just swap the bytes manually, or in a second call to
> >> LCMapStringW with LCMAP_BYTEREV and the correct character count...
> >> [...]
> > Hi Erik,
> >
> > I get
> > [...]
> Yes, that's the same.  Thanks for giving it a try--I should have
> included example output in my original message.
>=20
> You can see in the last case that without LCMAP_BYTEREV it writes the seq=
uence
>=20
> 0x0e 0x09 0x01 0x01 0x01 0x01 0x00
>=20
> with a terminating 0x00.  Bytes after that remain unchanged.  In the
> other two examples *with* LCMAP_BYTEREV, the terminating 0x00 gets
> swapped with the 0x07 after it, but this documented and expected
> behavior of LCMapStringW, and is already accounted for in Cygwin's
> wcsxfrm.  What is undocumented, and unexpected, is that it also byte
> swaps 3 more byte pairs after the actual sort key, which can corrupt
> memory unexpectedly.

Incredible piece of detective work.  Yeah, this looks very much like a
bug in LCMapStringW, embarrassingly one I didn't notice at the time of
writing the code.  I just tested this on W7 and the problem was already
present.

I don't know if this could be fixed by documentation.  There's just no
safe way to combine LCMAP_SORTKEY with LCMAP_BYTEREV it seems, unless
you always allocate a buffer at least twice as big as actually required
for the sort key.=20=20

So, yeah, I think manual swaping after calling LCMapStringW without
LCMAP_BYTEREV is the way to go.  I'll prepare a fix.


Thanks for tracking down and the testcase,
Corinna

--=20
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

--xo44VMWPx7vlQ2+2
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJZNtDyAAoJEPU2Bp2uRE+gmmkP/A4oSnEXyQhlm1ugyOIx9wUg
DKg1fiAHjnj/nSFkZFrBout+RvFx5SnkrT7gRnUxQMH81t8P2IBKCe8VXCOxjm5C
XFdG4CUlLVyVQRidsIsRKssYPfdbOA6K59lo2Hm6fVtPAlAcC1+Bn4EA29DGHF6v
+Rg42XkDGfSp7qI3J3Bez9oB3dnwqAk2i33APE1Jnw/WLPDjRW7JPPXulBHFY0rQ
xQpQW6BDmsQBpM/fj6xYMf7G1Mm1VivLY+1yTKKMUYMDIX6d4BRNXoabjEWT5Bor
HT+Uu3kXmffS4F505x4A0K/2dQUIPwN+XvhtSCBP7YRXUBIXaRboPDhbS/ahMGvS
z5OwlzruZgnznyUOw7mt3dKjtUk1v6bVvAQIlEy7XTvl9cw2wshESJ5XCSVKCkf2
Gk0L5TCo+KKD11a6gRLnUXKYyOk/UJJ7vtiow72JYuPjUMii2KbAUURnQvmuDyYz
FVPHGpkTbS39PuHugyuDAaO2XZIjFDVnIdASxvNo3mJYUAtDetYkWueLHQJidKrz
hzMg0jaQGNX1jYbrphZHaUZIim4ZTl5+T3p6ryj1mko9ksOBi1LZ5M41vRov6MUj
2YdwA1udXh0JolB+HYdtkKuMgKy2sirE17m4GCiBq3XahSrYu+WCY31acf/cIx2M
JjyUmGb0QliuiGxKwWCO
=EFXU
-----END PGP SIGNATURE-----

--xo44VMWPx7vlQ2+2--

- Raw text -