X-Recipient:	archive-cygwin AT delorie DOT com
DomainKey-Signature:	a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:date:from:to:subject:message-id:reply-to
	:references:mime-version:content-type:in-reply-to; q=dns; s=
	default; b=JLY4metU3zrw3G8vOrCIW9syx4f1Z5rHMQYacQv2QS6xAU0pkHeel
	6uJ5CPx9X46whs4+6GMyhMJcz7qA6juZLfWAduhE552Q1tezVZPHjWyVpWZYiPv+
	bqnpsRsl0YBGdbNgnzguDCDu1n/CVD9EBCOzpQfuMxD4FUbBjmYYZg=
DKIM-Signature:	v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:date:from:to:subject:message-id:reply-to
	:references:mime-version:content-type:in-reply-to; s=default;
	bh=SIKdlKv+9zVlOYJRwtCal4dDTcQ=; b=KHfq5vlr3R50868FiKziIXtr2gR3
	B61MhYygB8D4F555U++FIe7E68xOCF3hukbHJ11h4MOZi6SFR+KAEbxkkQ+amrvh
	YkMAWvdt+Bs2tVjXgeTwvHzGQcCaufDQ7ZV9A+iCvxwz0zWhMZ01DSR8U/UAOooM
	4SvIuFai/+yk3Gg=
Mailing-List:	contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id:	<cygwin.cygwin.com>
List-Subscribe:	<mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive:	<http://sourceware.org/ml/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender:	cygwin-owner AT cygwin DOT com
Mail-Followup-To:	cygwin AT cygwin DOT com
Delivered-To:	mailing list cygwin AT cygwin DOT com
Authentication-Results:	sourceware.org; auth=none
X-Virus-Found:	No
X-Spam-SWARE-Status:	No, score=-4.9 required=5.0 tests=AWL,BAYES_00,KHOP_PGP_SIGNED,NO_RELAYS autolearn=ham version=3.3.2
X-HELO:	calimero.vinschen.de
Date:	Fri, 30 Aug 2013 23:39:32 +0200
From:	Corinna Vinschen <corinna-cygwin AT cygwin DOT com>
To:	cygwin AT cygwin DOT com
Subject:	Re: xz -9 : Cannot allocate memory
Message-ID:	<20130830213932.GA2342@calimero.vinschen.de>
Reply-To:	cygwin AT cygwin DOT com
Mail-Followup-To:	cygwin AT cygwin DOT com
References:	<loom DOT 20130829T155739-66 AT post DOT gmane DOT org> <loom DOT 20130829T162752-851 AT post DOT gmane DOT org> <20130829151121 DOT GR21571 AT calimero DOT vinschen DOT de> <87ioyowbq9 DOT fsf AT Rainer DOT invalid> <loom DOT 20130830T111847-683 AT post DOT gmane DOT org> <20130830120244 DOT GV21571 AT calimero DOT vinschen DOT de> <loom DOT 20130830T143654-505 AT post DOT gmane DOT org> <20130830174631 DOT GA8831 AT calimero DOT vinschen DOT de> <20130830190048 DOT GA21571 AT calimero DOT vinschen DOT de> <20130830210349 DOT GA975 AT ednor DOT casa DOT cgf DOT cx>
MIME-Version:	1.0
In-Reply-To:	<20130830210349.GA975@ednor.casa.cgf.cx>
User-Agent:	Mutt/1.5.21 (2010-09-15)

--vtzGhvizbBRQ85DL
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Aug 30 17:03, Christopher Faylor wrote:
> On Fri, Aug 30, 2013 at 09:00:48PM +0200, Corinna Vinschen wrote:
> >On Aug 30 19:46, Corinna Vinschen wrote:
> >> On Aug 30 12:58, Achim Gratz wrote:
> >> > Corinna Vinschen <corinna-cygwin <at> cygwin.com> writes:
> >> > > Yes, looks normal and expected from what you observed.  mmap commi=
ts
> >> > > memory top-down and that was apparently the first free slot big en=
ough
> >> > > to fullfil the request.  The default heap size is 384 Megs and then
> >> > > there's apparently not enough space anywhere.
> >> >=20
> >> > Sorry if I'm dense, but does this mean that setting the default heap=
 size to
> >> > zero does in fact mean it's trying to use 374MiB... wait, yes it is:=
 the
> >> > memory map is the same when I'm setting the initial heap size to 384=
MiB with
> >> > peflags.
> >> >=20
> >> > So, even though that rather large heap is essentially unused at the =
point of
> >> > failure and there is enough memory free just beyond the heap, the al=
location
> >> > still fails because both the existing heap and the free space are bo=
th
> >> > smaller than what's requested?
> >>=20
> >> Well, good question.  You could debug the sbrk function and see why
> >> it fails to reserve the space.
> >
> >I just debugged this and it seems our sbrk implementation has a serious
> >problem to extend the heap if the new chunk of memory requires to commit
> >some of the existing heap and to reserve and commit some more space.
> >It tries to reserve memory using the wrong address and the wrong size.
> >It also uses a too simple method to commit the memory.  I'll apply a fix
> >shortly.
>=20
> It looked fine to me.
>=20
> The only code change that I did make was to change your coercion of
> pointers to ptrdiff_t types into (char *), since that is more in keeping
> with the rest of the code.
>=20
> I had a hard time reading the code with all of the cygheap-> pointers so
> I changed two functions to be a user_info_heap methods.  That, IMO,
> makes the code faster and more readable.

Good idea, that was an overdue change, I guess.

What still bugs me in sbrk is the fact that it prefers to reserve
subsequent chunks of heap in "chunk"-sized chunks (uh oh).  When using
the default heap size of 384 Megs, "chunk" and thus each subsequent heap
chunk will be 384 Megs as well if the space is still available.

That's an awful lot of memory, and it it could easily break later mmap's
or thread stack reservations on 32 bit.  Given that the default heap is
384 Megs already, I would prefer if we reserve subsequent heap memory in
chunks of 1 or 2 Megs only, to lower the pressure on the VM.

Sounds good?

> You probably saw that I fixed the USR1/USR2 issue.  That was a serious si=
gnal
> bug.  I think both of those warrant a new release in fact.

I try to do that on Sunday.


Corinna

--=20
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

--vtzGhvizbBRQ85DL
Content-Type: application/pgp-signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.14 (GNU/Linux)

iQIcBAEBAgAGBQJSIREUAAoJEPU2Bp2uRE+gF5QQAIq/pk0qxT7ByFXFcghLE6Xm
MH3+uoOo8xyNREr3SQODbSPo1fiow2rqd79GOJ/J0XPutfzJfwfHBvxG1OyOehcG
tdsix2+Na1RK63HJBM5tbp9FIYN5mvVj/hYr51ozKdxNBnNP2fMstqO7Qg3Wib8w
3r+rJwSQgjvSFt0YYwxzgbXmP9GWLCs6ERiJVv5kPi4ve4qqUnmfyvbNiwX13IAH
ca/yYW9vweKb4wffd3rBqQPKRDHSqWF5AdE/qb73JF0Y4YX7+s0FpDJlFEUUY9nG
jolQIEQIupXKxmYNkItFtomibxDov36k2zPxJDpXmF+1KvauL86WfZrutgmNozGx
BGvoxekOtfgMoAq2irgrtoad9EgW0jq9hnSrxv713l9O5whMUu5uCXVTo9XjKakC
VcyaTphklcSNRVCPbMiV5x+nSnUEYAzFyxGZDDf+MVFxSAuon+XpOL+ZuBGvAONQ
E2x4n3d1k6kokeo+o6Qg3hruZV6UwEFwJ3I+7siVNPND58wDuc+hkizjM95u3FMl
F+9zqodkkbKPINOHtx5Ux2gGA2qJt+3lbWwpBjleahWWXYxQdg2FrW9WFh3v7AOP
L5SA7c3X9JlI6a8CY3lE7IpzdL2vPvROJaXmTxI2wrX1UPUS7Mb0/yV726m19UqD
Ty7lRfc0CYquRfMaOfxM
=uM2s
-----END PGP SIGNATURE-----

--vtzGhvizbBRQ85DL--

- Raw text -