X-Recipient:	archive-cygwin AT delorie DOT com
DomainKey-Signature:	a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:date:from:to:subject:message-id:reply-to
	:references:mime-version:content-type:in-reply-to; q=dns; s=
	default; b=CtVk1h1UIh2aGbRWy5ldoK6J+Yo7V2DxyGD5xHlN3KSXMfZ5Ui3RD
	QZbCja2HlB9P0/Eay7sPuDol5lW93gB7tDPjtqw97LDsQUrdXIubsvhLxcHANxQJ
	SXG4K2B17GDl4mcnnnctR1hvPOUgyLVF7WIHlhS/6qmn6oQmgpVBWI=
DKIM-Signature:	v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:date:from:to:subject:message-id:reply-to
	:references:mime-version:content-type:in-reply-to; s=default;
	bh=Fmf35svowzt4jlwfK7R/baw3uF0=; b=GmyOaLKRlODMSEt6b+bafj48iw5n
	PvM0baRRMaKIRkt5Br75JzEcsMUSSp5mWc/CbHjsbJQUqDOe2Vrfc9Pw43aXwBlL
	Wsq2icrygliSZy7pHFnilAXYthgdQMVfVk+tRaqYqchV2hm7bZ8sjxOoMEO5lg1j
	BOuacpZRBviwZKc=
Mailing-List:	contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id:	<cygwin.cygwin.com>
List-Subscribe:	<mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive:	<http://sourceware.org/ml/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender:	cygwin-owner AT cygwin DOT com
Mail-Followup-To:	cygwin AT cygwin DOT com
Delivered-To:	mailing list cygwin AT cygwin DOT com
Authentication-Results:	sourceware.org; auth=none
X-Spam-SWARE-Status:	No, score=-101.2 required=5.0 tests=AWL,BAYES_00,GOOD_FROM_CORINNA_CYGWIN,KAM_LAZY_DOMAIN_SECURITY,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.2 spammy=
X-HELO:	mout.kundenserver.de
Date:	Fri, 17 Aug 2018 20:44:02 +0200
From:	Corinna Vinschen <corinna-cygwin AT cygwin DOT com>
To:	cygwin AT cygwin DOT com
Subject:	Re: memory reported in /proc/pid/status is wrongly scaled
Message-ID:	<20180817184402.GW3747@calimero.vinschen.de>
Reply-To:	cygwin AT cygwin DOT com
Mail-Followup-To:	cygwin AT cygwin DOT com
References:	<CAL0dvtiDxdcN2qUG2MEG0vrpkr=R2LkopK7ECV8YamWyo7K=ng AT mail DOT gmail DOT com> <20180817171454 DOT GV3747 AT calimero DOT vinschen DOT de>
MIME-Version:	1.0
In-Reply-To:	<20180817171454.GV3747@calimero.vinschen.de>
User-Agent:	Mutt/1.9.2 (2017-12-15)

--JKvC8Wn6JllzUPdT
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Aug 17 19:14, Corinna Vinschen wrote:
> On Aug 17 16:05, Livio Bertacco wrote:
> >  Hi,
> > While playing with reading process memory usage in Linux and cygwin, I
> > found that cygwin reports too large values in /proc/self/status (in 2.1=
0.0
> > and earlier).
> > Whenever I was allocating a few kB in my test program, the VmSize line =
in
> > /proc/self/status was growing several times faster.
> >=20
> > Small bash script to show the issue:
> > #!/bin/bash
> > pid=3D$$
> > vmsizesplit=3D($(grep VmSize /proc/$pid/status))
> > vmsize1=3D"${vmsizesplit[1]}"
> > echo Initial memory reported in status: $vmsize1 kB
> > echo Allocating a 1000 kB string (bash can use more memory)
> > eat=3D$(printf '%1024000s')
> > vmsizesplit=3D($(grep VmSize /proc/$pid/status))
> > vmsize2=3D"${vmsizesplit[1]}"
> > echo Current memory reported in status: $vmsize2 kB
> > echo Difference is $[$vmsize2-$vmsize1] kB
> >=20
> > Running this in cygwin on my laptop I get:
> > Initial memory reported in status: 84928 kB
> > Allocating a 1000 kB string (bash can use more memory)
> > Current memory reported in status: 106880 kB
> > Difference is 21952 kB
> >=20
> > While bash may use quite more than 1000 kb in this case, 22x times larg=
er
> > doesn't seem right.
> >=20
> > Checking source file fhandler_process.cc, the
> > function format_process_status which writes the "status" proc file
> > retrieves memory usage via get_mem_values. Get_mem_values obtains that =
info
> > from NtQueryInformationProcess/PagefileUsage which is in bytes, then it
> > scales it to pages dividing by wincap.page_size:
> > 1515: *vmsize =3D vmc.PagefileUsage / wincap.page_size ();
> >=20
> > Then format_process_status scales it back, in theory to bytes, and shif=
ts
> > it by 10 bits in order to print it out in kB:
> > 1219:  unsigned page_size =3D wincap.allocation_granularity ();
>=20
> Looks like this is the bug.  get_mem_values returns all values
> in multiple of OS page_size (4K), but format_process_status multiplies
> with allocation_granularity (64K), leading to 16 times overallocation
> value.  The other caller of get_mem_values, format_process_statm,
> returns number of pages.  This must be expressed in multiples of
> allocation_granularity since that's the virtual page_size in Cygwin.
> But in case of format_process_status we're looking at KB values, so
> patch 8a32c24a7bdb0, replaceing page_size with allocation_granularity,
> was incorrect.
>=20
> Good catch!
>=20
> I'll revert patch 8a32c24a7bdb0 for 2.11.0.

On second thought there's more wrong than just that.  Just dividing
by page_size or allocation_granularity results in too small values.
I applied patches to return more correct values and I made sure
the values in status and statm are consistently rounded up to
Cygwin's page size of 64K.

I uploaded new developer snapshots to https://cygwin.com/snapshots
for testing.  Please give them a try.


Thanks,
Corinna

--=20
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

--JKvC8Wn6JllzUPdT
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEoVYPmneWZnwT6kwF9TYGna5ET6AFAlt3F3EACgkQ9TYGna5E
T6C9fQ/+LGqf6j3cLdjeUq5Gih26eJaA5p0wKlaeAbQAD+yGQmNwv5sQp2eOPDjV
m2uwzhUtNAnWDfnWwQ7N14suPNuymr3cn824lcJO9hH9UXbHYur18YrRxL16WFEa
9J7T5Zow9xetEjcEuOJ412l/k4q5fTE46r1waUp4RO2Q4RuAhjaU6gQfm1/IR2Mh
NiPMKqv8GZdQK+z+t11Nc+AQjjNqRM0Ke+XnW2AMNC9GacHULywYst0zVIF6TJDV
R+Iro69MdSxoDkuCy2NIHOh5EVGfbds21U38IDOF4uEVSQTMWMmCqrmx12mIUKNp
KMZfvL7yWha/tq9p4gEWY2DftZo+qGCGGZMpeID71ionFxWZhwzDXiNo3WorjeqI
1G358TxHPgBXL8txFtS0IXDncRf7epMEnjj5qbFeFczgU/C3HBSmETx5O0w8mUU0
b/IYcONHPKpqX4ljlTjNj1vpCLiUM+O0Trxgu2ofKB/SZWn44BUB5qagVkuVC4zm
r6nSv+DBCUysaujvxX4893061BNdEVn/rUn4CSwzqB+e3C5HijVfCYljE1c3SyqV
r03hhV6ol8YTKqJJvXYLmxZPj69kgVVqqhdsbHacNlXoQPjbi7ekOmfE1ttWc8kl
ohC3Jsq6keuKoZtn/XDuhi6P9KiigJJGdSqh5CQf3EMwnC2cW0M=
=lIgF
-----END PGP SIGNATURE-----

--JKvC8Wn6JllzUPdT--

- Raw text -