DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:date:from:to:cc:subject:message-id:reply-to
	:references:mime-version:content-type:in-reply-to; q=dns; s=
	default; b=CYpvA/LLN+9BSoRk/svoGRdToesfvigRUhjBbCynfKegSWL1lxDia
	AqHy+m8aDTZrGANrU6bhhZlTcz/zOLnYsBivKbhSLpUYhAOpmBpUXK6gXx20UQBa
	gcGPC056r+PIBNjQOKoiJX4FX5lSiabx+cfj3bdE7R6mLxTOdsUtEA=
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Date: Tue, 4 Jun 2019 15:18:36 +0200
From: Corinna Vinschen <corinna-cygwin AT cygwin DOT com>
To: Stanislav Kascak <stanislav DOT kascak AT gmail DOT com>
Cc: cygwin AT cygwin DOT com
Subject: Re: possible problem with memory allocation using calloc/mmap/munmap
Message-ID: <20190604131836.GS3437@calimero.vinschen.de>
Reply-To: cygwin AT cygwin DOT com
Mail-Followup-To: Stanislav Kascak <stanislav DOT kascak AT gmail DOT com>,	cygwin AT cygwin DOT com
References: <CALLhcm4QGY+eP0_CRiSbJwQ12kOetvTK=6-AtC17x7d+QhGKTw AT mail DOT gmail DOT com> <20190603115456 DOT GG3437 AT calimero DOT vinschen DOT de> <CALLhcm4qaha=XuBWEqag7QH1veA82UrB94emFtFMd26dZ_ZGZQ AT mail DOT gmail DOT com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;	protocol="application/pgp-signature"; boundary="1GSL5ZULXUIqbbH1"
Content-Disposition: inline
In-Reply-To: <CALLhcm4qaha=XuBWEqag7QH1veA82UrB94emFtFMd26dZ_ZGZQ@mail.gmail.com>
User-Agent: Mutt/1.11.3 (2019-02-01)

--1GSL5ZULXUIqbbH1
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Jun  4 11:38, Stanislav Kascak wrote:
> > > It seems that when mmap() is called with length argument exceeding
> > > size of file, only memory to fit that file is allocated. munmap()
> > > however frees the full specified length. Since (at least on my
> > > computer) big chunk of memory allocated by calloc() is located after
> > > mmap() allocation, munmap() frees even memory of that calloc().
> >
> > Ken's right.  Due to the differences between mapping files on Windows
> > vs. Unix, Cygwin can't map beyond the file size + the remainder of the
> > last page.  Cygwin tries to workaround that on 32 bit by allocating
> > an anonymous mapping following the file mapping to keep the range free
> > from other mappings.  But on 64 bit this workaround doesn't work anymore
> > because the OS is missing an (undocumented) flag which allows to
> > create mappings on 4K boundaries, rather than just on 64K boundaries.
> >
> > I know this situation is unsatisfying, but I have no easy workaround
> > to allow this.  Cygwin could add the anonymous mapping on the next
> > 64K boundary on 64 bit, but that would result in a hole in the mapping
> > which seemed like a rather bad idea when porting mmap to 64 bit.
> >
> > Ken's also right that munmap is doing the right thing here.  If
> > anything's wrong, it's mmap's workaround for mappings beyond the file
> > length.  If only 64 bit would allow 4K-aligned mappings :(
>=20
> Thanks for the answer. It is appreciated.
> I understand the problem and difficulty to resolve it. Maybe returning
> an error from mmap (and putting a comment to code for its reason)
> would be sufficient. mmap caller could just adjust requested
> allocation size to file size. Without error, caller has no way of
> knowing memory was not allocated and segfault is then thrown in an
> unrelated memory segment which makes the root cause hard to track
> down. But, I do not know all the implication that could result from
> that, so evaluation of this approach is up to you.

Given that most of the required code already exists for 32 bit systems
(except under WOW64, suffering the same problem as the 64 bit WIndows
environment), I hacked a bit on this code this morning and I got your
testcase running fine.  The idea being that after a successful mmap the
expectation that a matching munmap does *not* unmap unrelated mappings
is valid.

In more depth, here's what Cygwin does on 32 bit, assuming a file size
of 100 bytes and a mapping request of 256K:

First Cygwin mmaps the file.  This results in a 4K mapping in Windows:

 file:    |-- 100b --|

 mapping: |-- 4K --....--|

Next Cygwin adds another mapping to fill up the range up to the next
64K allocation granularity boundary:

 |-- file 4K --|-- filler 60K --|

Eventually Cygwin adds another mapping to fullfill the entire mapping
request:

 |-- file 4K --|-- filler 60K --|-- filler 192K --|

The problem on WOW64 and real 64 bit is that it's impossible to map
the first filler.  However, this area in the VM will *never* be
allocated by other application functions due to the allocation
granularity of 64K!

So my workaround for 64 bit and WOW64 is to just skip allocating the
first filler:

 |-- file 4K --|-- THE VOID 60K --|-- filler 192K --|

The advantage is now that the following munmap of 256K will only
unmap the map for the file and the filler, but not the region you
calloced before, which formerly was accidentally mapped to the
filler region.  This just can't happen anymore now.

Would that be feasible?  If so I can push my patch and create a
developer snapshot for testing.


Thanks,
Corinna

--=20
Corinna Vinschen
Cygwin Maintainer

--1GSL5ZULXUIqbbH1
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEoVYPmneWZnwT6kwF9TYGna5ET6AFAlz2b6wACgkQ9TYGna5E
T6BQuRAAn3adR4hphhCQ1lG7v1g6EAMzxJ7rtepChmvhfPD36j2K07ZQWNr0FOMH
LO7a4794mndq6W68CDp+rbpNlIJSjwvMXP8/GvtykiSpI1x4j4wxQtHVXffRnGTb
OCkwCcmS8I10/HkXzxNG0+rmGKlSVbgJ/P53i0N8UBpn+v2d2dym0cER2boMSnbV
iUAgwPvnLeO2eACZpw76MGMwaj+ut99CgQg2KperG5mrvyCBoJeYceohTA3lXPXd
o+2jorZ6kNNaaEL6EUy3OQlIMgST9aH7glOP7F39S6OSntYrpWZBth+glSL9JmgJ
yi2gqCc50GAuSV9q6ODVVMcXxt0iajSoEgFhp7MYIxXe5rtgPrYMZ1RQY15nCuVX
rZyW+9TWt626cMR1RM7qpBNB4sOCou2rreEAPAaiIunXVh/snHSKlAf472QoiB+h
wq22DeDYxLBw12I4FX1BaOjOVrL9XoHms5IPjVVeuNHJbdYLkz2yY1ahGcSvoOqy
hnV72QoF5qyo2iduqUwKq3bLHFejgk0ByRSQb2S8DSMqlbwcvLResfFiMUykfKe6
yCTd2X2Y2R39HJNzYZdY01OUntCH0SZPVBqy9JMP7wsyZVCOfDl6vEifpVkPssGb
iXVOpYxXVnHZBMo7mKO1t+xsxneCSeaYiyZVsLqxgRxauaibaFA=
=+Kqb
-----END PGP SIGNATURE-----

--1GSL5ZULXUIqbbH1--