Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sources.redhat.com/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sources.redhat.com/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com
Message-ID: <3C378D50.FB46BFA8@pobox.com>
Date: Sat, 05 Jan 2002 16:33:36 -0700
From: "John A. Turner" <john DOT turner AT pobox DOT com>
X-Mailer: Mozilla 4.78 [en]C-{C-UDP; EBM-SONY1}  (Windows NT 5.0; U)
X-Accept-Language: en
MIME-Version: 1.0
To: cygwin AT cygwin DOT com
Subject: process memory limit
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

[member of the "Valiantly Trying to Ask Intelligently" club...]

I've been investigating an apparent limitation on process memory,
and I've gone about as far as I can without asking for some advice.

Brief Summary
-------------

it appears that a cygwin app can't alloc more than around 256MB,
even though:

o my machine has 2GB of physical mem and plenty of virtual
o the same app compiled with the Intel or MS compiler can
  allocate much more

after digging through the mail archives and docs, I've checked
out the sources in hopes of finding a parameter that could be
increased

Environment
-----------

dual 1.7GHz Xeons, 2GB RAM, 2GB page file
Win2kSP2 with all updates
up-to-date Cygwin 1.3.6 and friends
XFree86 4.1.0 with Server Test 53, LessTif, etc.

Details
-------

o there's a viz tool we use that I wanted to try and build on
  cygwin:

  http://www-xdiv.lanl.gov/XCM/gmv/GMVHome.html

o after a few false starts related to OpenGL and such, I was able
  to compile successfully and load a small example

  yes, it required xfree86 as well as lesstif, but as you'll see
  shortly, they don't seem to be a real factor, which is why I'm
  not sending this to cygwin-xfree

o we have a couple of much larger data sets that I wanted to try
  
  a colleague has a Linux box on which he has run one of the
  sets - he has 1GB physical mem - but when I try to load that
  set GMV falls over saying it can't allocate the necessary mem

o at first I thought it might be a windows thing, but after some
  time in the MS knowledge base without finding much, I decided to
  write a simple prog to test this - here it is:

#include <iostream>
#include <string>
using namespace std;

int main (int argc, char **argv)
{
    int meg=1024*1024;
    int mb = atoi(argv[2]);
    int n = mb*meg / 8;
    cout << "number of elements   :  " << n << endl;
    cout << "approx total mem req :  " << mb << " MB" << endl;

    double *x = new double[n];
    cout << "allocated..." << endl;

    for (int i = 0; i < n; ++i)
        x[i] = 1.0;
    cout << "initialized..." << endl;
    return 0;
}

  when I compile with cygwin g++:

/d/2002_01_04/memory$ g++ mem.cpp
/d/2002_01_04/memory$ a.exe -mb 255
number of elements   :  33423360
approx total mem req :  255 MB
allocated...
initialized...
/d/2002_01_04/memory$ a.exe -mb 256
number of elements   :  33554432
approx total mem req :  256 MB
Aborted (core dumped)

  whereas with MS VC++:

/d/2002_01_04/memory$ cl mem.cpp
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 12.00.8804 for 80x86
Copyright (C) Microsoft Corp 1984-1998. All rights reserved.

mem.cpp

[snip]

Microsoft (R) Incremental Linker Version 6.00.8447
Copyright (C) Microsoft Corp 1992-1998. All rights reserved.

/out:mem.exe 
mem.obj 
/d/2002_01_04/memory$ ./mem.exe -mb 256
number of elements   :  33554432
approx total mem req :  256 MB
allocated...
initialized...

  and in fact it works with much larger values as well (512, etc.) - and
  the same using Intel's compiler

o out of curiosity, tried the same experiments on my laptop, which has
  only 256 MB of physical mem, with identical results

o I thought that an interesting test would be to use mingw, but I don't
  have the "real" one installed, and a -mno-cygwin compile failed due to
  missing headers - I have to admit that I didn't pursue mingw any further

o searching the cygwin mailing lists turned up a couple of things, like:

http://sources.redhat.com/ml/cygwin/2001-03/msg01367.html

  on the user list and this:

http://sources.redhat.com/ml/cygwin-developers/2001-07/msg00083.html

  on the developer list, but they didn't really help, other than to give
  me the idea that maybe I could build my own cygwin dll with an increased
  limit

o perusing the docs yielded only this:

http://cygwin.com/faq/faq_4.html#SEC103

  but that doesn't help either

o next step was to check out winsup and see if I could find something I
  might be able to bump up - the most likely thing I found was the CYGHEAPSIZE
  define in src/winsup/cygwin/cygheap.h

o successfully built new cygwin1.dll and libcygwin.a, both without modifying
  CYGHEAPSIZE and after increasing it

so now I'd like to ask for further advice - specifically:

o am I even on the right track here, or is there more to it than this?
o how much of the stuff I compile do I need to install?  can I just drop
  the new cygwin1.dll and libcygwin.a into the appropriate places, or do
  I need to install other stuff as well (e.g. the new .a's in w32api)?

thanks in advance

-John

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/