From: "Daniel Colascione" <dan DOT colascione AT gmail DOT com>
To: <cygwin AT cygwin DOT com>
References: <000801cb2383$9c3ad3a0$d4b07ae0$@gmail.com> <20100714184922 DOT GA13548 AT ednor DOT casa DOT cgf DOT cx>
In-Reply-To: <20100714184922.GA13548@ednor.casa.cgf.cx>
Subject: RE: {lp,cb}Reserved2 under Windows 7 and file descriptors
Date: Wed, 14 Jul 2010 15:38:05 -0700
Message-ID: <001001cb23a5$3a879090$af96b1b0$@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain;	charset="us-ascii"
Content-Transfer-Encoding: 7bit
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com

> On Wed, Jul 14, 2010 at 11:37:26AM -0700, Daniel Colascione wrote:
> >Windows 7 WOW64 seems to DTRT with respect to copying the process data;
> >needs_count_in_si_lpres2 can be false except on Vista. I'm testing a
> >cygwin with that modification now and it works fine (persistent Windows
> >7 sporadic issues notwithstanding; those seem to be solved by rebaseall).
> >
> >lpReserved2 working properly leads to another idea: set *lpReserved2 to
> >a non-zero value even when needs_count_in_si_lpres2 is a false;
> >specifically, set it to three and pass the stdin, stdout, and stderr
> >handles and flags via child_info. (child_info would have to begin with
> >the appropriate values.) At the worst, it does no harm; but if the
> >child is using the MS C runtime, it'll pick up the stdin file
> >descriptors and flags from this data area. If the flags are set
> >properly, you should be able to get the C runtime to report _isatty()
> >is true for cygwin ptys. (Though of course GetFileType will still
> >reveal the thing to be a socket.)
> 
> You seem to be jumping into the middle of a discussion here.  How about
> providing some context?

This is from sigproc.cc:

  /* It appears that when running under WOW64 on Vista 64, the first DWORD
     value in the datastructure lpReserved2 is pointing to (msv_count in
     Cygwin), has to reflect the size of that datastructure as used in the
     Microsoft C runtime (a count value, counting the number of elements in
     two subsequent arrays, BYTE[count and HANDLE[count]), even though the C
     runtime isn't used.  Otherwise, if msv_count is 0 or too small, the
     datastructure gets overwritten.

     This seems to be a bug in Vista's WOW64, which apparently copies the
     lpReserved2 datastructure not using the cbReserved2 size information,
     but using the information given in the first DWORD within lpReserved2
     instead.  32 bit Windows and former WOW64 don't care if msv_count is 0
     or a sensible non-0 count value.  However, it's not clear if a non-0
     count doesn't result in trying to evaluate the content, so we do this
     really only for Vista 64 for now.

     Note: It turns out that a non-zero value *does* harm operation on
     XP 64 and 2K3 64 (Crash in CreateProcess call).

     The value is sizeof (child_info_*) / 5 which results in a count which
     covers the full datastructure, plus not more than 4 extra bytes.  This
     is ok as long as the child_info structure is cosily stored within a
bigger
     datastructure. */

While this workaround is indeed necessary for Vista, it's *not* necessary
for Windows 7, which again handles copying the data structure pointed to by
lpReserved2 properly. Therefore, needs_count_in_si_lpres2 can be false on
that platform. Setting that variable to something non-zero is dangerous
because it tells the CRT that that many file descriptor entries are in the
data block, opening up the possibility that child_info will be
misinterpreted as some inherited file descriptors for the CRT.

Furthermore, there is a very long-standing issue with Cygwin pty devices:
while Cygwin programs report true from isatty() when called on a Cygwin PTY,
MSVCRT applications do *not*. From their point of view, Cygwin ptys are not
ttys, which has led to all sorts of trouble over the years. Because Windows
lacks a pseudoconsole facility, it's not possible to solve the problem in
general (though some people, like the author of conin, have taken some steps
in that direction). 

However, due to the way the CRT works, we can fool it into thinking a
passed-in file descriptor is actually a tty. All you need to do is use 3 for
the value of *lpReserved2, then follow it with three flag bytes, then three
HANDLE values --- corresponding respectively to flags[fd0], flags[fd1],
flags[fd2] and fh[0], fh[fd1] and fh[fd2].  This information would be
followed by the normal child_info structure. If stdin, stdout, or stderr is
a Cygwin PTY, Cygwin can manually set the FDEV bit (described in the old
MSDOS headers) in corresponding flag byte, which will make _isatty() return
true in the child.

(Not that I've actually tried it --- it's just an idea.)


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple