Subject: RE: Brainstorming a fix for CTRL-C handling in an emacs shell buf fer (non-TTY)
From: Troy Noble <[email-masked]>
To: "'Andrew Innes'" <[email-masked]>, Troy Noble <[email-masked]>
Cc: "[email-masked]'" <[email-masked]>
Date: Thu, 10 May 2001 10:10:22 -0600

> First off, thank you for looking into this problem - it is a significant
> limitation for people using bash and Emacs on Windows, and it would be
> great to get it fixed.  If your patch below does that, I know a number
> of people who will be very happy. :-)

Sadly, my patch only fixes the symptom as Christopher Faylor has pointed
out.  He has asked that I find the root of the problem before they patch
anything.  Which I think is a reasonable request.

I'm now investigating, as time permits, why bash doesn't seem to be
signaling
its children when it receives a SIGINT and its STDIN is a pipe.  That may
actually provide the "true" solution.  The patch I submitted earlier does
work around the particular problem I was experiencing, but will not make
it into the source base, at least not in its present form.  And I
certainly agree with Christopher's reasoning in this regard.

I read your description of what you are trying to do, and it sounds
reasonable... I have two concerns though (possibly unfounded):
1. a process can only have one console (according to
    http://msdn.microsoft.com/library/psdk/winbase/conchar_6smd.htm).
   So all of your emacs shell buffers are going to be attached to the
   same console...  unless you start up one proxy process per shell
   buffer, which would be one way to get around this I suppose.
   If not, when a user opens two or more shell buffers... I'm not sure
   what would happen.  I would suspect the output would get interleaved,
   so when you do ReadConsoleOutput you might not be what you'd expect.
   I'd be sure to test that scenario if you get a chance.
2. If #1 proves not to be a problem, then I would presume you'd need
   a separate console buffer per emacs shell buffer.  So if a user
   opens several shell buffers (I sometimes use up to 10 or 15,
   using M-x rename-buffer between invocations) they might be allocating
   1MB per invocation.  That could add up to quite a bit of memory
   usage potentially.

> Would setting CYGWIN=tty (or presumably something that means the inverse
> of that - CYGWIN=notty perhaps?) tell bash/ncurses/whatever to treat the
> console as a "dumb" terminal?  If not, is there any other way to do
> that, or might it be possible to add a mechanism for doing that?

In my (very limited) experience, if you do not put "tty" in your
CYGWIN variable, cygwin does seem to default to using the older console
routines at least for interrupt detection (i.e. it uses a ctrl_c_handler).
When you run with CYGWIN=tty, it seems to not use a ctrl_c_handler.  Now
whether that corresponds to the behavior you are looking for, I'm
probably not qualified to say.  All I know for sure is that in
order for me to be able to stop non-cygwin child processes using CTRL-C
inside an emacs shell buffer I cannot put "tty" in my CYGWIN variable.

Sorry I can't be of more help.  Maybe Christopher or one of the hard-core
cygwin developers can give more details about tty/notty.  I'm really coming
into this fresh as an end-user trying to help out and contribute a
fix for problems I've encountered personally.  And I admittedly only
have a small part of the picture at the moment, having only read about
1/5 or less of the winsup/cygwin code base thusfar.

Troy

-----Original Message-----
From: Andrew Innes [mailto:[email-masked]]
Sent: Wednesday, May 09, 2001 4:47 PM
To: Troy Noble
Cc: [email-masked]'
Subject: Re: Brainstorming a fix for CTRL-C handling in an emacs shell
buf fer (non-TTY)

Hi Troy (and others),

First off, thank you for looking into this problem - it is a significant
limitation for people using bash and Emacs on Windows, and it would be
great to get it fixed.  If your patch below does that, I know a number
of people who will be very happy. :-)

You mention a setting CYGWIN=tty which I wasn't aware of, but which
sounds like it might be germane to a problem I'm trying to solve.
Perhaps you can tell me whether it would help, or whether there is some
other way to achieve what I want.

I've been working recently on a completely different approach to try and
tackle the broader issue of many programs (not just bash) not working
ideally in NT-Emacs' shell buffers, because they behave differently with
stdin et al connected to pipes instead of a console (typically by
buffering stdout), or because they directly access the console, eg. to
prompt for a password.  The approach I'm experimenting with is to run
apps with stdin/out/err connected to real console input and output
handles, and use input stuffing and screen scraping techniques
implemented in a "proxy" program to connect back with Emacs' pipes.

This is possible using the Win32 api functions WriteConsoleInput,
ReadConsoleOuput and friends (provided for the benefit of telnet servers
I believe, but still useful here).  It turns out that this approach can
work much better than I had always thought, mainly because it is
possible to put a console into a "raw" output mode.  In this raw mode,
output written using WriteConsole or WriteFile (which covers nearly all
command line programs) will get written verbatim to the console output
screen buffer in sequential cells.  This includes control characters
like TAB, CR and LF, and extended characters with the high-bit set -
they get written as the corresponding codepoints in the DOS codepage
selected for that console.  Output can be made to wrap at the end of
lines, so the screen buffer effectively becomes a single large buffer.

By combing the raw output mode with some clever tricks with the console
attributes that get set as characters are written, it is possible to
recover the exact byte sequence output by a program to the console (as
long as the polling interval is short enough that the console screen
buffer isn't ever completely filled - but on NT, the screen buffer can
be made very large, on the order of 1 MB or more, so the risk of this
can be made very small - certainly good enough for an Emacs shell buffer
on modern hardware).

Ironically, the only serious problem I currently have is that bash is
now console aware, and makes use of the more advanced APIs for directly
addressing the console buffer.  This is certainly good when running bash
in a normal console window, but unfortunately it really messes up my
attempts to screen scrape its output.

So I was wondering if there was a way to tell bash not to use its direct
screen addressing support even though it is apparently running in a
normal console window (with stdin et al connected to the console).  I
thought possibly bash (or ncurses or whatever actually provides this
functionality) may look at the setting of TERM and only enable this
support for a particular setting, but since TERM is already set to
"emacs" in an Emacs shell buffer, that doesn't seem to be the case.

Would setting CYGWIN=tty (or presumably something that means the inverse
of that - CYGWIN=notty perhaps?) tell bash/ncurses/whatever to treat the
console as a "dumb" terminal?  If not, is there any other way to do
that, or might it be possible to add a mechanism for doing that?

Thanks for any help you can offer.

AndrewI

On Sun, 6 May 2001 22:16:38 -0600 , Troy Noble <[email-masked]>
said:
>Christopher,
>
>Looks like t->getpgid() is OK,  it's the
>
t->getpgid () != myself->pid
>
>that's causing the behavior... and I think I understand why
>now.  Thanks for leading me in the right direction.
>
>What do you think about the patch below for exceptions.cc
>instead?  No extra environment variables required this time ;->
>
>Note that I also determined that FSF Emacs users SHOULD NOT use
>CYGWIN=tty if they plan to run inside a shell buffer.  Otherwise
>the CTRL-C handler doesn't get setup or invoked properly for
>reasons I probably don't need to elaborate on.
>
>This likely explains why some folks have reported not being able
>to kill other non-cygwin child processes (like java.exe for example)
>when running bash in a shell buffer.  They probably had CYGWIN=tty
>set.  You probably already knew that, I thought I'd just pass it
>along for others who might be having that same problem.
>
>
>--- exceptions.cc-orig	Fri May  4 23:07:20 2001
>+++ exceptions.cc	Sun May  6 21:47:03 2001
>@@ -11,6 +11,7 @@
> #include "winsup.h"
> #include <imagehlp.h>
> #include <errno.h>
>+#include <unistd.h>
> 
> #include "exceptions.h"
> #include "sync.h"
>@@ -904,8 +905,21 @@
>   tty_min *t = cygwin_shared->tty.get_tty (myself->ctty);
>   /* Ignore this if we're not the process group lead since it should be
>handled
>      *by* the process group leader. */
>-  if (t->getpgid () && (t->getpgid () != myself->pid ||
>-      (GetTickCount () - t->last_ctrl_c) < MIN_CTRL_C_SLOP))
>+    /* Note: t->getpgid() does not correctly reflect the pid of the
process
>+       group leader when stdin is a pipe (when isatty(0) == 0).
>+       So when a native Win32 program like NT Emacs opens a pipe and
>supplies
>+       that as stdin for a cygwin process, this handler should respond to
>the
>+       CTRL-C when pid == pgid rather than when pid == t->getpgid().
>+       Inside a console window or cygwin-aware XEmacs or rxvt for example,
>+       isatty(0) == 1 so the proper behavior is to kill the process whose
>+       pid == t->getpgid().
>+       NT Emacs users should never set "tty" in their CYGWIN environment
>+       variable if they plan to run bash in an Emacs buffer.
>+       */
>+  if (t->getpgid() &&
>+      ((!isatty(0) && myself->pid != myself->pgid) ||
>+       (isatty(0) && (t->getpgid () != myself->pid)) ||
>+       ((GetTickCount () - t->last_ctrl_c) < MIN_CTRL_C_SLOP)))
>     return TRUE;
>   else
>     /* Otherwise we just send a SIGINT to the process group and return
TRUE
>(to indicate
>
>
>
>Just as Andrew Innes said in
>
>    http://sources.redhat.com/ml/cygwin/2001-01/msg00928.html
>
>FSF Emacs 20.7.1 opens a pipe and passes the read end to the
>bash child process as its stdin.  I read through the code just
>to see what it was doing so I could understand what was going on.
>
>Back on the cygwin side...
>
>In dtable::init_std_file_from_handle at line 202 I see:
>
>      else if (GetFileType (handle) == FILE_TYPE_PIPE)
>
>Through a long sequence of steps, the end result is
>that the array of fds gets initialized in such a way that
>isatty(0) always returns 0.  Therefore the code that switches
>the tty's PGID when a child forks never kicks in.
>
>I should add... I see this as a GOOD thing that you would not
>want to change, otherwise you'd never be able to kill the process
>leader for a set of commands in a pipe.
>
>So the simplest solution seems to be to change the ctrl_c_handler
>per the above patch.
>
>I too wrote a simple win32 program that emulated the behavior
>of NT Emacs's method of pipe/stdin handling.  And was able
>to reproduce it quite easily.  I can send the sources along if
>you'd like... so you wouldn't have to download/install FSF Emacs
>if you didn't want to ;->
>
>An interesting way to reproduce all of this without
>running FSF Emacs or even writing a test program:
>1. put the following code in the exceptions.cc:ctrl_c_handler
>     printf("isatty(0)=%d t->getpgid()=%d myself->pid=%d pgid=%d ppid=%d
>progname=%s\n",
>	isatty(0), t->getpgid(), myself->pid, myself->pgid, myself->ppid,
myself->progname);
>2. start up a bash shell in a console window.
>3. create a file /tmp/tmp.sh that looks like:
>      sleep 10
>4. run bash </tmp/tmp.sh    (this ensures stdin is a pipe)
>5. press CTRL-C and you'll see something like:
>
>     isatty(0)=0 t->getpgid()=608 myself->pid=112 pgid=608 ppid=608
>         progname=c:\cygwin\bin\sleep.exe
>     isatty(0)=0 t->getpgid()=608 myself->pid=608 pgid=608 ppid=1468
>         progname=c:\cygwin\bin\bash.exe
>     isatty(0)=0 t->getpgid()=608 myself->pid=1468 pgid=1468 ppid=1
>         progname=C:\cygwin\bin\bash.exe
>
>Notice that the child shell (pid=608) is the pgid...
>rather than the child sleep process (pid=112).
>
>Sorry for the long post.  Hope this solves it once and for
>all.  I'm still willing to pursue further if you don't think
>this is the right approach yet.
>
>Thanks again for your help, Troy
>
>
>-----Original Message-----
>From: Christopher Faylor [mailto:[email-masked]]
>Sent: Saturday, May 05, 2001 2:25 PM
>To: [email-masked]'
>Subject: Re: Brainstorming a fix for CTRL-C handling in an emacs shell
>buf fer (non-TTY)
>
>
>On Sat, May 05, 2001 at 02:19:24PM -0600, Troy Noble wrote:
>>So where would you recommend we go from here?
>>
>>Maybe looking into the TTY code, and figuring out why it doesn't
>>like the way Emacs is opening the pipe to stdin on cygwin processes?
>
>Look at the ctrl_c_handler code and tell me why the test for t->getpgid()
>being nonzero is inadequate.
>
>I actually wrote a program to test this and it works fine for me.
>
>I'm not going to download Xemacs, however.  I've heard that it
>is unstable...
>
>cgf
>
>--
>Want to unsubscribe from this list?
>Check out: http://cygwin.com/ml/#unsubscribe-simple
>
>

--
Want to unsubscribe from this list?
Check out: http://cygwin.com/ml/#unsubscribe-simple