Date: Wed, 17 Jan 2007 12:53:11 +1100 (EST)
From: Luke Kendall <luke DOT kendall AT cisra DOT canon DOT com DOT au>
Subject: Re: Changed handling of "!" in /bin/sh?
To: cygwin <cygwin AT cygwin DOT com>
In-Reply-To: <45ACD3BE.8000402@byu.net>
MIME-Version: 1.0
Content-Type: TEXT/plain; CHARSET=US-ASCII
Message-Id: <20070117015312.43811842EF@pessard.research.canon.com.au>
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
Precedence: bulk
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com


Executive summary: thanks to Eric's reply and information, I have
a couple of workable soultions.  Thanks, Eric!

For people who want the details, see below.

On 16 Jan, Eric Blake wrote:
>  -----BEGIN PGP SIGNED MESSAGE-----
>  Hash: SHA1
>  
>  According to Luke Kendall on 1/15/2007 9:34 PM:
> > On 15 Jan, Eric Blake wrote:
>  >>  > SHELLOPTS=braceexpand:emacs:hashall:histexpand:history:igncr:interactive-comments:monitor 
>  >>   
>  >>  There you go.  You have "history" enabled in SHELLOPTS, which is a 
>  >>  non-POSIX extension, and explains why /bin/sh is not doing what you 
>  >>  expected.  Your shell scripts are inheriting interactive behavior, and 
>  >>  trying to do history expansion when they encounter !.
> > 
> > Ah!  I'm not setting that - it comes "for free". :-)
>  
>  Yes, bash always has a read-only variable named SHELLOPTS.  The difference
>  is whether it is exported to the environment, or local to the shell.
>  
> > 
> > Any idea how to turn it off?
>  
>  set +o history

Ah, okay.  But I'm still hoping for a solution that won't require me to
edit 200+ scripts (e.g. to add set +o history 2> /dev/null).

> >  I grepped in /etc/* and /etc/profile.d/*
> > for SHELLOPTS but didn't find it.
>  
>  And you won't, because by default the cygwin base files don't currently
>  export SHELLOPTS, for the very reason that it tends to interfere with
>  non-interactive scripts.

Although, as you imply, some of the options are specifically to control
some behaviours of non-interactive scripts.

> >  In the System Env Vars in Windows I
> > define SHELLOPTS to be igncr (only).
>  
>  There you go - that is the action that exported it into the environment,
>  instead of leaving it shell-local.

Yes, because I need a way to affect all the non-interactive scripts for
all the Cygwin users who access the shared drive with the scripts on
it, or who cvs checkout a local copy of scripts.

But thanks for explaining the operation.

> >  If I echo %SHELLOPTS% in a
> > cmd.exe window it's set to igncr only.  What's defining the other
> > things?
>  
>  bash, as part of it's normal operation, makes SHELLOPTS track all shell
>  options, not just the ones requested at startup.

Thanks for that explanation, too.


> >  It's a readonly env variable, isn't it?  I.e. I don't think I
> > can correct it from within a bash shell?
>  
>  You can correct it in bash indirectly by setting shell options.

Do you mean, like adding set +o history into /etc/profile?  Er, but
that would turn it off for interactive use.  And if I set igncr so that
everything can see it then it has a side effect of exporting the
SHELLOPTS, so then the automatically set options are of course in the
env so they affect every sub shell.

Ouch!

It seems like I'm in a catch 22 situation.

Unless I'm still misunderstanding?

> > We are working in a heterogeneous environment, and use CVS for source
> > code (and script) management, so that's why we want to allow CR/LF in
> > script files.
>  
>  Have you considered using text mounts for your script files instead?  I
>  intentionally ordered my recommendations in the release announcement in
>  the order that I thought were most supported; and setting SHELLOPTS is a
>  lower-rated feature (in my mind) than using proper line endings or proper
>  mount points.

The other day I mailed that this was no longer working.  Some more
investigation just now clarifies it: I misunderstood.

(For reference, I wrote Re: CR/LF problems after upgrade

    With a freshly-installed Cygwin from a mirror that's a few days old, and
    with c:\cygwin\bin mounted textmode and a network share directory of
    bash scripts also mounted in textmode, using scripts that had CR/LF
    endings, each blank line in the script caused an error, and there were
    other errors too.

    In short, I don't think the text mounts are doing their magic
    correctly at the moment.  (The scripts mentioned above did work
    correctly in much older Cygwins using text mounts.)
)

I have "\\samba\x" mounted as "X:", a textmode mount.  But (to avoid
having "x:/cygnus" in my ":"-separated PATH), I have "//samba/x/cygnus"
in my PATH.

I assumed (wrongly) that because of the text mount of that share that it
would do the CR/LF mapping.

I think a solution is therefore to mount "x:" in textmode and *also*
mount "//samba/x" in text mode. I just tried this, below, and it does
indeed fix the problem.  So that's one solution - thanks, Eric!

    mkdir /var/samba-x-mountpoint
    mount -tfx //samba/x /var/samba-x-mountpoint

> > Well, I think we have to use it to define igncr.  And all bash users 
> > who use it for interactive shells would expect to have a history, yes!
>  
>  Then write wrappers around your development tools that disable igncr, or
>  write a script and set BASH_ENV pointing to that script that only sets
>  igncr if the current shell is interactive (if $- contains i), rather than
>  setting SHELLOPTS.  Or use /bin/sh instead of /bin/bash as your shell.

Sorry, let me see if I understand.  We want igncr enabled, not disabled
(or the text mount idea above).  By developer tools do you mean all the
scripts?  I just read up on BASH_ENV: so I could have its script say:

    if expr "$-" : ".*i" > /dev/null
    then
	:
    else
	set +o history
    fi

Or, copy /bin/ash.exe to replace /bin/sh.exe.

> > Are the extra options being turned on automatically only for
> > interactive shells?
>  
>  Yes - the fact that you ran bash interactively first is what set the
>  interactive shell options that are in conflict with POSIX, and the fact
>  that you exported SHELLOPTS is why /bin/sh picked up on those same options.

Please let me restate, to check I understand what you mean by "you ran
bash interactively first": you simply  mean, I started an interactive shell?
(So that sees SHELLOPTS in the env because I put it there, then
the interactive shell options get set, which conflict with POSIX.)

> >  Do you mean that if I want POSIX behaviour from
> > shell scripts then I can't have history for interactive shells? 
>  
>  No, you just have to be more careful about how you go about getting igncr
>  behavior.

You mean, as in "follow the tips you outlined above"?  I think I understand.

> > Perhaps a simple workaround is to replace /bin/sh with /bin/ash?
>  
>  If you use /bin/sh as your login shell, then fewer options will be set in
>  SHELLOPTS automatically.

Do you mean, change /etc/passwd so it's /bin/sh, and then change .profile
to exec bash?

> I would not recommend using /bin/ash; the
>  version on cygwin is quite buggy, and lacking several POSIX-compliant
>  fixes that linux compilations have added in the past few years.

Oh!  I didn't know that, thanks.

Er, I just ran cmp on /bin/bash.exe and /bin/sh.exe and they are indeed
the same, and yet I'm having the problem.  I was talking about literally
replacing /bin/sh.exe, i.e. copying over a posix shell.  I think you
were talking about just changing /etc/passwd.

That raises another question, but I think I should ask in a separate 
email.

> > I'm puzzled that others haven't found the same problem, so I still
> > suspect there must be something unusual about my setup.
>  
>  Actually, others have reported the problem of SHELLOPTS interfering with
>  scripts before.  I just haven't had time to try and analyze how easy or
>  hard it would be to patch bash to unset interactive options when starting
>  non-interactively.

Ah, okay.

Anyway, I think I now have a few workarounds, thanks to your patient
explanations.

Many thanks,

luke


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/