X-Recipient: archive-cygwin AT delorie DOT com
DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org C1D663857712
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com;
	s=default; t=1680809245;
	bh=epjTO0FcM67P2twsj/jjOYjJ/lmErs/NHmwvkgUnEJ8=;
	h=References:In-Reply-To:Date:Subject:To:Cc:List-Id:
	 List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe:
	 From:Reply-To:From;
	b=KPGFVdUSfqwMn3GbVGrSPBBX7oU3BFeQB7kujoxDoaSr70GWVnRSADCmceQW8T9Fz
	 3S9GfRoHrpDY7xX2FfReYmYxwebtvE2AEQ96XQDdIVaeWSInH4wGnMu512X3kb3sv5
	 zu+U1ngVFWYQTelD7a+lS5yWMN5cwtU8HB01qiLY=
X-Original-To: cygwin AT cygwin DOT com
Delivered-To: cygwin AT cygwin DOT com
DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 3A2073858D28
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112; t=1680809207; x=1683401207;
 h=cc:to:subject:message-id:date:from:in-reply-to:references
 :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id
 :reply-to;
 bh=RgwvxHpq2JyzLseAKeWs3P8LO/4VO7BnC+dDGuEFm/s=;
 b=SNsE4JLpeIIXlxglJa4j+8FeUyYTSp84JcqPNOO3l70WFtqJAVZxtILeuQwxViXLo8
 pzO2n/wxR5kWWo7xen/6cSX1lWoALyfflqHq7RmEz5ljhFjQ9P0O8D+oWkSaS+RTgLj7
 AxL2enoHXjDyqa99QgC6S+MkU3S0ysE44i/TmMyR2KGMVzMH5bxVd3wMt0fUR/BRAmmO
 yG8hLWAvr8vKlhSA8SELYgxj6ZhqIEKp8z5njVMxRv138tGRUfC2gMKXIaAdrNQBD3Nu
 PCt6MHYmFDZulIpgOH+pvzUDHdGh+lmsv0CYfgvY1WmW2SzgHvpYoUh/LputnKQw3Ivj
 4glA==
X-Gm-Message-State: AAQBX9crkVUl0ueu/K2PXVMj7UId2+RQq1ngKgOyGk8X7R7bhiqlVTEl
 vFfde3fWb/725PaW2+3LafdLKh01l9rw3JADs4M=
X-Google-Smtp-Source: AKy350ZnCBOAI2lEJ/GBPVZXBZpG1wAbHH7WJ2NtAH8yHyaZ6hmE4WF3l3RIEiYWdI3Py063eq03S9tuOGmQJW6A02w=
X-Received: by 2002:a05:6402:184e:b0:502:1299:5fa5 with SMTP id
 v14-20020a056402184e00b0050212995fa5mr619392edy.16.1680809206638; Thu, 06 Apr
 2023 12:26:46 -0700 (PDT)
MIME-Version: 1.0
References: <DB6P18901MB005559D81FD4125B0D6DAA07A4919 AT DB6P18901MB0055 DOT EURP189 DOT PROD DOT OUTLOOK DOT COM>
 <20230406171859 DOT ud6jsdxgwtulu6cg AT lucy DOT dinwoodie DOT org>
In-Reply-To: <20230406171859.ud6jsdxgwtulu6cg@lucy.dinwoodie.org>
Date: Thu, 6 Apr 2023 15:26:35 -0400
Message-ID: <CAGHQO_4aFRV7_7W-1GjEYc7W0w4TzXN9imsfuL2uqeT0k1nxOw@mail.gmail.com>
Subject: Re: bash shell script: recently running, now failing
To: Adam Dinwoodie <adam AT dinwoodie DOT org>
Cc: Fergus Daly <fergusd84 AT outlook DOT com>,
        "cygwin AT cygwin DOT com" <cygwin AT cygwin DOT com>
X-Spam-Status: No, score=-0.3 required=5.0 tests=BAYES_00, DKIM_SIGNED,
 DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, HTML_MESSAGE,
 RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS,
 TXREP autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
 server2.sourceware.org
X-Content-Filtered-By: Mailman/MimeDel 2.1.29
X-BeenThere: cygwin AT cygwin DOT com
X-Mailman-Version: 2.1.29
List-Id: General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Archive: <https://cygwin.com/pipermail/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe: <https://cygwin.com/mailman/listinfo/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From: Scott Smith via Cygwin <cygwin AT cygwin DOT com>
Reply-To: Scott Smith <grimblefritz AT gmail DOT com>
Content-Type: text/plain; charset="utf-8"
Sender: "Cygwin" <cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com>
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from base64 to 8bit by delorie.com id 336JRoU8008525

Coming from an enterprise and supercomputing background, we were able to
control what shell was available, so bashisms weren't a problem any more
than dashisms, fishisms, kornisms, perl or python versionisms, etc, might
be.

But, when I was in a commercial environment, everything - shell, perl, C,
text tools, etc - had to be tested and usage adjusted accordingly. It
wasn't uncommon to have a starter script that performed compatibility tests
and forked to the appropriate version... of the same code, functionally.

Back to the OP's problem.

I haven't seen the bash source, but if I had to guess, based on behavior,
bash is only checking the line after the she-bang. With a little more
testing, I've concluded that the nul ^@ can appear anywhere other than in
line #2. My guess is that, after finding the she-bang, bash is reading one
more line to make the "is or is not binary" determination. I imagine that
is oodles less overhead than scanning the entire file.

I have not tested the behavior if the first line is not a she-bang, such as
if the script is run via argument to bash.



On Thu, Apr 6, 2023 at 1:19 PM Adam Dinwoodie via Cygwin <cygwin AT cygwin DOT com>
wrote:

> On Thu, Apr 06, 2023 at 04:43:51AM +0000, Fergus Daly via Cygwin wrote:
> > I have a "hash bang" bash shell script i.e. first line
> > #! /bin/sh
> > or equivalently
> > #! /bin/bash
> > For various reasons I want this file to be identified as binary so its
> second line
> > is the single character null \x00 showing up in some editors e.g. nano as
> >  ^@
> > This does not prevent the script from running to a successful conclusion.
> > Or not until recently. Now the script fails with
> > /home/user/bin/file.old.sh: cannot execute binary file
> > Q1 - was bash recently updated? Would this explain the changed behaviour?
> > Q2 - if so, is this newly introduced "glitch" known and presumably
> intended? Or
> > an unintended consequence that will be retracted in a later update?
> > I then altered the first line to
> > #! /bin/dash
> > whilst retaining the null character at line 2 and subsequent content
> also unaltered..
> > The altered script file.new.sh runs as previously to a successful
> conclusion.
> > Q3 - at 1/8 the size of bash and sh, I am not at all sure of the role
> and reach of dash.
> > Should the edit (dash replacing bash/sh) be incorporated elsewhere or
> would this be a
> > bad idea (and retained only locally in what is indeed an eccentric and
> one-off context)?
>
> Dash is smaller and much less feature-rich than Bash.  Whether Dash is a
> suitable replacement for Bash depends on how much (if at all) you're
> relying on Bash-specific functions.  For very simple scripts, the only
> difference is likely that Dash will be very slightly faster, but working
> out whether your script is using any "Bashisms" isn't always a trivial
> job.
>
> (I have previously been involved work in migrating scripts between Ksh
> and Bash, which is a similar-but-different problem, and there were *a
> lot* of surprises in how the two differed.)
>
> Depending on why you want the file to be identified as a binary, and how
> that identification is being done, you could move your null byte later
> in the file.  In particular, a pattern I've seen several times in Bash
> is to have a normal Bash script, finishing with an explicit `exit`,
> followed by an actual binary blob; this can be used to create things
> like self-extracting bundles, where the binary blob is a tarball and the
> script at the top of the file has the instructions for extracting the
> tarball.
>
> --
> Problem reports:      https://cygwin.com/problems.html
> FAQ:                  https://cygwin.com/faq/
> Documentation:        https://cygwin.com/docs.html
> Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple
>

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple