DMARC-Filter: OpenDMARC Filter v1.4.2 delorie.com 51JN8ukZ4177824
DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org C39C13858C42
MIME-Version: 1.0
References: <CALXu0Ue6z-QgfG3o7waBQ=n17xhAQAAH_Lx+0tyXGsKzhLMmtg@mail.gmail.com>
 <20250219162341.6c907a00b9bd7b0ae6f78c93@nifty.ne.jp>
In-Reply-To: <20250219162341.6c907a00b9bd7b0ae6f78c93@nifty.ne.jp>
Date: Thu, 20 Feb 2025 00:07:48 +0100
Message-ID: <CAKAoaQ=1QexEh_gzyRPpKsdJ4T4qz3BVM5YgXfnefC2YVcaUOA@mail.gmail.com>
Subject: Pipes/fifos/sockets&&|mmap()|'ed output files in POSIX shell x=$(cmd)
 and x=${ cmd ;
 } statements... / was: Re: [regression-3.6] Pipe between
 Cygwin and non Cygwin (CRT/URT) randomly loses characters
To: cygwin@cygwin.com
From: Roland Mainz via Cygwin <cygwin@cygwin.com>
Reply-To: Roland Mainz <roland.mainz@nrubsig.org>
Content-Type: text/plain; charset="utf-8"
Sender: "Cygwin" <cygwin-bounces~archive-cygwin=delorie.com@cygwin.com>
Content-Transfer-Encoding: 8bit

On Wed, Feb 19, 2025 at 8:23 AM Takashi Yano via Cygwin
<cygwin@cygwin.com> wrote:
>
> On Wed, 19 Feb 2025 07:59:00 +0100
> Cedric Blancher wrote:
> > Good morning!
> >
> > Cygwin 3.6.0-0.374.g4dd859d01c22.x86_64 on Win10/AMD64/64bit:
> >
> > Pipe between Cygwin and non Cygwin (CRT/URT) randomly loses
> > characters. We do not have a reproducer yet, but it is happening at
> > least hourly on our test machines. Sometimes we only get a single
> > character, and sometimes words are missing.
> >
> > Typically it happens like this:
> > bash -c 'typeset bar="$( powershell -Command ...) ; echo "${bar}""',
> > where bar does not contain the full output
>
> Please provide full command line that can reproduce your problem.
> I can not see any pipe in above command.

I think Cedric is referring to the "hidden" pipes/fifos/sockets a
POSIX shell might use internally:
1. The x=$(cmd) and x={ cmd ; } statements have "hidden"
pipes/fifos/sockets to read from the child.
Sometimes it's just "redirect output to file, run child, once child
quits |mmap()| the output into parent shell address space and use that
mapped memory variable content", sometimes it's a traditional pipe or
fifo.
And in case of shells who use |I_PEEK| (e.g. ksh93) they might use
sockets if the systems's fifo implementation does not support
|I_PEEK|.

2. x=$(cmd) is intended to do a |fork()| and run "cmd" from that child
process (the "(" ")" in x=$(cmd) referring to a "subshell"), but many
shells optimise the |fork()| away if it just runs a shell function or
shell builtin command. ksh93's x=${ cmd ; } does not fork because it's
not running in a subshell, and only redirects stdout to a
file/pipe/fifo/socket or |mmap()|'ed output, and uses the resulting
data as value for variable "x"

And that can make testing in this case HARD if you do not know what
the shell implementation is doing. We had "fun" like that in the
OpenSolaris ksh93-integration project long ago, and found LOTS of bugs
in the Solaris kernel&&libc.

> > Same happens with dash and ksh93.

And THAT statement is BAD, because it means basic shell functionality
is broken, affecting dash/bash/ksh93 - reading that IMO means Cygwin
has a problem in the pipe/fifo code... ;-(

Best way forward is to grab the shell testsuite (e.g. both bash and
ast-ksh/ksh93 ship with test suites), and run the pipe/command
substitution tests.

----

Bye,
Roland
-- 
  __ .  . __
 (o.\ \/ /.o) roland.mainz@nrubsig.org
  \__\/\/__/  MPEG specialist, C&&JAVA&&Sun&&Unix programmer
  /O /==\ O\  TEL +49 641 3992797
 (;O/ \/ \O;)

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple