X-Recipient:	archive-cygwin AT delorie DOT com
DKIM-Filter:	OpenDKIM Filter v2.11.0 sourceware.org 5D7F03857C73
DKIM-Signature:	v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com;
	s=default; t=1636964384;
	bh=fZekR7+KPHebE8gOf7WgfQ2UqIKIGBuIoXOyVJ5CLkE=;
	h=Date:To:Subject:In-Reply-To:References:List-Id:List-Unsubscribe:
	List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:
	From;
	b=FevgzQW/l68IfL2aiABld3zyXX1ryZbQXeXKS0Lc0tVuxsDi9Uky5ml2mlbu/U1IV
	18+htMugjKrldowesnMDrc/Z0lUhwOQgmoPdz5tdmkG8qHQulwzIynRw0mgRJSkiKz
	29B7x6v033HKgEaj3idu5WPO+Cl+sbYCjBCZC7nA=
X-Original-To:	cygwin AT cygwin DOT com
Delivered-To:	cygwin AT cygwin DOT com
DMARC-Filter:	OpenDMARC Filter v1.4.1 sourceware.org 1FD973858403
DKIM-Filter:	OpenDKIM Filter v2.10.3 conssluserg-05.nifty.com 1AF8IBlC020841
X-Nifty-SrcIP:	[110.4.221.123]
Date:	Mon, 15 Nov 2021 17:18:11 +0900
To:	cygwin AT cygwin DOT com
Subject:	Re: cygwin 3.3.x: another problem that may be related to pipes
Message-Id:	<20211115171811.844dce9cce2b4d13262d64f2@nifty.ne.jp>
In-Reply-To:	<115203324.649908.1636923059546@mail.yahoo.com>
References:	<115203324 DOT 649908 DOT 1636923059546 DOT ref AT mail DOT yahoo DOT com>
	<115203324 DOT 649908 DOT 1636923059546 AT mail DOT yahoo DOT com>
X-Mailer:	Sylpheed 3.7.0 (GTK+ 2.24.30; i686-pc-mingw32)
Mime-Version:	1.0
X-Spam-Status:	No, score=-11.4 required=5.0 tests=BAYES_00, DKIM_SIGNED,
	DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, NICE_REPLY_A,
	RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS,
	TXREP autolearn=ham autolearn_force=no version=3.4.4
X-Spam-Checker-Version:	SpamAssassin 3.4.4 (2020-01-24) on
	server2.sourceware.org
X-BeenThere:	cygwin AT cygwin DOT com
X-Mailman-Version:	2.1.29
List-Id:	General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Unsubscribe:	<https://cygwin.com/mailman/options/cygwin>,
	<mailto:cygwin-request AT cygwin DOT com?subject=unsubscribe>
List-Archive:	<https://cygwin.com/pipermail/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe:	<https://cygwin.com/mailman/listinfo/cygwin>,
	<mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From:	Takashi Yano via Cygwin <cygwin AT cygwin DOT com>
Reply-To:	Takashi Yano <takashi DOT yano AT nifty DOT ne DOT jp>
Cc:	bf <bf2006a AT yahoo DOT com>
Errors-To:	cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com
Sender:	"Cygwin" <cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com>
X-MIME-Autoconverted:	from base64 to 8bit by delorie.com id 1AF8JkAr025881

On Sun, 14 Nov 2021 20:50:59 +0000 (UTC)
bf wrote:
> I've a shell script that processes several hundred xz-compressed text files with a pipeline of the form:
> 
> find . -depth -type f  ... -print0 | \
> xargs -0tI @ xzcat -- @ | \
> iconv --byte-subst='�' --unicode-subst='�' --widechar-subst='�' -f MS-ANSI -t UTF-8 -- | \
> grep ... \
> sed ... | \
> sort ... | \
> sort ...
> 
> . Since updating to cygwin 3.3.x (but not on cygwin 3.2.x) the script consistently fails when decompressing a portion of the files, resulting in lost data, and error messages of the form:
> 
> xzcat: (stdout): Write error: Bad address

Thanks for the report.
I could reproduce your problem and found the cause.

raw_read()/raw_write() in fhandler_pipe.cc seems to need handling
of STATUS_PENDING in nonblocking mode.

I also confirmed the following patch fixes the issue. However, I
am not very sure that this is the right thing.

Corinna, Ken, what do you think?


From 2261c7d7df0173d2fb6755f6b9ccccf02b27015e Mon Sep 17 00:00:00 2001
From: Takashi Yano <takashi DOT yano AT nifty DOT ne DOT jp>
Date: Mon, 15 Nov 2021 12:30:53 +0900
Subject: [PATCH] Cygwin: pipe: Call CancelIo() if I/O is pending in
 nonblocking mode.

- Currently, handling for STATUS_PENDING in raw_read()/raw_write()
  in nonblocking mode is missing. This patch fixes the issue.

Addresses:
  https://cygwin.com/pipermail/cygwin/2021-November/249910.html
---
 winsup/cygwin/fhandler_pipe.cc | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/winsup/cygwin/fhandler_pipe.cc b/winsup/cygwin/fhandler_pipe.cc
index 1ebf4de10..0de01befb 100644
--- a/winsup/cygwin/fhandler_pipe.cc
+++ b/winsup/cygwin/fhandler_pipe.cc
@@ -355,6 +355,9 @@ fhandler_pipe::raw_read (void *ptr, size_t& len)
 	  else
 	    status = io.Status;
 	}
+      else if (status == STATUS_PENDING)
+	/* CancelIo() if STATUS_PENDING in nonblocking mode. */
+	CancelIo (get_handle ());
       if (isclosed ())  /* A signal handler might have closed the fd. */
 	{
 	  if (waitret == WAIT_OBJECT_0)
@@ -538,6 +541,9 @@ fhandler_pipe_fifo::raw_write (const void *ptr, size_t len)
 	  else
 	    status = io.Status;
 	}
+      else if (status == STATUS_PENDING)
+	/* CancelIo() if STATUS_PENDING in nonblocking mode. */
+	CancelIo (get_handle ());
       if (isclosed ())  /* A signal handler might have closed the fd. */
 	{
 	  if (waitret == WAIT_OBJECT_0)
-- 
2.33.0

-- 
Takashi Yano <takashi DOT yano AT nifty DOT ne DOT jp>

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple

- Raw text -