X-Recipient:	archive-cygwin AT delorie DOT com
DKIM-Filter:	OpenDKIM Filter v2.11.0 sourceware.org 8B8663857C7E
DKIM-Signature:	v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com;
	s=default; t=1630265993;
	bh=rZV96egWPhrrhGviJSYZVbR6ZjS3IYrjE7VfFqSsVUs=;
	h=Date:To:Subject:In-Reply-To:References:List-Id:List-Unsubscribe:
	List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:
	From;
	b=RHuvkpANJx7jGWllM1LbPB4Dv9yrbXLPS5hh8dSdh8Tu85Awuro5PP8feNuDdjWhd
	/45FCpKX9rZKoIb9rfl9t3dQ8+kBljyglrF2dQ2dBjNmgtTfzt++gWBPNgxG0rZ0Rt
	Akf6hdFISsyuDo+cIZpERKyphky5kSm0AUb5MonA=
X-Original-To:	cygwin AT cygwin DOT com
Delivered-To:	cygwin AT cygwin DOT com
DMARC-Filter:	OpenDMARC Filter v1.4.1 sourceware.org DA4DA3858C3B
DKIM-Filter:	OpenDKIM Filter v2.10.3 conssluserg-03.nifty.com 17TJbosU022335
X-Nifty-SrcIP:	[110.4.221.123]
Date:	Mon, 30 Aug 2021 04:37:56 +0900
To:	cygwin AT cygwin DOT com
Subject:	Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?
Message-Id:	<20210830043756.8aa0ada77db0bfbbe3889f62@nifty.ne.jp>
In-Reply-To:	<789f056a-f164-d71d-1dc9-230f5a41846d@cornell.edu>
References:	<41A583E1-C8E7-42AB-9F24-EEC33A41EC60 AT house DOT org>
	<20210825201845 DOT 07b6400b79dc5558a7761efe AT nifty DOT ne DOT jp>
	<f8106fe7-a2b8-d106-3061-4d888124f4b0 AT cornell DOT edu>
	<20210826062934 DOT 54f2f2216021c095bb7ba13b AT nifty DOT ne DOT jp>
	<d0a8c57d-1ed1-6b4f-c6e7-cbe0e2ec8a1c AT cornell DOT edu>
	<3b560051-ab27-f392-ca4b-d1fd9b5733b0 AT cornell DOT edu>
	<20210827202440 DOT 47706fc2fc07c5e9a1bc0047 AT nifty DOT ne DOT jp>
	<4f2cb5f3-ce9c-c617-f65f-841a5eca096e AT cornell DOT edu>
	<20210828022111 DOT 91ef5b4ff24f6da9fadb489e AT nifty DOT ne DOT jp>
	<YSn3L0W1M527utK0 AT calimero DOT vinschen DOT de>
	<20210828184102 DOT f2206a8a9e5fe5cf24bf5e45 AT nifty DOT ne DOT jp>
	<20210829180729 DOT 48b4e877f773cb3980c5766d AT nifty DOT ne DOT jp>
	<789f056a-f164-d71d-1dc9-230f5a41846d AT cornell DOT edu>
X-Mailer:	Sylpheed 3.7.0 (GTK+ 2.24.30; i686-pc-mingw32)
Mime-Version:	1.0
X-Spam-Status:	No, score=-4.3 required=5.0 tests=BAYES_00, DKIM_SIGNED,
	DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, NICE_REPLY_A, RCVD_IN_DNSWL_NONE,
	RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_PASS,
	TXREP autolearn=ham autolearn_force=no version=3.4.4
X-Spam-Checker-Version:	SpamAssassin 3.4.4 (2020-01-24) on
	server2.sourceware.org
X-BeenThere:	cygwin AT cygwin DOT com
X-Mailman-Version:	2.1.29
List-Id:	General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Unsubscribe:	<https://cygwin.com/mailman/options/cygwin>,
	<mailto:cygwin-request AT cygwin DOT com?subject=unsubscribe>
List-Archive:	<https://cygwin.com/pipermail/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe:	<https://cygwin.com/mailman/listinfo/cygwin>,
	<mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From:	Takashi Yano via Cygwin <cygwin AT cygwin DOT com>
Reply-To:	Takashi Yano <takashi DOT yano AT nifty DOT ne DOT jp>
Errors-To:	cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com
Sender:	"Cygwin" <cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com>

This is a multi-part message in MIME format.

--Multipart=_Mon__30_Aug_2021_04_37_56_+0900_47bqBXHA3vMzIuQt
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

On Sun, 29 Aug 2021 11:57:04 -0400
Ken Brown wrote:
> On 8/29/2021 5:07 AM, Takashi Yano via Cygwin wrote:
> > On Sat, 28 Aug 2021 18:41:02 +0900
> > Takashi Yano wrote:
> >> On Sat, 28 Aug 2021 10:43:27 +0200
> >> Corinna Vinschen wrote:
> >>> On Aug 28 02:21, Takashi Yano via Cygwin wrote:
> >>>> On Fri, 27 Aug 2021 12:00:50 -0400
> >>>> Ken Brown wrote:
> >>>>> Two years ago I thought I needed nt_create to avoid problems when calling
> >>>>> set_pipe_non_blocking.  Are you saying that's not an issue?  Is
> >>>>> set_pipe_non_blocking unnecessary?  Is that the point of your modification to
> >>>>> raw_read?
> >>>>
> >>>> Yes. Instead of making windows read function itself non-blocking,
> >>>> it is possible to check if the pipe can be read before read using
> >>>> PeekNamedPipe(). If the pipe cannot be read right now, EAGAIN is
> >>>> returned.
> >>>
> >>> The problem is this:
> >>>
> >>>    if (PeekNamedPipe())
> >>>      ReadFile(blocking);
> >>>
> >>> is not atomic.  I. e., if PeekNamedPipe succeeds, nothing keeps another
> >>> thread from draining the pipe between the PeekNamedPipe and the ReadFile
> >>> call.  And as soon as ReadFile runs, it hangs indefinitely and we can't
> >>> stop it via a signal.
> >>
> >> Hmm, you are right. Mutex guard seems to be necessary like pty code
> >> if we go this way.
> > 
> > I have found that set_pipe_non_blocking() succeeds for both read and
> > write pipes if the write pipe is created by CreateNamedPipe() and the
> > read pipe is created by CreateFile() contrary to the current create()
> > code. Therefore, not only nt_create() but also PeekNamedPipe() become
> > unnecessary.
> > 
> > Please see the revised patch attached.
> 
> That's a great idea.
> 
> I've applied your two patches to the topic/pipe branch.  I also rebased it and 
> did a forced push in order to bring in Corinna's loader script fix.  So you'll 
> have to do 'git fetch' and 'git rebase --hard origin/topic/pipe'.
> 
> Does this now fix all known problems with pipes?

Only the small thing remaining is pipe mode. If the pipe mode
is changed to byte mode, the following issue will be solved.
https://cygwin.com/pipermail/cygwin/2021-March/247987.html

How about the simple patch attached?

The comment in pipe code says:
     Note that the write side of the pipe is opened as PIPE_TYPE_MESSAGE.
     This *seems* to more closely mimic Linux pipe behavior and is
     definitely required for pty handling since fhandler_pty_master
     writes to the pipe in chunks, terminated by newline when CANON mode
     is specified.

This mentions about pty behaiviour in canonical mode, however, the
pty pipe is created as message mode even with this patch. Are there
any other reasons that message mode is preferred for pipe?

-- 
Takashi Yano <takashi DOT yano AT nifty DOT ne DOT jp>

--Multipart=_Mon__30_Aug_2021_04_37_56_+0900_47bqBXHA3vMzIuQt
Content-Type: application/octet-stream;
 name="0003-Cygwin-pipe-Set-byte-mode-as-default-rather-than-mes.patch"
Content-Disposition: attachment;
 filename="0003-Cygwin-pipe-Set-byte-mode-as-default-rather-than-mes.patch"
Content-Transfer-Encoding: base64

RnJvbSBlYzJkNDNmMjVmN2NmNGZjM2IwMWJkMmUxNWUzYjQ4ZTNjOTQ2MWQxIE1vbiBTZXAgMTcg
MDA6MDA6MDAgMjAwMQpGcm9tOiBUYWthc2hpIFlhbm8gPHRha2FzaGkueWFub0BuaWZ0eS5uZS5q
cD4KRGF0ZTogTW9uLCAzMCBBdWcgMjAyMSAwNDoxMjo1NiArMDkwMApTdWJqZWN0OiBbUEFUQ0gg
My8zXSBDeWd3aW46IHBpcGU6IFNldCBieXRlIG1vZGUgYXMgZGVmYXVsdCByYXRoZXIgdGhhbgog
bWVzc2FnZSBtb2RlLgoKQ3VycmVudGx5LCB0aGUgcGlwZSBtb2RlIGlzIG1lc3NhZ2UgYnkgZGVm
YXVsdC4gVGhpcyBjYXVzZXMgYSBpc3N1ZQp3aXRoIEMjICguTkVUKSBwcm9ncmFtcyBpZiB0aGV5
IGFyZSBwaXBlZC4gV2l0aCB0aGlzIHBhdGNoLCBkZWZhdWx0CnBpcGUgbW9kZSBpcyBjaGFuZ2Vk
IHRvIGJ5dGUgbW9kZS4gVGhlIHBpcGUgbW9kZSBjYW4gYmUgcmV2ZXJ0IHRvCm1lc3NhZ2UgbW9k
ZSBieSBzZXR0aW5nIENZR1dJTj1ub3BpcGVfYnl0ZS4KCkFkZHJlc3NlczogaHR0cHM6Ly9jeWd3
aW4uY29tL3BpcGVybWFpbC9jeWd3aW4vMjAyMS1NYXJjaC8yNDc5ODcuaHRtbAotLS0KIHdpbnN1
cC9jeWd3aW4vZ2xvYmFscy5jYyB8IDIgKy0KIDEgZmlsZSBjaGFuZ2VkLCAxIGluc2VydGlvbigr
KSwgMSBkZWxldGlvbigtKQoKZGlmZiAtLWdpdCBhL3dpbnN1cC9jeWd3aW4vZ2xvYmFscy5jYyBi
L3dpbnN1cC9jeWd3aW4vZ2xvYmFscy5jYwppbmRleCA0OGZiMzEyZGUuLjM1MzQzODFlYyAxMDA2
NDQKLS0tIGEvd2luc3VwL2N5Z3dpbi9nbG9iYWxzLmNjCisrKyBiL3dpbnN1cC9jeWd3aW4vZ2xv
YmFscy5jYwpAQCAtNjksNyArNjksNyBAQCBpbnQgTk9fQ09QWSBkeW5hbWljYWxseV9sb2FkZWQ7
CiAvKiBTb21lIENZR1dJTiBlbnZpcm9ubWVudCB2YXJpYWJsZSB2YXJpYWJsZXMuICovCiBib29s
IGFsbG93X2dsb2IgPSB0cnVlOwogYm9vbCBpZ25vcmVfY2FzZV93aXRoX2dsb2I7Ci1ib29sIHBp
cGVfYnl0ZTsKK2Jvb2wgcGlwZV9ieXRlID0gdHJ1ZTsKIGJvb2wgcmVzZXRfY29tOwogYm9vbCB3
aW5jbWRsbjsKIHdpbnN5bV90IGFsbG93X3dpbnN5bWxpbmtzID0gV1NZTV9kZWZhdWx0OwotLSAK
Mi4zMy4wCgo=

--Multipart=_Mon__30_Aug_2021_04_37_56_+0900_47bqBXHA3vMzIuQt
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline


-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple

--Multipart=_Mon__30_Aug_2021_04_37_56_+0900_47bqBXHA3vMzIuQt--

- Raw text -