X-Recipient: archive-cygwin AT delorie DOT com
DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org D55473858015
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com;
	s=default; t=1630252709;
	bh=Ka+prXihLC41GhYREEqbgBknJbmhjmeY6c5O2u9c3sE=;
	h=Subject:To:References:Date:In-Reply-To:List-Id:List-Unsubscribe:
	 List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:
	 From;
	b=XKlGQi5poV2IsaLBHtSLpP3gnKKAxnAyi9VP/FH7NAHhNIloPcrFxTjX0jEDJ1Iuw
	 D3hlij44ptPWMVwEZuA1Yi+9L+KVRDkvS+oG2wRAduz7dYZfmXaBZbGNRU8GNuN7b5
	 kyaSQdaO1i2hsbKHDGfIMGKDT91ADL+wS1CEshvA=
X-Original-To: cygwin AT cygwin DOT com
Delivered-To: cygwin AT cygwin DOT com
DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org BD5DD3858403
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=N5f75N4A/HoXfx3atuyBHaeuBaKpZ3b8HGxqpNged1wGsaQPpLeZhIf1j4qFug5Y/5Mxr+J+DAN86H3aI1f1XlGytQBwjqSkrKXoiqtn/WeITDg/m1kbBmMNSasX1oQfXvWAgN73xZ9CmVMSHih96O1Md0/MksfGhqIt0oIlfwTlzX0QWsqiAbcO7TYAmJlfxA62PD7BPBtrvy2QQ3kfM0uvHlJPMZLSSUCpaeknOkhLmk5vTsIb/YIIRxW7Gdyu1oDXpfsO70c1WSbSc7VdOTji5P9DbvTRgmFGbEjkDVFUayc0oGfFFjS6+Xs5/U1vSQtPgWbPT4IMtWQ+5BXs+g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; 
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=6MNxanTzMOAqA61fxpMLI+4Azhvjbhd+QbwJiojGkEY=;
 b=MdqhQBSZviMt1CSjIo5C+Iy0MErE8BxuYYmpL8QEF4HthYqtmCmjO+CPdS+zTp7QmP2aJ9Woxo3UIbbBZmeWqnZFOg5nT8FWJ0zZ5mAsNa0K+luCR2jRztu75r4Yk5NpfnEacDKwok5y1vxu/LxJn0sQvy52r/Gsny5qqXvbVjGTs1si8DfKzRx/0oZaXzaHDWl96tsssPGGvwUeUABYh9C4+wvTEtAYfK0IV/1w8UeaXZ+UtzhfKT+MPM8Yl5VY1vkwviRvKZ5X3GVLKlYGRh0N87KSp64KUgOa32PEzY+9h9mbYIjQyUAf3JMJvJIVu33DVcZZB7YadodjcnkqTA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=cornell.edu; dmarc=pass action=none header.from=cornell.edu;
 dkim=pass header.d=cornell.edu; arc=none
Subject: Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?
To: cygwin AT cygwin DOT com
References: <41A583E1-C8E7-42AB-9F24-EEC33A41EC60 AT house DOT org>
 <20210825201845 DOT 07b6400b79dc5558a7761efe AT nifty DOT ne DOT jp>
 <f8106fe7-a2b8-d106-3061-4d888124f4b0 AT cornell DOT edu>
 <20210826062934 DOT 54f2f2216021c095bb7ba13b AT nifty DOT ne DOT jp>
 <d0a8c57d-1ed1-6b4f-c6e7-cbe0e2ec8a1c AT cornell DOT edu>
 <3b560051-ab27-f392-ca4b-d1fd9b5733b0 AT cornell DOT edu>
 <20210827202440 DOT 47706fc2fc07c5e9a1bc0047 AT nifty DOT ne DOT jp>
 <4f2cb5f3-ce9c-c617-f65f-841a5eca096e AT cornell DOT edu>
 <20210828022111 DOT 91ef5b4ff24f6da9fadb489e AT nifty DOT ne DOT jp>
 <YSn3L0W1M527utK0 AT calimero DOT vinschen DOT de>
 <20210828184102 DOT f2206a8a9e5fe5cf24bf5e45 AT nifty DOT ne DOT jp>
 <20210829180729 DOT 48b4e877f773cb3980c5766d AT nifty DOT ne DOT jp>
Message-ID: <789f056a-f164-d71d-1dc9-230f5a41846d@cornell.edu>
Date: Sun, 29 Aug 2021 11:57:04 -0400
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <20210829180729.48b4e877f773cb3980c5766d@nifty.ne.jp>
Content-Language: en-US
X-ClientProxiedBy: CH0PR03CA0018.namprd03.prod.outlook.com
 (2603:10b6:610:b0::23) To BN7PR04MB4388.namprd04.prod.outlook.com
 (2603:10b6:406:f8::19)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
Received: from [192.168.1.211] (74.69.128.111) by
 CH0PR03CA0018.namprd03.prod.outlook.com (2603:10b6:610:b0::23) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4457.23 via Frontend Transport; Sun, 29 Aug 2021 15:57:05 +0000
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 14a5333f-8bc4-4d94-605f-08d96b05a5ff
X-MS-TrafficTypeDiagnostic: BN7PR04MB3956:
X-Microsoft-Antispam-PRVS: <BN7PR04MB3956849C7F1C48EC7B18CACCD8CA9 AT BN7PR04MB3956 DOT namprd04 DOT prod DOT outlook DOT com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: Vw004yVz70uSViL9VY4yEsEjaQ5JYqJABYlTqFgHW9Aig9jNtbCk7qc/kcIzDk6bYw91LytM8O3+tEvBpIwOQ8Ns1NnoJqZy5OQwaGM3ymf8zXUfm0ZJq9S19AhqFOkcKWTswGFsKZgmLMoXWqT7G5x49ZdUi0Eyn3f5zTwWqKP61D0+kYdtw39bx26O70S5fer0EqCKDB/S6gs5ibuw3gIcQeY3IXxk+PiSrI4qH+vKMijhPVIeLQF4wWg1n6ndf9AMAvu4X+7f5fEZ9fdAp2szC+vI23kOgsUuNX+xt+linwiV+L+4MQ3L9MjcCXgYf1nJ2msw7z8w1ROAhL2Q1RS1yk4QAk8+o391bQnSC2sRLsFhYjhdpvBwRAmb56EZd3BEHJQXxKvk8V1vwWTTJ45en1FopMPXErPWiTiHZy92ldepfuPBZTePuK+qhTS4wl+qfoejnWEBvtzMFIFWT9sI7eJuL76iHD8EqW0DTmGpb1SxFhm3bU8u9pxEx0wJ7vUmufPSge9vBxrdJPxbtQNlQykGx/zaYMV6afU7Gr4kzp6/kw/gWqrbnxUUTpKZhxAZKg9xQ3uTQ0QH/YlgulEta/C7roemFF4k7TN75Zp30WlvRO2qvXy5p/eB4XyGCOvmBl6xEmClNef/jaXmwbBopQdFnqcSt+7ky5LQNlTl7NNcXe1L/DFI+B+faipL3eyuNqjBn1tFAkzNs6tz1JUoTk9l5Dk9SMjrUaGb28g=
X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;
 IPV:NLI; SFV:NSPM; H:BN7PR04MB4388.namprd04.prod.outlook.com; PTR:; CAT:NONE;
 SFS:(4636009)(346002)(136003)(366004)(396003)(39860400002)(376002)(38100700002)(31696002)(5660300002)(478600001)(86362001)(36756003)(6916009)(53546011)(16576012)(786003)(316002)(83380400001)(75432002)(66476007)(2616005)(956004)(66556008)(66946007)(2906002)(186003)(8676002)(6486002)(31686004)(8936002)(26005)(45980500001)(43740500002);
 DIR:OUT; SFP:1102; 
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?Windows-1252?Q?rfPYK7/UsX163Ca5agMJyTWjtqQl4qkoGx0m3nufx11uXaTnjk7oBsaU?=
 =?Windows-1252?Q?uXHvS1o/EbGfcDxR6bidlckd3ZSVhRGwwU+hUtEKJLi1HH8Ed10WyHzh?=
 =?Windows-1252?Q?j7/IK8Uwpy0SxXogqVNrW/+y7MoWSeOnbCPvE0wqgkJZ1IaEs3cH74xB?=
 =?Windows-1252?Q?F+88h4PTeriTGCgrJcD4nmVnxnZ/BhZhfU+hDxiCxorSbeknxC9BPoIO?=
 =?Windows-1252?Q?3qJaQ86i2y4u21qy3DS3rkOFpjcjqCR2Dzqj8s4n/q9gP5Ua763vJvTW?=
 =?Windows-1252?Q?PPzzAxqudOLVliqCOr3L4JYzufWA7uY8yTe/MhG3McHZT1Z0HqO0roo5?=
 =?Windows-1252?Q?bivI/mJmYSqafvWmwbegHDblDv6SZrLCqRIYbkY3rH3qNBkXBogvEg6D?=
 =?Windows-1252?Q?s+9Is0AtEfXa94pGvwrzVOwyMbrmkvNCiQIAALPZ3a6XzN4B4E7UkSfS?=
 =?Windows-1252?Q?1VFKVU74bUpTz5n+/0nuUI8ptDd4GduBey/29jQUdIRr92qQonz2GgKb?=
 =?Windows-1252?Q?6IyrycSX8UOD6kR0zYj8luOMS1Dq8lygCMYAlp4kA0xnQqWGsE2iAMOI?=
 =?Windows-1252?Q?d6Fv5/llYlLqyWG19XGmE6Tkk9GPaeJq2DjUpsVjc5OSHJWoCURiluM6?=
 =?Windows-1252?Q?jEILtS3TZSYttKjXjwYYbMhBuuOIwIoL92KM92+l8qfzK12P1HXGdEM4?=
 =?Windows-1252?Q?UtsA9qoTbdeiwuGS6Pi1ym6By6PW1VBtek+NAcfy6pwv4XhCgwraiD2o?=
 =?Windows-1252?Q?WlW3V4k2tmhESK8DWE78JeMqhVh2Q/KC9+rLlZZ1Tc7aDiLw29RP+m/k?=
 =?Windows-1252?Q?UpOtpLQ4fDf7bb1raeGTJUQN1QKTWWJ6jLOLoyVFETAlGsbM4L+P+0wv?=
 =?Windows-1252?Q?PAWDiB674x/2ILIW0lKadegfdyv9p+2YBj/MoHG6045PKaZJN82p2tAh?=
 =?Windows-1252?Q?BiEDmOIdQYFWaPQgNTdvTiIYfJYPN0NJbs0v6xA20TidyLGqRJCo5NBX?=
 =?Windows-1252?Q?ri/k45JFRPoXoaX0tdsz+7DjawbWQFbG0a0OzkFOrTqKX7K/CPABdjz8?=
 =?Windows-1252?Q?2AkJvcrVA19g5gARxfBGb9oCHAvbEzhVr4Lmnmw2R1LN1P7L3Ho1mAA8?=
 =?Windows-1252?Q?holYReePtg4hwK5YMKHrM8bspWLJQjoIeJ5xZku6YQv/Lf/ltgVg/nKC?=
 =?Windows-1252?Q?BRMXXHoXKGUBOl0lprEeUSbQwe+bfUUEsIzXB2pVM7t6gUkzzOxuturs?=
 =?Windows-1252?Q?QfKvyJ6kATGKqZN+iUmArIHcOZmJws5dx0WkxLlJPR/Q+WOwj4XwbVCP?=
 =?Windows-1252?Q?3/kN8MIYeHgdGiH3DXePNAu4IdB+1mN+MV73aN2MAoZ8xa9Z23+ZPjjU?=
 =?Windows-1252?Q?RM8Meefp883gc/vkzWG/rRSMwbT20Y6rI52iNoRf+LvBFmxxVYBmurzN?=
X-OriginatorOrg: cornell.edu
X-MS-Exchange-CrossTenant-Network-Message-Id: 14a5333f-8bc4-4d94-605f-08d96b05a5ff
X-MS-Exchange-CrossTenant-AuthSource: BN7PR04MB4388.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Aug 2021 15:57:05.5190 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 5d7e4366-1b9b-45cf-8e79-b14b27df46e1
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: bE2Ze1aCpMsC1KCbii4qImzAPdONyf5zj9iq++p1Q892vFHhjrKaPrtPuZNWajEKY2/R1+p52s04aNJ6JF/1sg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PR04MB3956
X-Spam-Status: No, score=-3.6 required=5.0 tests=BAYES_00, DKIM_SIGNED,
 DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, MSGID_FROM_MTA_HEADER, NICE_REPLY_A,
 RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS,
 TXREP autolearn=ham autolearn_force=no version=3.4.4
X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on
 server2.sourceware.org
X-BeenThere: cygwin AT cygwin DOT com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Unsubscribe: <https://cygwin.com/mailman/options/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=unsubscribe>
List-Archive: <https://cygwin.com/pipermail/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe: <https://cygwin.com/mailman/listinfo/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From: Ken Brown via Cygwin <cygwin AT cygwin DOT com>
Reply-To: Ken Brown <kbrown AT cornell DOT edu>
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="us-ascii"; Format="flowed"
Errors-To: cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com
Sender: "Cygwin" <cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com>

On 8/29/2021 5:07 AM, Takashi Yano via Cygwin wrote:
> On Sat, 28 Aug 2021 18:41:02 +0900
> Takashi Yano wrote:
>> On Sat, 28 Aug 2021 10:43:27 +0200
>> Corinna Vinschen wrote:
>>> On Aug 28 02:21, Takashi Yano via Cygwin wrote:
>>>> On Fri, 27 Aug 2021 12:00:50 -0400
>>>> Ken Brown wrote:
>>>>> Two years ago I thought I needed nt_create to avoid problems when calling
>>>>> set_pipe_non_blocking.  Are you saying that's not an issue?  Is
>>>>> set_pipe_non_blocking unnecessary?  Is that the point of your modification to
>>>>> raw_read?
>>>>
>>>> Yes. Instead of making windows read function itself non-blocking,
>>>> it is possible to check if the pipe can be read before read using
>>>> PeekNamedPipe(). If the pipe cannot be read right now, EAGAIN is
>>>> returned.
>>>
>>> The problem is this:
>>>
>>>    if (PeekNamedPipe())
>>>      ReadFile(blocking);
>>>
>>> is not atomic.  I. e., if PeekNamedPipe succeeds, nothing keeps another
>>> thread from draining the pipe between the PeekNamedPipe and the ReadFile
>>> call.  And as soon as ReadFile runs, it hangs indefinitely and we can't
>>> stop it via a signal.
>>
>> Hmm, you are right. Mutex guard seems to be necessary like pty code
>> if we go this way.
> 
> I have found that set_pipe_non_blocking() succeeds for both read and
> write pipes if the write pipe is created by CreateNamedPipe() and the
> read pipe is created by CreateFile() contrary to the current create()
> code. Therefore, not only nt_create() but also PeekNamedPipe() become
> unnecessary.
> 
> Please see the revised patch attached.

That's a great idea.

I've applied your two patches to the topic/pipe branch.  I also rebased it and 
did a forced push in order to bring in Corinna's loader script fix.  So you'll 
have to do 'git fetch' and 'git rebase --hard origin/topic/pipe'.

Does this now fix all known problems with pipes?

Corinna, do you still see any benefit to switching to PIPE_NOWAIT?  AFAICT, it 
wouldn't decrease the code size at this point, so the only question is whether 
it might improve performance.

If you think it's worth trying, I'd be glad to code it up on a new branch, and 
we could compare the two.

Aside from that, I'm wondering how and when to merge the new pipe implementation 
to master.  It obviously needs much more widespread testing than it's gotten so 
far.  I'm a little nervous about it because I haven't thought about the details 
for two years, and no one other than me has tested it until a few days ago.

Ken

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple