X-Recipient: archive-cygwin AT delorie DOT com
DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org D258C3857C7E
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com;
	s=default; t=1630277009;
	bh=NcDPLgqpK5kQRFlLJkiJcdhIqUMU+l7ISB0u1oFX7v0=;
	h=Subject:To:References:Date:In-Reply-To:List-Id:List-Unsubscribe:
	 List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:
	 From;
	b=Z9o8Vbww+4K/sT5+f22LGZr6+kMcecytdAogRFW+LXgmIAn1sD/8HiX7D+uYAGAYI
	 MS5Y4n37UTBwJuXvReAu+ac87/wfIM0q0Lw4pGR0Az41JkvFPpS+/DmqE/JjBrN+EK
	 VZJBLvq6Wz9gRrtUKm7VGfUIQBIueWduZEiO/tCY=
X-Original-To: cygwin AT cygwin DOT com
Delivered-To: cygwin AT cygwin DOT com
DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 5BEAA385840A
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=RJKO4iemSQaZBKOMTa0BhXRYscb6TZbUxXpGA4AlJUf9jejE2cwnKzIFDFXdAJeOXlBBl4TNguaM61awWeYGtCcrn/4UwNEHD3PLu2lrnMjfxp2jXmQZ9CvQCW4MktB7ZKj7FdFUlzN1b+7MrpIJE3px1vTQs9FzrR+neGn6IMqqntivtFfxEebwqgv1eLYTwkt/MpQHbiMntuGITwYykKBU1TouVHNqD3a8PWRdSvZW/KIxhVWjHKWAhO9ek78gIh6/3I7ZmtAUdNTqXLr2pUCpAIOjesHAxSbYkrbxf+EyrMbzKc938LGnaQ3KY56IXsH+6/iyEWEaB7+BIHEh0w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; 
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Y8LWPQ7oAXW+RsRXwTUrkK2Y+AW3+6LHgo8jiLo7XB4=;
 b=eCZMeJFv5M6lkePqM+sqfcMNwZ/ierz6oWCOCZJ6pIoyHmCSaEhKJOxCSbTvVLfqb7qt6bUgoxlU5INBO++Zi3pa99t2YDVE7xr/CtRTJrXATopmT8SPkJr173/bloOkLdlAHtbIDLWaD7Ncfmjo0KOAdZRAGTc3JkQQBG8HNjrWmQf7HBRoDKHQJmTxX7b6aVFdRfnUW0k6d+2BeIwTn3fLcbDilR+0lBQtoj3DT0F3O2nqDpUdkefMyJ4J8PQmuIsZWUeRuF1HBGWgVYKwIDyaBDnsT90Bn/4pjVrEjp+1vFVb6pb71PToA5DVfPO+KZh6klNtElmu97ABXtIeHQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=cornell.edu; dmarc=pass action=none header.from=cornell.edu;
 dkim=pass header.d=cornell.edu; arc=none
Subject: Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?
To: cygwin AT cygwin DOT com
References: <20210825201845 DOT 07b6400b79dc5558a7761efe AT nifty DOT ne DOT jp>
 <f8106fe7-a2b8-d106-3061-4d888124f4b0 AT cornell DOT edu>
 <20210826062934 DOT 54f2f2216021c095bb7ba13b AT nifty DOT ne DOT jp>
 <d0a8c57d-1ed1-6b4f-c6e7-cbe0e2ec8a1c AT cornell DOT edu>
 <3b560051-ab27-f392-ca4b-d1fd9b5733b0 AT cornell DOT edu>
 <20210827202440 DOT 47706fc2fc07c5e9a1bc0047 AT nifty DOT ne DOT jp>
 <4f2cb5f3-ce9c-c617-f65f-841a5eca096e AT cornell DOT edu>
 <20210828022111 DOT 91ef5b4ff24f6da9fadb489e AT nifty DOT ne DOT jp>
 <YSn3L0W1M527utK0 AT calimero DOT vinschen DOT de>
 <20210828184102 DOT f2206a8a9e5fe5cf24bf5e45 AT nifty DOT ne DOT jp>
 <YSok0PoCQn2TPPrn AT calimero DOT vinschen DOT de>
 <20210829004346 DOT c2f80469abc3a07fd4b2918d AT nifty DOT ne DOT jp>
 <e8caa02f-be85-33bc-3f09-347c1cdb0923 AT cornell DOT edu>
 <20210829174124 DOT 0c1ae6c16a3e8da1f490abc7 AT nifty DOT ne DOT jp>
Message-ID: <6e9bb35e-6f4f-cf78-e515-549da487b5ef@cornell.edu>
Date: Sun, 29 Aug 2021 18:42:06 -0400
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <20210829174124.0c1ae6c16a3e8da1f490abc7@nifty.ne.jp>
Content-Language: en-US
X-ClientProxiedBy: CH2PR05CA0018.namprd05.prod.outlook.com (2603:10b6:610::31)
 To BN7PR04MB4388.namprd04.prod.outlook.com
 (2603:10b6:406:f8::19)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
Received: from [192.168.1.211] (74.69.128.111) by
 CH2PR05CA0018.namprd05.prod.outlook.com (2603:10b6:610::31) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4478.14 via Frontend Transport; Sun, 29 Aug 2021 22:42:07 +0000
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 4237ba4d-6301-414f-3f4a-08d96b3e3b1d
X-MS-TrafficTypeDiagnostic: BN8PR04MB6386:
X-Microsoft-Antispam-PRVS: <BN8PR04MB63866D7519FB0415EC02D07BD8CA9 AT BN8PR04MB6386 DOT namprd04 DOT prod DOT outlook DOT com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 0iw9fzJyUXVDMkT1WyFZc9WO4ExcWEwfpI5Z0yDTgjF3TLb7dlxVnuoelBxLgu29WHhHQB1ExlfvZNX7QSux17G1OCgQyFzn6n8g2ZECOL2CU58iSzWXc9QOPfrZ+7gqqHwMmcV58cbn/ndWcOErrs+ahiXnF3LBfrm1pTm3Vmm/BlDPk5VpZ4Hj4EqJGlX2qpKBH5BUJpnithWClxAC35lU7/csbtLyknYBYs+uxEcfb/p6YeUneuL4GCJu4SZpQyaK6v1ZhZhFcdQhnaIlkJBxDbcLF+fjfw4BquzybNU75bvy/nZ6dMWnw/VPSK2GLm9n9/3ckAbpth6sI+CD4kCQZvvUDtrrYDKNv7MSZNGdFwIxQCMnJAjUROUmORqu7JJ5tzpZuiCGiHVoBSN8233AZKLJUkzHBHNmA2/nAg4wTdf2EVIoD70Je3RwLMsUpd9tyxMSbWv8i0vDa94p1s/l1hIa6FUJPv+1ao87RbPNUQdCMIpL8f4AQ8P0xmkHeuXUiWzByWSjfgWK6h0m4kSlNIFUd4jfzRYVUJ1ojSti8Wp3ypTEclGgvu0wEMth6JUk9MqK9F+7oPfHiRgXmrxxgKBR2zsOq29MHhE64xxBsPSqpAhjOU1MVt4o3y/rBCEkVzdIKKtxCcMhqq+PyMusH6PBqxb+a3bZfZumfetD7CkgVKPemYTaKsGp4SDFplBCr4m3AJDVtBMmGX81pZPgMg2SkXIov+CspdlTH3+dbgZT8rH6+MCbp0Al+MLO7Y0pXxeFd7aYjErZ/+Fc4gjw2DRgMVGTfI4g5IYCpItemHrV8HD+oy0q2ovu8iu8
X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;
 IPV:NLI; SFV:NSPM; H:BN7PR04MB4388.namprd04.prod.outlook.com; PTR:; CAT:NONE;
 SFS:(4636009)(376002)(136003)(396003)(366004)(346002)(39860400002)(66946007)(2616005)(26005)(66476007)(2906002)(75432002)(31686004)(83380400001)(5660300002)(316002)(8676002)(8936002)(86362001)(53546011)(186003)(31696002)(66556008)(956004)(36756003)(478600001)(966005)(6486002)(38100700002)(6916009)(16576012)(786003)(43740500002)(45980500001);
 DIR:OUT; SFP:1102; 
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?Windows-1252?Q?r9R4jqrkZ1h/SIW2bgKeCjDm7hM5BgNouuk3uBTfBwKAjLXcBg0PK0nk?=
 =?Windows-1252?Q?5GwB7g6BGWNjalJldWflBFXM6NswDgTroXgRf4cEg/ush4Z3ZKMzze/3?=
 =?Windows-1252?Q?PCbVXACT/mtJplF3YqfUoKZBA0bcemBc2irw7BpayZ02TPVNCjVVaYPi?=
 =?Windows-1252?Q?nb/WwIp7anz1cxYS/dHT4+FE5+CEyCrQgz9ZSNGGW8wii/0aKfCo1F4+?=
 =?Windows-1252?Q?tVTlkN/tv6uPNvJG66teAxOCIZrm776MIxcPM/vU6uBePlwJWcwOx4GW?=
 =?Windows-1252?Q?HI4/hpsoOzY9TjYMt0DqXum7k3lfIIQJLjj3PayncBrpslBKpmTXIujo?=
 =?Windows-1252?Q?WQMbQ5j4fYjaAU4G6p0y/31dnNKvnlUu+VFsuNe9zrVk4qcxLlZc2ZLC?=
 =?Windows-1252?Q?NXHcVS6SpY0b8NLjEclD0DOMGbdMqL+Z6YR4duLb8XZQQ4fSXrZuAich?=
 =?Windows-1252?Q?L2lVH9utHNJlbcX5BfUumknUwGMySNZmm1H+TUJx9zUcW5yuQtIimeRE?=
 =?Windows-1252?Q?gCr9t3JAQquZxyuJ5b2/p8x0fwAJUEhketjlKOpfEOmzvwIgkdCUVXJM?=
 =?Windows-1252?Q?jsEkNBc8YL0yyk4Z0xZb/Mucu4NAFuRs/PCc4iZ7R5P77VFl99jg+PmG?=
 =?Windows-1252?Q?acQXkXIMvyCaooSH9g4a23aIbUS8L8SUaeGY5zdkPHJ9KfRXuP6War9z?=
 =?Windows-1252?Q?OrExgiItl41QXYhjYhMGM4Zy7NAv0reZQwuLfQLhtNtKUx8KHoKPU9gr?=
 =?Windows-1252?Q?zmD2PldPZyc7NWTGLFIAa80mVwK32Jj5E27IHpnHEcPNBpf68rkMMJSk?=
 =?Windows-1252?Q?qGYaBkzNAQoQBcH4Vj6GQtod/NoR3So/Ti82a9J/bhGkOgox4Wh7ixz4?=
 =?Windows-1252?Q?ENxLqing3+yMEp2jdx7COXPOsnx6PGAN4jh6DDgLIpKtZUfOPYxJQL5m?=
 =?Windows-1252?Q?iMNoxSZAukD0oNRAiTnxL2ExMzBTDIExv8Qkgq8Z2WeYLNagSQlKtZdz?=
 =?Windows-1252?Q?8vnYEp2Ma4D3u0F+IjsP47p0uICTjfGnFcqRqGfKas2Nc+EQItMHLYhm?=
 =?Windows-1252?Q?F+0gPxqkxwBdidye9ahK5KbylCSWSXjU72T3g02Y5RADTOas1o1VriIz?=
 =?Windows-1252?Q?wq7SsnDBDkXADpAQ2coGR88GgGDEfBX6ptLRF9a92CksLatAdEgB3rM4?=
 =?Windows-1252?Q?yF3GQWWAnxYso3wky2BpRJsSZB9emgOolG+icNaxnNaFibUr/cDVIlvZ?=
 =?Windows-1252?Q?2hB2DSkYsHczkR/xtBSsw8XC0I3Q59nmAEFzvKHAXUB6y9vqBcygoHUE?=
 =?Windows-1252?Q?Jjw+cCuXKQmtPuo5cCncYmx1m2uLXl0QIWofx8jPd8az2eYgQQv1w0IE?=
 =?Windows-1252?Q?oNEZXPp1y9ZCBiQGghofGNL0C6Vr28xjXPGluU7qOU4WFj6I0yE5BmXs?=
X-OriginatorOrg: cornell.edu
X-MS-Exchange-CrossTenant-Network-Message-Id: 4237ba4d-6301-414f-3f4a-08d96b3e3b1d
X-MS-Exchange-CrossTenant-AuthSource: BN7PR04MB4388.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Aug 2021 22:42:07.4232 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 5d7e4366-1b9b-45cf-8e79-b14b27df46e1
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: kZO7T4SnDPIxV/+blJdYjsdhJt6Nf5LUcSiAePjhCFc2+kM8K1mal2+565m4kVJkb/8DVTAGMh9GBF3S9fsDyg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR04MB6386
X-Spam-Status: No, score=-3.3 required=5.0 tests=BAYES_00, DKIM_SIGNED,
 DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, JMQ_SPF_NEUTRAL,
 MSGID_FROM_MTA_HEADER, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,
 SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4
X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on
 server2.sourceware.org
X-BeenThere: cygwin AT cygwin DOT com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Unsubscribe: <https://cygwin.com/mailman/options/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=unsubscribe>
List-Archive: <https://cygwin.com/pipermail/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe: <https://cygwin.com/mailman/listinfo/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From: Ken Brown via Cygwin <cygwin AT cygwin DOT com>
Reply-To: Ken Brown <kbrown AT cornell DOT edu>
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="us-ascii"; Format="flowed"
Errors-To: cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com
Sender: "Cygwin" <cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com>

On 8/29/2021 4:41 AM, Takashi Yano wrote:
> Hi Ken,
> 
> On Sat, 28 Aug 2021 16:55:52 -0400
> Ken Brown wrote:
>> On 8/28/2021 11:43 AM, Takashi Yano via Cygwin wrote:
>>> On Sat, 28 Aug 2021 13:58:08 +0200
>>> Corinna Vinschen wrote:
>>>> On Aug 28 18:41, Takashi Yano via Cygwin wrote:
>>>>> On Sat, 28 Aug 2021 10:43:27 +0200
>>>>> Corinna Vinschen wrote:
>>>>>> On Aug 28 02:21, Takashi Yano via Cygwin wrote:
>>>>>>> On Fri, 27 Aug 2021 12:00:50 -0400
>>>>>>> Ken Brown wrote:
>>>>>>>> Two years ago I thought I needed nt_create to avoid problems when calling
>>>>>>>> set_pipe_non_blocking.  Are you saying that's not an issue?  Is
>>>>>>>> set_pipe_non_blocking unnecessary?  Is that the point of your modification to
>>>>>>>> raw_read?
>>>>>>>
>>>>>>> Yes. Instead of making windows read function itself non-blocking,
>>>>>>> it is possible to check if the pipe can be read before read using
>>>>>>> PeekNamedPipe(). If the pipe cannot be read right now, EAGAIN is
>>>>>>> returned.
>>>>>>
>>>>>> The problem is this:
>>>>>>
>>>>>>     if (PeekNamedPipe())
>>>>>>       ReadFile(blocking);
>>>>>>
>>>>>> is not atomic.  I. e., if PeekNamedPipe succeeds, nothing keeps another
>>>>>> thread from draining the pipe between the PeekNamedPipe and the ReadFile
>>>>>> call.  And as soon as ReadFile runs, it hangs indefinitely and we can't
>>>>>> stop it via a signal.
>>>>>
>>>>> Hmm, you are right. Mutex guard seems to be necessary like pty code
>>>>> if we go this way.
>>>>>
>>>>>> Is a blocking ReadFile actually faster than a non-blocking read?  Or
>>>>>> does it mainly depend on BYTE vs. MESSAGE mode?
>>>>>
>>>>> Actually, I don't think so. Perhaps it is not essential problem of
>>>>> overlapped I/O but something is wrong with current pipe code.
>>>>>
>>>>>> What if the pipe is created non-blocking and stays non-blocking all the
>>>>>> time and uses BYTE mode all the time?  Just as sockets, it would always
>>>>>> only emulate blocking mode.  Wouldn't that drop code size a lot and fix
>>>>>> most problems?
>>>>>
>>>>> If 'non-blocking' means overlapped I/O, only the problem will be:
>>>>> https://cygwin.com/pipermail/cygwin/2021-March/247987.html
>>>>
>>>> Sorry if that wasn't clear, but I was not talking about overlapped I/O,
>>>> which we should get rid off, but of real non-blocking mode, which
>>>> Windows pipes are fortunately capable of.
>>>
>>> Do you mean, PIPE_NOWAIT flag? If this flags is specified in
>>> the read pipe, non-cygwin apps cannot read the pipe correctly.
>>
>> While waiting for Corinna's response to this, I have one more question.  Do you
>> understand why nt_create() failed and you had to revert to create()?  Was it an
>> access problem because nt_create requested FILE_WRITE_ATTRIBUTES?  Or did I make
>> some careless mistake in writing nt_create?
> 
> I am sorry but no. I don't understand why piping C# program via
> the pipe created by nt_create() has the issue. I tried to change
> setup parameters in nt_create(), however, I did not succeed it to
> work. I also couldn't find any mistake in nt_create() so far.
> 
> Win32 programs which use ReadFile() and WriteFile() work even
> with the pipe created by nt_create() as well as overlapped I/O.
> 
> What does C# program differ from legacy win32 program at all?

I don't know.

By the way, when I introduced nt_create(), my preference would have been to 
simply change create() to use the NT API, but I was afraid to do that because I 
didn't want to take a chance on breaking something.  That's still my preference, 
if we can find a way to work around this problem with C# programs.

Ken

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple