DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org A6A213858D3C
Subject: Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?
To: cygwin AT cygwin DOT com
References: <41A583E1-C8E7-42AB-9F24-EEC33A41EC60 AT house DOT org>
 <20210825201845 DOT 07b6400b79dc5558a7761efe AT nifty DOT ne DOT jp>
 <f8106fe7-a2b8-d106-3061-4d888124f4b0 AT cornell DOT edu>
 <CCE0AD29-F93E-4E4E-84AC-A6252EB1D3B7 AT house DOT org>
Message-ID: <bb75a792-ca44-b7c9-63c4-a246e10186a3@cornell.edu>
Date: Thu, 26 Aug 2021 11:44:39 -0400
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <CCE0AD29-F93E-4E4E-84AC-A6252EB1D3B7@house.org>
Content-Language: en-US
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
Received: from [IPv6:2603:7081:7e3f:3419:b191:35f5:6937:52a]
 (2603:7081:7e3f:3419:b191:35f5:6937:52a) by
 CH0PR08CA0005.namprd08.prod.outlook.com (2603:10b6:610:33::10) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4457.20 via Frontend Transport; Thu, 26 Aug 2021 15:44:40 +0000
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 0a362dbf-b46f-45e2-7f51-08d968a86b7a
X-MS-TrafficTypeDiagnostic: BN7PR04MB4052:
X-Microsoft-Antispam-PRVS: <BN7PR04MB4052054EB0844827A5F67F8DD8C79 AT BN7PR04MB4052 DOT namprd04 DOT prod DOT outlook DOT com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8273;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: n6k38t83Zsj8OkYQxAvCeqzh5CVRyMi1CowW24+ItltmNId7yt4dtfXS/sObn2B/jeAKPhHd+BHDTc7qiZDquqr8H5POksp/ROiXjxvDDm+OnkHWIQLUH5gf+PBuRMQ59rQJ7MMjJJWE8obv9uyco+zW1jF8UaFp+1iX/G9s/XgvP3hI0lZ63aeZ86LJcOhZN7RuABMyiTsMjCreHFkFhi6NRVtms1HLg1o0yMphVDOF39sSaQvr0dxqIpMXseeoD+YGVxkMRSSwek21RkgR8XVtPsCrXzcfEIiFdQmTcSZkftCX6p1fFRGUJpYHh0P+bOp93yD6eYp6sPVGW1mmpURu7wlOhhwmBYHiTVr8w6dx6NxJ3tnJ4KUg5WVd5aiFsrRgy+ydoznKGJTxtAHeLWvDOsn8b7eTzkRcEF/dFmJ8zkVeMUvCViTgQ688gnz/jbNmxsPQiZKoAaLe7l10wpJ9JGRCXWiq7CJJxEuHKbNV7jb4rTJljjAa2mbF7y6c1lLjNOQJV/vsKsKNg4eJzCtwp8pRAm7rFhXAGmNkEpKA+e1OtyCQDBq6ka+3dfwX4Eje1qHmEY8YDyjHPmZhlL4gFaWOQQvrWTjH2dYCyeymBbBlkcnW9Z9ptELx+pUOPF+oh+Bo0TI/gGjWUoaqOo0gXF143XsC7MYNBAdAi+/QKHtcyU3rUP+jlOyy535N7XvsgiX2mOmp1RrvvYErIMGYtTlEWIsIVg6VYN77spmj5WjOlsudO4H0Ecevm38wHb+CKNdzSBtZURohKRNOQE3/0iBvbtCpQVaPKG9JUMzIqawq0l4czrzxMWpJASvt
X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;
 IPV:NLI; SFV:NSPM; H:BN7PR04MB4388.namprd04.prod.outlook.com; PTR:; CAT:NONE;
 SFS:(4636009)(366004)(66556008)(316002)(31686004)(66946007)(66476007)(8936002)(5660300002)(786003)(6916009)(75432002)(53546011)(31696002)(86362001)(186003)(8676002)(2616005)(38100700002)(2906002)(508600001)(36756003)(6486002)(966005)(43740500002)(45980500001);
 DIR:OUT; SFP:1102; 
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?Windows-1252?Q?qcPJ/bNWNwpYlEBaiSgm+TV3A0Dj5zBWuNsqZZeWWE2UY/MtNeRhKb59?=
 =?Windows-1252?Q?fBEFZLAzu/+cqaNF3P8PF4aV5xeSIKgkq23FqlVPRxKFlZribsbV7SJa?=
 =?Windows-1252?Q?WeUui/ReX13JX9u8kQ6iZx5KqhVtKOLiPSrp3rQ9yVfjngSVb2HY8pXS?=
 =?Windows-1252?Q?rySI6F0qqDGen5HGzd5BD94cbCr67c8HQBPEjCj5oCpFHT1eW0OxCXCk?=
 =?Windows-1252?Q?bR3puuOyasEB7EpVRrmgOnvDFan95XcUEV7ND0j4VRf+QbLRkYoyXIx/?=
 =?Windows-1252?Q?GS4T8kHeYNGLgxN61wXr04wAZxvgldtSYkUI20Y0FcJMGXKFJT11PlM/?=
 =?Windows-1252?Q?MdRxSKB0CKOcPgkstdfzWhw9LFTuTWQmMyfxnHteqNAiphakEAhtar24?=
 =?Windows-1252?Q?i9GDqWzrqXhSNFcE4z+PJhSiZKhcPLS82fqcwDZi2o1I65IeRgak9q8D?=
 =?Windows-1252?Q?oyICdHU4F3TZCOxm4mRy91PfZbA5uG7zvfKppSVrPqlmhPUhCo2CRCen?=
 =?Windows-1252?Q?mzvBykXW59M2gY/o+YKIEuQGesz3fGbSISHxIonr0mV47FmAwxyKc8HQ?=
 =?Windows-1252?Q?qLLO+tWzhx5/PD1MpYeXUan4UtjYLlDeh1CZS5HsdbXdiNc57QramcfX?=
 =?Windows-1252?Q?58SPXOdHjVm/ns+AroBHfDMrbnEEIeFeNuGADPyc+zVsbQFAzmSeoFPt?=
 =?Windows-1252?Q?AQyg+aByT9/RbDTmCi34F5hwlYsyO2HtGRsQmYu0TLDVCyc5bX3l9m78?=
 =?Windows-1252?Q?ONIVjexpcWD6jq32a3YAzt/TQCC9SzRE+By9QTtLNRs1ettdvhy9t12j?=
 =?Windows-1252?Q?/e9U1/qcfZGVfWPzyLl2QcGfsSXy0XuhqA4IiIjPIy+VO/RMUAVv9KMn?=
 =?Windows-1252?Q?TV+Y5EhY0xSzxtpU4DYx4BvNRzaEjT4lGM1FxKl1ocj/7wuPLrs1h8FC?=
 =?Windows-1252?Q?NawV14vcBnwiOF3720nVpCOqr4RzVQSD98Ta4aXffCgfjFYE42d1mAah?=
 =?Windows-1252?Q?w2NWM0bKkOM/EF3cq1JecToSMAEf00X3QR0rzNwv3z2afgllUCqbMAGx?=
 =?Windows-1252?Q?vsVMv53VxXjdPgQr0BRioZwxPVIRptw6GTeB75AjgqVH5ljCyYS04L+H?=
 =?Windows-1252?Q?dSavcCZ7/Jhv82hwvE+P7NmDDgzWpXzlvFuyx+yQqUnpaQU5/5KnDAOt?=
 =?Windows-1252?Q?nryY/YSFLQX/Ovtqy1xwRXo8It/JsYwvrt9A5Yh1tq6vEJWTtToKlqvo?=
 =?Windows-1252?Q?gja7XEDXpUXUOHxcGhNPHsnlnNnZKoShdZdEJAQE7P7MZJXXFmiPTcb4?=
 =?Windows-1252?Q?j4PPS3MTkPYiqh8tlgjxkMG4FFNldKewTi4mdrEciDDOywHdCkcmkkjM?=
 =?Windows-1252?Q?J0kEB949QdVNQ4lzdVM8sx23F9xS70AmLtiypE6IsmvhjjNmQgm+Fjb8?=
 =?Windows-1252?Q?xx1NfSXtxR/oyxSiJIGi0s1KRAbRelshBOI+X+/5r8de7vM0FKj1twDe?=
 =?Windows-1252?Q?dD5PMzLZ?=
X-OriginatorOrg: cornell.edu
X-MS-Exchange-CrossTenant-Network-Message-Id: 0a362dbf-b46f-45e2-7f51-08d968a86b7a
X-MS-Exchange-CrossTenant-AuthSource: BN7PR04MB4388.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Aug 2021 15:44:41.8009 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 5d7e4366-1b9b-45cf-8e79-b14b27df46e1
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: KZ5/OAu6hoiO4mV0ANiZyzwScQOjnw7mtuyw6HHNLKcLeIWDtJug6/1ir4ZqosJvlNIIkvSMKXE/GHHQAWOnLA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PR04MB4052
X-Spam-Status: No, score=-4.0 required=5.0 tests=BAYES_00, DKIM_SIGNED,
 DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, JMQ_SPF_NEUTRAL,
 MSGID_FROM_MTA_HEADER, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,
 SPF_PASS, TXREP,
 T_SPF_HELO_TEMPERROR autolearn=ham autolearn_force=no version=3.4.4
X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on
 server2.sourceware.org
X-BeenThere: cygwin AT cygwin DOT com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Unsubscribe: <https://cygwin.com/mailman/options/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=unsubscribe>
List-Archive: <https://cygwin.com/pipermail/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe: <https://cygwin.com/mailman/listinfo/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From: Ken Brown via Cygwin <cygwin AT cygwin DOT com>
Reply-To: Ken Brown <kbrown AT cornell DOT edu>
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="us-ascii"; Format="flowed"
Errors-To: cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com
Sender: "Cygwin" <cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com>

On 8/25/2021 2:18 PM, Chris Roehrig wrote:
> On Wed Aug 25 2021, at 10:52 AM, Ken Brown via Cygwin <cygwin AT cygwin DOT com> wrote:
>> A couple years ago I had an idea for changing the pipe implementation to avoid overlapped I/O:
>>
>>   https://cygwin.com/pipermail/cygwin-patches/2019q2/009393.html
>>   https://cygwin.com/pipermail/cygwin-patches/2019q2/009423.html
>>
>> I never followed up on it.  But if you think it might help with this problem, I could dust it off and try to finish it.
>>
>> Ken
> 
> I'm not familiar enough with the innards of rsync, sshd or cygwin to know how this would work.
> Is it possible to have a new CYGWIN environment option to switch the pipe behaviour without requiring changes to the ssh or rsync source code (and without breaking any existing stuff)?

My proposed change would be purely internal to Cygwin.  There should be no 
user-visible change in behavior except, perhaps, improved performance.  (And we 
don't know yet whether there would be improved performance.)  So I don't see a 
need for a new CYGWIN option.

Ken

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple