DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 39A753858CD1
To: "cygwin AT cygwin DOT com" <cygwin AT cygwin DOT com>
Subject: Re: posix thread scaling issue
Thread-Topic: posix thread scaling issue
Thread-Index: AQHZ3dPAE/RVWXqTpkamsR5tXy0XwA==
Date: Sat, 2 Sep 2023 19:30:16 +0000
Message-ID: <SN7PR14MB700057482EDE3E37825FA92193EBA@SN7PR14MB7000.namprd14.prod.outlook.com>
Accept-Language: fr-CA, en-US
Content-Language: fr-CA
msip_labels: 
x-tmn: [adKNfCJoN4lkNNs+WZqkOzXBSBqHnW3l]
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: SN7PR14MB7000:EE_|BY5PR14MB3543:EE_
x-ms-office365-filtering-correlation-id: 085b6a54-973b-4cfa-66ed-08dbabeb093f
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: HXT00/sAxjiIU6+3/waPfwq/zt5oRoh3bffwDCWOb/7IJ9jlv76vYz5yCE8piGD2B8mYYI2zyT5GQh1or1GziELsTcOd3nRTANjsKHiQisDIbHmvCBhTfcV8NcZxrtRv7RXsd16ycQcOJtX5ng1ZIhbZGnD3FpC/GAalEzl1rEmm4OxaYnnjfCc8tIr2aZ0BY2rZioiBZQTXLctnzlfDkgHCf9AH5GvwEJEeRCskn88k1sC6VNfnWrgl6NSEW4ev+AshjM7cQyl7E+2ip5dn8qfyzdkQdXEC925U1xlgpYJS65wAIleyjzFAjvmHm/dSCa+JoPoLXtJKaH4cHdn1H78J4oTBy+C9nd0BBpGasd4xT/WK7vcOhcNuxLAtps9IbDehTMYi+ZwL+211cO7rT2vRWeSIW6D/sADPlYMqbmvengm/IMya3ojmXUxH2y6pxkz2d8WUhUjXoBrd0HikHixejqxWi2bKhoJaGU7eKvkaZLr0ctedRWNXapLXbjjGeQT1rtu7kADI7gCrZBdI5OgQYthDV+OI1sYp+gFSF/vGcCB9vXVhcQhbu0n0/w/c
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?fcQCb120SmLhbzKMBihELmFTQDuhDtBjYVOouGHmAfzC+z1FmSLLBjI/2H?=
 =?iso-8859-1?Q?pNJkiGEK1Bzux8QmOl+7lppl8jpfC3WtIQNSrhkZU3ajS+yEFsnjShHZFH?=
 =?iso-8859-1?Q?aXTfmT7UYscjcZeSeEvEy16wrw+M7XGui5+tGAB5DbwizpFX4SAUqK1DeR?=
 =?iso-8859-1?Q?bX3v8LIjS8hBv4f8ZQrSVp18gneCjPCubEBSSGKjoIlT9nVo6N91rtSc9G?=
 =?iso-8859-1?Q?kl3hZtkhDrbsSCl6+v8dEvbmvbGaW+UdKDh7lADFH3/zjSMsHr6zUZck42?=
 =?iso-8859-1?Q?6Z8B4Ljo3TpGv9jCcfxlUBrtkjQFqAD6J9Eyokrd9PPTCg147LNcV9km3n?=
 =?iso-8859-1?Q?QLXtpwevfFqUPHhpxNJJdK0tgbGVQXO88iZuub9vsqdR6Kz4AzZAakZvjD?=
 =?iso-8859-1?Q?F7+HTfGLcmjPbtJvr9gtaUikISHyaLgWEGRKNQnbyUwgYxhkVn0w6BaSfS?=
 =?iso-8859-1?Q?hVUUfi6OvkbVRAIB9GstU4kJaJcrvl+AQiAkQgWI5dYJc6hiwNwNifctN7?=
 =?iso-8859-1?Q?nVhcGrHA+zLkfywjzf/+oJNdh69Gje9c6gtqvI6QiSSPXgZdRLxor8NPLm?=
 =?iso-8859-1?Q?AXJy+GMxLSxaAs+v5HT0f5U46uGYpD87lA4uqATWDW67yrvEzgTf9HdKOU?=
 =?iso-8859-1?Q?eUKeDCaJUie7sSFMqvdurJ1TV2B2+ff0YXXpxEuH08m00qvXZZ6nYh6vQz?=
 =?iso-8859-1?Q?oIVO9Cz+rBHj89GphpGAKqVkVXaBwyYkOCzGNAlYRwe2xfiHjVzNZSOwHj?=
 =?iso-8859-1?Q?4KNDS6ZaZ0KR0s21f7B9KTrAuQ4UD4pw7LfJPjl5+SCXjp9XRD1j7Xwvm8?=
 =?iso-8859-1?Q?vrNwzVWnF4Id3y/NfQo3ODQQhyHvb8JNIa9Bfo9tssDyP/KJAT+9KALuGz?=
 =?iso-8859-1?Q?hldN9+JYffa5X/6EJr0pfOJcNWGJATsr2IPYLMyxAE+s630jpH1VmMfPT6?=
 =?iso-8859-1?Q?kgtaDd1Azl6Tgh9q8/ZRMr9mFmIPUKCoPjs4jYS3EP0PDRDEF4XKAak1dW?=
 =?iso-8859-1?Q?xbMrlvNKNkMPighoyV2bdyJenFWix33m8+AHEq9G0z/QNk3LwNDXPtafaz?=
 =?iso-8859-1?Q?ZMJQwDA1ojC4NJQXiCeHjooZR2aEQRGxl3/dTy/Rex8DYalm7vmlh0Xdhc?=
 =?iso-8859-1?Q?/jgWwbpC4nLGKnbMejqrQKKhJIV5CkhcrNilcvLOWWujK2jSlB247HjMAX?=
 =?iso-8859-1?Q?Wz7mNPRYUeE+v10gocbVrJRcEasEvvSmg+UQJN1nKPTTQOIxAmr5Ar1tCC?=
 =?iso-8859-1?Q?VzouO6On+4dC08ILJqJQ=3D=3D?=
MIME-Version: 1.0
X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-cd57b.templateTenant
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: SN7PR14MB7000.namprd14.prod.outlook.com
X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000
X-MS-Exchange-CrossTenant-Network-Message-Id: 085b6a54-973b-4cfa-66ed-08dbabeb093f
X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000
X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Sep 2023 19:30:16.3279 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR14MB3543
X-Spam-Status: No, score=-1.6 required=5.0 tests=BAYES_00, DKIM_SIGNED,
 DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, SPF_HELO_PASS,
 SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
 server2.sourceware.org
X-BeenThere: cygwin AT cygwin DOT com
X-Mailman-Version: 2.1.30
List-Id: General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Archive: <https://cygwin.com/pipermail/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe: <https://cygwin.com/mailman/listinfo/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From: =?utf-8?q?Andr=C3=A9_Bleau_via_Cygwin?= <cygwin AT cygwin DOT com>
Reply-To: =?iso-8859-1?Q?Andr=E9_Bleau?= <Andre_Bleau AT hotmail DOT com>
Content-Type: text/plain; charset="iso-8859-1"
Sender: "Cygwin" <cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com>
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by delorie.com id 382JUwUX003447

Jeff wrote:

> I have a program that is embarrassing parallel.
> On my older computer which has an epyc 7302 (16 cores,  32 threads) it
> scales very well using cygwin, and fully utilized all threads.
> On my new computer which has an epyc 7B13 (64 cores, 128 threads) it
> does not scale very well.

> According to the windows task manager, it only uses 74% of the cpu
> resources.
> The time it takes the program to run on windows is 166 seconds.
> Using the same hardware on a recent version of linux, I can get 100% cpu
> utilization and the program takes 100 seconds to run.

> I suspect there may be something in cygwin that doesn't scale well with
> lots of posix threads.
> I know this is a bit of an unusual situation, but you can buy a 128 core
> / 256 thread system now.

> Enclosed is the output of cygcheck.
> I updated my version of cygwin to be current as of today, Sep 2 2023.

> thanks,
> jeff

Hi Jeff,

Cygwin's memory allocation, and anything that uses it under the hood, such as some containers from the C++ standard library, don't scale well with many threads. I have observed even worse scalling than yours in my own massively multi-thread programs.

My advice would be to either rewrite your program to centralize memory allocation in a specialized thread that serves the other processing threads, or, if you don't need Posix things, compile with the Mingw cross-compiler, which produces code that uses M$ implementation for memory allocation, which scales better.

Regards,

- André Bleau

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple