X-Recipient:	archive-cygwin AT delorie DOT com
DKIM-Filter:	OpenDKIM Filter v2.11.0 sourceware.org 6C6AA3857809
DKIM-Signature:	v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com;
	s=default; t=1617722746;
	bh=9H6qDI3AS5bsUvD/XCFakXdfL0EoVP92hLdfAAY8hh8=;
	h=Subject:To:References:Date:In-Reply-To:List-Id:List-Unsubscribe:
	List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:
	From;
	b=eahdMPTVsScuZpEfxwFrD41P9Em6I87wgMmdkjeaWog+UPhzhYgFwTa7dOCJC59lB
	e4w8BXYpQAV6H1jXbnaG9H9glLvVxpSHA92r8VaP1pesmyXyvY4v7kh3QuWzXBGvKT
	6uHXtsd95uvseeUTYhdqxmnEhObJLNlhBLC+gMfs=
X-Original-To:	cygwin AT cygwin DOT com
Delivered-To:	cygwin AT cygwin DOT com
DMARC-Filter:	OpenDMARC Filter v1.3.2 sourceware.org 7E6213857809
ARC-Seal:	i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
	b=bsPbe39NYJuAIyzIhKbfdrVLncLpJXsBAg/L0iXeYo63Pu4RqudfoxDgVJC2/SjlguKLe2yC+hYSj+p3TF1K4kydsvlwQKNIs30G5zakXBTLQZhY78nL5ZvciXU23kR0+dRQm7cUUfaBRAphcJMezLNvBjc9ve8G/h9zbPK3d84vqIsuqXhSnSf4hlMgt5pQuFzxMC2ZmTsP6kBoZuxtliec7kEKyrSyUHplcIbNq25/EENv+aaQ5F9YkI0a6RAEWqTJlXZePutQ5O5GEgnIXYBPOYeYIZhqoVc7/gtwa+Iy7FVcGwaa3bACluGg9NDb1sXMHCrqeYFUDu8cKpD8xg==
ARC-Message-Signature:	i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
	s=arcselector9901;
	h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
	bh=729cEKMS697QcSRrS7X+CE5/a75XQMz2luocE0M1ygs=;
	b=E2vm9xWo8Se0OteEzB7u77nKcIIkTmg8e0yjD/A1MFPUq/NmV1N9TnMWMxqxJRMJoExzUstp7ZnOH4Rv4JHn2g4MQtsdpTIK0ZHklL1nTRFkgI5s/UvvXQQkqvMTVScqPqsY77XDGusiaXog98WFLu5vG6XZRJO4dqzcmxZXdDlT16gUU0Xzdk1NUPCD5oCn6JO7yD7ISqoF/sUv6XEftNHPfd7jZGN3kZNtDZUZ0rY5kUAMlWeXtMcCB5DIMv2W+FAKUqxWN1bXHkgA2rOmsJ4ek/V/FveTNej6jkQrjF4rOIcyC2AALdvr0z0kArFTEnIxUmv9okWrlPYx8NWdWQ==
ARC-Authentication-Results:	i=1; mx.microsoft.com 1; spf=pass
	smtp.mailfrom=cornell.edu; dmarc=pass action=none header.from=cornell.edu;
	dkim=pass header.d=cornell.edu; arc=none
Subject:	Re: AF_UNIX/SOCK_DGRAM is dropping messages
To:	sten DOT kristian DOT ivarsson AT gmail DOT com, cygwin AT cygwin DOT com
References:	<04cc01d71ffa$7d1e6cf0$775b46d0$@gmail.com>
	<YFo/fFC2bITvnVGr AT xps13> <00d901d7208e$97c05c50$c74114f0$@gmail.com>
	<860668bf-8cf9-0969-6a01-7fbf8b782db1 AT cornell DOT edu>
	<000901d72607$55dc5a90$01950fb0$@gmail.com>
	<3346cd1c-b93f-83c4-ff26-553ac95ec692 AT cornell DOT edu>
	<7c21a430-9609-7fd4-1a02-8b7c1978d2f8 AT cornell DOT edu>
	<001901d72af4$4009cd50$c01d67f0$@gmail.com>
Message-ID:	<c6082b0e-c11f-693c-7991-42b3b8bc64e4@cornell.edu>
Date:	Tue, 6 Apr 2021 11:24:53 -0400
User-Agent:	Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
	Thunderbird/78.9.0
In-Reply-To:	<001901d72af4$4009cd50$c01d67f0$@gmail.com>
X-Originating-IP:	[24.194.34.31]
X-ClientProxiedBy:	BL1PR13CA0053.namprd13.prod.outlook.com
	(2603:10b6:208:257::28) To BN7PR04MB4388.namprd04.prod.outlook.com
	(2603:10b6:406:f8::19)
MIME-Version:	1.0
X-MS-Exchange-MessageSentRepresentingType:	1
X-MS-PublicTrafficType:	Email
X-MS-Office365-Filtering-Correlation-Id:	00b3009d-8bdb-4bc1-1ccd-08d8f9103b66
X-MS-TrafficTypeDiagnostic:	BN7PR04MB3956:
X-Microsoft-Antispam-PRVS:	<BN7PR04MB395694A36163F3690C8717F5D8769 AT BN7PR04MB3956 DOT namprd04 DOT prod DOT outlook DOT com>
X-MS-Oob-TLC-OOBClassifiers:	OLM:10000;
X-MS-Exchange-SenderADCheck:	1
X-Microsoft-Antispam:	BCL:0;
X-Microsoft-Antispam-Message-Info:	9Wb+s0lfxNp5wKPF/bSUX3fLSsRbMfCgsFA/UmYDQthEZFSw9wTEWlZ1jEgH87QuuZxCD+2d4jytnQPGUP8LRBRE9djwnKuASw8ZRIJZMzA+lSvuS0y4xKCUGj6Narh4czw1u+s+8oX/z2LfifyjdsOyRmmOo30KW4ldsBL9ZPR5jgtK7WBpblzngu53ATeTliFKEwkIRWqon0nxxd7FpSfBuE4pYnqKao6J1ddu/m3RI5ZeSvyhF73kn7Pa9sQNA0NPU9tmBS4F5TTmrN8P76fJDnnYMFm0SuoXe7SE/Pn5tqGmBHxoun73hmBWP/PMXdfBtu8WVpjPDOpXlmAqzSB/ZHslCEnXthFt+DjmAwlqKuWz1Panb8aeL5KbNBQRqJfGblFx7QvAKxbRegMBwehHWF3hEfzFCzB+CjSaDVYomNKNNR0DkaEys34om9SGfXucPNDjxu8whqUT0UMao+ko+gvEbZ2gwlUBaa30YeXZ6CKLwxRAVlRWIzhMPnlm2W5ESPbQUHncD3pSU1zmFG8Yye/PKwlNbh2gLiekeWjhNUUI/nZFnnYKZQykWB16vNnGLS2V76Z4NehvgJg1z3j+0zb7/Y6+YrtItgANCOJhsy6cXytuW+x1kUYybmIGHS8snbOHKwKApChkhV21uRJP1l1fIcNSVekKNLoyeA4w/RV8YolLC0I6FCkejBdqSOKV9ufg3vWdFcmqgpj1IvhX7yb8kwQ+4zmoZwHfQXF4PCqC7QsbWkfoiZ5B/F/6
X-Forefront-Antispam-Report:	CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;
	IPV:NLI; SFV:NSPM; H:BN7PR04MB4388.namprd04.prod.outlook.com; PTR:; CAT:NONE;
	SFS:(4636009)(136003)(376002)(396003)(366004)(39860400002)(346002)(6666004)(83380400001)(15650500001)(786003)(31686004)(956004)(31696002)(8936002)(2906002)(8676002)(36756003)(75432002)(66574015)(38100700001)(6486002)(86362001)(478600001)(26005)(52116002)(16576012)(316002)(66556008)(16526019)(66946007)(2616005)(53546011)(66476007)(5660300002)(186003)(38350700001)(43740500002)(45980500001);
	DIR:OUT; SFP:1102;
X-MS-Exchange-AntiSpam-MessageData:	=?utf-8?B?KzF0cy9QV2lVRVJSNmMvUmNYdDNKM0o5dW8xd1p4MFlzMW1aVnFyWUxPLytJ?=
	=?utf-8?B?K0RKbkEzVTZPTXoyUXJPb0lmdnhzN3ArYWRseXFTVFcrV0xySG9UVTgzZ3Yz?=
	=?utf-8?B?LzBKZ3pSbitLUkduOWhZSEdFTk5LUGxobWpyUnVIUW1kTkFDNGxnTHpEang3?=
	=?utf-8?B?SDRkY3M4bFRTeGV1UXY4SVBEQ1Z3YjltWEVXMUNXK2xJbXQ0TjVNM3lhUGZo?=
	=?utf-8?B?S1pRRmVRZmJram9Nc2JXQ2huWWVpdmg1T053VWc3TFdtZytiMHE4SmZGcGdo?=
	=?utf-8?B?VXFwd3E2aEVqTFRCZXp3QTF0dHNlL2JtSVRXQVQ4Y09PRU11anhlVnZiRXdP?=
	=?utf-8?B?emMxUVZHUDZ3WVIrd1Q4cDRjYThXeG1UYXNsRnFMMnFONkJSbHd0Y29BbHhi?=
	=?utf-8?B?NDkvQjhLbUFMYXNyRmIzM09Ia01FaEkrNkx4eUtKREUwWkRLNHBIQUltMlkw?=
	=?utf-8?B?bU91WlBGMndpZnhLWVNKeFRQcWtRK1dJMlFFUG1GR0lic2FNNTdqR293MHNr?=
	=?utf-8?B?V3JJR0FzUkU0VXBMM25ZaTR3OUlDNlVHdDBUTU5pKzJYYnllOWtsaitKeVoy?=
	=?utf-8?B?eVRQVFhack5LRUdhY1JvOUtycXJscWpPbkNkdWJaOVg2OHkxM3NWYjErK2ZF?=
	=?utf-8?B?Q2daUnlKaEtjQkZ3aEN1Si9WUjRtRDFLUHI3dlY3UUs5ZUN3eS9sQ1ZHM0pq?=
	=?utf-8?B?Rm1QNzZLTFl2R1JQbXQydEZZcjEyRFJSWjh1WWxpbmZjc01xdXZJcUc1cTQv?=
	=?utf-8?B?NENlR0N5TGNjS1JXSGh3TTlnUEhsNVM4RWJBcVFWUkpRWC9IcVVhanVxRkwz?=
	=?utf-8?B?WXJnYTNITFp6YXErUklhRWJhd2Y4bnlmNGZtZlRsTUc4YUhDQm9oRmNDbHBU?=
	=?utf-8?B?dWRkM1V4SDFZd1daQWNOK2QxcWp0NzdsdjhlcWczWm1qMktEbFZHd0xLanlV?=
	=?utf-8?B?ZXRoUnBOd3FtTVpaemt1a3Z2aHRsNEYyOVc0V1NmRTJYd0NsOWswL0JKeTBF?=
	=?utf-8?B?WXQ5ZUtldXAreG1NZEpuQzBjUTZpQ1VzZHhYaVFtVGVtVWdMc3JMbGdoQVdr?=
	=?utf-8?B?RS9LMFBNU3k0OGxRa1FSZ0ZoMjduYkRoQnp6TklSNXF1enJKZzh2OHB2cUky?=
	=?utf-8?B?WS83dlh6VXphRXdkNEJ2UktkTHVtaW5VMVFsSElHUTloR0xrTm1ZWkx4ZzRT?=
	=?utf-8?B?RnZOOUFDMGFzMmdnZ3hjOXgvVkhJYWpkTzVBT1R4MnVtbGZZbUMvVGhOR2Rx?=
	=?utf-8?B?eTVqWnVJZTBlTE8yaytMMGJNb0F6My9wclBjK05mbHV6eGFCS1FQVUJmVUtv?=
	=?utf-8?B?MFRyblptNWppRkFGM1E4Y0lLQXJyWWFXZVh4dy9tUm1PRlQ3VW9wa3I1OFJh?=
	=?utf-8?B?ck1iend6cnpNaC8wcmVKNTlKK21MMTE4d3dNdDJJQmpCQ2RBSllkRWdiVkZY?=
	=?utf-8?B?Y0VrMlVjL3BNSU9UcnNtOXVQTnQvYlRvRDdGSHowRlhBLzNLWTlna1p5M214?=
	=?utf-8?B?UVB6bUhhdGRWV0lLL3hkTU9KUFNHejBnRkovcjErRWk5Yk9HZ3VsSzkrMENr?=
	=?utf-8?B?NWxDRitFa2gya2xINzZxMXRUV3VQS0FtVDdicUZWbjBrYmhjYng0N25WV3RP?=
	=?utf-8?B?RUxNMDBFcDl2Q3lxc2ZxR0hXMUpKeWRsQlp3NlpTbEdBc1hzemlRZktRaEg5?=
	=?utf-8?B?dzZnZGd5UTRURlE5Z0lyMUJOZzF0SEo1d0NyWDlqbmNLS2tPQkRrdm5LVWdJ?=
	=?utf-8?Q?Cj5q+ebUhqeeZ8QuMDx9Cx9rEhOI2UEYE2aQ0vd?=
X-OriginatorOrg:	cornell.edu
X-MS-Exchange-CrossTenant-Network-Message-Id:	00b3009d-8bdb-4bc1-1ccd-08d8f9103b66
X-MS-Exchange-CrossTenant-AuthSource:	BN7PR04MB4388.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs:	Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime:	06 Apr 2021 15:25:38.5465 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader:	Hosted
X-MS-Exchange-CrossTenant-Id:	5d7e4366-1b9b-45cf-8e79-b14b27df46e1
X-MS-Exchange-CrossTenant-MailboxType:	HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName:	wtDVamavkWTvAKSixayAXoVy5HyKDBPk9xmILXh2ZuBB72W7LdqKCoEk1vMPYXV41CfigmTMz9agION1aYS9tg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped:	BN7PR04MB3956
X-Spam-Status:	No, score=-3.9 required=5.0 tests=BAYES_00, DKIM_SIGNED,
	DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, JMQ_SPF_NEUTRAL,
	MSGID_FROM_MTA_HEADER, NICE_REPLY_A, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H2,
	SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2
X-Spam-Checker-Version:	SpamAssassin 3.4.2 (2018-09-13) on
	server2.sourceware.org
X-BeenThere:	cygwin AT cygwin DOT com
X-Mailman-Version:	2.1.29
List-Id:	General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Unsubscribe:	<https://cygwin.com/mailman/options/cygwin>,
	<mailto:cygwin-request AT cygwin DOT com?subject=unsubscribe>
List-Archive:	<https://cygwin.com/pipermail/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe:	<https://cygwin.com/mailman/listinfo/cygwin>,
	<mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From:	Ken Brown via Cygwin <cygwin AT cygwin DOT com>
Reply-To:	Ken Brown <kbrown AT cornell DOT edu>
Errors-To:	cygwin-bounces AT cygwin DOT com
Sender:	"Cygwin" <cygwin-bounces AT cygwin DOT com>
X-MIME-Autoconverted:	from base64 to 8bit by delorie.com id 136FPo6d027561

On 4/6/2021 10:50 AM, sten DOT kristian DOT ivarsson AT gmail DOT com wrote:
> 
>>>>>>>> Using AF_UNIX/SOCK_DGRAM with current version (3.2.0) seems to
>>>>>>>> drop messages or at least they are not received in the same order
>>>>>>>> they are  sent
>>>>
>>>> [snip]
>>>>
>>>>> Thanks for the test case.  I can confirm the problem.  I'm not
>>>>> familiar enough with the current AF_UNIX implementation to debug
>>>>> this easily.  I'd rather spend my time on the new implementation (on
>>>>> the topic/af_unix branch).  It turns out that your test case fails
>>>>> there too, but in a completely different way, due to a bug in sendto
>>>>> for datagrams.  I'll see if I can fix that bug and then try again.
>>>>>
>>>>> Ken
>>>>
>>>> Ok, too bad it wasn't our own code base but good that the "mystery"
>>>> is verified
>>>>
>>>> I finally succeed to build topic/af_unix (after finding out what
>>>> version of zlib was needed), but not with -D__WITH_AF_UNIX to
>>>> CXXFLAGS though and thus I haven’t tested it yet
>>>>
>>>> Is it sufficient to add the define to the "main" Makefile or do you
>>>> have to add it to all the Makefile:s ? I guess I can find out though
>>>
>>> I do it on the configure line, like this:
>>>
>>>    ../af_unix/configure CXXFLAGS="-g -O0 -D__WITH_AF_UNIX" --prefix=...
>>>
>>>> Is topic/af_unix fairly up to date with master branch ?
>>>
>>> Yes, I periodically cherry-pick commits from master to topic/af_unix.
>>> I'lldo that again right now.
>>>
>>>> Either way, I'll be glad to help out testing topic/af_unix
>>>
>>> Thanks!
>>
>> I've now pushed a fix for that sendto bug, and your test case runs without
>> error on the topic/af_unix branch.
> 
> It seems like the test-case do work now with topic/af_unix in blocking mode, but when using non-blocking (with MSG_DONTWAIT) there are some issues I think
> 
> 1. When the queue is empty with non-blocking recv(), errno is set to EPIPE but I think it should be EAGAIN (or maybe the pipe is getting broken for real of some reason ?)
> 
> 2. When using non-blocking recv() and no message is written at all, it seems like recv() blocks forever
> 
> 3. Using non-blocking recv() where the "client" does send less than "count" messages, sometimes recv() blocks forever (as well)
> 
> 
> My naïve analysis of this is that for the first issue (if any) the wrong errno is set and for the second issue it blocks if no sendto() is done after the first recv(), i.e. nothing kicks the "reader thread" in the butt to realise the queue is empty. It is not super clear though what POSIX says about creating blocking descriptors and then using non-blocking-flags with recv(), but this works in Linux any way
> 
> Let me know if I should provide more a specific explanation, but I think minor modifications of the test-case can provoke all behaviours. I think 2 and 3 are of the same reason though (as described above)

Thanks, I'll take a look.

Ken

> 
> 
>> By the way, I think the implementation of sendto/recv for datagrams is very
>> inefficient when there are repeated calls to sendto as in your test case.
>> Nevertheless, your test case actually runs slightly faster on the topic/af_unix
>> branch than it does on master (when the latter succeeds, which it does about
>> half the time for me).  So I'm not sure whether it's worth worrying about this.
> 
> Of course we would like the best throughput possible 😉
> 
>> Here's the issue, briefly.  The communication is done via a Windows named
>> pipe.
>>    The receiver creates the pipe when it creates and binds its socket.  It creates
>> only one pipe instance.  The sender connects to the pipe, writes, and closes its
>> handle.  But the pipe is not available for another sender to connect to until the
>> receiver reads the message, after which it disconnects the sender.
> 
> Ok, in our application we will use long lived descriptors and multiple writers that possible send large business messages (chunked into some smaller pieces per sendto()/recv())
> 
>> Ken[Kristian]
> 
> Best regards,
> Kristian
> 
--
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple

- Raw text -