DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 4F2AE394CC1F
Subject: Re: AF_UNIX/SOCK_DGRAM is dropping messages
To: cygwin AT cygwin DOT com
References: <04cc01d71ffa$7d1e6cf0$775b46d0$@gmail.com>
 <YFo/fFC2bITvnVGr AT xps13> <00d901d7208e$97c05c50$c74114f0$@gmail.com>
 <860668bf-8cf9-0969-6a01-7fbf8b782db1 AT cornell DOT edu>
 <000901d72607$55dc5a90$01950fb0$@gmail.com>
 <3346cd1c-b93f-83c4-ff26-553ac95ec692 AT cornell DOT edu>
 <7c21a430-9609-7fd4-1a02-8b7c1978d2f8 AT cornell DOT edu>
 <001901d72af4$4009cd50$c01d67f0$@gmail.com>
 <134074c1-4c0b-0842-b88b-536a1ed4aefe AT cornell DOT edu>
 <000e01d7306e$3c265580$b4730080$@gmail.com>
 <19cf8626-c653-76db-a409-730a5aa5c955 AT cornell DOT edu>
 <4380cdea-c95b-d9dc-50e3-e5adabb73b92 AT cornell DOT edu>
Message-ID: <c9d32619-ca02-34be-7a04-e1bbbd067261@cornell.edu>
Date: Wed, 14 Apr 2021 11:53:56 -0400
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.9.1
In-Reply-To: <4380cdea-c95b-d9dc-50e3-e5adabb73b92@cornell.edu>
Content-Language: en-US
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
Received: from [IPv6:2603:7081:7e41:6a00:58b0:dce6:ff81:dc78]
 (2603:7081:7e41:6a00:58b0:dce6:ff81:dc78) by
 CH2PR05CA0027.namprd05.prod.outlook.com (2603:10b6:610::40) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.4042.8 via Frontend Transport; Wed, 14 Apr 2021 15:53:57 +0000
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: ff79878f-b32a-4ad6-2ab5-08d8ff5d83b9
X-MS-TrafficTypeDiagnostic: BN8PR04MB6388:
X-Microsoft-Antispam-PRVS: <BN8PR04MB6388A4EE3656CCF4872C477ED84E9 AT BN8PR04MB6388 DOT namprd04 DOT prod DOT outlook DOT com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: DfrmuYOIqe7xiy6uF8c021hDgi9roCLfxSxGBVZijr/OVTTZ1GUIrh/FeHsVIDaVe16Avd+4x9qzNICn+VJK64I9cqzmlmza5W1JnPxAHtfawgFeribcmN3duslGEjzqRgiPXc8Y2VRbofslsKbE2OLY+uAnHne20IAKNcnqhITnoRmNW1OFtWiPfHxMD15YxDtxMQpHF1n0mZcpYsXZP+PQNcIsZlEKSxwN0M122iR4UEsItOhpXR9E87GS74ILC6pR9GevB6rBAMbjQ+7SauR9rNjAGihDee1x9kf3WDxCS77kkkBnQG9e99LUPTUCz5XPJH+/dKmem16D3GkqMC8cnRtJWFZVFW6E/+Nwn0AWXRwMhQ1rzEOdPoc7WISefhkm6SfvUWflisnwtI9YwNGiLQltLepF/NrmkK4tyboH9XvPxk92Vj57IhI11JWtk1Da/kZ+ipkOABJTToKZKyadscpNnqBsZMt5ngc1yzrJwbz23Dl+kN5ekuHy0eN1G3aP/xswUN3ZhAWKxKRARnsXxvFUIYG4ApjvYovBjH+gQt1Qng/QpowK9ia/9UNHdjQ5oguvqZkGrvTxpnjmg7Mes7dJ6kueV42aY62EUbHd8tIticPVO3XIfqLUPbGRoeq2EoSI8bfqB9GOCRkaBVa6D5IUcCwIETTzKPIH6j8GvxJCJxPSGhHsk1tG12gl/gQh+skS0SctZGP1uCrzmz/gk1vQrsecCSrCbaGz+3lVHndQYlHWbPWD5PoRpI2WLoQDr2lRfxmzuPvl2tiM3g==
X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;
 IPV:NLI; SFV:NSPM; H:BN7PR04MB4388.namprd04.prod.outlook.com; PTR:; CAT:NONE;
 SFS:(4636009)(39860400002)(346002)(136003)(396003)(376002)(366004)(5660300002)(83380400001)(786003)(6486002)(316002)(8676002)(38100700002)(86362001)(31686004)(2616005)(186003)(16526019)(8936002)(478600001)(966005)(6916009)(31696002)(15650500001)(75432002)(2906002)(53546011)(66476007)(66556008)(66946007)(36756003)(52116002)(66574015)(45980500001)(43740500002);
 DIR:OUT; SFP:1102; 
X-MS-Exchange-AntiSpam-MessageData: =?Windows-1252?Q?WI3KcnIATRZJ2pUwidD+9e5GxeNKZ6guvE8CsmWRPBB8Z5wuLYN9IUdL?=
 =?Windows-1252?Q?jax5GWRYUou5xajLUJp6+mvDNBkfXZo7liSiyFwSzoALEtSwdf45xrLK?=
 =?Windows-1252?Q?XE3dnHDAge12G+RgwGpowg/g73DSGUKaRS0YCSf1HWwlOYdFVb+aYuqP?=
 =?Windows-1252?Q?J9kYnF4XSJmK1bho0f6SWm5ZagXzCgasCNsUfx6jTXD5zNKyr9o9x+uj?=
 =?Windows-1252?Q?XZJ/nnCvExare1/K4IkZ0qu8MicN27GqlBhPZkrPXZue9Kvjlm+4nCTN?=
 =?Windows-1252?Q?w6Fp6BBlTB0v9mXdjs62ZdSDEqT/3S6MsNLcbR0Mo9br5+BB6yjENP0o?=
 =?Windows-1252?Q?7bAPxQTuLK29CwUWDYvCBLcsX46onLg/87xD78iwJk46hVRoNqSOtNEo?=
 =?Windows-1252?Q?ll0oPTpeQJHyfa+lasoSn2p+nol+0GCT9U0QCWyQn7UpgjOCyDmOh3gw?=
 =?Windows-1252?Q?aAPeWEQ1nlTlPl5OkitKMxQ1l2C8TLbExgmpd/NjGxnrQ2YECM0D3QWg?=
 =?Windows-1252?Q?XTdUgSNgevpNTU7PGLbSorna6zs7tbqknANXL/Y9HfLajH34XvKwDSh/?=
 =?Windows-1252?Q?6fRv+Lr9PC0zt8AxfhHIJx28fgAOdiH5STCPXeIqzTdab9Odb+tmFLKP?=
 =?Windows-1252?Q?hQgQqWq9rqUm/AQGN3Q88d6Ovk3qxIzQaOQm3Exi1p905sk+GTMDbLEv?=
 =?Windows-1252?Q?WqQsgRvi6ZL2pWUzpw2kdSDATcejF6bM6XIm35QFQE4HckdpBJo+va3z?=
 =?Windows-1252?Q?Cjc/eIqKAfq5w44cloW5Jm8LAl8T9fr50MM2W6LffAjMj16mQGfAMXah?=
 =?Windows-1252?Q?9K58hIrDpZD4wOeAav8UP8LpWz8mnugViqeoFfJogjLbL17Cf9AARMl7?=
 =?Windows-1252?Q?C6TwWpEorw6h6uqwR+eGoKJlW3Ye3VJxG9aE0oU0vvQcpwSezR19U40P?=
 =?Windows-1252?Q?evb1YU5NQwCeIDdUYwVdBZ2WKt+RTPftY3BXTNro9WbYoWiezi7wQ5ZG?=
 =?Windows-1252?Q?4oAmxlJdLQ6/N/A8CES888T/yUvvCtxcnB2U2taaVEIzo/IkWGFtVpmQ?=
 =?Windows-1252?Q?szOIcMrZ9CcQtWruaoceiXVhSIiJdmw6ioM/DqYdA36aEdH0qX/aBVCK?=
 =?Windows-1252?Q?f4/luupZQTJ3ukD7KOd2TQoZ7XjOioDzJwLSR+AaIkYM+WhU4cyU2xv1?=
 =?Windows-1252?Q?T900LFZ6AMNOl8pOvcEelGFMvbf5gk06Utb77040LHcU/U36HdlRoj8L?=
 =?Windows-1252?Q?OC2+QTfLKEG1u6DQR3Cyst2t0jzt1li/UVt3QppFgBERKQ4XqArFmcNv?=
 =?Windows-1252?Q?31H4fZs/NPT6neEpUoeCtCi/dvCSrz3dgFQ7ylIbpXrD5K1JpUazgEux?=
 =?Windows-1252?Q?AtwRizNVMJ/L6PHwA8TUe/PP0JpXXeD7UOT6Iv59exizfGfvzvTu9Z9K?=
 =?Windows-1252?Q?IWxXpilq2TBpZouUpIhAHPr90fGPSb2XCpXDvAYmWSmSTH/AbxwwlRsV?=
 =?Windows-1252?Q?wFWslI62?=
X-OriginatorOrg: cornell.edu
X-MS-Exchange-CrossTenant-Network-Message-Id: ff79878f-b32a-4ad6-2ab5-08d8ff5d83b9
X-MS-Exchange-CrossTenant-AuthSource: BN7PR04MB4388.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2021 15:53:58.2995 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 5d7e4366-1b9b-45cf-8e79-b14b27df46e1
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: jE89PBljgSbnSa5UmvuBdxc0jnhl1qUpWZkVHZSW7B1BwykWuo8jARy4iBuz7BTOSjjqTzK5EZY8x1+2CY8Piw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR04MB6388
X-Spam-Status: No, score=1.6 required=5.0 tests=BAYES_00, DKIM_INVALID,
 DKIM_SIGNED, JMQ_SPF_NEUTRAL, KAM_DMARC_STATUS, MSGID_FROM_MTA_HEADER,
 NICE_REPLY_A, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS,
 TXREP, URIBL_SBL, URIBL_SBL_A autolearn=no autolearn_force=no version=3.4.2
X-Spam-Level: *
X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on
 server2.sourceware.org
X-BeenThere: cygwin AT cygwin DOT com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Unsubscribe: <https://cygwin.com/mailman/options/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=unsubscribe>
List-Archive: <https://cygwin.com/pipermail/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe: <https://cygwin.com/mailman/listinfo/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From: Ken Brown via Cygwin <cygwin AT cygwin DOT com>
Reply-To: Ken Brown <kbrown AT cornell DOT edu>
Content-Type: text/plain; charset="windows-1252"; Format="flowed"
Errors-To: cygwin-bounces AT cygwin DOT com
Sender: "Cygwin" <cygwin-bounces AT cygwin DOT com>
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by delorie.com id 13EFs6MY017580

On 4/13/2021 6:43 PM, Ken Brown via Cygwin wrote:
> On 4/13/2021 10:47 AM, Ken Brown via Cygwin wrote:
>> On 4/13/2021 10:06 AM, sten DOT kristian DOT ivarsson AT gmail DOT com wrote:
>>> Hi Ken
>>>
>>>>>>>>>>>> Using AF_UNIX/SOCK_DGRAM with current version (3.2.0) seems
>>>> to
>>>>>>>>>>>> drop messages or at least they are not received in the same
>>>>>>>>>>>> order they are  sent
>>>>>>>>
>>>>>>>> [snip]
>>>>>>>>
>>>>>>>>> Thanks for the test case.  I can confirm the problem.  I'm not
>>>>>>>>> familiar enough with the current AF_UNIX implementation to debug
>>>>>>>>> this easily.  I'd rather spend my time on the new implementation
>>>>>>>>> (on the topic/af_unix branch).  It turns out that your test case
>>>>>>>>> fails there too, but in a completely different way, due to a bug
>>>>>>>>> in sendto for datagrams.  I'll see if I can fix that bug and then try 
>>>>>>>>> again.
>>>>>>>>>
>>>>>>>>> Ken
>>>>>>>>
>>>>>>>> Ok, too bad it wasn't our own code base but good that the "mystery"
>>>>>>>> is verified
>>>>>>>>
>>>>>>>> I finally succeed to build topic/af_unix (after finding out what
>>>>>>>> version of zlib was needed), but not with -D__WITH_AF_UNIX to
>>>>>>>> CXXFLAGS though and thus I haven’t tested it yet
>>>>>>>>
>>>>>>>> Is it sufficient to add the define to the "main" Makefile or do you
>>>>>>>> have to add it to all the Makefile:s ? I guess I can find out
>>>>>>>> though
>>>>>>>
>>>>>>> I do it on the configure line, like this:
>>>>>>>
>>>>>>>     ../af_unix/configure CXXFLAGS="-g -O0 -D__WITH_AF_UNIX" --
>>>> prefix=...
>>>>>>>
>>>>>>>> Is topic/af_unix fairly up to date with master branch ?
>>>>>>>
>>>>>>> Yes, I periodically cherry-pick commits from master to topic/af_unix.
>>>>>>> I'lldo that again right now.
>>>>>>>
>>>>>>>> Either way, I'll be glad to help out testing topic/af_unix
>>>>>>>
>>>>>>> Thanks!
>>>>>>
>>>>>> I've now pushed a fix for that sendto bug, and your test case runs
>>>>>> without error on the topic/af_unix branch.
>>>>>
>>>>> It seems like the test-case do work now with topic/af_unix in blocking
>>>>> mode, but when using non-blocking (with MSG_DONTWAIT) there are
>>>> some
>>>>> issues I think
>>>>>
>>>>> 1. When the queue is empty with non-blocking recv(), errno is set to
>>>>> EPIPE but I think it should be EAGAIN (or maybe the pipe is getting
>>>>> broken for real of some reason ?)
>>>>>
>>>>> 2. When using non-blocking recv() and no message is written at all, it
>>>>> seems like recv() blocks forever
>>>>>
>>>>> 3. Using non-blocking recv() where the "client" does send less than
>>>>> "count" messages, sometimes recv() blocks forever (as well)
>>>>>
>>>>>
>>>>> My naļve analysis of this is that for the first issue (if any) the
>>>>> wrong errno is set and for the second issue it blocks if no sendto()
>>>>> is done after the first recv(), i.e. nothing kicks the "reader thread"
>>>>> in the butt to realise the queue is empty. It is not super clear
>>>>> though what POSIX says about creating blocking descriptors and then
>>>>> using non-blocking-flags with recv(), but this works in Linux any way
>>>>
>>>> The explanation is actually much simpler.  In the recv code where a bound
>>>> datagram socket waits for a remote socket to connect to the pipe, I simply
>>>> forget to handle MSG_DONTWAIT.  I've pushed a fix.  Please retest.
>>>>
>>>> I should add that in all my work so far on the topic/af_unix branch, I've
>>>> thought mainly about stream sockets.  So there may still be things remaining
>>>> to be implemented for the datagram case.
>>>
>>> I finally got some time to test topic/af_unix in our "real" 
>>> cygwin-application (casual) and unfortunately very few of our unittests pass
>>>
>>> The symptoms are that there's unexpected eternal blocking, sometimes there's 
>>> unexpected EADDRNOTAVAIL, sometimes it looks like some memory corruption(and 
>>> core-dumps)
>>>
>>> Of course the memory corruption etc could be our self and the core-dumpsmight 
>>> be because of uncaught exceptions
>>>
>>> Needles to say is that all unittests pass on Linux, but of course 
>>> cygwin-topic/af_unix could act according to POSIX-standard and the behaviour 
>>> couldbe due to our own misinterpretation of how POSIX works
>>
>> More likely it's due to bugs in the topic/af_unix branch.  This is still very 
>> much a work in progress.
>>
>>> I will try to narrow down the quite complex logic and reproduce the problems
>>
>> That would be ideal.
>>
>>> If you of some reason wanna try it with casual, I'd be glad to help you out 
>>> (it should be easier now that last time (but there might be some 
>>> documentation missing for Cygwin still))
>>>
>>> https://bitbucket.org/casualcore/
>>
>> I'm going on vacation in a few days, but I might do this when I get back.
>>
>> Thanks for your testing.
> 
> By the way, if your code is using datagram sockets, then there are very serious 
> problems with our implementation (even aside from the performance issue that 
> we've already discussed).  For example, I don't know of any reasonable way for 
> select to test whether such a socket is ready for writing.  We'll need to solve 
> that somehow.

I'm going to follow-up on the cygwin-developers list.

Ken
--
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple