Mail Archives: cygwin/2021/03/24/05:18:11
| X-Recipient: | archive-cygwin AT delorie DOT com
|
| DKIM-Filter: | OpenDKIM Filter v2.11.0 sourceware.org 52EDF3861030
|
| DKIM-Signature: | v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com;
|
| s=default; t=1616577487;
|
| bh=/jaCGssqwwQWR3lUYCd5UMR/YmPfEj3+y0PRrVqjMOo=;
|
| h=To:References:In-Reply-To:Subject:Date:List-Id:List-Unsubscribe:
|
| List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:
|
| From;
|
| b=oRvwOTTXg9tjhuKb3OU6hS+5AlQWmpKHZKGPx1qMyGoUoDS+DL5ZnlDO8FskG1ngU
|
| 5/jClh9HfwrDLLLFi8DHiIfOSeiuhBuT29Ehe/PwpXwkAw5lcXHzho+763Cc4xVGF5
|
| 7p9oVqJksDu1V7/vwfSdc0GRXozvHWtnCrSStiYg=
|
| X-Original-To: | cygwin AT cygwin DOT com
|
| Delivered-To: | cygwin AT cygwin DOT com
|
| DMARC-Filter: | OpenDMARC Filter v1.3.2 sourceware.org 29BBB3858002
|
| X-Google-DKIM-Signature: | v=1; a=rsa-sha256; c=relaxed/relaxed;
|
| d=1e100.net; s=20161025;
|
| h=x-gm-message-state:from:to:cc:references:in-reply-to:subject:date
|
| :message-id:mime-version:content-language:thread-index;
|
| bh=UuysZKXZRAZd+oApi9xrFKiamjJz2rDYRppE180Q2Dg=;
|
| b=kbD427A3i7VjQkLG84pHMK7YKD9wv73vx2UXKmhqUaMnQ1ZTkj/aL+5uufVnD/K6oF
|
| b0Po0XqiGl8CdRDtzdU5EOHBy74mTuhDuuJ/BNrXiau31LDkEZsw3B1QPqVvk4duo2Az
|
| pyvz7hnG2V45+B88763sidFSv/ZzTr8S5XcHS3Qa0SiG+idh6fG7PXb78C1aoNX/5iEc
|
| KVXarhiqS1UygXUG5H80jSrWiyM0EoobiEc4OTwovoqAfiEgDaGjXA7bh6cINg/6kAhm
|
| +ffdGXxOQdqa0OCCv8stcbvlQAEQNBQEsPJM6IoVUsSmfDFgtuPXeBP+I+DI0/Pp6+2B
|
| /22g==
|
| X-Gm-Message-State: | AOAM531oLJOhfC7zDhXG0mvjaLviMeWd4o3OAhZFZt1fV7zq2pr/9GDR
|
| jQBT+AV4ySUA/x6Vb5P/jJH0ta4W7xfgLw==
|
| X-Google-Smtp-Source: | ABdhPJyCy656ez9ZzK/SfFtFzt/6XFLJnv//WWo029di0kmGbC4yQs9Y0RPoVi3xDLJsZEpc37vo2Q==
|
| X-Received: | by 2002:a2e:9d8f:: with SMTP id c15mr1506920ljj.494.1616577482910;
|
| Wed, 24 Mar 2021 02:18:02 -0700 (PDT)
|
| To: | "'Glenn Strauss'" <gs-cygwin DOT com AT gluelogic DOT com>
|
| References: | <04cc01d71ffa$7d1e6cf0$775b46d0$@gmail.com>
|
| <YFo/fFC2bITvnVGr AT xps13>
|
| In-Reply-To: | <YFo/fFC2bITvnVGr@xps13>
|
| Subject: | RE: AF_UNIX/SOCK_DGRAM is dropping messages
|
| Date: | Wed, 24 Mar 2021 10:18:02 +0100
|
| Message-ID: | <00d901d7208e$97c05c50$c74114f0$@gmail.com>
|
| MIME-Version: | 1.0
|
| X-Mailer: | Microsoft Outlook 16.0
|
| Thread-Index: | AQIPffBCgY7dkx32YYBd4buxXBOzegICwCl2qhIG+gA=
|
| X-Spam-Status: | No, score=-1.7 required=5.0 tests=BAYES_00, DKIM_SIGNED,
|
| DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,
|
| SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2
|
| X-Spam-Checker-Version: | SpamAssassin 3.4.2 (2018-09-13) on
|
| server2.sourceware.org
|
| X-BeenThere: | cygwin AT cygwin DOT com
|
| X-Mailman-Version: | 2.1.29
|
| List-Id: | General Cygwin discussions and problem reports <cygwin.cygwin.com>
|
| List-Archive: | <https://cygwin.com/pipermail/cygwin/>
|
| List-Post: | <mailto:cygwin AT cygwin DOT com>
|
| List-Help: | <mailto:cygwin-request AT cygwin DOT com?subject=help>
|
| List-Subscribe: | <https://cygwin.com/mailman/listinfo/cygwin>,
|
| <mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
|
| From: | Kristian Ivarsson via Cygwin <cygwin AT cygwin DOT com>
|
| Reply-To: | sten DOT kristian DOT ivarsson AT gmail DOT com
|
| Cc: | cygwin AT cygwin DOT com
|
| Sender: | "Cygwin" <cygwin-bounces AT cygwin DOT com>
|
This is a multipart message in MIME format.
------=_NextPart_000_00DA_01D72096.F985D5C0
Content-Type: text/plain;
charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Hi Glenn
Thanks for the reply, so more below
> > Hi all
> >
> > Using AF_UNIX/SOCK_DGRAM with current version (3.2.0) seems to drop =
messages or at least they are not received in the same order they are =
sent
> >
> > Attached C:ish (with C++ threads though) sample program that =
essentially creates a "client" that writes as much as possible and a =
"server" that consumes as much as possible
> >
> > It seems like some buffer is filled and then messages are dropped =
(or at least it appears so) (if someone is about to test this, the =
"sleep" might need to be adjusted in order to make this happen)
> >
> > Hopefully it's just a flaw in our application (and sample), but as =
far as we can see, this should work
> >
> >
> > Does anyone, perhaps named Ken, have any insightful thoughts about =
this ?
>=20
>=20
> > const int size =3D BUFSIZ * 2;
>=20
>=20
> > char buffer[size] =3D {};
> >
> > for( int idx =3D 0; idx < count; ++idx)
> > {
> > memcpy( buffer, &idx, sizeof idx);
> >
> > const ssize_t result =3D sendto( fd, buffer, size, 0, =
(struct
> > sockaddr*)&address, sizeof address);
>=20
>=20
> > const ssize_t result =3D recv( fd, buffer, size, 0);
> ...
> > int index =3D 0;
> > memcpy( &index, buffer, sizeof idx);
>=20
> This appears to be a programming error, unrelated to Cygwin.
>=20
> I know that what you provided was an example test case, but you might =
want to check if your app is sending way too much when the actual =
payload size is much smaller. In the example you provided, you are =
sending 16KB instead of 4 bytes for the count.
To send a larger buffer (in this case 16 KB) is intentional, but just =
the sizeof int is relevant. The reason is just to send many bytes and =
verify that they end up on the other side in correct order
> Is your code handling partial read/recv and partial write/sendto? (It =
is definitely a bug in the use of recv() in the sample code.)=20
It was not and the updated version does not either, but that is not the =
issue though but I added a test to verify that the whole chunk is =
sent/read
> Partial reads and writes can occur more frequently with non-blocking =
sockets, but it is still good defensive programming to detect and handle =
partial read/writes.
That might be the case, but this is blocking attempts though (or maybe =
I've misunderstood the flags ?), but regardless of that, the test-case =
is not about how to handle partial writes/reads though, but to kind of =
show that messages seems to be lost, but of cource code need not to be =
flawed so thanx for the feedback
It almost seems like it is UDP-semantics and that packages can get lost =
or end up in non sequential order, and of course SOCK_DGRAM tells you =
that, but the posix description says "UNIX domain datagram sockets are =
always reliable and don't reorder datagrams"
It seems like when an internal buffer or so of 64 KB is filled the rest =
of the packages are dropped until consumed, so in this case the 32 first =
packages arrive in correct order but after that any random package (with =
index > 32) seems to end up at the "server"
> It goes without saying that if your protocol sends a fixed size chunk =
of data, that you should ensure that you read the entire fixed size, =
even if only using part of the data.
That's done in the updated version, or at least verified
> Cheers, Glenn
Best regards,
Kristian
------=_NextPart_000_00DA_01D72096.F985D5C0
Content-Type: text/plain;
name="af_unix.cpp"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
filename="af_unix.cpp"
#include <sys/socket.h>
#include <sys/un.h>
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <thread>
#include <chrono>
// $ g++ --std=3Dgnu++17 af_unix.cpp
const char* const path =3D "address";
const int count =3D 1000;
const int size =3D BUFSIZ * 2;
int client()
{
const int fd =3D socket( AF_UNIX, SOCK_DGRAM, 0);
if( fd =3D=3D -1)
{
perror( "socket error");
return -1;
}
struct sockaddr_un address{};
strcpy( address.sun_path, path);
address.sun_family =3D AF_UNIX;
char buffer[size] =3D {};
for( int idx =3D 0; idx < count; ++idx)
{
memcpy( buffer, &idx, sizeof idx);
const ssize_t result =3D sendto( fd, buffer, size, 0, (struct =
sockaddr*)&address, sizeof address);
if( result =3D=3D -1 || result !=3D size)
{
perror( "sendto error");
return -1;
}
}
close( fd);
return 0;
}
int server()
{
const int fd =3D socket( AF_UNIX, SOCK_DGRAM, 0);
if( fd =3D=3D -1)
{
perror( "socket error");
return -1;
}
struct sockaddr_un address{};
strcpy( address.sun_path, path);
address.sun_family =3D AF_UNIX;
const int result =3D bind( fd, (struct sockaddr*)&address, sizeof =
address);
if( result =3D=3D -1)
{
perror( "bind error");
return -1;
}
return fd;
}
int main( int argc, char* argv[])
{
const int fd =3D server( );
if( fd !=3D -1)
{
fprintf( stdout, "%d\tnumber of packages\n", count);
fprintf( stdout, "%d\tbytes per package\n", size);
std::thread{ [&](){client( );}}.detach();
std::this_thread::sleep_for( std::chrono::microseconds( 500));
=20
char buffer[size] =3D {};
for( int idx =3D 0; idx < count; ++idx)
{
const ssize_t result =3D recv( fd, buffer, size, 0);
if( result =3D=3D -1 || result !=3D size)
{
perror("recv error");
unlink( path);
return -1;
}
int index =3D 0;
memcpy( &index, buffer, sizeof idx);
if( index !=3D idx)
{
fprintf( stderr, "expected %d but got %d\n", idx, =
index);
unlink( path);
return -1;
}
}
close( fd);
unlink( path);
}
return 0;
}
------=_NextPart_000_00DA_01D72096.F985D5C0
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation: https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
------=_NextPart_000_00DA_01D72096.F985D5C0--
- Raw text -