X-Recipient:	archive-cygwin AT delorie DOT com
DKIM-Filter:	OpenDKIM Filter v2.11.0 sourceware.org EA6BF3894C3C
DKIM-Signature:	v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com;
	s=default; t=1601291008;
	bh=mnXmmJlBmRbxhfvku048FwFLV5fR0tRPESVa40j7OJI=;
	h=Subject:To:References:Date:In-Reply-To:List-Id:List-Unsubscribe:
	List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:
	From;
	b=r8oNE5vM4DolWQebcQOIe6rc2NTOvG6WkyiDCaiLHWg4n9oQaIgMnfy7ACfFzCCV8
	Qt2fX8V8vRUdYB5BGB19eeksKg8YQdnmRDRKvkxVnusnXWpqfEZV3iJj7fPumdVWFR
	J+JFpygCU7RcdgGILns3hzbmrG2sr6hx8sszNXnE=
X-Original-To:	cygwin AT cygwin DOT com
Delivered-To:	cygwin AT cygwin DOT com
DMARC-Filter:	OpenDMARC Filter v1.3.2 sourceware.org EE3963857C7C
Subject:	Re: Problems with native Unix domain sockets on Win 10/2019
To:	Ken Brown <kbrown AT cornell DOT edu>, cygwin AT cygwin DOT com
References:	<2b0aeab4-983d-e1d7-301f-edfeeb38cc85 AT oracle DOT com>
	<db0f2634-328c-baaa-1cdb-5bd3c145c9e0 AT cornell DOT edu>
	<bb34a767-0cb5-1d48-7f9b-ad914762f9f7 AT oracle DOT com>
	<97d2b3af-224a-6873-fb4a-55a0ae9cd379 AT cornell DOT edu>
	<d9a6467d-e797-8917-3240-e79d55dcfb38 AT oracle DOT com>
	<3e3cfe17-7fda-b063-4885-9114db9e748d AT cornell DOT edu>
	<70b5577f-2cf1-0110-5d3b-cb2bd8ee6df2 AT cornell DOT edu>
	<69ad720c-8ea6-d3bb-b0a5-5556c4550091 AT oracle DOT com>
Message-ID:	<2d85550f-d753-4055-8b93-35e5287a9a93@oracle.com>
Date:	Mon, 28 Sep 2020 12:03:13 +0100
User-Agent:	Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:68.0)
	Gecko/20100101 Thunderbird/68.12.0
MIME-Version:	1.0
In-Reply-To:	<69ad720c-8ea6-d3bb-b0a5-5556c4550091@oracle.com>
X-Proofpoint-Virus-Version:	vendor=nai engine=6000 definitions=9757
	signatures=668680
X-Proofpoint-Spam-Details:	rule=notspam policy=default score=0 spamscore=0
	mlxlogscore=999
	suspectscore=0 adultscore=0 malwarescore=0 phishscore=0 bulkscore=0
	mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1
	engine=8.12.0-2006250000 definitions=main-2009280088
X-Proofpoint-Virus-Version:	vendor=nai engine=6000 definitions=9757
	signatures=668680
X-Proofpoint-Spam-Details:	rule=notspam policy=default score=0 mlxlogscore=999
	suspectscore=0
	lowpriorityscore=0 spamscore=0 clxscore=1015 mlxscore=0 impostorscore=0
	malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 priorityscore=1501
	classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000
	definitions=main-2009280088
X-Spam-Status:	No, score=-2.6 required=5.0 tests=BAYES_00, BODY_8BITS,
	DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF,
	NICE_REPLY_A, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL,
	SPF_HELO_PASS, SPF_PASS, TXREP,
	UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.2
X-Spam-Checker-Version:	SpamAssassin 3.4.2 (2018-09-13) on
	server2.sourceware.org
X-BeenThere:	cygwin AT cygwin DOT com
X-Mailman-Version:	2.1.29
List-Id:	General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Unsubscribe:	<https://cygwin.com/mailman/options/cygwin>,
	<mailto:cygwin-request AT cygwin DOT com?subject=unsubscribe>
List-Archive:	<https://cygwin.com/pipermail/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe:	<https://cygwin.com/mailman/listinfo/cygwin>,
	<mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From:	Michael McMahon via Cygwin <cygwin AT cygwin DOT com>
Reply-To:	Michael McMahon <michael DOT x DOT mcmahon AT oracle DOT com>
Errors-To:	cygwin-bounces AT cygwin DOT com
Sender:	"Cygwin" <cygwin-bounces AT cygwin DOT com>
X-MIME-Autoconverted:	from quoted-printable to 8bit by delorie.com id 08SB3sw4008927

On 26/09/2020 08:30, Michael McMahon via Cygwin wrote:
> 
> 
> On 25/09/2020 21:30, Ken Brown wrote:
>> On 9/25/2020 2:50 PM, Ken Brown via Cygwin wrote:
>>> On 9/25/2020 10:29 AM, Michael McMahon wrote:
>>>>
>>>>
>>>> On 25/09/2020 14:19, Ken Brown wrote:
>>>>> On 9/24/2020 8:01 AM, Michael McMahon wrote:
>>>>>>
>>>>>>
>>>>>> On 24/09/2020 12:26, Ken Brown wrote:
>>>>>>> On 9/23/2020 7:25 AM, Michael McMahon via Cygwin wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I searched for related issues but haven't found anything.
>>>>>>>>
>>>>>>>> I am having some trouble with Windows native Unix domain sockets
>>>>>>>> (a recent feature in Windows 10 and 2019 server) and Cygwin.
>>>>>>>> I think I possibly know the cause since I had to investigate a 
>>>>>>>> similar
>>>>>>>> looking issue on another platform built on Windows.
>>>>>>>>
>>>>>>>> The problem is that cygwin commands don't seem to recognise 
>>>>>>>> native Unix
>>>>>>>> domain sockets correctly. For example, the socket "foo.sock" should
>>>>>>>> have the same ownership and similar permissions to other files
>>>>>>>> in the example below:
>>>>>>>>
>>>>>>>> $ ls -lrt
>>>>>>>> total 2181303
>>>>>>>>
>>>>>>>> -rw-r--r--  1 mimcmah      None             1259   Sep 23 10:22 
>>>>>>>> test.c
>>>>>>>> -rwxr-xr-x  1 mimcmah      None             3680   Sep 23 10:22 
>>>>>>>> test.obj
>>>>>>>> -rwxr-xr-x  1 mimcmah      None             121344 Sep 23 10:22 
>>>>>>>> test.exe
>>>>>>>> -rw-r-----  1 Unknown+User Unknown+Group         0 Sep 23 10:23 
>>>>>>>> foo.sock
>>>>>>>> -rw-r--r--  1 mimcmah      None             144356 Sep 23 10:27 
>>>>>>>> check.ot
>>>>>>>>
>>>>>>>> A bigger problem is that foo.sock can't be deleted with the 
>>>>>>>> cygwin "rm"
>>>>>>>> command.
>>>>>>>>
>>>>>>>> $ rm -f foo.sock
>>>>>>>> rm: cannot remove 'foo.sock': Permission denied
>>>>>>>>
>>>>>>>> $ chmod 777 foo.sock
>>>>>>>> chmod: changing permissions of 'foo.sock': Permission denied
>>>>>>>>
>>>>>>>> $ cmd /c del foo.sock
>>>>>>>>
>>>>>>>> But, native Windows commands are okay, as the third example shows.
>>>>>>>>
>>>>>>>> I think the problem may relate to the way native Unix domain 
>>>>>>>> sockets are
>>>>>>>> implemented in Windows and the resulting special handling required.
>>>>>>>> They are implemented as NTFS reparse points and when opening them
>>>>>>>> with CreateFile, you need to specify the 
>>>>>>>> FILE_FLAG_OPEN_REPARSE_POINT
>>>>>>>> flag. Otherwise, you get an ERROR_CANT_ACCESS_FILE. There are other
>>>>>>>> complications unfortunately, which I'd be happy to discuss further.
>>>>>>>>
>>>>>>>> But, to reproduce it, you can compile the attached code snippet
>>>>>>>> which creates foo.sock in the current directory. Obviously, this
>>>>>>>> only works on recent versions of Windows 10 and 2019 server.
>>>>>>>
>>>>>>> Cygwin doesn't currently support native Windows AF_UNIX sockets, 
>>>>>>> as you've discovered.  See
>>>>>>>
>>>>>>> https://urldefense.com/v3/__https://cygwin.com/pipermail/cygwin/2020-June/245088.html__;!!GqivPVa7Brio!P7lIFI4rYAtWh8_DtCbRCxT-M_E4vwQ0qwzQ0p656T73BpJ0jbUkLI_bXdA6mmSL9lJcSQ$ 
>>>>>>>
>>>>>>> for the current state of AF_UNIX sockets on Cygwin, including the 
>>>>>>> possibility of using native Windows AF_UNIX sockets on systems 
>>>>>>> that support them.
>>>>>>>
>>>>>>> If all you want is for Cygwin to recognize such sockets and allow 
>>>>>>> you to apply rm, chmod, etc., I don't think it would be hard to 
>>>>>>> add that capability.  But I doubt if that's all you want.
>>>>>>>
>>>>>>> Further discussion of this will have to wait until Corinna is 
>>>>>>> available.
>>>>>>>
>>>>>>
>>>>>> Thanks for the info. It's mainly about recognition of sockets for
>>>>>> regular commands. Since these objects can exist on Windows 
>>>>>> filesystems
>>>>>> now, potentially created by any kind of Windows application,
>>>>>> it would be great if Cygwin could handle them, irrespective of 
>>>>>> whether
>>>>>> the Cygwin development environment does. Though that sounds like a
>>>>>> good idea too.
>>>>>
>>>>> I think this has a simple fix (attached), but I can't easily test 
>>>>> it because your test program doesn't compile for me.  First, I got
>>>>>
>>>>> $ gcc -o native_unix_socket native_unix_socket.c
>>>>> native_unix_socket.c:5:10: fatal error: WS2tcpip.h: No such file or 
>>>>> directory
>>>>>      5 | #include <WS2tcpip.h>
>>>>>        |          ^~~~~~~~~~~~
>>>>> compilation terminated.
>>>>>
>>>>> I fixed this by making the include file name lower case.  (My 
>>>>> system is case sensitive, so it matters.)
>>>>>
>>>>> Next:
>>>>>
>>>>> $ gcc -o native_unix_socket native_unix_socket.c
>>>>> native_unix_socket.c:8:10: fatal error: afunix.h: No such file or 
>>>>> directory
>>>>>      8 | #include <afunix.h>
>>>>>        |          ^~~~~~~~~~
>>>>> compilation terminated.
>>>>>
>>>>> There's no file afunix.h in the Cygwin distribution, but I located 
>>>>> it online and pasted in the contents.  The program now compiles but 
>>>>> fails to link:
>>>>>
>>>>> $ gcc -o native_unix_socket native_unix_socket.c
>>>>> /usr/lib/gcc/x86_64-pc-cygwin/10/../../../../x86_64-pc-cygwin/bin/ld: 
>>>>> /tmp/cc74urPr.o:native_unix_socket.c:(.text+0x3b): undefined 
>>>>> reference to `__imp_WSAStartup'
>>>>> /tmp/cc74urPr.o:native_unix_socket.c:(.text+0x3b): relocation 
>>>>> truncated to fit: R_X86_64_PC32 against undefined symbol 
>>>>> `__imp_WSAStartup'
>>>>> /usr/lib/gcc/x86_64-pc-cygwin/10/../../../../x86_64-pc-cygwin/bin/ld: 
>>>>> /tmp/cc74urPr.o:native_unix_socket.c:(.text+0xf2): undefined 
>>>>> reference to `__imp_WSAGetLastError'
>>>>> /tmp/cc74urPr.o:native_unix_socket.c:(.text+0xf2): relocation 
>>>>> truncated to fit: R_X86_64_PC32 against undefined symbol 
>>>>> `__imp_WSAGetLastError'
>>>>> /usr/lib/gcc/x86_64-pc-cygwin/10/../../../../x86_64-pc-cygwin/bin/ld: 
>>>>> /tmp/cc74urPr.o:native_unix_socket.c:(.text+0x13d): undefined 
>>>>> reference to `__imp_WSAGetLastError'
>>>>> /tmp/cc74urPr.o:native_unix_socket.c:(.text+0x13d): relocation 
>>>>> truncated to fit: R_X86_64_PC32 against undefined symbol 
>>>>> `__imp_WSAGetLastError'
>>>>> collect2: error: ld returned 1 exit status
>>>>>
>>>>> This is probably easy to fix too, but I don't feel like tracking it 
>>>>> down. Please send compilation instructions (that use Cygwin tools).
>>>>>
>>>>> Ken
>>>>
>>>> Hi
>>>>
>>>> Sorry, I had compiled it in a native Visual C environment.
>>>>
>>>> Assuming you have afunix.h in the current directory.
>>>>
>>>> gcc -o native_unix_socket -I. native_unix_socket.c -lws2_32
>>>>
>>>> should do it.
>>>
>>> Thanks, that works.  But now I can't reproduce your problem.  Here's 
>>> what I see, using Cygwin 3.1.7 without applying my patch:
>>>
>>> $ ./native_unix_socket.exe
>>> getsockname works
>>> fam = 1, len = 11
>>> offsetof clen = 9
>>> strlen = 8
>>> name = foo.sock
>>>
>>> $ ls -l foo.sock
>>> -rwxr-xr-x 1 kbrown None 0 2020-09-25 14:39 foo.sock*
>>>
>>> $ chmod 644 foo.sock
>>>
>>> $ ls -l foo.sock
>>> -rw-r--r-- 1 kbrown None 0 2020-09-25 14:39 foo.sock
>>>
>>> $ rm foo.sock
>>>
>>> $ ls -l foo.sock
>>> ls: cannot access 'foo.sock': No such file or directory
>>>
>>> I'm running 64-bit Cygwin on Windows 10 1909.
>>
>> I just ran the 'rm' command under gdb to see what's going on, and it 
>> seems that foo.sock is not being recognized as a reparse point.  So 
>> maybe your test program, when compiled and run under Cygwin, doesn't 
>> actually produce a native Windows AF_UNIX socket.  And when I try to 
>> run it in a Windows Command Prompt, I get
>>
>> bind failed 10050
>> getsockname failed 10022
>>
>> Can you make your version of the test executable available for me to 
>> try?  Or tell me some other way to create a native Windows AF_UNIX 
>> socket?
>>
>> Ken
> 
> That is all very strange. I have checked both the gcc compiled and MS
> compiled executables on my system (2019 server) and they are both
> definitely producing native AF_UNIX sockets.
> 
> I can email you the two exe files. They are both quite small. But, first
> I want to check the patch status of my test system.
> 

So, it turns out that this issue only happens on some of our test
systems. It does not happen on a personal copy of Windows 10 on my
laptop either.

I also noticed that some native Windows commands don't work properly on
any affected system (eg 'attrib' or 'fsutil'). Though 'fsutil' can be
used to verify that the reparse point is created correctly.

Possibly, this was a Windows bug that has been fixed. It never made
sense that you had to open socket files using the 
FILE_FLAG_OPEN_REPARSE_POINT flag, because you would have to know in
advance that the file is a socket to be able to do this (or else be
prepared to have to open the file twice). But, I don't fully
understand yet, why some systems are affected and others not.
All seem to be patched up to date.

In any case, I think it's clear this isn't a Cygwin issue.
So, apologies for the noise, and thanks for the assistance!

Regards,
Michael.
--
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple

- Raw text -