X-Recipient:	archive-cygwin AT delorie DOT com
X-Original-To:	cygwin AT cygwin DOT com
Delivered-To:	cygwin AT cygwin DOT com
DMARC-Filter:	OpenDMARC Filter v1.3.2 sourceware.org 5DA51386F455
Authentication-Results:	sourceware.org;
	dmarc=none (p=none dis=none) header.from=maxrnd.com
Authentication-Results:	sourceware.org; spf=none smtp.mailfrom=mark AT maxrnd DOT com
Subject:	Re: segfault on 32bit cygwin snapshot
To:	cygwin AT cygwin DOT com
References:	<9d7b9dc2-cb92-498b-7655-e9c618114c87 AT gmail DOT com>
	<20210221072954 DOT db2dcbd523ed366e4dfcb0d0 AT nifty DOT ne DOT jp>
	<7480c946-8e02-aba2-c06f-6b39f630699f AT gmail DOT com>
	<20210301095546 DOT dce31a474bd0cec2c3518f87 AT nifty DOT ne DOT jp>
	<20210301212542 DOT 8b1749f92af62c01b008f25a AT nifty DOT ne DOT jp>
	<20210302200308 DOT 62db4fe01f78fb35a538784f AT nifty DOT ne DOT jp>
	<YD5eXbxBWbUUSwcM AT calimero DOT vinschen DOT de>
	<20210303185621 DOT b048287526901af6a4c8200a AT nifty DOT ne DOT jp>
	<YD9sSea32T7GqlJr AT calimero DOT vinschen DOT de>
From:	Mark Geisert <mark AT maxrnd DOT com>
Message-ID:	<e15e6d1f-7680-8f80-871e-0d224a3ed682@maxrnd.com>
Date:	Thu, 4 Mar 2021 01:05:14 -0800
User-Agent:	Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101
	Firefox/52.0 SeaMonkey/2.49.4
MIME-Version:	1.0
In-Reply-To:	<YD9sSea32T7GqlJr@calimero.vinschen.de>
X-Spam-Status:	No, score=-4.4 required=5.0 tests=BAYES_00, KAM_DMARC_STATUS,
	KAM_LAZY_DOMAIN_SECURITY, NICE_REPLY_A, SPF_HELO_NONE, SPF_NONE,
	TXREP autolearn=no autolearn_force=no version=3.4.2
X-Spam-Checker-Version:	SpamAssassin 3.4.2 (2018-09-13) on
	server2.sourceware.org
X-BeenThere:	cygwin AT cygwin DOT com
X-Mailman-Version:	2.1.29
List-Id:	General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Unsubscribe:	<https://cygwin.com/mailman/options/cygwin>,
	<mailto:cygwin-request AT cygwin DOT com?subject=unsubscribe>
List-Archive:	<https://cygwin.com/pipermail/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe:	<https://cygwin.com/mailman/listinfo/cygwin>,
	<mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
Errors-To:	cygwin-bounces AT cygwin DOT com
Sender:	"Cygwin" <cygwin-bounces AT cygwin DOT com>

Hi Corinna,

Corinna Vinschen via Cygwin wrote:
> [Ping Mark Geisert]
> 
> On Mar  3 18:56, Takashi Yano via Cygwin wrote:
>> Hi Corinna,
>>
>> On Tue, 2 Mar 2021 16:48:45 +0100
>> Corinna Vinschen wrote:
>>> On Mar  2 20:03, Takashi Yano via Cygwin wrote:
>>>>> The following check code does not work as expected if
>>>>> newly build exe file is linked with old dll which calls
>>>>> uname() as in this case.
>>>>>
>>>>>    if (CYGWIN_VERSION_CHECK_FOR_UNAME_X)
>>>>>      return uname_x (in_name);
>>>>>
>>>>> Any idea?
>>>>
>>>> Ping Corinna?
>>>
>>> I have no idea how we could fix that, other than by rebuilding the DLLs
>>> which call uname, too.  We can't check the Cygwin build of all DLLs an
>>> executable is linked to.
>>
>> I have checked all /usr/bin/*.dll in my cygwin installation and found
>> following dlls call uname() rather than uname_x().
>> [...]
>> Do you think rebuilding all of these (or maybe more) dlls is only
>> the solution?
> 
> No, we could also drop the above code snippet.
> 
> Here's the problem: When we changed some datatypes in the early 2000s,
> the old entry points have been conserved for backward compatibility,
> while the new function using the new datatypes got a new name, e. g.,
> stat vs. _stat64.
> 
> Next, libcygwin.a gets changed so that a newly built executable (using
> the new datatypes as defined in the standard headers) calling stat is
> redirected to _stat64.
> 
> All is well for the next 15+ years or so.
> 
> Then we discover that the exact same mechanism fails to work for
> uname vs. the new uname_x in python.  What happened?
> 
> It turned out that python called uname dynamically Rather then just
> calling uname, it calls dlopen();dlsym("uname");
> 
> This actually fetches the symbol for uname, not the symbol for uname_x.
> The good old mechanism used for ages on standard function, fails as soon
> as the caller uses dynamic loading of symbols.  Surprise, surprise!
> It was just never taken into consideration that standard libc functions
> might be called dynamically, given it usually doesn't make sense.
> 
> Given that this affects *all* of these redirected functions, we're in a
> bit of a fix.  Fortunately, for all other functions this only affects 32
> bit Cygwin, because the 64 bit version never had this backward
> compatibility problem.
> 
> Therefore, uname vs. uname_x is the only function affecting 64 bit
> Cygwin as well, and that's why I added the above crude redirection only
> to uname in the first place.
> 
> So what we can do is this:
> 
> - Either all old DLLs calling uname must be rebuilt.

.. or patched (somehow).  Also, IIUC you're both talking about Cygwin-supplied 
DLLs, not user builds of Cygwin or private DLLs.  Dunno how we'd find every last one.

> - Or we remove the above code snippet again and behave like for all
>    other redirected functions on 32 bit as well.  Python's os.uname is
>    broken, but all the affected DLL sstart working again.

I think you need to keep the above code snippet to handle old exe running with 
current Cygwin DLL, no?

> Is there a way around that?  I'm not quite sure, so let's brain storm
> a bit, ok?
> 
> - One thing we could try is to remove the above code, but add a python
>    hack to dlsym instead.  This would let the "old" DLLs work again as
>    before and for python we could add a hack to dlsym, along these lines:
> 
>      if (CYGWIN_VERSION_CHECK_FOR_UNAME_X
>      	&& modulehandle == cygwin1.dll
> 	&& strcmp (symname, "uname"))
>        symname = "uname_x";
> 
> Thoughts?  Other ideas?

That's a sly fix, but it seems that it would do the job.  That's good!

On a different tack, I was thinking about how run time code could tell the 
difference between the versions of uname() being called.  It can't.  I looked at 
glibc and FreeBSD libc to see if they had to deal with this compatibility issue. 
It seems they don't, or I couldn't locate it if they do.

But FreeBSD has an approach that looked interesting in another way.  They have the 
standard uname() function taking the one usual arg.  But it's just a wrapper on a 
worker function that takes the original arg plus another arg specifying the size 
of the fields in struct utsname.  Paraphrasing, using Cygwin names:
         int uname(struct utsname *uptr)
         {
             return uname_x((int) _UTSNAME_LENGTH, uptr);
         }
They allow the user to override what we call _UTSNAME_LENGTH.  That seems like an 
invitation to exploit so I don't care for that.  But what I was thinking is if we 
make that first arg to uname_x() be a uintptr_t, we could tell (with pretty good 
confidence) at run time inside uname_x() if we were passed a length (from new code 
passing two args) or a ptr-to-buf (from old code just passing one arg).

I am unsure if this is useful.  I am trying to consider the cross-product of exes 
and dlls we care about and whether we can identify which combination we have at 
run time.  I think I need some time with a pad of paper, and maybe a drink.

..mark
--
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple

- Raw text -