X-Recipient:	archive-cygwin AT delorie DOT com
DomainKey-Signature:	a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:from:to:subject:date:message-id:content-type
	:content-transfer-encoding:mime-version; q=dns; s=default; b=SVy
	b+OXalgYtj7xG6VibZ3JJo2mOy5e7KgfKZ0HxXbYQ/Fv2ttINNtgmuPV/Hbf9CIy
	mV7P7yLpo6cUHloy3xlsFYNbpo4vREQXRoUy3NIfXXGqYnMW/R8DoUACrYkQDaGl
	S17/Rk8RAvx9QSxd71qlSxaJfDUukEGTtVpItcdg=
DKIM-Signature:	v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:from:to:subject:date:message-id:content-type
	:content-transfer-encoding:mime-version; s=default; bh=soenEocyz
	2T/YabBAaROB5lSshI=; b=FViMIknoRGDDoNtQ+yPPMWqajgHYyhWwVKUL0QyaM
	ru9jORtyjNVzBLrcLG7NjhXznyVhjXzCp28VonzgjfXOcpS3YJt5obPXPPQOgcIo
	Fdz/RkPK6iEDns8khIj8pH/rPjGhwIvJCUHno0k/IsY6+XHRHmrCw4BEXoljnibP
	ZA=
Mailing-List:	contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id:	<cygwin.cygwin.com>
List-Subscribe:	<mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive:	<http://sourceware.org/ml/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender:	cygwin-owner AT cygwin DOT com
Mail-Followup-To:	cygwin AT cygwin DOT com
Delivered-To:	mailing list cygwin AT cygwin DOT com
Authentication-Results:	sourceware.org; auth=none
X-Virus-Found:	No
X-Spam-SWARE-Status:	No, score=1.9 required=5.0 tests=BAYES_50,FORGED_HOTMAIL_RCVD2,FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SPF_HELO_PASS,SPF_PASS autolearn=no version=3.3.2 spammy=approximately, inherited, Jay, jay
X-HELO:	NAM01-BN3-obe.outbound.protection.outlook.com
From:	Jay K <jayk123 AT hotmail DOT com>
To:	"cygwin AT cygwin DOT com" <cygwin AT cygwin DOT com>
Subject:	fast/native fork?
Date:	Sun, 21 Jan 2018 08:32:17 +0000
Message-ID:	<MWHPR18MB1214E3A077433426273EFE00E6ED0@MWHPR18MB1214.namprd18.prod.outlook.com>
x-incomingtopheadermarker:	OriginalChecksum:353F701BCE4FD88F3AD79A9CC51EE6AA37234793108806544D6BBBD7FDBC9AB6;UpperCasedChecksum:E2600A04E77608512CB623CD565B71A607D74A7446AB01464329ABE6956A2AEB;SizeAsReceived:6851;Count:43
x-tmn:	[jslgc+BCb/xUt/uoKfOPj6goH5qRMW3eVpIx7CAcxeMqz4Hn6oRxG2p9rmfFflNK]
x-ms-publictraffictype:	Email
x-microsoft-exchange-diagnostics:	1;BN3NAM01HT063;6:+1nzUQMMDLLhMGZq8+seHkBsYXoPNANjJVN3Jp+2CtcuS4AlTQVszqLC2aHkT1Efg1L1H1auv+kLj5t9MLou9nISiKsYiS3cFMmKym7DwuFB2VSLyjj7zIF8BKgOJznhCyG0l669xslT/C70oMiH0KiYmdNND55FFCkfK+tdJTbZQY0scYjSK0JCg/ZVFJDdeHPPg7wEeokoq0naQDM7ZcGQEui6vx0cbieGH4JCS+czRuFmJ2ccWMiRL7mUq9iBRnidFZ666AhX0xodTpYe4lIrzbFTdIQSZg4OYJuHtbnaBn0ad18+fTCQNfdvpr6ERKcaFzwk1J7YfpomR5Udb+pN1AfmYao4lHZgG/GJdDI=;5:isGcza0KXUClZHwKGwuKeu8FJgKgPGHhrsYAM+ITb3u5mM7X95EKDhXzUgwmq0cuDkBVY+NhOqmgnt07YdlHCw0Q1EXdao4d4FgOCMRnhpG2wdeWE6iRD2VegjxEyu6OInX6DOGRA2MBBZ+9M6+ZCqj6gTMtsxLHrRI5Ok9BKyw=;24:dqFbGhM4vd3iuvFpEc1GFtlhHT8gtMn4p8C70YWBLqxbMOIOlHM0bogZU9Gg/Ojxoh2qIFzhpACqObD5Y86GGXSeIybxCjMfWL7XkMREHdY=;7:dJYtjsv57ZUcr+pIXSj5KsXCYs3oSgJQxsWjl8Po/L4Eip6aTPxcNPnpdtSByNm3s37it7ZSb29x+HlbjNdmbeOIQTx0l1PRGCKDJtjR6MptminVGRdxulmKMYMR1ir16okfINPvfeBQvKlSidsLfOiv7MG0g1lf6CIRmVVKPk/EKmk3lXxRLzvpxBjvCbsUuBF25v1ShD7yt7r6lvSBfEOiwbxg0tG0R51nuVO5T36YcFzbYXpEOLjR66wG3MCf
x-incomingheadercount:	43
x-eopattributedmessage:	0
x-microsoft-antispam:	UriScan:;BCL:0;PCL:0;RULEID:(7020095)(201702061074)(5061506573)(5061507331)(1603103135)(2017031320274)(2017031324274)(2017031323274)(2017031322404)(1603101448)(1601125374)(1701031045);SRVR:BN3NAM01HT063;
x-ms-traffictypediagnostic:	BN3NAM01HT063:
x-ms-office365-filtering-correlation-id:	fd71aac5-58e6-452d-0552-08d560a97b4c
x-exchange-antispam-report-cfa-test:	BCL:0;PCL:0;RULEID:(444000031);SRVR:BN3NAM01HT063;BCL:0;PCL:0;RULEID:(100000803101)(100110400095);SRVR:BN3NAM01HT063;
x-forefront-prvs:	0559FB9674
x-forefront-antispam-report:	SFV:NSPM;SFS:(7070007)(98901004);DIR:OUT;SFP:1901;SCL:1;SRVR:BN3NAM01HT063;H:MWHPR18MB1214.namprd18.prod.outlook.com;FPR:;SPF:None;LANG:;
spamdiagnosticoutput:	1:99
spamdiagnosticmetadata:	NSPM
MIME-Version:	1.0
X-OriginatorOrg:	hotmail.com
X-MS-Exchange-CrossTenant-Network-Message-Id:	fd71aac5-58e6-452d-0552-08d560a97b4c
X-MS-Exchange-CrossTenant-originalarrivaltime:	21 Jan 2018 08:32:17.6331 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader:	Internet
X-MS-Exchange-CrossTenant-id:	84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa
X-MS-Exchange-Transport-CrossTenantHeadersStamped:	BN3NAM01HT063
X-MIME-Autoconverted:	from quoted-printable to 8bit by delorie.com id w0L8WVhF004760

I have some desire to discuss fork.
I know it is an old and difficult topic.


I found this:

 "Cygwin fork and RtlCloneUserProcess"
 https://social.msdn.microsoft.com/Forums/windowsdesktop/en-US/afdf1b68-1f3e-47f5-94cf-51e397afe073/cygwin-fork-and-rtlcloneuserprocess?forum=windowsgeneraldevelopmentissues



NT has had fork since v1.
The Posix subsystem used it.

You didn't need Vista's introduction of RtlCloneUserProcess.

This from 2005 alludes to how to make it work:
  https://www.winvistatips.com/threads/question-about-ntcreateprocess.186504/

but I have difficult questions for you -- anyone, but including Corinna.

What do you expect it to do?


I mean, consider that there is no pthread_atfork or an analog in Win32.
 
Dlls at all levels of the Win32 stack, might have
process-specific state, that needs to be reinitialized.
Maybe they were holding locks in a worker thread.
Maybe they had the pid cached. Dumb, but it works w/o fork.
The usual problems that pthread_atfork is meant to solve.
 
Which memory do you expect to be inherited copy on write, and which
memory do you expect to revert to whatever it is when a process starts (or a
dll is loaded)?

You could hope for something like calling DllMain(process attach) of all the
dlls could help, but it can't, at least because dlls just depend on the
default data in the image, they don't write the defaults to memory
at every load.


ntdll.dll is special. It somehow knows fork occured and can reinitialize itself.
ntdll.dll is special -- it used by the Posix subsystems (SFU/SUA/Interix/etc.) and always had to work with fork.
  (Up until Windows 10 and changes for WSL, ntdll.dll was loaded into
  all usermode processes: Win32, Posix, OS/2. This was specifically
  changed for WSL.)


But no other dll expects this.

  
Now, I have some wierd ideas. Let's brainstorm a little?


Can you somehow leave the child process in limbo waiting for exec, know
when you have waited too long (because *anything* else happened) and only
then do the expensive copying?

Or there is the very problem of getting to the exec call and having the exec parameters?

I mean, what if you actually knew 100% that exec would come very soon after fork?
 What would/could you do then?

 
And when exec does not follow, what do you do?
How much of the child process is inherited from the parent, vs. how much
is reinitialized such as for a new process?

If there a solution that optimizes the guaranteed to exec case,
can you almost just assume it? Breaking the rare (?) program that does not exec?
You could even omit fork.
You'd have fork_slow and fork_before_exec.
People would have to ifdef Cygwin and chose what they want.
Or the default could be fork_before_exec, breaking a small number
of programs, that could be easily ported.


Ok, how about this?
Can you implement exec using only ntdll.dll, avoiding kernel32.dll?
 And the small/zero number of other things valid/used between fork and exec?
And assume exec follows fork?
If so then that is a solution:
 learn how to use native fork
 and have exec only use ntdll.dll
 
 That will give you a fast fork + exec sequence.
 
 Or, can you in the new process, just reinitialize kernel32.dll and kernelbase.dll,
 and only use them for exec?
 
 
It doesn't do anything for fork without exec but I still don't understand
how that is supposed to work in Win32.
 
 
 Or how about this:
 Again, if you assume exec is coming, and you just need fork 
 to do the minimum -- basically to get a pid.
 
 fork calls CreateProcess, with a helper .exe, suspended
 fork calls CreateThread, passing it the register context of the creating thread
 the helper thread suspends the creating thread, and takes over its
 register context (including rip and rsp, approximately), shortly thereafter
 the helper thread in the creating process reaches exec.
 
 At this point, somehow, it adjusts everything..hand waving.
 
 How do you implement exec today? Does fork actually get the pid
 of the new child, and exec in the child somehow "replaces" the executable, or
 does exec create a second child, with another pid, and the original
 child just waits for it, and returns its exit code as its own?
 
 Or do you have indirection on pids, and cygwin pids are not win32 pids?
 
Oh, that's right, setjmp/longjmp.
fork calls setjmp in the parent.
The first return continues until exec in the parent and then
returns the new pid the second time?
  Given an arrival at exec, in the parent instead of the child,
  the usual child part of fork need never run at all.


To repeat: To what extent, if any, can we assume exec follows fork?
And what can be done with this idea?
I understand the more general model, where exec does not follow fork.
But how common is it? How would Cygwin fair if by default fork+exec
was fast, fork w/o exec didn't work, and people ported those somehow?


Fyi:  
 https://mikeash.com/pyblog/friday-qa-2012-01-20-fork-safety.html
 
 
 - Jay

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

- Raw text -