X-Recipient:	archive-cygwin AT delorie DOT com
DKIM-Filter:	OpenDKIM Filter v2.11.0 sourceware.org B18E63858032
DKIM-Signature:	v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com;
	s=default; t=1709751346;
	bh=OH8GRxlIyvOLKzL3QJdi9pbqlUfKUtU6n+kUM8lk0nc=;
	h=Date:Subject:To:References:In-Reply-To:List-Id:List-Unsubscribe:
	List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:
	From;
	b=KW/9rWtyU0IQVwa4/A/YGfLYi0erFb9g0T2d2fI8CLanb5sEDCHj0ni9GBRBFQij7
	Poa9U1Kh5Ro2mzboa7Id23RjLSz+9TSFaiqFeU6hNI4r4WKZsZeAyaalTwAy2aeTm9
	es1nVNb8AfpmPk5xpqKofySRiCSlPZ/ly3KOxa/I=
X-Original-To:	cygwin AT cygwin DOT com
Delivered-To:	cygwin AT cygwin DOT com
DMARC-Filter:	OpenDMARC Filter v1.4.2 sourceware.org B3DB23858D34
ARC-Filter:	OpenARC Filter v1.0.0 sourceware.org B3DB23858D34
ARC-Seal:	i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1709751326; cv=none;
	b=bY/Gw9e+ZTRUe1m5SBdAgGAim39+VVHrnCDWQoIXx8fhPr5oKMnLnJ1jKIufr5bILF0rmbJNwcfl/LfCvy2uaZUr5jCT8svbQG6+w1fEjxlDbGJ3mdHmDuNpAYREGDzp3RjzG1fhOJKnRQVCQisXV4dDjdjNn2wIqJKhETXkKSE=
ARC-Message-Signature:	i=1; a=rsa-sha256; d=sourceware.org; s=key;
	t=1709751326; c=relaxed/simple;
	bh=ANP4ZONQuBhWn2mjh2Fxy6OqWOIcrWVZhWjT07xQFQ0=;
	h=DKIM-Signature:Message-ID:Date:MIME-Version:Subject:To:From;
	b=p0J0mTPBOIwuklW7yJNOOnjMNX15sNzGokDppLxtCIAOoiXsQPiygWYRtanf7seNvtXd94aQNGa5vHnlnvw56OS1wwy/IeBgIyquNWTAe/PMXmKWzGrWx8IeT2SXKkWVjNz7Za1dtDEs6FPik14+84n+ve4mJM7JopA/Xs0eN14=
ARC-Authentication-Results:	i=1; server2.sourceware.org
X-MDAV-Result:	clean
X-MDAV-Processed:	sequoia-grove.ad.secure-endpoints.com, Wed,
	06 Mar 2024 13:55:22 -0500
X-Spam-Processed:	sequoia-grove.ad.secure-endpoints.com, Wed,
	06 Mar 2024 13:55:22 -0500
	(not processed: message from trusted or authenticated source)
X-MDRemoteIP:	2603:7000:73c:bb00:2477:ddf9:9ccb:a8de
X-MDHelo:	[IPV6:2603:7000:73c:bb00:2477:ddf9:9ccb:a8de]
X-MDArrival-Date:	Wed, 06 Mar 2024 13:55:22 -0500
X-MDOrigin-Country:	US, NA
X-Authenticated-Sender:	acct-jaltman AT secure-endpoints DOT com
X-Return-Path:	prvs=17951a7cf7=jaltman AT secure-endpoints DOT com
X-Envelope-From:	jaltman AT secure-endpoints DOT com
X-MDaemon-Deliver-To:	cygwin AT cygwin DOT com
Message-ID:	<7d9fe460-5704-424b-a89b-e34ef2176d38@secure-endpoints.com>
Date:	Wed, 6 Mar 2024 13:55:17 -0500
MIME-Version:	1.0
User-Agent:	Mozilla Thunderbird
Subject:	Re: ls/stat on OneDrive causes download of files
To:	cygwin AT cygwin DOT com
References:	<CAC9GOO-RbcR5YOPyk6qVDuVSdDK4M=f1V8bHvz7yvoaHjGUZnQ AT mail DOT gmail DOT com>
	<ZehuDAKd3ceXlfba AT calimero DOT vinschen DOT de>
	<ZehvbhVizZTBSaRR AT calimero DOT vinschen DOT de>
	<208962d1-78cd-4e2d-9e4a-a1edcbf5e809 AT SystematicSW DOT ab DOT ca>
	<ZeilkJK7Csryuzkc AT calimero DOT vinschen DOT de>
Organization:	Secure Endpoints, Inc.
Autocrypt:	addr=jaltman AT secure-endpoints DOT com; keydata=
	xsFNBEwLlO0BEACu6yWFkd1+qwsGg8ZzgslSkcAKhSegWt5j86DpaRL0W8fxg6YjxwEPvwoH
	BGa/rpSdBd1gkmzeYxD3hVZdj75r6nVS9f/mxNQzW+o1sW4vaeSxKgZSQz5RqHmwPDcqQP66
	+ZSnjV+G88MKwZ9DIzA9AwpJhNAAlAlj3OvsQVsxd1ipc6C4/U3qjHL7Ih22UbPBM71ltIZx
	kqcrAlXPnUTeraJXtfzYbq4mJFJ9JC6/o1NRSjsBvRD+ADxlG50+KccZN4SS5xxdGuh1tA9U
	TydYBQB3YtJbq7CYau2kIYt/3HnyLYGo1s6Ti6cuAJJ/40iIE1xkqhvMiIz/Q+1ztmksJbLQ
	aCtW8kF42nF8MpPdIPTSPr2uGvpRtCjRbh4lgMXgyNUx1wpCEY0X11xce++H8HySmFwryE2y
	kkxUQeMUjaaXZDHYUSyQz7riChFiZ9ax9dmX0wUY/A05v0qcualglpk4wJ2kcsGKUEGkLvnV
	wwvya8zifPwKOw5JlGPvzX8t2m7jB2GXKzvVAsImqOqnDBTKUXWQQZCW9Rqt7acdE8bQ2vqr
	vP+3Ykf4SrPwcuNCDt6QSgjVbhc3hA3hCtE1iW/HhuBAzKiuzJ9era+q9QjTtLPIkQDHRpcC
	MMWvK0Y1uQ34Ql1BfKRA4gc8A7CuVUY6+Ga7PuJWd+FSglvmKQARAQABzS1KZWZmcmV5IEFs
	dG1hbiA8amFsdG1hbkBzZWN1cmUtZW5kcG9pbnRzLmNvbT7CwXYEEwECACAFAkwLltMCGyMG
	CwkIBwMCBBUCCAMEFgIDAQIeAQIXgAAKCRD3enNVkraaBIO5D/sHkpJx2Vz7mgUDthBuro7f
	y/FK9MxC6dUTF0H54OjRXPBRQmzDDzpTNmG+8mtPO+FR1QlkupQJpGgFjpUdVnjlq2nGF8sH
	ecCCIKylI9VM3+bqZOEiRWHYxmmfMNDLJTBY5ES8YFsiOvWiCmKJe5YicwysBEcjD75O5JsX
	2ylRJ/NRF3VLI2A/jE1HI7DDgZ8hDkYU2yhV37wUHZPQUS5XYiSaUtn71MI2kIS8rHRPrxHg
	qmRB4uKJDVVW1jdo0/43YAJQr8CitlstluZMmXYOjGZdiG9MDdtWJz7KxEQsQD9XLPvb/aD5
	sGCoBbu26ODnSkz0k2UM5uWknul4zbgt6s2V/HD+uwxmbhJEpzMeLMbWNVDXQH+EcmOnfVYV
	ZalkMA19zkGt0jEbNq3CTx2zSsg1Gq+j6UzRm+kANo+VN3y2izRVtVreQrw1MX6DhvwQsqdI
	gVVINvTaa+2rZhHaDXRsnb3jYaqsvLo7gOUGdwGp3jin4YNHAR8JZ9e+ueUMDEFiYdCHMPJf
	y2HgVh4vNTtO9P6BDIG/DdpOPSlOgKKulbE0BGKMWCkqwAr42SDF0lk5uCvOkEpfDHPLgDbK
	rBehbmhCidGiZsR5ijdhh65CgKFUrBY50/aBeSSjMUCo2kC10oV+N5SdjbOH/M5TC8Yt1oUf
	ATH/Dt7/FZ/GHM7BTQRMC5TtARAA88hJdpgcg2RU/uAWfAL46XZHA59cVpPNNly1tPWCSbG6
	+ONH6nOG/NarmNVxX6Mb9YRkEU6wmrZS85inz3otdyz/zlyNSWma8qGNUlMbiwFQqfXWVBAP
	GoRC0a0aJrd4IayLuvv1UqEwx7Otp7y5RNHtRv35/kho0Z+UheYVdGm2I06xIc+aNKW2LO7R
	5BNtjpADPIG+NSdsVIeamhAWPvLrwbf6mUb//eA9pF0w0QixLVrH/cCoz+S27gCGJvY6zF22
	NgdhnkIqNz8E/LKt6S36ZI9Mw/ixpQTozqRmdNzVQNgTHUZClbJj4iq1EPHB7XqpxOv+awrx
	Sxq2jt8GFD0rU+sAuzW+F7cBoIw434/IrxKYwcPHpHLEVQ1tLP7d3ZpZR30p3oqoliGiLsWv
	HHxyXjuMBF4XJ6MRXmD65/qOhuo2DKduHMNlmxzgSzvWgXZeNJq+OcS8jQZDt2Na2pMKjWyt
	au7xQu2ndm0FwS48ngMrDYRQMxzL1NfnBnT9BCwjiU+/6NBSwcNKIqyea9IpTwsVfkF4/iui
	7xD9+LtzqeUkBAe7q5jEJmJMZhAfh7usZGT8TGxXegCaF4Jwz2nxS4Fv7VRza/yUAOJlc0da
	R11TPeiUNCQWY7PpL1AXO9vaSyjFuOzTnU8vzXvI9fGoxIxKGRQpKMU8PROIFw0AEQEAAcLB
	XwQYAQIACQUCTAuU7QIbDAAKCRD3enNVkraaBIxXD/4xlaBwW2TLFfMvlcY/2XDSm6NO4JaJ
	G2Nzp35xaaBVwMVzWvI+GgTgKNSFot9f4jiLBNQdnq3UKoEThR2ORKVL0ZJS1QYR7yyrOo0M
	teDSy8ofU1FJ6xu4ND3ekOjP20BTrihDpqUdahir2uaRfMkwM+0imOlcutGMhJNF/LAjrhoD
	p9SeDMYBXZ1wfrbrEo/EEu0PbkGyzqPyEPqwN1iSJkcAnjuIA0rTf1jQtJAaDov7yHsSRwUM
	+qTGsjOGQAN3wtYwjPpw7hI01sE+x0uq0pVeo4qeWTZ2TE4Vtp8FKXFAkqnP878q+kNk9Ve+
	DRs8UlRfa9Lgf5ETjXOTVGaT/UGxi9B4oo8k0lzvM/A1txexL/lLw8AULhUeGtyS6D2X9vFi
	6azna+o918R9BV86uXPiDOf1nMwqKchNCxmgH9vd0aQm8TKCrWAW4kU1Ig6aMNuZiWloVZfK
	rmWizbgeGKE9rhNPNqxkqBaA4lrJ8L6bdKbhAOe3NQjO2vUAXB53JphlF74GwEsh+85i9/yI
	bvwJVcsFYhdZz7fCAUOcnFkGnyrwIgkizQ3xXShPW8mqkgUk4kYMnucC4kG/E7pI/4lke5X5
	X9vroXRHB7tkpAgT46SqSM/XTwCaseXG9orDgz3duRTUp6K0++S/qsqTakGVmjD5917A1HqW
	fMmiKA==
In-Reply-To:	<ZeilkJK7Csryuzkc@calimero.vinschen.de>
X-MDCFSigsAdded:	secure-endpoints.com
X-Spam-Status:	No, score=-2.1 required=5.0 tests=BAYES_00, DKIM_SIGNED,
	DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_DNSWL_NONE, SPF_HELO_PASS,
	SPF_PASS, TXREP,
	T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version:	SpamAssassin 3.4.6 (2021-04-09) on
	server2.sourceware.org
X-BeenThere:	cygwin AT cygwin DOT com
X-Mailman-Version:	2.1.30
List-Id:	General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Unsubscribe:	<https://cygwin.com/mailman/options/cygwin>,
	<mailto:cygwin-request AT cygwin DOT com?subject=unsubscribe>
List-Archive:	<https://cygwin.com/pipermail/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe:	<https://cygwin.com/mailman/listinfo/cygwin>,
	<mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From:	Jeffrey Altman via Cygwin <cygwin AT cygwin DOT com>
Reply-To:	Jeffrey Altman <jaltman AT secure-endpoints DOT com>
Errors-To:	cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com
Sender:	"Cygwin" <cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com>

On 3/6/2024 12:19 PM, Corinna Vinschen via Cygwin wrote:
> We can add an explicit call to
>
>    RtlSetProcessPlaceholderCompatibilityMode (PHCM_EXPOSE_PLACEHOLDERS);
>
> and we can recognize the IO_REPARSE_TAG_FILE_PLACEHOLDER and
> IO_REPARSE_TAG_CLOUD_* tags during symlink evaluation, but even then
> we still have to know what the reparse point buffer actually contains.
>
> Given that the content of reparse points with these reparse tags are
> undocumented, some people using cloud services should examine these
> reparse points so we can add some suitable code to Cygwin.
>
>
> Corinna
I'm not an expert in this area by any means but here are my 
recollections from when Microsoft presented in-person on cloud 
placeholders to filter and filesystem developers many years ago.

Files and directories that are placeholders should have either the 
FILE_ATTRIBUTE_RECALL_ON_DATA_ACCESS or FILE_ATTRIBUTE_RECALL_ON_OPEN 
file attributes set. When these attributes are set, applications and 
mini filters are advised not to "read" or "open" the files or 
directories unless they absolutely need to because doing so will cause 
the placeholder to be replaced by an object containing the actual data 
which might take a long time to fetch, might cost the end user money, or 
might fail depending upon the network connectivity. In particular, 
anti-malware should ignore them during scans and only analyze the data 
when it is fetched locally by an end user application.

I believe that IO_REPARSE_TAG_FILE_PLACEHOLDER was replaced by 
IO_REPARSE_TAG_CLOUD_1 ..IO_REPARSE_TAG_CLOUD_F. Any reparse tag 
attached to a placeholder object is for the interpretation of the filter 
associated with the back-end storage and not for the consumption of 
applications. The content of the reparse tags can be back-end 
proprietary; different reparse data for onedrive, icloud, dropbox, etc.

The default ProcessPlaceholderCompaibilityMode is 
PHCM_EXPOSE_PLACEHOLDERS which makes the FILE_ATTRIBUTE flags and 
reparse tags visible. Microsoft maintains a database of processes for 
which PHCM_DISGUISE_PLACEHOLDER is set which hides that information. Its 
unclear to me that explicitly setting the placeholder compatibility mode 
is useful.

I'm not sure that exposing the object as a symlink is a good idea. A 
posix symlink is an object whose type and target information cannot 
change. In the case of a placeholder, the placeholder is silently 
replaced by the actual object either when the object is opened or the 
object's data is accessed. An application that believes it knows that 
the object is a symlink will be mighty confused when it turns out to be 
a file or a directory.

Perhaps the question that needs to be asked is whether there are opens 
that can be skipped if an object is known to not be locally present 
(either of the FILE_ATTRIBUTE flags are set)?

Jeffrey Altman



-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple

- Raw text -