X-Recipient:	archive-cygwin AT delorie DOT com
DKIM-Filter:	OpenDKIM Filter v2.11.0 sourceware.org 3B69E3858C36
DKIM-Signature:	v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com;
	s=default; t=1709733338;
	bh=zvo6msY2WIMJG21u1/unEY9q/I4to5V/KB58vksCNhY=;
	h=Date:Subject:To:References:In-Reply-To:List-Id:List-Unsubscribe:
	List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:
	From;
	b=aK7cyPSmw8WKenwmEJvZlljQmSTg0bnzdbnVFTZDIQWHd1yuMY2ZAFhDlW7kpWZpp
	dU/devkd4n/6d8RcY0MOxLDH9DtoUBCp8uWwmxI+Jptl0h+wJz0iGVmlX2citcDMZV
	P3adIWM/dA/bVf1kpSSBW7+3s57kF6XWshq0DY0Q=
X-Original-To:	cygwin AT cygwin DOT com
Delivered-To:	cygwin AT cygwin DOT com
DMARC-Filter:	OpenDMARC Filter v1.4.2 sourceware.org E80653858034
ARC-Filter:	OpenARC Filter v1.0.0 sourceware.org E80653858034
ARC-Seal:	i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1709733284; cv=none;
	b=rGQS53+c90HAOl8irTotVNJ7WOz0mLyUM5JODCcdk7tUDAxpgingYUkDf3rGpnkIG10o2q5DIJxHZSTKtz31oUlRpHKIuQTIIhkMd6PO7X6emBbk3+ZGlOvDhUSuAgqq9UDbO1UswCHcgR0tVrEACwJnACme5QGPYuocmdmSYB0=
ARC-Message-Signature:	i=1; a=rsa-sha256; d=sourceware.org; s=key;
	t=1709733284; c=relaxed/simple;
	bh=8rP1dTasS36hG+V8YxbKr1t9MUev6d90NedD50ga3Hs=;
	h=Message-ID:Date:MIME-Version:Subject:To:From;
	b=ehFdCpyVVAGemHPtwAfNquEBO26Ujd09dYQPuqY7OI+kZWyOl3YV+J/yc9Of61Y5E1erHHIyVGPho1ec3Q8Axj47IgGCRFg/pQf8nGlt4ukvmn4xnNSYVb0YNL6RdgB2Y4f32lm1Rh+LPHQytqD4raVEYfAP1gDKXmLieBw3/58=
ARC-Authentication-Results:	i=1; server2.sourceware.org
Message-ID:	<208962d1-78cd-4e2d-9e4a-a1edcbf5e809@SystematicSW.ab.ca>
Date:	Wed, 6 Mar 2024 06:54:39 -0700
MIME-Version:	1.0
User-Agent:	Mozilla Thunderbird
Subject:	Re: ls/stat on OneDrive causes download of files
To:	cygwin AT cygwin DOT com
References:	<CAC9GOO-RbcR5YOPyk6qVDuVSdDK4M=f1V8bHvz7yvoaHjGUZnQ AT mail DOT gmail DOT com>
	<ZehuDAKd3ceXlfba AT calimero DOT vinschen DOT de>
	<ZehvbhVizZTBSaRR AT calimero DOT vinschen DOT de>
Organization:	Systematic Software
In-Reply-To:	<ZehvbhVizZTBSaRR@calimero.vinschen.de>
X-Rspamd-Queue-Id:	E1B3520029
X-Spam-Status:	No, score=-2.1 required=5.0 tests=BAYES_00, KAM_DMARC_STATUS,
	RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_PASS,
	SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE,
	UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6
X-Stat-Signature:	ifbw9orrx5dra3i3mxh79ecr9yqkiozf
X-Rspamd-Server:	rspamout04
X-Session-Marker:	427269616E2E496E676C69734053797374656D6174696353572E61622E6361
X-Session-ID:	U2FsdGVkX1/dOvHJ37m1NoNkCgJRgDlmb7kVGzGxCjg=
X-HE-Tag:	1709733279-307392
X-HE-Meta:	U2FsdGVkX18DNvQmXzBJFXjKYELFZ1hv1SfnvmTGqvy4jvXxR/iKeIQEoUe++KuAgFe2mLBCM3Ec76ZhwrXVOZ0HC6UUw1OlWPFx1AfIVFHgRaANxp9CzcVNwO8Vt2si8NmZBh+VKJ4jobf/tupfAfIZC9pONl0sduJLIdYHKwK9qUXBUqENKtgm4WhPHzBw13cxLKjsve1FxhYO8BobOz82QFgFJa/YFKvtRQVAC/0gDO/JheXJSu230fea6sA46XpCGW16G2OuVA7RnYe/1x+lEoxdOCdzgWxOXPawK/jBMd+ZiQ8n38gXpeBdYpJ7z8uOt2m70VtDkS1XGE1xvbPDxvbbxIXCAgLDhm8afvqYW6lpaRfy7Qnv5XerThYWjJ9p1Fq8llTImI3gXz7L95GVsqDfCguP
X-Spam-Checker-Version:	SpamAssassin 3.4.6 (2021-04-09) on
	server2.sourceware.org
X-BeenThere:	cygwin AT cygwin DOT com
X-Mailman-Version:	2.1.30
List-Id:	General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Unsubscribe:	<https://cygwin.com/mailman/options/cygwin>,
	<mailto:cygwin-request AT cygwin DOT com?subject=unsubscribe>
List-Archive:	<https://cygwin.com/pipermail/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe:	<https://cygwin.com/mailman/listinfo/cygwin>,
	<mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From:	Brian Inglis via Cygwin <cygwin AT cygwin DOT com>
Reply-To:	cygwin AT cygwin DOT com
Cc:	Brian Inglis <Brian DOT Inglis AT SystematicSW DOT ab DOT ca>
Errors-To:	cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com
Sender:	"Cygwin" <cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com>
X-MIME-Autoconverted:	from base64 to 8bit by delorie.com id 426Dtec23741699

On 2024-03-06 06:28, Corinna Vinschen via Cygwin wrote:
> On Mar  6 14:22, Corinna Vinschen via Cygwin wrote:
>> On Mar  5 19:54, Marcin Wisnicki via Cygwin wrote:
>>> If I invoke ls or anything else that does stat inside OneDrive folder
>>> it will trigger download of all files.
>>>
>>> OneDrive uses placeholder files[1] to represent remote files.
>>>
>>> I'm guessing reading file content in stat is to support detection of
>>> actually executable files as in here[2]?
>>>
>>> I think this should be disabled on non-hydrated placeholder files.
>>> Running `find` or 'ls -R` and having your entire OneDrive downloaded
>>> is extremely problematic.
>>>
>>> I could live without executable scripts in the OneDrive folder and
>>> it's easy to mark files as always offline to solve it.
>>>
>>> Another idea is to skip checking files with extensions known to be
>>> non-executable such as jpg (or just any extensions that is not known
>>> to be executable).
>>
>> Nothing of this makes sense from a POSIX library POV.  The library can
>> either not handle placeholder files specially, as today, or it can
>> handle them all the same way.
>>
>> Given these placeholder files are actually reparse points of type
>> IO_REPARSE_TAG_FILE_PLACEHOLDER, we can handle them as symbolic links.
>>
>> However, the structure of the IO_REPARSE_TAG_FILE_PLACEHOLDER reparse
>> data buffer is undocumented.  It would be helpful if somebody using
>> OneDrive would examine the content of the attached REPARSE_DATA_BUFFER.
>>
>>> [2] https://github.com/msys2/msys2-runtime/blob/msys2-3.4.10/winsup/cygwin/fhandler/disk_file.cc#L548
>>
>> The NtReadFile call at this point is not the problem.  It would be
>> helpful to point to Cygwin's source instead of MSYS2, btw.
> 
> Oh, btw., this is from
> https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/c8e77b37-3909-4fe6-a4ea-2b9d423b1ee4:
> 
>    IO_REPARSE_TAG_FILE_PLACEHOLDER
>    0x80000015
> 
>      Obsolete.
>      ---------
>      Used by Windows Shell for legacy placeholder files in Windows 8.1.
>      Server-side interpretation only, not meaningful over the wire.
> 
> So even if we support them, what is their replacement in W10 and later?

May or not help:

https://stackoverflow.com/questions/59152220/cant-get-reparse-point-information-for-the-onedrive-folder
-- 
Take care. Thanks, Brian Inglis              Calgary, Alberta, Canada

La perfection est atteinte                   Perfection is achieved
non pas lorsqu'il n'y a plus rien à ajouter  not when there is no more to add
mais lorsqu'il n'y a plus rien à retirer     but when there is no more to cut
                                 -- Antoine de Saint-Exupéry


-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple

- Raw text -