X-Recipient:	archive-cygwin AT delorie DOT com
DKIM-Filter:	OpenDKIM Filter v2.11.0 sourceware.org AB9B53857C4B
DKIM-Signature:	v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com;
	s=default; t=1709731748;
	bh=4GYY9qNwpYxLnqMymjm0HYfP9C1sTUqmfLlbHhREbK0=;
	h=Date:To:Subject:References:In-Reply-To:List-Id:List-Unsubscribe:
	List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:
	From;
	b=NxNGPJ7sDISZbpHN83ukGydLB4ZIkiv8G4wlkLa/VHrKtSWyCsRf6hXFxHBa4hmML
	8YvOYOTsfsWc6+og5gFAP82KSzVIYRAh20B5vhI3f+7IcROnDVxmjwwIXji/0uhWi3
	rG6lSYUzaT4qoX6mWwL+VOtCs5f81pyG9l5QDJfc=
X-Original-To:	cygwin AT cygwin DOT com
Delivered-To:	cygwin AT cygwin DOT com
DKIM-Filter:	OpenDKIM Filter v2.11.0 sourceware.org 60C5C38582A3
Date:	Wed, 6 Mar 2024 14:28:14 +0100
To:	cygwin AT cygwin DOT com
Subject:	Re: ls/stat on OneDrive causes download of files
Message-ID:	<ZehvbhVizZTBSaRR@calimero.vinschen.de>
Mail-Followup-To:	cygwin AT cygwin DOT com
References:	<CAC9GOO-RbcR5YOPyk6qVDuVSdDK4M=f1V8bHvz7yvoaHjGUZnQ AT mail DOT gmail DOT com>
	<ZehuDAKd3ceXlfba AT calimero DOT vinschen DOT de>
MIME-Version:	1.0
In-Reply-To:	<ZehuDAKd3ceXlfba@calimero.vinschen.de>
X-BeenThere:	cygwin AT cygwin DOT com
X-Mailman-Version:	2.1.30
List-Id:	General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Unsubscribe:	<https://cygwin.com/mailman/options/cygwin>,
	<mailto:cygwin-request AT cygwin DOT com?subject=unsubscribe>
List-Archive:	<https://cygwin.com/pipermail/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe:	<https://cygwin.com/mailman/listinfo/cygwin>,
	<mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From:	Corinna Vinschen via Cygwin <cygwin AT cygwin DOT com>
Reply-To:	cygwin AT cygwin DOT com
Cc:	Corinna Vinschen <corinna-cygwin AT cygwin DOT com>
Errors-To:	cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com
Sender:	"Cygwin" <cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com>

On Mar  6 14:22, Corinna Vinschen via Cygwin wrote:
> On Mar  5 19:54, Marcin Wisnicki via Cygwin wrote:
> > If I invoke ls or anything else that does stat inside OneDrive folder
> > it will trigger download of all files.
> > 
> > OneDrive uses placeholder files[1] to represent remote files.
> > 
> > I'm guessing reading file content in stat is to support detection of
> > actually executable files as in here[2]?
> > 
> > I think this should be disabled on non-hydrated placeholder files.
> > Running `find` or 'ls -R` and having your entire OneDrive downloaded
> > is extremely problematic.
> > 
> > I could live without executable scripts in the OneDrive folder and
> > it's easy to mark files as always offline to solve it.
> > 
> > Another idea is to skip checking files with extensions known to be
> > non-executable such as jpg (or just any extensions that is not known
> > to be executable).
> 
> Nothing of this makes sense from a POSIX library POV.  The library can
> either not handle placeholder files specially, as today, or it can
> handle them all the same way.
> 
> Given these placeholder files are actually reparse points of type
> IO_REPARSE_TAG_FILE_PLACEHOLDER, we can handle them as symbolic links.
> 
> However, the structure of the IO_REPARSE_TAG_FILE_PLACEHOLDER reparse
> data buffer is undocumented.  It would be helpful if somebody using
> OneDrive would examine the content of the attached REPARSE_DATA_BUFFER.
> 
> > [2] https://github.com/msys2/msys2-runtime/blob/msys2-3.4.10/winsup/cygwin/fhandler/disk_file.cc#L548
> 
> The NtReadFile call at this point is not the problem.  It would be
> helpful to point to Cygwin's source instead of MSYS2, btw.

Oh, btw., this is from
https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/c8e77b37-3909-4fe6-a4ea-2b9d423b1ee4:

  IO_REPARSE_TAG_FILE_PLACEHOLDER
  0x80000015

    Obsolete.
    ---------
    Used by Windows Shell for legacy placeholder files in Windows 8.1.
    Server-side interpretation only, not meaningful over the wire.

So even if we support them, what is their replacement in W10 and later?


Corinna

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple

- Raw text -