X-Recipient: archive-cygwin AT delorie DOT com
DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 148BC3857C73
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com;
	s=default; t=1709751691;
	bh=IyYUKkRkop0Zn//RJ3biMnHXMwUSb4YB45UO2SrSrQo=;
	h=Date:To:Subject:References:In-Reply-To:List-Id:List-Unsubscribe:
	 List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:
	 From;
	b=CNMHyo0msAsL4AKWYQja+8Npq9ztjkeltWGbQvBQ1Q6xIuzJIGh++mz3ab/MKU+zC
	 tP1WTvOLNOXFTPSeP6w44xk8NuU4m51Y9duJm+M6Kbcgd4hpqHOu4Tkby6rYLCa/if
	 IQEcsz5KQ9FvSgtNRA7BzCfLIjBfQO1GjhM+ofF4=
X-Original-To: cygwin AT cygwin DOT com
Delivered-To: cygwin AT cygwin DOT com
DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 748683858D34
Date: Wed, 6 Mar 2024 20:00:36 +0100
To: cygwin AT cygwin DOT com
Subject: Re: ls/stat on OneDrive causes download of files
Message-ID: <Zei9VLMhI1JxywPO@calimero.vinschen.de>
Mail-Followup-To: cygwin AT cygwin DOT com
References: <CAC9GOO-RbcR5YOPyk6qVDuVSdDK4M=f1V8bHvz7yvoaHjGUZnQ AT mail DOT gmail DOT com>
 <ZehuDAKd3ceXlfba AT calimero DOT vinschen DOT de>
 <ZehvbhVizZTBSaRR AT calimero DOT vinschen DOT de>
 <208962d1-78cd-4e2d-9e4a-a1edcbf5e809 AT SystematicSW DOT ab DOT ca>
 <ZeilkJK7Csryuzkc AT calimero DOT vinschen DOT de>
MIME-Version: 1.0
Content-Disposition: inline
In-Reply-To: <ZeilkJK7Csryuzkc@calimero.vinschen.de>
X-BeenThere: cygwin AT cygwin DOT com
X-Mailman-Version: 2.1.30
Precedence: list
List-Id: General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Unsubscribe: <https://cygwin.com/mailman/options/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=unsubscribe>
List-Archive: <https://cygwin.com/pipermail/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe: <https://cygwin.com/mailman/listinfo/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From: Corinna Vinschen via Cygwin <cygwin AT cygwin DOT com>
Reply-To: cygwin AT cygwin DOT com
Cc: Corinna Vinschen <corinna-cygwin AT cygwin DOT com>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com
Sender: "Cygwin" <cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com>

On Mar  6 18:19, Corinna Vinschen via Cygwin wrote:
> On Mar  6 06:54, Brian Inglis via Cygwin wrote:
> > On 2024-03-06 06:28, Corinna Vinschen via Cygwin wrote:
> > > On Mar  6 14:22, Corinna Vinschen via Cygwin wrote:
> > > > Given these placeholder files are actually reparse points of type
> > > > IO_REPARSE_TAG_FILE_PLACEHOLDER, we can handle them as symbolic links.
> > > > 
> > > > However, the structure of the IO_REPARSE_TAG_FILE_PLACEHOLDER reparse
> > > > data buffer is undocumented.  It would be helpful if somebody using
> > > > OneDrive would examine the content of the attached REPARSE_DATA_BUFFER.
> > > > 
> > > > > [2] https://github.com/msys2/msys2-runtime/blob/msys2-3.4.10/winsup/cygwin/fhandler/disk_file.cc#L548
> > > > 
> > > > The NtReadFile call at this point is not the problem.  It would be
> > > > helpful to point to Cygwin's source instead of MSYS2, btw.
> > > 
> > > Oh, btw., this is from
> > > https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/c8e77b37-3909-4fe6-a4ea-2b9d423b1ee4:
> > > 
> > >    IO_REPARSE_TAG_FILE_PLACEHOLDER
> > >    0x80000015
> > > 
> > >      Obsolete.
> > >      ---------
> > >      Used by Windows Shell for legacy placeholder files in Windows 8.1.
> > >      Server-side interpretation only, not meaningful over the wire.
> > > 
> > > So even if we support them, what is their replacement in W10 and later?
> > 
> > May or not help:
> > 
> > https://stackoverflow.com/questions/59152220/cant-get-reparse-point-information-for-the-onedrive-folder
> 
> We can add an explicit call to
> 
>   RtlSetProcessPlaceholderCompatibilityMode (PHCM_EXPOSE_PLACEHOLDERS);
> 
> and we can recognize the IO_REPARSE_TAG_FILE_PLACEHOLDER and
> IO_REPARSE_TAG_CLOUD_* tags during symlink evaluation, but even then
> we still have to know what the reparse point buffer actually contains.
> 
> Given that the content of reparse points with these reparse tags are
> undocumented, some people using cloud services should examine these
> reparse points so we can add some suitable code to Cygwin.

Reading further on this it seems that one cannot easily compare these
reparse points with symlinks.

The tag values are 0x80000015 for IO_REPARSE_TAG_FILE_PLACEHOLDER and
0x9000001AL up to 0x9000F01AL for the IO_REPARSE_TAG_CLOUD_* tags.  None
of them have the name surrogate bit set, so they don't "represent
another named entity in the system".  In other words, these reparse
points represent themselves rather than pointing to some other file, as
symlinks do.

Additionally the IO_REPARSE_TAG_CLOUD_* tags all have the directory bit
set so it seems they are used on the parent(?) directory of the local
OneDrive copy only, but not on the files inside it.

Bottom line:

I wonder if the real deal is not the reparse tag and the reparse
content, but whether or not the file has the FILE_ATTRIBUTE_OFFLINE flag
set.

If so, we can try to disable any action within path conversion, as 
well as in our stat(2) and readdir(3) implementation which would
trigger onlining an offline file.

Can anybody confirm that the idea is right, or if I'm something missing?


Corinna

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple