X-Recipient: archive-cygwin AT delorie DOT com
DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 25D823858012
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com;
	s=default; t=1709731401;
	bh=BUIjIsUymGmTFky21ZmJgKP1J9bmXI1+BXqMVhElPFE=;
	h=Date:To:Subject:References:In-Reply-To:List-Id:List-Unsubscribe:
	 List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:
	 From;
	b=wWxdGSp2CvYryYZhVUcHUWBjzEHPMGv5Eo+NgKfVk1Fh8D7qnviG9EbP+czvkuWRn
	 JIeXO/RcxQ+BbfAUOAORAesA712RrEvdCxV6r7u6Sn0vvhAD9w2cBpeB9Xr4caN14M
	 xmplUxrys87oDxrafSpzkMuTKAgZAgqbSMmbztCo=
X-Original-To: cygwin AT cygwin DOT com
Delivered-To: cygwin AT cygwin DOT com
DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org E3BBF385843A
Date: Wed, 6 Mar 2024 14:22:20 +0100
To: cygwin AT cygwin DOT com
Subject: Re: ls/stat on OneDrive causes download of files
Message-ID: <ZehuDAKd3ceXlfba@calimero.vinschen.de>
Mail-Followup-To: cygwin AT cygwin DOT com
References: <CAC9GOO-RbcR5YOPyk6qVDuVSdDK4M=f1V8bHvz7yvoaHjGUZnQ AT mail DOT gmail DOT com>
MIME-Version: 1.0
Content-Disposition: inline
In-Reply-To: <CAC9GOO-RbcR5YOPyk6qVDuVSdDK4M=f1V8bHvz7yvoaHjGUZnQ@mail.gmail.com>
X-BeenThere: cygwin AT cygwin DOT com
X-Mailman-Version: 2.1.30
Precedence: list
List-Id: General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Unsubscribe: <https://cygwin.com/mailman/options/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=unsubscribe>
List-Archive: <https://cygwin.com/pipermail/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe: <https://cygwin.com/mailman/listinfo/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From: Corinna Vinschen via Cygwin <cygwin AT cygwin DOT com>
Reply-To: cygwin AT cygwin DOT com
Cc: Corinna Vinschen <corinna-cygwin AT cygwin DOT com>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com
Sender: "Cygwin" <cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com>

On Mar  5 19:54, Marcin Wisnicki via Cygwin wrote:
> If I invoke ls or anything else that does stat inside OneDrive folder
> it will trigger download of all files.
> 
> OneDrive uses placeholder files[1] to represent remote files.
> 
> I'm guessing reading file content in stat is to support detection of
> actually executable files as in here[2]?
> 
> I think this should be disabled on non-hydrated placeholder files.
> Running `find` or 'ls -R` and having your entire OneDrive downloaded
> is extremely problematic.
> 
> I could live without executable scripts in the OneDrive folder and
> it's easy to mark files as always offline to solve it.
> 
> Another idea is to skip checking files with extensions known to be
> non-executable such as jpg (or just any extensions that is not known
> to be executable).

Nothing of this makes sense from a POSIX library POV.  The library can
either not handle placeholder files specially, as today, or it can
handle them all the same way.

Given these placeholder files are actually reparse points of type
IO_REPARSE_TAG_FILE_PLACEHOLDER, we can handle them as symbolic links.

However, the structure of the IO_REPARSE_TAG_FILE_PLACEHOLDER reparse
data buffer is undocumented.  It would be helpful if somebody using
OneDrive would examine the content of the attached REPARSE_DATA_BUFFER.

> [2] https://github.com/msys2/msys2-runtime/blob/msys2-3.4.10/winsup/cygwin/fhandler/disk_file.cc#L548

The NtReadFile call at this point is not the problem.  It would be
helpful to point to Cygwin's source instead of MSYS2, btw.


Corinna

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple