Date: Fri, 8 Mar 2024 14:56:24 +0100
To: Jeffrey Altman <jaltman@secure-endpoints.com>
Subject: Re: ls/stat on OneDrive causes download of files
Message-ID: <ZesZCFdvz4uxLjKg@calimero.vinschen.de>
Mail-Followup-To: Jeffrey Altman <jaltman@secure-endpoints.com>,
 Thomas Wolff <towo@towo.net>, cygwin@cygwin.com
References: <CAC9GOO-RbcR5YOPyk6qVDuVSdDK4M=f1V8bHvz7yvoaHjGUZnQ@mail.gmail.com>
 <ZehuDAKd3ceXlfba@calimero.vinschen.de>
 <ZehvbhVizZTBSaRR@calimero.vinschen.de>
 <208962d1-78cd-4e2d-9e4a-a1edcbf5e809@SystematicSW.ab.ca>
 <ZeilkJK7Csryuzkc@calimero.vinschen.de>
 <7d9fe460-5704-424b-a89b-e34ef2176d38@secure-endpoints.com>
 <ZerqWzAr3b_mpork@calimero.vinschen.de>
 <a5ed70b0-8ac4-453f-b5cf-ce8f77f6bdf7@towo.net>
 <64b65385-cad3-4807-b6c3-e463507da164@secure-endpoints.com>
MIME-Version: 1.0
Content-Disposition: inline
In-Reply-To: <64b65385-cad3-4807-b6c3-e463507da164@secure-endpoints.com>
Precedence: list
From: Corinna Vinschen via Cygwin <cygwin@cygwin.com>
Reply-To: cygwin@cygwin.com
Cc: Corinna Vinschen <corinna-cygwin@cygwin.com>, cygwin@cygwin.com
Content-Type: text/plain; charset="utf-8"
Errors-To: cygwin-bounces+archive-cygwin=delorie.com@cygwin.com
Sender: "Cygwin" <cygwin-bounces+archive-cygwin=delorie.com@cygwin.com>
Content-Transfer-Encoding: 8bit

On Mar  8 08:15, Jeffrey Altman via Cygwin wrote:
> On 3/8/2024 7:52 AM, Thomas Wolff via Cygwin wrote:
> > Am 08.03.2024 um 11:37 schrieb Corinna Vinschen via Cygwin:
> > > Yesterday I stumbled over a certain NtCreateFile flag:
> > > 
> > >    FILE_OPEN_NO_RECALL (0x00400000)
> > > 
> > >      Instructs any filters that perform offline storage or
> > > virtualization
> > >      to not recall the contents of the file as a result of this open.
> > > 
> > > MS-CIFS described it like this:
> > > 
> > >    FILE_OPEN_NO_RECALL
> > >    0x00400000
> > > 
> > >      In a hierarchical storage management environment, this option
> > >      requests that the file SHOULD NOT be recalled from tertiary storage
> > >      such as tape. A file recall can take up to several minutes in a
> > >      hierarchical storage management environment. The clients can
> > > specify
> > >      this option to avoid such delays.
> > > 
> > > This sounds like we could simply add this flag to all NtOpenFile
> > > used for path conversion or stat-like calls, without having to care
> > > for any file attributes specificially.
> > > 
> > > Does that make sense?
> > Sounds good, without even studying the other details...
> > I speculate some more handling would still be needed to avoid executable
> > detection via magic tags.
> > 
> Agreed.   FILE_OPEN_NO_RECALL has been defined for at least a decade but was
> not documented by Microsoft relatively recently.

Thanks for the feedback, guys.

> Another suggestion would be to try opening the file with
> FILE_READ_ATTRIBUTES instead of GENERIC_READ if the file data is not
> required.  See
> 
> https://github.com/microsoft/BuildXL/commit/4fb8e7ce07d243ccd95de0d66da551538a794493

Cygwin uses the minimum of required permissions in NtCreateFile/
NtOpenFile calls anyway.

I'm just running a test cygwin DLL locally with a lot of added
FILE_OPEN_NO_RECALL bits and a couple of added attribute checks for
being offline to allow skipping some code.

I think I'll push this change in a bit so we get a test release out
so people using OneDrive can test.


Thanks,
Corinna

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple