X-Recipient: archive-cygwin AT delorie DOT com
DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org D7A85385C422
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com;
	s=default; t=1709906242;
	bh=KBIw165E3LtpKH3Z+NTmTPKa/2IpVVM0DqOeuquGKUw=;
	h=Date:To:Subject:References:In-Reply-To:List-Id:List-Unsubscribe:
	 List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:
	 From;
	b=laeB9Pr7sfLF9rJ0kJseq/4qnrVXN3vZEtNQSRFPesqcsOJ6WX/UOZr1HvZBMq6o3
	 5WOwl5DT0tB6NhpfsoTgTzsKUo7mxoF9/wBNexsl/OE5uekcaP8oafHM9TH0YKuk5S
	 JpMyeXV/dGe5jYosbJqf3BtwL4djgZ0YHqp2cWPw=
X-Original-To: cygwin AT cygwin DOT com
Delivered-To: cygwin AT cygwin DOT com
DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 24F283858D33
Date: Fri, 8 Mar 2024 14:56:24 +0100
To: Jeffrey Altman <jaltman AT secure-endpoints DOT com>
Subject: Re: ls/stat on OneDrive causes download of files
Message-ID: <ZesZCFdvz4uxLjKg@calimero.vinschen.de>
Mail-Followup-To: Jeffrey Altman <jaltman AT secure-endpoints DOT com>,
 Thomas Wolff <towo AT towo DOT net>, cygwin AT cygwin DOT com
References: <CAC9GOO-RbcR5YOPyk6qVDuVSdDK4M=f1V8bHvz7yvoaHjGUZnQ AT mail DOT gmail DOT com>
 <ZehuDAKd3ceXlfba AT calimero DOT vinschen DOT de>
 <ZehvbhVizZTBSaRR AT calimero DOT vinschen DOT de>
 <208962d1-78cd-4e2d-9e4a-a1edcbf5e809 AT SystematicSW DOT ab DOT ca>
 <ZeilkJK7Csryuzkc AT calimero DOT vinschen DOT de>
 <7d9fe460-5704-424b-a89b-e34ef2176d38 AT secure-endpoints DOT com>
 <ZerqWzAr3b_mpork AT calimero DOT vinschen DOT de>
 <a5ed70b0-8ac4-453f-b5cf-ce8f77f6bdf7 AT towo DOT net>
 <64b65385-cad3-4807-b6c3-e463507da164 AT secure-endpoints DOT com>
MIME-Version: 1.0
Content-Disposition: inline
In-Reply-To: <64b65385-cad3-4807-b6c3-e463507da164@secure-endpoints.com>
X-BeenThere: cygwin AT cygwin DOT com
X-Mailman-Version: 2.1.30
Precedence: list
List-Id: General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Unsubscribe: <https://cygwin.com/mailman/options/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=unsubscribe>
List-Archive: <https://cygwin.com/pipermail/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe: <https://cygwin.com/mailman/listinfo/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From: Corinna Vinschen via Cygwin <cygwin AT cygwin DOT com>
Reply-To: cygwin AT cygwin DOT com
Cc: Corinna Vinschen <corinna-cygwin AT cygwin DOT com>, cygwin AT cygwin DOT com
Content-Type: text/plain; charset="utf-8"
Errors-To: cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com
Sender: "Cygwin" <cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com>
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from base64 to 8bit by delorie.com id 428DvOq8360485

On Mar  8 08:15, Jeffrey Altman via Cygwin wrote:
> On 3/8/2024 7:52 AM, Thomas Wolff via Cygwin wrote:
> > Am 08.03.2024 um 11:37 schrieb Corinna Vinschen via Cygwin:
> > > Yesterday I stumbled over a certain NtCreateFile flag:
> > > 
> > >    FILE_OPEN_NO_RECALL (0x00400000)
> > > 
> > >      Instructs any filters that perform offline storage or
> > > virtualization
> > >      to not recall the contents of the file as a result of this open.
> > > 
> > > MS-CIFS described it like this:
> > > 
> > >    FILE_OPEN_NO_RECALL
> > >    0x00400000
> > > 
> > >      In a hierarchical storage management environment, this option
> > >      requests that the file SHOULD NOT be recalled from tertiary storage
> > >      such as tape. A file recall can take up to several minutes in a
> > >      hierarchical storage management environment. The clients can
> > > specify
> > >      this option to avoid such delays.
> > > 
> > > This sounds like we could simply add this flag to all NtOpenFile
> > > used for path conversion or stat-like calls, without having to care
> > > for any file attributes specificially.
> > > 
> > > Does that make sense?
> > Sounds good, without even studying the other details...
> > I speculate some more handling would still be needed to avoid executable
> > detection via magic tags.
> > 
> Agreed.   FILE_OPEN_NO_RECALL has been defined for at least a decade but was
> not documented by Microsoft relatively recently.

Thanks for the feedback, guys.

> Another suggestion would be to try opening the file with
> FILE_READ_ATTRIBUTES instead of GENERIC_READ if the file data is not
> required.  See
> 
> https://github.com/microsoft/BuildXL/commit/4fb8e7ce07d243ccd95de0d66da551538a794493

Cygwin uses the minimum of required permissions in NtCreateFile/
NtOpenFile calls anyway.

I'm just running a test cygwin DLL locally with a lot of added
FILE_OPEN_NO_RECALL bits and a couple of added attribute checks for
being offline to allow skipping some code.

I think I'll push this change in a bit so we get a test release out
so people using OneDrive can test.


Thanks,
Corinna

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple