DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:date:from:to:subject:message-id:reply-to
	:references:mime-version:content-type:in-reply-to; q=dns; s=
	default; b=jRpLdnL5ZuQ878sNny5iPfwgKjcI8/s6bOXBShAaKSmiBec/Z/SK5
	Z46biM5Uosfw+ggk5bGKSqfP3hFIdT05uE191bJ5rcdUxZuwhWiyfBeU/HnuFDS8
	X9xUqob2c3bD/ygevz+4sj4dsuEMLZFNTlT1EQ9MSLPqkIKrNCYnzk=
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Date: Wed, 23 Apr 2014 10:40:56 +0200
From: Corinna Vinschen <corinna-cygwin AT cygwin DOT com>
To: cygwin AT cygwin DOT com
Subject: Re: fstat st_size on open files on Parallels filesystem is wrong
Message-ID: <20140423084056.GJ2339@calimero.vinschen.de>
Reply-To: cygwin AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
References: <21333 DOT 25325 DOT 11106 DOT 958642 AT compute01 DOT cs DOT columbia DOT edu> <227151856 DOT 20140421223417 AT yandex DOT ru> <21333 DOT 26515 DOT 393838 DOT 380071 AT compute01 DOT cs DOT columbia DOT edu> <20140422081628 DOT GC2339 AT calimero DOT vinschen DOT de> <21334 DOT 55207 DOT 784319 DOT 488271 AT compute01 DOT cs DOT columbia DOT edu>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha1;	protocol="application/pgp-signature"; boundary="eDB11BtaWSyaBkpc"
Content-Disposition: inline
In-Reply-To: <21334.55207.784319.488271@compute01.cs.columbia.edu>
User-Agent: Mutt/1.5.21 (2010-09-15)

--eDB11BtaWSyaBkpc
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Apr 22 16:57, lennox AT cs DOT columbia DOT edu wrote:
> On Tuesday, April 22 2014, "Corinna Vinschen" wrote to "cygwin at cygwin.=
com" saying:
> > On Apr 21 14:46, lennox at cs.columbia.edu wrote:
> > > [...]
> > > This is using Posix APIs -- open() / write() -- not C APIs, fopen() /
> > > fwrite(), so there shouldn't be a buffer?  Notice that the test behav=
es as I
> > > expect for a file on NTFS.
> > >=20
> > > Adding a call to fsync() prior to the fstat() call doesn't change any=
thing.
> >=20
> > This is actually a bad sign.  The problem you're describing occurs on
> > NFS, too.  If you write to the file, a subsequent call to fetch stat
> > attributes does not return the actual size of the file, but the size at
> > the time the handle has been opened.
> >=20
> > However, on NFS, a call to FlushFileBuffers helps to kick stat back into
> > shape.  That's the Win32 function called from fsync as well.  What is
> > Cygwin supposed to do if that doesn't work?
>=20
> Okay, interesting further investigation.
>=20
> The Parallels filesystem appears to work correctly when I repeat the test
> case using Windows kernel32 APIs -- specifically, FileInformationByHandle=
 --
> so something's different between the kernel32 APIs and the ntdll APIs that
> Cygwin uses.
>=20
> Sample code for Win32 test attached.  Works identically with Cygwin, MinG=
W,
> or Visual C++.
>=20
> Just spitballing here, but -- I see that cygwin's file_get_fnoi function
> (which is where fhandler_base::fstat_by_handle gets its size parameter)
> tries NtQueryInformationFile(FileNetworkOpenInformation) before
> NtQueryInformationFile(FileStandardInformation).  Is it possible that the
> Parallels filesystem isn't filling out all fields of that API properly?  =
Is
> there a straightforward way I could debug this?

You could use the same functions.  Since you had a look into the sources
anyway, you probably saw the comments:

  /* Some FSes (Netapps) don't implement FileNetworkOpenInformation. */
  [...call NtQueryInformationFile(FileNetworkOpenInformation) unless
      the skip_network_open_inf flag is given...]
  if (status =3D=3D STATUS_INVALID_PARAMETER)
     /* Apart from accessing Netapps, this also occurs when accessing SMB
	share root dirs hosted on NT4. */
     [...call NtQueryInformationFile(FileBasicInformation) and
              NtQueryInformationFile(FileStandardInformation)...]

Rather than calling GetFileInformationByHandle, try this

  #include <winternl.h>

  [...]

  NTSTATUS status;
  IO_STATUS_BLOCK io;
  FILE_NETWORK_OPEN_INFORMATION fnoi;
=20=20
  status =3D NtQueryInformationFile (file, &io, &fnoi, sizeof fnoi,
				   FileNetworkOpenInformation);
  if (!NT_SUCCESS (status))
    {
      fprintf (stderr, "NtQueryInformationFile: 0x%08x\n", status);
      [...]
    }
  printf("%s: NtQueryInformationFile: nFileSize=3D%" PRIu64 "\n",
	 argv[i], fnoi.EndOfFile.QuadPart);

Maybe that's really the problem.  If so, either the EndOfFile info
is wrong, or (more likely) the call to NtQueryInformationFile returns
with a non-0 status code, which would be interesting to know.  It's
probably not STATUS_INVALID_PARAMETER, otherwise you probably wouldn't
have seen a problem at all.

Replacing the above call with

  FILE_STANDARD_INFORMATION fsi;
  status =3D NtQueryInformationFile (file, &io, &fsi, sizeof fsi,
				   FileStandardInformation);
  [...]
  printf("%s: NtQueryInformationFile: nFileSize=3D%" PRIu64 "\n",
	 argv[i], fsi.EndOfFile.QuadPart);

should then give the correct result.

Also, to add handling for the Parallels filesystem to Cygwin, I'd
need the info printed by the getVolInfo tool from the csih package:

  $ /usr/lib/csih/getVolInfo /cygdrive/z


Corinna

--=20
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

--eDB11BtaWSyaBkpc
Content-Type: application/pgp-signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBAgAGBQJTV3yYAAoJEPU2Bp2uRE+gqKcP/1ArA4fpFeZoNOWraRgLNxlu
V20ltT5Ls+ErC4MIf5hQrxABOc4SUODmsLXgkmOdcbRXgn/OR/RMWbr3m1XDKEyL
hTUG7mf4ko4T+BuLHOW9iwgnkVCKw11XFozjLO+egN2AFck6hHrM16jtsWBmYS2L
jPB3idmoHBz00oIYGUt0kT28miZ1BcaO+MSusdzm7/KXbT5guKTsBpSfjERJx9E6
Sm0LxQMkmKzvvXaP2oOqyxaxIzhwu3wp+kki6F2JRv1P21cVt1K8/wgC1/79Ayxh
hIV3/ZBwvHyVJE4f5KBueKp9xfLYy6HE6rzOJjsaYCbwunXH/6eDCnM8od/EZQ+P
hoIPeNrqYnTK3TTpUyPjE9Z63J/ihIiJR1XH4PiUbd/UCVgz7Rll65aj1C9d+jdC
vW3HzdVgD4ddCSvczBd0VZzAJPFv25KmUfDH4VStjY/DsrCvJOUhQywK4ec42QpU
+PNPH6yMGfX7oAHG+1lgq46T/LUjVo6SCBKqWH45Ja7mU2rRFwhqEJ5lcH7yiq5p
cHcbgsOIJ+wax2qnYkDwVEXIfZDf75iHC9rzMvMMsO5l4udpK7FpdqaKu2p3twYs
9N6+EyRnIfnQZXVFO9nA5Pb7Yiy1pMZrXLQMebnTWTaPeweOh/yjdDk5G0Udbt7Q
nlgERudKE1SmI4RhWCfb
=VrZ/
-----END PGP SIGNATURE-----

--eDB11BtaWSyaBkpc--