DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:message-id:date:mime-version:content-type
	:content-transfer-encoding:to:subject:in-reply-to:references
	:from; q=dns; s=default; b=gW0oF8ok6jWr+BycEMERjHZx42BrM5kuf1a8C
	yrtJtVdUioHr8nvCigcA1m96drty3gJl6FpVE96BeyYYrbK33vdYQV4aCUh3Nc1o
	+NfzoQV098TZ5smV0SO6z5Q1AM+leSXnBRNCVacVKXAmPk7qI1p+zGRPNJeNC8v9
	au9nkA=
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Message-ID: <21334.55207.784319.488271@compute01.cs.columbia.edu>
Date: Tue, 22 Apr 2014 16:57:11 -0400
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="WFG9/5G373"
Content-Transfer-Encoding: 7bit
To: cygwin AT cygwin DOT com
Subject: Re: fstat st_size on open files on Parallels filesystem is wrong
In-Reply-To: <20140422081628.GC2339@calimero.vinschen.de>
References: <21333 DOT 25325 DOT 11106 DOT 958642 AT compute01 DOT cs DOT columbia DOT edu>	<227151856 DOT 20140421223417 AT yandex DOT ru>	<21333 DOT 26515 DOT 393838 DOT 380071 AT compute01 DOT cs DOT columbia DOT edu>	<20140422081628 DOT GC2339 AT calimero DOT vinschen DOT de>
From: lennox AT cs DOT columbia DOT edu

--WFG9/5G373
Content-Type: text/plain; charset=utf-8
Content-Description: message body text
Content-Transfer-Encoding: quoted-printable

On Tuesday, April 22 2014, "Corinna Vinschen" wrote to "cygwin at cygwin.co=
m" saying:

> On Apr 21 14:46, lennox at cs.columbia.edu wrote:
> > On Monday, April 21 2014, "Andrey Repin" wrote to "lennox at
> > cs.columbia.edu, cygwin at cygwin.com" saying:
> >=20
> > > Greetings, lennox at cs.columbia.edu!
> > >=20
> > > > I=E2=80=99m running cygwin64 1.7.29 in a Windows 8.1 Pro virtual ma=
chine, running in
> > > > Parallels Desktop 9.0.24229 on Mac OS X 10.9.2.
> > >=20
> > > > Parallels Desktop automatically mounts my Mac OS X home directory a=
s a Z:
> > > > drive in Windows.  Cygwin mount reports this drive as being type "p=
rlsf".
> > >=20
> > > > Unfortunately, I've discovered that if I have an open file on this
> > > > filesystem which has been written to, the size returned by Cygwin f=
stat() on
> > > > the open file is wrong.  A stat() of the file after it's been close=
d is
> > > > correct.
> > >=20
> > > > This has the consequence that emacs always thinks saved files have =
been
> > > > modified externally, since emacs looks at files' sizes (as well as =
their
> > > > modification times) to detect external changes.  This makes emacs
> > > > near-unusable.
> > >=20
> > > > This problem does not occur for files in my Cygwin home directory, =
or other
> > > > locations mounted on my Windows C: drive.
> > >=20
> > > > I've attached a simple unit test program that illustrates the probl=
em.
> > > > I've also attached my cygcheck -s -v -r output.
> > >=20
> > > > Any ideas?  Is this a Cygwin bug, a Parallels bug, or something els=
e?
> > > > Glancing over the Cygwin code, I see that there are a few cases whe=
re fstat
> > > > has special cases for certain filesystem types.
> > >=20
> > > You never flushing the buffer in your test code, or I'm reading it wr=
ong?
> >=20
> > This is using Posix APIs -- open() / write() -- not C APIs, fopen() /
> > fwrite(), so there shouldn't be a buffer?  Notice that the test behaves=
 as I
> > expect for a file on NTFS.
> >=20
> > Adding a call to fsync() prior to the fstat() call doesn't change anyth=
ing.
>=20
> This is actually a bad sign.  The problem you're describing occurs on
> NFS, too.  If you write to the file, a subsequent call to fetch stat
> attributes does not return the actual size of the file, but the size at
> the time the handle has been opened.
>=20
> However, on NFS, a call to FlushFileBuffers helps to kick stat back into
> shape.  That's the Win32 function called from fsync as well.  What is
> Cygwin supposed to do if that doesn't work?

Okay, interesting further investigation.

The Parallels filesystem appears to work correctly when I repeat the test
case using Windows kernel32 APIs -- specifically, FileInformationByHandle --
so something's different between the kernel32 APIs and the ntdll APIs that
Cygwin uses.

Sample code for Win32 test attached.  Works identically with Cygwin, MinGW,
or Visual C++.

Just spitballing here, but -- I see that cygwin's file_get_fnoi function
(which is where fhandler_base::fstat_by_handle gets its size parameter)
tries NtQueryInformationFile(FileNetworkOpenInformation) before
NtQueryInformationFile(FileStandardInformation).  Is it possible that the
Parallels filesystem isn't filling out all fields of that API properly?  Is
there a straightforward way I could debug this?


$ ./stat-size-test.exe '/cygdrive/z/foo'
/cygdrive/z/foo: fstat: st_size=3D0
/cygdrive/z/foo: stat: st_size=3D12

$ ./win32-size-test.exe 'z:\foo'
z:\foo: FileInformationByHandle: nFileSize=3D12

$ mount
C:/cygwin64/bin on /usr/bin type ntfs (binary,auto)
C:/cygwin64/lib on /usr/lib type ntfs (binary,auto)
C:/cygwin64 on / type ntfs (binary,auto)
C: on /cygdrive/c type ntfs (binary,posix=3D0,user,noumount,auto)
S: on /cygdrive/s type ntfs (binary,posix=3D0,user,noumount,auto)
U: on /cygdrive/u type ntfs (binary,posix=3D0,user,noumount,auto)
Z: on /cygdrive/z type prlsf (binary,posix=3D0,user,noumount,auto)


--WFG9/5G373
Content-Type: text/plain; name="win32-size-test.c"
Content-Disposition: inline;
	filename="win32-size-test.c"
Content-Transfer-Encoding: 7bit

#include <windows.h>
#include <stdio.h>
#include <inttypes.h>

#if __LP64__
#define PRI_DWORD "u"
#else
#define PRI_DWORD "lu"
#endif

static const char* geterr(DWORD err)
{
	static char msg[1024];

	FormatMessage(FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_IGNORE_INSERTS |
		FORMAT_MESSAGE_MAX_WIDTH_MASK, NULL, err,
		MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT),
		(LPSTR)msg, sizeof(msg), NULL);

	return msg;
}

int main(int argc, char* argv[])
{
	int i;
	int status = 0;
	for (i = 1; i < argc; i++) {
		HANDLE file;
		DWORD err, bytesWritten;
		BY_HANDLE_FILE_INFORMATION fileInfo;

		file = CreateFileA(argv[i], GENERIC_WRITE, FILE_SHARE_WRITE,
			NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL);
		if (file == INVALID_HANDLE_VALUE) {
			err = GetLastError();
			fprintf(stderr, "%s: CreateFile: %s (err %" PRI_DWORD")\n", argv[i],
				geterr(err), err);
			status++;
			continue;
		}

		if (!WriteFile(file, "Hello world!\n", 12, &bytesWritten, NULL) ||
			bytesWritten != 12) {
			err = GetLastError();
			fprintf(stderr, "%s: WriteFile: %s (err %" PRI_DWORD ")\n", argv[i],
				geterr(err), err);
			CloseHandle(file);
			status++;
			continue;
		}

		if (!GetFileInformationByHandle(file, &fileInfo)) {
			err = GetLastError();
			fprintf(stderr, "%s: GetFileInformationByHandle: %s (err %" PRI_DWORD ")\n",
				argv[i], geterr(err), err);
			CloseHandle(file);
			status++;
			continue;
		}

		printf("%s: FileInformationByHandle: nFileSize=%" PRIu64 "\n",
			argv[i], (uint64_t)fileInfo.nFileSizeHigh << 32 | fileInfo.nFileSizeLow);

		if (!CloseHandle(file)) {
			err = GetLastError();
			fprintf(stderr, "%s: CloseHandle: %s (err %" PRI_DWORD ")\n", argv[i],
				geterr(err), err);
			status++;
			continue;
		}
	}
	return status;
}

--WFG9/5G373
Content-Type: text/plain; charset=us-ascii
Content-Description: .signature
Content-Transfer-Encoding: 7bit


-- 
Jonathan Lennox
lennox at cs.columbia.edu


--WFG9/5G373
Content-Type: text/plain; charset=us-ascii

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
--WFG9/5G373--