X-Recipient: archive-cygwin AT delorie DOT com
DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org E6CD63959E66
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com;
	s=default; t=1629384553;
	bh=L0S6pp3ijR2ecc3Vs9Yg58s0MSGke5tNJVwmeLX1LLs=;
	h=Date:To:Subject:References:In-Reply-To:List-Id:List-Unsubscribe:
	 List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:
	 From;
	b=bVsSEdoizTqF47EQpSP2apFOYjLS5Xvz6o4pVgoZDDJai9IS0kIgmBIsLGp1QdvCk
	 9iU4ZajBHJ8hhh9k3dTJRRyFa/1b5Mg30+4tyZddYoCMSNswUQL6vEZgM8DYmsz1+j
	 IEUr+gmvuSQGstozfeRd13QAYTTe9RrzgekKyUjU=
X-Original-To: cygwin AT cygwin DOT com
Delivered-To: cygwin AT cygwin DOT com
DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 51032385840A
Date: Thu, 19 Aug 2021 16:48:30 +0200
To: cygwin AT cygwin DOT com
Subject: Re: Duplicates in /proc/partitions
Message-ID: <YR5vPnkH6RN6eapZ@calimero.vinschen.de>
Mail-Followup-To: cygwin AT cygwin DOT com
References: <YRqzOZAiDEfkHBM+@calimero.vinschen.de>
 <DM8PR09MB70958FD69B70B19E187BD682A5FE9 AT DM8PR09MB7095 DOT namprd09 DOT prod DOT outlook DOT com>
 <YRu++uklGOPynvdf AT calimero DOT vinschen DOT de>
 <DM8PR09MB709560E2652D35DBE54217C9A5FF9 AT DM8PR09MB7095 DOT namprd09 DOT prod DOT outlook DOT com>
 <YR0g29R5sQpEio9A AT calimero DOT vinschen DOT de>
 <DM8PR09MB7095852064F71BCCBBFB998AA5FF9 AT DM8PR09MB7095 DOT namprd09 DOT prod DOT outlook DOT com>
 <DM8PR09MB7095EE5F6B0192EC6F42823BA5FF9 AT DM8PR09MB7095 DOT namprd09 DOT prod DOT outlook DOT com>
 <DM8PR09MB7095D416926BFC028E6BBC21A5FF9 AT DM8PR09MB7095 DOT namprd09 DOT prod DOT outlook DOT com>
 <YR4sYNqO+bVYuCHd AT calimero DOT vinschen DOT de>
 <YR4/yRCTTTz/nXXl AT calimero DOT vinschen DOT de>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="1YWYokcy67A1TOBd"
Content-Disposition: inline
In-Reply-To: <YR4/yRCTTTz/nXXl@calimero.vinschen.de>
X-Provags-ID: V03:K1:X6eCcUKDhiR2S7mdYfKBkilICxkIs1GJFlwJEZnhLLQHqHxhWaT
 Z5Qhrsq1eDbsSDlh3EDZY8wyleG2SPc1FELCfZNwmSAl0hAXinmszHrX5waMrmD0XhhQ+uX
 rtOJObMm9sH9ZgmLpq3nyk1z58qPJTVGjpiKwoFsnDksl8lxbKZZGp6wX+f83CjWpQZLVBe
 wYqmmLnca0SOkm4Ou+Ujw==
X-UI-Out-Filterresults: notjunk:1;V03:K0:Fj3Zro6vUOM=:tjmyQOBcCbjaAiTmsezUvz
 igL0MEAcId0rh8WNI2Sgd3hoU9U++VdAOD0ZeOEMLbWQ6QXwbZHWzN7fiIO5LJK2yIuu568VD
 yr3fcOY2QGk7P5wKgWvcMLrWpwcDRCo6ieHXRuf9bgXaJs8FxpGSUMkKKWQU8ieV7vgshbU7v
 c99sCmI+jpl3CsY1ZuhLZnUQGas12LHpCvNLs6ci0kDkY2E1jDuQ6OYUL+ZD+zInLtI4ZHIey
 sgFBOFMR0fdCJos5HQOqj1bVtB/bSo81XTDNfqc7a/EX70+TfxGjr7hDMg7JtVuQ77Ad9KXAo
 QQkM9TGZCJHWqX9tRs0Bu7yplpx3n/5huFDn63EwPxAkZEDRMsROtYxEqyVWFiF2P1p/cCw5H
 FQQw3cgahQqberTWwVxsEMqH5jbJrY/cApy3R3qS6LUI7ic1oE2zISmYkDNmtzpjS2u9vYKK2
 miQiExs4Rw+ptYskmdAsPFWRz5s7LwWfaqJXFMZmsBSpvJW4jpXGbqysLN0oOKhGWUzucGnlK
 F3KGeOm2EneXU3mHsGZJ7kUASlgPoHjuGNpm+hmlYp5olVqczgyN0Uoiizjn0fkvXBpmeyYKE
 VpAgSuLtB4yk4MNSNbGsawNRdK+bgJON8cs/VOeEm/Iau0kMlPRTj6YV1sLUmkWw7MVCB4j2K
 uPOvQpR8XrfoTiMMBR+x+N3sixyl8CwjGDE3sXW8Q+8ObysphCUvLtVNCMTcnjP1OWBtnUBy8
 UwymE4CJIxPJEwcUZk5k8+OrKASC1aZHMPuu6LJHikZuOm+SJ7luYfuLzok8lJtyafsPkewaT
 o/2FTeD2Yth8Z2TOfWHbGE05lN1kHe0O77kwdo+6Jf3ImPYZVcgrLlciXSIXnmOhJQOgFf1v3
 vm2xkLRMD9RziPZvaNtQ==
X-Spam-Status: No, score=-100.2 required=5.0 tests=BAYES_00,
 GOOD_FROM_CORINNA_CYGWIN, KAM_DMARC_NONE, KAM_DMARC_STATUS, RCVD_IN_MSPIKE_H4,
 RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_NEUTRAL,
 TXREP autolearn=ham autolearn_force=no version=3.4.4
X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on
 server2.sourceware.org
X-BeenThere: cygwin AT cygwin DOT com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Unsubscribe: <https://cygwin.com/mailman/options/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=unsubscribe>
List-Archive: <https://cygwin.com/pipermail/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe: <https://cygwin.com/mailman/listinfo/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From: Corinna Vinschen via Cygwin <cygwin AT cygwin DOT com>
Reply-To: cygwin AT cygwin DOT com
Cc: Corinna Vinschen <corinna-cygwin AT cygwin DOT com>
Errors-To: cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com
Sender: "Cygwin" <cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com>


--1YWYokcy67A1TOBd
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline

Hi Anton,

On Aug 19 13:26, Corinna Vinschen via Cygwin wrote:
> On Aug 19 12:03, Corinna Vinschen via Cygwin wrote:
> > On Aug 18 18:36, Lavrentiev, Anton (NIH/NLM/NCBI) [C] via Cygwin wrote:
> > > > And I just confirmed that if I print the context right before NtOpenFile(), and just do
> > > > the "continue",
> > > 
> > > But then I also tried this:  I removed the "continue" before "NtOpenFile()" and allowed it to proceed,
> > > but I moved NtClose(devhdl) right after the "printf" statement (that we were tweaking), and
> > > inserted the "continue" there, so it does not proceed with enumerating of the partitions.
> > > And so again, I got the output that matches my disks without the duplication.  So this actually
> > > exonerates NtOpenFile() :-)
> > > [...]
> > 
> > Unfortunately I can't reproduce the issue.  I created a couple of
> > virtual drives and added them to my W10 and my W7 VM, but to no avail.
> 
> Never mind.  I now can reproduce the problem at will.  The trick is to
> create objects inside the \Device directory while the loop iterating
> over the directory is running.
> 
> As shitty as it is, a NtOpenDirectoryObject/NtQueryDirectoryObject/NtClose
> loop is not working atomically.  If a new object is inserted into the
> dir preceeding the currently handled entry (which, on a reliable system
> should *never* occur), the entry is moved by one, and the next
> NtQueryDirectoryObject call returns the same object again.
> 
> Windows never disappoints when one is looking for really ugly problems.
> 
> The good news is that NtQueryDirectoryObject is also capable to take a
> big buffer and return as much entries as fit in in the buffer.  I'll
> give it a try now in the vain hope that this way to call
> NtQueryDirectoryObject returns reliable, atomic-like results...

I tested this on both of my VMs (W7, W10) and the result looks promising.

I implemented this method in the Cygwin DLL locally, for all places
utilizing the NtQueryDirectoryObject call, except for one call where
this isn't possible, unfortunately.  That's readdir on /proc/sys
directories, which is supposed to return one entry on each invocation
only and is thus stateful.

Funny enough this problem may also explain why ps(1) output sometimes
showed some Cygwin processes twice.  Same method, same problem.

Anyway, would you mind to test the below new proc_partition.c as well
as the latest snapshot I just uploaded to https://cygwin.com/snapshots/?


Thanks,
Corinna

--1YWYokcy67A1TOBd
Content-Type: text/plain; charset=utf-8
Content-Disposition: attachment; filename="proc_partition.c"

#include <stdio.h>
#include <windows.h>
#include <winioctl.h>
#include <winternl.h>
#include <wchar.h>
#include <wctype.h>
#include <ntstatus.h>

NTSTATUS NTAPI NtOpenDirectoryObject (PHANDLE, ACCESS_MASK, POBJECT_ATTRIBUTES);
NTSTATUS NTAPI NtQueryDirectoryObject (HANDLE, PVOID, ULONG, BOOLEAN,
				       BOOLEAN, PULONG, PULONG);

typedef struct _DIRECTORY_BASIC_INFORMATION
{
  UNICODE_STRING ObjectName;
  UNICODE_STRING ObjectTypeName;
} DIRECTORY_BASIC_INFORMATION, *PDIRECTORY_BASIC_INFORMATION;

#define DIRECTORY_QUERY 1
#define NT_MAX_PATH	32768

char buf[NT_MAX_PATH];
char ioctl_buf[NT_MAX_PATH];
WCHAR mp_buf[NT_MAX_PATH];

typedef enum { false, true } bool;

int
main ()
{
  OBJECT_ATTRIBUTES attr;
  IO_STATUS_BLOCK io;
  NTSTATUS status;
  HANDLE dirhdl;
  WCHAR fpath[MAX_PATH];
  WCHAR gpath[MAX_PATH];
  DWORD len;

  /* Note that device ids and names are just faked here, not using the internal
     Cygwin code to generate device major/minor numbers and device names.
     There's *no* guarantee that the device name and the device major number
     is the same as in Cygwin. */
  char dev_name = '`';

  /* Open \Device object directory. */
  wchar_t wpath[MAX_PATH] = L"\\Device";
  UNICODE_STRING upath = {14, 16, wpath};

  InitializeObjectAttributes (&attr, &upath, OBJ_CASE_INSENSITIVE, NULL, NULL);
  status = NtOpenDirectoryObject (&dirhdl, DIRECTORY_QUERY, &attr);
  if (!NT_SUCCESS (status))
    {
      fprintf (stderr, "NtOpenDirectoryObject, status 0x%08x", status);
      return 1;
    }

  /* Traverse \Device directory ... */
  PDIRECTORY_BASIC_INFORMATION dbi_buf = (PDIRECTORY_BASIC_INFORMATION)
					 alloca (65536);
  PDIRECTORY_BASIC_INFORMATION dbi = dbi_buf;
  BOOLEAN restart = TRUE;
  bool got_one = false;
  bool last_run = false;
  ULONG context = 0;
  ULONG bytes_read = 0;
  while (!last_run)
    {
      status = NtQueryDirectoryObject (dirhdl, dbi_buf, 65536, FALSE, restart,
				       &context, &bytes_read);
      if (!NT_SUCCESS (status))
	{
	  fprintf (stderr, "NtQueryDirectoryObject(), status 0x%08x", status);
	  return 1;
	}
      if (status != STATUS_MORE_ENTRIES)
	last_run = true;
      restart = FALSE;
      printf ("bytes_read = %lu, context = %lu, status = 0x%08x\n",
	      (unsigned long) bytes_read, (unsigned long) context,
	      status);
      for (dbi = dbi_buf; dbi->ObjectName.Length > 0; dbi++)
	{
	  HANDLE devhdl;
	  PARTITION_INFORMATION_EX *pix = NULL;
	  PARTITION_INFORMATION *pi = NULL;
	  DWORD bytes_read;
	  DWORD part_cnt = 0;
	  unsigned long long size;

	  /* ... and check for a "Harddisk[0-9]*" entry. */
	  if (dbi->ObjectName.Length < 9 * sizeof (WCHAR)
	      || wcsncasecmp (dbi->ObjectName.Buffer, L"Harddisk", 8) != 0
	      || !iswdigit (dbi->ObjectName.Buffer[8]))
	    continue;
	  /* Got it.  Now construct the path to the entire disk, which is
	     "\\Device\\HarddiskX\\Partition0", and open the disk with
	     minimum permissions. */
	  //unsigned long drive_num = wcstoul (dbi->ObjectName.Buffer + 8, NULL, 10);
	  wcscpy (wpath, dbi->ObjectName.Buffer);
	  PWCHAR wpart = wpath + dbi->ObjectName.Length / sizeof (WCHAR);
	  wcpcpy (wpart, L"\\Partition0");
	  upath.Length = dbi->ObjectName.Length + 22;
	  upath.MaximumLength = upath.Length + sizeof (WCHAR);
	  InitializeObjectAttributes (&attr, &upath, OBJ_CASE_INSENSITIVE,
				      dirhdl, NULL);
	  status = NtOpenFile (&devhdl, READ_CONTROL, &attr, &io,
			       FILE_SHARE_VALID_FLAGS, 0);
	  if (!NT_SUCCESS (status))
	    {
	      fprintf (stderr, "NtOpenFile(%ls), status 0x%08x",
		       upath.Buffer, status);
	      continue;
	    }
	  if (!got_one)
	    {
	      printf ("major minor  #blocks  name   win-mounts\n\n");
	      got_one = true;
	    }
	  /* Fetch partition info for the entire disk to get its size. */
	  if (DeviceIoControl (devhdl, IOCTL_DISK_GET_PARTITION_INFO_EX, NULL, 0,
			       ioctl_buf, NT_MAX_PATH, &bytes_read, NULL))
	    {
	      pix = (PARTITION_INFORMATION_EX *) ioctl_buf;
	      size = pix->PartitionLength.QuadPart;
	    }
	  else if (DeviceIoControl (devhdl, IOCTL_DISK_GET_PARTITION_INFO, NULL, 0,
				    ioctl_buf, NT_MAX_PATH, &bytes_read, NULL))
	    {
	      pi = (PARTITION_INFORMATION *) ioctl_buf;
	      size = pi->PartitionLength.QuadPart;
	    }
	  else
	    {
	      fprintf (stderr, "DeviceIoControl (%ls, "
		       "IOCTL_DISK_GET_PARTITION_INFO{_EX}) %lu",
		       upath.Buffer, (unsigned long) GetLastError ());
	      size = 0;
	    }
	  //device dev (drive_num, 0);
	  ++dev_name;
	  printf ("%5d %5d %9llu sd%c (%lu, %ls)\n",
		  8, (dev_name - 'a') * 16, size >> 10, dev_name,
		  (unsigned long) context, dbi->ObjectName.Buffer);

	  /* Fetch drive layout info to get size of all partitions on the disk. */
	  if (DeviceIoControl (devhdl, IOCTL_DISK_GET_DRIVE_LAYOUT_EX,
			       NULL, 0, ioctl_buf, NT_MAX_PATH, &bytes_read, NULL))
	    {
	      PDRIVE_LAYOUT_INFORMATION_EX pdlix = (PDRIVE_LAYOUT_INFORMATION_EX)
						   ioctl_buf;
	      part_cnt = pdlix->PartitionCount;
	      pix = pdlix->PartitionEntry;
	    }
	  else if (DeviceIoControl (devhdl, IOCTL_DISK_GET_DRIVE_LAYOUT,
				    NULL, 0, ioctl_buf, NT_MAX_PATH, &bytes_read, NULL))
	    {
	      PDRIVE_LAYOUT_INFORMATION pdli = (PDRIVE_LAYOUT_INFORMATION) ioctl_buf;
	      part_cnt = pdli->PartitionCount;
	      pi = pdli->PartitionEntry;
	    }
	  else
	    fprintf (stderr, "DeviceIoControl(%ls, "
		      "IOCTL_DISK_GET_DRIVE_LAYOUT{_EX}): %lu",
		      upath.Buffer, (unsigned long) GetLastError ());

	  /* Loop over partitions. */
	  if (pix || pi)
	    for (DWORD i = 0; i < part_cnt && i < 64; ++i)
	      {
		DWORD part_num;

		if (pix)
		  {
		    size = pix->PartitionLength.QuadPart;
		    part_num = pix->PartitionNumber;
		    ++pix;
		  }
		else
		  {
		    size = pi->PartitionLength.QuadPart;
		    part_num = pi->PartitionNumber;
		    ++pi;
		  }
		/* A partition number of 0 denotes an extended partition or a
		   filler entry as described in fhandler_dev_floppy::lock_partition.
		   Just skip. */
		if (part_num == 0)
		  continue;

		//device dev (drive_num, part_num);
		printf ("%5d %5d %9llu sd%c%d   ",
			8, (dev_name - 'a') * 16 + part_num, size >> 10,
			dev_name, part_num);

		/* Check if the partition is mounted in Windows and, if so,
		   print the mount point list. */
		swprintf (fpath, sizeof fpath,
			  L"\\\\?\\GLOBALROOT\\Device\\%ls\\Partition%u\\",
			  dbi->ObjectName.Buffer, part_num);
		if (GetVolumeNameForVolumeMountPointW (fpath, gpath, MAX_PATH)
		    && GetVolumePathNamesForVolumeNameW (gpath, mp_buf,
							 NT_MAX_PATH, &len))
		  {
		    for (PWCHAR p = mp_buf; *p; p = wcschr (p, L'\0') + 1)
		      printf (" %ls", p);
		  }
		puts ("");
	      }
	  NtClose (devhdl);
	}
    }
  NtClose (dirhdl);

  if (!got_one)
    return 1;

  return 0;
}

--1YWYokcy67A1TOBd
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline


-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple

--1YWYokcy67A1TOBd--