X-Recipient: archive-cygwin AT delorie DOT com DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org E6CD63959E66 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com; s=default; t=1629384553; bh=L0S6pp3ijR2ecc3Vs9Yg58s0MSGke5tNJVwmeLX1LLs=; h=Date:To:Subject:References:In-Reply-To:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=bVsSEdoizTqF47EQpSP2apFOYjLS5Xvz6o4pVgoZDDJai9IS0kIgmBIsLGp1QdvCk 9iU4ZajBHJ8hhh9k3dTJRRyFa/1b5Mg30+4tyZddYoCMSNswUQL6vEZgM8DYmsz1+j IEUr+gmvuSQGstozfeRd13QAYTTe9RrzgekKyUjU= X-Original-To: cygwin AT cygwin DOT com Delivered-To: cygwin AT cygwin DOT com DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 51032385840A Date: Thu, 19 Aug 2021 16:48:30 +0200 To: cygwin AT cygwin DOT com Subject: Re: Duplicates in /proc/partitions Message-ID: Mail-Followup-To: cygwin AT cygwin DOT com References: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="1YWYokcy67A1TOBd" Content-Disposition: inline In-Reply-To: X-Provags-ID: V03:K1:X6eCcUKDhiR2S7mdYfKBkilICxkIs1GJFlwJEZnhLLQHqHxhWaT Z5Qhrsq1eDbsSDlh3EDZY8wyleG2SPc1FELCfZNwmSAl0hAXinmszHrX5waMrmD0XhhQ+uX rtOJObMm9sH9ZgmLpq3nyk1z58qPJTVGjpiKwoFsnDksl8lxbKZZGp6wX+f83CjWpQZLVBe wYqmmLnca0SOkm4Ou+Ujw== X-UI-Out-Filterresults: notjunk:1;V03:K0:Fj3Zro6vUOM=:tjmyQOBcCbjaAiTmsezUvz igL0MEAcId0rh8WNI2Sgd3hoU9U++VdAOD0ZeOEMLbWQ6QXwbZHWzN7fiIO5LJK2yIuu568VD yr3fcOY2QGk7P5wKgWvcMLrWpwcDRCo6ieHXRuf9bgXaJs8FxpGSUMkKKWQU8ieV7vgshbU7v c99sCmI+jpl3CsY1ZuhLZnUQGas12LHpCvNLs6ci0kDkY2E1jDuQ6OYUL+ZD+zInLtI4ZHIey sgFBOFMR0fdCJos5HQOqj1bVtB/bSo81XTDNfqc7a/EX70+TfxGjr7hDMg7JtVuQ77Ad9KXAo QQkM9TGZCJHWqX9tRs0Bu7yplpx3n/5huFDn63EwPxAkZEDRMsROtYxEqyVWFiF2P1p/cCw5H FQQw3cgahQqberTWwVxsEMqH5jbJrY/cApy3R3qS6LUI7ic1oE2zISmYkDNmtzpjS2u9vYKK2 miQiExs4Rw+ptYskmdAsPFWRz5s7LwWfaqJXFMZmsBSpvJW4jpXGbqysLN0oOKhGWUzucGnlK F3KGeOm2EneXU3mHsGZJ7kUASlgPoHjuGNpm+hmlYp5olVqczgyN0Uoiizjn0fkvXBpmeyYKE VpAgSuLtB4yk4MNSNbGsawNRdK+bgJON8cs/VOeEm/Iau0kMlPRTj6YV1sLUmkWw7MVCB4j2K uPOvQpR8XrfoTiMMBR+x+N3sixyl8CwjGDE3sXW8Q+8ObysphCUvLtVNCMTcnjP1OWBtnUBy8 UwymE4CJIxPJEwcUZk5k8+OrKASC1aZHMPuu6LJHikZuOm+SJ7luYfuLzok8lJtyafsPkewaT o/2FTeD2Yth8Z2TOfWHbGE05lN1kHe0O77kwdo+6Jf3ImPYZVcgrLlciXSIXnmOhJQOgFf1v3 vm2xkLRMD9RziPZvaNtQ== X-Spam-Status: No, score=-100.2 required=5.0 tests=BAYES_00, GOOD_FROM_CORINNA_CYGWIN, KAM_DMARC_NONE, KAM_DMARC_STATUS, RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_NEUTRAL, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: cygwin AT cygwin DOT com X-Mailman-Version: 2.1.29 Precedence: list List-Id: General Cygwin discussions and problem reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Corinna Vinschen via Cygwin Reply-To: cygwin AT cygwin DOT com Cc: Corinna Vinschen Errors-To: cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com Sender: "Cygwin" --1YWYokcy67A1TOBd Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Hi Anton, On Aug 19 13:26, Corinna Vinschen via Cygwin wrote: > On Aug 19 12:03, Corinna Vinschen via Cygwin wrote: > > On Aug 18 18:36, Lavrentiev, Anton (NIH/NLM/NCBI) [C] via Cygwin wrote: > > > > And I just confirmed that if I print the context right before NtOpenFile(), and just do > > > > the "continue", > > > > > > But then I also tried this: I removed the "continue" before "NtOpenFile()" and allowed it to proceed, > > > but I moved NtClose(devhdl) right after the "printf" statement (that we were tweaking), and > > > inserted the "continue" there, so it does not proceed with enumerating of the partitions. > > > And so again, I got the output that matches my disks without the duplication. So this actually > > > exonerates NtOpenFile() :-) > > > [...] > > > > Unfortunately I can't reproduce the issue. I created a couple of > > virtual drives and added them to my W10 and my W7 VM, but to no avail. > > Never mind. I now can reproduce the problem at will. The trick is to > create objects inside the \Device directory while the loop iterating > over the directory is running. > > As shitty as it is, a NtOpenDirectoryObject/NtQueryDirectoryObject/NtClose > loop is not working atomically. If a new object is inserted into the > dir preceeding the currently handled entry (which, on a reliable system > should *never* occur), the entry is moved by one, and the next > NtQueryDirectoryObject call returns the same object again. > > Windows never disappoints when one is looking for really ugly problems. > > The good news is that NtQueryDirectoryObject is also capable to take a > big buffer and return as much entries as fit in in the buffer. I'll > give it a try now in the vain hope that this way to call > NtQueryDirectoryObject returns reliable, atomic-like results... I tested this on both of my VMs (W7, W10) and the result looks promising. I implemented this method in the Cygwin DLL locally, for all places utilizing the NtQueryDirectoryObject call, except for one call where this isn't possible, unfortunately. That's readdir on /proc/sys directories, which is supposed to return one entry on each invocation only and is thus stateful. Funny enough this problem may also explain why ps(1) output sometimes showed some Cygwin processes twice. Same method, same problem. Anyway, would you mind to test the below new proc_partition.c as well as the latest snapshot I just uploaded to https://cygwin.com/snapshots/? Thanks, Corinna --1YWYokcy67A1TOBd Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="proc_partition.c" #include #include #include #include #include #include #include NTSTATUS NTAPI NtOpenDirectoryObject (PHANDLE, ACCESS_MASK, POBJECT_ATTRIBUTES); NTSTATUS NTAPI NtQueryDirectoryObject (HANDLE, PVOID, ULONG, BOOLEAN, BOOLEAN, PULONG, PULONG); typedef struct _DIRECTORY_BASIC_INFORMATION { UNICODE_STRING ObjectName; UNICODE_STRING ObjectTypeName; } DIRECTORY_BASIC_INFORMATION, *PDIRECTORY_BASIC_INFORMATION; #define DIRECTORY_QUERY 1 #define NT_MAX_PATH 32768 char buf[NT_MAX_PATH]; char ioctl_buf[NT_MAX_PATH]; WCHAR mp_buf[NT_MAX_PATH]; typedef enum { false, true } bool; int main () { OBJECT_ATTRIBUTES attr; IO_STATUS_BLOCK io; NTSTATUS status; HANDLE dirhdl; WCHAR fpath[MAX_PATH]; WCHAR gpath[MAX_PATH]; DWORD len; /* Note that device ids and names are just faked here, not using the internal Cygwin code to generate device major/minor numbers and device names. There's *no* guarantee that the device name and the device major number is the same as in Cygwin. */ char dev_name = '`'; /* Open \Device object directory. */ wchar_t wpath[MAX_PATH] = L"\\Device"; UNICODE_STRING upath = {14, 16, wpath}; InitializeObjectAttributes (&attr, &upath, OBJ_CASE_INSENSITIVE, NULL, NULL); status = NtOpenDirectoryObject (&dirhdl, DIRECTORY_QUERY, &attr); if (!NT_SUCCESS (status)) { fprintf (stderr, "NtOpenDirectoryObject, status 0x%08x", status); return 1; } /* Traverse \Device directory ... */ PDIRECTORY_BASIC_INFORMATION dbi_buf = (PDIRECTORY_BASIC_INFORMATION) alloca (65536); PDIRECTORY_BASIC_INFORMATION dbi = dbi_buf; BOOLEAN restart = TRUE; bool got_one = false; bool last_run = false; ULONG context = 0; ULONG bytes_read = 0; while (!last_run) { status = NtQueryDirectoryObject (dirhdl, dbi_buf, 65536, FALSE, restart, &context, &bytes_read); if (!NT_SUCCESS (status)) { fprintf (stderr, "NtQueryDirectoryObject(), status 0x%08x", status); return 1; } if (status != STATUS_MORE_ENTRIES) last_run = true; restart = FALSE; printf ("bytes_read = %lu, context = %lu, status = 0x%08x\n", (unsigned long) bytes_read, (unsigned long) context, status); for (dbi = dbi_buf; dbi->ObjectName.Length > 0; dbi++) { HANDLE devhdl; PARTITION_INFORMATION_EX *pix = NULL; PARTITION_INFORMATION *pi = NULL; DWORD bytes_read; DWORD part_cnt = 0; unsigned long long size; /* ... and check for a "Harddisk[0-9]*" entry. */ if (dbi->ObjectName.Length < 9 * sizeof (WCHAR) || wcsncasecmp (dbi->ObjectName.Buffer, L"Harddisk", 8) != 0 || !iswdigit (dbi->ObjectName.Buffer[8])) continue; /* Got it. Now construct the path to the entire disk, which is "\\Device\\HarddiskX\\Partition0", and open the disk with minimum permissions. */ //unsigned long drive_num = wcstoul (dbi->ObjectName.Buffer + 8, NULL, 10); wcscpy (wpath, dbi->ObjectName.Buffer); PWCHAR wpart = wpath + dbi->ObjectName.Length / sizeof (WCHAR); wcpcpy (wpart, L"\\Partition0"); upath.Length = dbi->ObjectName.Length + 22; upath.MaximumLength = upath.Length + sizeof (WCHAR); InitializeObjectAttributes (&attr, &upath, OBJ_CASE_INSENSITIVE, dirhdl, NULL); status = NtOpenFile (&devhdl, READ_CONTROL, &attr, &io, FILE_SHARE_VALID_FLAGS, 0); if (!NT_SUCCESS (status)) { fprintf (stderr, "NtOpenFile(%ls), status 0x%08x", upath.Buffer, status); continue; } if (!got_one) { printf ("major minor #blocks name win-mounts\n\n"); got_one = true; } /* Fetch partition info for the entire disk to get its size. */ if (DeviceIoControl (devhdl, IOCTL_DISK_GET_PARTITION_INFO_EX, NULL, 0, ioctl_buf, NT_MAX_PATH, &bytes_read, NULL)) { pix = (PARTITION_INFORMATION_EX *) ioctl_buf; size = pix->PartitionLength.QuadPart; } else if (DeviceIoControl (devhdl, IOCTL_DISK_GET_PARTITION_INFO, NULL, 0, ioctl_buf, NT_MAX_PATH, &bytes_read, NULL)) { pi = (PARTITION_INFORMATION *) ioctl_buf; size = pi->PartitionLength.QuadPart; } else { fprintf (stderr, "DeviceIoControl (%ls, " "IOCTL_DISK_GET_PARTITION_INFO{_EX}) %lu", upath.Buffer, (unsigned long) GetLastError ()); size = 0; } //device dev (drive_num, 0); ++dev_name; printf ("%5d %5d %9llu sd%c (%lu, %ls)\n", 8, (dev_name - 'a') * 16, size >> 10, dev_name, (unsigned long) context, dbi->ObjectName.Buffer); /* Fetch drive layout info to get size of all partitions on the disk. */ if (DeviceIoControl (devhdl, IOCTL_DISK_GET_DRIVE_LAYOUT_EX, NULL, 0, ioctl_buf, NT_MAX_PATH, &bytes_read, NULL)) { PDRIVE_LAYOUT_INFORMATION_EX pdlix = (PDRIVE_LAYOUT_INFORMATION_EX) ioctl_buf; part_cnt = pdlix->PartitionCount; pix = pdlix->PartitionEntry; } else if (DeviceIoControl (devhdl, IOCTL_DISK_GET_DRIVE_LAYOUT, NULL, 0, ioctl_buf, NT_MAX_PATH, &bytes_read, NULL)) { PDRIVE_LAYOUT_INFORMATION pdli = (PDRIVE_LAYOUT_INFORMATION) ioctl_buf; part_cnt = pdli->PartitionCount; pi = pdli->PartitionEntry; } else fprintf (stderr, "DeviceIoControl(%ls, " "IOCTL_DISK_GET_DRIVE_LAYOUT{_EX}): %lu", upath.Buffer, (unsigned long) GetLastError ()); /* Loop over partitions. */ if (pix || pi) for (DWORD i = 0; i < part_cnt && i < 64; ++i) { DWORD part_num; if (pix) { size = pix->PartitionLength.QuadPart; part_num = pix->PartitionNumber; ++pix; } else { size = pi->PartitionLength.QuadPart; part_num = pi->PartitionNumber; ++pi; } /* A partition number of 0 denotes an extended partition or a filler entry as described in fhandler_dev_floppy::lock_partition. Just skip. */ if (part_num == 0) continue; //device dev (drive_num, part_num); printf ("%5d %5d %9llu sd%c%d ", 8, (dev_name - 'a') * 16 + part_num, size >> 10, dev_name, part_num); /* Check if the partition is mounted in Windows and, if so, print the mount point list. */ swprintf (fpath, sizeof fpath, L"\\\\?\\GLOBALROOT\\Device\\%ls\\Partition%u\\", dbi->ObjectName.Buffer, part_num); if (GetVolumeNameForVolumeMountPointW (fpath, gpath, MAX_PATH) && GetVolumePathNamesForVolumeNameW (gpath, mp_buf, NT_MAX_PATH, &len)) { for (PWCHAR p = mp_buf; *p; p = wcschr (p, L'\0') + 1) printf (" %ls", p); } puts (""); } NtClose (devhdl); } } NtClose (dirhdl); if (!got_one) return 1; return 0; } --1YWYokcy67A1TOBd Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation: https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple --1YWYokcy67A1TOBd--