X-Recipient: archive-cygwin AT delorie DOT com DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:from:to:subject:date:message-id:content-type :content-id:content-transfer-encoding:mime-version; q=dns; s= default; b=SdTRqnHM+yeOtcBFarL11+Ilp6R8kqjbxvsh4Zr07ILxhm0eeHgZG zV3CG5jePsp77aIr8MpE0sy/NDYWJMIcp/KccjhjiLQwUPT9KRxrcq96hrOwotwO YpwwheDDAvl5lBFt68BGGSaS2yue9s9TIEm/PUQArru5SU7iLbIUWw= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:from:to:subject:date:message-id:content-type :content-id:content-transfer-encoding:mime-version; s=default; bh=qL9nG6jX4mfbCLlg2o+8lSTOWtk=; b=mQERM4gN4ghw8p4eYP3Jd/ZQKvx8 uXm+3eOH3UtNXidPfThSwr7g5wIUse7EgdAY90i/ODLpcwsM5pKSC2jvY0DgKzVA Eo0YLrrDPYbb11pCw15FWDTnFmADKS4OSLgq1hgwueBqE5/5JaPDFf7WqbC6EA96 nVCm2dz+7pKHk4U= Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner AT cygwin DOT com Mail-Followup-To: cygwin AT cygwin DOT com Delivered-To: mailing list cygwin AT cygwin DOT com Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-0.1 required=5.0 tests=AWL,BAYES_20,MIME_BASE64_BLANKS,RCVD_IN_DNSWL_NONE,SPF_HELO_PASS,SPF_PASS autolearn=ham version=3.3.2 spammy=originating, slowly, symlinks, shortcuts X-HELO: NAM01-SN1-obe.outbound.protection.outlook.com From: Bill Zissimopoulos To: "cygwin AT cygwin DOT com" Subject: FUSE, symbolic links and special files Date: Thu, 25 Aug 2016 11:46:31 +0000 Message-ID: authentication-results: spf=none (sender IP is ) smtp.mailfrom=billziss AT navimatics DOT com; x-ms-exchange-messagesentrepresentingtype: 1 x-ms-office365-filtering-correlation-id: e8b555e8-5bf3-4dac-b984-08d3ccdd7586 x-microsoft-exchange-diagnostics: 1;CY1PR07MB2199;6:LjSjr81FPt3hyAMjnw9bim7C5s3zbaR1tC9iE8Epzj05bX8vzz5xwvU8n0aLIZ5fEtkewdq+LGygQSpw7ZbEFBhMpzStclmTcwcwwpi/JBlKFjcdZyZ79oTXGPFsI5RHrzFPjcvV+2APUsahke4nK+LsVDRu5Sy7bp/zyrlp6rFmYYPdwabHIbADKdFJa97Gd4qbEStEkSdSKLePEPdj+YdijuN2uWCAc9/Q3fBRG80eP/HwR2MXTL80jvApiGd5Tnf6LH14bWQzDAZxRDnAvjMo8nJeYpYCpVrtRJqsCcuzzJz/sjJ5O8bNrW1FkkQn;5:H6ymhGCN7z5HgR/BJgZ/AaHnFeKW2WH9TDPSZLJLRSm3PNJj7q6kYsTtY8hBsyRI9KqAj7wMo6kDbRWvAq8FVqvkIFw+NZwzkdM1VY1BDBSTZExJ8sold872wlAC1Jzhhn2YGMYz5yZTswHCdcE3OA==;24:UolcsPbCpuXPMZr4C59BYL6nBmfkC7e8X19o7G+ljs206KmCWZRlmbyyb/2JjzgmtzDEBGy5qS42bKUud8SYjkLBPfx1b71gdCFmJE5ws7Y=;7:/nwKrjk0tLVedlc8FW9k5zTfR0wLphfOf/8JeTvqLSX8XhqZPuxM/VdKdyXdJvr9n7ullrmRrkSE2hyiXr4JBp7ioHGqxNV7piA8RY/IKKWleO0VtquHzHVCpFzHYRR/0gNW1qD2RXOSyzl+LG1lb9ocPowa6Rjd9CRyto4aVuLrrnTM5su3QlC0lAm5EDyytBELNmIa7tjt53Cmqn8/nvMPiCWAgPmUOuTPZ6YU/ycRwLmwb5U0HMG81w+8lqMG x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:CY1PR07MB2199; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(158342451672863)(176295241369792)(21532816269658); x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(6040176)(601004)(2401047)(5005006)(8121501046)(10201501046)(3002001)(6043046)(6042046);SRVR:CY1PR07MB2199;BCL:0;PCL:0;RULEID:;SRVR:CY1PR07MB2199; x-forefront-prvs: 0045236D47 x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(979002)(6009001)(7916002)(199003)(189002)(377454003)(3905003)(107886002)(110136002)(97736004)(68736007)(5890100001)(189998001)(92566002)(2501003)(11100500001)(5640700001)(66066001)(36756003)(106356001)(105586002)(305945005)(99286002)(2351001)(122556002)(87936001)(106116001)(77096005)(54356999)(2900100001)(101416001)(86362001)(10400500002)(15975445007)(50986999)(450100001)(3280700002)(5660300001)(229853001)(19580395003)(102836003)(5002640100001)(8936002)(8676002)(2906002)(3846002)(7846002)(1730700003)(7736002)(81156014)(586003)(81166006)(3660700001)(6116002)(94096001)(969003)(989001)(999001)(1009001)(1019001);DIR:OUT;SFP:1102;SCL:1;SRVR:CY1PR07MB2199;H:CY1PR07MB2199.namprd07.prod.outlook.com;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; received-spf: None (protection.outlook.com: navimatics.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="utf-8" Content-ID: <4569FD13AFEE1946AE28FC399E87DA3F AT namprd07 DOT prod DOT outlook DOT com> MIME-Version: 1.0 X-OriginatorOrg: navimatics.com X-MS-Exchange-CrossTenant-originalarrivaltime: 25 Aug 2016 11:46:31.9082 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 21071be9-4f9a-413b-89ac-8353a5d2410a X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR07MB2199 X-IsSubscribed: yes Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by delorie.com id u7PBl1kl000438 While on vacation I have been (slowly) working to add reparse point and symbolic link support for WinFsp and FUSE for Cygwin. This work is mostly complete and is currently being tested. I am writing to the Cygwin list because I want to resolve a problem that Herbert Stocker originally brought up: >F) Pipes: >> [Quick experiment: >> >> $ mkfifo foo; cmd /c dir 'foo*' | grep foo; rm foo >> 06/16/2016 11:02 PM 130 foo.lnk >> ] >> >> Ok, so they are shortcuts. Naturally they are supported. > > I think they are not. > The mkfifo system call will have Cygwin create a .lnk file and > WinFsp will forward it as such to the file system process. The > sytem calls readdir or open will then have the file system > process tell WinFsp that there is a .lnk file and Cygwin will > translate this back to a fifo, so in this sense it does work. > > But the file system will see a file (with name *.lnk) where it > should see a pipe (mknod call with 'mode' set to S_IFIFO). > IMHO one could say this is a break of the FUSE API. > > Practically it will break: > - file systems that special-treat pipe files (or .lnk files). > > - If one uses sshfs to connect to a Linux based server and > issues the command mkfifo foo from Cygwin, the server will > end up with a .lnk file instead of a pipe special file. > > - Imagine something like mysqlfs, which stores the stuff in a > database. When you run SQL statements to analyze the data > in the file system, you won't see the pipes as such. Or if > you open the file system from Linux you'll see the .lnk > files. Herbert is of course right. A .lnk file is not a FIFO to any system other than Cygwin. I have been thinking about how to solve this problem and I believe I have an answer in the form of reparse points. Reparse points can be viewed as a form of special metadata that can be attached to a file or directory. The interesting thing about reparse points is that they can have special meaning to a file system driver (NTFS/WinFsp), a filter driver (e.g. a hierarchical storage system) or even an application (Cygwin). NTFS uses reparse points to implement symbolic links and places severe limitations on them. For one, it differentiates between file and directory symlinks. It also requires the SE_CREATE_SYMBOLIC_LINK_PRIVILEGE privilege to be held to create a symbolic link account. Interestingly it does not perform any extraneous access checks on other reparse points. IMO this makes reparse points very interesting because it offers a path for WinFsp and FUSE for Cygwin to provide special file support. Turns out that Microsoft already has a solution for special files on NFS: https://msdn.microsoft.com/en-us/library/dn617178.aspx I see no reason that the same solution cannot be used for FUSE for Cygwin as well. In the following OP is the originating process, CW is the Cygwin layer, WL is the WinFsp layer and FL is the FUSE layer. OP: mkfifo("myfifo") CW: NtCreateFile, NtDeviceIoControlFile(FSCTL_SET_REPARSE_POINT) [NFS_SPECFILE_FIFO] WL: IRP_MJ_FILE_SYSTEM_CONTROL/FSCTL_SET_REPARSE_POINT [NFS_SPECFILE_FIFO] FL: fuse_operations::mknod("myfifo", S_IFIFO) Regarding symbolic link support specifically I am still undecided on whether the right thing to do is to use NTFS symbolic links (IO_REPARSE_TAG_SYMLINK) or use NFS_SPECFILE_LNK for the FUSE layer. My inclination is to support both and let the FUSE file system developer decide on their preference. Any comments welcome. Bill