delorie.com/archives/browse.cgi   search  
Mail Archives: cygwin/2016/06/19/07:20:51

X-Recipient: archive-cygwin AT delorie DOT com
DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
:list-unsubscribe:list-subscribe:list-archive:list-post
:list-help:sender:subject:to:references:from:message-id:date
:mime-version:in-reply-to:content-type
:content-transfer-encoding; q=dns; s=default; b=YOOradtPeNgMBZQD
Y9tTsB7+pRqFIlJcyEYVJK/6G+QSC1qbv021fMan2Mz0Nav1v6wKrpQ9kIL7Aio/
VPZ63A4EUVHzglcZGI8ychlcT+NXxSlMi8qAc6vnl7nZPgeQv9gmnMi7QaCag7Ze
5owv0v+spPsi6hI5tLDQWDd6+i8=
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id
:list-unsubscribe:list-subscribe:list-archive:list-post
:list-help:sender:subject:to:references:from:message-id:date
:mime-version:in-reply-to:content-type
:content-transfer-encoding; s=default; bh=KlxnlZ7JO3muVdEvuUDoUG
ihJ7w=; b=Ib4FQMv3aCmLKy84mRhE6JvFK2tNZDlUpEfOG1eF6wfhkJCJr5+ySb
kqEKSCWjtkaAxhkbCDmUbIxhY+nsYMpJMcI49XVOXM/b+Cm8NOw1dMxt8h1VKIf6
ueyCZBcczVb0WFeCvoLYp/JL9m2Z3x4wkHVFgY0Vn6ldOrwVEO8Gw=
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id: <cygwin.cygwin.com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com
Authentication-Results: sourceware.org; auth=none
X-Virus-Found: No
X-Spam-SWARE-Status: No, score=-1.7 required=5.0 tests=AWL,BAYES_00,FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,SPF_PASS autolearn=ham version=3.3.2 spammy=narrow, hallo, guest, lnk
X-HELO: mout.gmx.net
Subject: Re: FUSE for Cygwin
To: cygwin AT cygwin DOT com
References: <D389ABB9.931F%billziss AT navimatics DOT com>
From: Herbert Stocker <hersto AT gmx DOT de>
Message-ID: <57667FEF.5070801@gmx.de>
Date: Sun, 19 Jun 2016 13:20:15 +0200
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:38.0) Gecko/20100101 Thunderbird/38.1.0
MIME-Version: 1.0
In-Reply-To: <D389ABB9.931F%billziss@navimatics.com>
X-UI-Out-Filterresults: notjunk:1;V01:K0:z9JNrvNPQp0=:I2w7DNB66WcXQ5lPVH6w15 oRAswwTOwDNYgfWoWk7vOkmfuq/CoF8gWgFY8XoDXsZTr2RYDXvrQXKEJg0pjreBtFkgdqq7M dyYuzpNzSXn2Qr35MUpOeq/SWnX/YkhlAg1KVuuiCZ0T4wU9Ah2NcxXTx0259dalv50KRv5H1 +g2b7SmpE82GDSV3VJVtJRSVpCzmi5tWBgKnsovMdUfcP7MWM9lmfLHMs7m/dqY/esBxE74wL dRtydwgXkYRygjUOSKBiNAvIx6dgNBKKWcWOKi2e+Ml1yKZsaqbj05ce9LDk0KZXHis1Yw2mx QpZTjg6HCVrm7NN9JvO/sIYq0msvxkHNdD9PXossKhIdVQXKe2Fk70Zbw4RAHuKPEtikE/k/3 cAFE9rne95xDfhYt3gfyv1JvyGSEG2GTYPiSJkpGVUJdqLvGWwezGL1HEJo86jAc/BFXoeQ3H 3LuNVGU7OM9lgQOdpRUntHQfN9o5dWVjlRSK9S0M35D1aV72cmN5HKnV6CX8dXmowYqoJEpmZ XD4KyiyO46+Gd60R84hZvVZMc/8XiJvl/J/Z6jlZqVA5nuOEBCbUV57ZnaDUsFB/m0ebxVCoQ WPwZQBxTWO59OgSojsOYcPMBYriMRsGkjjc5/X5zt0rbc7AwwDaSqSK2dKuaQnKQ8fXgpd/Fv w0U5GkRMeiJFJsFR+ZJ6vzyHK5v2mZ7F8UrX/sXkY7+8UlxdRDfEiYs+2FGGDviRnAE+2MjsJ 995YardxLleveugZoqlOQEZFT4RnTlIXfoS7xo8exJgJIAyIus30MKjI+EjfczwBmnNrPHTm9 dZ3akTo
X-IsSubscribed: yes

Hi Bill,

this is now my proposal of an alternative mode for WinFsp to support
Cygwin based FUSE file systems. I'll call it mode (4).
It's actually my initial idea that i have in mind for some time but
did not propose to implement because i have some constraints that
prevent me from completing it.

Maybe i should first note that i'm not part of Cygwin, i'm just a
user of Cygwin who is happy to have Cygwin. However i know how to
code.

To repeat your 3 modes of operation (in my words):
(1) WinFsp's own API. The file system is a Windows process that
     can use all file system features that Windows (and WinFsp)
     provides.

(2) A FUE API for Win32. The file system is a native Windows process
     but receives requests as defined in fuse.h . It eases porting
     for the file system and does not require Cygwin be installed.

(3) The file system is a Cygwin process and thus can use all POSIX
     features of Cygwin. It needs no porting or only little.

What i don't like on (3) is that when a Cygwin process accesses the
FUSE file system there are two Cygwin processes whose communication
is translated from Posix to Win32 and then back (which is done again
for the response).

You sketched this as follows:
> Let's examine the lifetime of a call to creat(). Suppose a Cygwin process
> does creat("/cygdrive/z/foo*bar"). In the following OP is the "originating
> process", CW is the "Cygwin runtime", NT is NTOS, WD is the "WinFsp FSD",
> WL is the "WinFsp DLL", FL is the "FUSE layer", and FS is the "user mode
> FUSE file system".
>
> OP: creat("/cygdrive/z/foo*bar")
> CW: NtCreateFile(L"<DEVICE>\\foo\xf02abar")     <--- Cygwin translation
> NT: IRP_MJ_CREATE L"\\foo\xf02abar"
> WD: FspFsctlTransactCreateKind L"\\foo\xf02abar"
> WL: FSP_FILE_SYSTEM_INTERFACE::Create L"\\foo\xf02abar"
> FL: fuse_operations::create "/foo*bar"          <--- WinFsp/FUSE
> translation
> FS: somehow satisfies fuse_operations::create
> [snip return path]


So my proposal is basically this:

(4) The file system is a Cygwin process and Cygwin is extended to
     pass file system requests to that process if they fall into the
     respective path (where the file system is mounted.)

     This way the file system is exported to Cygwin only. But another
     tool exports this to the Win32 world.

Initially i thought for the second part we could use that part of
VirtualBox Guest Additions that makes the host file system available
in a windows guest. But now we have WinFsp for this...

For the first part of (4),
Cygwin does have a place that translates Unix paths to Win32 paths.
This part follows the settings in /etc/fstab so that arbitrary Win32
paths can be "mounted" to arbitrary Posix paths with settings applied.
This could be extended so that processes that link to a Cygwin version
of libfuse can register for certain paths.

Cygwin does not have a "kernel", but a Cygwin1.dll which is loaded
into every Cygwin process. On places where it does path lookups or
interferes with files (read, write, extended attributes, etc) it would
require some means to communicate with the file system process and
to wait for the response.

This would make Cygwin processes accessing the file system 1st class
citicens while Windows processes wouldn't lose compared to mode (3).
In mode (3) Cygwin processes would be 2nd class, as i'll explain below.

But the drawback of my approach is that it requires to reimplement
FUSE for Cygwin. If i could, i would volunteer to work on it. i like
Cygwin and i like FUSE, so i sure would like to work on that.

Mode (4) would avoid the following issues coming from the double
translation of mode (3):

A) Besides the double double translations of every request, there
    is also the need for four Kernel/Usermode transitions, which take
    their time.

B) We could have hardlinks for the Cygwin world only, and you have no
    need to implement them in FSD or the DLL. Though you could add that
    feature later not not at all.

    (Or would that be an issue e.g. with oplocs?)

C) i guess this way we could support fuse with all other languages
    besides C/C++.

D) Symlinks won't go through reparse points.
    You'd have no need to implement that or to follow Cygwin should
    Cygwin improve its way of translation.
    BTW, are code pages involved there?

E) Same for uid/gid to SID mapping.
    No need to implement or follow Cygwin.
    And how about the case where a uid/gid has no correspoinding SID?
    Can this happen?

F) Pipes:
 > [Quick experiment:
 >
 > $ mkfifo foo; cmd /c dir 'foo*' | grep foo; rm foo
 > 06/16/2016  11:02 PM               130 foo.lnk
 > ]
 >
 > Ok, so they are shortcuts. Naturally they are supported.

    I think they are not.
    The mkfifo system call will have Cygwin create a .lnk file and
    WinFsp will forward it as such to the file system process. The
    sytem calls readdir or open will then have the file system
    process tell WinFsp that there is a .lnk file and Cygwin will
    translate this back to a fifo, so in this sense it does work.

    But the file system will see a file (with name *.lnk) where it
    should see a pipe (mknod call with 'mode' set to S_IFIFO).
    IMHO one could say this is a break of the FUSE API.

    Practically it will break:
     - file systems that special-treat pipe files (or .lnk files).

     - If one uses sshfs to connect to a Linux based server and
       issues the command mkfifo foo from Cygwin, the server will
       end up with a .lnk file instead of a pipe special file.

     - Imagine something like mysqlfs, which stores the stuff in a
       database. When you run SQL statements to analyze the data
       in the file system, you won't see the pipes as such. Or if
       you open the file system from Linux you'll see the .lnk
       files.

    To fix this with mode (3) you'd have to recognize these .lnk
    files and forward them to the file system as pipes, and you'd
    have to generate .lnk files on the fly when the file system
    says there is a pipe file (e.g. on readdir).

G) Case sensitivity.
    Windows is usually case insensitive but case
    preserving on file names. But there is a registry setting that
    allows one to activate case sensitivity. Then programs that use
    the Win32 API in a certain way can have case sensitivity.
    Cygwin has an option to do so, too. It is an option in the
    /etc/fstab file. (And i use it for all drives.)

    With this you can say   touch a A   in the Cygwin command line
    and you'll see two files in Windows Explorer. Double clicking one
    of them will always open the first file though.

    i guess FUSE file systems ported to mode (3) or (2) will expect
    file names to be case sensitive.
 
https://cygwin.com/cygwin-ug-net/using-specialnames.html#pathnames-casesensitive

    What happens if you use sshfs to connect to a Linux box and then
    do the following in cmd.exe:
      # echo Hallo >Foo.txt
      # type foo.Txt

H) Currently you have either "disk style" file system that you mount
    as a drive letter or a "network style" file sytem that you mount
    via UNC path, e.g. \\sshfs\name AT host . Drive letters are a name
    space of only 26 entries which is very narrow, but you can
    also mount them to empty directories on other drives (which must
    support reparse points i think). I don't know if that's possible
    with UNC paths too.

    That means that Cygwin processes will see them as
    //sshfs/name AT host and have to use symlinks or /etc/fstab settings
    to "mount" them elsewhere.

It looks like mode (4) is much more work because it requires FUSE
to be reimplemented/ported to Cygwin. But it's lesser work on WinFsp
and would have no issues due to double translations. It would make
Cygwin processes 1st class on FUSE file systems.

So my question is to Corinna, would it be possible and feasible
to implement this in Cygwin?


best regards,

Herbert


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

- Raw text -


  webmaster     delorie software   privacy  
  Copyright © 2019   by DJ Delorie     Updated Jul 2019