Mail Archives: cygwin/2010/03/30/19:08:54
I've spent some time trying to diagnose the previously reported
git-fetch failures running on cygwin 1.7-x (up to and including 1.7.2).
The failure manifests itself as an incomplete transfer of data using
git's native transfer protocol, and happens either talking directly to a
git-daemon via sockets or using a pipe tunneled through an ssh connection.
git started life as a large number of shell scripts wrapping a few
compiled programs, and retains the architecture of multiple processes
communicating via pipes even though most of the shell scripts were
converted to C. For the given problem, the situation is as follows:
The main git process either opens ssh as a subprocess to execute
'git-upload-pack' on the server, or opens a socket to talk to a
git-daemon on the server (running under inetd). Either way, the parent
git process now has two file handles for communicating to the server,
and redirects these to subprocesses. The data sent from server back to
the client is multiplexed as packets of up to 64 KBytes, each packet
having a header giving a channel number and the number of bytes to
follow. Channel 1 is for the data, channel 2 is typically stderr and is
used as such in this case.
Next, git starts a sideband demux process, connecting its stdin to
stdout from the server process above. This demux process prints the
information channel (channel 2) to stderr, and the data channel is
written back to the demux process's stdout.
Next, git starts an index-pack process with stdin connected to the demux
process's stdout.
All of the above connections and i/o processing are done using
bog-standard fork, pipe, open, read, write, and associated file
commands. Git has no trouble opening all of the processes and pipes
under Cygwin 1.7.2. However, in general before the data transfer is
complete the demux process issues an error upon getting an incomplete
packet (fewer bytes transmitted than declared in the header), and
debugging shows the error code of the read() is ENOENT, indicating the
pipe or socket has been terminated early. The index-pack process also
issues an error due to receipt of an incomplete pack.
I am able to get this error very reliably trying to clone a repository
about 50 MBytes in size across my office network (100-base T). The error
shows up randomly but in the range 2% to 10% of the total repository
having transferred. The same repository clones just fine when connecting
to the same server using VPN over a relatively slow internet connection.
Also, the same git executable on the same computer works flawlessly a
Cygwin 1.5 installation.
I'm guessing the error is in the Cygwin dll, and a further guess is that
it is triggered by hitting buffer limits on pipes between the several
processes running inside of git as the error does not occur using when
the transfer rate is slow.
I'm not sure where to continue investigating, but it is quite obvious
the problem is not in git itself (works fine on cygwin 1.5 not to
mention Linux) or in ssh (the failure occurs using the direct protocol
without ssh). I'm not familiar with the internals of the dll, but I'm
willing to help dig if someone can give me some hints on where to look.
Mark Levedahl
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
- Raw text -