Mail Archives: cygwin/2002/04/29/15:51:19
A few notes:
Using rsync does not help the following problem: I have four favorite
mirrors, plus a "private" setup-compatible website from my friend Bob.
setup.exe will show me a merged view of the contents of all of those
sites, insuring that I get the latest version of a package wherever it
is found. By merging in the "private" site, setup enables me to get my
friend Bob's latest version of "King of Frobozz, Tyrant" game using the
same setup tool/view, along with the latest apache package or binutils
package from the "official" cygwin mirrors.
rsync can't merge.
I could rsync each of the four mirrors down to my machine into separate
directories (assuming the mirrors all had anonymous rsync servers), and
rsync Bob's private website (assuming his webhost provided rsync --
hah!). But even then, setup can't merge multiple local directories, at
least not right now.
So, we still need some variant of setup.exe that can merge multiple
setup-compatible sites, and "mirror" only the latest stuff into a single
local "release" tree / setup.ini.
Also, rsync does some stuff we don't really care about: since most of
the files are compressed, rsync's sophisticated "download changed
chunks" algorithm is pretty useless. In compressed files, the first
byte that is different in the uncompressed file will cause a cascade of
changes throughout the remainder of the compressed file. So: 1 byte
diff in uncompressed file == entire compressed file is different.
OTOH, since we typically (never?) upload a modified version of a given
file USING THE SAME NAME, we don't really care too much about this.
"Our" mirroring tool can just use the md5 hashes on the mirror and
locally to determine the rare instance when a given file has had its
contents changed -- but then we'd download the ENTIRE file, not "changed
chunks". (However, if this ever happens, isn't it a sign that something
went wrong somewhere?)
Also, there's really no need to use rsync's compression engine, since
most of the data to be transferred has already been compressed.
Finally, the local "setup.ini" will never(?) match the setup.ini at any
one of the mirrors or private sites; it will be a merger of them. So,
rsync can't do that.
So, if someone cares enough to do the work, there is still a need for
the "additional tool" that uses the setup.exe codebase to do
"cygwin-specific distributed mirroring". See my earlier rant.
But, the rsync method should enable folks with slow connections to limp
along for a while...without trying to turn setup.exe ITSELF into a
mirror tool.
--Chuck
--
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Bug reporting: http://cygwin.com/bugs.html
Documentation: http://cygwin.com/docs.html
FAQ: http://cygwin.com/faq/
- Raw text -