X-Recipient: archive-cygwin AT delorie DOT com
X-Spam-Check-By: sourceware.org
Date: Tue, 10 Jun 2008 15:54:15 +1000 (EST)
From: Luke Kendall <luke DOT kendall AT cisra DOT canon DOT com DOT au>
Subject: Checking a mirror is complete (Was: Re: diff: Resource temporarily   unavailable)
To: cygwin AT cygwin DOT com
In-Reply-To: <484E0D71.4090609@cisra.canon.com.au>
MIME-Version: 1.0
Content-Type: TEXT/plain; CHARSET=US-ASCII
Message-Id: <20080610055416.7F6E18430C@pessard.research.canon.com.au>
X-IsSubscribed: yes
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
Precedence: bulk
List-Id: <cygwin.cygwin.com>
List-Unsubscribe: <mailto:cygwin-unsubscribe-archive-cygwin=delorie DOT com AT cygwin DOT com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com

On 10 Jun, luke wrote:
>  Larry Hall (Cygwin) wrote:
[...snip]

> > Why not install a new cygwin (and diff if necessary) package and check it
> > out for yourself?  As long as you have the original versions for the
> > newer package(s), you can always reinstall them if you don't like what
> > you see.
>  
>  Okay.
>  
>  <grumble>Now to find a mirror with a complete Cygwin.  Sadly, one must 
>  download the whole Cygwin mirror and then check it, because no mirror 
>  site runs the md5cygchk script I posted here a year or two ago.</grumble>
>  
>  Hmm, maybe it's not too bad using rsync - we may just be able to fetch 
>  the missing few dozen files.  Hmm, if that does work, we should be able 
>  to automate to rsync from several servers routinely, to construct a 
>  union from several sites that are each >99% complete.

Yes, this is what we'll try:

 - rsync, *not* additively (i.e. with deletions) from our chosen main
   mirror site (currently incomplete for about 2 months),
 - run md5cycgchk on our local mirror
 - if that shows an incomplete mirror:
   - rsync *additively* from another site
   - run md5cygchk on the result.
   - Maybe repeat once or twice with other sites if still needed
     (i.e. mirror still incomplete).

This way obsolete packages will eventually get pruned when the main
mirror site gets corrected, and even if one or a few sites don't mirror
properly from cygwin.com, even for a long time, we can still create a
complete mirror (with high probability).

luke

PS: 
md5cygchk is just a script that checks if all packages are present and
correct according to the md5sum files, and records the result in a
success or fail text file that can be checked for: even from a dumb
Windows batch file.



--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/