Mailing-List:	contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Subscribe:	<mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive:	<http://sourceware.org/ml/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender:	cygwin-owner AT cygwin DOT com
Mail-Followup-To:	cygwin AT cygwin DOT com
Delivered-To:	mailing list cygwin AT cygwin DOT com
MIME-Version:	1.0
Subject:	RE: Cygwin Time Machine
Date:	Thu, 10 Feb 2005 14:35:14 -0500
Message-ID:	<336E93A5D77EC246881C518F41A47CCA45A95E@EXCHANGE1.atl.air2web.com>
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
From:	"Richard Campbell" <richard DOT campbell AT air2web DOT com>
To:	<cygwin AT cygwin DOT com>
X-IsSubscribed:	yes
X-MIME-Autoconverted:	from quoted-printable to 8bit by delorie.com id j1AJZREr008591

Peter Castro wrote:
>
> (if there's a way to make wget ignore robots.txt, I'd love to know
it!)

http://www.gnu.org/software/wget/faq.html#3.0
"3.0 How can I make GNU wget ignore a robots.txt file?

Try using:

wget -erobots=off http://your.site.here "

-Richard Campbell

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

- Raw text -