Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com
Content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Subject: RE: Cygwin Time Machine
Date: Thu, 10 Feb 2005 14:35:14 -0500
Message-ID: <336E93A5D77EC246881C518F41A47CCA45A95E@EXCHANGE1.atl.air2web.com>
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
From: "Richard Campbell" <richard DOT campbell AT air2web DOT com>
To: <cygwin AT cygwin DOT com>
X-IsSubscribed: yes
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by delorie.com id j1AJZREr008591

Peter Castro wrote:
>
> (if there's a way to make wget ignore robots.txt, I'd love to know
it!)

http://www.gnu.org/software/wget/faq.html#3.0
"3.0 How can I make GNU wget ignore a robots.txt file?

Try using:

wget -erobots=off http://your.site.here "

-Richard Campbell

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/