delorie.com/archives/browse.cgi | search |
DJ Delorie wrote: > >> By the way: The internet archive does not seem to store >> geda-project.org in its data base. >> http://wayback.archive.org/web/*/http://www.geda-project.org >> Is this deliberate? > > No. There is a robots.txt: http://geda-project.org/robots.txt /----------- # robots.txt for *.seul.org # questions/comments to seul AT seul DOT org User-agent: * Disallow: /wiki User-agent: * Disallow: /cgi-bin User-agent: Slurp Disallow: /cgi-bin User-agent: Scooter Disallow: /cgi-bin User-agent: GoogleBot Disallow: /cgi-bin \------------- The internet is served by their own web crawler and by the one of alexa. Seems to me, that the robots.txt disallows them to enter the wiki. ---<)kaimartin(>---
webmaster | delorie software privacy |
Copyright © 2019 by DJ Delorie | Updated Jul 2019 |