Welcome to WebmasterWorld Guest from 126.96.36.199
I need to write a robots.text file but i do not have access to server logs, i need to know the names of these spiders, in particular the altavistaUK and miragoUK bots
does anyone know of an extensive and upto date list of spider names
The "Scooters" are the crawlers, I do not submit to AVuk.
Henry is the only mirago bot I know of.
The names tend to exclude version numbers, and
I don't go back and check they're still active once added to the page.
I also have a page on search engines at
This includes a list of around a dozen UK
search engines, but doesn't include their spider
I seem to be posting this URL a few times :-)
Is there a resource page on this site I should
be submitting these links to?