Page is a not externally linkable
- Search Engines
-- Alternative Search Engines
---- ArchitextSpider going mad?
gekoviola - 5:22 am on Jun 13, 2001 (gmt 0)Thread source:: http://www.webmasterworld.com/alternative_search_engines/329.htm
After a quick look at my logs, everyday I see that the ArchiText Spider IS going mad, he make thousands of requests for non-existant urls.
I've noticed this behavior happened also 2 years ago.
Brought to you by WebmasterWorld: http://www.webmasterworld.com