Forum Moderators: open
This on is interesting because I found it in a log of a site that uses an IP delivery system. This bot grabbed about 50 pages from this particular site.
However, this IP is not on the list, so requesting any single page from the site will return the content, but that content will not contain any links to the other pages.
That would suggest that this bot is crawling from a list of urls collected from other search engines, rather than following links in a normal fashion.
dave
