RankFlex.com Webspider

Forum Moderators: open

Message Too Old, No Replies

RankFlex.com Webspider

incrediBILL

7:26 am on Oct 29, 2010 (gmt 0)

Doesn't ask for robots.txt and does this crazy probe:

83.169.26.5 "GET / HTTP/1.1" "RankFlex.com Webspider"
83.169.26.5 "GET /index.php HTTP/1.1" "RankFlex.com Webspider"
83.169.26.5 "GET /default.php HTTP/1.1""RankFlex.com Webspider"
83.169.26.5 "GET /index.html HTTP/1.1" "RankFlex.com Webspider"
83.169.26.5 "GET /index.htm HTTP/1.1" "RankFlex.com Webspider"
83.169.26.5 "GET /default.htm HTTP/1.1" "RankFlex.com Webspider"
83.169.26.5 "GET /default.html HTTP/1.1" "RankFlex.com Webspider"
83.169.26.5 "GET /index.jsp HTTP/1.1" "RankFlex.com Webspider"
83.169.26.5 "GET /default.jsp HTTP/1.1" "RankFlex.com Webspider"
83.169.26.5 "GET /rankflex_content_duplication_test HTTP/1.1" "RankFlex.com Webspider"

crawls from vwp4525.webpack.hosteurope.de

Dijkgraaf

12:12 am on Nov 1, 2010 (gmt 0)

Looks like it is trying to see whether the same page gets served up using different default page names, and comparing it against the 404 page and the actual default page.