Welcome to WebmasterWorld Guest from 188.8.131.52
My mother tongue is French, i will try to make sense, OK?
I recently recieved the visit of sneaky robot. It seems to provoke a 404 to test the server.
Here is a clip from the log.
184.108.40.206 - - [04/Jan/2001:02:34:16 -0500] "GET /robots.txt HTTP/1.0" 200 354
220.127.116.11 - - [04/Jan/2001:02:34:22 -0500] "GET / HTTP/1.0" 200 14788
18.104.22.168 - - [04/Jan/2001:02:34:20 -0500] "GET /test404response462450495.html HTTP/1.0"
The site uses java script redirect to the index.htm file, only because I use frames.
A couple of real content pages where optimised for targeted engines, but I neither use entry pages nor cloaking. I visited digital integrity web site to discover that they sell robots.
Can some one teach me about this robot, please?
Edited by: Macguru
Like if you copied news report from cnn.com and posted it on your site without their approval. Digital Integrity is one of those companies that searches around the web looking to see if this news source is posted on any other sites.
Or if you post photographs on your site that you didn't license from the photographer.
These are the types of things they spider for. If you didn't nick anything from other peoples' sites without their consent then you have nothing to worry about.