Can anyone tell me anything about an oddly behaved (apparent) spider using this dns: 80.184.32.134? It was referred to my site from an or ordinary organic link I have on a similar site. Then it proceeded to browse like a regular visitor .. but ..
It sucked in every last page I have, including the images. I put up one image per page usually. The pattern was totally mechanical.
Index page => menu #1
Menu #1 => page A, then downloads image A.
Back to Index.
Index page => menu #1
Menu #1 => page B, then downloads image B.
Back to Index.
This repeated until all pages on menu #1 were exhausted. Next back to index.html, and the whole silly waltz repeated with menu #2. Then Menus 3 and 4. Finally it went nuts on my sitemap and found all my oddball pages.
This was extremely methodical, with hits coming from 3 to 10 seconds apart. NO human browses like that.
It never looked at robots.txt. There was no referral string or identifier. Spiders usually only suck up my html files. Image bots come later for those.
I've never seen this before, my only clue was the dns.
Any clues? - Larry
