Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- msnbot-media


lucy24 - 10:15 pm on Feb 11, 2013 (gmt 0)


What I'm referring to is this:

http://example.com/random-numbers-here/valid-page.html

Gotcha. I see that pattern sometimes* from inept robots. Yandex sometimes, but not bing yet. Except that generally it isn't a wholly imaginary directory. It's a real directory-- just not the one that contains the file they're asking for. And it isn't due to linking errors from my end.

Hm. I guess you can't downgrade a robot for Low Technical Quality ;)


* Shortly after posting, I went over to chew on logs. And what should pop up but
/paintings/myrats/vwegfmnwngg.html
That's the "real location, bogus filename" variant.

:: wandering off to feed 'vwegfmnwngg' into transcoder to see if it yields a real word in any major legacy font ::


Thread source:: http://www.webmasterworld.com/search_engine_spiders/4470273.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com