| Welcome to WebmasterWorld Guest from 126.96.36.199 |
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
|Pubcon Platinum Sponsor|
|Google Crawling Without Identification?|
| 3:45 pm on Mar 15, 2008 (gmt 0)|
3/15/2008 2:48:33 PM----188.8.131.52----/Default.asp ----Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:184.108.40.206) Gecko/2007030919 Firefox/220.127.116.11
3/15/2008 2:48:37 PM----18.104.22.168----/Default.asp ----Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:22.214.171.124) Gecko/20080201 Firefox/126.96.36.199
Today, took one page and css file, no robots.txt and no images.
Does anybody know about a specific reason as to why ?
| 4:04 pm on Mar 15, 2008 (gmt 0)|
Might be a human inspection from Google, rather than a googlebot crawl. That Google IP address does not have reverse-DNS set up, and googlebot crawlers all do.
| 10:45 pm on Mar 15, 2008 (gmt 0)|
What made me suspicious was 1 IP number, a 4 second interval and 2 different user-agents which together usually means not much good.
Weird way of behaving, that's why I checked the IP number.
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
© Webmaster World 1996-2013 all rights reserved