homepage Welcome to WebmasterWorld Guest from 54.227.182.191
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

    
Google Crawling Without Identification?
Staffa




msg:3601499
 3:45 pm on Mar 15, 2008 (gmt 0)

3/15/2008 2:48:33 PM----74.125.16.70----/Default.asp ----Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.3) Gecko/2007030919 Firefox/2.0.0.3
3/15/2008 2:48:37 PM----74.125.16.70----/Default.asp ----Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12

Today, took one page and css file, no robots.txt and no images.

Does anybody know about a specific reason as to why ?

 

tedster




msg:3601509
 4:04 pm on Mar 15, 2008 (gmt 0)

Might be a human inspection from Google, rather than a googlebot crawl. That Google IP address does not have reverse-DNS set up, and googlebot crawlers all do.

Staffa




msg:3601661
 10:45 pm on Mar 15, 2008 (gmt 0)

Thanks, tedster.

What made me suspicious was 1 IP number, a 4 second interval and 2 different user-agents which together usually means not much good.
Weird way of behaving, that's why I checked the IP number.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved