homepage Welcome to WebmasterWorld Guest from 54.161.155.142
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Visit PubCon.com
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Reasons for using Googlebot user agent?
DiscoStu




msg:4100611
 7:29 pm on Mar 18, 2010 (gmt 0)

What are the reasons for wanting to occasionally switching to the Googlebot user agent? I know people do it to check for cloaking, but are there other reasons? Just curious to hear how people use it...

[edited by: incrediBILL at 8:48 pm (utc) on Mar 18, 2010]
[edit reason] fixed typo [/edit]

 

incrediBILL




msg:4100668
 8:53 pm on Mar 18, 2010 (gmt 0)

Snooping people do this to see how Google sees your website, mostly for SEO purposes.

Some sites only send text to Google, no formatting.

Some sites load their text with extra keywords for Google to see.

However, smart sites like mine validate Googlebot and will boot you to the curb for faking so you'll never see what I'm showing Google, never, ever.

keyplyr




msg:4100786
 1:12 am on Mar 19, 2010 (gmt 0)

However, smart sites like mine validate Googlebot and will boot you to the curb for faking so you'll never see what I'm showing Google, never, ever.

+1

sonjay




msg:4100803
 2:30 am on Mar 19, 2010 (gmt 0)

At least one site (and we probably all know which one I'm talking about) shows Googlebot answers to questions, but only shows the questions to real people, along with a link to sign up for a paid subscription. Unless you tell your browser to send a Googlebot UA; then you get the answers too. That site does not validate Googlebot.

jmccormac




msg:4101462
 9:25 am on Mar 20, 2010 (gmt 0)

Scrapers use this in an attempt to bypass controls. Some sites will whitelist the large search engine bots but apply access controls and limits to ordinary browsers.

Regards...jmcc

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved