Welcome to WebmasterWorld Guest from 54.159.50.111

Forum Moderators: Ocean10000 & incrediBILL

Message Too Old, No Replies

Reasons for using Googlebot user agent?

     
7:29 pm on Mar 18, 2010 (gmt 0)

Full Member

5+ Year Member

joined:Aug 28, 2007
posts:248
votes: 0


What are the reasons for wanting to occasionally switching to the Googlebot user agent? I know people do it to check for cloaking, but are there other reasons? Just curious to hear how people use it...

[edited by: incrediBILL at 8:48 pm (utc) on Mar 18, 2010]
[edit reason] fixed typo [/edit]

8:53 pm on Mar 18, 2010 (gmt 0)

Administrator from US 

WebmasterWorld Administrator incredibill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Jan 25, 2005
posts:14624
votes: 88


Snooping people do this to see how Google sees your website, mostly for SEO purposes.

Some sites only send text to Google, no formatting.

Some sites load their text with extra keywords for Google to see.

However, smart sites like mine validate Googlebot and will boot you to the curb for faking so you'll never see what I'm showing Google, never, ever.
1:12 am on Mar 19, 2010 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:5821
votes: 64


However, smart sites like mine validate Googlebot and will boot you to the curb for faking so you'll never see what I'm showing Google, never, ever.

+1
2:30 am on Mar 19, 2010 (gmt 0)

Preferred Member

10+ Year Member

joined:Feb 13, 2003
posts:590
votes: 0


At least one site (and we probably all know which one I'm talking about) shows Googlebot answers to questions, but only shows the questions to real people, along with a link to sign up for a paid subscription. Unless you tell your browser to send a Googlebot UA; then you get the answers too. That site does not validate Googlebot.
9:25 am on Mar 20, 2010 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Aug 30, 2002
posts:2416
votes: 24


Scrapers use this in an attempt to bypass controls. Some sites will whitelist the large search engine bots but apply access controls and limits to ordinary browsers.

Regards...jmcc