Forum Moderators: open

Message Too Old, No Replies

Googlebot and HTTP 'Referer'

Does Google collect links then spider them?

         

netnerd

5:20 pm on Aug 31, 2003 (gmt 0)

10+ Year Member



I have a question which i know someone will be able to answer for me!

When googlebot spiders links, is the 'referrer' the page before on the web, or is it googlebot?

I.E. if i want to count the clicks from a certain url1 to my site at url2, will clicks generated by googlebot spidering my site show up as referred from url1 or from googlebot itself in a windows log?

Hope you guys can help!

ciml

1:12 pm on Sep 1, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Googlebot always sends a "Googlebot/2.1 (+http://www.googlebot.com/bot.html)" as the HTTP 'referer'.

netnerd

1:29 pm on Sep 1, 2003 (gmt 0)

10+ Year Member



Great!

Thats what i was hoping to hear!

killroy

1:32 pm on Sep 1, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Actually that's the user_agent. googlebot doesn't send any referer information

SN