Forum Moderators: open

Message Too Old, No Replies

How to Get Notified at What Time Google Spider Hits Your Page?

         

aakash

4:39 am on Jan 14, 2005 (gmt 0)

10+ Year Member



Do any webmaster here, have any idea about how to get information about google visiting your page. I was wondering If I could get a notification or something like that as soon as google spider visit site.

Brett_Tabke

10:29 pm on Jan 14, 2005 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



whew - if I did that, my isp would kick me off for email abuse.

Yesterday:
43304 -8.1% Mozilla/5.0 (compatible; Yahoo! Slurp; [help.yahoo.com...]
28465 -5.31% Googlebot/2.1 (+http://www.google.com/bot.html)
7639 -1.61% msnbot/0.3 (+http://search.msn.com/msnbot.htm)

pmkpmk

10:49 pm on Jan 14, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Subscribe to the Google web/news alerts, and submit the query string "site:mysite.com"

BillyS

1:36 am on Jan 15, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Yesterday:
43304 -8.1% Mozilla/5.0 (compatible; Yahoo! Slurp; [help.yahoo.com...]
28465 -5.31% Googlebot/2.1 (+http://www.google.com/bot.html)
7639 -1.61% msnbot/0.3 (+http://search.msn.com/msnbot.htm)

Wow, this site serves 540,000 pages a day!

ogletree

1:40 am on Jan 15, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I'm sure somebody asking that rarely if ever gets hit. Yes there is a way. You can just put some server side code on the page (php, asp) that detects the referer and if it is gbot have it send you a message. Sticky me if you need some help.

BroadProspect

8:16 am on Jan 15, 2005 (gmt 0)

10+ Year Member



there is a free service out there that automatically sends you an email notification whenever a search engine crawler (spider/agent/robot) is scanning your website.
due to the TOS here I can not point the URL.
I suggest you search for it in google
/BP

aakash

11:33 am on Jan 15, 2005 (gmt 0)

10+ Year Member



Well, thanks all for your response. Did somebody create any scripts to detect the spider. Let us know.

Thanks

Brett_Tabke

3:51 am on Jan 16, 2005 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



lol - check *any* of the script archives - there are many out there in every flavor.

ganderla

6:35 am on Jan 16, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



7639 -1.61% msnbot/0.3 (+http://search.msn.com/msnbot.htm)

Brett, At one time weren't you banning the MSN bot from your site?

webnewton

11:24 am on Jan 18, 2005 (gmt 0)

10+ Year Member



For what spcific purpose do you need that Aakash anyways?

aakash

11:39 am on Jan 18, 2005 (gmt 0)

10+ Year Member



just for kicks, you mind

webnewton

7:02 am on Jan 21, 2005 (gmt 0)

10+ Year Member



Don't mind at all. Have fun. :>)