Forum Moderators: open

Message Too Old, No Replies

         

StanTheMan

1:36 pm on Feb 25, 2003 (gmt 0)

10+ Year Member



Advice For A Newbie!

How would I findout the way google has crawled my site?, How deep and What pages it missed out and so on.

How do you guys go about doing so?

Regards

jpjones

1:47 pm on Feb 25, 2003 (gmt 0)

10+ Year Member



Hi Martin,

Look at the raw logs for the site, looking for any requests by the user agent "Googlebot/2.1" - this is Google's bot.

You can then match up the requests to your site pages and work out what has been crawled and what hasn't.

JP

jpjones

1:52 pm on Feb 25, 2003 (gmt 0)

10+ Year Member



Just to add to the above - you can also recognise googlebot by it's IP Address.

Look at this forum [webmasterworld.com] for further information.

JP

WebRankInfo

7:28 pm on Mar 15, 2003 (gmt 0)

10+ Year Member



for those of you with PHP support, you may try GoogleStats.
It is a free PHP application detecting the visits of search engines robots, and presenting statistics in tables and graphics.
It is available in french, english, german and spanish.
htpp://www.googlestats.com/english/

Enjoy!