Forum Moderators: open

Message Too Old, No Replies

Google is not showing all spidered pages in spider log files

         

Nitin

5:33 am on Apr 28, 2003 (gmt 0)

10+ Year Member



Google is spidering my discussion pages having dynamic URL's with two parameters,but it is not showing in spider log files.

I am new to this field please guide.

Thanks for your time.

TheDave

6:26 am on Apr 28, 2003 (gmt 0)

10+ Year Member



I'm not quite sure what your question is, but I'll have a go. Are you looking in your website logs to see if googlebot has been, but not seeing it there? Are you wondering why there are pages in google which according to your logs havn't been spidered yet? The answer may be that google will index links as well as pages, and it is the links you are seeing in the google search results. You can tell if it is only a link by looking in the google SERP (SERP - Search Engine Results Page) and seeing if it has a description beneath the link, and also if there is a link to the cached copy. No description and no cache generally mean that google has indexed the links, but not spidered the pages. I hope this answers you question, but please explain further if it doesn't.

Nitin

6:48 am on Apr 28, 2003 (gmt 0)

10+ Year Member



Thanks Dave,

SERP is showing description as well as cached copy.It means Googlebot has spidered my pages,still not showing in log files.

TheDave

7:38 am on Apr 28, 2003 (gmt 0)

10+ Year Member



All I can suggest then, if you are determined to find googlebot in your logs, is to dig deeper/further back into your logs, maybe even get the raw log file and search that if you havent already (ie., load it in a text editor and go Edit->Find)