Forum Moderators: open

Message Too Old, No Replies

Googlebot only looks at one page?

Why is this - googlebot only looks at my index page?

         

netnerd

9:58 am on May 13, 2003 (gmt 0)

10+ Year Member



I have a new site which has text links to the other pages from the main index page.

Ihave no robots.txt file - in my logs googlebot takes a look only at robots.txt and index.asp.

64.68.82.67 - blah blah 80 GET /index.asp - 200 0 0 227 460 HTTP/1.0 Googlebot/2.1+(+http://www.googlebot.com/bot.html)

SHould this be ok for future deep crawling or do i need to worry that it didnt work through the whole site?

creative craig

12:42 pm on May 13, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



It shouldnt be a problem, just keep watching and when the deep crawl comes along it should use those text links to go through the whole site.

Craig

netnerd

1:07 pm on May 13, 2003 (gmt 0)

10+ Year Member



Ok Craig - when is this deep crawl likely to be?

creative craig

1:09 pm on May 13, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Good question, when the update is over, or is that when it has started ;)

No one actualy knows at the moment, Google are trying a few new things this month so the time scale is out of the window.

Craig