Forum Moderators: open

Message Too Old, No Replies

Deep Crawl question for new pages on my site

         

nadsab

4:37 am on Apr 6, 2003 (gmt 0)

10+ Year Member



Hi,

I had a small three page site and I just upgraded it this weekend, it now has 15 pages, nice titles for each page with keywords in the titles matching the content of the site, all 15 pages have the exact same internal link menu and all of the pages are linked to each other.

I was wondering, to get ready for the deep crawl, should I be doing anything else to get ready for it so that Goggle will index all my pages? Do I need to be adding any robot scripts or anything like that? Like should I have a robots.txt file in my site for google?

Or do I need to place a <META NAME="ROBOTS" CONTENT="ALL">
tag in my head so google will index all my pages?

Thanks.

troels nybo nielsen

7:09 am on Apr 6, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



robots.txt files and meta tags are not necessary.

Most important are

1. Have good inbound links from well established websites.
2. The more pages you have, the higher PR you need to have Google crawl your entire site. With only 15 pages this should not be a problem.
3. Have nice, clean easily accesible code where the links are easy to find for the bot.
4. Have a good linking structure. Every single page should have several both inbound and outbound links. No page should be more than three steps away from the main page. You have no problems here.

But your tight linking structure may be a problem _after_ your site has been crawled. Ask yourself if your internal linking is natural and a benefit for your human visitors. Every page linking to all other pages sounds a bit excessive. Google tend to penalise pages with links that seem to have been made not for human visitors but for robots.

nadsab

3:53 pm on Apr 6, 2003 (gmt 0)

10+ Year Member



Thanks troels nybo nielsen,

So if I have 15 pages with lots of internal site links to other pages on the site, and all pages are linked to other pages in the site, but none (or few) of them have outbound links to other outside sites, would this cause google to not crawl those pages or give me a very low rank for those pages? My pr is 4 now for my index.html.

For every page linking to all other pages you say could be a problem? I have a menu on the left column containing titles/links to all the other pages. U think this would reduce my rank? It makes the site easier to navigate I think, but do you think I should take it off and on each page link to just a few other internal site pages? Like have three or four internal links and three or four external links on each page? Does google like to have equal numbers of inside and outside links on each page or is that not a factor for PR?

garylo

4:08 pm on Apr 6, 2003 (gmt 0)

10+ Year Member



You should be OK, no need to make any modifications.