Forum Moderators: DixonJones
I am not new to web sites, but I am to maximising search engine exposure as I am setting up some article databases. One of these has over 1600 articles and yet when I look at the AW Stats in CPanel, it shows me today that Google has visited only 139 pages, MSN 323 and Ask Jeeves 58.
The site has a sitemap installed, so I can't fathom these discrepancies. Also, do these reflect only the pages found or are they new pages found by the bots?
I hope this was the right Forum into which to place this. If not, mod please move.
Regards,
Ric
I am not a specialist when it comes to search engine spiders but I can tell you that what you are observing is quite normal. Don't expect a search engine to read your entire site in one go. It will take some time. A hundred hits per day is perfectly ok. Some spiders are more greedy initially while GoogleBot seems to be rather conservative in the beginning. After a couple of weeks you will notice that GoogleBot does a good job detecting changes and new pages.
You might want to search Google for[site:www.yourdomain.com] (without the square brackets) in order to see how many pages G has indexed already.
Google does have my pages indexed, but only 179 of them. This is a far cry from the total number, in fact about one tenth of the site. I shall try and exercise patience.
Kind regards,
Ric
One of the rumors about Google is that they compare site growth with traffic growth. Google can get an estimate of a site's traffic from the toolbar data they collect. By exposing too many pages at once a young site with little traffic can easily look unnatural.
Site went up towards the end of March. It is an article site and receives over 100 submissions a day so I can't really control the growth of the site unless I delete submissions. But the "growth" would be visible on a daily basis and as each day goes by the percentage growth would effectively be less because the submission rates are about the same, but the total page numbers are growing.
Kind regards,
Ric
One of the things you can do is to make it easier for the search engines to index new submissions. A common way to do that is to link to the new pages from somewhere at the top of the home page. Kind of blogging style. This will not speed up the initial crawling of your site, but it will reduce the time it takes the search engine to pickup new pages once it has been completely crawled and indexed. If the new pages are hidden in the link structure of you site it might take considerably longer for the search engines to find them.
You might also want to consolidate multiple submissions onto one page. But that really depends on the kind of content. I am not sure what the ideal page size is but I would guess that it's somewhere between 100 to 1000 words. If your average "submission" contains fewer words I would consider grouping multiple submission on one page.
Quite frankly, I'm not sure how this is done because I am not familiar with php. html/xhtml and css is more my style with static pages. This is a free script and is encrypted using ioncube or something like that, so you can't even change the pages around or read the original code.
Perhaps it will be a matter of "Rome wasn't built in a day!"
Kind regards,
Ric
[edited by: Receptional at 9:19 pm (utc) on May 7, 2006]
[edit reason] No URL drops - Please see TOS. [/edit]
