Welcome to WebmasterWorld Guest from 54.196.244.186

Message Too Old, No Replies

Pages being spidered too slow because of database?

     
6:06 am on Jun 17, 2006 (gmt 0)

Full Member

10+ Year Member

joined:Nov 24, 2003
posts:206
votes: 0


I have a bout 50,000 products in my database and I use asp/mysql for generating dynamic pages, all unique descriptions and keywords. The only problem is...it takes about 4-5 seconds for the pages to load, not because the page's filesize is too big but because of the server speed (I'm guessing) and the large database (50,000 records). Could that be the two factors that slow down google spidering those pages? I've had those pages up for about two weeks now and only about 800 got spidered so far. I see some of my competitors doing the same exact thing, but they have around 500,000 pages spidered by google and they started doing it at the same time I did. (they also have exact linking structure, backlinks to the site, age of the site, google pr and all the other little factors are the same). Their pages load instantly though, so should I get a faster server and reduce the database size to speed up google spidering the pages? Any ideas would be greatly appreciated.
10:29 pm on June 17, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member tedster is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:May 26, 2000
posts:37301
votes: 0


In my experience, Google really does care about extremely slow server response, especially when it happens regularly and is not just once in a while. They don't want their users to get a poor experince from a results page they serve up.

Yes, I'd say do whatever you need to in order to tune the database functioning. But also, realize that with a relitively nerw set of pages, the new Google crawling pattern may take a while to get deep into the site.