Forum Moderators: open
I have a content site (fan website for something.) and I have a flatfile style database that people can search off of in order to find the product they want to see information about.
So if they search this database base for "widget bob" they get a link to the profile page of "widget bob" like this:
www.mysite.com/profile.cgi?n=001
There are about 800 of these "profiles" in my database.
People often use search strings from google (or any other search engine) like "widget bob". So I want google to realize I have a "widget bob" page. Though it is not static so it can't be spidered, well google does spider these type of my pages if I happen to have a link to that exact profile somewhere on one of my pages.
So I thought ok I created another script that automatically generates a static html page called something like all_widgets.html and in it its a list of all the entries in the database and each in the list is a link to their profile.
Now these page has more then one purpose, yes I created it so google might spider all of those links like it does when it sees one on my site else where, and two it has proved a popular page for visitors who want the long list.
Goggle spidered the all_widgets.html page right away, but it hasn't picked up all of the links inside of the all_widgets.html like I hoped.
Do you think it just hasn't got to them (only been a few weeks, but google usually spiders my pages really page, as it picked up the new all_widgets.html within the first week), or maybe because it has 800+ links in it that it is to large and google see's it as bad etc...
Any thoughts or suggestions?
thanx! I have really enjoyed this forum since I found out about webmasterworld earlier this month when I was searching for answers as to why my site which ranked #2 for its main keywords dropped into no where land! :) Least I made it back to page 2, waaa!
-Marie
Welcome to WebmasterWorld
If you take a look at Google's guidelines [google.com], you'll see that they recommend 100 links or less. If you break up the page into several smaller pages (maybe by listing them in alphabetical order with a separate page for ech letter) you will have better luck getting all your pages indexed.
It will probably work eventually, but as WebGuerrilla says, breaking it up should give you better results.
Try breaking the links up in chunks alphabetically, so that they each have about 70 links. This will give them room to grow. Don't worry if some pages go above 100 links, but if you try to keep it under that number, it gives you a good safety margin.
