homepage Welcome to WebmasterWorld Guest from 54.196.196.62
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld
Visit PubCon.com
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
Search engine friendly sitemap for 20,000+ page site and growing
advice needed
mayday88




msg:3894716
 4:19 pm on Apr 17, 2009 (gmt 0)

Hello:

I have a classified ads website with approx 20,000 ads at present. I have submitted sitemaps to Google, etc, and only about 30% of my pages have been indexed. I have read an archived post on this website that stated the following: "There are a number of products available to enable this that can produce the static HTML pages off your database quite easily.. This would benefit the user and the search engines in indexing your site."

Basically, I am hoping to find a solution that would allow me to generate a sitemap to enable the major search engines index all of my ads. Also, new ads are added daily.

Does anyone have any advice for me?

Thanks-
Stephen

 

goodroi




msg:3896067
 1:00 pm on Apr 20, 2009 (gmt 0)

Welcome to WebmasterWorld!

All of your pages will probably never be indexed. Often with extremely large sites you will not achieve 100% indexing.

Here are some ways to help increase the percentage of your site that the search engines will index:
-Submit a sitemap (submitting doesn't guarantee indexing)
-Interlink your pages
-Make sure there is significant unique text
-Develop external links deep into your content

good luck!

tadden




msg:3896331
 6:28 pm on Apr 20, 2009 (gmt 0)

I am having the same issue with my jobs site. Thanks for the advice

ogletree




msg:3896358
 6:50 pm on Apr 20, 2009 (gmt 0)

Site maps do not get pages indexed. It is just a way for you to give url's to google to add to its list to check. The best way for you to get all your pages indexed is to properly link your internal pages. Do not have more than 100 links on one page. If all your pages have a template and a tiny amount of unique material you will not get all your pages indexed. Reduce the footprint of your template and do something so that there is more unique text on each page. Your linking structure should look like a pyramid. Interlink your internal pages like wikipedia does as well.

mayday88




msg:3896407
 8:01 pm on Apr 20, 2009 (gmt 0)

Thanks to all for the great advice so far.

mayday88




msg:3896460
 8:55 pm on Apr 20, 2009 (gmt 0)

RE: If all your pages have a template and a tiny amount of unique material you will not get all your pages indexed. Reduce the footprint of your template and do something so that there is more unique text on each page.
>> The footprint is relatively small on the individual ad pages. The unique text would be the details of the item for sale.

RE: Your linking structure should look like a pyramid.
>> Not sure how to go about accomplishing this on a classified ads website. Content reflects pages that are unique to one another.

RE: Interlink your internal pages like wikipedia does as well.
>> Seems like a big task, and again, most of the site's content reflects pages that are unique to one another.

ogletree




msg:3896483
 9:24 pm on Apr 20, 2009 (gmt 0)

In your case Craigslist is a good linking strategy.

mayday88




msg:3896526
 9:54 pm on Apr 20, 2009 (gmt 0)

I'm not quite sure what you mean...

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved