Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

Generating a huge sitemap on shared host

Creating a sitemap with thousands of URL'son a shared hosting plan.

5:54 pm on May 23, 2011 (gmt 0)

New User

5+ Year Member

joined:May 18, 2011
posts: 12
votes: 0

I'm looking into creating a sitemap for my site running joomla and phpbb3. I've encountered a couple problems and have a few questions on what's really needed and how to get around these problems. The site has thousands of pages because of the message board. This is the "meat and potatoes" of the site and my goal as the webmaster is to grow the user base so that I get more enjoyment out of the site and maybe make enough coin to pay the bills.

Question 1 - Is there a noticeable benefit to having a sitemap containg URL's for every relevant topic compared to a sitemap with URL's to the main pages? Would manually creating a sitemap that goes a view levels deep into the forum be sufficient?

I ask this because I am having issues generating a sitemap. I tried using gsitecrawler and, because of the size, it puts a strain on my host and takes a LONG time to run. I am on a shared hosting plan and got scolded for going above "industry standard usage levels" last time I ran the crawler. I was able to explain that the usage was an anomaly and they granted me a window to get the sitemap created. This leads me to question 2:

Question 2 - If I use gsitecrawler, how often will I need to run it and produce an up to date sitemap? I think the most I could weasel out of my host is once a month, would this be sufficient?

I also looked into using googles python script. (http://sitemap-generators.googlecode.com/svn/trunk/docs/en/sitemap-generator.html) I'm not even sure this would work with my message board as it uses " create Sitemaps from URL lists, web server directories, or from access logs." Plus, my limited host comes into play again as they want a $40 per year fee to be able to access via SSH. Also, I'm thinking this would create usage issues as well, yes?

Question 3 - Is the google python script a viable alternative? Is there any other method you can suggest to accomplish my goal considering my limitations?

As always, thanks for your help!

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members