homepage Welcome to WebmasterWorld Guest from 107.20.25.215
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

    
Google Releases New Sitemap Generator Tool
pageoneresults




msg:3826006
 11:05 am on Jan 14, 2009 (gmt 0)

While most Sitemap generators either crawl websites or list the files on a server, we have created a different kind of Sitemap generator that uses several ways to find URLs on your website and then allows you to automatically create and maintain different kinds of Sitemap files.

A new Google Sitemap Generator for your website
2009-01-13 - [googlewebmastercentral.blogspot.com...]

 

tedster




msg:3826291
 5:38 pm on Jan 14, 2009 (gmt 0)

It's a server plug-in, so it requires admin level server access - but it does work for both Apache and IIS. Lots of user options here as well as three different methods for discovering urls on the site. It sounds like a well-thought out project.

notsosmart




msg:3826351
 7:03 pm on Jan 14, 2009 (gmt 0)

Good to know that the $25 i spent on < a commercial sitemap generator > (for like the tenth time) the other day was wasted.

(It's not really about the money - $25 - rather I'm mad at myself for constantly paying for things I could get for free. Oh well).

[edited by: Robert_Charlton at 7:54 pm (utc) on Jan. 14, 2009]
[edit reason] edited specific [/edit]

goodroi




msg:3826355
 7:16 pm on Jan 14, 2009 (gmt 0)

if your 404 page is returning a 200 status code your competition could use this to stuff your google sitemap with errors. i am not even going to get into the fun you could have with sites that generate urls on the fly. oh the potential chaos that a disobedient bot could cause on certain websites in selective situations.

maximillianos




msg:3826361
 7:23 pm on Jan 14, 2009 (gmt 0)

I thought the point of a sitemap was to notify search engines of content that is not being found. Ie - content that is not showing up in our log files.

Though I can see where it would help expose content that other search engines are sending traffic to... It doesn't beat building your own sitemap tool... at least in my opinion.

KHWeb




msg:3826383
 8:17 pm on Jan 14, 2009 (gmt 0)

Goodroi,

Regarding your comment:
"...the fun you could have with sites that generate urls on the fly. oh the potential chaos that a disobedient bot could cause on certain websites in selective situations. "

Can you please explain further? Most of my sites are generating urls on the fly and indeed I am getting google sitemaps errors with urls that don't exist and just go to a custom 404 page. The url rewrite was done for SEO purposes...but definitely I don't want this to create another problem and most importantly an opportunity for malicious competitors and bots to affect our ranking.

How can a bot or a malicious competitor create this havoc and how can I stop this?

Thanks,

KH

trinorthlighting




msg:3826452
 9:40 pm on Jan 14, 2009 (gmt 0)

What they should do, incorporate google analytics, google-bot and any other bots with google webmasters. Pull the url's from there dump them into a space in google webmaster tools and give the option to the webmaster to review them to create their own site map.

goodroi




msg:3826695
 2:50 am on Jan 15, 2009 (gmt 0)

hi KHWeb,

this new sitemap generator tool creates a sitemap file based on the urls that receive traffic. if someone wants to, they can send you fake traffic to any url. you may not want those urls automatically included in your sitemap. this new sitemap generator will include those urls since it is based on traffic. i dont want to go much more into it because i dont want to encourage people to go looking for trouble.

a clever person in just the right situation could have fun. imho the potential for trouble here is limited so you should not be very worried.

nealrodriguez




msg:3827321
 7:04 pm on Jan 15, 2009 (gmt 0)

if someone wants to, they can send you fake traffic to any url. you may not want those urls automatically included in your sitemap.

why can't you robots.txt those urls?

nealrodriguez




msg:3827325
 7:12 pm on Jan 15, 2009 (gmt 0)

why can't you robots.txt those urls?

whoops! just answered my own question; i haven't installed it, but the doc reads the following

Under URL query fields, read the special privacy notice and then specify URL query fields that you want to include in Sitemaps. Because Google Sitemap Generator removes all query fields from URLs in order to protect user privacy, you must explicitly specify any query field that you want URLs to include.

papachumba




msg:3830520
 5:56 pm on Jan 20, 2009 (gmt 0)

i installed it on our corporate IIS 6 running Winblows Server 2003
HttpRewrite & IIS Zip Compression enabled on the box

After installation wouldn't run properly due to mis-configured permissions. I set the correct permissions, after which it gave me another error and i just gave up and uninstalled...

The search for a perfect sitemap generator continues...

Quadrille




msg:3830530
 6:13 pm on Jan 20, 2009 (gmt 0)

What about us technophobes?

Why can't Google cook up a reliable online sitemap generator, that does not have artificial linits (eg 500 pages), or fees, or a pageful of promos, or code that hides links to the 3rd party system?

Why does Google claim to want everyone to have a fair chance - then discriminate against the non-technical?

I don't expect answers, BTW; I've said similar on Google's group, and was told "Why would they offer a tool that spiders, when they'll be spidering the site anyway" - so why bother with sitemaps at all?

civgroup




msg:3836750
 7:48 pm on Jan 28, 2009 (gmt 0)

If you run your own server (or at least have root) this is not that difficult to install - I ran into a couple of snags but the Google Group for this tool was enough to get me going.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved