Welcome to WebmasterWorld Guest from 34.231.247.139

Forum Moderators: goodroi

Message Too Old, No Replies

Who to create XML sitemap exclude URL parameter in Xenu's Link Sleuth

Best practices to using Xenu's Link Sleuth

     
7:15 am on Sep 25, 2012 (gmt 0)

New User

5+ Year Member

joined:Aug 11, 2011
posts: 9
votes: 0


Hi All,

I have website which is having 3000+ article/links posted on it. And just week ago it's been redesign and now I've been wondering to create latest XML sitemap for the same.

So problem just that when I run the URL in Xenu's Link Sleuth to fetch my website links, its crawling 9000+ links with adding URL Parameter, like "?Action=" and all.

So how can I skip this URLs with addition parameter and just get crawl canonical URLs to create light weight XML sitemap.

Thanks in Advance.
9:32 am on Sept 25, 2012 (gmt 0)

Administrator

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 10, 2004
posts:11824
votes: 237


xenu is simply going to discover the urls you link to internally, so if xenu found non-canonical url parameters you have to fix this in your web application.
10:46 am on Sept 25, 2012 (gmt 0)

New User

5+ Year Member

joined:Aug 11, 2011
posts: 9
votes: 0


Ok that's fine, but is't their any way to skip this links in Xenu itself.

Like if you look at the functionality of the ScreamingFrogSEOSpider it's only crawl main pages and totally skip those all link with additional parameter in it.

Truly Appreciate you feedback, Thanks
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members