Welcome to WebmasterWorld Guest from 54.226.25.231

Forum Moderators: goodroi

Message Too Old, No Replies

Who to create XML sitemap exclude URL parameter in Xenu's Link Sleuth

Best practices to using Xenu's Link Sleuth

     

rajesh magar

7:15 am on Sep 25, 2012 (gmt 0)



Hi All,

I have website which is having 3000+ article/links posted on it. And just week ago it's been redesign and now I've been wondering to create latest XML sitemap for the same.

So problem just that when I run the URL in Xenu's Link Sleuth to fetch my website links, its crawling 9000+ links with adding URL Parameter, like "?Action=" and all.

So how can I skip this URLs with addition parameter and just get crawl canonical URLs to create light weight XML sitemap.

Thanks in Advance.

phranque

9:32 am on Sep 25, 2012 (gmt 0)

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



xenu is simply going to discover the urls you link to internally, so if xenu found non-canonical url parameters you have to fix this in your web application.

rajesh magar

10:46 am on Sep 25, 2012 (gmt 0)



Ok that's fine, but is't their any way to skip this links in Xenu itself.

Like if you look at the functionality of the ScreamingFrogSEOSpider it's only crawl main pages and totally skip those all link with additional parameter in it.

Truly Appreciate you feedback, Thanks
 

Featured Threads

Hot Threads This Week

Hot Threads This Month