| Welcome to WebmasterWorld Guest from 220.127.116.11 |
register, login, search, subscribe, help, library, PubCon, announcements, recent posts, open posts,
|Subscribe to WebmasterWorld|
|Who to create XML sitemap exclude URL parameter in Xenu's Link Sleuth|
Best practices to using Xenu's Link Sleuth
| 7:15 am on Sep 25, 2012 (gmt 0)|
I have website which is having 3000+ article/links posted on it. And just week ago it's been redesign and now I've been wondering to create latest XML sitemap for the same.
So problem just that when I run the URL in Xenu's Link Sleuth to fetch my website links, its crawling 9000+ links with adding URL Parameter, like "?Action=" and all.
So how can I skip this URLs with addition parameter and just get crawl canonical URLs to create light weight XML sitemap.
Thanks in Advance.
| 9:32 am on Sep 25, 2012 (gmt 0)|
xenu is simply going to discover the urls you link to internally, so if xenu found non-canonical url parameters you have to fix this in your web application.
| 10:46 am on Sep 25, 2012 (gmt 0)|
Ok that's fine, but is't their any way to skip this links in Xenu itself.
Like if you look at the functionality of the ScreamingFrogSEOSpider it's only crawl main pages and totally skip those all link with additional parameter in it.
Truly Appreciate you feedback, Thanks
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
WebmasterWorld ® and PubCon ® are a Registered Trademarks of Pubcon Inc.
© Pubcon Inc. 1996-2012 all rights reserved