- - Search Engines
- -- Sitemaps, Meta Data, and robots.txt
- ---- Who to create XML sitemap exclude URL parameter in Xenu's Link Sleuth
rajesh_magar - 10:46 am on Sep 25, 2012 (gmt 0)
Ok that's fine, but is't their any way to skip this links in Xenu itself.
Like if you look at the functionality of the ScreamingFrogSEOSpider it's only crawl main pages and totally skip those all link with additional parameter in it.
Truly Appreciate you feedback, Thanks
Thread source: http://www.webmasterworld.com/robots_txt/4499601.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com