Page is a not externally linkable
fsmobilez - 8:07 pm on Sep 23, 2008 (gmt 0)
the robots.txt urls cached pages are just turning to We're sorry, but we could not process your request for the cache of http://example.com/demo.php?cat_id=ahhf&jtype=1. Please click here to check the current page or check for previous versions at the Internet Archive. So what u say is it properly blocked by yahoo as well and yes most of the pages are crawled with jtype=1 example below www.example.com/demo/jokes_category.php?cat_id=78&jtype=1 as i have blocked them and now i want all search engines to crawl the urls without extra parameter one of the example is www.example.com/demo/jokes_category.php?cat_id=78 so can i submit a new sitemap with new requried urls or just wait for few weeks,months as i dont know whether robots blocked urls will be removed from search engines or not.
well i have search in yahoo