So just adding my sitemap to robots is enough. But HOW in first place Ask.com will find my robots.txt? I mean how do I submit either my robots.txt or sitemap.xml or home page. Couldn't find that info on Ask.
phranque
2:07 am on Jul 20, 2009 (gmt 0)
any "well-behaved" robot will request and respect robots.txt before crawling your site. some bots will check robots.txt before each page request and others will do it once per "session".
eeek
8:38 am on Aug 22, 2009 (gmt 0)
Ask.com will use an XML Sitemap, too - you need to use the robots.txt autodiscovery method.