Welcome to WebmasterWorld Guest from 184.108.40.206 , register , free tools , login , search , pro membership , help , library , announcements , recent posts , open posts Pubcon Platinum Sponsor 2014
Disallowing https Can this be done via robots.txt? yosmc msg:1528083 1:36 am on Apr 8, 2005 (gmt 0) I'm looking for a way to keep the search engines from crawling the https versions of my html pages. Currently my best bet is to try it via mod_rewrite, but maybe there's an easier way to do it via robots.txt? Thanks in advance!
Lord Majestic msg:1528084 8:45 pm on Apr 9, 2005 (gmt 0)
Yes you can do that, but make sure that robots.txt is located in root of https://www.example.com