Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

Disallowing https

Can this be done via robots.txt?

1:36 am on Apr 8, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Feb 8, 2003
votes: 0

I'm looking for a way to keep the search engines from crawling the https versions of my html pages. Currently my best bet is to try it via mod_rewrite, but maybe there's an easier way to do it via robots.txt? Thanks in advance!
8:45 pm on Apr 9, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Aug 8, 2004
votes: 0

Yes you can do that, but make sure that robots.txt is located in root of [example.com...]