Forum Moderators: open

Message Too Old, No Replies

excluding https from bots through robots.txt

         

Crush

6:39 pm on Apr 4, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Noticed a lot of https pages had been indexed and g is getting confused.

What is the best way to exclude https pages on an IIS server if we do not have a seperate folder for the https: . The site is all done in asapi_rewrite

What we did is:

[ourdomain.com...]

User-agent: *
Disallow: /

I am worries that bots might ignore http as well :0/

Lord Majestic

6:42 pm on Apr 4, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Make robots.txt a script which would return appropriate text depending on whether protocol of request was HTTP or HTTPS.