Forum Moderators: goodroi

Message Too Old, No Replies

Combining htaccess and robots.txt?

         

Lady18

4:29 am on Oct 23, 2006 (gmt 0)

10+ Year Member



Please forgive my ignorance - I'm a complete novice at this stuff.

I just put a .htaccess file in the root of my file manager - meaning it's not in any other folder/directory, including public_html. It's all stuff like
"RewriteCond %{HTTP_USER_AGENT} ^BlackWidow [OR]
RewriteCond %{HTTP_USER_AGENT} ^Bot\ mailto:craftbot@yahoo.com [OR]"
etc.

What I'd like to know is: Can I add stuff from a suggested robots.txt file, like
"Disallow: /mywebsite/cgi-bin/
Disallow: /mywebsite/style_images/", etc.
to my current .htaccess? Or do I need a separate robots.txt file?

Either way, both should be in the root - meaning outside of public_html, right?

I read [webmasterworld.com...] ... .

Lady18

4:58 am on Oct 23, 2006 (gmt 0)

10+ Year Member



...it's not in any other folder/directory, including public_html.
I'm sorry! My .htaccess is inside the public-html folder, but should it be?

benevolent001

5:04 am on Oct 23, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Hello

.htaccess file and robotx should be kept separate

.htaccess file is for Apache Server
.htaccess file can be kept in any folder inside public_html or in that folder itself if you want to apply some rules to all folders inside public_html

if you want to apply some conditions in some specific folder only you can just move the htaccess file in that folder too

Robots.txt file tells the search engine to if you want to allow the website pages to be read by them? or are search engines allowed to ccess specific part of website

So you can see that both of the files have some specific functions and should be kept different

Lady18

5:33 am on Oct 23, 2006 (gmt 0)

10+ Year Member



Thanks much!

Lady18

9:33 pm on Oct 23, 2006 (gmt 0)

10+ Year Member



Just one more question: You're saying it's ok to have the same identical .htaccess file in the public_html folder

and in

public_html >> my site's folder

Right?

benevolent001

6:11 am on Oct 24, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



htaccess rules are applicable to all the files and sub folders in the location in which it is present

So its better to keep the htaccess file in your mysite folder if you are hosting multiple website in same public_html folder

Lady18

11:37 pm on Oct 24, 2006 (gmt 0)

10+ Year Member



Thanks much