homepage Welcome to WebmasterWorld Guest from 54.227.40.166
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Visit PubCon.com
Home / Forums Index / Code, Content, and Presentation / Apache Web Server
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL & phranque

Apache Web Server Forum

    
Way of using .htaccess to only allow some bots ?
Dexie




msg:4254801
 8:11 pm on Jan 18, 2011 (gmt 0)

Using .htaccess is there any way of only allowing the legit robots to visit please?

Any help much appreciated.

Dexie

 

g1smd




msg:4254808
 8:17 pm on Jan 18, 2011 (gmt 0)

Yes, but you will have to define what you consider to be legit - by UA and by IP.

Armed with that information you can start to craft some rules, and latterly some code.

Dexie




msg:4254829
 8:59 pm on Jan 18, 2011 (gmt 0)

Thanks g1smd - which bots would generally be considered as legit please? Google, MSN, Yahoo, any others?

Samizdata




msg:4254867
 10:37 pm on Jan 18, 2011 (gmt 0)

which bots would generally be considered as legit please?

Those you wish to allow - each webmaster has their own preferences.

There is no quick fix, you will need to study discussions in the Spider Forum:

[webmasterworld.com...]

...

Dexie




msg:4255032
 7:34 am on Jan 19, 2011 (gmt 0)

Thanks, will take a look there.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Code, Content, and Presentation / Apache Web Server
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved