Welcome to WebmasterWorld Guest from 54.225.20.73

Forum Moderators: Ocean10000 & incrediBILL & phranque

Message Too Old, No Replies

Way of using .htaccess to only allow some bots ?

     
8:11 pm on Jan 18, 2011 (gmt 0)

Preferred Member

10+ Year Member

joined:Nov 26, 2004
posts: 394
votes: 0


Using .htaccess is there any way of only allowing the legit robots to visit please?

Any help much appreciated.

Dexie
8:17 pm on Jan 18, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2002
posts:18903
votes: 0


Yes, but you will have to define what you consider to be legit - by UA and by IP.

Armed with that information you can start to craft some rules, and latterly some code.
8:59 pm on Jan 18, 2011 (gmt 0)

Preferred Member

10+ Year Member

joined:Nov 26, 2004
posts: 394
votes: 0


Thanks g1smd - which bots would generally be considered as legit please? Google, MSN, Yahoo, any others?
10:37 pm on Jan 18, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Aug 29, 2006
posts:1312
votes: 0


which bots would generally be considered as legit please?

Those you wish to allow - each webmaster has their own preferences.

There is no quick fix, you will need to study discussions in the Spider Forum:

[webmasterworld.com...]

...
7:34 am on Jan 19, 2011 (gmt 0)

Preferred Member

10+ Year Member

joined:Nov 26, 2004
posts: 394
votes: 0


Thanks, will take a look there.
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members