Welcome to WebmasterWorld Guest from 54.144.243.34

Forum Moderators: Ocean10000 & incrediBILL & phranque

Need to redirect non-spiders with htaccess

   
6:11 am on Oct 21, 2003 (gmt 0)

10+ Year Member



I have a script that creates a text only version of my Perl based forum for robots to spider.

I am currently using a javascript redirect with a meta refresh in the <no script> tag to send visitors to the real board.

I know Google and maybe others do not like pages that redirect, so I want to use ModRewrite to redirect everyone except top bots to the real forum.

While I have a basic knowledge of ModRewrite, I'm not up to this. Any help is appreciated.

3:00 pm on Oct 21, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Generally, we prefer to help people DEBUG their mod_rewrite code at WebmasterWorld, not write it for them. Although someone may jump in here and offer a solution, you could start with this Intro to mod_rewrite [webmasterworld.com] (and follow the links it includes if it's too basic) or try a post in the Commercial Exchange forum.
3:14 pm on Oct 21, 2003 (gmt 0)

10+ Year Member



hi guillermo5000.

i see that you write perl scripts. i loaded shed loads of spider c-blocks into my database then wrote a script that checked if the visitor was a spider or not. depending on this check you can serve different content.

you can maintain a table of visitorips which you can purge say every half an hour.

i actually used this for logging purposes rather than serving different content but the principle is the same. i now have a system where i have visits and pageviews for both humans and spiders.

why not implement something like this in perl? just remember to update your spider list every now and then.

5:31 pm on Oct 21, 2003 (gmt 0)

10+ Year Member



incy: Thanks, but I do not write perl. I just use a 3rd party board that is written in Perl.

Dave: Thank you. I'll look elsewhere.

 

Featured Threads

My Threads

Hot Threads This Week

Hot Threads This Month