Welcome to WebmasterWorld Guest from 54.226.133.245

Forum Moderators: Ocean10000 & incrediBILL & phranque

Message Too Old, No Replies

Need to redirect non-spiders with htaccess

     
6:11 am on Oct 21, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:May 16, 2003
posts:109
votes: 0


I have a script that creates a text only version of my Perl based forum for robots to spider.

I am currently using a javascript redirect with a meta refresh in the <no script> tag to send visitors to the real board.

I know Google and maybe others do not like pages that redirect, so I want to use ModRewrite to redirect everyone except top bots to the real forum.

While I have a basic knowledge of ModRewrite, I'm not up to this. Any help is appreciated.

3:00 pm on Oct 21, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Sept 21, 1999
posts:2141
votes: 0


Generally, we prefer to help people DEBUG their mod_rewrite code at WebmasterWorld, not write it for them. Although someone may jump in here and offer a solution, you could start with this Intro to mod_rewrite [webmasterworld.com] (and follow the links it includes if it's too basic) or try a post in the Commercial Exchange forum.
3:14 pm on Oct 21, 2003 (gmt 0)

Full Member

10+ Year Member

joined:Feb 5, 2002
posts:333
votes: 0


hi guillermo5000.

i see that you write perl scripts. i loaded shed loads of spider c-blocks into my database then wrote a script that checked if the visitor was a spider or not. depending on this check you can serve different content.

you can maintain a table of visitorips which you can purge say every half an hour.

i actually used this for logging purposes rather than serving different content but the principle is the same. i now have a system where i have visits and pageviews for both humans and spiders.

why not implement something like this in perl? just remember to update your spider list every now and then.

5:31 pm on Oct 21, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:May 16, 2003
posts:109
votes: 0


incy: Thanks, but I do not write perl. I just use a 3rd party board that is written in Perl.

Dave: Thank you. I'll look elsewhere.