homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Code, Content, and Presentation / Apache Web Server
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL & phranque

Apache Web Server Forum

Need to redirect non-spiders with htaccess

10+ Year Member

Msg#: 471 posted 6:11 am on Oct 21, 2003 (gmt 0)

I have a script that creates a text only version of my Perl based forum for robots to spider.

I am currently using a javascript redirect with a meta refresh in the <no script> tag to send visitors to the real board.

I know Google and maybe others do not like pages that redirect, so I want to use ModRewrite to redirect everyone except top bots to the real forum.

While I have a basic knowledge of ModRewrite, I'm not up to this. Any help is appreciated.



WebmasterWorld Senior Member 10+ Year Member

Msg#: 471 posted 3:00 pm on Oct 21, 2003 (gmt 0)

Generally, we prefer to help people DEBUG their mod_rewrite code at WebmasterWorld, not write it for them. Although someone may jump in here and offer a solution, you could start with this Intro to mod_rewrite [webmasterworld.com] (and follow the links it includes if it's too basic) or try a post in the Commercial Exchange forum.


10+ Year Member

Msg#: 471 posted 3:14 pm on Oct 21, 2003 (gmt 0)

hi guillermo5000.

i see that you write perl scripts. i loaded shed loads of spider c-blocks into my database then wrote a script that checked if the visitor was a spider or not. depending on this check you can serve different content.

you can maintain a table of visitorips which you can purge say every half an hour.

i actually used this for logging purposes rather than serving different content but the principle is the same. i now have a system where i have visits and pageviews for both humans and spiders.

why not implement something like this in perl? just remember to update your spider list every now and then.


10+ Year Member

Msg#: 471 posted 5:31 pm on Oct 21, 2003 (gmt 0)

incy: Thanks, but I do not write perl. I just use a 3rd party board that is written in Perl.

Dave: Thank you. I'll look elsewhere.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Code, Content, and Presentation / Apache Web Server
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved