Welcome to WebmasterWorld Guest from 3.81.29.226

Forum Moderators: Ocean10000 & phranque

Message Too Old, No Replies

Newbie Needs .htaccess advice

Server throws a "wobbly"

     
1:00 pm on Dec 15, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Nov 21, 2003
posts:53
votes: 0


I recently installed some .htaccess rewrites with the purpose of converting long php urls into search engine friendly ones.

This has worked extremely well with the exception of a "teathing" problem.

Every so often the server throws up a 500 error. I've checked the error log and this seems to be the culprit:

[Mon Dec 15 01:59:50 2003] [alert] [client ****.****.xxx.xxx] /www/virtual/mysitename/www.mysitename.com/.htaccess: RewriteCond: cannot compile regular expression '^Alexibot'

Is it that something is over-running my server and causing it to run out of cachse or something similar?

Thank you kindly in anticipation.

This is my htaccess:

# $Author: zx $
# $Date: 2003/08/17 14:03:21 $

# deny most common except .php
<FilesMatch "\.(inc¦tpl¦h¦ihtml¦sql¦ini¦conf¦class¦bin¦spd¦theme¦module)$">
deny from all
</FilesMatch>

<Limit GET PUT POST>
Order Allow,Deny
Allow from all
</Limit>

RewriteEngine on

#The next lines check for Email Spammers Robots and redirect them to a fake page
RewriteCond %{HTTP_USER_AGENT} ^Alexibot [OR]
RewriteCond %{HTTP_USER_AGENT} ^asterias [OR]
RewriteCond %{HTTP_USER_AGENT} ^BackDoorBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^Black.Hole [OR]
RewriteCond %{HTTP_USER_AGENT} ^BlackWidow [OR]
RewriteCond %{HTTP_USER_AGENT} ^BlowFish [OR]
RewriteCond %{HTTP_USER_AGENT} ^BotALot [OR]
RewriteCond %{HTTP_USER_AGENT} ^BuiltBotTough [OR]
RewriteCond %{HTTP_USER_AGENT} ^Bullseye [OR]
RewriteCond %{HTTP_USER_AGENT} ^BunnySlippers [OR]
RewriteCond %{HTTP_USER_AGENT} ^Cegbfeieh [OR]
RewriteCond %{HTTP_USER_AGENT} ^CheeseBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^CherryPicker [OR]
RewriteCond %{HTTP_USER_AGENT} ^ChinaClaw [OR]
RewriteCond %{HTTP_USER_AGENT} ^CopyRightCheck [OR]
RewriteCond %{HTTP_USER_AGENT} ^cosmos [OR]
RewriteCond %{HTTP_USER_AGENT} ^Crescent [OR]
RewriteCond %{HTTP_USER_AGENT} ^Custo [OR]
RewriteCond %{HTTP_USER_AGENT} ^DISCo [OR]
RewriteCond %{HTTP_USER_AGENT} ^DittoSpyder [OR]
RewriteCond %{HTTP_USER_AGENT} ^Download\ Demon [OR]
RewriteCond %{HTTP_USER_AGENT} ^eCatch [OR]
RewriteCond %{HTTP_USER_AGENT} ^EirGrabber [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailCollector [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailSiphon [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailWolf [OR]
RewriteCond %{HTTP_USER_AGENT} ^EroCrawler [OR]
RewriteCond %{HTTP_USER_AGENT} ^Express\ WebPictures [OR]
RewriteCond %{HTTP_USER_AGENT} ^ExtractorPro [OR]
RewriteCond %{HTTP_USER_AGENT} ^EyeNetIE [OR]
RewriteCond %{HTTP_USER_AGENT} ^FlashGet [OR]
RewriteCond %{HTTP_USER_AGENT} ^Foobot [OR]
RewriteCond %{HTTP_USER_AGENT} ^FrontPage [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetWeb! [OR]
RewriteCond %{HTTP_USER_AGENT} ^Go-Ahead-Got-It [OR]
RewriteCond %{HTTP_USER_AGENT} ^Googlebot-Image [OR]
RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla [OR]
RewriteCond %{HTTP_USER_AGENT} ^GrabNet [OR]
RewriteCond %{HTTP_USER_AGENT} ^Grafula [OR]
RewriteCond %{HTTP_USER_AGENT} ^Harvest [OR]
RewriteCond %{HTTP_USER_AGENT} ^hloader [OR]
RewriteCond %{HTTP_USER_AGENT} ^HMView [OR]
RewriteCond %{HTTP_USER_AGENT} ^httplib [OR]
RewriteCond %{HTTP_USER_AGENT} ^HTTrack [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^humanlinks [OR]
RewriteCond %{HTTP_USER_AGENT} ^ia_archiver [OR]
RewriteCond %{HTTP_USER_AGENT} ^Image\ Stripper [OR]
RewriteCond %{HTTP_USER_AGENT} ^Image\ Sucker [OR]
RewriteCond %{HTTP_USER_AGENT} ^Indy\ Library [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^InfoNaviRobot [OR]
RewriteCond %{HTTP_USER_AGENT} ^InterGET [OR]
RewriteCond %{HTTP_USER_AGENT} ^Internet\ Ninja [OR]
RewriteCond %{HTTP_USER_AGENT} ^JennyBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^JetCar [OR]
RewriteCond %{HTTP_USER_AGENT} ^JOC\ Web\ Spider [OR]
RewriteCond %{HTTP_USER_AGENT} ^Kenjin.Spider [OR]
RewriteCond %{HTTP_USER_AGENT} ^Keyword.Density [OR]
RewriteCond %{HTTP_USER_AGENT} ^larbin [OR]
RewriteCond %{HTTP_USER_AGENT} ^LeechFTP [OR]
RewriteCond %{HTTP_USER_AGENT} ^LexiBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^libWeb/clsHTTP [OR]
RewriteCond %{HTTP_USER_AGENT} ^LinkextractorPro [OR]
RewriteCond %{HTTP_USER_AGENT} ^LinkScan/8.1a.Unix [OR]
RewriteCond %{HTTP_USER_AGENT} ^LinkWalker [OR]
RewriteCond %{HTTP_USER_AGENT} ^lwp-trivial [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mass\ Downloader [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mata.Hari [OR]
RewriteCond %{HTTP_USER_AGENT} ^Microsoft.URL [OR]
RewriteCond %{HTTP_USER_AGENT} ^MIDown\ tool [OR]
RewriteCond %{HTTP_USER_AGENT} ^MIIxpc [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mister.PiX [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mister\ PiX [OR]
RewriteCond %{HTTP_USER_AGENT} ^moget [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mozilla/2 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mozilla/3.Mozilla/2.01 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mozilla.*NEWT [OR]
RewriteCond %{HTTP_USER_AGENT} ^Navroad [OR]
RewriteCond %{HTTP_USER_AGENT} ^NearSite [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetAnts [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetMechanic [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetSpider [OR]
RewriteCond %{HTTP_USER_AGENT} ^Net\ Vampire [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetZIP [OR]
RewriteCond %{HTTP_USER_AGENT} ^NICErsPRO [OR]
RewriteCond %{HTTP_USER_AGENT} ^NPBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^Octopus [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline.Explorer [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Explorer [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Navigator [OR]
RewriteCond %{HTTP_USER_AGENT} ^Openfind [OR]
RewriteCond %{HTTP_USER_AGENT} ^PageGrabber [OR]
RewriteCond %{HTTP_USER_AGENT} ^Papa\ Foto [OR]
RewriteCond %{HTTP_USER_AGENT} ^pavuk [OR]
RewriteCond %{HTTP_USER_AGENT} ^pcBrowser [OR]
RewriteCond %{HTTP_USER_AGENT} ^ProPowerBot/2.14 [OR]
RewriteCond %{HTTP_USER_AGENT} ^ProWebWalker [OR]
RewriteCond %{HTTP_USER_AGENT} ^ProWebWalker [OR]
RewriteCond %{HTTP_USER_AGENT} ^QueryN.Metasearch [OR]
RewriteCond %{HTTP_USER_AGENT} ^ReGet [OR]
RewriteCond %{HTTP_USER_AGENT} ^RepoMonkey [OR]
RewriteCond %{HTTP_USER_AGENT} ^RMA [OR]
RewriteCond %{HTTP_USER_AGENT} ^SiteSnagger [OR]
RewriteCond %{HTTP_USER_AGENT} ^SlySearch [OR]
RewriteCond %{HTTP_USER_AGENT} ^SmartDownload [OR]
RewriteCond %{HTTP_USER_AGENT} ^SpankBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^spanner [OR]
RewriteCond %{HTTP_USER_AGENT} ^SuperBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^SuperHTTP [OR]
RewriteCond %{HTTP_USER_AGENT} ^Surfbot [OR]
RewriteCond %{HTTP_USER_AGENT} ^suzuran [OR]
RewriteCond %{HTTP_USER_AGENT} ^Szukacz/1.4 [OR]
RewriteCond %{HTTP_USER_AGENT} ^tAkeOut [OR]
RewriteCond %{HTTP_USER_AGENT} ^Teleport [OR]
RewriteCond %{HTTP_USER_AGENT} ^Teleport\ Pro [OR]
RewriteCond %{HTTP_USER_AGENT} ^Telesoft [OR]
RewriteCond %{HTTP_USER_AGENT} ^The.Intraformant [OR]
RewriteCond %{HTTP_USER_AGENT} ^TheNomad [OR]
RewriteCond %{HTTP_USER_AGENT} ^TightTwatBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^Titan [OR]
RewriteCond %{HTTP_USER_AGENT} ^toCrawl/UrlDispatcher [OR]
RewriteCond %{HTTP_USER_AGENT} ^toCrawl/UrlDispatcher [OR]
RewriteCond %{HTTP_USER_AGENT} ^True_Robot [OR]
RewriteCond %{HTTP_USER_AGENT} ^turingos [OR]
RewriteCond %{HTTP_USER_AGENT} ^TurnitinBot/1.5 [OR]
RewriteCond %{HTTP_USER_AGENT} ^URLy.Warning [OR]
RewriteCond %{HTTP_USER_AGENT} ^VCI [OR]
RewriteCond %{HTTP_USER_AGENT} ^VoidEYE [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebAuto [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebBandit [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebCopier [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebEMailExtrac.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebEnhancer [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebFetch [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebGo\ IS [OR]
RewriteCond %{HTTP_USER_AGENT} ^Web.Image.Collector [OR]
RewriteCond %{HTTP_USER_AGENT} ^Web\ Image\ Collector [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebLeacher [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebmasterWorldForumBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebReaper [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebSauger [OR]
RewriteCond %{HTTP_USER_AGENT} ^Website\ eXtractor [OR]
RewriteCond %{HTTP_USER_AGENT} ^Website.Quester [OR]
RewriteCond %{HTTP_USER_AGENT} ^Website\ Quester [OR]
RewriteCond %{HTTP_USER_AGENT} ^Webster.Pro [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebStripper [OR]
RewriteCond %{HTTP_USER_AGENT} ^Web\ Sucker [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebWhacker [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZip [OR]
RewriteCond %{HTTP_USER_AGENT} ^Wget [OR]
RewriteCond %{HTTP_USER_AGENT} ^Widow [OR]
RewriteCond %{HTTP_USER_AGENT} ^[Ww]eb[Bb]andit [OR]
RewriteCond %{HTTP_USER_AGENT} ^WWW-Collector-E [OR]
RewriteCond %{HTTP_USER_AGENT} ^WWWOFFLE [OR]
RewriteCond %{HTTP_USER_AGENT} ^Xaldon\ WebSpider [OR]
RewriteCond %{HTTP_USER_AGENT} ^Xenu's [OR]
RewriteCond %{HTTP_USER_AGENT} ^Zeus
RewriteRule ^.* - [F,L]

#RewriteCond %{HTTP_USER_AGENT} ^-?$
#RewriteRule ^.*$ /www/nukecops/noID.php [L]

RewriteRule /index.html /index.php

#Articles
RewriteRule ^article-([0-9]*)-([a-z]*)-([0-9]*)-([0-9]*).html modules.php?name=News&file=article&sid=$1&mode=$2&order=$3&thold=$4
RewriteRule ^article([1-9][0-9]*).* modules.php?name=News&file=article&sid=$1
RewriteRule ^article-topic-([0-9]*).html modules.php?name=News&new_topic=$1
RewriteRule ^allnews.html modules.php?name=News&file=allindex

#Content
RewriteRule ^content-([0-9]*)-page([0-9]*).html modules.php?name=Content&pa=showpage&pid=$1&page=$2
RewriteRule ^content-cat-([0-9]*).html modules.php?name=Content&pa=list_pages_categories&cid=$1
RewriteRule ^content-([0-9]*).html modules.php?name=Content&pa=showpage&pid=$1
RewriteRule ^content.html modules.php?name=Content

#FAQ
RewriteRule ^faq.html modules.php?name=FAQ
RewriteRule ^faq-([0-9]*)-([a-zA-Z0-9\+\&\-\/%[:punct:]\.\;]*).html modules.php?name=FAQ&myfaq=yes&id_cat=$1&categories=$2

#Forums
RewriteRule ^forumx([0-9]+)-([0-9]+)-([0-9]+).html modules.php?name=Forums&file=viewforum&f=$1&topicdays=$2&start=$3
RewriteRule ^forum([1-9][0-9]*)-(mark¦start)([a-zA-Z0-9_]*).html modules.php?name=Forums&file=viewforum&f=$1&$2=$3
RewriteRule ^forum([1-9][0-9]*).* modules.php?name=Forums&file=viewforum&f=$1
RewriteRule ^postx([0-9]+)-([0-9]+)-([0-9]+).html modules.php?name=Forums&file=viewtopic&t=$1&start=$2&postdays=$3&postorder=asc&highlight=
RewriteRule ^postxf([0-9]+)-([0-9]+)-([0-9]+).html modules.php?name=Forums&file=viewtopic&t=$1&postdays=$2&postorder=asc&start=$3
RewriteRule ^posts([0-9]*)-(start¦view¦highlight)([[:punct:]/:\-\'(){}.&=_a-zA-Z0-9\ ]*).html modules.php?name=Forums&file=viewtopic&t=$1&$2=$3
RewriteRule ^topic([1-9][0-9]*)-([a-zA-Z0-9]*).html modules.php?name=Forums&file=viewtopic&t=$1&$2=topic&start=0
RewriteRule ^post(p¦t)([1-9][0-9]*).* modules.php?name=Forums&file=viewtopic&$1=$2
RewriteRule ^register-coppa-([a-zA-Z0-9_]*).html modules.php?name=Forums&file=profile&mode=register&agreed=true&coppa=$1
RewriteRule ^register-([a-zA-Z0-9_]*).html modules.php?name=Forums&file=profile&mode=register&agreed=$1
RewriteRule ^poster([a-zA-Z0-9_]*)-([0-9_&]*)_([a-zA-Z0-9_&=[:punct:]\.\;]*).html modules.php?name=Forums&file=modcp&mode=$1&p=$2&t=$3
RewriteRule ^modcp([0-9]*)-([a-zA-Z0-9]*).html modules.php?name=Forums&file=modcp&t=$1&amp;mode=$2
RewriteRule ^forums-([a-zA-Z0-9_]*)-([a-zA-Z0-9_]*)-([0-9]*)-([1-9][0-9]*).html modules.php?name=Forums&file=$1&mode=$2&t=$3&popup=$4
RewriteRule ^forums-([a-zA-Z0-9_]*)-([a-zA-Z0-9_]*)-(p¦t¦f¦u¦popup)-([0-9]*).html modules.php?name=Forums&file=$1&mode=$2&$3=$4
RewriteRule ^forums-([a-zA-Z0-9_]*)-([a-zA-Z0-9_]*).html modules.php?name=Forums&file=$1&mode=$2
RewriteRule ^forum([a-zA-Z0-9_]*)-([a-zA-Z0-9_]*)-([a-zA-Z0-9_]*)-([a-zA-Z0-9_]*).html modules.php?name=Forums&file=search&$1=$2&$3=$4
RewriteRule ^forums([a-zA-Z0-9_]*)-([a-zA-Z0-9_]*)-([a-zA-Z0-9_]*).html modules.php?name=Forums&file=$1&$2=$3
RewriteRule ^forums.html modules.php?name=Forums&file=index
RewriteRule ^forum-([a-zA-Z0-9_]*).html modules.php?name=Forums&file=$1
RewriteRule ^forums.html modules.php?name=Forums

#Downloads
RewriteRule ^downloadview-(comments¦details¦editorial)-([0-9]*)-([[:punct:]/:\-\'{}()._&a-zA-Z0-9+=]*).html modules.php?name=Downloads&d_op=viewdownload$1&lid=$2&ttitle=$3
RewriteRule ^download-mod-([0-9]*).html modules.php?name=Downloads&d_op=modifydownloadrequest&lid=$1
RewriteRule ^download-broken-([0-9]*).html modules.php?name=Downloads&d_op=brokendownload&lid=$1
RewriteRule ^download-shownew-([0-9]*).html modules.php?name=Downloads&d_op=NewDownloads&newdownloadshowdays=$1
RewriteRule ^download-seldate-([a-zA-Z0-9+]*).html modules.php?name=Downloads&d_op=NewDownloadsDate&selectdate=$1
RewriteRule ^download-file-([0-9]*).html modules.php?name=Downloads&d_op=getit&lid=$1
RewriteRule ^ratedownload-([0-9]*)-([[:punct:]/:\-\'{}()._&a-zA-Z0-9+=]*).html modules.php?name=Downloads&d_op=ratedownload&lid=$1&ttitle=$2
RewriteRule ^ratedownload-([0-9]*).html modules.php?name=Downloads&d_op=ratedownload&lid=$1
RewriteRule ^download-(TopRated¦MostPopular)-([0-9]*)-(num¦percent).html modules.php?name=Downloads&d_op=$1&ratenum=$2&ratetype=$3
RewriteRule ^downloads-(AddDownload¦NewDownloads¦MostPopular¦TopRated).html modules.php?name=Downloads&d_op=$1
RewriteRule ^download([0-9]*)-orderby([a-zA-Z0-9+]*).html modules.php?name=Downloads&d_op=viewdownload&cid=$1&orderby=$2
RewriteRule ^download([0-9]*)-([0-9]*)-([a-zA-Z0-9+]*)-([0-9]*).html modules.php?name=Downloads&d_op=viewdownload&cid=$1&min=$2&orderby=$3&show=$4
RewriteRule ^download-search-([[:punct:]/:\-\'{}()._&a-zA-Z0-9+=]*)-([0-9]*)-([a-zA-Z0-9+]*)-([0-9]*).html modules.php?name=Downloads&d_op=search&query=$1&min=$2&orderby=$3&show=$4
RewriteRule ^download-search-([[:punct:]/:\-\'{}()._&a-zA-Z0-9+=]*)-([a-zA-Z0-9+]*).html modules.php?name=Downloads&d_op=search&query=$1&orderby=$2
RewriteRule ^download-search-([[:punct:]/:\-\'{}()._&a-zA-Z0-9+=]*).html modules.php?name=Downloads&d_op=search&query=$1
RewriteRule ^download-outside-([0-9]*).html modules.php?name=Downloads&d_op=outsidedownloadsetup&lid=$1
RewriteRule ^downloads-cat([0-9]*).html modules.php?name=Downloads&d_op=viewdownload&cid=$1
RewriteRule ^downloads.html modules.php?name=Downloads

#Journal
RewriteRule ^journal-search-([a-zA-Z0-9]*).html modules.php?name=Journal&file=search&bywhat=aid&exact=1&forwhat=$1
RewriteRule ^journal(search¦delete¦commentkill)-(bywhat¦jid¦onwhat)([a-zA-Z0-9]*)-(forwhat¦ref)([a-zA-Z0-9]*).html modules.php?name=Journal&file=$1&$2=$3&$4=$5
RewriteRule ^journal-([a-zA-Z0-9+]*)-(jid¦onwhat¦disp¦op¦disp)-([a-zA-Z0-9+]*).html modules.php?name=Journal&file=$1&$2=$3
RewriteRule ^journal(file¦op)-([a-zA-Z0-9]*).html modules.php?name=Journal&$1=$2
RewriteRule ^journal.html modules.php?name=Journal

#Stories Archive
RewriteRule ^archive.html modules.php?name=Stories_Archive
RewriteRule ^archive-([0-9]*)-([0-9]*)-([a-zA-Z]*).* modules.php?name=Stories_Archive&sa=show_month&year=$1&month=$2&month_l=$3

#Reviews
RewriteRule ^reviews-new.html modules.php?name=Reviews&rop=write_review
RewriteRule ^reviews-preview.html modules.php?name=Reviews&rop=preview_review
RewriteRule ^reviews-([0-9]*)-page([0-9]*).html modules.php?name=Reviews&rop=showcontent&id=$1&page=$2
RewriteRule ^reviews-([0-9]*).html modules.php?name=Reviews&rop=showcontent&id=$1
RewriteRule ^reviews-comment-([0-9]*)-([[:punct:]?/:\-\'{}()._&a-zA-Z0-9+=\%]*).* modules.php?name=Reviews&rop=postcomment&id=$1&title=$2
RewriteRule ^reviews-([0-9]*)-delete.html modules.php?name=Reviews&rop=del_review&id_del=$1
RewriteRule ^reviews-([0-9]*)-edit.html modules.php?name=Reviews&rop=mod_review&id=$1
RewriteRule ^reviews-([0-9]*)-delcomment-([0-9]*).html modules.php?name=Reviews&rop=del_comment&cid=$1&id=$2
RewriteRule ^reviews-([a-zA-Z0-9]*)-orderby-([a-z]*)-([a-zA-Z]*).html modules.php?name=Reviews&rop=$1&field=$2&order=$3
RewriteRule ^reviews-sortby-([a-zA-Z0-9]*).html modules.php?name=Reviews&rop=$1
RewriteRule ^reviews.html modules.php?name=Reviews

#Sections
RewriteRule ^sections-viewarticle([0-9]*)-page([0-9]*).html modules.php?name=Sections&op=viewarticle&artid=$1&page=$2
RewriteRule ^sections-(printpage¦viewarticle)-([0-9]*).html modules.php?name=Sections&op=$1&artid=$2
RewriteRule ^sections-listarticles-([0-9]*).html modules.php?name=Sections&op=listarticles&secid=$1
RewriteRule ^sections.html modules.php?name=Sections

#Statistics
RewriteRule ^stats-([0-9]*)-([0-9]*)-([0-9]*).html modules.php?name=Statistics&op=DailyStats&year=$1&month=$2&date=$3
RewriteRule ^stats-([0-9]*)-([0-9]*).html modules.php?name=Statistics&op=MonthlyStats&year=$1&month=$2
RewriteRule ^stats-([0-9]*).html modules.php?name=Statistics&op=YearlyStats&year=$1
RewriteRule ^advstats.html modules.php?name=Statistics&op=Stats
RewriteRule ^stats.html modules.php?name=Statistics

#Surveys
RewriteRule ^surveys.html modules.php?name=Surveys
RewriteRule ^survey([a-zA-Z0-9]*)results.html modules.php?name=Surveys&op=results&pollID=$1
RewriteRule ^survey([a-zA-Z0-9]*).html modules.php?name=Surveys&pollID=$1

#Submit
RewriteRule ^submit.html modules.php?name=Submit_News

#Search
#RewriteRule ^search.html modules.php?name=Search

#Top10
RewriteRule ^top-([[:print:]]*).html modules.php?name=Top&zx=$1
RewriteRule ^top.html modules.php?name=Top

#Topics
RewriteRule ^topics.html modules.php?name=Topics

#Your Account
RewriteRule ^userinfo-([a-zA-Z0-9_-]*).html modules.php?name=Your_Account&op=userinfo&username=$1

#WebLinks
RewriteRule ^viewlinkcomments-([0-9]+)-([[:punct:]/:\-\'(){}.&=_a-zA-Z0-9\ ]*).html modules.php?name=Web_Links&l_op=viewlinkcomments&lid=$1&ttitle=$2
RewriteRule ^viewlinkdetails-([0-9]+)-([[:punct:]/:\-\'(){}.&=_a-zA-Z0-9\ ]*).html modules.php?name=Web_Links&l_op=viewlinkdetails&lid=$1&ttitle=$2
RewriteRule ^vieweditorial-([0-9]+)-([[:punct:]/:\-\'(){}.&=_a-zA-Z0-9\ ]*).html modules.php?name=Web_Links&l_op=viewlinkeditorial&lid=$1&ttitle=$2
RewriteRule ^modifylink-([0-9]+).html modules.php?name=Web_Links&l_op=modifylinkrequest&lid=$1
RewriteRule ^brokenlink-([0-9]+).html modules.php?name=Web_Links&l_op=brokenlink&lid=$1
RewriteRule ^outsidelink-([0-9]+).html modules.php?name=Web_Links&l_op=outsidelinksetup&lid=$1
RewriteRule ^linkop-(MostPopular¦TopRated)-([0-9]+)-(num¦percent).html modules.php?name=Web_Links&l_op=$1&ratenum=$2&ratetype=$3
RewriteRule ^newlinks-([0-9]+).html modules.php\?name=Web_Links&l_op=NewLinks&newlinkshowdays=$1
RewriteRule ^linksnew-([0-9]+).html modules.php?op=modload&name=Web_Links&file=index&l_op=NewLinksDate&selectdate=$1
RewriteRule ^linkop-(AddLink¦MostPopular¦NewLinks¦RandomLink¦TopRated).html modules.php?op=modload&name=Web_Links&file=index&l_op=$1
RewriteRule ^ratelink-([0-9]*)-([[:punct:]/:\-\'(){}.&=_a-zA-Z0-9\ ]*).html modules.php?op=modload&name=Web_Links&file=index&l_op=ratelink&lid=$1&ttitle=$2
RewriteRule ^viewlink-([0-9]*).html modules.php?name=Web_Links&l_op=visit&lid=$1
RewriteRule ^links-([0-9]*)-([a-zA-Z0-9]*).html modules.php?name=Web_Links&file=index&l_op=viewlink&cid=$1&orderby=$2
RewriteRule ^links-([0-9]*)-([0-9]*)-([a-zA-Z0-9]*)-([0-9]*).html modules.php?name=Web_Links&file=index&l_op=viewlink&cid=$1&min=$2&orderby=$3&show=$4
RewriteRule ^link-([0-9]*).html modules.php?name=Web_Links&l_op=viewlink&cid=$1
RewriteRule ^links-search-([a-zA-Z0-9]*).html modules.php?name=Web_Links&l_op=search&query=$1
RewriteRule ^links-search-([a-zA-Z0-9]*)-orderby-([a-zA-Z]*).html modules.php?name=Web_Links&l_op=search&query=$1&orderby=$2
RewriteRule ^links-search-([a-zA-Z0-9]*)-([0-9]*)-orderby-([a-zA-Z]*)-([0-9]*) modules.php?name=Web_Links&l_op=search&query=$1&min=$2&orderby=$3&show=$4
RewriteRule ^links.html modules.php?name=Web_Links

3:55 pm on Dec 15, 2003 (gmt 0)

New User

10+ Year Member

joined:Dec 15, 2003
posts:15
votes: 0


I've also considered using a very long htaccess file like that to block annoying spambots / email harvesters for example, but, and I didn´t really research this any further, wouldn´t a HUGE file like that cause a big performance hit?

I know it´s not an answer to your question but it´d be interesting for you as well while we´re on this topic.

5:58 pm on Dec 15, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Mar 31, 2002
posts:25430
votes: 0


Bridge,

Like most folks here, I can't afford the time to read and analyze that whole file - it is simply too long and unfamiliar.

But here are some things to try:

Narrow down the problem - Take a look at your raw log file and extract the actual request that caused the 500 error. What page was it requesting, and using what HTTP method?

Try swapping the order of Alexibot and the next user-agent string in the list. Does the 500-Error problem follow the user-agent, or stay on the first RewriteRule?

Place a copy of your .htaccess in a new 'test' subdirectory. Use a user-agent spoofing tool to pretend you are Alexibot and request pages from your site. What error(s) do you get? Comment out large code sections and see if anything changes. Isolate and reduce problems.

Comment:
I would recommend that you add an [L] flag to all (or most) of the RewriteRules in your file. This will stop mod_rewrite processing for the current request after a RewriteRule pattern matches and the URL has been rewritten. Otherwise, mod_rewrite will do the rewrite, and then continue on to try to match the new URL to the patterns in the following RewriteRules. In most cases, this is not needed, and is a great waste of time and CPU cycles. The [L] flag is documented in the Apache modrewrite RewriteRule directive section.

.

Profiler,
The code could be made more efficient for .htaccess use by combining RewriteCond patterns on one line using (user-agent1¦useragent2¦user-agent3) and this is a good idea once you know which user-agents are real problems. Initially, though, it's easier to maintain if you go one per line. Surprisingly, for use in httpd.conf, the one-per-line approach is fastest. This is due to the difference in mod_rewrite rule processing -- interpreted vs. compiled -- in .htaccess and httpd.conf respectively.

That said, the performance hit you see with most .htaccess files is completely swamped by other factors such as network transmission delays, server disk access, server memory page-swapping delays, script parsing delays, etc. Mod-Rewrite and the other modules are fairly-well-optimized code, and are 'wired in' directly to the Apache server code, so they are usually quite efficient compared to external scripts that must access server resources through calls to the operating system.

Once a site has been running awhile it *is* a good idea to go through lists like the one above, and drop the ones that never visit or that only show up infrequently. They can be stopped by other means if necessary, rather than having the server constantly testing requests against their user-agent name.

Jim

8:21 am on Dec 16, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Nov 21, 2003
posts:53
votes: 0


Thanks.

I will be trying out your recommendations over the next couple of days and will inform of the outcome.

Nice one!