homepage Welcome to WebmasterWorld Guest from 54.234.141.47
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Marketing and Biz Dev / Cloaking
Forum Library, Charter, Moderator: open

Cloaking Forum

    
Is there a way to block single keyword referrals from Live Search?
Marcia

WebmasterWorld Senior Member marcia us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 3577130 posted 10:23 am on Feb 17, 2008 (gmt 0)

There's no way that these "visitors" are arriving by finding the site(s) with those single word searches, and I have a strong suspicion that it would be best to block them altogether.

Is there a way to block those altogether from accessing a site? Or at least not going any further than the first page they hit?

 

volatilegx

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 3577130 posted 6:46 pm on Feb 17, 2008 (gmt 0)

You could always do some scripting to check for a microsoft live reference in the HTTP_REFERER header.

This could be done via mod_rewrite, and send such hits over to a script for further processing, such as adding their IP to a 403 ban list, etc.

Marcia

WebmasterWorld Senior Member marcia us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 3577130 posted 7:16 am on Feb 18, 2008 (gmt 0)

I don't want all from Live banned, just the ones coming in on one word searches, because they're scrapers.

jake66

5+ Year Member



 
Msg#: 3577130 posted 8:32 am on Feb 18, 2008 (gmt 0)

I use this (in .htaccess):
RewriteCond %{HTTP_REFERER} (milf) [NC,OR]
RewriteCond %{HTTP_REFERER} (x x x) [NC,OR]
RewriteRule .* - [F]

to block the referral spammers, which will work for you too if all of your people are sending referrers.

If I am not mistaken, I got this code a while back off webmasterworld. :)

Marcia

WebmasterWorld Senior Member marcia us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 3577130 posted 10:35 am on Feb 18, 2008 (gmt 0)

>>if all of your people are sending referrers

They aren't all unwelcome and not all are a PITA. Most visitors are 100% legitimate and more than welcome. If someone comes in from search using a search phrase such as "discount yellow widgets" or "yellow widget stores," then it's a living, breathing human being and they'll find what they're looking for. But if they arrive with the single search word "widgets" then it's software running for either data mining (no biggie) or to scrape for MFA sites, and the latter are a nuisance and represent a hazard for some sites if they go far enough, which many do.

I do know there's a way to sniff for s/e referral query strings and then process accordingly, but it was a long time ago, so my memory on it is very dim.

>>If I am not mistaken, I got this code a while back off webmasterworld.

Jake, I've been spending a lot of time "reading" in the code forums at WebmasterWorld (which I heartily recommend, rather than just copying and pasting), and unless I can "read" what something is doing, I won't use it until I know what it'll do.

RewriteCond %{HTTP_REFERER} (milf) [NC,OR]
RewriteCond %{HTTP_REFERER} (x x x) [NC,OR]
RewriteRule .* - [F]

Do (milf) and (x x x) represent what actual query words could be? If so, then that could do it.

[edited by: Marcia at 10:49 am (utc) on Feb. 18, 2008]

jake66

5+ Year Member



 
Msg#: 3577130 posted 11:26 am on Feb 18, 2008 (gmt 0)

milf and x x x are actual words I filter, the x's of course need to remove the spaces because WebmasterWorld filter the "word"..

And I do not copy/paste anything until I believe it won't break my site either. Only posted the reference as a working, live example that already works. :)

I use it myself, which is why I mentioned "I got this code a while back off webmasterworld" -- not to claim this as my work.

Believe me, I do not spend my time looking for ways to make myself look smart by answering questions I don't understand.

By the way you described your problem.. the remedy sounded as simple as the htaccess filter I use to block spam bots, that is the only reason I posted it.

Marcia

WebmasterWorld Senior Member marcia us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 3577130 posted 11:34 am on Feb 18, 2008 (gmt 0)

By the way you described your problem.. the remedy sounded as simple as the htaccess filter I use to block spam bots, that is the only reason I posted it.

If those are the actual query words, then that might just do the trick. I've got a fairly new domain not doing much of anything on other engines yet that gets those requests constantly, it sure couldn't hurt to try.

Thanks, will do!

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Marketing and Biz Dev / Cloaking
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved