Welcome to WebmasterWorld Guest from 54.226.146.15

Forum Moderators: open

Message Too Old, No Replies

Is there a way to block single keyword referrals from Live Search?

     

Marcia

10:23 am on Feb 17, 2008 (gmt 0)

WebmasterWorld Senior Member marcia is a WebmasterWorld Top Contributor of All Time 10+ Year Member



There's no way that these "visitors" are arriving by finding the site(s) with those single word searches, and I have a strong suspicion that it would be best to block them altogether.

Is there a way to block those altogether from accessing a site? Or at least not going any further than the first page they hit?

volatilegx

6:46 pm on Feb 17, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



You could always do some scripting to check for a microsoft live reference in the HTTP_REFERER header.

This could be done via mod_rewrite, and send such hits over to a script for further processing, such as adding their IP to a 403 ban list, etc.

Marcia

7:16 am on Feb 18, 2008 (gmt 0)

WebmasterWorld Senior Member marcia is a WebmasterWorld Top Contributor of All Time 10+ Year Member



I don't want all from Live banned, just the ones coming in on one word searches, because they're scrapers.

jake66

8:32 am on Feb 18, 2008 (gmt 0)

5+ Year Member



I use this (in .htaccess):
RewriteCond %{HTTP_REFERER} (milf) [NC,OR]
RewriteCond %{HTTP_REFERER} (x x x) [NC,OR]
RewriteRule .* - [F]

to block the referral spammers, which will work for you too if all of your people are sending referrers.

If I am not mistaken, I got this code a while back off webmasterworld. :)

Marcia

10:35 am on Feb 18, 2008 (gmt 0)

WebmasterWorld Senior Member marcia is a WebmasterWorld Top Contributor of All Time 10+ Year Member



>>if all of your people are sending referrers

They aren't all unwelcome and not all are a PITA. Most visitors are 100% legitimate and more than welcome. If someone comes in from search using a search phrase such as "discount yellow widgets" or "yellow widget stores," then it's a living, breathing human being and they'll find what they're looking for. But if they arrive with the single search word "widgets" then it's software running for either data mining (no biggie) or to scrape for MFA sites, and the latter are a nuisance and represent a hazard for some sites if they go far enough, which many do.

I do know there's a way to sniff for s/e referral query strings and then process accordingly, but it was a long time ago, so my memory on it is very dim.

>>If I am not mistaken, I got this code a while back off webmasterworld.

Jake, I've been spending a lot of time "reading" in the code forums at WebmasterWorld (which I heartily recommend, rather than just copying and pasting), and unless I can "read" what something is doing, I won't use it until I know what it'll do.

RewriteCond %{HTTP_REFERER} (milf) [NC,OR]
RewriteCond %{HTTP_REFERER} (x x x) [NC,OR]
RewriteRule .* - [F]

Do (milf) and (x x x) represent what actual query words could be? If so, then that could do it.

[edited by: Marcia at 10:49 am (utc) on Feb. 18, 2008]

jake66

11:26 am on Feb 18, 2008 (gmt 0)

5+ Year Member



milf and x x x are actual words I filter, the x's of course need to remove the spaces because WebmasterWorld filter the "word"..

And I do not copy/paste anything until I believe it won't break my site either. Only posted the reference as a working, live example that already works. :)

I use it myself, which is why I mentioned "I got this code a while back off webmasterworld" -- not to claim this as my work.

Believe me, I do not spend my time looking for ways to make myself look smart by answering questions I don't understand.

By the way you described your problem.. the remedy sounded as simple as the htaccess filter I use to block spam bots, that is the only reason I posted it.

Marcia

11:34 am on Feb 18, 2008 (gmt 0)

WebmasterWorld Senior Member marcia is a WebmasterWorld Top Contributor of All Time 10+ Year Member



By the way you described your problem.. the remedy sounded as simple as the htaccess filter I use to block spam bots, that is the only reason I posted it.

If those are the actual query words, then that might just do the trick. I've got a fairly new domain not doing much of anything on other engines yet that gets those requests constantly, it sure couldn't hurt to try.

Thanks, will do!

 

Featured Threads

Hot Threads This Week

Hot Threads This Month