homepage Welcome to WebmasterWorld Guest from 54.204.73.126
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
1900 hits from images.yahoo.com
slipkid



 
Msg#: 4493624 posted 6:04 am on Sep 11, 2012 (gmt 0)

Yesterday, my website
came under continual attack (DoS?) from a yahoo referrer/user agent listed below. The hits occurred every 10-15 seconds and were continuous when they stopped in the early am. At least 1900 hits (I think).

The is the referrer/ua. (URI etc. changed)

24.190.103.173 - - [07/Sep/2012:02:57:48 -0400] "GET
/cgi-bin/referers.cgi?http://images.search.yahoo.com/images/view;_ylt=A0PDoTHcCUlQh38AzK.JzbkF;_ylu=X3oDMTBlMTQ4cGxyBHNlYwNzcgRzbGsDaW1n?back=http%3A%2F%2Fimages.search.yahoo.com%2Fsearch%2Fimages%3Fp%3Dkeyword_one%2Bkeyword_two %26_adv_prop%3Dimage%26va%3Dkeyword_one%2Bkeyword_two%26fr%3Dyfp-t-701%26tab%3Dorganic%26ri%3D122&w=750&h=500&imgurl=www.example.com%2Fpicture_gallery%2Fimages%2Flocation_of_image%2image.jpg& rurl=http%3A%2F%2Fwww.example.com%2Fpicture_gallery%2Fimage_location.html&size=81.9+KB&name=image_title%29&p=keyword_one+keyword_two&oid=c8a97c65e40bca9a6331f36da03145c4&fr2=&fr=yfp-t-701&tt=image_title%2529&b=121&ni=112&no=122&ts=&tab=organic&sigr=123rbb8eu&sigb=14545eqhn&sigi=13aikubif&.crumb=NZ.bhUZyY2s
HTTP/1.1" 404 486
"http://www.example.com/picture_gallery/image_location.html"
"Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; GTB6.5;
.NET CLR 1.0.3705; .NET CLR 1.1.4322; Media Center PC 4.0; .NET CLR
2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)"

According to my logs, this user agent seems to want the image contained in the folder www.example.com/picture_gallery//images/location_of_image/image.jpg.

Weird.

Hosting company said no impact to their system because bytes served were low and server was returning 404.

The "GET" references a perl logging script using a 1px by 1px web beacon.

[edited by: incrediBILL at 2:45 am (utc) on Sep 12, 2012]
[edit reason] broke up long referer [/edit]

 

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4493624 posted 9:38 am on Sep 11, 2012 (gmt 0)

deny from 24.190.103.173
or
RewriteCond %{REMOTE_ADDR} ^24\.190\.(9[6-9]|10[0-3])\.

If you'd like to lessen the innocents?

#UA contains GTB and comes from Optimum WRRNNJ
RewriteCond %{HTTP_USER_AGENT} GTB
RewriteCond %{REMOTE_ADDR} ^24\.190\.(9[6-9]|10[0-3])\.
RewriteRule .* - [F]

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4493624 posted 6:30 pm on Sep 11, 2012 (gmt 0)

@slipkid

I wouldn't block 24.190.103.173 because it's a cable ISP and you'd be blocking real visitors.

See if you can block something unique to the UA.

slipkid



 
Msg#: 4493624 posted 6:41 pm on Sep 11, 2012 (gmt 0)

@ keyplyr

I kind of figured it had something to do with a user's mouse. Researched the URI and found as indicated that it was coming from New Jersey.

Don't use Google ToolBar... so not familar if it had anything to do with the constant stream of hits.

My pics are the more popular pages on my site, so don't want to exclude them in robots.txt.

I consdier this a one-off problem, and will monitor to see if it happens again.

Thanks all for the help.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4493624 posted 9:58 pm on Sep 11, 2012 (gmt 0)

Yesterday, my website came under continual attack (DoS?) from a yahoo referrer/user agent listed below.

Putting a search engine in the forged-referer slot is a tried and true approach. Most of the time the exact wording is wrong, so you can block them even if you don't want to block the honest users coming in from real searches.

Now, personally I don't care much for yahoo so their image search goes straight into the hotlink bin without checking to see whether it's real or not. But ymmv.

I kind of figured it had something to do with a user's mouse.

Huh. Most people would blame the user's cat. But to each his own :)

slipkid



 
Msg#: 4493624 posted 11:15 pm on Sep 11, 2012 (gmt 0)

Partial to dogs, hate cats. Would not give credit to a cat's intelligence to hit a mouse button every fifteen secods...

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4493624 posted 11:43 pm on Sep 11, 2012 (gmt 0)


Now, personally I don't care much for yahoo so their image search goes straight into the hotlink bin without checking to see whether it's real or not. But ymmv

I get triple digit daily traffic from Yahoo/Bing/Google image search, but I guess if you don't want traffic coming to your site then blocking them is an alternative. And BTW, once again this has nothing to do with hot-linking, at least not from the major SEs.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4493624 posted 2:12 am on Sep 12, 2012 (gmt 0)

once again this has nothing to do with hot-linking

Your server can't tell the difference between a hotlink and a "google image search sent me". (Uh... You did know that, didn't you? :() They both come through as referers, so any routine aimed at one kind will automatically pick up the other. Which is why at least half of my current hotlink exemptions are for assorted legitimate* google functions. Conversely, certain image directories are roboted-out because I know by direct experience that people aren't interested in the pages; they're just collecting hotlink fodder.


* For a given definition of "legitimate". I know some people have serious issues with Translate, but mine are perfectly respectable and there's no reason to block them.

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4493624 posted 2:57 am on Sep 12, 2012 (gmt 0)

[quote]For a given definition of "legitimate". I know some people have serious issues with Translate, but mine are perfectly respectable and there's no reason to block them. [quote/]

Opinions are like. . . . ;)

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4493624 posted 6:56 am on Sep 12, 2012 (gmt 0)

Your server can't tell the difference between a hotlink and a "google image search sent me".

Sure I can because Google does not hotlink my images, at least not what I consider hotlinking. They are doing my bidding :)

I use a script that checks a few things any time a request is made for a file residing on my server from a remote source. It also busts the display of said image if the referrer isn't my site, and the human instantly gets pulled to my page where the image is.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4493624 posted 7:35 am on Sep 12, 2012 (gmt 0)

Sure I can

YOU can. Your server can't. You didn't actually read my post, did you?

the human instantly gets pulled to my page

Well, that's one way to use Image Search to generate traffic.

thetrasher

5+ Year Member



 
Msg#: 4493624 posted 1:16 pm on Sep 12, 2012 (gmt 0)

I'm a little confused.

yahoo referrer/user agent

Sorry, but I see neither a Yahoo referrer nor a Yahoo user agent.

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4493624 posted 2:20 pm on Sep 12, 2012 (gmt 0)

A0PDoTHcCUlQh38AzK.JzbkF;_ylu=X3oDMTBlMTQ4cGxyBHNlYwNzcgRzbGsDaW1n?back=http%3A%2F%2Fimages.search.yahoo.com%2Fsearch%2Fimages%3Fp%3Dkeywor

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4493624 posted 8:12 pm on Sep 12, 2012 (gmt 0)

I see neither a Yahoo referrer nor a Yahoo user agent.

Technically you're right. But I think the sample line is a request sent to the OP's analytics program. In that case his own page would be listed as the referer, while the referer for that page would go into the request's query string. Go back a few lines in the logs and you'll find the original page request, with Yahoo in the referer line.

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4493624 posted 11:26 pm on Sep 12, 2012 (gmt 0)


YOU can. Your server can't. You didn't actually read my post, did you?

I read your entire post. I answered it accordingly. You didn't actually read my post, did you? LOL

e.g. my SERVER can tell the difference because of the script I have in place. Sorry, not going into any more detail on a public forum.

Anyway, as stated above, I enjoy the traffic resulting from image searches and do not consider it hotlinking since they have my full approval to do so.

slipkid



 
Msg#: 4493624 posted 1:10 am on Sep 13, 2012 (gmt 0)

I see neither a Yahoo referrer nor a Yahoo user agent.


I agree with what Lucy24 has pointed out.

I am still learning how to frame issues on the forum.

Elsmarc

10+ Year Member



 
Msg#: 4493624 posted 3:00 am on Sep 13, 2012 (gmt 0)

Goodness. All you need is an .htaccess file and use the RewriteCond %{HTTP_REFERER} bit. Just Google something like stop hotlinking. It's not rocket science.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved