Forum Moderators: open

Message Too Old, No Replies

Hit from Homeland Security

         

slipkid

3:13 pm on May 15, 2014 (gmt 0)

10+ Year Member




216.81.81.83 - - [15/May/2014:05:30:52 -0400] "GET example.jpg HTTP/1.1" 200 24239 "-" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"

Grabbed a picture and left.

216.81.80.0 - 216.81.95.255
Deny from 216.81.80.0/20

Yeah, right.

incrediBILL

9:07 pm on May 18, 2014 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Could be a hot linked image in a document.

Do you have something someone there might want to use in a web page, report or background?

slipkid

9:34 pm on May 18, 2014 (gmt 0)

10+ Year Member



Bill:

I have a hot-linking block in .htaccess with exclusions for 6 search engines.

The picture was a photograph of the U.S.S. Arizona Memorial.

What website Homeland Security was visiting that may have hot-linked the image or if HS went to my website instead of grabbing a photo from the US Park Service or National Archives leaves me bewildered.

Maybe I should ban all hot-linking?

not2easy

10:33 pm on May 18, 2014 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



Don't worry, the same IP shows up now and then to grab something from others' sites, it appears to be scrapers with spoofed IPs, so blocking the real HLS won't help much. See this thread for more info: [webmasterworld.com...]

slipkid

10:39 pm on May 18, 2014 (gmt 0)

10+ Year Member



Thanks for the info.

incrediBILL

11:00 pm on May 18, 2014 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Let's not start with that old wives tale about spoofing again, it doesn't work like that.

[en.wikipedia.org...]
"The machine that receives spoofed packets will send a response back to the forged source address, which means that this technique is mainly used when the attacker does not care about the response or the attacker has some way of guessing the response."

The picture was a photograph of the U.S.S. Arizona Memorial.


There you have it, maybe someone bookmarked the image.

lucy24

11:18 pm on May 18, 2014 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Was something left out of the first post? The quoted log entry shows a blank referer, so where does the hotlinking idea come from?

incrediBILL

11:50 pm on May 18, 2014 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



referrer is an OPTIONAL header field set by the browser, scraper or if bookmarked is NULL.

I've had to block null referrers from hotlinking before, its not uncommon.

lucy24

2:47 am on May 19, 2014 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



How do you know it's a hotlink? Or are you using the term in a broader sense of "anything that doesn't explicitly name my site as referer"?

not2easy

5:23 pm on May 19, 2014 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



In calling them scrapers with spoofed IPs it was because I looked at the access logs and I can clearly see that it is not human activity, they spent 1 second grabbing random images and .html files. The UA and referer on the first hit did not remain and switched from:
"http://www.bing.com/search?q=BLUE+WIDGETS+&src=IE-SearchBox" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 3.0.04506.30; .NET CLR 3.0.04506.648; .NET CLR 3.0.4506.2152; .NET 

to:
"-" "Mozilla/4.0 (compatible;)"

with the same time stamp and IP logged. That one pulled 15 files in one second. Several similar hits were seen coming from all kinds of "official" sites in that month as detailed in the linked thread. I do not see them routinely and kind of think it was more likely to have been done by kids than dedicated scrapers.

This particular case may be entirely different and may have been a hotlinked image on a bookmarked page.

lucy24

10:03 pm on May 19, 2014 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Spoofed UA, sure. They want to sneak past the barriers by putting up a humanoid user-agent. But spoofed IP? That would mean the end of civilization as we know it.

Let's not start with that old wives tale

But, but
::splutter::
but not2easy is an old wife :(

keyplyr

10:31 pm on May 19, 2014 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



not2easy is an old wife
Who knew?

RE: the 2 UAs. From my experience, the "Mozilla/4.0 (compatible;)" is a default for most download & harvest tools. So IMO the visitor came in with the first UA (an old IE browser) saw something they just had to have, then used their tool.

I see it all the time.

[added]
I'm also of the belief that IP addresses cannot be spoofed, unless you mean a proxy, which would of course also need to reveal it's authentic IP address to receive the packets.

incrediBILL

4:17 pm on May 20, 2014 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



How do you know it's a hotlink?


I just use that term any time you see images or other media extracted from the site without the page being loaded.

People do bookmark images, which I've done in the past, and bookmarked images don't have referrers. Based on the OP's description of the image, and that it was a one time event, is what I would conclude.

How someone could 'hotlink' an image and not get referrers from a browser would be to use an image server, a proxy if you will, where their server grabs your image on demand. Meaning the browser never directly connects with your site. I had someone do something like this to gain access to about 40K screen shots I have using some PHP script.

Not anymore ;)

Hard to say for sure but it's always more amusing to speculate when DHS, DOD, etc. is doing it.

While we're on the topic, once upon a time I made a blog post about the government using a specific scraper service that scraped and sold 'reports' to them. Must've hit a real nerve as I quickly saw a bunch of focused hits from that company, DHS, DOD, and of all places, Arlington apts. as someone was reading from home. I kept waiting for the black helicopters to land in my backyard because I figured out their little secret. Nothing happened, my car didn't explode, but I'll bet I have a file.

It's almost as much fun as some of my blog posts that got immediate hits from corporate lawyers for those companies as I was sure I was about to get sued a time or two based on the volume of hits from those law offices.

That's when I decided to stop looking.

As much fun as it is seeing the DHS on your site, it can lead to paranoia, sleeplessness, needless changes to .htaccess, and worse!