Is it possible ;)

[wilderness]

from Saint Petersburg, Russia to Bell South in two seconds ;)

87.237.113.z - - [22/Feb/2007:09:57:37 -0800] "GET /MyFolder/MyPage.html HTTP/1.0" 403 - "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
65.83.25.zzz - - [22/Feb/2007:09:57:39 -0800] "GET /SameFolder/SamePage.html HTTP/1.1" 200 27767 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"

[volatilegx]

are these related in any way besides the short time in between requests?

[wilderness]

Hey Dan,
You mean besides being the identical page request and UA?

The refers were blank (as provided) so I'm unable to determine if a search was ontopic.

The page itself is a frequently visited page and frequently ontopic search result.

The RU request was 403'd and no images were requested.
The US request was accompanied by all page images.

BTW is two seconds a short time ;)

[GaryK]

I've seen this user agent close to 1.5 million times since early 2000. The last time on the 18th of this month. It's very popular.

[wilderness]

I've seen this user agent close to 1.5 million times since early 2000. The last time on the 18th of this month. It's very popular.

;) ;)

But how many times did it come from Saint Petersburg, Russia and then two seconds later, Bell South ;)
with both requesting the same page!

[wilderness]

For general interests?

The page can be difficult to find unless the search is highly customized (I do get some occassional widget interests from Russia and I have opened an extensive folder via whitelisting recently that offers links to many of my pages NOT located in that extensive folder. This formerly closed folder excluded nearly all NON-North American IP ranges.)

A google on "Ben White", possibly a common name doesn't provide much. I stopped flipping through pages after 20 without seeing my page listed.

IF, however I add ("Ben White" +specific term)than my page (s) come up at the top of listings.
Utilizing "other" terms, my page (s) come up very close to top.

The entire point is that this page is not exactly easy to find, at least randomly.

[GaryK]

The entire point is that this page is not exactly easy to find, at least randomly.

I'll grant you that part is odd otherwise it could be chalked up to random chance. Guess I'll put a watch on that UA to see what it's been up to lately. I know it's malformed but I've got it parented by IE6. At least for now.

[zCat]

I see the following pattern quite a lot: IP address 1 performs a GET, then shortly after IP address 2 - from a completely different IP range - performs a POST on the same page. IP addresses are typically residential. I guess some kind of zombie botnet. Trivial to block at the application level though.

[wilderness]

zCat,
here's one for ya.
Took place in less than two minutes.
Note the repetitous UA over three of the four IP ranges.

I've removed the lines for image requests, which only accompanied the requests from the IP 70.156.68.zzz
(I've left the favicon request in tact)

Note; two entirely different IP ranges from MS utililzed.
An additional range from Sprint and a 4th from an MCI sub-net.

63.80.56.zz - - [28/Feb/2007:07:55:53 -0800] "GET /MyFolder/MyPage.html HTTP/1.0" 403 - "-" "Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; ....../1.0 )"
70.56.168.zzz - - [28/Feb/2007:07:55:59 -0800] "GET /SameFolder/SamePage.html HTTP/1.1" 200 80184 "Sort of on Topic Google" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.0.3705; .NET CLR 1.1.4322; .NET CLR 2.0.50727)"
63.80.56.zz - - [28/Feb/2007:07:56:00 -0800] "GET /SameFolder/ HTTP/1.0" 403 - "-" "Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; ....../1.0 )"
63.80.56.zz - - [28/Feb/2007:07:56:05 -0800] "GET /MyFolder HTTP/1.0" 403 - "-" "Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; ....../1.0 )"
63.80.56.zz - - [28/Feb/2007:07:56:11 -0800] "GET / HTTP/1.0" 403 - "-" "Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; ....../1.0 )"
63.80.56.zz - - [28/Feb/2007:07:56:12 -0800] "GET /SameFolder/SamePage.html HTTP/1.0" 403 - "-" "Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; ....../1.0 )"
204.249.11.zz - - [28/Feb/2007:07:56:15 -0800] "GET /favicon.ico HTTP/1.0" 403 - "-" "Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; ....../1.0 )"
207.46.134.zz - - [28/Feb/2007:07:56:15 -0800] "GET /favicon.ico HTTP/1.1" 403 - "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3"
207.46.134.zz - - [28/Feb/2007:07:56:16 -0800] "GET / HTTP/1.1" 403 - "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3"
70.56.168.zzz - - [28/Feb/2007:07:56:16 -0800] "GET /favicon.ico HTTP/1.1" 200 318 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.0.3705; .NET CLR 1.1.4322; .NET CLR 2.0.50727)"
63.80.56.zz - - [28/Feb/2007:07:56:17 -0800] "GET /SameFolder/ HTTP/1.0" 403 - "-" "Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; ....../1.0 )"
204.249.11.19 - - [28/Feb/2007:07:56:22 -0800] "GET / HTTP/1.0" 403 - "-" "Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; ....../1.0 )"
204.249.11.zz - - [28/Feb/2007:07:56:22 -0800] "GET /favicon.ico HTTP/1.0" 403 - "-" "Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; ....../1.0 )"
63.80.56.zz - - [28/Feb/2007:07:56:23 -0800] "GET /SameFolder HTTP/1.0" 403 - "-" "Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; ....../1.0 )"
204.249.11.zz - - [28/Feb/2007:07:56:29 -0800] "GET / HTTP/1.0" 403 - "-" "Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; ....../1.0 )"
63.80.56.zz - - [28/Feb/2007:07:56:29 -0800] "GET / HTTP/1.0" 403 - "-" "Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; ....../1.0 )"
63.80.56.zz - - [28/Feb/2007:07:56:30 -0800] "GET /SameFolder/SamePage.html HTTP/1.0" 403 - "-" "Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; ....../1.0 )"
204.249.11.zz - - [28/Feb/2007:07:56:30 -0800] "GET /favicon.ico HTTP/1.0" 403 - "-" "Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; ....../1.0 )"
63.80.56.zz - - [28/Feb/2007:07:56:35 -0800] "GET /SameFolder/ HTTP/1.0" 403 - "-" "Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; ....../1.0 )"
204.249.11.zz - - [28/Feb/2007:07:56:37 -0800] "GET / HTTP/1.0" 403 - "-" "Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; ....../1.0 )"
63.80.56.zz - - [28/Feb/2007:07:56:41 -0800] "GET /SameFolder HTTP/1.0" 403 - "-" "Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; ....../1.0 )"
63.80.56.zz - - [28/Feb/2007:07:56:47 -0800] "GET / HTTP/1.0" 403 - "-" "Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; ....../1.0 )"

[GaryK]

Don, I looked for the same pattern last weeks analysis that you originally posted about and I saw it too. Same UA, different page requests, vastly different parts of the world, all within seconds. It's a bit much to be coincidence.

More often what I see is a valid browser UA spiders a page and if nothing goes wrong a SE UA gets the same page. None of them reads robots.txt. Sometimes the first UA will fall into a trap. The second UA avoids it and avoids getting banned. I can't ban the first UA cause it's usually for a major browser.

[zCat]

This is the kind of stuff I see:

69.168.51.118 - - [27/Aug/2006:17:05:22 +0200] "GET /blog/blahblah.html HTTP/1.1" 200 7325 "http://example.com/" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
67.175.103.235 - - [27/Aug/2006:17:05:23 +0200] "POST /blog/blahblah.html#comments HTTP/1.1" 200 7555 "http://example.com/blog/blahblah.html" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"

Since I fixed things so that whatever's posted from this kind of "session" lands in data nirvana, this sort of activity seems to have dropped off.

(Note to self: must get life, over the last few years I have developed the ability to spot non-human website activity just by watching logs...)

[benevolent001]

Might be offtopic , sorry for this

is it possible SE checking same checking same page at same time from different locations around the world to see if the website owner showing different content to them or fooling SE

i know its perfectly legal to show up different content , which is more relevant to user according to location

[terrabyte]

What if the user was using Tor?
Although that probably wouldn't explain hitting the same page twice in a row.

Does TOR change the user agent at all?