iaea.org spider

new theory

[yobb]

Today I looked for informations about this misterious spider and I found this:

www.iaea.org/inis/ws/index.html

Maybe the spider is related to this project ?

[wilderness]

Alexa.com (aka ia_archiver)
Has but one purpose
[download.alexa.com...]
This software has a use which the user is unable to see which entirely benefits Alexa, in that by using their software YOU add links to their database.

They also offer this explanation
[alexa.com...]

ia_archiver is the name of the robot the Alexa software uses.

Alexa has NOTHING to do with Nuclear regulations or research.

Unless Alexa is probing that site also ;-) TIC

[bird]

And why should the alexa toolbar fetch those individual pages again, seperately from IE? They are already present in the browser window, where it has full access to their content.

Not that it would necessarily be beyond alexa to use a fake referrer, but the connection doesn't quite convince me yet. Have you checked the toolbar's network traffic?

[littleman]

Wilderness, the iaea.org is a very odd bot that fakes a referrer to iaea.org, it will come out of seemingly random IPs, sometimes dialups, and sometimes proxies. People have been trying to figure it out for a while. Yobb was not talking about Alexa or the ia_archiver UA.

[webmasterworld.com...]
[webmasterworld.com...]
[webmasterworld.com...]
[webmasterworld.com...]

Yobb, it could be, I guess someone should email them and ask.

[wilderness]

Hey bird,
I'm not a fan of Alexa :-(
They do have a genuine source which I use occasionally which they are affiliated with.
Other wise I don't go near anything associated with Alexa.

My sites are Standardbreds horse sites.
Early last year a directory Standardbred type site was using the Alexa software to compile phony statistics on website traffic rankings using the software.
Of course the software only compiles stats based on the URL's included by the statisticain (sp?.)

Hence my negativity.

[wilderness]

Hey Littleman,
I get an occasional referal from "iae"
which when I use that referring URL in my logs appears as the Nuclear site.

I have another theory for you :-)
Take a look at this URL
[archive.org...]

Who runs the resources?
Does it have a defined bot?
Some pages upon search the user is notified that the site requested that these pages not be included in the archives.
Ex: Detroit news articles prior to 2000

I have not used this to look at any of my old pages and as a result cannot tell you what shows up in the logs.
However . . .
I'm willing to wager. . .

[bird]

The fact that www.archive.org is fueled by Alexa (aka. ia_archiver) is well known. Although many people like to browse the archive, almost nobody likes Alexa and it's robot.

But I still don't see any convincing connection between this organization on one side, and the accesses with the obviously fake iaea referrer that we all find in our logs.

Wilderness, do you have any arguments as to why you consider the two to have anything to do with each other? Or did you simply post your first reply to the wrong thread?

(Unfortunately, yobb's hypothesis doesn't really convince me either...)

[volatilegx]

My personal opinion is that the spider is someone's personal bot and that person has something against the nuclear power industry. He could be providing the referrer information to sneak past mod rewrite instructions that disallow access to GET requests with no referrer, so he doesn't look like a bot.

Just a wild guess.