Forum Moderators: open

Message Too Old, No Replies

211.154.211.209

heads up, spider faking user agent Googlebot

         

Key_Master

5:20 pm on Aug 5, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Resolves to a Chinese domain, possibly an E-mail scavenger.

IP: 211.154.211.209
User Agent: Googlebot/2.1 (+http://www.googlebot.com/bot.html)

More incriminating evidence [google.com]

william_dw

6:02 pm on Aug 5, 2002 (gmt 0)

10+ Year Member



I'm probably wrong (I often am),
but from that evidence link there's a IMDB 404, which if you look at the cached version shows some diagnostic information.

Seeing as that diagnostic info is based upon the requesting agent, the only way I can see that google could get to see that page is if it was the one doing the requesting.

Althoughhhhh, looking at this again, it looks like google saw that info because the URL it was crawling was a proxy which pulled the page from IMDB.

If that's the case, it might not be an email crawler (I've never seen an email crawler that works by pretending to be a proxy that you can call at will for pages, there would be more efficient ways to find email rich pages, like pulling google search results for 'forum', or asking google for searches for ':', which it likes to use instead of the @ symbol).

It could be some people from china trying to view information on films about the horrible oppressive west that their government is 'protecting' them from.

Ho hum,
No better than when I started writing LOL.

volatilegx

6:10 pm on Aug 5, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Thanks... have to keep an eye out for this one!

Jaf

12:03 am on Aug 8, 2002 (gmt 0)

10+ Year Member



I know of at least one webmaster (not me) who has been known to change his user agent to be GoogleBot :-)

wilderness

1:42 am on Aug 9, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



from today's IAR Newsletter

2. Don't Get Googled by Hackers!
August 8, 2002
The popular search engine houses a flaw in its toolbar that hackers can use to execute multiple tasks; Google responds with fix.
[internetnews.com...]