homepage Welcome to WebmasterWorld Guest from 50.17.79.35
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Mediabot coming through Yahoo! Proxy
volatilegx




msg:407271
 2:18 pm on May 3, 2006 (gmt 0)

Can anybody confirm that this is a valid Mediabot user agent?

05/02/06 10:15:44 BST
IP: 66.94.237.140
Host: proxy1.search.scd.yahoo.net
UA: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 4.0), Mediapartners-Google/2.1


05/02/06 10:15:44 BST
IP: 66.94.237.141
Host: proxy2.search.scd.yahoo.net
UA: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 4.0), Mediapartners-Google/2.1

 

Brett_Tabke




msg:407272
 2:12 am on May 4, 2006 (gmt 0)

hmmm. someone at the hoo have a little fun?

Pfui




msg:407273
 8:40 am on May 4, 2006 (gmt 0)

Spidertrack.org [spidertrack.org] shows the same curious combination (hit dates unclear). Scroll down to the teal+yellow highlighted text on this cached page [72.14.207.104], or more conveniently --

IP: 66.94.237.140
Host: proxy1.search.scd.yahoo.net
UA: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 4.0), Mediapartners-Google/2.1

IP: 66.94.237.141
Host: proxy2.search.scd.yahoo.net
UA: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 4.0), Mediapartners-Google/2.1

IP: 66.94.237.142
Host: proxy3.search.scd.yahoo.net
UA: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 4.0), Mediapartners-Google/2.1

(Deja vu all over again:)

activeco




msg:407274
 6:58 am on May 7, 2006 (gmt 0)

<speculation>

Yahoo is checking if someone hides Adsense code from them.
If so, do they prefer non-adsense pages in their organic search?

volatilegx




msg:407275
 7:46 pm on May 7, 2006 (gmt 0)

Want my guess? It's a proxy that somebody figured out how to use, and they are trying to decloak pages.

activeco




msg:407276
 9:57 pm on May 7, 2006 (gmt 0)

Hm, it seemed it was an enigma in the past too:

[webmasterworld.com...]
[webmasterworld.com...]
[webmasterworld.com...]
[webmasterworld.com...]
[webmasterworld.com...]
[webmasterworld.com...]

Brett_Tabke




msg:407277
 10:05 pm on May 7, 2006 (gmt 0)

Sure that isn't bablefish?

activeco




msg:407278
 11:15 pm on May 7, 2006 (gmt 0)

Done some research.
All the subdomains before "search.scd.yahoo.*" are tied to different "departments" like "bff*" is babelfish (try [bff1.search.scd.yahoo.com...] ), "a*" is Alltheweb (try [a1.search.scd.yahoo.com...] ), etc.

It looks like subdomain "scd" is connected with Yahoo Search Web Services, which are explained here:
[pcquest.com...]

It could also be that general public who use YS Web Services, could manipulate it. E.g.:
[api.search.yahoo.com...]

Note this line: "ws02.search.scd.yahoo.com compressed/chunked..."
ws=webServer?

So it is possible that proxy1/2/3 (there are three of them) is used in some way by the public too.

activeco




msg:407279
 11:59 pm on May 7, 2006 (gmt 0)

P.S.

Pardon me for not quoting the WebmasterWorld results first regarding YSWS, such as [webmasterworld.com...] .

The link above from PCQuest was very neat explanation.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved