homepage Welcome to WebmasterWorld Guest from 54.242.126.9
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Mediabot coming through Yahoo! Proxy
volatilegx

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 3241 posted 2:18 pm on May 3, 2006 (gmt 0)

Can anybody confirm that this is a valid Mediabot user agent?

05/02/06 10:15:44 BST
IP: 66.94.237.140
Host: proxy1.search.scd.yahoo.net
UA: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 4.0), Mediapartners-Google/2.1


05/02/06 10:15:44 BST
IP: 66.94.237.141
Host: proxy2.search.scd.yahoo.net
UA: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 4.0), Mediapartners-Google/2.1

 

Brett_Tabke

WebmasterWorld Administrator brett_tabke us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 3241 posted 2:12 am on May 4, 2006 (gmt 0)

hmmm. someone at the hoo have a little fun?

Pfui

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 3241 posted 8:40 am on May 4, 2006 (gmt 0)

Spidertrack.org [spidertrack.org] shows the same curious combination (hit dates unclear). Scroll down to the teal+yellow highlighted text on this cached page [72.14.207.104], or more conveniently --

IP: 66.94.237.140
Host: proxy1.search.scd.yahoo.net
UA: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 4.0), Mediapartners-Google/2.1

IP: 66.94.237.141
Host: proxy2.search.scd.yahoo.net
UA: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 4.0), Mediapartners-Google/2.1

IP: 66.94.237.142
Host: proxy3.search.scd.yahoo.net
UA: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 4.0), Mediapartners-Google/2.1

(Deja vu all over again:)

activeco

10+ Year Member



 
Msg#: 3241 posted 6:58 am on May 7, 2006 (gmt 0)

<speculation>

Yahoo is checking if someone hides Adsense code from them.
If so, do they prefer non-adsense pages in their organic search?

volatilegx

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 3241 posted 7:46 pm on May 7, 2006 (gmt 0)

Want my guess? It's a proxy that somebody figured out how to use, and they are trying to decloak pages.

activeco

10+ Year Member



 
Msg#: 3241 posted 9:57 pm on May 7, 2006 (gmt 0)

Hm, it seemed it was an enigma in the past too:

[webmasterworld.com...]
[webmasterworld.com...]
[webmasterworld.com...]
[webmasterworld.com...]
[webmasterworld.com...]
[webmasterworld.com...]

Brett_Tabke

WebmasterWorld Administrator brett_tabke us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 3241 posted 10:05 pm on May 7, 2006 (gmt 0)

Sure that isn't bablefish?

activeco

10+ Year Member



 
Msg#: 3241 posted 11:15 pm on May 7, 2006 (gmt 0)

Done some research.
All the subdomains before "search.scd.yahoo.*" are tied to different "departments" like "bff*" is babelfish (try [bff1.search.scd.yahoo.com...] ), "a*" is Alltheweb (try [a1.search.scd.yahoo.com...] ), etc.

It looks like subdomain "scd" is connected with Yahoo Search Web Services, which are explained here:
[pcquest.com...]

It could also be that general public who use YS Web Services, could manipulate it. E.g.:
[api.search.yahoo.com...]

Note this line: "ws02.search.scd.yahoo.com compressed/chunked..."
ws=webServer?

So it is possible that proxy1/2/3 (there are three of them) is used in some way by the public too.

activeco

10+ Year Member



 
Msg#: 3241 posted 11:59 pm on May 7, 2006 (gmt 0)

P.S.

Pardon me for not quoting the WebmasterWorld results first regarding YSWS, such as [webmasterworld.com...] .

The link above from PCQuest was very neat explanation.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved