Welcome to WebmasterWorld Guest from 54.146.195.24

Forum Moderators: eljefe3 & skibum

Media.net bot ?

     
8:37 pm on May 6, 2018 (gmt 0)

Preferred Member

Top Contributors Of The Month

joined:Mar 25, 2018
posts:500
votes: 101


Since there are experts of bots identification, I am wondering if anyone know about a "media.net" bot (like there is an adsense bot).

A couple of times, I wrote to media.net to ask how they were fetching the pages of a site, to select ads, and if they had a "robot" doing this. I asked this to be sure to whistle it. They claimed they had no "bot", but they didn't tell me how they were identifying the content of a page (industrial secrets?).

But they must have a way, to crawl pages, no? Or may be they manually review each site, and set the themes of the site globally ?

ps: I meant "media.net" in the title, if someone can fix it.
9:17 pm on May 6, 2018 (gmt 0)

Moderator from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 890


I keep a diligent eye on most all UAs and have not identified anything calling itself "media.net" bot.

There are thousands and thousands of generically named scraping agents [webmasterworld.com] so they are likely using one of those non-identified UAs.

For example:
http-client
9:27 pm on May 6, 2018 (gmt 0)

Preferred Member

Top Contributors Of The Month

joined:Mar 25, 2018
posts:500
votes: 101


I don't have the skill and experience of keyplyr, but I have a "feeling" that "may be", "eventually", media.net has something running from xlhost.com .

Since , I started the topic, I made a little test. I created a page, without incoming links, with a media.net code and without any bot/ip filtering. I visited the page myself, and in the same second, there was a hit from an IP located at xlhost.com , with a referrer field being set to the page itself , and with a user agent, which was "mine" (I tried changing the user agent string to check). So I guess it means media.net is running a bot that way. (and may be from other networks, but I tried several times, and each time it was different IPs from xlhost.com).
9:34 pm on May 6, 2018 (gmt 0)

Moderator from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 890


Yeah, many bots masquerade as human browsers.
11:25 am on May 7, 2018 (gmt 0)

Preferred Member

Top Contributors Of The Month

joined:Mar 25, 2018
posts:500
votes: 101


I am not going to bother much longer, but just in case, someone, one day is investigating on the subject... , EACH time I visit my test page, there is a hit from an IP at xlhost, and the ad starts loading only after the hit from xlhost occurred, which as a result is making ads slow to show.
6:43 pm on May 7, 2018 (gmt 0)

Moderator from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 890


EACH time I visit my test page, there is a hit from an IP at xlhost
Because you have a tracking cookie on your hard drive set through your browser.
7:02 pm on May 7, 2018 (gmt 0)

Preferred Member

Top Contributors Of The Month

joined:Mar 25, 2018
posts:500
votes: 101


I didn't understand.
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members