homepage Welcome to WebmasterWorld Guest from 54.235.16.159
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Visit PubCon.com
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Nokia6300/ (botmobi)
didn't ask for robotos text, has Amazon IP
idiotgirl




msg:3680750
 1:26 pm on Jun 22, 2008 (gmt 0)

This little critter showed up with a default list of directories to scan, and dined on a quick list of 404's for breakfast. The IP is under Amazon.Com. find.mobi/bot.html says it asks for robots.txt, but not that I can see. Haven't seen this one before. I thought bypassing robots.txt seemed a little unfriendly...

67.202.23.* - - [21/Jun/2008:23:49:02 -0400] "GET / HTTP/1.0" 200 5718 "-" "Nokia6300/2.0 (05.50) Profile/MIDP-2.0 Configuration/CLDC-1.1 (botmobi find.mobi/bot.html find@mtld.mobi)"
67.202.23.* - - [21/Jun/2008:23:49:02 -0400] "GET /css/style.css HTTP/1.1" 200 4585 "http://www.exampledomain.com/" "Nokia6300/2.0 (05.50) Profile/MIDP-2.0 Configuration/CLDC-1.1 (botmobi find.mobi/bot.html find@mtld.mobi)"
67.202.23.* - - [21/Jun/2008:23:49:03 -0400] "GET /wap/ HTTP/1.0" 404 1101 "-" "Nokia6300/2.0 (05.50) Profile/MIDP-2.0 Configuration/CLDC-1.1 (botmobi find.mobi/bot.html find@mtld.mobi)"
67.202.23.* - - [21/Jun/2008:23:49:03 -0400] "GET /m/ HTTP/1.0" 404 1101 "-" "Nokia6300/2.0 (05.50) Profile/MIDP-2.0 Configuration/CLDC-1.1 (botmobi find.mobi/bot.html find@mtld.mobi)"
67.202.23.* - - [21/Jun/2008:23:49:04 -0400] "GET /mobi/ HTTP/1.0" 404 1101 "-" "Nokia6300/2.0 (05.50) Profile/MIDP-2.0 Configuration/CLDC-1.1 (botmobi find.mobi/bot.html find@mtld.mobi)"
67.202.23.* - - [21/Jun/2008:23:49:04 -0400] "GET /mobile/ HTTP/1.0" 404 1101 "-" "Nokia6300/2.0 (05.50) Profile/MIDP-2.0 Configuration/CLDC-1.1 (botmobi find.mobi/bot.html find@mtld.mobi)"
67.202.23.* - - [21/Jun/2008:23:49:04 -0400] "GET /imode/ HTTP/1.0" 404 1101 "-" "Nokia6300/2.0 (05.50) Profile/MIDP-2.0 Configuration/CLDC-1.1 (botmobi find.mobi/bot.html find@mtld.mobi)"
67.202.23.* - - [21/Jun/2008:23:49:05 -0400] "GET /xhtml/ HTTP/1.0" 404 1101 "-" "Nokia6300/2.0 (05.50) Profile/MIDP-2.0 Configuration/CLDC-1.1 (botmobi find.mobi/bot.html find@mtld.mobi)

[edited by: incrediBILL at 6:48 pm (utc) on June 22, 2008]
[edit reason] Obscured IPs [/edit]

 

g1smd




msg:3680941
 7:18 pm on Jun 22, 2008 (gmt 0)

Odd that it claims to be coming from a Nokia mobile phone too...

jdMorgan




msg:3680983
 8:49 pm on Jun 22, 2008 (gmt 0)

It is typical of robots crawling mobile sites (or looking for mobile sites) to spoof a "typical" mobile device such as a Nokia phone. You'll notice that Googlebot-Mobile and Yahoo's YahooSeeker/M1A1-R2D2 do exactly the same thing.

I've also got my eye on this one, but it looks like a link checker for the dot mobi directory so far, and I have backlinks from that directory.

Jim

idiotgirl




msg:3681235
 6:27 am on Jun 23, 2008 (gmt 0)

The weird thing is, this site that was visited has nothing in the dot mobi directory, nor do the directories the bot requested even exist. It came in, didn't look for rorbots text, quickly checked a pre-determined list of directories, then scrammed. And the IP block it came from belongs to Amazon, or so it appears.

keyplyr




msg:3681292
 9:05 am on Jun 23, 2008 (gmt 0)

It's been showing at my sites requesting the same directories.

nativenewyorker




msg:3688495
 2:16 pm on Jul 2, 2008 (gmt 0)

This stupid bot from .mobi has been hammering my non-mobile site for the last couple hours from all of these IPs. It looks like they're trying to drum up .mobi domain name registrations by spamming my logs.

67.202.17.***
67.202.22.***
67.202.30.***
67.202.36.***
67.202.41.***
67.202.46.***
67.202.53.***
67.202.55.***
67.202.58.***
67.202.60.***
67.202.61.***
75.101.193.***
75.101.195.***
75.101.206.***
75.101.212.***
75.101.218.***
75.101.223.***
75.101.232.***
75.101.233.***
75.101.235.***
75.101.236.***
75.101.240.***

jdMorgan




msg:3690062
 1:04 am on Jul 4, 2008 (gmt 0)

If you read their robot info page, they explain that they try to 'discover' mobile pages on sites by trying m.example.com, mobi.example.com, example.com/m/, example.com/mobi, and several other variations. While this may be annoying, please realize that the mobile Web is not "developed" like the "old" Web is. In many cases (including some sites of mine), the only backlinks are from directories, because there are no other relevant mobile sites** to link to or to get links from. So I can understand why they're doing this.

However, on my sites, they *did* check robots.txt, so I dunno what;s going on if they are not checking it on some sites... maybe a glitch.

Anyway, the "Nokia" in the user-agent, and the fetches of "mobile-related" subdomains and subdirectories is not mysterious.

** If you're looking for a niche market, get in now!

Jim

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved