Welcome to WebmasterWorld Guest from 54.166.87.123

Forum Moderators: Ocean10000 & incrediBILL

Message Too Old, No Replies

Exalead

Thumbnail bot

   
1:19 am on Jun 15, 2006 (gmt 0)

10+ Year Member



193.47.80.83-193.47.80.92

UA=mozilla/5.0 (compatible; konqueror/3.4; linux) khtml/3.4.1 (like gecko)
(previously NG/2.0)

Reverse DNS=thumb0.exabot.com to thumb9.exabot.com

Still opening a "preview" of your site in a frame and removing adsense code.
[webmasterworld.com...]

5:32 am on Jun 15, 2006 (gmt 0)

5+ Year Member



I've been seeing Exabot-Images crawler occasionally recently. But I have Exabot disallowed in robots.txt plus my /images/ folder is disallowed and it has respected that consistently.

193.47.80.140 - - [15/Jun/2006:04:25:08 +1000] "GET /robots.txt HTTP/1.1" 200 1622 "-" "Exabot-Images/1.0"

5:50 pm on Jun 15, 2006 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



fiestagirl, is the UA you provided exactly as-is? All lowercase and including the "previously" paren?

mozilla/5.0 (compatible; konqueror/3.4; linux) khtml/3.4.1 (like gecko) (previously NG/2.0)

I ask because I usually see generic Konqueror UAs as upper- and lower-cased, e.g.:

Mozilla/5.0 (compatible; Konqueror/2.0.1; X11); Supports MD5-Digest; Supports gzip encoding
Mozilla/5.0 (compatible; Konqueror/2.2.2; Linux 2.4.14-xfs; X11; i686)
Mozilla/5.0 (compatible; Konqueror/3.4; Linux) KHTML/3.4.3 (like Gecko) (Kubuntu package 4:3.4.3-0ubuntu2)

Regardless, thank goodness for [NC,OR] :)

6:58 pm on Jun 15, 2006 (gmt 0)

WebmasterWorld Administrator incredibill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Yahoo Image Search opens a 'preview" as well which disables certain features in MSIE:
[webmasterworld.com...]

I just installed a "fix" so my site detects Yahoo displays my page in preview and MSIE the "preview" being displayed says:

"Sorry, we can't display this page because Yahoo causes the page to malfunction.
CLICK HERE to see the page operating properly in a new window."

Now Google Image Search on the other hand does the same thing but my frame buster blasts out of their framing in both MSIE and Firefox, as well as breaks out of Yahoo in Firefox.

I'm looking at implementing the same trick for this search engine.

When exalead frames your site it uses a referrer like this:

"http://www.exalead.com/search?C=<gibberish>"

Opposed to when you click the link to open the site:

"http://www.exalead.com/search/C=<gibberish>q=www.yourdomain.com"

Should be easy enough to cloak this so visitors see something like Yahooligans now see.

6:27 pm on Jun 16, 2006 (gmt 0)

10+ Year Member



The "previously" was my addition, sorry for the confusion. All lower case.

mozilla/5.0 (compatible; konqueror/3.4; linux) khtml/3.4.1 (like gecko)

I also noticed that when I previewed my site, it generated a visit from their ip group, with my user agent.

Query string in this form:
http ://www.exalead.com/search?C=0MlEA.....

4:21 am on Jun 24, 2006 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



Finally got around to checking current and old logs. No sign of the all-lowercase variety, via UA or IP. FWIW...

crawl103.exabot.com
"Exabot-Images/1.0"
/robots.txt

tc-gw.exabot.com
"Exabot/2.0"
/robots.txt

Granted, I've never allowed Exava under its Host, its IP, under Become's IPs as sub-domains of exava.com, or any of these bots --

"Exalead NG/MimeLive Client (convert/http/0.143)"
"NG/2.0"
"Mozilla/4.7 [en](Exabot@exava.com)"
"Mozilla/4.7 [en](BecomeBot@exava.com)"
"Mozilla/5.0 (compatible; BecomeBot/1.23; +http://www.become.com/webmasters.html)"
"Exabot/2.0"
"Exabot-Images/1.0"

-- yet the last/newest requested specific .gif files, including some from an all-bots-restricted area.

Too many bots.

Too many (attempted) crawls.

Toodle-oo.