homepage Welcome to WebmasterWorld Guest from 54.196.168.78
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Jakarta Commons
wilderness




msg:4441610
 7:08 pm on Apr 16, 2012 (gmt 0)

There's multiple old threads on Jakarta Commons, which Ive had denied for the longest time.

Unfortunately, and considering my content (especially the requested page; I've some K-12 orgs that spider pages regularly), I had to consider/explore an exception for the IP based upon their potential clientele.

Unfortunately, the majority of their "ALL Partners", seem to be 3rd party services, as opposed to primarily educational orgs.

"We provide a full range of services that support the use of open source technologies in education, including Moodle, Sakai and others. With this effort, we are supporting teaching and learning wherever it is happening, around the globe."
69.196.241.zzz - - [16/Apr/2012:18:20:39 +0100] "GET /MyFolder/SubFolder/MyPage.html HTTP/1.1" 403 - "-" "Jakarta Commons-HttpClient/3.1"

 

incrediBILL




msg:4441631
 7:42 pm on Apr 16, 2012 (gmt 0)

Well teach them to supply a valid user agent and maybe they'll learn something.

Trust me, none of the rest of us will be making any exceptions for sloppy and lazy programming.

keyplyr




msg:4441640
 8:24 pm on Apr 16, 2012 (gmt 0)

...the majority of their "ALL Partners", seem to be 3rd party services, as opposed to primarily educational orgs.


I exchanged email with at least one of these 3rd party companies a couple years ago attempting to convince them to use a specific UA with info page. They avoided the obvious, skirting responsibility and hiding behind the edu argument. Well IMO that's BS as they are in biz selling the index to schools, who never investigate any further. I don't see much difference between selling their index to schools or anyone else, they are still profiting from my content. I still block Jakarta Commons.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved