1. Home
  2. Forums Index
  3. General SEO Issues / Crawler, Spider, and User Agent ID 10:00 pm May 4, 2026

Forum Moderators: open

Message Too Old, No Replies

CsCrawler

University of Kassel, Germany

         

bull

9:08 pm on Nov 6, 2005 (gmt 0)

10+ Year Member



141.51.167.* - - [06/Nov/2005:10:17:56 +0100] "GET /foo.html HTTP/1.1" 403 - "" "Hi! I'm CsCrawler, my homepage: [kde.cs.uni-kassel.de...] RPT-HTTPClient/0.3-3"

no robots.txt

bull

6:54 am on Nov 8, 2005 (gmt 0)

10+ Year Member



have to correct:
did fetch robots.txt
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

  1. Home
  2. Forums Index
  3. General SEO Issues / Crawler, Spider, and User Agent ID 10:00 pm May 4, 2026