Forum Moderators: open

Message Too Old, No Replies

cj.com spider?

         

jp_19

1:56 pm on Sep 25, 2003 (gmt 0)

10+ Year Member



I keep seeing mx4.cj.com indexing "/" and "/robots.txt". I don't have an account with them, so what's the deal? Anyone know what they are looking for?

Thanks.
-jp

Sadguy01

2:10 pm on Sep 25, 2003 (gmt 0)

10+ Year Member



It's a mail server of Commission Junction. When an advertiser gets a request from you to join their program, they're probably clicking through to your homepage from their CJ mail.

jp_19

2:35 pm on Sep 25, 2003 (gmt 0)

10+ Year Member



We aren't applying to any affiliate programs and this is a daily occurance. If someone was clicking through, then it would come from their IP. This is a spider.

jp_19

3:17 pm on Sep 25, 2003 (gmt 0)

10+ Year Member



Just dug a little deeper in the logs:

mx4.cj.com - - [25/Sep/2003:06:28:04 -0400] "GET /robots.txt HTTP/1.1" 200 105 "-" "CJNetworkQuality; [cj.com...]
mx4.cj.com - - [25/Sep/2003:06:28:04 -0400] "GET / HTTP/1.1" 200 16965 "-" "CJNetworkQuality; [cj.com...]

After reading:
[cj.com...]

I guess I need to ask CJ why they are spidering our site, since we don't have an account.

Thanks.
-jp

fiestagirl

4:05 pm on Sep 25, 2003 (gmt 0)

10+ Year Member



There is some more info about this here from June 03:
[webmasterworld.com...]