Forum Moderators: open

Message Too Old, No Replies

BeB-cart

Why is a shopping cart crawling my site?

         

GaryK

7:34 pm on Nov 9, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



BeB-cart 3.0.2 (2008-11-05)
123.198.1.nnn
pc601c8.accauc00.ap.so-net.ne.jp

netname: So-net
descr: So-net Entertainment Corporation
descr: 2-1-1 Osaki, Shinagawa-ku, Tokyo, 141-6010, Japan
country: JP

It was well behaved, but I don't understand why a shopping cart is downloading my browscap.ini file.

incrediBILL

3:27 pm on Nov 15, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Did it just download one file or crawl your site?

It's possible it uses your browscap.ini and has a built in update function.

GaryK

5:33 pm on Nov 15, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



It didn't do a full crawl, but it took more than just my browscap.ini file. The pattern looked like a human doing the browsing. After it looked around it waited a minute then took the PHP version of my browscap.ini file and left. Normally an auto-updater looks at one of my version files first to see if it needs to download the full file.

Unfortunately I can't find out anything more about the site this UA came from because it appears to be a one-page site with no text. Not even in the meta tags. Just a single graphic that's entirely in Japanese.

Pinging the IP Address results in a timeout so I'm not even sure if the IP belongs to the cart's website.

I don't want to ban it without cause. And yet right now in a strict technical sense I do have cause because automated agents are supposed to check one of my version files before downloading one of my data files.

I'll know tonight after midnight GMT if it visited again this past week.