Welcome to WebmasterWorld Guest from 184.108.40.206
Forum Moderators: bakedjake
jung.excite.com.......57 requests for robots.txt
daal.excite.com.......46 requests for robots.txt
tympani.excite.com....29 requests for robots.txt
Anyone else see anything like this? Any ideas what excite might stand to gain by requesting a single robots.txt file over 100 times in the course of a weekend?
Hit about 10 of my sites going after just the robots.txt file and the root html page. A few times it would request the robots file 10x in a row. I think the total amount of requests were in the 70-80 range per site. Just on those two files alone.
.....43 (14 others under 10)
which adds up to 13424 requests on a single day, 50 times more than any other file on the site. This number has increased every day since May 22. I've tried to find an e-mail address at excite.com where I can notify them about the problem but have been unsuccessful. Help would be appreciated.
Excite hasn't a) spidered my site itself, b) sent me any traffic, or c) shown any pages differing in any way from Ink results for our company name. I've pretty much written them off, but the sudden assault on my robots.txt definitely threw me off.