Welcome to WebmasterWorld Guest from 18.104.22.168
Forum Moderators: open
On the first of the month, Slurp/cat visited our site and ravished it. It spidered the same dozen pages about one hundred times each.
Yet on the 2nd and 3rd of the month Slurp/Si turned up, asked for robots.txt (of which there isn't one) recieved the usual 404 and left without looking at a single page.
Being fairly new to this sort of thing, I am left baffled. Any sugestions would be welcome.