Forum Moderators: open
I have them in the robots exclusion.
Which Google follows however they persist upon continuiosly reading.
Apparently Google needs reassurance ;-)
Almost simultaneously spidering was begun by PICA bot which i had removed from my robots because they hadn't visited in so long.
Now their back in although Pica can't read the robots.
It seems they keep getting a 403 ;-)