Basically this is the problem, my robots.txt says this:
Which according to the protocol, as far as I can tell, is wrong. But it is approved by the validator. Google obeys it but the Turnitin Robot (and possibly others) do not.
I have made changes like this:
i.e. the main issues seemed to be the wild card and the absence of the full path.
Any suggestions? - is there another more recent protocol that I am missing or is this a bug?
PS I do hope this is the correct place to post this message!