joined:Aug 27, 2002
Asmentioned in the title, my new domain gets visited by Slurp, but only the index.html and the robots.txt get hit.
I have links to inner pages on the index.html which are followed by other spiders (msnbot, googlebot) and the robots.txt isn't disallowing slurp.
What could be the reason for this "ignorance"?
It seems not to be technical.
any help is veryy much appreciated.