Welcome to WebmasterWorld Guest from 126.96.36.199
Forum Moderators: goodroi
# robots.txt for mydomain.com
Or will I have to add the follwing at the end of the file:
If a add this, will "ia_archiver" back off as soon as it reads that it's not welcome, or will it interpret the whole robots.txt and match the "global allow" entry?
PS: Hope I am in the right forum. Couldn't find a more appropriate one in the "Forum Index".