Forum Moderators: goodroi
The info page says it's a distributed crawler, so just like my policy for the cronic robots.txt violater Grub, I banned the user agent and the entire IP block associated with the offending IP.
There was nothing to reject,
You rejected my offer.
no information that could have reasonably lead to establishing whether something was wrong with my implementation of robots.txt standard.
You're in denial. You can't say that the information was insufficient without actually examining the information.
You're in denial.
One of us sure is in denial ;)
Now where is that robots.txt you so persistently denied to me?
I cannot locate the post with the robots.txt information in it - please post your robots.txt information so the forum can examine it and validate your claims about the member's spider.
You're just trolling. What you're asking has already been answered multiple times in this thread.
you're in denial. You can't say that the information was insufficient without actually examining the information.
lol. I cant add any value to that statement...
...without actually examining the information
What information are you referring to? Do you have a robots.txt file you could post here? That might clear up the misconception that there is no information to examine.
Dont worry I did read it all and again
I doubt that considering...
Why dont you post it now to shut everyone up
You must have missed the post where after multiple offers I finally rescinded my offer due to his attitude.
...after multiple offers I finally rescinded my offer
No need to offer, you can post the offending robots.txt file here without any preamble - we're not fussy.
What information are you referring to? Do you have a robots.txt file you could post here? That might clear up the misconception that there is no information to examine.
Again, that just shows that you either haven't read the thread or you're trolling. Already asked and answered multiple times in this thread.
No need to offer, you can post the offending robots.txt file here without any preamble - we're not fussy.
See above.
You must have missed the post where after multiple offers I finally rescinded my offer due to his attitude.
I didnt miss it, Its still not the point - you had problem and posted information, The bot owner asked for more information as what you had gave wasnt enough to look into it and also asked for the robots.txt file and any URLS. You didnt post the information. Regardless of what the bot owner said it would be in the best interest for everyone to see the file.
