Forum Moderators: goodroi
The info page says it's a distributed crawler, so just like my policy for the cronic robots.txt violater Grub, I banned the user agent and the entire IP block associated with the offending IP.
there's nothing more annoying than a user complaining about bugs without showing proof.
That's one perspective. I'm particularly annoyed by arrogant developers who respond with insults and sarcasm. Much of this thread could have been avoided if the bot owner hadn't offended me.
By the way, the proof part has already been covered multiple times in this thread.
...if the bot owner hadn't offended me
Ah ok - I see there was no real problem with the spider at all - this was a personal vendetta thingy - cool, although off-topic for this forum.
I can't find the post where you provided the data to back up your claim,
Did you find the posts where the bot owner rejected my offers to supply the data to back up my claim?
please repost - someone who understands robots.txt and troubleshooting in general will be able to assist you.
Again, you're way behind. That's already been covered. I wasn't looking for assistance. The bot owner superficially acted like he wanted assistance, but then rejected my assistance.
...but then rejected my assistance
I guess that occurred outside this thread - is that when the personal offense took place as well?
Ah ok - I see there was no real problem with the spider at all
Perhaps you missed the posts where the bot owner verified that his bot has in fact violated robots.txt
this was a personal vendetta thingy
Nope. You're obviously either trolling or you haven't read the whole thread.
Perhaps you missed the posts where the bot owner verified that his bot has in fact violated robots.txt
I never said that my bot is flawless, however I can't verify bug report if it lacks specifics.
I wasn't looking for assistance.
Your attitude suggests to me that you were looking for some cheap bashing without any desire to back up your words with proof. Generally the people who act like that have no proof in the first place.
but then rejected my assistanceI guess that occurred outside this thread
You haven't read the whole thread have you? Either that or you're trolling.
...the bot owner verified that his bot has in fact violated robots.txt
He violated your robots.txt? Could you post a copy of it here? Someone should be able to figure out if the mistake is with the spider or your ruleset.
Did you even read the thread? That was answered way back at the beginning and rehashed many times throughout the thread.
I had fun reading all 4 pages.
But from what i was reading you are not being very cooperative with Lord Majestic when he clearly is asking for the information he needs to look into the problem and you are not giving him the exact information he asked for. Infact you could have stopped this 4 page fun read if you just gave him that information. Infact im rather amazed at how long this thread has went on for.
I say infact a lot eh?
