Forum Moderators: goodroi
The info page says it's a distributed crawler, so just like my policy for the cronic robots.txt violater Grub, I banned the user agent and the entire IP block associated with the offending IP.
there's nothing more annoying than a user complaining about bugs without showing proof.
That's one perspective. I'm particularly annoyed by arrogant developers who respond with insults and sarcasm. Much of this thread could have been avoided if the bot owner hadn't offended me.
By the way, the proof part has already been covered multiple times in this thread.
...if the bot owner hadn't offended me
Ah ok - I see there was no real problem with the spider at all - this was a personal vendetta thingy - cool, although off-topic for this forum.
I can't find the post where you provided the data to back up your claim,
Did you find the posts where the bot owner rejected my offers to supply the data to back up my claim?
please repost - someone who understands robots.txt and troubleshooting in general will be able to assist you.
Again, you're way behind. That's already been covered. I wasn't looking for assistance. The bot owner superficially acted like he wanted assistance, but then rejected my assistance.
...but then rejected my assistance
I guess that occurred outside this thread - is that when the personal offense took place as well?
Ah ok - I see there was no real problem with the spider at all
Perhaps you missed the posts where the bot owner verified that his bot has in fact violated robots.txt
this was a personal vendetta thingy
Nope. You're obviously either trolling or you haven't read the whole thread.
Perhaps you missed the posts where the bot owner verified that his bot has in fact violated robots.txt
I never said that my bot is flawless, however I can't verify bug report if it lacks specifics.
I wasn't looking for assistance.
Your attitude suggests to me that you were looking for some cheap bashing without any desire to back up your words with proof. Generally the people who act like that have no proof in the first place.
but then rejected my assistanceI guess that occurred outside this thread
You haven't read the whole thread have you? Either that or you're trolling.
...the bot owner verified that his bot has in fact violated robots.txt
He violated your robots.txt? Could you post a copy of it here? Someone should be able to figure out if the mistake is with the spider or your ruleset.
Did you even read the thread? That was answered way back at the beginning and rehashed many times throughout the thread.
I had fun reading all 4 pages.
But from what i was reading you are not being very cooperative with Lord Majestic when he clearly is asking for the information he needs to look into the problem and you are not giving him the exact information he asked for. Infact you could have stopped this 4 page fun read if you just gave him that information. Infact im rather amazed at how long this thread has went on for.
I say infact a lot eh?
I never said that my bot is flawless, however I can't verify bug report if it lacks specifics.
Already covered multiple times in this thread. I offered more information. You rejected my offer. You kept on with the sarcasm and insults. I withdrew my offer.
Your attitude suggests to me that you were looking for some cheap bashing without any desire to back up your words with proof. Generally the people who act like that have no proof in the first place.
And your attitude suggest to me an arrogant jackass, which is why I withdrew my offer of assistance. I offered more data. You rejected the offer. Then you try to say that I have no proof because you rejected my offer to supply you more information.
This is all just repeating what has been covered multiple times in this thread.
He violated your robots.txt? Could you post a copy of it here? Someone should be able to figure out if the mistake is with the spider or your ruleset.
Again, an obvious troll. That's already been covered multiple times in this thread.
And your attitude suggest to me an arrogant jackass
I will let the readers judge on who is who :)
Since you progressed to name calling I take it that this discussion is pretty much over. I am still interested in your domain name to ensure future compliance with your robots.txt I am happy to add your site to a very short list of permanently banned sites. I promise not to actually look at your site, and in a few years I will tell you whether I noticed its abscense from the index or not :)
[edited by: Lord_Majestic at 7:27 pm (utc) on June 14, 2005]
...you try to say that I have no proof
Ah cool, I take it to mean you have proof - can you please post the robots.txt file for the forum members to examine?
But from what i was reading you are not being very cooperative with Lord Majestic when he clearly is asking for the information he needs to look into the problem and you are not giving him the exact information he asked for.
That exact subject has already been covered multiple times in this thread so rather the repeat it here, I suggest you try reading the thread more objectively so you won't gloss over statements you might have missed.
Infact you could have stopped this 4 page fun read if you just gave him that information.
Or he might have stopped the thread by examining the information I offered rather than rejecting it as insufficient without even examining it.
Infact im rather amazed at how long this thread has went on for.
You and me both.
I take it to mean you have proof - can you please post the robots.txt file for the forum members to examine?
Still trolling? Already asked and answered multiple times in this thread.
Or he might have stopped the thread by examining the information I offered rather than rejecting it as insufficient without even examining it.
There was nothing to reject, no information that could have reasonably lead to establishing whether something was wrong with my implementation of robots.txt standard.
...answered multiple times in this thread
I cannot locate the post with the robots.txt information in it - please post your robots.txt information so the forum can examine it and validate your claims about the member's spider.
I take it that this discussion is pretty much over.
I thought it was over long ago since the same things just kept getting repeated over and over.
I am still interested in your domain name to ensure future compliance with your robots.txt
Already covered multiple times in this thread.
I won't quote the rest of your post since it's just more of the same sarcasm that fanned the flames of this thread to begin with.
Or he might have stopped the thread by examining the information I offered rather than rejecting it as insufficient without even examining it.
Dont worry I did read it all and again it all comes down to you posting your robots.txt - That has been covered so many times (you say) in this thread.
Infact (Yes I know again) - Why dont you post it now to shut everyone up :) You know - to add a twist to this thread!
There was nothing to reject,
You rejected my offer.
no information that could have reasonably lead to establishing whether something was wrong with my implementation of robots.txt standard.
You're in denial. You can't say that the information was insufficient without actually examining the information.
You're in denial.
One of us sure is in denial ;)
Now where is that robots.txt you so persistently denied to me?
I cannot locate the post with the robots.txt information in it - please post your robots.txt information so the forum can examine it and validate your claims about the member's spider.
You're just trolling. What you're asking has already been answered multiple times in this thread.
you're in denial. You can't say that the information was insufficient without actually examining the information.
lol. I cant add any value to that statement...
...without actually examining the information
What information are you referring to? Do you have a robots.txt file you could post here? That might clear up the misconception that there is no information to examine.
Dont worry I did read it all and again
I doubt that considering...
Why dont you post it now to shut everyone up
You must have missed the post where after multiple offers I finally rescinded my offer due to his attitude.
...after multiple offers I finally rescinded my offer
No need to offer, you can post the offending robots.txt file here without any preamble - we're not fussy.
What information are you referring to? Do you have a robots.txt file you could post here? That might clear up the misconception that there is no information to examine.
Again, that just shows that you either haven't read the thread or you're trolling. Already asked and answered multiple times in this thread.
No need to offer, you can post the offending robots.txt file here without any preamble - we're not fussy.
See above.
You must have missed the post where after multiple offers I finally rescinded my offer due to his attitude.
I didnt miss it, Its still not the point - you had problem and posted information, The bot owner asked for more information as what you had gave wasnt enough to look into it and also asked for the robots.txt file and any URLS. You didnt post the information. Regardless of what the bot owner said it would be in the best interest for everyone to see the file.
