Welcome to WebmasterWorld Guest from 18.104.22.168
Forum Moderators: mack
joined:Jan 20, 2003
Personally, bandwith is cheap for me and I'm willing to feed MSNbot as long as it doesn't impact performance for users.
So how much bandwith are you allowing?
Fortunately, bandwidth is cheap for my US based sites, so I just let MSNbot get her fill. Maybe that saved some of you a few bytes. ;)
There is a robots.txt now. So don't assume that I was lazy. Don't make up scenarios when you don't know the full extent of the story.
As for the "rude" part, had I not added that word, you guys wouldn't have perceived my message as rude, just very direct. It's all in the packaging.
However, calling someone "lazy" is rude and disrespectful.
Remember guys, TOS 4 and 19 applies to ALL of our members, regardless of who employs them.
What about items 18 and 20? The search company representatives are not standard posters, they occupy a special position on these forums as far as I can tell, and represent clear commercial interests, but do provide a certain insight into those commercial entities, as long as you bring your salt with you.
It would be slow as crap, but I can't think of any other reason why it would do what it's doing. Think about this... they have a cached snapshot of your page which is used for text queries just like Google but maybe they're going a step further and actually comparing the results to the "live" page to make sure that it hasn't changed since they cached it, and that it's not a dead link.
I don't think this model would be very efficient in a production environment - in fact I KNOW it wouldn't - but I can see some advantages for them to run this way while testing.
joined:Apr 22, 2004