Yahoo Consumes 6 GB in three days

No response from anyone from Yahoo

[Harry]

I've asked Yahoo experts to look into this matter and I have had no responses yet. What I don't understand is why their robots have so much problems with an html Web site. Except for the Invision Board and OSC, 90% is static.

Are the spiders getting stuck somewhere in OSC or Invision?

Yahoo's consumption is worse than Microsoft's. Their response, so far is totally sub standard. Folks from Google reply within 24 hours to all sorts of concerns. Sometimes, they reply within an hour.

I guess I've been spoiled by good service.

My $100 question is how come Google can program a spider that doesn't get caught in a Web site with 90% html and Yahoo and Microsoft can't?

I never get bandwidth issues with Google - touches wood - Yet, Microsoft and Yahoo are nothing but trouble. The only solution that works is banning them completely - but surely, in an interdependent relationship, that can't really work?