I think the answer is very easy: once a site earns (or tries to earn) money from the infringing content, be it directly or indirectly. Let's say, you run a hobby site on your own server with your own domain, www.widgets.com. You do not have any advertising or affiliate links. The site is ad free. Someday you decide to embed a video that matches your topic perfectly and explains stuff to your visitors. As you still do not run any ads, you should be fine. (BTW, I'm not a lawyer.)
Your traffic rises, and you start to embed more videos. Again, word of mouth brings more visitors to your site, and someday you look at the traffic, thinking - "well, if I put ads on the site, then I can make a lot $ per month". And you do. And sooner or later you will find a nasty letter from some IP holders in your mailbox.
I believe you are crossing the border once you are trying to monetize a site. No ads, no affiliate program, no subscription fees = non commercial. Ads and/or affiliate program and/or subscription fees = commercial.
I do not believe that building up traffic per se is already commercial nature - but right holders might see this differently. The problem definitely comes when you are trying to monetize that traffic. If you find a buyer for your website full of infringing videos, fine, take the money and run. But the buyer will now have the trouble of monetizing. And I guess he will have a hard time to do so.
In fact, on Google Video there are relatively few infringing videos. Also, they are doing deals with media companies. So your statement is not fully correct. However, when you look at YouTube, your question is very valid (especially as they no wbelong to Google). The answer is: They are trying to get away with it. :-) I guess they won't, but that's for the courts to decide. Just the fact that roughly 30,000 videos needed to be removed recently tells you something about the sheer size of the problem. This was still just scratching the surface.
It will be interesting to see who wins this battle.