It's easy to see the spammy looking sites as black hat but some very professional looking sites could be using black hat practices. It's more if they are doing things Google or other search engines would see as cheating to get up in the rankings.
In other words the bad guy may be sneaky and wear a white hat instead of his black hat.
For me, Black Hat goes beyond spammy sites. It implies dirty or illegal tricks like:
1) Phony links traded for for real (<a> href=) links. 2) Scraped / plagiarized and/or uncredited content 3) Blatant KW stuffing, especially when cloaked or masked. 4) Link farms, link rings/triangles and variations. 5) Framing the contents of other sites without permission or notification. 6) Click attacks on competitors adsense or other ads. 7) Denial of Service attacks (DOS) 8) Rewriting pages of other sites to capture them within the BH domain. 9) 301 page hi-jacking. 10) Machine Spamming blogs or guestbooks to gain links. 11) Slander.
That's only 11 things off the top of my head. There must be many many more. -Larry