I check (Only works if they don't have their own ip) the IP of the web site. Also sometimes a web host leaves extra HTML and if they do they usually leave a commet tag with it saying there company name.
I check (Only works if they don't have their own ip) the IP of the web site.
Checking the IP is a good way, to a certain extent. If the host is colocated and using IPs assigned by the datacenter, then you can find out "where" the site is hosted, but not necessarily "who" hosts the site. You could check DNS, but the hosted site may use their own nameservers and not those of the host.
Not quite sure why you think checking the IP only works if they don't have their own. We give out static IPs to all hosting accounts, so you can certainly find out if we host a site by checking the IP (as we own the IPs as well).
If all else fails, why not just contact the site:
"Hi, I was found your site on [insert search engine here] and was impressed with how fast it is. Where is it hosted, because my site seems to be really slow in comparison..."