Yeah, I really don't think they can use bounce by itself and definitely not site-wide imo ... The only way I could see them using it is 'click-back, re-click' and imo it would have to be 'constant' and the surrounding results could not have a similar pattern.
Beyond that, it gets really fuzzy, because it's really tough (impossible?) to tell if the searcher found the answer they were looking for quickly on a well designed site or if the site was 'garbage' when they got there ... Really, once you stop looking at the bounce rate of your site and try to gauge what a bounce rate tells you for a site you can't 'see' (design, etc.) about a visitor you don't know it gets super noisy, because a bounce can mean so many different things in so many different situations.