Or it may mean they're poking all around the site trying to figure out why their keywords aren't there and why the site appeared on their search when they can't find anything relevant. I have frequently spent more time on unrelated sites for this reason, and I doubt I'm alone.
On the other hand, on a well-designed site I should be able to find the piece of information I seek almost instantaneously. Then I can move on.
A more useful metric would be to track which site is the last site they visit on the search, without coming back to the search page. That should mean they found what they are seeking.