Good question arnick!
Speaking as TL, for our stuff, we use a proprietary algorithm to compute the city center based on population density, number of businesses, etc. Geometric center is "close enough" about 60% of the time, but fails miserably when you get cities near a major body of water.
Zip code centroids are computed as one would expect.
So, our distance estimate will be different if you input a zip vs. a city.
One thing I've noticed is that some local search engines will pick a zip code as the center of the city, and use the centroid of that zip code for all of its distance calculations when a user inputs a city. Often, they'll pick the zip code of the main post office. This fails miserably in a town like Cleveland, where the main post office is about 4 miles from what would be considered the "city center".
FWIW: Y! Local is using the same centroid for Chicago that we are. We made a significant change about a year ago to how we compute those centroids, and they're probably using similar data.
Oddly enough, the centroids that are being computed by Y! and us are not what I would pick as the manual centroid for Chicago, and probably throws off downtown queries quite a bit. This is a hard question, though - what do you pick as the center? Magnificent mile, which is where most tourists would consider the center of the city? Or the loop, which is where most residents would consider the center? Neither of those are the geometric or density center, BTW.
One idea I've been throwing around is that perhaps the city center should not be a point, but an area. That makes distance calculations harder, but for downtown-type queries, it makes a lot of sense.
Ex: Embassy Suites in Chicago [beta.truelocal.com] - I would not say that the first two Embassy Suites are 3 miles from the city center - they're both very much in the city center for someone who may be performing that query, but that search result doesn't give enough information for you to know that.
The whole city center thing is a tough problem.