lucy24 - 5:17 am on Nov 8, 2012 (gmt 0)

As promised, I came back and did more testing. And as I suspected, it's browser-dependent.

I kept the "child" at 100px square, and fiddled with the "parent".

Webkit (Safari, Chrome) and Mozilla (Firefox): minimum size to avoid overflow is width 101, height 102. I do not pretend to understand this, but them's the numbers.
Opera: 101 for both
MSIE 5 pause for hysterical laughter: 100px for both

The scrollbar itself is probably OS-dependent. On my system it's 15 pixels wide, so that's how much you have to add to the parent in order to keep both sides from jumping to a scroll bar as soon as one side becomes too short.