As promised, I came back and did more testing. And as I suspected, it's browser-dependent.
I kept the "child" at 100px square, and fiddled with the "parent".
Webkit (Safari, Chrome) and Mozilla (Firefox): minimum size to avoid overflow is width 101, height 102. I do not pretend to understand this, but them's the numbers.
Opera: 101 for both
MSIE 5 pause for hysterical laughter: 100px for both
The scrollbar itself is probably OS-dependent. On my system it's 15 pixels wide, so that's how much you have to add to the parent in order to keep both sides from jumping to a scroll bar as soon as one side becomes too short.