One way: Put image plus number in paired groups.
<div class = "holder">
<div class = "picture">
{picture here}
</div>
<div class = "number">
{number here}
</div>
</div>
If you set the "holder" div to "display: inline-block" you can march them across the page, each one taking only as much space as it needs. The image doesn't absolutely have to be inside of something; it's just tidier if you can keep it structurally parallel to its caption. Or number, or whatever it is.
If you set the "picture" and "number" divs (both of them) to "display: table-cell" you get them side by side and you can set the number part to "vertical-align: middle".
display: table-cell is not used all that often-- except of course for actual table cells, where it is the default-- but it's essential in one situation. A table cell is the only block-level element that allows vertical alignment. That's alignment within itself, not in relation to other nearby elements (like superscripted text). Note that table cells don't have margins; you'll need padding instead.
Matter of fact, you can probably run your existing html through a quick global replace using a Regular Expression and convert them all to divs.
If you want a sneak preview, don't even change the html. Just redefine the existing elements in css:
table.mypictures {display: inline-block;}
and so on.
There is a time and a place for floats. This isn't one of them ;)
And, if you later change your mind and decide you want the number to go below the picture instead of to the side, simply change the display properties for picture and caption to "block". (Or, in the case of a div, don't say anything at all; block is the default.)
