Depending on what CMS you are using, plugins may already exist for this. Of course, if you're planning on rolling your own comment system then you may have rolled your CMS, too. ;)
As far as caching goes, it should not preclude the opportunity to have an AJAX infinite scroll comment system. You may have to do some "hacky" things to break through cached pages, but if you do alot of aggressive caching now then you've probably already know what I'm talking about.