This seems a little strange to me.
There are multiple ways do to e-commerce on a website, to me one of the main reasons you would pick a wordpress based e-commerce solution would be the fact it can integrate into an existing site. Putting it on a separate install defeats the purpose as I see it - you might as well use a dedicated e-commerce system like opencart or prestashop etc if you are going down that path.
Just my thoughts, others may not agree.
From an SEO point of view, the fact it's different installs wouldn't affect much as search engines look at the generated code, not the back end scripting and database that is generated code. Obviously all the normal stuff re setting up the site properly to be search engine friendly still apply.
Which e-commerce plugin are you using? If it were me I would look at which plugins you are using, and search to see if there are any known issues. Then would do a test install to see how it goes, and test etc. Obviously new versions could create new problems, but it's less likely than if you don't test at the start.
Relying on a general comment from a host about "some plugins don't play nice with some e-commerce software" is just that, a blanket statement that likely may not apply to your situation.