I am not sure you can avoid this unless the business was incorporated in the state you plan to live in. However I am not a CPA so you should talk to one, but my understanding from my own business is you are always responsible for the state (and possibly local) taxes that a corporation, LLC, or partnership make in the state it is set up in. In addition, you are responsible for federal and possibly local taxes both where the business is at and where you will live in the future.
This isn't necessarily a bad thing, but it is extra paperwork you need to maintain, but hopefully the extra income you are making would be an incentive to keep all of this paperwork and keep the business regardless of where it is located.
However one HUGE piece of advice I would give since you mentioned you had a partner is definitely incorporate in some form, i.e. LLC or Corp. and make sure you have a fully written partnership agreement that details EXACTLY who is responsible for what, how the money will be divided and re-invested, and remedies for dissolving the company if things don't work out. I have many nightmare stories I can tell you about partnerships that went bad even though everyone "were friends" when it was formed. A written partnership agreement will be worth its weight in gold in the future if a disagreement occurs and will save you a ton of heartache. See both an attorney and CPA about having this drafted.