Google remembers urls and visits them directly, I don't think you have much to worry about. Google does keep track of how many internal links point to each individual url but it doesn't sound like your planned changes will dramatically change this.
Category pages in general: Google doesn't return them very often unless they are built in such a way as to have useful content. Google doesn't want to send visitors into a list of links of any kind really. I have made my category pages "noindex" and not lost traffic at all in the past(noindex pages still pass rank). I would not change the category urls however since they do have some weight in terms of pagerank flow, keep old urls whenever possible.
If you must change a category url then make sure you 301 redirect the old to the new and update all links to reflect the change. Above all else focus on improving visitor experience first. Since your content page urls aren't changing I wouldn't worry about losing traffic to them. Also, before ditching old category urls, do a link audit to see if you have solid incoming links pointed to them. If you do it's possible your sites rank may stumble a bit as the value is not carried 100% through a 301 redirect. If this is the case get in touch with the link owners and ask them to update the link on their site.