It is possible to rent an Amazon EC2 instance and put a MySQL database on it. But practically that solution is nothing different from a dedicated server you have now already.
The other option is to use the Amazon Relational Database Service which runs MySQL 5.1. This service is still in beta though.
The main problem you will run into are security, because your database now has to be open for requests over the Internet which adds additional vectors for hackers, and latency. Especially with web pages which are built using multiple SQL queries you may see a noticeable slowdown of page generation speed due to the time needed to transfer the SQL requests to the Amazon server and sending the responses back to the web server front-end.
Amazon EC2 instances are paid per instance per runtime. Having more instances each running one database will cost you more money than having all databases in one instance. You can have multiple databases on one EC2 instance, so from a financial point of view it would be wise to run all databases from one EC2 instance.
I am not sure about the functionality and pricing of Amazon RDS instances. Because it is fully compatible with MySQL 5.1, you should be able to create more than one database in one instance, which is also more cost-effective than having one instance per database.