I've been running a bot for a while that goes around to a few sites and continually fetches and stores (in MySQL) data. It's been working great for about 3 months, but I'm working on some major upgrades. The next version will traverse log-ins (such as with sites like nytimes.com).
My code uses cURL (via PHP) to access the pages. I do fine with sending the POST variables, just need to know how to store/send cookies.