Originally posted by Gregg Bolinger:
I really like http://www.dreamhost.com. They have One click installs for everything you requested.
I can second Gregg's opinion about DreamHost. I've been running a couple of domains with them for a bit over a year and I'm quite happy with their service.
The only downside I can think of for your case is that DreamHost doesn't support
Java so you're down to implementing your web applications in PHP or Ruby on Rails (or good ol' CGI scripts...).
Anyway, if you decide to sign up, do consider using some promotion code. I'm running the community website for
Agile Finland with DreamHost and there's a promotion code listed in the footer, which gives you a $50 discount and contributes slightly less to the hosting bills of agilefinland.com.