While A2Hosting has a pretty darn good bang for the buck, I recently had to look for a web host with even more advanced features. For instance, I needed to be able to compile a more recent version of the git DVCS, and use it on my account of course.
So I looked around some more and settled on WebFaction. I had wanted to try them out for a while now based on all the good reviews you can find on the Internet on them, and now that I have I must say that I’m pretty impressed.
I was able to compile git using my SSH account, along with the dependencies to be able to use git-svn, and it all went pretty flawlessly.
The control panel at WebFaction is awesome, easy to use, and it seems one can host just about everything on their servers: anything PHP, all sorts of Python and Ruby frameworks, it just looks fantastic.
In most cases you are running your own private Apache daemon, so you control your own configuration, add the modules you want, etc.
Pretty good stuff, check them out!