This may not be a great big help as I am still a very new user to SME....
But I've but a simple server using a Via 600 mhz fanless board, and 512 mb of RAM. Outside of some odd slowness when first connecting to the box (by either SSH or IMAP), the server really flys.
I'm presently running both Mysql and Postgres, ClamAV and Spam Assasin, with some other various bits and bobs. I'm kinda like yourself, I need to run about a dozen low-mid level web sites and email servers.
So your spec would easily do circles around mine! I think you should be ok.
Geoffrey