FYI, I know next to nothing about Linux in general, and only a little more about SME. But I don't think you can just remove Apache without a bunch of work. As is, SME has fairly low system requirements (not sure how much more the web stuff you require would add to that). It would probably be a lot of work for a small gain to swap Xitami in place of Apache. We have a Cyrix PR300 with 64MB acting as gateway/router and IMAP/web server for our small LAN (about 20 people) and I haven't noticed any slowdown on anything...
Again, I'm no expert, but I'd guess that it's more trouble than it's worth. FWIW, I did use Xitami on my Windows box 4+ years ago and liked it then.