Ok, well the theory in what you've done sounds correct, so the problem most likely is a small step along the way.
So from what you're written I gather you have:
1. forwarded subdomain from ISP to external IP address of SME server
So, sme.yourserver.com is now pointing to your SME external IP, I take it you've given it time to propagate the DNS entry?
2. forwarded the relevant port on the SME server to internal address of terminal server
So, you're incoming port is set up to forward to the IP internally of the terminal server with the destination port 3389? (as with my example above) Try forwarding
both the TCP and UDP ports with the same source and destination if you haven't already.
3. attempted to connect from home via RDP to subdomain
That's right, I have a setup similar to this and it works. So, pretty much you'll just have to work back and figure it out...can you rdp to you terminal server internally? If you try to rdp to your server from home, does it work if you rdp to the external ip instead of the dns entry you made?
Check everything, remember to connect you will have to rdp with the format
yourhostnameorip:1234 where 1234 is the source port you set up.