most likely, your being rejected because your isp blocks all smtp traffic, except through their servers. setup your server to go through theirs.
need more details to the specifics of your system as to why it's slow. Could be your server isn't powerful enough. you be you have outgoing authentication on when none is needed.
of course, you can always search first.