Log files are generally stored in /var/log/. The main system log file is /var/log/messages, but some systems like mail and the web server use separate log files (all in that directory). By default, the system logs lots of stuff.
For future reference, you didn't need to format and reinstall, so long as you still had physical access to the box. You could have rebooted in single-user mode, and changed the root password back to something usable (this has been discussed here before; a search should find more details).
IMO, as I said above, the Linksys router won't measurably improve your security by itself. It especially won't help if you're forwarding everything to the SME machine. Whoever attacked your machine almost certainly did so by accessing one of your open services, and you're continuing (or trying) to keep those open to the outside world. So if, for example, they exploited a vulnerability in Apache (the web server), they'd still be able to exploit the same vulnerability, since they still have access to Apache. If you don't want public access to your services, set the system up as a private server and gateway, and they'll be blocked. Either way, the Linksys router is a waste of money in your application.
On the question of mail ports, it isn't quite as simple as you suggest. Mail usually travels from server to server using SMTP, which uses port 25. So, mail coming in to your server (in most cases) comes in on port 25, and outbound mail leaves on port 25. That's the case if your system is the primary mail server for your domain. If your system is set up to use fetchmail for all incoming mail, then the status of port 25 isn't relevant.
On your specific question about how to set up the Linksys router, I really don't have an answer--as I've said above, it's not really likely to do you any good.