I'm not really an expert but some pointers

first of all the server shouldn't have to restart after using portforwarding, second thing is sme uses iptables.
does the protocal your using need tcp and udp ??
once i had something happen like that and i think it was mainly due to using both tcp and udp on the same port, but it could have only been me

try removing udp and and see how you go.
oh yeah you haven't over looked forwarding the port on an external router
