Yes, SME server allows you to choose netmasks other than CIDR /8, /16 and /24 (i.e. other than 255.0.0.0, 255.255.0.0 and 255.255.255.0 respectively).
You can also increase the number of IP addresses by using a router and the local networks panel. However, SME server won't provide DHCP services to the additional network(s).