Check out Proxim
http://www.proxim.com. They have wireless gatways that have RADIUS authentication built in, as well as per user bandwidth levels & throttling. The kit is not that expensive either.
Linux software for running an ISP? I know of two open source packages - one is from NoCat
http://www.nocat.net which is designed from the ground up for providing wireless services, and the other is Freeside
http://www.sisd.com/freeside/ which handles billing & administration. Both I believe are written in Perl.
As for SME being able to handle it - I have a Cobalt RAQ4 which at the moment has 120 websites being hosted on it and provides email for 3 times as many users. Add to that the constant FTP uploads and downloads, 80 MySQL databases and you can imagine that the poor old RAQ is under a lot of strain - no it's not, it handles it easily, and that's a Linux machine with 256mb of RAM and a lowly 400Mhz processor.
So I believe SME as a barebones Linux OS on a half decent machine will run your wireless ISP - but don't go chasing AOL - you may need two machines for that!