Please expand on "the connection is dropping".
Can they authenticate and connect, but then the connection drops later, or are they failing to authenticate?
In my experience (small home network), using the one certificate for every user is not a problem. If you want to maximise security, you could generate and deploy a new cert for each user but it depends on how much admin you want to do.