You don't really uninstall a source rpm using the rpm command, but it doesn't matter, as uninstaling it won't change anything. There's certainly no need to reformat.
If you aren't a developer, don't even look at .src.rpm files; they won't do anything for you. If, however, you've inadvertently installed one, don't worry; it won't hurt anything.