Your catchall account does not get emptied for several reasons:
1) Mail contains invalid email addresses and emails gets rejected during the process of downloading, leaving them in the catchall mailbox
2) Mail in the catchall box does not pass the various filter criteria set up on the SME Server and it gets rejected "at the front door", leaving it behind in the catchall mailbox.
Ultimately, this will bring the mail system down. A collection cycle will not complete before the next collection cycle starts and the problem grows. The catchall mailbox size also grows and probably / possibly exceeds the storage limits set by the catchall provider.
I worked around this issue at the mail MX record level (DNS), setting the SME Server IP to priority 0 and the catchall mailbox IP to Priority 5. For me, this approach works extremely well. There are probably other ways to approach this also. I absolutely get why you like the concept of a catchall mailbox.