Koozali.org: home of the SME Server
Obsolete Releases => SME Server 9.x => Topic started by: ber on February 08, 2019, 04:17:38 AM
-
Hi guys. My admin inbox has been bloated because of a monitoring software wrongly configured. Trying to cleanout the inbox and the deleted folder is almost inaccessible using webmail or a email client software- 64K plus emails.
Is there a quick script i can run to clean out the admin inbox and the deleted.
Thanks in advance.
-
Thanks for asking -- I discovered looking into this that I had over 500K emails in my admin Inbox...
You can delete the extra emails from the command line.
These commands will completely empty the Inbox for admin:
find /home/e-smith/Maildir/cur -name "*$(config get SystemName)*" -exec 'rm' "{}" \;
find /home/e-smith/Maildir/new -name "*$(config get SystemName)*" -exec 'rm' "{}" \;
Notes:
* /home/e-smith/Maildir - admin's mail folder
* /home/e-smith/Maildir/new - new messages - never touched by an email client
* /home/e-smith/maildir/cur - messages that have been accessed by an email client (whether "read" or "unread")
* $(config get SystemName) - get the config variable "SystemName". SME includes this value in the name of each message filename by default. You could probably replace -name "*$(config get SystemName)*" with -type f, but I did my testing using the commands shown.
If you want to delete only the emails related to the mail loop (or whatever), find some text specific to the unwanted messages, modify the "find" command, and feed the output to "xargs rm":
In this example, I want to remove only the emails containing the string "MaxHwTemp":
find /home/e-smith/Maildir/cur -name "*$(config get SystemName)*" -exec grep -l "MaxHwTemp" "{}" \; |xargs rm
find /home/e-smith/Maildir/cur -name "*$(config get SystemName)*" -exec grep -l "MaxHwTemp" "{}" \; |xargs rm
If you want to avoid slowing your server down while the cleanup is running, put "nice" at the start of each command line:
nice find /home/e-smith/Maildir/cur -name "*$(config get SystemName)*" -exec 'rm' "{}" \;
nice find /home/e-smith/Maildir/new -name "*$(config get SystemName)*" -exec 'rm' "{}" \;
Be patient.
My server (ARM processor, spinning HDD) was deleting messages at a rate of 25 - 55 per second without "nice" and (unexpectedly) closer to 80 per second with "nice". 500K messages will take over 5 hours to clean up at 25/second, or 1.5 hrs at 80/sec
There is no output from the running command. Consequently, the workstation you're using could go to sleep and disconnect the session unexpectedly. To avoid this, run the commands from the console, or from a desktop that does not go to sleep, or use "screen" so you can start it up and disconnect until it's done.
https://www.rackaid.com/blog/linux-screen-tutorial-and-how-to/
[edit]fix time estimate for 25 messages per sec
-
[Update]
The commands I posted yesterday reduced the message count in /home/e-smith/Maildir/cur from 500,000+ to 947, but afterwards I could not access the Admin mailbox using sogo, and my system was pegged at a load average of 40+ with the CPU at 95%+, with lots of busy "dovecot-lda" processes.
Since I didn't need anything from the Admin mailbox, I tried doing this:
cd /home/e-smith
sv d dovecot
mv Maildir Maildir.xxx
mkdir Maildir
chown admin:admin Maildir
sv u dovecot
At this point, I could get to the Admin mailbox from sogo (no messages), and the various dovecot-specific index, acl, and uid files appeared in the new /home/e-smith/Maildir folder. However, my system's load average was still 40+ with 95%+ CPU.
On the assumption that the busy processes were related to dovecot attempting to re-index the Admin mailbox I did this:
kill -9 $(pgrep dovecot-lda)
signal-event email-update
CPU was better, but load average was still around 20 (not 0.20). lsof showed that there were still files in use by "imap" from the renamed "Maildir.xxx" folder, so I did this to kill those processes:
lsof |grep Maildir.xxx |awk '{print $2}' |sort |uniq |xargs kill -9
I think I could have avoided the various kill commands if I had started out by renaming and recreating "Maildir" yesterday instead of deleting the individual messages and leaving dovecot to clean up the indexes on its own, or possibly if I had included "sv d imap" along with "sv d dovecot" before renaming Maildir, plus "sv u imap" after the rename.
-
I think there is something you can do to delete out the old dovecot index files.
That lets it just reindex the few left rather than having to contrast & compare what has changed.
Have done it in the past but can't remember which files to dump. Sure there is stuff online.
-
Simply delete the old index file they will be created from scratch. Which is faster than erroring on all deleted mail still in the index.
-
Hi mmccarn
After running the script you recommended the server is now constantly running very high CPU usage. Can you advise how to fix.
[root@server e-smith]# top
top - 09:28:04 up 17:40, 1 user, load average: 10.65, 6.47, 5.82
Tasks: 239 total, 10 running, 229 sleeping, 0 stopped, 0 zombie
Cpu(s): 98.2%us, 1.7%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.2%hi, 0.0%si, 0.0%st
Mem: 4045920k total, 3164504k used, 881416k free, 303560k buffers
Swap: 16506876k total, 62204k used, 16444672k free, 585516k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
7748 qpsmtpd 20 0 248m 139m 3588 R 24.4 3.5 0:20.09 qpsmtpd-forkser
7740 qpsmtpd 20 0 252m 142m 3588 R 24.1 3.6 0:21.66 qpsmtpd-forkser
7749 qpsmtpd 20 0 249m 140m 3588 R 24.1 3.6 0:20.29 qpsmtpd-forkser
7753 qpsmtpd 20 0 249m 140m 3588 R 24.1 3.5 0:18.67 qpsmtpd-forkser
7743 qpsmtpd 20 0 250m 140m 3588 R 23.4 3.6 0:21.04 qpsmtpd-forkser
7744 qpsmtpd 20 0 251m 142m 3588 R 23.4 3.6 0:21.87 qpsmtpd-forkser
7752 qpsmtpd 20 0 248m 139m 3588 R 23.4 3.5 0:18.95 qpsmtpd-forkser
7758 qpsmtpd 20 0 248m 139m 3588 R 23.4 3.5 0:18.46 qpsmtpd-forkser
7836 root 20 0 35008 4300 1028 S 1.0 0.1 0:00.07 plymouthd
7976 root 20 0 106m 1872 1232 S 0.7 0.0 0:00.02 ifdown-eth
258 root 20 0 0 0 0 S 0.3 0.0 1:11.37 md1_raid1
964 root 20 0 0 0 0 S 0.3 0.0 0:42.39 kondemand/0
965 root 20 0 0 0 0 S 0.3 0.0 0:41.16 kondemand/1
2081 qpsmtpd 20 0 108m 22m 3644 S 0.3 0.6 0:03.19 qpsmtpd-forkser
7736 root 20 0 15160 1380 920 R 0.3 0.0 0:00.54 top
7840 root 20 0 106m 1788 1256 S 0.3 0.0 0:00.02 rc
7949 root 20 0 106m 1972 1288 S 0.3 0.0 0:00.01 K90network
...
-
I'm not sure you told us the whole story here. I presume you are still receiving lots of messages?
Check the forums for:
qpsmtpd-forkser high load
First thing to do is check your logs.
/var/log/qpsmptd/current
/var/log/messages
Somthing is keeping qpsmtpd busy, so what is it?