Topic: BCC Emails

[devtay]

Hello all. I have been using BCC for some time on my server and now have a ton of email (big surprise right) in my maillog Inbox. I don't usually mess with this account unless I need to search for a particular email. There are so many emails in my inbox now that I can't search or log in and view emails. I  have been through the forums and used pine, logged in and back out to refresh the dovecot database, tried moving messages out, etc. There are so many emails it's just impossible to find anything. For example, using select in pine results in me sitting forever until the server connection times out. So, I just decided to create a new user and change the BccUser to the new address. After all of that, I am going to be back in the same boat again in a few months.

I was wondering if anyone has any best practices or solutions for taking care of a massive amount of email. Is there an email client that anyone recommends? I have used thunderbird and outlook to try and solve this issue, but both lock up after a while. I need something that I can leave alone for a long time and not have to worry about it. Searching should be easy. Any suggestions are appreciated.

[CharlieBrady]

Google have great expertise in storage and searching. And email.

[Jáder]

MailArchiva

https://www.mailarchiva.com/

http://wiki.contribs.org/MailArchiva

Never used myself... but it's available.

[mmccarn]

How much mail is "a ton of email"?

Thunderbird (since the release of v3.x) provides near-instant search capability to 8GB+ of imap email on my SME server.

[devtay]

I did think about google. Their limit is around 8GB for the free accounts so I may have to look into a paid account. I use gmail personally so I know I can pull the messages off of the server with a gmail account.

Currently in my maillog account I have almost 32GB of email on the server. There are approximatley 120k emails in the inbox folder. I know, way too many. We send/receive around 800 emails a day (not counting the 5,000 or so that spamassassin takes care of for me).

I actually used to use mailarchiva but was having so much trouble with the indexes corrupting that I stopped using it. It didn't like the no date header emails created by my Dell networked printers. I couldn't get mailarchiva to install on SME per the wiki so I had to build up an Ubuntu server to get it running. Much prefer RHEL/CentOs/SME to Ubuntu/Debian.

The Thunderbird/IMAP 8GB statement has me curious. I am running a Dell PowerEdge 2970 Server - Quad-Core AMD Opteron 2373HE@2.1 Ghz with 4GB RAM - and the 64 bit kernel of SME. That should be plenty of power to do whatever I need to but it's sometimes not. When I use Outlook for example to connect IMAP, the processor load spikes and everyone starts complaining about slow email. I just thought IMAP was too processor intensive for that much data so I stopped using it. Maybe I need to look at some tuning on the server or add some memory. Still 31GB of email is a ton. Not sure what the max memory for my server is. Will check more into that.

I'm not trying to thread hijack, just interested in the searching/IMAP/Thunderbird comment.

Thanks so much for replying and offering advice.

[Stefano]

use imapsync and one or more "fake" accounts to divide your email by age..

HTH

[mmccarn]

I'm not trying to thread hijack, just interested in the searching/IMAP/Thunderbird comment.

I am amazed at the speed of Thunderbird's search function.  It handles my somewhat over-large mailbox much better than Outlook (I would use Thunderbird exclusively, except that my office mail server doesn't play well with Thunderbird and message "tagging").

I recently had to search all of my company's email from 2002 for some missing emails - I restored everything, moved it all into one email account, setup Thunderbird IMAP access, waited an hour or two while the offline cache was built, then had no trouble searching. (didn't find anything - we were mostly POP3, and often using the ISP's SMTP server at that point...).

Thunderbird also has an auto-archiving addon (or maybe it's a built-in setting) that will partition your email by year & month automatically.  I think you'd see a benefit from breaking up your single monolithic Inbox.

[devtay]

I think you'd see a benefit from breaking up your single monolithic Inbox.

  You know it! I think I am going to create another account on the server and play a bit with IMAP/Thunderbird. I try to segregate emails based on month and year anyways so that's enough for me to spend the effort. I do know that when I used it before, I didn't wait for the cache and did it during business hours. That probably had a major effect on what I was trying to get done.

Thanks again.

[mmccarn]

... and did it during business hours.

Here's another idea:  Setup a second SME server, and deliver the BCC emails to that server - then you can play around with it during business hours and have no effect on the live server.

You could also use the 2nd box as an Affa backup server, giving you a "warm spare" backup in case your primary SME has any problem.

[devtay]

You know it! I think I am going to create another account on the server and play a bit with IMAP/Thunderbird. I try to segregate emails based on month and year anyways so that's enough for me to spend the effort. I do know that when I used it before, I didn't wait for the cache and did it during business hours. That probably had a major effect on what I was trying to get done.

Thanks again.

I am now using Thunderbird/IMAP and it's working like a dream. I have about 3 GB (about 3 months worth) of email in the account right now and the archive function is doing a great job. Searching is a breeze and screaming fast. I think my new procedure will be to update the IMAP index every few days. This really fixes the issue.

I am also looking at imapsych and the Affa backup server as an option. The extra server I have is not compatible with SME 8 so I'm going to think about that one for a bit.

Thanks for the help with this issue.