Topic: Simple Mail Archiving with Thunderbird

[david000]

To follow on from another thread I'm trying to use Thunderbird's 'save as' function from a local folder to archive our legacy mail.

After a bit of testing it all seemed to work ok, I ended up with lots of .eml files on my NAS which could be zipped up and dropped to a cold USB drive.

However when I moved onto the main folder with 50k of emails in it, Thunderbird is randomly stopping the 'Save As'. The difficulty being that the order in which Thunderbird is exporting the selected files is unclear so I can't tell what's been 'Saved As' and what hasn't.

I appreciate this isn't a Thunderbird Support place but wondered if anyone has heard of this or can suggest a solution.

My next stop is the Thunderbird Archive function but I'm unclear how the resulting file will be accessible x years down the line.

Thanks in advance...

[bunkobugsy]

https://wiki.koozali.org/User_homes_admin_access can be used to directly access user homes,
then folder by folder copy/move out emails (sorted by date) from Maildir folders and rename files to *.eml.

Don't copy anything to Maildir folders without setting ownership to user:user, otherwise users won't have access to them.

I use this custom-template: /etc/e-smith/templates-custom/etc/smb.conf/55adminhomes

[admin_homes]
comment = home_directories
path = /home/e-smith/files/users/
valid users = admin root
admin users = admin root
write list = admin root
public = no
browsable = no
writable = yes

then:
expand-template /etc/smb.conf

systemctl restart smb

now go to  \\serverIP\admin_homes  (with the two-panel file manager of your choice)

[david000]

https://wiki.koozali.org/User_homes_admin_access can be used to directly access user homes,
then folder by folder copy/move out emails (sorted by date) from Maildir folders and rename files to *.eml.

Don't copy anything to Maildir folders without setting ownership to user:user, otherwise users won't have access to them.

I use this custom-template: /etc/e-smith/templates-custom/etc/smb.conf/55adminhomes

[admin_homes]
comment = home_directories
path = /home/e-smith/files/users/
valid users = admin root
admin users = admin root
write list = admin root
public = no
browsable = no
writable = yes

then:
expand-template /etc/smb.conf

systemctl restart smb

now go to  \\serverIP\admin_homes  (with the two-panel file manager of your choice)

Thank you for this - just the job. I need to brush up (!) on my linux skills but this will be a quick job than Thunderbird.

[david000]

Thank you for this - just the job. I need to brush up (!) on my linux skills but this will be a quick job than Thunderbird.

It looks like the reason Thunderbird is failing is because of 'special characters' in (at least) one of the emails. So far that appears to be from one of our German suppliers although so far just the one.

[ReetP]

Another point with TB and large IMAP mailboxes is it starts to get slow and messy over 10,000 or so emails.

I usually go through folder every 6 months or so and manually archive them.

If you use the TB 'archive' function it drops them all in one folder named by the year, and it doesn't separate Inbox/Sent hence I do it manually.

There are various scripts around that can archive mails for you in a cron job etc - they are essentially just text files so easy to move.

Again as suggested, just watch ownership.

[david000]

Another point with TB and large IMAP mailboxes is it starts to get slow and messy over 10,000 or so emails.

If you use the TB 'archive' function it drops them all in one folder named by the year, and it doesn't separate Inbox/Sent hence I do it manually.

I can confirm the 10k thing, and sometimes less !

I did look at TB 'archive' but it wasn't clear (to me) how easy it would be to access the result ?  Presumably through TB ?   I'm hoping that the individual email text files I've saved will open in whatever I have to hand in the future.

 

[ReetP]

FWIW after I looked for something to do what I wanted and couldn't find anything that worked for me, I have been working on a mail archive script that can run for multiple users and move files to Archive directories a la Thunderbird.

It is based on this:

https://github.com/netconstructor/Imap4-Archiver.git

It now moves Inbox and Sent to:

Archive/$year/Inbox
Archive/$year/Sent

It works against any server using the imap login - not just a local ones. (It needs Dovecot extras and the admin permission setting to handle all accounts)

Yes, it could probably have been done in bash to just copy files which would have been faster, but I liked slow and steady

I will work for a given set of years, or a more simple 'everything' before x days.

I still have some work to do but will push it to the koozali repo in due course.

Potentially it could be integrated to the smanager, and hopefully will be able to run via a cron as well.

I'll post back when I have more news.

[Stefano]

take a look at this

https://github.com/s1t5/mail-archiver

[Stefano]

and this one
https://openarchiver.com/

[ReetP]

Thanks Stefano.

I really wanted a simple thing I could run here, do it on multiple servers, and possibly on a cron job.

There are some complex pieces out there but they are way to much for what I wanted - really just move to an archive folder. They don't need indexing or anything fancy.

They probably started like I did and kept building

[Stefano]

Hi mate

IIRC you're on proxmox.

both solutions are available here https://community-scripts.github.io/ProxmoxVE/scripts?id=dashy

so the deploy and config effort is low

HTH

[ReetP]

IIRC you're on proxmox.

I am, though these still don't really do what I want.

I just want mails shifted into the Archive folders on the same account, not shifted to another store somewhere else (though that may be an option at another time). I have them backed up regardless so it is just duplicating effort.

I have tried both apps as an experiment but they are very messy when you start combining multiple accounts.

Open archiver is more basic - it really does look like a backup system without much other functionality.

Mail archiver has more options but does not do what I want. Fine for simple backing up, restoring, delete over X days etc. It would be my preferred choice for general backups.

For me it would be OK for say cleaning up admin accounts or similar, but with lots of user accounts it would get messy really fast. It also doesn't sort to new folders, which is what I really want.

My wife has 176Gb mails to manage, and doesn't necessarily want them all removed. But needs older mails shifting out of the default Inbox/Sent some other folders which are still searchable on her Thunderbird - she isn't going to go search through some other app.

I can then remove/move Archive folders over X years easily.

So the apps are good if they do what you want, but aren't what I need