Topic: Bulk mail delete ?

[david000]

Hi,

I'm looking at implementing a mail retention policy where we keep mail for x years.  I've fully decided but as we're in the UK 7 years might be reasonable.

At the moment we have mail going back to 2013 so I need to a way to delete anything on the system prior a given date. Manually doing that in round cube looks to be a bit of a PITA.

We use webmail \ roundcube to access mail on our local SME Server.

Any suggestions on how to best do this ?

Also, if there are UK users I'd be interested in your take on retention policy.

[ReetP]

Hi,

I'm looking at implementing a mail retention policy where we keep mail for x years.  I've fully decided but as we're in the UK 7 years might be reasonable.

At the moment we have mail going back to 2013 so I need to a way to delete anything on the system prior a given date. Manually doing that in round cube looks to be a bit of a PITA.

Any suggestions on how to best do this ?

Think these are two of the more well known apps for managing imap accounts - sure there are others:

https://imapsync.lamiral.info/
https://github.com/lefcha/imapfilter


You could also possibly do a fancy bash script to find and delete files over X age.

eg

https://unix.stackexchange.com/questions/194863/delete-files-older-than-x-days

Also, if there are UK users I'd be interested in your take on retention policy.

I don't delete anything. Storage is cheap.....

If I did I'd cold archive anything over say 10. You could use imapsync to a safe space and then back that up to a secure medium.

Others may have other thoughts!

[david000]

Think these are two of the more well known apps for managing imap accounts - sure there are others:

https://imapsync.lamiral.info/
https://github.com/lefcha/imapfilter


You could also possibly do a fancy bash script to find and delete files over X age.

eg

https://unix.stackexchange.com/questions/194863/delete-files-older-than-x-days


I don't delete anything. Storage is cheap.....

If I did I'd cold archive anything over say 10. You could use imapsync to a safe space and then back that up to a secure medium.

Others may have other thoughts!

Thanks, imapsync might be a plan, especially if I can rig that on a windows machine.

[ReetP]

Thanks, imapsync might be a plan, especially if I can rig that on a windows machine.

Sounds like a plan!

[mmccarn]

I don't delete anything. Storage is cheap.....

If you don't delete the old data in the EU you can be required to produce it in litigation -- or simply by a disgruntled customer asking your country's data authority to demand it of you.
 (Storage may be cheap, but GDPR penalties or lawsuits probably aren't...)

* imapsync
My most recent (albeit still at least 7 years old) notes on installing imapsync onto a SME server

Code: [Select]

  #IMAPSYNC v1.5.x
  cd /var/git/imapsync & git pull
  # https://github.com/imapsync/imapsync
  # GIT REPO https://github.com/imapsync/imapsync.git(presumably I had done mkdir -p /var/git; cd /var/git; git pull https://github.com/imapsync/imapsync.git

* fancy bash script
I created a script many years ago to move suspicious emails within each user's Maildir: rbl-recheck.sh

From that I've created this quick command to move emails older than a specified date  for each user into a new "Archived" folder, so you could see what gets moved.

* ARCHIVETO must start with "." and should not have spaces
* ARCHIVEDATE can be any valid argument to "date -d" -- for example "5 years ago"
* Change "DEBUG=echo" to "DEBUG=" to actually run the commands instead of listing them

It should be reasonably simple to modify this to move the archived emails to a totally different mailbox, if that's what you wanted to do...

Code: [Select]

ARCHIVETO=".Archived"; \
ARCHIVEDATE="1/1/2023"; \
DEBUG=echo; \
find /home/e-smith/files/users \
-type f \
-not -path $ARCHIVETO \
-daystart \
\( -path */cur/* -or -path */new/* -or -path */tmp/* \) \
-not -newermt $(date +%Y-%m-%d -d "$ARCHIVEDATE") \
-print0 |\
while read -d $'\0' MAILFILE; do \
if [[ $MAILFILE == */Maildir/cur/* ]]; then \
  if [ ! -d "${MAILFILE/Maildir?cur*/Maildir/$ARCHIVETO/cur}" ]; then  \
    $DEBUG mkdir -p "${MAILFILE/Maildir?cur*/Maildir/$ARCHIVETO/{cur,new,tmp}}" ; \
    $DEBUG chown -R $(stat -c \"%U:%G\" $MAILFILE) "${MAILFILE/Maildir?cur*/Maildir/$ARCHIVETO}" ;\
  fi ; \
  $DEBUG mv "$MAILFILE" "${MAILFILE/Maildir?cur/Maildir/$ARCHIVETO/cur}"; \
fi ; \
done
* Notes & Vague Memories
- Email folder names must start with a dot to be recognized - eg ".Archived"
- Email folders must contain all three of /cur/, /new/, and /tmp/ to be recognized
- All of the folders must be "owned" by the user accessing the mailbox

I had another script on a Kerio Connect server that scanned the email folders and output email stats - date, from, to, subject, etc -- if that sounds useful let me know.

[ReetP]

If you don't delete the old data in the EU you can be required to produce it in litigation -- or simply by a disgruntled customer asking your country's data authority to demand it of you.
 (Storage may be cheap, but GDPR penalties or lawsuits probably aren't...)

Yup, but regrettably we often have to refer back to stuff more than 6 years. (it's a bit obscure as to why, but it happens - and frequently users don't write things down.... paperless offices and all that jazz)

Yes, we may have to produce it for GDPR - we're UK/EU, but I have no qualms about anything we hold and the penalties are really for when you can't, don't, or won't supply it, rather than when you can.

So holding it isn't really an issue for us.

From that I've created this quick command to move emails older than a specified date  for each user into a new "Archived" folder, so you could see what gets moved.

That really will be handy as I move stuff to archived folders by year split with Sent and Received - mainly to keep the folder sizes down for Thunderbird.

Be nice to script that!