Koozali.org: home of the SME Server

Contribs.org Forums => General Discussion => Topic started by: Drifting on June 01, 2018, 01:17:28 PM

Title: Lost email from sent items?
Post by: Drifting on June 01, 2018, 01:17:28 PM: Confused of the planet here.

I have a small charity I look after, and they reported that their sent items have been vanishing in Outlook! Not all, just some? Is there anyway I can check or search for the lost emails, without resorting to wading through logs. I have my reasons and not lazy, just borderline eyesight I am afraid.

Any suggestions greatly appreciated.

Paul.
Title: Re: Lost email from sent items?
Post by: ReetP on June 01, 2018, 01:36:48 PM: You need to tell us more info.

Server version

What version if Outlook?

How does Outlook handle mail? POP or IMAP?
Title: Re: Lost email from sent items?
Post by: Drifting on June 01, 2018, 01:43:45 PM: Quote from: ReetP on June 01, 2018, 01:36:48 PM
You need to tell us more info.

Server version

What version if Outlook?

How does Outlook handle mail? POP or IMAP?

Sorry, that was more of a general question.

It is the latest SME9.2 fully updated.
Outlook 2010
Using IMAP.

So far two people have complained that an email sent does not appear in the sent items. However the recipients received them fine. It was random, two this morning for one person, and one yesterday for another. Cannot get on the workstations as yet, as I would usually blame Outlook and a massive PST file. Just wondering if there was anyway to search for an email on the server users mailbox to see if it was there.

Thanks for the quick response appreciated.
Paul
Title: Re: Lost email from sent items?
Post by: Stefano on June 01, 2018, 01:55:05 PM: Outlook and IMAP is a no sense
Use Thunderbird or, if you can, install sogo and use the webmail
Title: Re: Lost email from sent items?
Post by: ReetP on June 01, 2018, 01:57:01 PM: Ok.

I'd agree with Stefano, but I know lots have to stick with the steaming pile of sh1t that is Outlook :-)

You can check their real Sent items by logging in with webmail and have a look around (I often have to do this with my users when they lose stuff, usually by dragging accidentally to a different folder)

You can also ssh in and take a look in the users mail directories.

They are here:

/home/e-smith/files/users

Question... I seem to remember that Outlook with IMAP doesn't automatically save to the server Sent folder. You need a rule to copy them across with older versions, and some settings for newer ones. Have you done that?

eg: https://www.msoutlook.info/question/486

If they aren't on the server then they are stored in the pst file (I guess) and they have either accidentally moved them some how, or the pst file could be corrupt ?
Title: Re: Lost email from sent items?
Post by: Jean-Philippe Pialasse on June 01, 2018, 02:53:19 PM: check the settings for sent folder and other model, draft, junk....

Depending on the mail client they do not use the same as default, Thunderbird vs Horde is a good example, so you might see your sent mail in 2 different folder if you do not set yourself .

ALso if your user clicked on cancel because it was too long, the mail might have been sent but not yet copied to imap folder, it can be long if you have a lot of user searching in their IMAP folders at the same time and you reach the maximum connection rate.
Title: Re: Lost email from sent items?
Post by: Drifting on June 01, 2018, 03:11:44 PM: Hi Guys, thanks for the input.
And I feel much the same as you about Outlook, but the problem is that everyone knows it.

Yes, I was aware on the IMAP folders and added the sent items on the server as the default in Outlook. Already did a comparison on the webmail and Outlook. They are identical. Which is why I was asking the sages on here if they had any idea why it would just vanish? Yet still be sent!

Did a compaction of one PST file, just to make sure, and that is fine, it's also quite small compared to some at only 6MB.

I do not know the mechanics of how the mail gets copied to the sent items. So the suggestion on the mailbox might be a cunning plan. Now to brush up on my searching skills within Linux :-)

Thanks again for all your help

Paul.
Title: Re: Lost email from sent items?
Post by: Stefano on June 01, 2018, 03:38:41 PM: some strange rule created by the users?
Title: Re: Lost email from sent items?
Post by: Jean-Philippe Pialasse on June 01, 2018, 04:10:20 PM: I more and more use symlink on the server, started with spam folders as some clients like ios Mail do not allow to set the folder. When you know the alternatives in your languages and with your mail clients you can do a symlink in all users Maildir so you do not have to handle the manual setting for all users.

Stefano has a good point with rules created by users.

Yes mails first get send using SMTP protocol, then client use IMAP protocol to push a copy on the sent folder.
Title: Re: Lost email from sent items?
Post by: mmccarn on June 02, 2018, 04:07:08 PM: You can search for recent Sent Items on the server using something like this:
Code: [Select]
cd /home/e-smith/files/users find . -ctime -1 |grep -i sent* -ctime -1: find all files created in the last 1 day
* grep -i sent: include in the output anything containing "sent", ignoring case (-i)

I've seen different email clients use "Sent", "Sent Items" and "Sent Messages". If I change "-ctime -1" to "-ctime -10", I see that I have all three of these on my account:
Quote from: find . -ctime -10 |grep -i sent
./support/Maildir/.Sent Items ./support/Maildir/.Sent Items/maildirfolder ./support/Maildir/.Sent Items/tmp ./support/Maildir/.Sent Items/new ./support/Maildir/.Sent Items/dovecot-uidlist ./support/Maildir/.Sent Items/dovecot.index.log ./support/Maildir/.Sent Items/dovecot.index.cache ./support/Maildir/.Sent Items/cur ./support/Maildir/.Sent Items/cur/1527351935.M478949P8602.office,S=7396,W=7528:2,S ./support/Maildir/.Sent Items/cur/1527350066.M257951P5934.office,S=1223,W=1265:2,S ./support/Maildir/.Sent Items/cur/1527350029.M591954P5878.office,S=699,W=724:2,S ./support/Maildir/.Sent Items/cur/1527601499.M711966P8065.office,S=7090,W=7247:2,S ./mmccarn/Maildir/.Sent Items ./mmccarn/Maildir/.Sent Items/tmp ./mmccarn/Maildir/.Sent Items/dovecot-uidlist ./mmccarn/Maildir/.Sent Items/dovecot.index.log ./mmccarn/Maildir/.Sent Items/dovecot.index.cache ./mmccarn/Maildir/.Sent Items/cur ./mmccarn/Maildir/.Sent Items/cur/1527707147.M277894P22460.office,S=39516,W=40289:2,S ./mmccarn/Maildir/.Sent Items/cur/1527352381.M370254P8981.office,S=8841,W=8993:2,S ./mmccarn/Maildir/.Sent Items/cur/1527945032.M147900P7953.office,S=1253,W=1289:2,S ./mmccarn/Maildir/.Sent Items/cur/1527350162.M171903P6617.office,S=1628,W=1679:2,S ./mmccarn/Maildir/.Sent Items/cur/1527854968.M733872P11508.office,S=718,W=743:2,S ./mmccarn/Maildir/.sent-mail/dovecot.index.cache ./mmccarn/Maildir/.Sent Messages/dovecot.index.cache ./rmccarn/Maildir/.Sent ./rmccarn/Maildir/.Sent/dovecot.index.cache ./rmccarn/Maildir/.Sent Messages ./rmccarn/Maildir/.Sent Messages/dovecot-uidlist ./rmccarn/Maildir/.Sent Messages/dovecot.index.log ./rmccarn/Maildir/.Sent Messages/dovecot.index.cache

There is also a setting in Outlook for "save replies in same folder as message", which can scatter the sent items throughout a user's email folders.

Here's a command that should find "sent" items in any folder for a specific user
Code: [Select]
USER=mmccarn cd /home/e-smith/files/users/$USER find . -type f -ctime -1 -exec grep -H ^From\:\ .*$USER "{}" \;* USER=mmccarn: run this command for the "mmccarn" user account
* -type f: include only files in the output
* -ctime -1: include only items created in the last 1 day
* -exec: run the specified command, replacing "{}" with the result from "find"
* grep -H: include the filename in the output from grep
* ^From\:\ .*$USER: find lines that begin with "From: ", and also include the username specified

Quote from: USER=mmccarn
cd /home/e-smith/files/users/$USER
find . -type f -ctime -10 -exec grep -H ^From\:\ .*$USER "{}" \;
./Maildir/.Sent Items/cur/1527707147.M277894P22460.office,S=39516,W=40289:2,S:From: "Michael J McCarn" <mmccarn@mmsionline.us> ./Maildir/.Sent Items/cur/1527352381.M370254P8981.office,S=8841,W=8993:2,S:From: "Michael J McCarn" <mmccarn@mmsionline.us> ./Maildir/.Sent Items/cur/1527945032.M147900P7953.office,S=1253,W=1289:2,S:From: "Michael J McCarn" <mmccarn@mmsionline.us> ./Maildir/.Sent Items/cur/1527350162.M171903P6617.office,S=1628,W=1679:2,S:From: "Michael J McCarn" <mmccarn@mmsionline.us> ./Maildir/.Sent Items/cur/1527854968.M733872P11508.office,S=718,W=743:2,S:From: "Michael J McCarn" <mmccarn@mmsionline.us> ./Maildir/cur/1527855192.M632509P11827.office,S=8686,W=8837:2,:From: M McC <mmccarn@mmsionline.us>

The last one looks odd, but I can review it using "less", and surrounding the message name with single quotes (it's an email test that matches on "^From: .*mmccarn" in the body of the email)
Code: [Select]
less './Maildir/cur/1527855192.M632509P11827.office,S=8686,W=8837:2,'
Title: Re: Lost email from sent items?
Post by: janet on June 02, 2018, 10:27:29 PM: Drifting

This comment by JPP is probably meaningful:

"the mail might have been sent but not yet copied to imap folder, it can be long if you have a lot of user searching in their IMAP folders at the same time and you reach the maximum connection rate."

I would suggest you look at slowness of connection or lack of connection (etc) to the sme mail server while doing the IMAP copy.
Title: Re: Lost email from sent items?
Post by: Drifting on June 06, 2018, 01:52:53 PM: MMCARN

What a fantastic response, learnt a lot from that, thank you !

And everyone else who helped, greatly appreciated. I think the root cause is outlook and the size of the pst file, that an the inordinate amount of email on the server. Basically told them to clean up their mailboxes, and reduce the Outlook PST files below 600MB which it seems was the recommended maximum anyway. One was over 6GB!

Thanks guys you save my bacon as always.

Paul.
Title: Re: Lost email from sent items?
Post by: JohnG on June 07, 2018, 01:42:26 AM: Yep, 6GB is considered huge by microsoft but I've seen ones 25G+... and that usually results in crazy random corruption issues.

Personally I would start fresh and make the current pst an archive and create a new default pst.

As for the missing sent items, I'm thinking it's simply different clients using different sent folders as was mentioned earlier.