Topic: Disappearing IMAP email /redux/

[lproven]

I'm having problems with one client machine. The user's email is stored on their SME Server, a Dell PowerEdge 800 running 6.0.1. She's using IMAP via Mozilla Thunderbird. The problem appeared originally on Thunderbird 1.0 but it's still there on Thunderbird 1.03.

It's happened twice now, and is pretty much as described here, back in 2001:

http://forums.contribs.org/index.php?topic=2372.msg7927#msg7927

All of the messages in her INBOX folder have disappeared except for 18 received in the last couple of days. Messages in subfolders are unaffected. The view of the folder is exactly the same in Outlook 2000 SP3 in "Internet Mail Only" mode.

Last time this happened, I was able to retrieve all the messages into another client and transfer them over with POP3, but that led to lots of duplicates and was a pain to do. Is there an easier way?

P.S. Rebooting the server makes no difference.

[jackl]

Iproven,
 Have you checked the ownership of the emails from a console they should show as usersname/usersname. I have seen this before if for example ownership is changed to root/root while the emails are still there the user will not see them with an imap mail client. Change them back  with chown to correct ownership and voila messages will reappear.
Hope this might help

Regards
jack

[CharlieBrady]

I'm having problems with one client machine. The user's email is stored on their SME Server, a Dell PowerEdge 800 running 6.0.1. She's using IMAP via Mozilla Thunderbird. The problem appeared originally on Thunderbird 1.0 but it's still there on Thunderbird 1.03.

It's happened twice now, and is pretty much as described here, back in 2001:

http://forums.contribs.org/index.php?topic=2372.msg7927#msg7927

I very much doubt that what you are seeing now has anything to do with the issue you reference from 2001. That was a very short-lived problem (if you applied the update) and the IMAP daemon in use with 6.0.1 is completely different.

Please describe as carefully as you can what the problem is you are now seeing. Maybe then someone can help you.

[lproven]

Iproven,
 Have you checked the ownership of the emails from a console they should show as usersname/usersname.

That's "L" for "Liam" :¬)

Pardon my ignorance, but where would I go to do that? I didn't think that messages in IMAP folders would show up in the filesystem anyhwhere.

The users don't /ever/ use the Unix CLI on the machine. I don't think they realise it has one!

[lproven]

I very much doubt that what you are seeing now has anything to do with the issue you reference from 2001. That was a very short-lived problem (if you applied the update) and the IMAP daemon in use with 6.0.1 is completely different.

Please describe as carefully as you can what the problem is you are now seeing. Maybe then someone can help you.

Old problem: I'm aware of that, but I found that old message when searching contribs.org and it matches the problem I'm having very well.

IMAP daemon different: Oh? I wasn't aware of that.

The description in that post stands.

Twice now, the user has come in and logged into their machine in the morning to find that most of their mail messages have disappeared. The server hasn't rebooted; rebooting the client, quitting & restarting the mail client, even using a different email client, makes no difference.

Last time, re-downloading the messages with POP3 helped, but it took some hours to refile all the multiple thousands of mails across about 50 folders. I am keen to avoid this if possible but I may have to abandon IMAP at this rate.

Anything else I can tell you?

[jackl]

Hi Liam

Sorry about getting your name wrong, but here is where you would find a users Mailbox

/home/e-smith/files/users/username/Maildir

The directory cur is normally the inbox and you should also be able to see any folders the user has created.
A good way to view these is to use midnight commander by typing mc at CLI, navigate to the users mail folder and check ownership. This may not be your problem but I think it is worth a look.

Regards

Jackl

[CharlieBrady]

Old problem: I'm aware of that, but I found that old message when searching contribs.org and it matches the problem I'm having very well.

Sure, but the cause must be different.

The description in that post stands.

OK, but it doesn't provide enough information to replicate the problem, or to diagnose it.

All mail messages are stored as files, one message per file. Messages are either in foldername/new or foldername/cur, depending on whether a client has queried the folder or not.
Messages aren't deleted unless a client asks for them to be deleted (as a POP client does after it's fetched them, for instance). Message files aren't deleted via IMAP until a client both deletes the message and then asks for the folder to be purged of deleted messages.

If the messages can be refound, then they weren't deleted. You need to work out why the client wasn't showing them to the user.

Did you ever try webmail, to see if the webmail client could see the messages?

[lproven]

Jackl:

Thanks for that hint! I didn't realise SMES used Maildir; I thought it was in mailbox or something similar. I'll go look immediately.

Charlie Brady:

I'm not sure what more I can say. It happens consistently - twice now. The user uses a notebook to attach. but she's working on the office Ethernet LAN. The client is Thunderbird 1.0, now upgraded to 1.0.2, but the messages are still invisible with Outlook 2000 SP3 in Interet Email Only mode, and Outlook Express 6. I tried Eudora 6.2 but it won't connect at all. Logging in from a desktop machine running W2K SP4 and Outlook Express 6 showed no difference. All machines are running the latest Norton Antivirus.

The server is SME 6.01 on a 2.8GHz Dell PowerEdge SL800 (IIRC) with 512M RAM. Disks are hardware-mirrored 120GB PATA on a Dell CERC controller, in one big partition.

Clients are on a 100Mb/s Ethernet network, running through a hub, not a switch. There are 3 client desktops & 1 laptop; occasionally a 2nd or 3rd laptop connectes. SMES is doing DHCP. Server is in private-only mode, firewalled on a 5765Kb/s ADSL connection via a standalone P133 running Smoothwall 2.0 with all the latest patches, controlling an Alcatel Speedtouch USB modem.

The user swears blind that she just comes in to work one morning and finds that all her old mail messages have disappeared with no intervention from her, and that this has happened twice now, some months apart. The remaining messages go back about 3 days; stuff in subfolders is unaffected. Nothing appears in the trash.

I'll go look at this again this afternoon with the hint from Jackl.

I work in support myself; I know what it's like to receive inadequate information, but in this case, I don't know what more to say. Tell me what you need to know!

[mdo]

Could it be that a pop3 mail client has connected at some stage and retrieved all her mail from the inbox (all other folders are not affected). Maybe you change that user's password in her standard mail client?

Michael

[Morgo]

Hey all,

I have just started having exactly the same issue as Liam. Checked Cur for messages as post suggests and all have dissapeared exect for 2. Changed mail clients as Liam did and problem happened couple of days latter. Anyone please !

Paul

[CharlieBrady]

I have just started having exactly the same issue as Liam. Checked Cur for messages as post suggests and all have dissapeared exect for 2. Changed mail clients as Liam did and problem happened couple of days latter.

If messages disappear from ~/Maildir/cur it's because a client program has asked for them to be deleted. Typically a pop client will delete messages after downloading them. The messages themselves will then be on local storage wherever that client program is.

[Morgo]

Thanks Charlie,

There is no PoP client checking messages. I have switched from outlook to thunderbird now webmail and it is still happening. I have changed the password for the user in case someone is checking it remotely without my knowledge.

Paul

[lproven]

OK. I've checked; there are no other messages than the ones in /cur in the user's maildir. She absolutely assures me that she has not used any other mail client, and I believe her, because she would not know *how* to configure another computer or program for POP3 access. The machine also has Outlook 2000 SP3 and I've added Outlook Express back in, but neither is configured for POP access.

In other words, there is something going wrong here. Something has caused the SMES box to delete all old messages before a certain date from the user's IMAP root folder and they appear to be genuinely gone. And now it's long enough ago that the backups have been overwritten. :¬(

[lproven]

Update: I've fixed webmail on the server. It did have this problem:

http://forums.contribs.org/index.php?topic=20210.0

Using Webmail the messages are missing as well.

Also, I tried retrieving them using POP3. None of the deleted messages were recovered. So, in other words, she's lost hundreds of archived emails.

[CharlieBrady]

In other words, there is something going wrong here. Something has caused the SMES box to delete all old messages before a certain date from the user's IMAP root folder and they appear to be genuinely gone.

I'll stick with what I said earlier - the only things which will delete mail messages are the imap and pop daemons, and they only do it when requested to by a mail client.