Topic: Delete User/Mailbox - Hard-disk space not correct?

[dbuechel]

Hey,

I have a big problem with our Hard-disk space when I delete a user account.

Example:
Our Hard-disk space have when 46G - Used 9,4G

I create a user and over fetchmail start the connect to a pop3 account and download over 23500 emails

Then I have 25G used hard-disk space - all ok.

But now, when I delete the user over the server-manager, it delete the mail account on the hard-disk but my Hard-disk space have used 17GB

This is a different from 7.6G from start the user account!

What will not be deleted?
What can I do to clean up the hard drive?

[Stefano]

Do you have alwaysbcc enabled? Did you forward emails to other accounts?

[dbuechel]

No - there is only one user account maillog and the admin account.

[Stefano]

Well.. Check users' quota, maillog make me think you have alwaysbcc enabled

[dbuechel]

Really - I've tested the now twice with a fresh installation.
I create only the user maillog, and run fetchmail.

There is only ONE user folder:
/home/e-smith/files/users/  => admin

And I don't use this:
=> Keep a copy of all emails

config setprop qpsmtpd Bcc enabled
signal-event email-update

[brianr]

In my experience deleting files in Linux does not always free the space instantly, I guess the garbage collection is run as a background task.  I'd have expected it to complete with an hour or so though.

I think we'd need to see the evidence ("df" and "du"  before and after)?

[Stefano]

please, post the output of

Code: [Select]

config show qpsmtpd

thank you

[dbuechel]

qpsmtpd=service
    Bcc=disabled
    BccMode=cc
    BccUser=maillog
    DNSBL=disabled
    LogLevel=6
    MaxScannerSize=25000000
    RBLList=bl.spamcop.net:dnsbl-1.uceprotect.net:dnsbl-2.uceprotect.net:psbl.surriel.com:zen.spamhaus.org
    RHSBL=disabled
    RelayRequiresAuth=disabled
    SBLList=multi.surbl.org:black.uribl.com:rhsbl.sorbs.net
    TlsBeforeAuth=1
    access=public
    qplogsumm=disabled
    status=enabled

[Stefano]

ok

Code: [Select]

cd /home/e-smith/files/users
du -s -m * | sort -nr


[dbuechel]

Only:

1   admin

see picture

[Stefano]

ok..

Code: [Select]

cd /
du -s -m * | sort -nr | head


[dbuechel]

[root@willmsmail /]# du -s -m * | sort -nr | head
du: Zugriff auf „proc/3157/task/3157/fd/4“ nicht möglich: Datei oder Verzeichnis nicht gefunden
du: Zugriff auf „proc/3157/task/3157/fdinfo/4“ nicht möglich: Datei oder Verzeichnis nicht gefunden
du: Zugriff auf „proc/3157/fd/4“ nicht möglich: Datei oder Verzeichnis nicht gefunden
du: Zugriff auf „proc/3157/fdinfo/4“ nicht möglich: Datei oder Verzeichnis nicht gefunden
14845   var
907     usr
322     lib
77      home
71      boot
37      etc
20      sbin
8       bin
3       dev
2       tmp
[root@willmsmail /]#

in english:

you: could not access "proc/3157/task/3157/fd/4": file or directory not found
you: could not access "proc/3157/task/3157/fdinfo/4": file or directory not found
you: could not access on "proc/3157/fd/4": file or directory not found
you: could not access on "proc/3157/fdinfo/4": file or directory not found

[Stefano]

well.. you have 14845 mb (i.e. about 14 Gb) in /var, which is strange IMHO

let's investigate

Code: [Select]

cd /var
du -s -m * | sort -nr | head

if necessary, repeat this step

[dbuechel]

[root@willmsmail /]# cd /var
[root@willmsmail var]# du -s -m * | sort -nr | head
14397   spool
175     log
119     clamav
87      lib
63      cache
3       service
3       qmail
2       www
1       yp
1       tmp
[root@willmsmail var]# cd /spool
-bash: cd: /spool: Datei oder Verzeichnis nicht gefunden

[dbuechel]

[root@willmsmail spool]# du -s -m * | sort -nr | head
14397   qpsmtpd
1       vbox
1       squid
1       spamd
1       samba
1       plymouth
1       mail
1       lpd
1       cron
1       clamav

In this folder:

14397   qpsmtpd

all email´s inside!

Can I delete all files?

[Stefano]

well, you should not have all that files there..
on a quite used mail server of mine I read:

Code: [Select]

[root@fileserver spool]$ du -s -m * | sort -nr
35      spamd
2       qpsmtpd
1       vbox
1       squid
1       samba
1       pnp4nagios
1       plymouth
1       nagios
1       mail
1       lpd
1       exim
1       cron
1       clamav
1       at
1       anacron

something is wrong with your server..

[dbuechel]

something is wrong with your server..

I install only SME-Server then all updates and then fetchmail - nothing more.

How can that happen?

[Stefano]

Code: [Select]

cd /var/spool/qpsmtpd
ls

what do you see?

[dbuechel]

msg-1464695222-2667-9582   msg-1464770443-3672-6146   tnef-2667-161.doc
msg-1464695222-2667-9583   msg-1464770443-3672-6147   tnef-2667-162.doc
msg-1464695222-2667-9584   msg-1464770443-3672-6148   tnef-2667-163.doc
msg-1464695222-2667-9585   msg-1464770443-3672-6149   tnef-2667-164.doc
msg-1464695222-2667-9586   msg-1464770444-3672-6150   tnef-2667-165.doc
msg-1464695225-2667-9587   msg-1464770444-3672-6151   tnef-2667-166.doc
msg-1464695229-2667-9588   msg-1464770444-3672-6152   tnef-2667-167.doc
msg-1464695231-2667-9589   msg-1464770444-3672-6153   tnef-2667-168.doc
msg-1464695236-2667-9590   msg-1464770444-3672-6154   tnef-2667-169.doc
msg-1464695236-2667-9591   msg-1464770444-3672-6155   tnef-2667-170.doc
msg-1464695236-2667-9592   msg-1464770445-3672-6156   tnef-2667-171.doc
msg-1464695236-2667-9593   msg-1464770445-3672-6157   tnef-2667-172.doc
msg-1464695237-2667-9594   msg-1464770445-3672-6158   tnef-2667-173.doc
msg-1464695237-2667-9595   msg-1464770445-3672-6159   tnef-2667-174.doc
msg-1464695238-2667-9596   msg-1464770445-3672-6160   tnef-2667-175.doc
msg-1464695238-2667-9597   msg-1464770446-3672-6161   tnef-2667-176.doc
msg-1464695238-2667-9598   msg-1464770446-3672-6162   tnef-2667-177.doc
msg-1464695241-2667-9599   msg-1464770446-3672-6163   tnef-2667-178.doc
msg-1464695245-2667-9600   msg-1464770446-3672-6164   tnef-2667-179.doc
msg-1464695245-2667-9601   msg-1464770446-3672-6165   tnef-2667-180.doc
msg-1464695245-2667-9602   msg-1464770446-3672-6166   tnef-2667-181.doc
msg-1464695246-2667-9603   msg-1464770446-3672-6167   tnef-2667-182.doc
msg-1464695246-2667-9604   msg-1464770446-3672-6168   tnef-2667-183.doc
msg-1464695246-2667-9605   msg-1464770447-3672-6169   tnef-2667-184.doc
msg-1464695247-2667-9606   msg-1464770447-3672-6170   tnef-2667-185.doc

and many many more !

[Stefano]

they should have

Code: [Select]

-rw-r--r--  1 qpsmtpd clamav

perms and ownership..

I think you can safely delete them but if I were in your shoes, I'd like to know why are there so many files there..
wait for some other guys to comment

[dbuechel]

I wait

[dbuechel]

Question:

Is it a time problem?

We have email´s with 20MB and more. With many files inside!

[dbuechel]

About

Code: [Select]

-rw-r--r--  1 qpsmtpd clamav
-bash:-rw-r--r--: command not found.

^^

[Stefano]

try

Code: [Select]

ls -la

when you are in /var/spool/qpsmtpd

NOTE: be aware that if you don't know what you're doing, playing with CLI could be very dangerous.. I didn't tell you to try that "command".. if you're not familiar with CLI, before doing anything, ask for clarification

[dbuechel]

Code: [Select]

-rw-r--r--     1 qpsmtpd clamav    3299  1. Jun 09:48 tnef-3672-73.doc
-rw-r--r--     1 qpsmtpd clamav     400  1. Jun 09:48 tnef-3672-74.doc
-rw-r--r--     1 qpsmtpd clamav       4  1. Jun 09:52 tnef-3672-75.doc
-rw-r--r--     1 qpsmtpd clamav       8  1. Jun 09:52 tnef-3672-76.doc
-rw-r--r--     1 qpsmtpd clamav      33  1. Jun 09:52 tnef-3672-77.doc
-rw-r--r--     1 qpsmtpd clamav       2  1. Jun 09:52 tnef-3672-78.doc
-rw-r--r--     1 qpsmtpd clamav      14  1. Jun 09:52 tnef-3672-79.doc
-rw-r--r--     1 qpsmtpd clamav      14  1. Jun 09:52 tnef-3672-80.doc
-rw-r--r--     1 qpsmtpd clamav      24  1. Jun 09:52 tnef-3672-81.doc
-rw-r--r--     1 qpsmtpd clamav      22  1. Jun 09:52 tnef-3672-82.doc
-rw-r--r--     1 qpsmtpd clamav       4  1. Jun 09:52 tnef-3672-83.doc
-rw-r--r--     1 qpsmtpd clamav       2  1. Jun 09:52 tnef-3672-84.doc
-rw-r--r--     1 qpsmtpd clamav   24952  1. Jun 09:52 tnef-3672-85.doc
-rw-r--r--     1 qpsmtpd clamav      14  1. Jun 09:52 tnef-3672-86.doc
-rw-r--r--     1 qpsmtpd clamav      13  1. Jun 09:52 tnef-3672-87.doc
-rw-r--r--     1 qpsmtpd clamav      14  1. Jun 09:52 tnef-3672-88.doc
-rw-r--r--     1 qpsmtpd clamav      14  1. Jun 09:52 tnef-3672-89.doc
-rw-r--r--     1 qpsmtpd clamav    3542  1. Jun 09:52 tnef-3672-90.doc
-rw-r--r--     1 qpsmtpd clamav     396  1. Jun 09:52 tnef-3672-91.doc
-rw-r--r--     1 qpsmtpd clamav      14  1. Jun 09:52 tnef-3672-92.doc
-rw-r--r--     1 qpsmtpd clamav      13  1. Jun 09:52 tnef-3672-93.doc
-rw-r--r--     1 qpsmtpd clamav      14  1. Jun 09:52 tnef-3672-94.doc
-rw-r--r--     1 qpsmtpd clamav      14  1. Jun 09:52 tnef-3672-95.doc
-rw-r--r--     1 qpsmtpd clamav    7661  1. Jun 09:52 tnef-3672-96.doc
-rw-r--r--     1 qpsmtpd clamav     400  1. Jun 09:52 tnef-3672-97.doc
-rw-r--r--     1 qpsmtpd clamav      14  1. Jun 09:52 tnef-3672-98.doc
-rw-r--r--     1 qpsmtpd clamav      13  1. Jun 09:52 tnef-3672-99.doc


NOTE: be aware that if you don't know what you're doing, playing with CLI could be very dangerous.. I didn't tell you to try that "command".. if you're not familiar with CLI, before doing anything, ask for clarification

No Problem

clamav - time problem?

[JohnG]

is this helpful?

Mail left in /var/spool/qpsmtpd https://forums.contribs.org/index.php/topic,36588.msg162763.html#msg162763

[Daniel B.]

dbuechel should check /var/log/qpsmtpd/current /var/log/maillog and /var/log/clamd/current which might contain relevant information. A bug should be opened

[CharlieBrady]

In my experience deleting files in Linux does not always free the space instantly, I guess the garbage collection is run as a background task.

There's no garbage collection running as a background task.

There are two cases where space won't be freed when you delete a file. One is where there are multiple links to a file (i.e. where there are two or more directory entries, but only one file), and the other is when there is a process running which has an open file descriptor to the file - in which case the file becomes an anonymous file (i.e occupies space, but has no directory entries). When the last process which has an open file descriptor closes the descriptor, or exits, then the space will be freed.

[CharlieBrady]

is this helpful?

Mail left in /var/spool/qpsmtpd

This is a good case where qpsmtpd/clamd should be using anonymous files, so that it doesn't accidentally leave temporary files around (for example, if it crashes). An anonymous file is one which is opened, then deleted immediately while an open file descriptor is kept, so that data can be written to and read from the file.

[CharlieBrady]

I think you can safely delete them but if I were in your shoes, I'd like to know why are there so many files there..
wait for some other guys to comment

Definitely a bug.

[dbuechel]

Hi,

I made yesterday following:

Delete all fils in /var/spool/qpsmtpd over MC

Create the user Maillog again.
Create fetchmail to our server and start the service.

In the beginning everything looked good. Message has been scanned and then deleted.

Fetchmail was ever 15 minutes to check.

This morning the following:
Webmail tell me 23829 Emails.

In /var/spool/qpsmtpd => 22868 files !!!

But!

fetchmail[25935]: 23842 Nachrichten (23837 gesehene) für

Different from 5 mails.

When I look in my thunderbird - 23842 is correct - but ONE Email is unread!

This email is ~70 MB big - only 13 jpg-files.

I look in Webmail from SME -Server and search this email - he don't download it.

But fetchmail download other emails with big attachment. - see picture.

I found something interesting:

@40000000575040f10516835c --- Stopped at Thu Jun  2 16:21:27 2016
@40000000575040f105168744 Closing the main socket.
@40000000575040f105172384 Socket file removed.
@40000000575040f936a94b34 Limits: Global size limit set to 104857600 bytes.
@40000000575040f936a94f1c Limits: File size limit set to 15728640 bytes.
@40000000575040f936a95304 Limits: Recursion level limit set to 8.
@40000000575040f936a95304 Limits: Files limit set to 1500.
@40000000575040f936a95304 Limits: Core-dump limit is 0.
@40000000575040f936a956ec Limits: MaxEmbeddedPE limit set to 10485760 bytes.
@40000000575040f936a956ec Limits: MaxHTMLNormalize limit set to 10485760 bytes.
@40000000575040f936a956ec Limits: MaxHTMLNoTags limit set to 2097152 bytes.
@40000000575040f936a9b894 Limits: MaxScriptNormalize limit set to 5242880 bytes.
@40000000575040f936a9bc7c Limits: MaxZipTypeRcg limit set to 1048576 bytes.
@40000000575040f936a9bc7c Limits: MaxPartitions limit set to 50.
@40000000575040f936a9bc7c Limits: MaxIconsPE limit set to 100.
@40000000575040f936a9c064 Limits: MaxRecHWP3 limit set to 16.
@40000000575040f936a9c064 Limits: PCREMatchLimit limit set to 10000.
@40000000575040f936a9dbbc Limits: PCRERecMatchLimit limit set to 5000.
@40000000575040f936a9dfa4 Limits: PCREMaxFileSize limit set to 26214400.
@40000000575040f936a9dfa4 Archive support enabled.
@40000000575040f936a9dfa4 Algorithmic detection enabled.
@40000000575040f936a9dfa4 Portable Executable support enabled.
@40000000575040f936a9e38c ELF support enabled.
@40000000575040f936a9e38c Mail files support enabled.
@40000000575040f936a9e38c OLE2 support enabled.
@40000000575040f936a9e38c PDF support enabled.
@40000000575040f936a9e774 SWF support enabled.
@40000000575040f936a9f32c HTML support enabled.
@40000000575040f936a9f32c XMLDOCS support enabled.
@40000000575040f936a9f32c HWP3 support enabled.
@40000000575040f936a9f714 Heuristic: precedence enabled
@40000000575040f936aa06b4 Self checking every 1800 seconds.
@40000000575040f936aa0a9c Listening daemon: PID: 12125
@40000000575040f936aa0a9c MaxQueue set to: 100
@400000005750433925128ffc Reading databases from /var/clamav
@40000000575043421b1c0e4c Database correctly reloaded (4431156 signatures)
@4000000057504a4a1849eb1c SelfCheck: Database status OK.
@400000005750515203c2f1ac SelfCheck: Database status OK.
@4000000057505866138a20ec SelfCheck: Database status OK.
@4000000057505b181f682b34 /var/spool/qpsmtpd/1464883982:8587:10560: Win.Trojan.Agent-1349043 FOUND
@4000000057505df32dd7ec7c /var/spool/qpsmtpd/1464884713:8587:11841: Heuristics.Phishing.Email.SpoofedDomain FOUND
@4000000057505f8608c840bc SelfCheck: Database status OK.
@4000000057505fbd1f2e0864 LibClamAV Warning: cli_tnef: file truncated, returning CLEAN
@400000005750662f016619d4 /var/spool/qpsmtpd/1464886820:8587:14587: Win.Trojan.Agent-1335858 FOUND
@40000000575066960a9f5a4c SelfCheck: Database status OK.
@4000000057506dbc125a5764 SelfCheck: Database status OK.
@4000000057506eee0cbd6404 LibClamAV Warning: Bytecode run timed out in interpreter after 322730000 opcodes
@4000000057506eee0cbd67ec LibClamAV Warning: Bytcode 47 failed to run: Time limit reached
@4000000057507508007a5274 SelfCheck: Database status OK.
@4000000057507b812c774ef4 Reading databases from /var/clamav
@4000000057507b8a1b79e9a4 Database correctly reloaded (4432344 signatures)
@4000000057508293190b56bc SelfCheck: Database status OK.
@4000000057508c1927871654 SelfCheck: Database status OK.
@40000000575093212ded5cc4 SelfCheck: Database status OK.
@4000000057509a2933e78b54 SelfCheck: Database status OK.
@400000005750a1313a01f7cc SelfCheck: Database status OK.
@400000005750ac6d11fc5ccc SelfCheck: Database status OK.
@400000005750b6f905a00294 SelfCheck: Database status OK.
@400000005750be0109b150cc SelfCheck: Database status OK.
@400000005750c5090fc7063c SelfCheck: Database status OK.
@400000005750cc11163a1234 SelfCheck: Database status OK.
@400000005750d3191c50582c SelfCheck: Database status OK.
@400000005750da21226c7a24 SelfCheck: Database status OK.
@400000005750e129286a2f0c SelfCheck: Database status OK.
@400000005750e8312e83b8f4 SelfCheck: Database status OK.
@400000005750ef39347e29ec SelfCheck: Database status OK.
@400000005750f6413a955274 SelfCheck: Database status OK.
@4000000057510059336641ac SelfCheck: Database modification detected. Forcing reload.
@400000005751005a117de5cc Reading databases from /var/clamav
@40000000575100630f27d06c Database correctly reloaded (4439155 signatures)
@40000000575100632bc7bda4 Reading databases from /var/clamav
@400000005751006c289c08c4 Database correctly reloaded (4439155 signatures)
@40000000575107d501db4204 SelfCheck: Database status OK.
@4000000057510edd07e435d4 SelfCheck: Database status OK.
@40000000575115e50ebd087c SelfCheck: Database status OK.
@4000000057511c873a5d98d4 Reading databases from /var/clamav
@4000000057511c903722112c Database correctly reloaded (4440649 signatures)
@40000000575123f618212984 SelfCheck: Database status OK.

The first file in /var/spool/qpsmtpd is from 2. Jun 16.34

^^

I think is a bug

[Daniel B.]

When I look in my thunderbird - 23842 is correct - but ONE Email is unread!

This email is ~70 MB big - only 13 jpg-files.

I look in Webmail from SME -Server and search this email - he don't download it.

That's not surprising. 70MB is well over the default size limit

I think is a bug

Then please open a bug, and you'll be asked for specific information so we can track, and fix the issue

[dbuechel]

For the limitation of emails, it's 15MB by default. - correct?

I increase

Code: [Select]

config setprop qmail MaxMessageSize 100000000

[dbuechel]

I make now the following:

I delete the user maillog.
I delete all files in  /var/spool/qpsmtpd

I deactivated over server-manager the function - check email with antivirus.

Then:
I create the user maillog again - start fetchmail.

[Daniel B.]

I delete the user maillog.
I delete all files in  /var/spool/qpsmtpd

You shouldn't have. You'd better open a bug when the problem exists so devs can analyze and fix it

[JohnG]

I make now the following:

I delete the user maillog.
I delete all files in  /var/spool/qpsmtpd

I deactivated over server-manager the function - check email with antivirus.

Then:
I create the user maillog again - start fetchmail.

Any shutdowns and/or reboots?