Koozali.org: home of the SME Server

procmail:_Couldn't_create_ files in /var/spool/mail/

Offline smeghead

  • *
  • 557
  • +0/-0
procmail:_Couldn't_create_ files in /var/spool/mail/
« on: December 25, 2021, 07:06:27 AM »
/var/log/qmail/current (and older logs) show the following issue for some active users:


delivery 17889: success: procmail:_Couldn't_create_"/var/spool/mail/<username>"/did_0+0+2/


Figured it might be just be log noise given the delivery states it was successful but a dig though the /var/spool/mail folder shows some user files missing & some files with different permissions based on a check of a coupla other servers.

To fix I wrote a basic script that dumps out the users from the accounts db & then processes the files in the /var/spool/mail folder against that user list:

#! /bin/sh

# Create input file with usernames
db accounts show | grep "=user$" | cut -d '=' -f1 > /root/db_userlist.txt

cd /var/spool/mail

# Loop through users in input file and create any missing user files and then set correct ownership and permissions
for F in $(cat /root/db_userlist.txt) ; do
   echo Processing User $F
   touch $F
   chown $F:mail $F
   chmod 660 $F
        echo Processing User $F Complete
done;

Once run this eliminates the issue in the log file.

Can a few other peeps cross check this for me & see if they have the same issue, if so I'll post a bug.

FYI - this server has all the following contribs installed (dump from /sbin/e-smith/audittools/newrpms)

GeoIP.x86_64                          1.6.12-9.el7.sme       @smecontribs
GeoIP-GeoLite-data.noarch             2018.06-7.el7.sme      @smecontribs
GeoIP-GeoLite-data-extra.noarch       2018.06-7.el7.sme      @smecontribs
denyhosts.noarch                      3.1-12.el7.sme         @smecontribs
e-smith-base.x86_64                   5.8.1-6.el7.sme        @smeupdates-testing
e-smith-cvm-unix-local.noarch         2.6.0-4.el7.sme        @smeupdates-testing
fail2ban-sendmail.noarch              0.11.2-3.el7           @smecontribs
fail2ban-server.noarch                0.11.2-3.el7           @smecontribs
gd3php.x86_64                         2.3.3-4.el7.remi       @remi-safe
jpgraph.noarch                        4.3.4-1.el7.sme        @smecontribs
libicu69.x86_64                       69.1-2.el7.remi        @remi-safe
mod_maxminddb.x86_64                  1.1.0-1.el7.sme        @smecontribs
msttcorefonts.noarch                  2.1-1.of.el7           @smecontribs
perl-B-Hooks-EndOfScope.noarch        0.24-1.of.el7          @smecontribs
perl-B-Hooks-OP-Check.x86_64          0.22-1.of.el7          @smecontribs
perl-Class-Load-XS.x86_64             0.10-1.of.el7          @smecontribs
perl-Class-Method-Modifiers.noarch    2.13-1.of.el7          @smecontribs
perl-Class-XSAccessor.x86_64          1.19-2.el7             @smecontribs
perl-Clone-PP.noarch                  1.06-1.of.el7          @smecontribs
perl-Data-Dumper-Concise.noarch       2.022-1.of.el7         @smecontribs
perl-Data-IEEE754.noarch              0.01-1.of.el7          @smecontribs
perl-Data-Printer.noarch              0.35-1.of.el7          @smecontribs
perl-Data-Validate-IP.noarch          0.27-13.el7            @smecontribs
perl-DateTime.x86_64                  2:1.10-1.of.el7        @smecontribs
perl-DateTime-Locale.noarch           0.4500-1.of.el7        @smecontribs
perl-Devel-GlobalDestruction.noarch   0.14-1.of.el7          @smecontribs
perl-Devel-OverloadInfo.noarch        0.007-1.of.el7         @smecontribs
perl-Devel-StackTrace.noarch          1:2.04-1.of.el7        @smecontribs
perl-Eval-Closure.noarch              0.14-1.of.el7          @smecontribs
perl-File-HomeDir.noarch              1.002-1.of.el7         @smecontribs
perl-Geo-IP.x86_64                    1.45-1.of.el7          @smecontribs
perl-GeoIP2.noarch                    2.001002-1.of.el7      @smecontribs
perl-Hash-FieldHash.x86_64            0.14-1.of.el7          @smecontribs
perl-IO-Socket-IP.noarch              0.38-1.of.el7          @smeupdates-testing
perl-IO-Socket-SSL.noarch             2.027-1.of.el7         @smeupdates-testing
perl-JSON.noarch                      2.90-1.of.el7          @smeupdates-testing
perl-Lexical-SealRequireHints.x86_64  0.011-1.of.el7         @smecontribs
perl-List-AllUtils.noarch             0.08-1.of.el7          @smecontribs
perl-MIME-Lite.noarch                 3.030-1.el7            @smecontribs
perl-MLDBM.noarch                     2.05-1.el7             @smecontribs
perl-MRO-Compat.noarch                0.12-2.el7             @smecontribs
perl-Math-Int128.x86_64               0.18-1.of.el7          @smecontribs
perl-Math-Int64.x86_64                0.52-1.el7             @smecontribs
perl-MaxMind-DB-Common.noarch         0.040000-1.of.el7      @smecontribs
perl-MaxMind-DB-Reader.noarch         1.000004-1.of.el7      @smecontribs
perl-Module-Implementation.noarch     0.09-1.of.el7          @smecontribs
perl-Module-Runtime.noarch            0.016-1.of.el7         @smecontribs
perl-Module-Runtime-Conflicts.noarch  0.003-1.of.el7         @smecontribs
perl-Moo.noarch                       2.004004-2.of.el7      @smecontribs
perl-MooX-StrictConstructor.noarch    0.006-1.of.el7         @smecontribs
perl-MooX-Types-MooseLike.noarch      0.25-1.of.el7          @smecontribs
perl-Moose.x86_64                     2.2015-1.of.el7        @smecontribs
perl-Net-Works.noarch                 0.21-1.of.el7          @smecontribs
perl-Params-Classify.x86_64           0.013-7.el7            @smecontribs
perl-Params-Validate.x86_64           1.30-1.of.el7          @smecontribs
perl-Role-Tiny.noarch                 2.001004-1.of.el7      @smecontribs
perl-Sort-Naturally.noarch            1.03-8.el7             @smecontribs
perl-Sub-Exporter-Progressive.noarch  0.001013-1.of.el7      @smecontribs
perl-Sub-Identify.x86_64              0.14-1.of.el7          @smecontribs
perl-Sub-Install.noarch               0.928-1.of.el7         @smecontribs
perl-Sub-Name.x86_64                  0.26-1.of.el7          @smecontribs
perl-Sub-Quote.noarch                 2.006006-1.of.el7      @smecontribs
perl-Test-Warnings.noarch             0.030-1.of.el7         @smecontribs
perl-Throwable.noarch                 0.200011-1.of.el7      @smecontribs
perl-Try-Tiny.noarch                  0.22-1.of.el7          @smecontribs
perl-Unicode-IMAPUtf7.noarch          2.01-1.of.el7          @smecontribs
perl-Variable-Magic.x86_64            0.62-1.of.el7          @smecontribs
perl-bareword-filehandles.x86_64      0.007-1.of.el7         @smecontribs
perl-indirect.x86_64                  0.39-1.of.el7          @smecontribs
perl-multidimensional.x86_64          0.014-1.of.el7         @smecontribs
perl-namespace-autoclean.noarch       0.29-1.of.el7          @smecontribs
perl-namespace-clean.noarch           0.27-1.of.el7          @smecontribs
perl-strictures.noarch                2.000006-1.of.el7      @smecontribs
php81-php.x86_64                      8.1.1-1.el7.remi       @remi-safe
php81-php-bcmath.x86_64               8.1.1-1.el7.remi       @remi-safe
php81-php-cli.x86_64                  8.1.1-1.el7.remi       @remi-safe
php81-php-common.x86_64               8.1.1-1.el7.remi       @remi-safe
php81-php-enchant.x86_64              8.1.1-1.el7.remi       @remi-safe
php81-php-fpm.x86_64                  8.1.1-1.el7.remi       @remi-safe
php81-php-gd.x86_64                   8.1.1-1.el7.remi       @remi-safe
php81-php-imap.x86_64                 8.1.1-1.el7.remi       @remi-safe
php81-php-intl.x86_64                 8.1.1-1.el7.remi       @remi-safe
php81-php-ldap.x86_64                 8.1.1-1.el7.remi       @remi-safe
php81-php-mbstring.x86_64             8.1.1-1.el7.remi       @remi-safe
php81-php-mysqlnd.x86_64              8.1.1-1.el7.remi       @remi-safe
php81-php-opcache.x86_64              8.1.1-1.el7.remi       @remi-safe
php81-php-pdo.x86_64                  8.1.1-1.el7.remi       @remi-safe
php81-php-pear.noarch                 1:1.10.13-1.el7.remi   @remi-safe
php81-php-pecl-xmlrpc.x86_64          1.0.0~rc3-1.el7.remi   @remi-safe
php81-php-pecl-zip.x86_64             1.20.0-1.el7.remi      @remi-safe
php81-php-process.x86_64              8.1.1-1.el7.remi       @remi-safe
php81-php-snmp.x86_64                 8.1.1-1.el7.remi       @remi-safe
php81-php-soap.x86_64                 8.1.1-1.el7.remi       @remi-safe
php81-php-sodium.x86_64               8.1.1-1.el7.remi       @remi-safe
php81-php-tidy.x86_64                 8.1.1-1.el7.remi       @remi-safe
php81-php-xml.x86_64                  8.1.1-1.el7.remi       @remi-safe
php81-runtime.x86_64                  8.1-1.el7.remi         @remi-safe
rkhunter.noarch                       1.4.6-3.el7            @smecontribs
smeserver-check4updates.noarch        0.0.4-4.el7.sme        @smecontribs
smeserver-clamav.noarch               2.7.0-11.el7.sme       @smeupdates-testing
smeserver-crontab_manager.noarch      2.4-7.el7.sme          @smecontribs
smeserver-denyhosts.noarch            2.9-16.el7.sme         @smecontribs
smeserver-dhcp-dns.noarch             1.2.0-5.el7.sme        @smecontribs
smeserver-dhcpmanager.noarch          2.0.4-12.el7.sme       @smecontribs
smeserver-diskusage.noarch            0.2.0-5.el7.sme        @smecontribs
smeserver-domains.noarch              1.4-4                  @smecontribs
smeserver-dovecot-extras.noarch       0.1.6-4.el7.sme        @smecontribs
smeserver-durep.noarch                1.5.0-9.el7.sme        @smecontribs
smeserver-email-management.noarch     1.3-5.el7.sme          @smecontribs
smeserver-fail2ban.noarch             9:0.1.18-24.el7.sme    @smecontribs
smeserver-geoip.noarch                1.2-18.el7.sme         @smecontribs
smeserver-groupmembers-panel.noarch   1.0.4-3.el7.sme        @smecontribs
smeserver-hwinfo.noarch               1.2-5.el7.sme          @smecontribs
smeserver-lazy_admin_tools.noarch     1.1-6.el7.sme          @smecontribs
smeserver-mailalias.noarch            0.1-8.el7.sme          @smecontribs
smeserver-mailsorting.noarch          1.4-14.el7.sme         @smecontribs
smeserver-mailstats.noarch            1.1-16.el7.sme         @smecontribs
smeserver-mod_maxminddb.noarch        1.1.0-11.el7.sme       @smecontribs
smeserver-motd.noarch                 0.1-8                  @smecontribs
smeserver-php.x86_64                  3.0.0-42.el7.sme       @smeupdates-testing
smeserver-qmHandle.noarch             1.4-16.el7.sme         @smecontribs
smeserver-remoteuseraccess.noarch     1.3-6.el7.sme          @smecontribs
smeserver-rkhunter.noarch             1.4.0-7.el7.sme        @smecontribs
smeserver-rsync-backup2disk.noarch    0.0.2-3                @smecontribs
smeserver-service_control.noarch      2.2-16.el7.sme         @smecontribs
smeserver-smbstatus.noarch            1.2-3                  @smecontribs
smeserver-unjunkmgr.noarch            3.1-5.el7.sme          @smetest
smeserver-userpanel.noarch            1.4-3.el7.sme          @smecontribs
smeserver-userpanels.noarch           1.3-3.el7.sme          @smecontribs
smeserver-vacation.noarch             1.1-33.el7.sme         @smecontribs
smeserver-wbl.noarch                  0.5.0-5.el7.sme        @smecontribs
smeserver-xt_geoip.noarch             1.3.1-11.el7.sme       @smecontribs
smeserver-yum.noarch                  2.6.0-59.el7.sme       @smeupdates-testing
spamassassin.x86_64                   3.4.6-1.el7.sme        @smeupdates-testing
xtables-addons.x86_64                 2.14-01.el7.sme        @smecontribs
xtables-addons-kmod.x86_64            2.14-05.el7.sme        @smecontribs
..................

Offline ReetP

  • *
  • 3,722
  • +5/-0
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline Jean-Philippe Pialasse

  • *
  • 2,747
  • +11/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: procmail:_Couldn't_create_ files in /var/spool/mail/
« Reply #2 on: December 26, 2021, 06:17:52 AM »
what is the history of the server ?

Offline smeghead

  • *
  • 557
  • +0/-0
Re: procmail:_Couldn't_create_ files in /var/spool/mail/
« Reply #3 on: December 26, 2021, 08:24:45 AM »
.. built late August this year, baremetal not VM.

Yum'd up to date as of 2 days ago.

Hosts email only for 2 domains; approx 10 users added

Public websites for domains are hosted elsewhere on the Internet; Hostnames panel has been updated to that effect.

Additional contribs installed as per orig post.
..................

Offline Jean-Philippe Pialasse

  • *
  • 2,747
  • +11/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: procmail:_Couldn't_create_ files in /var/spool/mail/
« Reply #4 on: December 26, 2021, 02:00:40 PM »
install from iso? from centos2sme?
only fresh install or restored sme9 from
backup?
if so what kind of backup and restore? (console, workstation, migratehelper, rsync, affa…)

we have no crystal balls.  help us to help you…

Offline smeghead

  • *
  • 557
  • +0/-0
Re: procmail:_Couldn't_create_ files in /var/spool/mail/
« Reply #5 on: December 26, 2021, 02:31:54 PM »
install from iso? from centos2sme?
only fresh install or restored sme9 from
backup?
if so what kind of backup and restore? (console, workstation, migratehelper, rsync, affa…)

we have no crystal balls.  help us to help you…

Clean install ISO, no restore.

To me clean install means just that, no other messing with the system bar installing contribs once the box is yum'd up to date.

The SME backup/restore process I have found over the years to be clumsy given that most servers I work on have many contribs and restores presuppose none, historically lotsa stuff is broken with such restores.  I find DAR to be a pain, much prefer TGZ as for me at least it's WAY more flexible.  On the odd occasion, on a build, that I need to restore something from a (workstation) backup I will copy the full backup file(s) to the SME box as backup files, say TGZ files, & extract what I need to the correct location, usually only email data files, then run a signal-event post-upgrade process to be sure the perms & ownership are correct (no reason they shouldn't be but doesn't hurt).  This is only ever done after the users are set up on the box.

Servers are always installed in public server/gateway mode.
..................

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: procmail:_Couldn't_create_ files in /var/spool/mail/
« Reply #6 on: December 26, 2021, 04:53:02 PM »
Quote
Clean install ISO, no restore.

Yes, we just want to understand the history. As JP said, we are blind and cannot see how you got to this point.

You do not have a clean install now because you have restored users/data somehow (we'd like to know how - you make some explanation of different restore methods but don't explain for THIS server). You have added contribs which again changes things. Between those is the likely culprit for the error.

It would have been good to know the permissions before you started trying to fix things. Not so easy to find out what has happened. Did you record the perms?

Did you restore anything in /var/* ?

Quote
The SME backup/restore process I have found over the years to be clumsy

It isn't 'clumsy' but it probably isn't a complex as you want.

There are a number of reasons for this.

The main one is upstream changes to packages & configurations that are out of our control.

v8-v9 required a clean install/restore primarily due to changes in rpms by RH. That complicated things.

v9-v10 mainly due to systemd, plus other changes, particularly httpd/php to try and accomodate user requirements eg multiple PHP versions.

That means any custom templates (note the use of the word 'custom' as in 'non standard') would quite possibly break a new install (which you can see has happened a lot), and there is no simple way to get around that. Custom configs are pretty well infinite. We can't accomodate everything - we do our best, and tend to orientate things to those who help.

Of course the other practical limitation is manpower. We don't have anywhere enough. There are a handful of part time unpaid devs who do this because we enjoy it. That's it.

If you want to effect change and make things less 'clumsy' then rather than sit back, take the easy route and criticise, please get involved and help code your solution. We'll be delighted.

You may then understand some of the  complexities involved.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline smeghead

  • *
  • 557
  • +0/-0
Re: procmail:_Couldn't_create_ files in /var/spool/mail/
« Reply #7 on: December 26, 2021, 07:42:05 PM »
.. I have indicated the state/condition of the server, fairly explicitly I thought; contribs added, user email data restored only, configuration of this server done via SME itself.  Again as stated ONLY user email data restored.

On the clumsy issue for me I beg to differ.  While the TGZ file could get a bit unwieldy for servers with a large amount of data using DAR for me is even more problematic.  I live most of my IT world in the Windows environment & TGZ is very simple to handle, DAR not.  There is one program & one GUI for DAR under Windows, they work ok but somewhat limited.  Given the range of archive formats around, that are pretty common to both Linux & Windows, I would have thought a more prevalent format in both environments would be better option.  I'm sure there were reasons DAR was chosen they are just unknown to the 'great unwashed'.

Lastly am I criticizing, not really, or at least trying to be constructive about it, is reported/discussing an issue criticizing?  I point out my issue, & why I do things the way I do.  Nothing can be perfect for everyone, understood, I don't expect it to be so, but to that end I explain my issue & anything I do to work around it.  I do this for 2 reasons, firstly so others can refer to their systems to see if the issue is specific to my builds, and contribute to a bit of discovery, secondly so others may benefit from any workaround/fix I may have come up with.

I also ask here so that if it does seem to be a bug of some sort I can then make a post to bugzilla inc a fix if I have one.

It's poss the client may have a had a tinker themselves (I always try to educate as I go) & messed with something they shouldn't have, unlikely but possible, Ill ask.
..................

Offline sages

  • *
  • 182
  • +0/-0
    • http://www.sages.com.au
Re: procmail:_Couldn't_create_ files in /var/spool/mail/
« Reply #8 on: December 26, 2021, 11:57:21 PM »
.. I have indicated the state/condition of the server, fairly explicitly I thought; contribs added, user email data restored only, configuration of this server done via SME itself.  Again as stated ONLY user email data restored.
Well somewhere between when you posted explicit explanation and the unpaid volunteer people who are trying to help you read your post there has been some form of communications failure. You could try posting that you have explicitly described the configuration and changes (if any) that have been made and see if that helps clarify things or perhaps you could consider working with the unpaid volunteers to reduce the amount of unpaid volunteer time and effort they have to apply to help you.
Perhaps given your knowledge of the system and that basically the unpaid volunteers appear to be struggling to understand what may or may not have been done to your customer's (paid customer perhaps?) you could provide some further information. Just wondering who is asking for the unpaid volunteer help on what may turn out to be a paid customers system? Nothing wrong with that but it might be something to think about in how the issue is approached and (hopefully) eventually resolved. Have a nice day.
...

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: procmail:_Couldn't_create_ files in /var/spool/mail/
« Reply #9 on: December 27, 2021, 01:19:48 AM »
Quote
I have indicated the state/condition of the server, fairly explicitly I thought;

No, you said 'clean install' and then added a load of other stuff that does nothing but confuse. Hence the questions.

Quote
On the clumsy issue for me I beg to differ.
You can differ and beg all you like.

My point remains.

If you have better solutions then we look forward to your code.

I think you will find that tgz is the standard backup, and recommended for upgrades, though we do have a number of alternative methods for your delectation and delight.

https://wiki.koozali.org/SME_Server:Documentation:Administration_Manual:Chapter5

We do try to provide alternative choices where possible. However, I'm not sure what relevance the backup format has to do with upgrading.

None of it will change the fundamental issues of underlying system changes not being compatible with a multitude of flakey custom hacks.

If we tried to cater for them all we'd spend so long trying to fix the issues that we'd never write any code.

Again, if you took time to explore the code and understand the issues and complexities involved you may get a greater appreciation of the situation.

And if you do find it trivial to fix we'll be happy to commit your code.

Of course this is now becoming an XY problem.

Please take time to have a read of this:

https://forums.contribs.org/index.php/topic,54724.0.html

And of course if you have details of the issue prior to your attempt at a fix then please share it here as we can't help much without it, or how to create and then debug it.

Thanks.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation