Topic: Can't send or receive email

[RCFerguson]

I have a 7.5.1 server that stopped sending and receiving external email the other day.  In looking at the logs I'm seeing some errors:

messages:
Oct  3 02:35:15 server kernel: Bad page state at prep_new_page (in process 'spamd', page c10cf7e0)
Oct  3 02:35:15 server kernel: flags:0x2000106c mapping:c1aff6b4 mapcount:0 count:2
Oct  3 02:35:15 server kernel: Backtrace:
Oct  3 02:35:15 server kernel:  [bad_page+88/137] bad_page+0x58/0x89
Oct  3 02:35:15 server kernel:  [<c0150461>] bad_page+0x58/0x89
Oct  3 02:35:15 server kernel:  [prep_new_page+36/58] prep_new_page+0x24/0x3a
Oct  3 02:35:15 server kernel:  [<c015086e>] prep_new_page+0x24/0x3a
Oct  3 02:35:15 server kernel:  [buffered_rmqueue+452/487] buffered_rmqueue+0x1c4/0x1e7
Oct  3 02:35:15 server kernel:  [<c0150ed5>] buffered_rmqueue+0x1c4/0x1e7
Oct  3 02:35:15 server kernel:  [__alloc_pages+180/689] __alloc_pages+0xb4/0x2b1
Oct  3 02:35:15 server kernel:  [<c0150fac>] __alloc_pages+0xb4/0x2b1
Oct  3 02:35:15 server kernel:  [anon_vma_prepare+31/459] anon_vma_prepare+0x1f/0x1cb
Oct  3 02:35:15 server kernel:  [<c0162be7>] anon_vma_prepare+0x1f/0x1cb
Oct  3 02:35:15 server kernel:  [do_wp_page+468/1058] do_wp_page+0x1d4/0x422
Oct  3 02:35:15 server kernel:  [<c015d1f8>] do_wp_page+0x1d4/0x422
Oct  3 02:35:15 server kernel:  [common_interrupt+24/32] common_interrupt+0x18/0x20
Oct  3 02:35:15 server kernel:  [<c0324024>] common_interrupt+0x18/0x20
Oct  3 02:35:15 server kernel:  [handle_mm_fault+310/530] handle_mm_fault+0x136/0x212
Oct  3 02:35:15 server kernel:  [<c015e46d>] handle_mm_fault+0x136/0x212
Oct  3 02:35:15 server kernel:  [do_page_fault+428/1244] do_page_fault+0x1ac/0x4dc
Oct  3 02:35:15 server kernel:  [<c011decd>] do_page_fault+0x1ac/0x4dc
Oct  3 02:35:15 server kernel:  [vma_merge+225/357] vma_merge+0xe1/0x165
Oct  3 02:35:15 server kernel:  [<c015faec>] vma_merge+0xe1/0x165
Oct  3 02:35:15 server kernel:  [recalc_task_prio+296/307] recalc_task_prio+0x128/0x133
Oct  3 02:35:15 server kernel:  [<c011f972>] recalc_task_prio+0x128/0x133
Oct  3 02:35:15 server kernel:  [do_IRQ+810/823] do_IRQ+0x32a/0x337
Oct  3 02:35:15 server kernel:  [<c0108a8e>] do_IRQ+0x32a/0x337
Oct  3 02:35:15 server kernel:  [do_page_fault+0/1244] do_page_fault+0x0/0x4dc
Oct  3 02:35:15 server kernel:  [<c011dd21>] do_page_fault+0x0/0x4dc
Oct  3 02:35:15 server kernel:  [error_code+47/56] error_code+0x2f/0x38
Oct  3 02:35:15 server kernel:  [<c03240e3>] error_code+0x2f/0x38
Oct  3 02:35:15 server kernel: Trying to fix it up, but a reboot is needed
Oct  3 02:35:15 server kernel: klogd 1.4.1, ---------- state change ----------
Oct  3 02:35:16 server kernel: Inspecting /boot/System.map-2.6.9-89.0.25.EL

and qpsmtpd/current:
@400000004caba7ea0086932c BEGIN failed--compilation aborted at /usr/lib/perl5/vendor_perl/5.8.5/MIME/Head.pm line 125.
@400000004caba7ea0086ae84 Compilation failed in require at /usr/lib/perl5/vendor_perl/5.8.5/MIME/Parser.pm line 147.
@400000004caba7ea0086c9dc BEGIN failed--compilation aborted at /usr/lib/perl5/vendor_perl/5.8.5/MIME/Parser.pm line 147.
@400000004caba7ea0086e91c Compilation failed in require at /usr/share/qpsmtpd/plugins/tnef2mime line 24.
@400000004caba7ea00875a64 BEGIN failed--compilation aborted at /usr/share/qpsmtpd/plugins/tnef2mime line 24.
@400000004caba7eb183ac434 eval MIME/Field/ParamVal.pm did not return a true value at /usr/lib/perl5/vendor_perl/5.8.5/MIME/Head.pm line 125.


Obviously something is corrupted...can anyone tell me what?

[RCFerguson]

Just noticed something very strange:

[root@server Field]# ls -la
total 24
drwxr-xr-x  2 root   root   4096 Jun 20 14:34 .
drwxr-xr-x  5 root   root   4096 Jun 20 14:34 ..
-rw-r--r--  1 root   root   1194 Mar 17  2006 ContDisp.pm
-rw-r--r--  1 root   root   1310 Mar 17  2006 ConTraEnc.pm
drwxr-sr-x  2 dnslog dnslog 4096 Oct  2 21:42 ContType.pm
drwxr-sr-x  2 dnslog dnslog 4096 Oct  3 00:07 ParamVal.pm

It shows that ContType.om and ParamVal.pm are both DIRECTORIES, not FILES, and are owned by dnslog.  I checked a working sme server installation and these are both FILES, not DIRECTORIES, and are owned by root.  I wonder if the previous reboot wasn't clean, and inodes got messed up?  Can I safely rename or delete these two directories and replace them with the correct files?  Both of these directories have current dns logfiles in them.  What tells tinydns where to keep it's log and state files?

[RCFerguson]

Digging into it more, it appears that the two directories, /usr/lib/perl5/vendor_perl/5.8.5/MIME/Field/ContType.pm and /usr/lib/perl5/vendor_perl/5.8.5/MIME/Field/ParamVal.pm are identical to the directories /var/log/dnscache and /var/log/dnscache.forwarder, so there must be some haywire inodes.  How do I fix?

[chris burnat]

Did you do anything to this server around the time this problem showed itself? i.e. Installed some package(s), or changed configuration from the CL, or whatever.
If affirmative, now would be a good time to provide this information.   If negative, then you should report this to the bug tracker, it may be a bug. Provide as much information as possible, the Devs do not have to time to trawl the forum for details. It may be useful providing the output of the following commands:
/sbin/e-smith/audittools/newrpms
/sbin/e-smith/audittools/templates