Koozali.org: home of the SME Server

IBM System X 3250 4364 problem

Offline toothandnail

  • ****
  • 139
  • +0/-0
IBM System X 3250 4364 problem
« on: September 08, 2015, 03:10:26 PM »
I look after a business that has two of these machines, one as a gateway/mail server, the other as a dedicated file server. The machines both have LSI hardware raid controllers running in mirror mode. They've been ultra reliable, other than a couple of hardware glitches, but the load on them has grown to the point they need upgrading. Virus checking alone was taking more than 12 hours per night on both of them...

The file server has now been updated to a more recent System X machine, which is working very well with SME 9 64-bit. It will be a few months before the gateway machine is upgraded, so I hoped to get a bit better performance out of the existing hardware by upgrading the memory in the retired file server and using it to replace the existing gateway machine, running SME 9 64-bit.

SME 9 installs, but has a major problem on the first reboot after the install. After a lot of testing, I came to the conclusion that the most likely cause was that the kernel/drivers on the SME 9 install CD don't like the LSI controller. As a final test, I tried installing SME 9.1 beta 2. That went through the install without problems, and is running without any problems, so it looks as though my diagnosis was correct. Problem is, this is a busy production server, and I don't think I can risk running the beta on it in the production environment.

So I'm wondering  if there is any way that I can rebuild the standard SME 9 iso, but update the kernel/firmware to something more recent? That would allow me to get some performance improvements out of the system until such time as the company budget allows for replacing it with a new, more powerful box.

I've rebuilt Arch isos to customise them for my use, but I don't know enough about CentOS/SME to know if that can be done, or how to do it if it can be done...

Paul.

Offline stephdl

  • *
  • 1,523
  • +0/-0
    • Linux et Geekeries
Re: IBM System X 3250 4364 problem
« Reply #1 on: September 08, 2015, 03:46:29 PM »
SME 9 installs, but has a major problem on the first reboot after the install. After a lot of testing, I came to the conclusion that the most likely cause was that the kernel/drivers on the SME 9 install CD don't like the LSI controller. As a final test, I tried installing SME 9.1 beta 2. That went through the install without problems, and is running without any problems, so it looks as though my diagnosis was correct. Problem is, this is a busy production server, and I don't think I can risk running the beta on it in the production environment.

we planned to release the final sme9.1 at the end of September, either you wait for it, or you play with the sme9.1beta2 during this time. For me it is not worth the time you will lose. It is better that you give a hand in Bugzilla to help the development :)

I know Guys, I'm a dreamer
See http://wiki.contribs.org/Koozali_Foundation
irc : Freenode #sme_server #sme-fr

!!! Please write your knowledge to the Wiki !!!

Offline toothandnail

  • ****
  • 139
  • +0/-0
Re: IBM System X 3250 4364 problem
« Reply #2 on: September 08, 2015, 04:05:04 PM »
we planned to release the final sme9.1 at the end of September, either you wait for it, or you play with the sme9.1beta2 during this time. For me it is not worth the time you will lose. It is better that you give a hand in Bugzilla to help the development :)

Thanks. I should probably have asked if there was any tentative time table for the 9.1 release.

If its that close, I'll leave it alone. If nothing else, it will be a while before I can make the swap - given how busy the company is, it has to be done out of hours. Takes a while to restore 60+ GB of mail...

Quote
I know Guys, I'm a dreamer

:) I may be able to test the 9.1 beta in a less demanding production environment. Have to see if I can get that set up.

Paul.

Offline Stefano

  • *
  • 10,894
  • +3/-0
Re: IBM System X 3250 4364 problem
« Reply #3 on: September 08, 2015, 04:16:28 PM »
is the new machine RH6 certified?
SME9.1 will have a new kernel, but I'm quite sure you can have the same just upgrading your server..
Try this:
- install SME9 from cd
- try to reboot..
- if it fails, boot from a SME9.1beta iso in rescue mode and re-create the kernel image file (search the wiki and the forums for mkinitrd)
- reboot again.. it should work.. then update your server via yum

just guessing, not tested, should work, YMMV :-)

Offline toothandnail

  • ****
  • 139
  • +0/-0
Re: IBM System X 3250 4364 problem
« Reply #4 on: September 08, 2015, 04:43:25 PM »
is the new machine RH6 certified?

:) The new one, yes. The X3250 is a bit old for that - its RHEL 3 certified. First available towards the end of 2006, the samples I'm working with were bought sometime in early 2008.

Quote
SME9.1 will have a new kernel, but I'm quite sure you can have the same just upgrading your server..
Try this:
- install SME9 from cd
- try to reboot..
- if it fails, boot from a SME9.1beta iso in rescue mode and re-create the kernel image file (search the wiki and the forums for mkinitrd)
- reboot again.. it should work.. then update your server via yum

just guessing, not tested, should work, YMMV :-)

Hmm. I might get time to give this a try. It works fine with the original 9.1 beta 2 kernel, and I upgraded it to the latest kernel, which also works.

Installed from the original SME 9 CD, it attempts to boot, but does so in super-slow mode. Spends over an hour booting, and is very sluggish even when it gets to the login prompt.

Thanks for the suggestion. I'll give it a try and see how that works.

Paul.

Offline janet

  • *****
  • 4,812
  • +0/-0
Re: IBM System X 3250 4364 problem
« Reply #5 on: September 08, 2015, 05:43:23 PM »
Paul

Keep in mind that SME server beta releases are not usually "brand new" code. They use extensive code from previous stable releases & the underlying CentOS operating system code is already stable final release code.
Typically the changes between "point" versions of the same number version (eg 9.x) have code improvements & tweaks & some package version updates, rather than huge amounts of new code.

As such, over the years, SME server point releases or advanced betas of those, have proven to be quite stable, & it is known that many people have used them in production environments without experiencing serious problems.

I would suggest you test a beta release before deploying it, just as you would or should test any final release before using it.

Really, if your hardware requires a newer CentOS version to support newer hardware etc, then you have no choice.
As long as you understand that you may need to run an update or two on a beta system if any bugs are found, I think it is OK to use a beta release (of SME server).
I doubt that you would have any problems with basic functionality.
Please search before asking, an answer may already exist.
The Search & other links to useful information are at top of Forum.

Offline CharlieBrady

  • *
  • 6,918
  • +3/-0
Re: IBM System X 3250 4364 problem
« Reply #6 on: September 09, 2015, 01:20:05 PM »
After a lot of testing, I came to the conclusion that the most likely cause was that the kernel/drivers on the SME 9 install CD don't like the LSI controller.

If that is the issue, then you can consider ditching the LSI controller, and running hard drives from the native controller, using software RAID.

Offline Stefano

  • *
  • 10,894
  • +3/-0
Re: IBM System X 3250 4364 problem
« Reply #7 on: September 09, 2015, 01:25:18 PM »
If that is the issue, then you can consider ditching the LSI controller, and running hard drives from the native controller, using software RAID.

this is a solution indeed.. but it seems that OP only needs a more uptodate kernel

Offline toothandnail

  • ****
  • 139
  • +0/-0
Re: IBM System X 3250 4364 problem
« Reply #8 on: September 09, 2015, 10:51:25 PM »
If that is the issue, then you can consider ditching the LSI controller, and running hard drives from the native controller, using software RAID.

That was one of the first options I considered - if I'd specified the machines in the first place, I would most likely not have opted for the LSI controller, since Linux software raid would have done the job.

Unfortunately, in those machines, reverting to standard SATA controllers requires a different backplane. Which were originally supplied, but have since been discarded. So it would be an expensive option to use.

It looks as though a later kernel in SME 9 will fix the problem, or waiting until 9.1 comes out. Maybe I can do some testing of 9.1 while waiting.

Paul.

Offline toothandnail

  • ****
  • 139
  • +0/-0
Re: IBM System X 3250 4364 problem
« Reply #9 on: September 09, 2015, 10:56:20 PM »
this is a solution indeed.. but it seems that OP only needs a more uptodate kernel

:) Or a bit of patience to see when 9.1 is released. Though it might be interesting to see if I can get an updated kernel into a fresh 9.0 install.

Paul.

Offline Stefano

  • *
  • 10,894
  • +3/-0
Re: IBM System X 3250 4364 problem
« Reply #10 on: September 09, 2015, 11:00:03 PM »
you can, as long as you install SME9, reboot with SME9.1b cd and recreate the initrd file ;-)

Offline janet

  • *****
  • 4,812
  • +0/-0
Re: IBM System X 3250 4364 problem
« Reply #11 on: September 10, 2015, 02:48:49 AM »
toothandnail

Quote
:) Or a bit of patience to see when 9.1 is released.

Did you read my post about the "qualty" of SME betas    ?
http://forums.contribs.org/index.php/topic,51924.msg265033.html#msg265033

I think 9.1 beta 2 would be OK to use "as is", as it appears to be very close to release
You can upgrade or do a new install using a CD/DVD

Also read this comment on the main Wiki page
"July 1st, 2015. SME Server 9.1 beta 2 and SME Server 8.2 RC1 are now available for public testing. Please visit our testing area on one of our mirrors here and give them a spin. Please report all feedback to our bug tracker. The new ISO's passed initial and extensive tests as stable so challenge us! "
« Last Edit: September 10, 2015, 06:30:44 AM by janet »
Please search before asking, an answer may already exist.
The Search & other links to useful information are at top of Forum.

Offline janet

  • *****
  • 4,812
  • +0/-0
Re: IBM System X 3250 4364 problem
« Reply #12 on: September 10, 2015, 06:37:56 AM »
I was unable to test v9.1betas in the last few months due to holidays, & re-arranging home & equipment locations, but last week got a VirtualBox VM up & running with sme9.0 fresh install with a few contribs.
So today I just upgraded from CD my home VM from sme9.0 to 9.1beta2, & no problems.
No extensive testing done as yet, but basic functionality is there.
I am running the latest smeserver-ddclient contrib & a few others & everything is happening as it should.
There is nothing to indicate this is a beta, except the name.
Please search before asking, an answer may already exist.
The Search & other links to useful information are at top of Forum.