Koozali.org: home of the SME Server

Scenario: NVMe raid1 wish to add 4x 2Tb drives for raid5/6

Offline cpits

  • 10
  • +0/-0
Re: Scenario: NVMe raid1 wish to add 4x 2Tb drives for raid5/6
« Reply #15 on: September 01, 2021, 04:54:58 AM »
I have run the bios update, from version 52 to 62. Seems quite stable presently so we will be monitoring over the next day or 2.

8168 realtek you will at least need to install elrepo kmod for it and not using the 8169 driver.

there are a few posts in the forum about that. 

and again looking at google maybe also need to  update your bios.

Offline sages

  • *
  • 182
  • +0/-0
    • http://www.sages.com.au
Re: Scenario: NVMe raid1 wish to add 4x 2Tb drives for raid5/6
« Reply #16 on: September 01, 2021, 05:00:44 AM »
For those of you who will be TL;DR the key to this is _netdev

The links will explain why in more detail. Note this could be a LVM, ext4 or any other type of partiton.

As Jean said:

So the simple solution here is as per Terrys code above.

OK Mr Fage. Fun over. Get back to testing my hacks :lol:

The way I read Terry's code the _netdev line in /etc/fstab is actually commented out. _netdev is needed for network filesystems to ensure the network is up before trying to mount.
« Last Edit: September 01, 2021, 05:02:32 AM by sages »
...

Offline TerryF

  • grumpy old man
  • *
  • 1,826
  • +6/-0
Re: Scenario: NVMe raid1 wish to add 4x 2Tb drives for raid5/6
« Reply #17 on: September 01, 2021, 05:49:21 AM »
The way I read Terry's code the _netdev line in /etc/fstab is actually commented out. _netdev is needed for network filesystems to ensure the network is up before trying to mount.

Theres is a issue with the Raid6 being processed before the network is up, causes the OS to derop into emergency mode, see the links I posted, there are two methods to overcome..

adding _netdev to the fstab entry OR  noauto,x-systemd.automount I have cjosen to use the latter, see the references listed..

#/dev/md11 /home/e-smith/files/shares  ext4  _netdev  0 0
/dev/md11 /home/e-smith/files/shares  ext4  noauto,x-systemd.automount  0 0

or using UUID
#UUID=29dbfbc8-5562-4800-9c35-2733af1c74d2 /home/e-smith/files/shares  ext4  _netdev  0 0
UUID=29dbfbc8-5562-4800-9c35-2733af1c74d2 /home/e-smith/files/shares  ext4  noauto,x-systemd.automount  0 0
--
qui scribit bis legit

Offline sages

  • *
  • 182
  • +0/-0
    • http://www.sages.com.au
Re: Scenario: NVMe raid1 wish to add 4x 2Tb drives for raid5/6
« Reply #18 on: September 01, 2021, 05:53:25 AM »
Got it.
Raid 1 on a vm seemed to be happy without either. But have used the systemd variant and it works fine.
...

Offline TerryF

  • grumpy old man
  • *
  • 1,826
  • +6/-0
Re: Scenario: NVMe raid1 wish to add 4x 2Tb drives for raid5/6
« Reply #19 on: September 01, 2021, 05:58:32 AM »
yep, default install Raid 1 or even default install with 4 or more disks will get you a Raid6 all fine, its when the second raid is added the issue arose, good fun :-) totally new to me...
--
qui scribit bis legit

Offline TerryF

  • grumpy old man
  • *
  • 1,826
  • +6/-0
Re: Scenario: NVMe raid1 wish to add 4x 2Tb drives for raid5/6
« Reply #20 on: September 01, 2021, 06:03:27 AM »
Of course we all know what happens now :-) order placed for some new largish hard drives :-) home server already on ssd but just a single data drive .. just need to formulate the $$ excuse for Mrs TerryF  :-)
--
qui scribit bis legit

Offline Jean-Philippe Pialasse

  • *
  • 2,763
  • +11/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: Scenario: NVMe raid1 wish to add 4x 2Tb drives for raid5/6
« Reply #21 on: September 01, 2021, 08:22:55 AM »
with Centos 6 you had to rebuild the initram to avoid this issue https://forums.centos.org/viewtopic.php?t=65257

_netdev or x-systemd.automount will just lead to prevent the mount of disk during the init.

because your init need to load the needed modules for raid 6. also you had to declare your raid at grub. rd_MD_UUID=

for Centos 7, this needs some changes too (about the same but different):

https://forums.centos.org/viewtopic.php?t=65655
https://forums.centos.org/viewtopic.php?t=54901

rd.md.uuid=$youruuid or rd.auto=1 in GRUB_CMDLINE_LINUX line of /etc/default/grub
and then rebuild grub.conf and rebuild initramfs

#dracut --regenerate-all -fv --mdadmconf --fstab --add=mdraid --add-drivers="raid1 raid10 raid456"

then as long as adhoc module is added to the module to load, next initramfs reneration on update should add it to new kernel initramfs
« Last Edit: September 01, 2021, 08:24:44 AM by Jean-Philippe Pialasse »

Offline cpits

  • 10
  • +0/-0
Re: Scenario: NVMe raid1 wish to add 4x 2Tb drives for raid5/6
« Reply #22 on: March 06, 2022, 02:53:15 AM »
UPDATE:
Recently our area suffered a blackout, whilst connected to a UPS the system blew a number of resistors near the cpu socket and we have the motherboard being replaced under warranty.

We've had to purchase a new board in the meantime to get our server back online, so hopefully we get this raid config underway!

I have run the bios update, from version 52 to 62. Seems quite stable presently so we will be monitoring over the next day or 2.

Offline TerryF

  • grumpy old man
  • *
  • 1,826
  • +6/-0
Re: Scenario: NVMe raid1 wish to add 4x 2Tb drives for raid5/6
« Reply #23 on: March 06, 2022, 04:16:06 AM »
UPDATE:
Recently our area suffered a blackout, whilst connected to a UPS the system blew a number of resistors near the cpu socket and we have the motherboard being replaced under warranty.

Damn...  :shock: :shock: :shock:
--
qui scribit bis legit