Topic: Software Raid rebuilding on every startup?

[Dan Williams]

Hi,
I have two "exact drives" on the primary channel, for raid1.
I have experimented, and found that once the server is up, it syncs the drives etc, and all is good.
However, I have found that every time I power down the server, and then power back up at a later time, it re-syncs the drives, every time.
I have installed the contrib from Darrell to monotor this, and this is what I am seeing:
+++++++++++++++++
ALARM! RAID configuration problem

Current configuration is:

Personalities : [raid1]
read_ahead 1024 sectors
md2 : active raid1 hdb1[1] hda1[0] 264960 blocks [2/2] [UU] resync=DELAYED
md0 : active raid1 hdb5[1] hda5[0] 15936 blocks [2/2] [UU] resync=DELAYED
md1 : active raid1 hdb6[1] hda6[0] 38796864 blocks [2/2] [UU] resync=9% finish=353.8min
unused devices:

Last known good configuration was:

Personalities : [raid1]
read_ahead 1024 sectors
md2 : active raid1 hdb1[1] hda1[0] 264960 blocks [2/2] [UU]
md0 : active raid1 hdb5[1] hda5[0] 15936 blocks [2/2] [UU]
md1 : active raid1 hdb6[1] hda6[0] 38796864 blocks [2/2] [UU]
unused devices:
++++++++++++++++++++++
Is this normal behaviour?
This is what is in the messages log:

+++++++++++++++++++++++
Jan 12 10:07:21 wrpserver kernel: autodetecting RAID arrays
Jan 12 10:07:21 wrpserver kernel: (read) hda1's sb offset: 264960 [events: 00000011]
Jan 12 10:07:21 wrpserver kernel: (read) hda5's sb offset: 15936 [events: 00000011]
Jan 12 10:07:21 wrpserver kernel: (read) hda6's sb offset: 38796864 [events: 00000011]
Jan 12 10:07:21 wrpserver kernel: (read) hdb1's sb offset: 264960 [events: 00000011]
Jan 12 10:07:21 wrpserver kernel: (read) hdb5's sb offset: 15936 [events: 00000011]
Jan 12 10:07:21 wrpserver kernel: (read) hdb6's sb offset: 38796864 [events: 00000011]
Jan 12 10:07:21 wrpserver kernel: autorun ...
Jan 12 10:07:21 wrpserver kernel: considering hdb6 ...
Jan 12 10:07:21 wrpserver kernel:   adding hdb6 ...
Jan 12 10:07:21 wrpserver kernel:   adding hda6 ...
Jan 12 10:07:21 wrpserver kernel: created md1
Jan 12 10:07:21 wrpserver kernel: bind
Jan 12 10:07:21 wrpserver kernel: bind
Jan 12 10:07:21 wrpserver kernel: running:
Jan 12 10:07:21 wrpserver kernel: now!
Jan 12 10:07:21 wrpserver kernel: hdb6's event counter: 00000011
Jan 12 10:07:21 wrpserver kernel: hda6's event counter: 00000011
Jan 12 10:07:21 wrpserver kernel: md: md1: raid array is not clean -- starting background reconstruction
Jan 12 10:07:21 wrpserver kernel: request_module[md-personality-3]: Root fs not mounted
Jan 12 10:07:21 wrpserver kernel: do_md_run() returned -22
Jan 12 10:07:21 wrpserver kernel: unbind
Jan 12 10:07:21 wrpserver kernel: export_rdev(hdb6)
Jan 12 10:07:21 wrpserver kernel: unbind
Jan 12 10:07:21 wrpserver kernel: export_rdev(hda6)
Jan 12 10:07:21 wrpserver kernel: md1 stopped.
Jan 12 10:07:21 wrpserver kernel: considering hdb5 ...
Jan 12 10:07:21 wrpserver kernel:   adding hdb5 ...
Jan 12 10:07:21 wrpserver kernel:   adding hda5 ...
Jan 12 10:07:21 wrpserver kernel: created md0
Jan 12 10:07:22 wrpserver kernel: bind
Jan 12 10:07:22 wrpserver kernel: bind
Jan 12 10:07:22 wrpserver kernel: running:
Jan 12 10:07:22 wrpserver kernel: now!
Jan 12 10:07:22 wrpserver kernel: hdb5's event counter: 00000011
Jan 12 10:07:22 wrpserver kernel: hda5's event counter: 00000011
Jan 12 10:07:22 wrpserver kernel: md: md0: raid array is not clean -- starting background reconstruction
Jan 12 10:07:22 wrpserver kernel: request_module[md-personality-3]: Root fs not mounted
Jan 12 10:07:22 wrpserver kernel: do_md_run() returned -22
Jan 12 10:07:22 wrpserver kernel: unbind
Jan 12 10:07:22 wrpserver kernel: export_rdev(hdb5)
Jan 12 10:07:22 wrpserver kernel: unbind
Jan 12 10:07:22 wrpserver kernel: export_rdev(hda5)
Jan 12 10:07:22 wrpserver kernel: md0 stopped.
Jan 12 10:07:22 wrpserver kernel: considering hdb1 ...
Jan 12 10:07:22 wrpserver kernel:   adding hdb1 ...
Jan 12 10:07:22 wrpserver kernel:   adding hda1 ...
Jan 12 10:07:22 wrpserver kernel: created md2
Jan 12 10:07:22 wrpserver kernel: bind
Jan 12 10:07:22 wrpserver kernel: bind
Jan 12 10:07:22 wrpserver kernel: running:
Jan 12 10:07:22 wrpserver kernel: now!
Jan 12 10:07:22 wrpserver kernel: hdb1's event counter: 00000011
Jan 12 10:07:22 wrpserver kernel: hda1's event counter: 00000011
Jan 12 10:07:22 wrpserver kernel: md: md2: raid array is not clean -- starting background reconstruction
Jan 12 10:07:22 wrpserver kernel: request_module[md-personality-3]: Root fs not mounted
Jan 12 10:07:22 wrpserver kernel: do_md_run() returned -22
Jan 12 10:07:22 wrpserver kernel: unbind
Jan 12 10:07:22 wrpserver kernel: export_rdev(hdb1)
Jan 12 10:07:22 wrpserver kernel: unbind
Jan 12 10:07:22 wrpserver kernel: export_rdev(hda1)
Jan 12 10:07:22 wrpserver kernel: md2 stopped.
Jan 12 10:07:22 wrpserver kernel: ... autorun DONE.
Jan 12 10:07:22 wrpserver kernel: apm: BIOS version 1.2 Flags 0x03 (Driver version 1.13)
Jan 12 10:07:22 wrpserver kernel: VFS: Mounted root (ext2 filesystem).
Jan 12 10:07:22 wrpserver kernel: raid1 personality registered
Jan 12 10:07:22 wrpserver kernel: autodetecting RAID arrays
Jan 12 10:07:22 wrpserver kernel: (read) hda1's sb offset: 264960 [events: 00000011]
Jan 12 10:07:22 wrpserver kernel: (read) hda5's sb offset: 15936 [events: 00000011]
Jan 12 10:07:22 wrpserver kernel: (read) hda6's sb offset: 38796864 [events: 00000011]
Jan 12 10:07:22 wrpserver kernel: (read) hdb1's sb offset: 264960 [events: 00000011]
Jan 12 10:07:22 wrpserver kernel: (read) hdb5's sb offset: 15936 [events: 00000011]
Jan 12 10:07:22 wrpserver kernel: (read) hdb6's sb offset: 38796864 [events: 00000011]
Jan 12 10:07:22 wrpserver kernel: autorun ...
Jan 12 10:07:22 wrpserver kernel: considering hdb6 ...
Jan 12 10:07:22 wrpserver kernel:   adding hdb6 ...
Jan 12 10:07:22 wrpserver kernel:   adding hda6 ...
Jan 12 10:07:22 wrpserver kernel: created md1
Jan 12 10:07:22 wrpserver kernel: bind
Jan 12 10:07:22 wrpserver kernel: bind
Jan 12 10:07:22 wrpserver kernel: running:
Jan 12 10:07:22 wrpserver kernel: now!
Jan 12 10:07:22 wrpserver kernel: hdb6's event counter: 00000011
Jan 12 10:07:22 wrpserver kernel: hda6's event counter: 00000011
Jan 12 10:07:22 wrpserver kernel: md: md1: raid array is not clean -- starting background reconstruction
Jan 12 10:07:22 wrpserver kernel: md1: max total readahead window set to 128k
Jan 12 10:07:22 wrpserver kernel: md1: 1 data-disks, max readahead per data-disk: 128k
Jan 12 10:07:22 wrpserver kernel: raid1: device hdb6 operational as mirror 1
Jan 12 10:07:22 wrpserver kernel: raid1: device hda6 operational as mirror 0
Jan 12 10:07:22 wrpserver kernel: raid1: raid set md1 not clean; reconstructing mirrors
Jan 12 10:07:22 wrpserver kernel: raid1: raid set md1 active with 2 out of 2 mirrors
Jan 12 10:07:22 wrpserver kernel: md: updating md1 RAID superblock on device
Jan 12 10:07:22 wrpserver kernel: hdb6 [events: 00000012](write) hdb6's sb offset: 38796864
Jan 12 10:07:22 wrpserver kernel: md: syncing RAID array md1
Jan 12 10:07:22 wrpserver kernel: md: minimum _guaranteed_ reconstruction speed: 100 KB/sec.
Jan 12 10:07:22 wrpserver kernel: md: using maximum available idle IO bandwith for reconstruction.
Jan 12 10:07:22 wrpserver kernel: md: using 128k window.
Jan 12 10:07:22 wrpserver kernel: hda6 [events: 00000012](write) hda6's sb offset: 38796864
Jan 12 10:07:22 wrpserver kernel: .
Jan 12 10:07:22 wrpserver kernel: considering hdb5 ...
Jan 12 10:07:22 wrpserver kernel:   adding hdb5 ...
Jan 12 10:07:22 wrpserver kernel:   adding hda5 ...
Jan 12 10:07:22 wrpserver kernel: created md0
Jan 12 10:07:22 wrpserver kernel: bind
Jan 12 10:07:22 wrpserver kernel: bind
Jan 12 10:07:22 wrpserver kernel: running:
Jan 12 10:07:22 wrpserver kernel: now!
Jan 12 10:07:22 wrpserver kernel: hdb5's event counter: 00000011
Jan 12 10:07:22 wrpserver kernel: hda5's event counter: 00000011
Jan 12 10:07:22 wrpserver kernel: md: md0: raid array is not clean -- starting background reconstruction
Jan 12 10:07:22 wrpserver kernel: md0: max total readahead window set to 128k
Jan 12 10:07:22 wrpserver kernel: md0: 1 data-disks, max readahead per data-disk: 128k
Jan 12 10:07:22 wrpserver kernel: raid1: device hdb5 operational as mirror 1
Jan 12 10:07:22 wrpserver kernel: raid1: device hda5 operational as mirror 0
Jan 12 10:07:22 wrpserver kernel: raid1: raid set md0 not clean; reconstructing mirrors
Jan 12 10:07:22 wrpserver kernel: raid1: raid set md0 active with 2 out of 2 mirrors
Jan 12 10:07:22 wrpserver kernel: md: updating md0 RAID superblock on device
Jan 12 10:07:22 wrpserver kernel: hdb5 [events: 00000012](write) hdb5's sb offset: 15936
Jan 12 10:07:22 wrpserver kernel: md: serializing resync, md0 has overlapping physical units with md1!
Jan 12 10:07:22 wrpserver kernel: hda5 [events: 00000012](write) hda5's sb offset: 15936
Jan 12 10:07:22 wrpserver kernel: .
Jan 12 10:07:22 wrpserver kernel: considering hdb1 ...
Jan 12 10:07:22 wrpserver kernel:   adding hdb1 ...
Jan 12 10:07:22 wrpserver kernel:   adding hda1 ...
Jan 12 10:07:22 wrpserver kernel: created md2
Jan 12 10:07:22 wrpserver kernel: bind
Jan 12 10:07:22 wrpserver kernel: bind
Jan 12 10:07:22 wrpserver kernel: running:
Jan 12 10:07:22 wrpserver kernel: now!
Jan 12 10:07:22 wrpserver kernel: hdb1's event counter: 00000011
Jan 12 10:07:22 wrpserver kernel: hda1's event counter: 00000011
Jan 12 10:07:22 wrpserver kernel: md: md2: raid array is not clean -- starting background reconstruction
Jan 12 10:07:22 wrpserver kernel: md2: max total readahead window set to 128k
Jan 12 10:07:22 wrpserver kernel: md2: 1 data-disks, max readahead per data-disk: 128k
Jan 12 10:07:22 wrpserver kernel: raid1: device hdb1 operational as mirror 1
Jan 12 10:07:22 wrpserver kernel: raid1: device hda1 operational as mirror 0
Jan 12 10:07:22 wrpserver kernel: raid1: raid set md2 not clean; reconstructing mirrors
Jan 12 10:07:22 wrpserver kernel: raid1: raid set md2 active with 2 out of 2 mirrors
Jan 12 10:07:22 wrpserver kernel: md: updating md2 RAID superblock on device
Jan 12 10:07:22 wrpserver kernel: hdb1 [events: 00000012](write) hdb1's sb offset: 264960
Jan 12 10:07:22 wrpserver kernel: md: serializing resync, md2 has overlapping physical units with md1!
Jan 12 10:07:22 wrpserver kernel: hda1 [events: 00000012](write) hda1's sb offset: 264960
Jan 12 10:07:22 wrpserver kernel: .
Jan 12 10:07:22 wrpserver kernel: ... autorun DONE.
Jan 12 10:07:22 wrpserver kernel: VFS: Mounted root (ext2 filesystem) readonly.
Jan 12 10:07:22 wrpserver kernel: change_root: old root has d_count=1

Dan

[Welsh]

You shouldn't powerdown your server. The system will think you have power problem. So when it turn back on, it will resync the mirror disk. If your do couple of time, your HD will blow.

If you need to shutdown server, at the command
#shutdown -h now

[Dan Williams]

Hi,
I neglected to mention that I use the proper shutdown command
shutdown -h now.
Dan

[Kenneth Wellin]

Hi Dan,

What brand are your disks? I've seen the same problem with some Seagate disks.

Regards,

Kenneth

[Dan Williams]

Yes, they are seagate,
Dan

[Kenneth Wellin]

Hi again,

I thought they were....according to Seagate (I don't have the link at hand right now) their newer ATA drives will not run in a raid configuration...don't ask me what the technical problem is. I haven't looked into the problem in detail but noted that this is what Seagate states.

Regards,

Kenneth

[Dan Williams]

Thanks for the heads up, that sucks, Seagate just lost my business, I use about 20 hard drives a month.
Dan

[Dan Williams]

Just a note about this situation,
I contacted Seagate regarding this issue, and like you said they are aware of it.
The thing they did offer me was this, to swap the drives out that I have, for drives that they say are optimized or the raid.
Dan

[Ari]

Just on an aside, the hard drives should be set up as the masters on both the primary AND secondary channels for software raid.

Although it tends to work just fine in a master/slave config on one channel, software raid tends to perform better when 'duplexed' and mirrored rather than just mirrored.

Ari