I'm having a bizarre issue and all the pages that I've searched don't quite match my problem.
Basically, I can't access my small raid 1 array that comprises two 1TB WD Red disks (sdb and sdc in the fdisk check below).
Here are the usual checks (if I miss one please let me know):
fdisk You'll probably need to scroll the box below to see it all, also I have no idea what all the loops are...
$> sudo fdisk -l
Disk /dev/loop0: 140.7 MiB, 147496960 bytes, 288080 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/loop1: 13 MiB, 13619200 bytes, 26600 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/loop2: 3.7 MiB, 3878912 bytes, 7576 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/loop3: 91 MiB, 95408128 bytes, 186344 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/loop4: 2.3 MiB, 2355200 bytes, 4600 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/loop5: 14.5 MiB, 15208448 bytes, 29704 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/loop6: 34.6 MiB, 36216832 bytes, 70736 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/loop7: 88.5 MiB, 92778496 bytes, 181208 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/sda: 223.6 GiB, 240057409536 bytes, 468862128 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 28352AE2-4322-4627-9BE2-DFBEDBAFF1BF
Device Start End Sectors Size Type
/dev/sda1 2048 1050623 1048576 512M EFI System
/dev/sda2 1050624 468860927 467810304 223.1G Linux filesystem
GPT PMBR size mismatch (1953519879 != 1953525167) will be corrected by w(rite).
Disk /dev/sdb: 931.5 GiB, 1000204886016 bytes, 1953525168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 84416481-C343-40E7-A8EB-3680B26FEF19
Device Start End Sectors Size Type
/dev/sdb1 2048 1953519615 1953517568 931.5G Linux filesystem
GPT PMBR size mismatch (1953519879 != 1953525167) will be corrected by w(rite).
Disk /dev/sdc: 931.5 GiB, 1000204886016 bytes, 1953525168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 84416481-C343-40E7-A8EB-3680B26FEF19
Device Start End Sectors Size Type
/dev/sdc1 2048 1953519615 1953517568 931.5G Linux filesystem
Disk /dev/sdd: 119.2 GiB, 128035676160 bytes, 250069680 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 4A8AA6CA-61E4-43A2-B616-EAD50214A106
Device Start End Sectors Size Type
/dev/sdd1 2048 999423 997376 487M EFI System
/dev/sdd2 999424 17000447 16001024 7.6G Linux swap
GPT PMBR size mismatch (1953519879 != 1953519615) will be corrected by w(rite).
Disk /dev/md126: 931.5 GiB, 1000202043392 bytes, 1953519616 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: dos
Disk identifier: 0x00000000
Device Boot Start End Sectors Size Id Type
/dev/md126p1 1 1953519879 1953519879 931.5G ee GPT
Partition 1 does not start on physical sector boundary.mdstat
$> cat /proc/mdstat
Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10]
md126 : active (auto-read-only) raid1 sdb[1] sdc[0] 976759808 blocks super external:/md127/0 [2/2] [UU]
md127 : inactive sdc[1](S) sdb[0](S) 5552 blocks super external:imsm
unused devices: <none>mdadm.conf
$> sudo cat /etc/mdadm/mdadm.conf
# mdadm.conf
#
# !NB! Run update-initramfs -u after updating this file.
# !NB! This will ensure that initramfs has an uptodate copy.
#
# Please refer to mdadm.conf(5) for information about this file.
#
# by default (built-in), scan all partitions (/proc/partitions) and all
# containers for MD superblocks. alternatively, specify devices to scan, using
# wildcards if desired.
#DEVICE partitions containers
# automatically tag new arrays as belonging to the local system
HOMEHOST <system>
# instruct the monitoring daemon where to send mail alerts
MAILADDR root
# definitions of existing MD arrays
ARRAY metadata=imsm UUID=fe0bb25b:d021df67:4d7fe09f:a30a6e08
ARRAY /dev/md/Volume1 container=fe0bb25b:d021df67:4d7fe09f:a30a6e08 member=0 UUID=3d2e36ef:e2314e97:11933fe5:f38135b1
ARRAY /dev/md/0 metadata=1.2 UUID=7d7acef8:cde50639:d9c04370:fbf727c6 name=chugster:0
# This configuration was auto-generated on Wed, 07 Aug 2019 00:10:23 +0100 by mkconfmdadm -E /dev/sdb
$> sudo mdadm -E /dev/sdb
/dev/sdb: Magic : Intel Raid ISM Cfg Sig. Version : 1.1.00 Orig Family : c1155891 Family : c1155891 Generation : 000000d2 Attributes : All supported UUID : fe0bb25b:d021df67:4d7fe09f:a30a6e08 Checksum : 03482b05 correct MPB Sectors : 1 Disks : 2 RAID Devices : 1 Disk00 Serial : WD-WXV1E74D9L1F State : active Id : 00000002 Usable Size : 1953519616 (931.51 GiB 1000.20 GB)
[Volume1]: UUID : 3d2e36ef:e2314e97:11933fe5:f38135b1 RAID Level : 1 Members : 2 Slots : [UU] Failed disk : none This Slot : 0 Sector Size : 512 Array Size : 1953519616 (931.51 GiB 1000.20 GB) Per Dev Size : 1953519880 (931.51 GiB 1000.20 GB) Sector Offset : 0 Num Stripes : 7630936 Chunk Size : 64 KiB Reserved : 0 Migrate State : idle Map State : normal Dirty State : clean RWH Policy : off Disk01 Serial : WD-WXV1E747PDZD State : active Id : 00000003 Usable Size : 1953519616 (931.51 GiB 1000.20 GB)mdadm -E /dev/sdc
$> sudo mdadm -E /dev/sdc
/dev/sdc: Magic : Intel Raid ISM Cfg Sig. Version : 1.1.00 Orig Family : c1155891 Family : c1155891 Generation : 000000d2 Attributes : All supported UUID : fe0bb25b:d021df67:4d7fe09f:a30a6e08 Checksum : 03482b05 correct MPB Sectors : 1 Disks : 2 RAID Devices : 1 Disk01 Serial : WD-WXV1E747PDZD State : active Id : 00000003 Usable Size : 1953519616 (931.51 GiB 1000.20 GB)
[Volume1]: UUID : 3d2e36ef:e2314e97:11933fe5:f38135b1 RAID Level : 1 Members : 2 Slots : [UU] Failed disk : none This Slot : 1 Sector Size : 512 Array Size : 1953519616 (931.51 GiB 1000.20 GB) Per Dev Size : 1953519880 (931.51 GiB 1000.20 GB) Sector Offset : 0 Num Stripes : 7630936 Chunk Size : 64 KiB Reserved : 0 Migrate State : idle Map State : normal Dirty State : clean RWH Policy : off Disk00 Serial : WD-WXV1E74D9L1F State : active Id : 00000002 Usable Size : 1953519616 (931.51 GiB 1000.20 GB)mdadm detail scan
$> sudo mdadm --detail --scan
ARRAY /dev/md/imsm0 metadata=imsm UUID=fe0bb25b:d021df67:4d7fe09f:a30a6e08
ARRAY /dev/md/Volume1 container=/dev/md/imsm0 member=0 UUID=3d2e36ef:e2314e97:11933fe5:f38135b1So as a little background, sdc failed with a missing superblock, but I read something somewhere that allowed me to "patch" sdc using the uuid of sdb. So, now "mdadm -E /dev/sdc" shows information rather than saying the superblock is missing. I'm not sure whether what I've done is the right thing to do.
If I try and assemble the raid it says that /dev/md127 doesn't exist in mdadm.conf. If I try to regenerate the mdadm.conf, then it doesn't add /dev/md127.
Basically, I have no idea how to reassemble the raid, or why it failed in the first place. Disk utility says that both of the disks are ok.
If all else fails, can I remove md127 from the array, mount the array with one disk (md126), remove all partitions on what is currently sdc and then add it back to the array?
Your help is very much appreciated.
Andrew
Edit 1It may also help to know that all this happened when I reinstalled the OS - to go from 14.4 to 18.4.
Edit 2
I have just noticed that I can examine sdb1 but not sdc1:
$> sudo mdadm --examine /dev/sdb1
/dev/sdb1: Magic : a92b4efc Version : 1.2 Feature Map : 0x0 Array UUID : 7d7acef8:cde50639:d9c04370:fbf727c6 Name : chugster:0 (local to host chugster) Creation Time : Tue Aug 6 23:38:40 2019 Raid Level : linear Raid Devices : 2 Avail Dev Size : 1953253376 (931.38 GiB 1000.07 GB) Used Dev Size : 0 Data Offset : 264192 sectors Super Offset : 8 sectors Unused Space : before=264112 sectors, after=0 sectors State : clean Device UUID : beeda35f:a7c7f529:33e2c551:4bc87bfc Update Time : Tue Aug 6 23:38:40 2019 Bad Block Log : 512 entries available at offset 8 sectors Checksum : f2302886 - correct Events : 0 Rounding : 0K Device Role : Active device 0 Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
$> sudo mdadm --examine /dev/sdc1
mdadm: cannot open /dev/sdc1: No such file or directoryI think something is seriously messed up with /dev/sdc. I'm not sure how to remove /dev/sdc from the array, given that /dev/sdc1 doesn't exist. Also I'm assuming that I remove it from md127, but that doesn't feel right, perhaps I should be trying to remove it from /dev/md/Volume1? The other thing that is concerning me is that /proc/mdstat suggests that the superblock for md126 is on md127, or am I reading that wrong?
Edit 3Made a correction
21 Answer
I really hate fake raid. It's a HW feature that users gravitate towards because they equate HW == better where it all it really does is complicate your storage setup and make it more brittle. The only time fake raid matters is when you want to dual boot and share the same volume between multiple operating systems. Otherwise run away from it like it's the plague.
The thing that really stands out to me is that you have partitions that are tagged with a filesystem which appear to span the entire disk, yet you assign the entire block device to the RAID. That's how you corrupt data. It could have gotten mounted at some point or fsck was run on it at boot which "repaired it" and that's when your superblock got corrupted.
It's fine to partition disks that are assigned to a RAID, just make sure you tag them as type FD (linux raid autodetect) so these sorts of collisions don't happen. The file system goes on the MD device.
At this point. I would boot from USB disk. Bring the array online. Forcibly remove "sdc", dd the entire thing with zeros, and then add it back to the array for a full resync.
Or just start over. You said you have a backup. disassemble the array, zero the superblocks or just dd if=/dev/zero of=/... and this time just use md, no fake raid. I advise that you create a single partition on each disk that spans all the space and tag it as FD so this doesn't happen again.
Good luck.
A side note concerning fake raid.
"The recommended software RAID implementation in Linux* is the open source MD RAID package. Intel has enhanced MD RAID to support RST metadata and OROM and it is validated and supported by Intel for server platforms. There is a growing interest at OEMs in having Intel extend the validation and support for RST on mobile, desktop and workstation platforms in a Windows and Linux dual-boot environment"
Which reads "HW vendors are lazy and don't want to deal with operating systems so they want to prebuild systems 'with a RAID' and pretend they added value for the customer"
