LVM volume not accesible after accidental drive removal, now LV Status is suspended

I have a home server with a pool of 3 drives, and the OS installed in a thumb drive, this thumb drive had physical errors and I had to reinstall the OS, the LVM pool was working well after thumb drive crash, but I accidentally selected one of the drives to manually install grub and things went wrong.

I don't know how/why this drive got detached from the pool and LV stopped working. I've trying for months and have followed different guides and instructions, I partially restored the pool, but it is still not working.

The only clue that I have now for the reason of the error is:

lvdisplay -m --- Logical volume --- LV Path /dev/DATA/shared LV Name shared VG Name DATA LV UUID WfdvYe-pga2-dyUn-d2gd-L0xS-KBee-x2PPzP LV Write Access read/write LV Creation host, time , LV Status suspended # open 0 LV Size 6.37 TiB Current LE 1669260 Segments 3 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 252:0 --- Segments --- Logical extent 0 to 476931: Type linear Physical volume /dev/sdb1 Physical extents 0 to 476931 Logical extent 476932 to 953863: Type linear Physical volume /dev/sda1 Physical extents 0 to 476931 Logical extent 953864 to 1669259: Type linear Physical volume /dev/sdc Physical extents 0 to 715395

 pvdisplay --- Physical volume --- PV Name /dev/sdb1 VG Name DATA PV Size 1.82 TiB / not usable 1.09 MiB Allocatable yes (but full) PE Size 4.00 MiB Total PE 476932 Free PE 0 Allocated PE 476932 PV UUID kcSzc0-tGcn-gA33-w27c-J4MP-Jekh-btQdKr --- Physical volume --- PV Name /dev/sda1 VG Name DATA PV Size 1.82 TiB / not usable 1.09 MiB Allocatable yes (but full) PE Size 4.00 MiB Total PE 476932 Free PE 0 Allocated PE 476932 PV UUID 0gZJv2-iRwc-r5Z3-VPEb-HUzc-eyAO-LIu2zt --- Physical volume --- PV Name /dev/sdc VG Name DATA PV Size 2.73 TiB / not usable 472.00 KiB Allocatable yes PE Size 4.00 MiB Total PE 715397 Free PE 1 Allocated PE 715396 PV UUID YicF5n-Xb8y-f3GL-kFuM-stxZ-gpmv-0E8mie "/dev/sdd5" is a new physical volume of "1.82 TiB" --- NEW Physical volume --- PV Name /dev/sdd5 VG Name PV Size 1.82 TiB Allocatable NO PE Size 0 Total PE 0 Free PE 0 Allocated PE 0 PV UUID LmJ6VD-LHS5-0BVb-mqzB-gbbc-8dRX-M74APj

Note: The 4th drive sdd5 was never part of the LVM DATA pool, and have no data.

sudo fdisk -l
Disk /dev/sda: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xd53b391a Device Boot Start End Blocks Id System
/dev/sda1 1 3907029167 1953514583+ 8e Linux LVM
Disk /dev/sdc: 3000.6 GB, 3000592982016 bytes
255 heads, 63 sectors/track, 364801 cylinders, total 5860533168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x6b4f5902 Device Boot Start End Blocks Id System
Disk /dev/sdd: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00081354 Device Boot Start End Blocks Id System
/dev/sdd1 2048 499711 248832 83 Linux
/dev/sdd2 501758 3907028991 1953263617 5 Extended
Partition 2 does not start on physical sector boundary.
/dev/sdd5 501760 3907028991 1953263616 8e Linux LVM
Disk /dev/sdb: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x19e4252c Device Boot Start End Blocks Id System
/dev/sdb1 2048 3902855167 1951426560 8e Linux LVM
/dev/sdb2 3902857214 3907028991 2085889 5 Extended
/dev/sdb5 3902857216 3907028991 2085888 8e Linux LVM
Disk /dev/sde: 16.1 GB, 16097738752 bytes
64 heads, 32 sectors/track, 15352 cylinders, total 31440896 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000cad44 Device Boot Start End Blocks Id System
/dev/sde1 2046 6125567 3061761 5 Extended
/dev/sde2 * 6125568 31438847 12656640 83 Linux
/dev/sde5 2048 1953791 975872 83 Linux
/dev/sde6 1955840 6125567 2084864 82 Linux swap / Solaris

*Note: The 16GB drive is the thumbdrive where OS is installed

updated result of fdisk -lon Jan-20th after execute @psusi suggestions

Disk /dev/sda: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xd53b391a Device Boot Start End Blocks Id System
/dev/sda1 1 3907029167 1953514583+ 8e Linux LVM
Disk /dev/sdb: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x19e4252c Device Boot Start End Blocks Id System
/dev/sdb1 2048 3907028991 1953513472 83 Linux
Disk /dev/sdc: 3000.6 GB, 3000592982016 bytes
255 heads, 63 sectors/track, 364801 cylinders, total 5860533168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x6b4f5902 Device Boot Start End Blocks Id System
Disk /dev/sdd: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00081354 Device Boot Start End Blocks Id System
/dev/sdd1 2048 499711 248832 83 Linux
/dev/sdd2 501758 3907028991 1953263617 5 Extended
Partition 2 does not start on physical sector boundary.
/dev/sdd5 501760 3907028991 1953263616 8e Linux LVM
Disk /dev/sde: 16.1 GB, 16097738752 bytes
64 heads, 32 sectors/track, 15352 cylinders, total 31440896 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000cad44 Device Boot Start End Blocks Id System
/dev/sde1 2046 6125567 3061761 5 Extended
/dev/sde2 * 6125568 31438847 12656640 83 Linux
/dev/sde5 2048 1953791 975872 83 Linux
/dev/sde6 1955840 6125567 2084864 82 Linux swap / Solaris

1 Answer

You seem to have clobbered your partition table on sdb. According to lvm, it originally had a single partition occupying the whole disk, and now sdb1 is too small, and much of the space is now unallocated, or allocated to another partition that says it is used by lvm, but lvm says it isn't. You should be able to use parted to blow away all of the partitions on the disk, and recreate the single whole disk partition to repair this.

sudo parted /dev/sdb
rm 5
rm 2
rm 1
mkpart primary ext2 2048s 100%
quit