Re: Problem with partially activate logical volume

fsck might be able to fix it enough that debugfs and/or mount works,
but it may also eliminate all the data, but since you have a clone and
no data it is probably worth a shot.

fsck -f -y   will either get you some data or not and make you have to
clone from the copy if you have something else to  try.

Did you try the "-c" option on debug fs?

On Thu, Aug 4, 2022 at 2:07 AM Ken Bass <[email protected]> wrote:
>
>
> That's pretty much it. Whenever any app attempts to read a block from the missing drive, I get the "Buffer I/O error" message. So, even though my recovery apps can scan the LV, marking blocks on the last drive as missing/unknown/etc., they can't display any recovered data - which I know does exist. Looking at raw data from the apps' scans, I can see directory entries, as well as files. I'm sure the inodes and bitmaps are still there for some of these, I just can't really reverse engineer and follow them through. But isn't that what the apps are supposed to do?
>
> As for debugfs: pretty much the same issue: in order to use it, I need to open the fs. But that (in debugfs) fails as well. So it can't help much. Unless I'm missing something about debugfs.
>
> The one thing I haven't tried is to use vgreduce to remove the missing PV; but that will also remove the LV as well, which is why I haven't tried it yet.
>
> Sorry I haven't replied sooner, but it takes a long time (days) to clone, then scan 16Tb...
>
> So, please any suggestions are greatly appreciated, as well as needed.
>
> ken
>
> (I know: No backup; got burned; it hurts; and I will now always have backups. 'Nuf said.)
>
>
> On Thu, Jul 28, 2022 at 3:12 AM Roger James <[email protected]> wrote:
>>
>> The procedure outlined should at least get you back to a state where the lv is consistent but with blank sectors where the data is missing. I would suggest using dd to make a backup partition image. Then you can either work on that or the original to mend the fs.
>>
>> On 27 July 2022 11:50:07 Roger Heflin <[email protected]> wrote:
>>
>>> I don't believe that is going to work.
>>>
>>> His issue is that the filesystem is refusing to work because of the
>>> missing data.
>>>
>>> man debugfs
>>>
>>> It will let you manually look at the metadata and structures of the
>>> ext2/3/4 fs.  You will likely need to use the "-c" option.
>>>
>>> It will be very manual and you should probably read up on the fs
>>> structure a bit.
>>>
>>> A data recovery company could get most of the data back, but they
>>> charge 5k-10k per TB, so likely close to 100k US$.
>>>
>>> And the issues will be that 1/3 of the metadata was on the missing
>>> disk, and some of the data was on the missing disk.
>>>
>>> I was able to do debugfs /dev/sda2  (my /boot) and do an ls and list
>>> out the files and then do a dump <filename> /tmp/junk.out and copy out
>>> that file.
>>>
>>> So the issue will be writing up a script to do lses and find all of
>>> the files and dump all of the files to someplace else.
>>>
>>> On Wed, Jul 27, 2022 at 2:39 AM Roger James <[email protected]> wrote:
>>>>
>>>>
>>>> Try https://www.linuxsysadmins.com/recover-a-deleted-physical-volume/?amp
>>>>
>>>> On 26 July 2022 09:16:32 Ken Bass <[email protected]> wrote:
>>>>>
>>>>>
>>>>> (fwiw: I am new to this list, so please bear with me.)
>>>>>
>>>>> Background: I have a very large (20TB) logical volume consisting of 3 drives. One of those drives unexpectedloy died (isn't that always the case :-)). The drive that failed happened to be the last PV. So I am assuming that there is still 2/3 of the data still intact and, to some extent, recoverable. Although, apparently the ext4 fs is not recognised.
>>>>>
>>>>> I activated the LV partially (via -P). But running any utility on that (eg: dumpe2fs, e2fsck, ...) I get many of these  in dmesg:
>>>>>
>>>>> "Buffer I/O error on dev dm-0, logical block xxxxxxx, async page read."  The thing is, the xxxxxxx block is on the missing drive/pv.
>>>>>
>>>>> I have also tried some recovery software, but eventually get these same messages, and the data recovered is not really useful.
>>>>>
>>>>> Please help! How can I get passed that dmesg error, and move on. 14TB recovered is better than 0.
>>>>>
>>>>> TIA
>>>>> ken
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> linux-lvm mailing list
>>>>> [email protected]
>>>>> https://listman.redhat.com/mailman/listinfo/linux-lvm
>>>>> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>>>>>
>>>>
>>>> _______________________________________________
>>>> linux-lvm mailing list
>>>> [email protected]
>>>> https://listman.redhat.com/mailman/listinfo/linux-lvm
>>>> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>>>
>>>
>>> _______________________________________________
>>> linux-lvm mailing list
>>> [email protected]
>>> https://listman.redhat.com/mailman/listinfo/linux-lvm
>>> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>>
>>
>> _______________________________________________
>> linux-lvm mailing list
>> [email protected]
>> https://listman.redhat.com/mailman/listinfo/linux-lvm
>> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>
> _______________________________________________
> linux-lvm mailing list
> [email protected]
> https://listman.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

_______________________________________________
linux-lvm mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/