Problem with lost configuration in LSM

Hello all who's left on this list,

I haven't had any troubles with my systems for years now so I haven't been
active on this list, but now I have a problem that's becoming urgent.

What happened is that I had a failed disk that I replaced and I was only
able to reestablish part of the mirroring that was there before.

The machine is an alpha 1200 and I have drives in all 7 bays, with the scsi
bus split and two controllers, so I have 4 drives on one controller and 3 on
the other.

I have my disks named as dsk0..dsk6 with dsk0, dsk1, dsk2 and dsk6 on the
first scsi controller on the 4 top bays, and then dsk3, dsk4 and dsk5 on the
other controller.

The dsk0 is the boot disk, with root, swap, usr and var in dsk0a, dsk0b,
dsk0d and dsk0f respectively. That disk was encapsulated and it is mirrored
on dsk3.

I was thinking I had several configuration copies, and I pretty sure I did,
but now after replacing the dsk3 that had failed, after trying to
reestablish all the mirroring, something must've happened and there are no
longer any configuration copies found.

To replace the failed disk, I swapped it physically and used hwmgr to scan
the new disk and then I swapped the name dsk3 with the newly created dsk10
with dsfmgr so I would have that new drive in the same location and with the
same name as the previous one.

So far, this worked fine. Then I reestablished the mirroring on the root
volume using voldiskadm with option 4 first to remove the failed disk and
then re-add it with the option 5 which worked fine.
I did the same thing with the swap volume and that also went fine.
However when I tried doing this with the usr and var volumes, I was getting
some errors such as those:

lsm:volplex: ERROR: Volume usrvol, plex usr-pl-02, block 0: Plex write:
    Error: Write failure
lsm:volplex: ERROR: sd usr-sd-02 in plex usr-pl-02 failed during attach
lsm:volplex: ERROR: I/O error on plex usr-pl-02, not attached to volume
usrvol
lsm:volplex: ERROR:  DRL write to volume usrvol failed:
    No such device or address
lsm:volplex: ERROR:  Write of active log to volume usrvol failed:
    No such device or address
lsm:volplex: ERROR: I/O error on plex usrvol-01, not attached to volume
usrvol
lsm:volplex: ERROR: Volume varvol, plex var-pl-02, block 0: Plex write:
    Error: Write failure
lsm:volplex: ERROR: sd var-sd-02 in plex var-pl-02 failed during attach
lsm:volplex: ERROR: I/O error on plex var-pl-02, not attached to volume
varvol
lsm:volplex: ERROR:  DRL write to volume varvol failed:
    No such device or address
lsm:volplex: ERROR:  Write of active log to volume varvol failed:
    No such device or address
lsm:volplex: ERROR: I/O error on plex varvol-01, not attached to volume
varvol

I think it's mostly because of the configuration copies and the fpa logging.
I had to reverse to the previous state on both and then finally I was able
to get the usr volume back to normal as well, but was left with that var
volume that still wouldn't want to come back to mirroring.

At some point I was no longer able to get any disk or other info from the
commands like voldisk... Volprint... etc... And I thought the vold daemon
had hung or something like that, so I tried issuing a "vold -k", which
didn't change anything. I tried stopping the vold using voldctl to be able
to restart it manually but it starts and has no configuration copies to
find.

Trying again to use voldisk, I get this:

voldisk list
lsm:voldisk: ERROR: Cannot get records from vold: Record not in disk group

So vold is running, responding, but has no configuration copies to find.

I only have one disk group, rootdg and besides the mirror on dsk0/dsk3 for
the root/swap/usr/var boot disk, I have dsk1/dsk4 and dsk2/dsk5 as mirrors
for other data volumes and some space reserved on each pairs for an extra
swap volume (3 swap volumes total).

For now everything still works on the machine but with an lsm configuration
that's broken I could get in serious trouble if something else happened now.

I was thinking I should try to add more configuration copies, but since
nothing works right now I think it wouldn't work. And I'm not certain of the
exact commands to use to manually add configuration copies and specify where
to put them.

Are there lsm gurus left on this list who could give me a hint?

How can I get out of this bad situation?


Thanks all,


-- 
Didier Godefroy
mailto:[email protected]