Re: Bug?: Minor numbers in true evms volumes are *not* persitant!

Rainer Krienke <[email protected]> wrote on 01/02/2007 12:25:38 PM:

> Hello,

Hello, Rainer

> I am running two files servers with SuSE SLES10 (kernel 2.6.16.27, evms
> 2.5.5). The (NFS) file servers have disks (5TB hardware raids) attached
and
> on each raid I manage the storage available using evmsgui. I create EVMS
> volumes (true EVMS vols, based on the LVM2 region manager). This
basically
> works fine.

> However lately I had to reboot the file servers and after this reboot
some NFS
> clients which had not been rebooted showed a strange behavior. They tried
to
> NFS mount (or still had mounted) a volume say
server1:/export/evms_volume1
> to /import/volume1 from the rebooted file server. The mount was ok but
the
> strange thing was that the mounted volume on the NFS client did not
contain
> what server1:/export/evms_volume1 contains but instead the contents of
say
> server1:/export/volume3. So the mount was sucessful but the contens was
from
> another NFS volume exported by the same fileserver.  This really turns
out to
> be a nightmare.

> I tracked this bug down to the fact that after a reboot "sometimes" the
minor
> device numbers of the NFS exported EVMS volumes in /dev/evms/ simply
change.
> Since the major/minor numbers are used in NFS to identify a NFS volume,
NFS
> clients suddenly mount the wrong one.

> Today I set up a test system and the effect was there again, eg after I
had
> created some EVMS volumes based on the LVM2 region manager using evmsgui
and
> then rebooted the machine. One or two of the 5 EVMS volumes I previously
had
> created suddenly had *different* minor device numbers than they had
before
> the reboot.

> Is this desired behaviour in EVMS?

No, it's not desired, as proved by the problem you are seeing. :)  But
neither is it unexpected.  EVMS uses Device Mapper to implement many of its
devices in the kernel.  EVMS does not make any effort to preserve minor
numbers for Device Mapper devices, although it could.  (We'd have to come
up with a design to adequately handle what happens when EVMS tries to
create the Device Mapper device for the volume and the minor number is
already in use.)

> I cannot imagine but since it happens on
> several machines I really ask myself. If its not desired does anyone have
an
> idea what might happen or how the minor numbers of existing EVMS volumes
are
> assigned and when they might be reassigned?

Device Mapper assigns a minor number to the device when it is created.
Assuming the disk configuration doesn't change between reboots, EVMS will
discover and create the devices in the same order and Device Mapper will
assign the minor numbers in the same order.  If something causes EVMS to
discover and create the devices in a different order, then the Device
Mapper devices may not get the same minor numbers as they did on the
previous system boot.

> This might help me to find out
> whats wrong. On the machine at boot time first evms_activate is started
by
> the boot script boot.evms then later boot.lvm is started. So evms scannes
all
> disks and then a little later lvm does the same again. Could this be the
> cause of this strange trouble?

Yes, this could be a problem.  If you are using EVMS to discover the LVM
devices, then you should not let the native LVM discover the devices.  The
native LVM will find the same regions/LVM volumes and will also use Device
Mapper to create devices for them.  Try disabling boot.lvm and see if the
problem goes away.

In general, if you are using EVMS devices then you should disable LVM and
software RAID in your boot init scripts, unless you really know what you
are doing.

> The bug seems to occur whenever an EVMS volume has been created/deleted
and
> the machine is then rebooted.

Any time you create or delete an EVMS volume it changes the disk
configuration, which can change which minor numbers get assigned to which
devices.  When the disk configuration changes it can change the order of
which devices EVMS finds first and therefore the order in which the Device
Mapper devices get created.

> Any ideas, or hints how to go on searching?

> Thanks
> Rainer Krienke

Hope this helps.

Steve D.


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Evms-devel mailing list
[email protected]
To subscribe/unsubscribe, please visit:
https://lists.sourceforge.net/lists/listinfo/evms-devel