Re: [Evms-devel] Getting EVMS working again after mdadm --grow

[Steve D. <st...@us...>]

Brian Kuschak <bku...@ya...> wrote on 11/04/2008 03:17:21 PM:

> > OK.  I'm crawling through the LVM2 plug-in code.  I see
> > the place where it
> > logs the warning, "Object %s recorded size
> > (%"PRIu64") does not match
> > actual size (%"PRIu64")\n", which is
> > what you see in your log.  It is the
> > else side of an if clause that checks if it should display
> > the device size
> > prompt.  Apparently, the setting for the device size prompt
> > is "no",
> > otherwise instead of logging a message it would have put up
> > a prompt asking
> > you if the object was indeed a PV.  Look at your
> > /etc/evsm.conf file.  In
> > the "lvm2" section there is an option
> > "device_size_prompt".  I will bet it
> > is set to "no".  Try changing it to
> > "yes".  When you are prompted whether
> > md/md0 is an LVM2 PV, answer "yes".  Be sure to
> > answer "no" for the other
> > objects, such as sd[acdef].  Answering "yes" for
> > md/md0 will cause the LVM2
> > plug-in to write the correct object size to the LVM
> > metadata.  Once you are
> > finished, change the setting of
> > "device_size_prompt" back to "no" or
> > else
> > you will be asked if sd[acdef] are PVs every time you start
> > EVMS.
>
> I followed your instructions and it appears that things are working
> again!  Yea!!

Woohoo!

> I do still see these messages about size mismatch, though:
>
> Nov 04 13:12:24 netdisk _6_ LVM2: read_pv_label: Found LVM2 label on
> object sde, sector 1.
> Nov 04 13:12:24 netdisk _3_ LVM2: check_pv_size: Object sde recorded
> size (2500527104) does not match actual size (1250263728)
> Nov 04 13:12:24 netdisk _6_ LVM2: read_pv_label: Found LVM2 label on
> object sdd, sector 1.
> Nov 04 13:12:24 netdisk _3_ LVM2: check_pv_size: Object sdd recorded
> size (2500527104) does not match actual size (1250263728)
> Nov 04 13:12:24 netdisk _6_ LVM2: read_pv_label: Found LVM2 label on
> object sdc, sector 1.
> Nov 04 13:12:24 netdisk _3_ LVM2: check_pv_size: Object sdc recorded
> size (625163264) does not match actual size (312581808)
> Nov 04 13:12:24 netdisk _6_ LVM2: read_pv_label: Found LVM2 label on
> object sda, sector 1.
> Nov 04 13:12:24 netdisk _3_ LVM2: check_pv_size: Object sda recorded
> size (625163264) does not match actual size (321672960)
>
> Do I need to worry about these?

No, don't worry about them.  As I mentioned in my previous note, just set
"device_size_prompt" back to "no" in /etc/evsm.conf.

You are always guaranteed to get at least one of those messages when you
use MD devices in an LVM container.  At least one of the MD disks is going
to have the LVM2 metadata at the front of the disk because that disk is at
the beginning of the MD device.  The size recorded in the LVM metadata is
the size of the MD device.  But when the LVM2 plug-in examines the disk,
say, sdd, it will see that the size in the LVM metadata does not match the
size of sdd.  The LVM2 plug-in should ignore the disk and wait until md/md0
is created.  When it examines md/md0 it will see that the size in the
metadata matches the size of the device and will use it as a PV.

On a RAID5 array you should only have one disk that contains the LVM2
metadata, because only one disk contains the first sectors of the MD
device.  That you have more than one disk presenting the error probably
means you have stale LVM metadata on those other disks.  It can be ignored.
The metadata will be part of the free space on the md/md0 device.

> One thing that is different, however, is that previously the volumes
> page showed /dev/evms/big_volume as my mounted object, and this is
> what I activate in my sysinit script.  After the fixes, this is now
> shown as /dev/evms/lvm2/lvm_on_raid_group_120/big_volume.  I updated
> my init script with this object and it mounts properly.  I'm just
> curious: why the change?
>
> -Brian

Your volume has changed from an "EVMS volume" to a "compatibility volume".
When you create an EVMS volume you give it a name that is meaningful to
you.  A compatibility volume cannot be named.  Its name is compatible with
what Linux would call the volume.  In your case, under standard Linux the
volume would be accessed as /dev/lvm2/lvm_on_raid_group_120/big_volume.

Why did the volume change?  EVMS keeps its information about an EVMS volume
on the last two sectors of the device.  The volume size presented is two
sectors less than the size of the device.  When you expanded the big_volume
LV the EVMS metadata was no longer in the last two sectors of the device.
Without the EVMS metadata, EVMS created a compatibility volume instead of
an EVMS volume.

What can you do about it?  You *may* be able to select the volume and be
presented with an option to convert it to an EVMS volume.  EVMS will not
present the option if it cannot safely overwrite the last two sectors on
the LV.  Sometimes a file system doesn't allocate all of the available
sectors.  For example, a file system that allocates in 4KB chunks (8
sectors) will leave 5 sectors free on a device with 800005 sectors.  If
that is the case then EVMS can use the unused sectors.  Sometimes EVMS can
shrink the volume to make room for the two sectors it needs.  I am going to
guess that is your case you will not be able to convert the volume to an
EVMS volume.  LVM serves up volumes in multiples of its chunk size, say,
4MB, which is most likely a multiple of the XFS allocation size, so there
won't be any unused sectors.  XFS does not do shrinking, so EVMS will not
be able to create the space that it needs for its metadata.  EVMS won't be
able to create its metadata for an EVMS volume, so it won't let you convert
it.  :(  At least you have it working using the compatibility volume.

Steve D.