#11 CSM support for AoE devices

Clustering (1)
Kevin Taylor

The current EVMS builds do not seem to correctly
identify Ata-over-Ethernet devices (/dev/etherd/e0.0,
etc) as capable of being used with the Cluster Segment
Manager. Support for AoE devices in CSM would help
enable volume management for a very cost-effective and
readily available method of shared storage.


  • Logged In: YES

    The CSM accepts any unused disk or segment. I can think of
    two reasons why it may not recognize /dev/etherd/e0.0 as

    1) Does EVMS pick up /dev/etherd/e0.0 as a disk? That is,
    does it appear on the "Disks" tab in evmsgui, on the
    "Logical Disks" panel (panel 6) in evmsn, or in the list of
    queried disks ("q:d" command) in evms? If not, it is
    probably because it does not pass the "include" and
    "exclude" filters for disks in /etc/evms.conf. See the
    "legacy_devices" and "sysfs_devices" sections in /etc/evms.conf.

    2) Is the disk available? That is, does it appear on the
    "Available Objects" tab in evmsgui or the "Available
    Objects" panel (panel 1) in evmsn? There are several
    reasons the disk may not be available.
    a) The disk may be made into a volume, i.e.,
    /dev/evms/etherd/e0.0. (The first time EVMS sees a disk it
    makes a volume for it since that is how native Linux sees
    the disk.) If so, you can delete the volume.
    b) The disk may have partitions on it or it may be a part of
    an LVM group or RAID. If so, then according to the EVMS
    architecture it cannot be used for anything else. You will
    have to delete the partitions ("segments" in EVMS-speak),
    LVM group, or RAID (backing up any data you want to save),
    add the CSM to the disk, and recreate the segment/LVM
    group/RAID using the segment produced by the CSM that
    represents the disk.

    If the disk appears as an available object and the CSM won't
    use it then we will have to debug it further to determine
    the cause. It should work. :)

    Steve D.

  • Kevin Taylor
    Kevin Taylor

    Logged In: YES

    Hi Steve,

    Yes, it was perplexing to me as well. Let me delve into
    more detail on the two items you sent:

    1) It does indeed show up as a disk under "LocalDskMgr" as

    2) Yes, it does show up under "Available Objects", again as
    "etherd/e0.0". Further, it does see it as something that
    action can be taken on, just not CSM. With this pristine
    disk object, I am given options to:
    - Segments: Add BBR
    - Containers: LVM or LVM2 region manager
    - Regions: LVM2 or one of 4 MD Region Managers
    - Features: Add drive-linking or snapshotting
    - Create EVMS volume
    - Create compatibility volume

    I can not get CSM to show up anywhere as an option, so hence
    I thought it might be something specific to it.

    I am using the EVMS which comes with SuSE 10, version 2.5.3


  • Logged In: YES

    Hi, Kevin.

    Here is another thing to check. Is heartbeat running and
    does the cluster membership have quorum? Run "evmsccm -v"
    to see if the membership has quorum.

    The CSM will not allow any configuration changes if the
    cluster membership does not have quorum. That's because it
    needs to make sure the other nodes pick up the configuration

    If the membership does have quorum then I'll need some more
    information to debug the problem. Run with the logging
    level set to "debug", e.g., "evmsgui -d debug", and bring up
    the Add Segment Manager to Storage Object menu (which
    doesn't list the CSM as an option). Then gzip the log
    (/var/log/evms-engine.log) and send it to me, steved@us.ibm.com.

    Steve D.