openQRM Solution in Germany for Oracle

  • Dietmar Simons
    Dietmar Simons


    I would tell you something about our openQRM solution for one of my customer in Germany.

    At first, the hardware:
    1x Dell PowerEdge 1950, 1x Intel QuadCore Xeon CPU, 2 GB memory, LSI 7104 Fibre-Channel 4 GBit
       (for the openQRM Server)
    3x Dell PowerEdge 1950, 2x Intel QuadCore Xeon CPU, 4 GB memory, LSI 7104 Fibre-Channel 4 GBit
       (for the nodes)
    1x Qlogic SANbox Express 1400 (4 Gbit)
    1x Infortrend Eonstor A12F-G2422 SAN-Box

    and now the software:
    -openQRM 3.1.2:
    -Linux CentOS 4.4

    My first Goal for my customer:
    Three nodes as Xen Dom0 and in summary twelve Xen DomU on this. The Xen DomU boot over NFS from the  QRM-server and mount the blockdevices direct from the SAN-Box. In the Xen DomU should run Oracle 9i, Oracle 10g, Oracle 10gR2, Lotus Domino Server, Samba 3, OpenLDAP, Apache and DNS.

    The QRM server stored the filesystem images on the SAN-Box over FC.

    I start with this project in November 2006, so I installed first openQRM 2.2.x and find the first problems, like only one NIC on the nodes and Dom0/DomU is supported. No support for FibreChannel or other blockdevices, no SMP and it was not possible to change the Xen Configuration over the openQRM GUI.

    I must have two NIC's on den Xen Dom0 and DomU, because my customer have a management network and a production network run. And if the Xen Dom0 and DomU have only one NIC, all of the network traffic runs over the QRM-Server, wich will act as router in this case.

    After a lot of emails and many phone-calls with Matt from Qlusters, he realised my wishes and changes very fast, so the result was the openQRM Xen-plugin 3.1.2.

    Now it support more than one NIC, SMP, mounting blockdevices from the Fibre-Channel, changes in the Xen-Configuration on a running Xen-Partition from the openQRM GUI.

    OpenQRM have no update function :-(
    So I must deinstall openQRM 2.2.x from the QRM Server, install openQRM 3.1.2 and have to make a new configuration in the GUI.

    After this I could do everything what I would do :-)

    I add in the resource configuration tab in the field "Resource parameters":
    XEN_PARTITION_NETWORK="00:00:09:09:00:01/xenbr1" XEN_PARTITION_HARDDISK="sda11/sda11/w, sda12/sda12w, sda13/sda13/w"
    and stop the Xen-Partion and start it again. Important: A restart change nothing!

    I create a few Virtual Enviroments and install the software:
    -Lotus Domino: successful
    -Oracle 9i: successful
    -Oracle 10g: successful
    -Oracle 10gR2: successful
    -Samba, DNS, LDAP, Apache: successful

    The cpu and network performance in the Xen DomU is very good.

    1st problem:
    But the performance in a Xen Dom0/DomU with the blockdevices in the SAN is very poor, remember: it is 4 Gbit FC.
    A copy from one blockdevice to an other blockdevice is no problem and a 560 MB file is copied in a few seconds.
    When I copy a 560MB file from a directory to an other directory on the same blockdevice it needs more than 90 seconds.

    When I do the same thing on the QRM Server everything is fine, every copy with the 560 MB file needs a few seconds.

    I search since January 2007 for a solution for this failure and find nothing :-(

    2nd problem:
    The nodes have 4 GB memory, but after a boot with the Xen-Hypervisor I see only 3.2 GB, not more.
    I look in the web and find the reason: the Xen-Hypervisor in openQRM was not compile with PAE and hughmem options.

    After an email and a phone-call with Matt he told me how I can change this.

    You find my little HowTo here:

    My first problem could not be solved :-(

    Now I realized this:
    The nodes boot from the QRM Server over NFS a normal CentOS 4.4 and mount die blockdevices from the SAN. I install the software like Oracle and so on again and my customer can now work.

    I mean openQRM is a great software solution and the developer from Qlusters do a great job.

    Special Thanks to Matt from Qlusters for his endless endurance and great support work!!!

    Dietmar Simons


    Simons IT-Service
    Dietmar Simons
    Matthiasstr. 18a
    41468 Neuss

    Phone +49 2131 15 34 98
    Fax +49 2131 15 34 99

    • Matt

      Hi Dietmar,

      thank you so much for those detailed informations about your very interesting setup !
      This is very useful for the other users of the Xen-plugin.

      all the best,