Re: [Dump-users] slowwwwwww

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

> On Nov 6, 2007 7:59 AM, Stelian Pop <st...@po...> wrote:
>> Le dimanche 04 novembre 2007 à 15:11 -0800, Todd and Margo Chester a
>> écrit :
>>> 2.6.18-8.1.8.el5
>>>
>>> Hi All,
>> Hi,
>>> After I wiped my hard drives clean of CentOS 4.4
>>> (RHEL 4.4 clone) and installed CentOS 5 (RHEL 5),
>>> I noticed that everything was slower.  Especially,
>>> dump, which was 3.5 times slower.
>>>
>>> So, I did some tests from my install CD/DVD's in "rescue mode"
>>> and the System Rescue CD from http://www.sysresccd.org.
>>> I used the same backup script (dump) that I use in normal
>>> boot mode. I stopped the backup after dump gave the first
>>> estimate of time:
>>>
>>>     "dump" CentOS4.4 (Kernel 2.6.9-42.EL) install CD in rescue mode:
>>>          transfer rate 8468 KB/s, estimated finish 2:09
>>>
>>>     "dump" CentOS5 (Kernel 2.6.18) install DVD in rescue mode:
>>>          transfer rate 4422 KB/s, estimated finish 4:10
>>>
>>>     "dump" System Rescue CD (Kernel 2.6.22.9):
>>>          transfer rate 4370 KB/s, estimated finish 4:15
>>>
>>> Has anyone else seen this?  If so, were you able to fix it?
>> I guess you'll have to do some cross-tests yourself... The problem could
>> be related to the filesystem read, the output write, or dump itself.
>>
>> Try dumping to /dev/null, and compare the speeds on both OSes.
>>
>> Try using the same dump version on both OSes to rule out dump problems.
>>
>> If the two tests above do not give any indication, it is probably a
>> kernel problem (either in the filesystem driver or the disk driver).
>> Maybe running some filesystem performance tests (bonnie etc) will show
>> the problem.
>>
>> Stelian.
>> --
>> Stelian Pop <st...@po...>
>>

James Roth wrote:
 > Did you use hdparm to check / enable DMA on the hard drive? What are
 > the drives and how are they configured?
 >

Hi Stelian and James,

     I read through the man page for hdparm and could not figure
out how to read my DMA settings with hdparam.  :'(
How would I test this?

I take it "bonnie++" will mean nothing to any of us
until I have data from a CentOS5 and a CentOS4.5
machine to compare with.  (I will be at the
4.5 customers site next Tuesday and will test
him for a comparison.)

Also, the problem occurs when backing up to tape
as well as hard drive.  Of the four computers below,
all are using RAID controllers as the source of the
data to be backed up.

And, it is not just dump that is slow, everything
is slow.  dump is just the easiest to get number
from.

Basically, I have three identical computers in play: two
with the problem and one without (it is running Cent OS 4.5).
This is extensively documented, including fresh dmesg's, on

          http://bugs.centos.org/view.php?id=2382

Also, there is another party having the same problem
with a different processor, motherboard, chipset,
RAID controller and tape drive.  He is documented over at

http://www.centos.org/modules/newbb/viewtopic.php?topic_id=10659&start=0#forumpost34209

To summarize my three computers:

      My CentOS5: uname -r 2.6.18-8.1.14.el5
      My (and my customer's) motherboards (3) are the Supermicro x7dal-e:

http://www.supermicro.com/products/motherboard/Xeon1333/5000X/X7DAL-E.cfm

      The LSI Megaraid cards are the SATA-150-4 and SATA 300-4XLP:

http://www.lsi.com/storage_home/products_home/internal_raid/megaraid_sata/megaraid_sata_1504/index.html 

http://www.lsi.com/storage_home/products_home/internal_raid/megaraid_sata/megaraid_sata_300_4xlp/index.html 

      The tape drive is a Exabyte VXA-3.
      http://exabyte.com/products/products/get_products.cfm?prod_id=641
      run from an Adaptec 19160 SCSI-U160 LVDS Controller

      The backup hard drive is an external SATA drive (eSATA)

      Computer A: VXA-3, CentOS 5 , LSI SATA 300-4XLP

      DUMP: 17815320 blocks (17397.77MB) on 1 volume(s)
      DUMP: finished in 6473 seconds, throughput 2,752 kBytes/sec

      DUMP: 5038380 blocks (4920.29MB) on 1 volume(s)
      DUMP: finished in 1872 seconds, throughput 2,691 kBytes/sec
      Note: 3 times slower than Computer B

      Computer B: VXA-3, CentOS 4.5 , LSI SATA-150-4

      DUMP: 35933440 blocks (35091.25MB) on 1 volume(s)
      DUMP: finished in 4396 seconds, throughput 8,174 kBytes/sec

      DUMP: 27096640 blocks (26461.56MB) on 1 volume(s)
      DUMP: finished in 3532 seconds, throughput 7,671 kBytes/sec

      Computer C: eSATA, CentOS 4.4 & 5 , LSI SATA-150-4

      CentOS 4.4, 1:05 hours, approx 52 GB backup file 13,333 kBytes/sec
      CentOS 5.0, 3:16 hours, approx 43 GB backup file 3,656 kBytes/sec
      Note: 3.6 times slower

      Computer C: misc speed tests

      cp winxp.hdd /dev/null (7.7 GB), HDD=LSI, Cent 5: 1:44 min,
           Cent OS 4.4: 1:43 min
      cp estat.hdd /dev/null (7.7 GB), HDD=eSata, Cent 5: 3:08 min,
           Cent OS 4.4: 3:07 min

      cp winxp.hdd eraseme (7.7 GB), HDD=LSI, Cent 5: 4:50 min,
           Cent OS 4.4: 4:48 min
      cp estat.hdd eraseme (7.7 GB), HDD=eSata, Cent 5: 7:22 min,
           Cent OS 4.4: 6:54 min

      Parallels' start up of winxp, HDD=LSI, 44 sec
      startup of Acrobat 8 Pro: 1st time: 5 sec; 2nd time: 2 sec

      Parallels' start up of eSata, HDD=eSata, 38 sec
      startup of Acrobat 8 Pro: 1st time: 7 sec; 2nd time: 1 sec

The [forth] computer that the other reporter is using
(cut and paste from the CentOS posting):

      MB Tyan S2892
      2x3ware9508 Raid Controlles 16x300Gb SATA drives.
      1xAIC7902W onboard controller for connect external SCSI devices.
      2xAXUS Brownie BR1600U3P SCSI Ultra 160 device with
            16x250Gb IDE drives in each.
      1xHP Storage Works Ultrium 460 LTO-2 Tape drive
      1xQuantum LTO-2 Tape Drive
      1xHP Storage Works Ultrium 920 LTO-3 Tape drive

      For backup we are use tapes in these Tape drives.

      As you can see this configuration use RAID massives
      but not all of them on RAID cards. Some of these
      RAIDs are on the RAID Storages.

      Now I cannot represent adequate information about
      slower but can write some data for analises.
      We write on tape different information in few
      sessions and this count is different but we can see
      from logs time of start, count of sessions, size of
      total writed data.

      RHEL3:
      Start - End : Size : Sessions
      2007/07/23-13:07:01 - 2007/07/23-16:40:30 : 115.737 GB. : 5
      2007/07/23-13:07:36 - 2007/07/23-16:52:54 : 163.738 GB. : 8
      2007/07/25-09:50:42 - 2007/07/25-12:26:21 : 190.159 GB. : 8
      2007/07/27-09:20:40 - 2007/07/27-10:44:21 : 115.603 GB. : 7
      2007/07/30-09:49:18 - 2007/07/30-12:10:03 : 182.304 GB. : 10

      CentOS 5:
      2007/10/05-16:16:06 - 2007/10/06-16:32:30 : 98.534 GB. : 4
      2007/10/05-16:26:00 - 2007/10/08-16:37:49 : 100.837 GB. : 2
      2007/10/08-16:34:40 - 2007/10/09-17:12:23 : 176.883 GB. : 9
      2007/10/08-16:41:36 - 2007/10/09-05:17:06 : 100.837 GB. : 4
      2007/10/09-16:48:48 - 2007/10/09-22:22:14 : 73.3732 GB. : 4

Many thanks,
-T