Menu
▾
▴
[Scst-devel] Performance of block device I/O versus filesystem I/O
|
From: Bart V. A. <bar...@gm...> - 2009-07-18 11:29:38
|
Hello Vlad,
During the last few weeks it has been stated several times that block device
I/O is not faster than I/O through a file system (XFS) using the same block
device. Since I was curious to know how big the performance difference is, I
ran a test (see below for the results). The conclusions are as follows when
comparing file system throughput to block I/O throughput:
* Same throughput for block sizes of 8 KB and above.
* Better write throughput (+80%) and lower read throughput (-30%) for the 4
KB block size.
* Lower throughput for small block sizes (512 bytes - 2 KB).
The details are as follows:
Kernel version: 2.6.27.23-0.1-default (unmodified openSUSE 11.1 kernel).
I/O scheduler for both disks: CFQ.
Scheduler parameters:
$ (cd /sys/class/block/sda/queue/iosched && for p in *; do echo $p $(<$p);
done)
back_seek_max 16384
back_seek_penalty 2
fifo_expire_async 248
fifo_expire_sync 124
quantum 4
slice_async 40
slice_async_rq 2
slice_idle 8
slice_sync 100
Block I/O test:
$ cat /proc/mdstat | grep -A2 ^md4
md4 : active raid1 sdb6[1] sda6[0]
5759168 blocks [2/2] [UU]
$ ~bart/software/scst/scripts/blockdev-perftest -s 27 /dev/md4
blocksize W W W W(avg, W(std, W R
R R R(avg, R(std R
(bytes) (s) (s) (s) MB/s) ,MB/s) (IOPS) (s)
(s) (s) MB/s) ,MB/s) (IOPS)
67108864 2.359 2.318 2.330 54.802 0.407 0.856 2.410
2.276 2.340 54.685 1.280 0.854
33554432 2.364 2.414 2.343 53.937 0.673 1.686 2.408
2.420 2.349 53.505 0.700 1.672
16777216 2.376 2.373 2.354 54.067 0.219 3.379 2.405
2.594 2.762 49.638 2.817 3.102
8388608 2.522 2.334 2.413 52.876 1.674 6.610 2.307
2.453 2.347 54.069 1.385 6.759
4194304 2.398 2.379 2.323 54.101 0.735 13.525 2.421
2.344 2.388 53.692 0.717 13.423
2097152 2.388 2.571 2.366 52.492 1.923 26.246 2.334
2.855 2.323 51.595 4.782 25.798
1048576 2.340 4.380 2.387 45.850 11.764 45.850 2.345
2.419 2.419 53.472 0.790 53.472
524288 2.286 2.423 2.346 54.468 1.295 108.935 2.302
2.511 2.389 53.381 1.895 106.761
262144 2.327 2.346 2.637 52.704 2.943 210.815 2.461
2.261 2.428 53.780 2.026 215.120
131072 2.261 2.379 2.390 54.654 1.385 437.229 2.397
3.403 2.435 47.862 7.252 382.892
65536 2.328 2.333 2.391 54.468 0.660 871.487 2.415
2.357 2.530 52.637 1.539 842.196
32768 2.420 2.416 2.378 53.233 0.421 1703.470 2.349
2.436 2.387 53.563 0.796 1714.008
16384 2.407 2.533 2.475 51.814 1.086 3316.115 2.444
2.278 2.390 54.040 1.599 3458.553
8192 2.804 5.077 3.848 34.709 8.405 4442.799 2.562
2.617 3.793 44.209 7.412 5658.716
4096 7.040 7.971 9.277 16.013 1.790 4099.204 3.932
3.991 4.093 31.967 0.527 8183.521
2048 10.631 8.660 8.226 14.128 1.510 7233.307 6.912
7.064 6.860 18.432 0.228 9437.334
1024 18.510 15.788 15.998 7.675 0.539 7858.690 12.425
14.150 12.681 9.814 0.550 10049.624
512 31.726 32.322 30.501 4.064 0.099 8322.509 24.604
24.142 25.465 5.177 0.114 10602.283
File I/O test:
$ xfs_info .
meta-data=/dev/md4 isize=256 agcount=4, agsize=359948 blks
= sectsz=512 attr=2
data = bsize=4096 blocks=1439792, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0
log =internal bsize=4096 blocks=2560, version=2
= sectsz=512 sunit=0 blks, lazy-count=0
realtime =none extsz=4096 blocks=0, rtextents=0
$ grep $(df . | tail -n 1 | awk '{print $1}') /proc/mounts
/dev/md4 /mnt xfs rw,noatime,attr2,noquota 0 0
$ dd if=/dev/zero of=block bs=1M count=1K
$ ~bart/software/scst/scripts/blockdev-perftest -s 27 block
blocksize W W W W(avg, W(std, W R
R R R(avg, R(std R
(bytes) (s) (s) (s) MB/s) ,MB/s) (IOPS) (s)
(s) (s) MB/s) ,MB/s) (IOPS)
67108864 2.268 2.297 2.297 55.957 0.341 0.874 2.314
2.406 2.270 54.966 1.328 0.859
33554432 2.374 2.297 2.372 54.534 0.840 1.704 2.342
2.437 2.431 53.269 0.974 1.665
16777216 2.367 2.297 2.381 54.519 0.858 3.407 2.303
2.613 2.640 51.017 3.226 3.189
8388608 2.947 2.422 2.364 50.143 4.775 6.268 2.372
2.349 2.380 54.078 0.305 6.760
4194304 2.406 2.291 2.398 54.141 1.217 13.535 2.348
2.438 2.608 52.029 2.242 13.007
2097152 2.485 2.376 2.387 52.999 1.053 26.500 2.446
2.341 2.339 53.915 1.117 26.958
1048576 2.666 2.905 2.385 48.580 3.938 48.580 2.253
2.445 2.416 54.047 1.975 54.047
524288 2.453 2.356 2.307 53.993 1.374 107.986 2.401
2.401 2.463 52.862 0.638 105.723
262144 2.388 2.299 2.388 54.290 0.987 217.160 2.403
2.419 2.420 53.015 0.172 212.061
131072 2.400 2.319 2.424 53.774 1.020 430.196 2.669
2.823 2.335 49.372 3.994 394.974
65536 2.286 2.281 2.427 54.941 1.561 879.057 2.278
2.371 2.223 55.920 1.475 894.722
32768 2.287 2.313 2.374 55.074 0.864 1762.362 2.402
2.383 2.354 53.799 0.450 1721.577
16384 2.603 2.362 2.390 52.309 2.233 3347.758 2.321
2.412 2.275 54.827 1.327 3508.948
8192 3.781 4.673 2.987 34.697 6.340 4441.222 2.658
2.772 2.664 47.455 0.908 6074.212
4096 4.311 4.432 4.462 29.087 0.434 7446.351 3.932
6.450 7.697 23.008 6.873 5890.070
2048 10.438 8.387 8.317 14.305 1.445 7324.099 7.083
7.248 7.296 17.759 0.227 9092.691
1024 18.131 16.329 17.091 7.463 0.319 7641.814 14.189
14.311 15.228 8.790 0.274 9001.167
512 32.458 32.974 32.882 3.906 0.027 7999.645 26.410
25.133 26.174 4.943 0.107 10123.800
Ratio of file I/O throughput to block I/O throughput:
blocksize W R
(bytes)
67108864 1.02 1.01
33554432 1.01 1.00
16777216 1.01 1.03
8388608 0.95 1.00
4194304 1.00 0.97
2097152 1.01 1.04
1048576 1.06 1.01
524288 0.99 0.99
262144 1.03 0.99
131072 0.98 1.03
65536 1.01 1.06
32768 1.03 1.00
16384 1.01 1.01
8192 1.00 1.07
4096 1.82 0.72
2048 1.01 0.96
1024 0.97 0.90
512 0.96 0.95
Bart.
|