Munin helps showing performance changes

<p>Using btrfs on a networked backup server looked like a good idea, what with the
data integrity checksumming and all.</p>

<p>Reformatting it to ext4 gave a decent increase in write performance, and will
hopefully give fewer server crashes per week (from "many" to "none" is the goal)
Just before this wipe-and-reinstall, "umount" had been hanging for a few hours,
and the admin got a tad annoyed.</p>

<p>This is on Ubuntu 12.04.4 LTS (GNU/Linux 3.12.8-031208-generic x86_64). The
"disk" in question is a single 28T device on a nearby disk shelf.</p>

<p>Now, why this performance difference? We have <a href="http://munin-monitoring.org/">Munin</a> installed, and showing graphs
from before and after the change gives us a few clues.</p>

<h2>Network througput</h2>

<p>Network throughput increased to the limit. Looks like it is time to move to
active/active bonding, instead of active/passive.</p>

<p><a href="http://fnord.no//tag/munin/../../posts/2014-02/disk-performance/if_bond0-day.png"><img alt="Bandwidth graph for bond0" class="img" height="111" src="http://fnord.no//tag/munin/../../posts/2014-02/disk-performance/200x200-if_bond0-day.png" width="200" /></a></p>

<h2>Storage graphs</h2>

<p>The number of operations went down, while the request size increased massively.
This allowed much more data to be written to the device.</p>

<p><a href="http://fnord.no//tag/munin/../../posts/2014-02/disk-performance/diskstats_iops__sdb-day.png"><img alt="IO operations per second" class="img" height="115" src="http://fnord.no//tag/munin/../../posts/2014-02/disk-performance/200x200-diskstats_iops__sdb-day.png" width="200" /></a>
<a href="http://fnord.no//tag/munin/../../posts/2014-02/disk-performance/diskstats_latency__sdb-day.png"><img alt="IO latency per second" class="img" height="125" src="http://fnord.no//tag/munin/../../posts/2014-02/disk-performance/200x200-diskstats_latency__sdb-day.png" width="200" /></a>
<a href="http://fnord.no//tag/munin/../../posts/2014-02/disk-performance/diskstats_throughput__sdb-day.png"><img alt="IO throughput per second" class="img" height="111" src="http://fnord.no//tag/munin/../../posts/2014-02/disk-performance/200x200-diskstats_throughput__sdb-day.png" width="200" /></a>
<a href="http://fnord.no//tag/munin/../../posts/2014-02/disk-performance/diskstats_utilization__sdb-day.png"><img alt="IO device utilization" class="img" height="111" src="http://fnord.no//tag/munin/../../posts/2014-02/disk-performance/200x200-diskstats_utilization__sdb-day.png" width="200" /></a></p> link

Posted by SourceForge Robot 2014-02-21