Hi Josh,

Thanks for the quick response.  Here's the output from netstat -m while running iperf -s (both on freenas server):

$ netstat -m

305/2005/2310 mbufs in use (current/cache/total)
288/746/1034/262144 mbuf clusters in use (current/cache/total/max)
288/736 mbuf+clusters out of packet secondary zone in use (current/cache)
0/247/247/131072 4k (page size) jumbo clusters in use (current/cache/total/max)
0/0/0/65536 9k jumbo clusters in use (current/cache/total/max)
0/0/0/32768 16k jumbo clusters in use (current/cache/total/max)
652K/2981K/3633K bytes allocated to network (current/cache/total)
0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters delayed (4k/9k/16k)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)
0/0/0 sfbufs in use (current/peak/max)
0 requests for sfbufs denied
0 requests for sfbufs delayed
0 requests for I/O initiated by sendfile
0 calls to protocol drain routines

And the corresponding output from the iperf client:

$ iperf -d -t 60 -i 5 -c freenas
WARNING: option -d is not valid for server mode
------------------------------------------------------------
Client connecting to freenas, TCP port 5001
TCP window size: 21.0 KByte (default)
------------------------------------------------------------
[  3] local 10.40.0.135 port 47951 connected with 10.40.0.30 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0- 5.0 sec  19.0 MBytes  31.9 Mbits/sec
[  3]  5.0-10.0 sec  4.50 MBytes  7.55 Mbits/sec
[  3] 10.0-15.0 sec  5.00 MBytes  8.39 Mbits/sec
[  3] 15.0-20.0 sec  65.1 MBytes   109 Mbits/sec
[  3] 20.0-25.0 sec  46.6 MBytes  78.2 Mbits/sec
[  3] 25.0-30.0 sec  77.8 MBytes   130 Mbits/sec
[  3] 30.0-35.0 sec  8.75 MBytes  14.7 Mbits/sec
[  3] 35.0-40.0 sec   191 MBytes   320 Mbits/sec 3
[  3] 40.0-45.0 sec   384 KBytes   629 Kbits/sec
[  3] 45.0-50.0 sec  4.62 MBytes  7.76 Mbits/sec
[  3] 50.0-55.0 sec  0.00 Bytes  0.00 bits/sec
[  3] 55.0-60.0 sec   274 MBytes   460 Mbits/sec
[  3]  0.0-60.0 sec   697 MBytes  97.5 Mbits/sec

FWIW, I did notice that my SSH terminal slowed to a crawl while this test was in progress.  I became curious if it was a packet loss problem, but I didn't see very larger numbers of packets retransmited (via netstat -s) on either the client or the server.  iperf never gets above 10% CPU usage on the server, and nothing else significant was running at the time.  Lastly, I also noticed that the results are better (though still not particularly consistent) when I don't do a bidirectional test:

$ iperf -t 60 -i 5 -c freenas
------------------------------------------------------------
Client connecting to freenas, TCP port 5001
TCP window size: 21.0 KByte (default)
------------------------------------------------------------
[  3] local 10.40.0.135 port 48161 connected with 10.40.0.30 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0- 5.0 sec   180 MBytes   302 Mbits/sec
[  3]  5.0-10.0 sec   358 MBytes   601 Mbits/sec
[  3] 10.0-15.0 sec   445 MBytes   746 Mbits/sec
[  3] 15.0-20.0 sec   348 MBytes   585 Mbits/sec
[  3] 20.0-25.0 sec   427 MBytes   717 Mbits/sec
[  3] 25.0-30.0 sec   429 MBytes   720 Mbits/sec
[  3] 30.0-35.0 sec   135 MBytes   226 Mbits/sec
[  3] 35.0-40.0 sec  32.2 MBytes  54.1 Mbits/sec
[  3] 40.0-45.0 sec   148 MBytes   247 Mbits/sec
[  3] 45.0-50.0 sec   349 MBytes   586 Mbits/sec
[  3] 50.0-55.0 sec   391 MBytes   656 Mbits/sec
[  3] 55.0-60.0 sec   396 MBytes   664 Mbits/sec
[  3]  0.0-60.0 sec  3.55 GBytes   509 Mbits/sec

Thanks again for your help!

Cheers,
Tobias

On Wed, Aug 7, 2013 at 12:52 AM, Josh Paetzel <josh@ixsystems.com> wrote:
The E-350 is a great cpu and chipset combination for a home media server, but it's a touch underpowered for doing wire speed gigE, especially to one client. 

The realtek is not a great NIC but it's good enough. I suspect you're seeing jittery performance due to some default tuning not being optimal for your setup. 

The linux test shows the hardware is capable of blasting bits across the wire at near gigE line rate, (which is a lot different in terms of CPU horsepower needed than doing TCP at gigE) but definitely tells us it's likely not a hardware fault. 

Run netstat -m while you have an iperf in progress and send the results of that. 

Thanks,

Josh Paetzel
Director of IT, iXsystems
Servers for Open Source

On Aug 6, 2013, at 8:11 PM, Tobias McNulty <tobias@caktusgroup.com> wrote:

Hello all,

I have an ASRock E350M1 motherboard [1] on which I recently installed the newly released FreeNAS 9.0 64-bit.  Transfer speeds were not as fast as I expected (~ 14 MB/sec over the network, but ~ 37 MB/sec writing straight to disk on the FreeNAS machine itself).  Through trial and error I may have identified the network as the bottleneck.  This motherboard has a Realtek RTL8111E ethernet controller.  Not great, I know.

I tested the network with iperf from my laptop (through a gigabit switch) and get widely varying results [2]; anywhere from 5-7 Mbits/sec up to 629 Mbits/sec (though usually on the far lower end).  That said, when I booted the same machine into Ubuntu Linux 12.04 (Linux kernel: 3.2), I received incredibly consistent results in the 932 Mbits/sec range.  In both of these cases, the E350M1 was the iperf *server* and my laptop was the *client.*

Now, when I run the iperf *server* on my laptop and the iperf *client* on the FreeNAS machine, I get much more consistent results, in the 790Mbits/sec range.  The iperf client command I'm using is:

iperf -d -t 60 -i 5 -c <server IP>

And iperf reports itself to be the same version on both my laptop and the FreeNAS machine:

iperf version 2.0.5 (08 Jul 2010) pthreads

Does anyone know what might explain these results, or how I could dig further to figure out what's causing the highly unreliable transfer speeds when the FreeNAS machine is the server?  Given that it's quite fast and consistent in Linux I expect it's a driver issue, but I do not know how or where to begin debugging such things in FreeNAS or BSD in general.

Many thanks in advance for your help.

Cheers,
Tobias

[2] See below:

$ iperf -d -t 60 -i 5 -c freenas
WARNING: option -d is not valid for server mode
------------------------------------------------------------
Client connecting to freenas, TCP port 5001
TCP window size: 21.0 KByte (default)
------------------------------------------------------------
[  3] local 10.30.0.135 port 47552 connected with 10.30.0.30 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0- 5.0 sec  8.75 MBytes  14.7 Mbits/sec
[  3]  5.0-10.0 sec  0.00 Bytes  0.00 bits/sec
[  3] 10.0-15.0 sec  4.62 MBytes  7.76 Mbits/sec
[  3] 15.0-20.0 sec  5.50 MBytes  9.23 Mbits/sec
[  3] 20.0-25.0 sec  0.00 Bytes  0.00 bits/sec
[  3] 25.0-30.0 sec   142 MBytes   237 Mbits/sec
[  3] 30.0-35.0 sec   384 KBytes   629 Kbits/sec
[  3] 35.0-40.0 sec   115 MBytes   192 Mbits/sec
[  3] 40.0-45.0 sec   384 KBytes   629 Kbits/sec
[  3] 45.0-50.0 sec   185 MBytes   310 Mbits/sec
[  3] 50.0-55.0 sec  0.00 Bytes  0.00 bits/sec
[  3] 55.0-60.0 sec   116 MBytes   195 Mbits/sec
[  3]  0.0-60.8 sec   577 MBytes  79.6 Mbits/sec
-- 
Tobias McNulty, Managing Member
Caktus Consulting Group, LLC
http://www.caktusgroup.com
------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
Freenas-devel mailing list
Freenas-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freenas-devel



--
Tobias McNulty, Managing Member
Caktus Consulting Group, LLC
http://www.caktusgroup.com