Menu
▾
▴
Re: [Lksctp-developers] SCTP send and receive buffer utilization
|
From: Dilip D. <dil...@hp...> - 2012-10-24 11:43:34
|
Hi Vlad, I think I may have answered my own question (correct if I'm wrong): TX_QUEUE = 279702 is well between the values of net.core.wmem_default=262144 and net.core.wmem_max=1048576 and therefore due to sctp auto buffer tuning the TCP_QUEUE value is correct and could at times be greater than net.core.wmem_default. -DilipD. On Tue, 2012-10-23 at 19:58 -0400, Dilip Daya wrote: > Hi Vlad, > > > Environment: > sysctl -w net.core.wmem_max=1048576 > sysctl -w net.core.rmem_max=1048576 > sysctl -w net.core.wmem_default=262144 > sysctl -w net.core.rmem_default=16384 > sysctl -w net.sctp.sndbuf_policy=1 > > server: sctp_test -H 192.168.3.62 -P 2345 -h 192.168.3.61 -p 6789 -l > client: sctp_test -H 192.168.3.61 -P 6789 -h 192.168.3.62 -p 2345 -s \ > -a 1 -c 6 -m 65515 -x 2 > > > Thanks to you and Neil for your responses so far...that helped, but > could you confirm whether the following code line: > > - http://lxr.linux.no/linux+v3.6.1/net/sctp/socket.c#L134 > > takes effect when net.sctp.sndbuf_policy = 1 ? > > => I find that when net.sctp.sndbuf_policy = 1 the above still show > sndbuf_policy = 0. > > > => I'm still grappling with output from instrumented sctp to > produce /proc/net/sctp/assocs: > > i.e. TX_QUEUE is greater than sk_sndbuf -- is this expected > behavior ? > > ASSOC SOCK STY SST ST HBKT ASSOC-ID TX_QUEUE > ffff88011e140000 ffff880122779040 0 7 3 35872 1 279702 > > sk_sndbuf peer_rwnd RX_QUEUE sk_rcvbuf UID INODE LPORT RPORT ... > 262144 1136 768 16384 0 917 6789 2345 ... > > > > Thanks. > -DilipD. > > > On Sun, 2012-10-21 at 15:44 -0400, Vlad Yasevich wrote: > > > > > > > > On Oct 20, 2012, at 4:01 PM, Dilip Daya <dil...@hp...> wrote: > > > > > Hi Neil, > > > > > > On Fri, 2012-10-19 at 19:32 -0400, Neil Horman wrote: > > >> On Fri, Oct 19, 2012 at 12:55:47PM -0400, Dilip Daya wrote: > > >>> Is there a way to calculate the sctp send and receive buffer > > >>> utilization? i.e. > > >>> > > >>> Environment: > > >>> * 3.6.1-stable (x86_64) > > >>> > > >>> * sctpspray client to server: /usr/sbin/sctpsprayd > > >>> # sctpspray -c 1000 -d 100000 -l 16384 -s 10000 192.168.3.62 > > >>> > > >>> * net.sctp.rcvbuf_policy = 1 > > >>> net.sctp.sndbuf_policy = 0 > > >>> > > >>> * > > >>> # sysctl -a | grep [rw]mem > > >>> net.core.wmem_max = 1048576 > > >>> net.core.wmem_default = 9216 > > >>> net.core.rmem_max = 1048576 > > >>> net.core.rmem_default = 42080 > > >>> net.sctp.sctp_wmem = 4096 16384 4194304 > > >>> net.sctp.sctp_rmem = 4096 865500 4194304 > > >>> > > >>> * Output from /proc/net/sctp/assocs: > > >>> > > >>> ASSOC SOCK STY SST ST HBKT ASSOC-ID TX_QUEUE RX_QUEUE UID INODE > > >>> LPORT RPORT LADDRS <-> RADDRS HBINT INS OUTS MAXRT T1X T2X RTXC > > >>> ffff880220e65800 ffff880221f33a40 0 7 3 58414 9 1579040 > > >>> 0 0 27174 46026 11776 10.27.82.50 192.168.1.61 192.168.3.61 > > >>> 192.168.4.61 192.168.5.61 192.168.6.61 <-> *192.168.3.62 10.27.82.60 > > >>> 192.168.2.62 192.168.7.62 192.168.5.62 192.168.6.62 192.168.4.62 > > >>> 7500 10 10 10 0 0 0 > > >>> > > >>> > > >>> net.core.rmem_max = 1048576 \ > > >>> TX_QUEUE = 1579040 / ~ 50% more than net.core.rmem_max - > > >>> why? > > >>> > > >>> TX_QUEUE = sctp_association->sndbuf_used > > >>> - This is the sndbuf size in use for the association. > > >>> - This corresponds to the sndbuf size for the association, as specified > > >>> in the sk->sndbuf > > >>> > > >>> > > >>> Likewise what is the correct method to calculate the sctp receive buffer > > >>> (RX_QUEUE) utilization? Should this be compared to net.core.wmem_max ? > > >>> > > >>> RX_QUEUE = sctp_association->rmem_alloc > > >>> - This is the amount of memory that this association has allocated > > >>> in the receive path at any given time. > > >>> > > >>> - Get the sndbuf space available at the time on the association: > > >>> sk_wmem_alloc_get(sctp_association->base.sk) > > >>> | > > >>> v > > >>> (sock->sk_wmem_alloc - 1) > > >>> | > > >>> v > > >>> write allocations = (transmit queue bytes committed - 1) > > >>> > > >>> > > >>> Thanking you in advance. > > >>> -DilipD. > > >>> > > >>> > > >>> ------------------------------------------------------------------------------ > > >>> Everyone hates slow websites. So do we. > > >>> Make your web apps faster with AppDynamics > > >>> Download AppDynamics Lite for free today: > > >>> http://p.sf.net/sfu/appdyn_sfd2d_oct > > >>> _______________________________________________ > > >>> Lksctp-developers mailing list > > >>> Lks...@li... > > >>> https://lists.sourceforge.net/lists/listinfo/lksctp-developers > > >>> > > >> netstat -p sctp > > >> That should tell you the per socket tx/rx memory usage at any point in time > > >> Neil > > > > > > netstat option "-p": > > > > > > -p, --program > > > Show the PID and name of the program to which each socket belongs. > > > > > > i.e. > > > > > > # netstat -p > > > Active Internet connections (w/o servers) > > > Proto Recv-Q Send-Q Local Address Foreign Address > > > State PID/Program name > > > ... > > > > > > Questions are: > > > > > > 1. Wouldn't "Recv-Q" and "Send-Q" from "netstat -p" be the same as > > > TX_QUEUE and RX_QUEUE from /proc/net/sctp/assocs ? > > > > > > Recv-Q > > > The count of bytes not copied by the user program connected to > > > this socket. > > > > > > Send-Q > > > The count of bytes not acknowledged by the remote host. > > > > > > > > > 2. Is max value for TX_QUEUE and RX_QUEUE determined by > > > net.core.wmem_max and net.core.rmem_max respectively ? > > > > > > > > > 3. If (1) is correct, then why is TX_QUEUE value bigger than the max > > > in some cases? > > > > > > I think I've read somewhere that SCTP does not care or check if > > > TX/RX_QUEUE value is over the max. Is this correct? > > > > > > > > > 4. Need to calculate TX_QUEUE/RX_QUEUE % utilization by say something > > > like (if correct): > > > > > > % TX_QUEUE utilization = (TX_QUEUE / net.core.wmem_max) * 100 > > > > I'd use the sk_rcvbuf/sndbuf values instead, but this is generally correct. the trouble is that current queue lengths are transient and can change very quickly. So your utilization is valid for a given instant, not lifetime of association. > > > > -vlad > > > > > > > > same method for RX_QUEUE. > > > > > > > > > Thanking you in advance. > > > -DilipD. > > > > > > > > > ------------------------------------------------------------------------------ > > > Everyone hates slow websites. So do we. > > > Make your web apps faster with AppDynamics > > > Download AppDynamics Lite for free today: > > > http://p.sf.net/sfu/appdyn_sfd2d_oct > > > _______________________________________________ > > > Lksctp-developers mailing list > > > Lks...@li... > > > https://lists.sourceforge.net/lists/listinfo/lksctp-developers |