#498 STUN hangs.

1.7.x
closed-fixed
core (110)
5
2012-04-10
2012-03-30
Nick Altmann
No

opensips-1.7.2 SVN #8787

After about 12 hours of opensips uptime it uses more, than 90% of cpu and STUN doesn't work.
This messages in log:
opensips-2-0 WARNING:core:fm_malloc: Not enough free memory, will attempt defragmenation

I tried this patch:
diff -durN config.h config.h
--- config.h (revision 8837)
+++ config.h (working copy)
@@ -114,8 +114,8 @@

#define SRV_MAX_PREFIX_LEN SRV_TLS_PREFIX_LEN

-#define PKG_MEM_SIZE 2 /*!< Used only if PKG_MALLOC is defined*/
-#define SHM_MEM_SIZE 32 /*!< Used if SH_MEM is defined*/
+#define PKG_MEM_SIZE 128 /*!< Used only if PKG_MALLOC is defined*/
+#define SHM_MEM_SIZE 512 /*!< Used if SH_MEM is defined*/

#define TIMER_TICK 1 /*!< one second */
#define UTIMER_TICK 100*1000 /*!< 100 miliseconds*/

and also -m parameter. It doesn't help.

Discussion

1 2 > >> (Page 1 of 2)
    • assigned_to: nobody --> vladut-paiu
     
  • Hi Nick,

    Can you please follow the following tutorial in order to get a memory dump, so we can further investigate this ?
    http://www.opensips.org/Resources/DocsTsMem

    Regards,
    Vlad

     
  • Nick Altmann
    Nick Altmann
    2012-04-03

    I need some time to collect information. I'll post it as soon, as it's ready.

     
  • Hi Nick,

    Also, when the issue happens again ( OpenSIPS goes to high CPU load ), please use top or any other monitoring tool to identify the OpenSIPS PID that is using more than 90% of CPU, and then do
    gdb opensips PID
    and inside gdb do
    bt full

    and send us the output of that.

    Regards,
    Vlad

     
  • Nick Altmann
    Nick Altmann
    2012-04-04

    I recompiled opensips with DBG_QM_MALLOC, but I cannot enable memlog=1 because it's production server on high load, but I enabled memdump=2.
    So, after about 8 hours of uptime, I received complains from users about non working STUN, I tested it and it really didn't work.
    But I didn't see 100% CPU load of main opensips process as earlier.
    There is gdb's "bt full output":
    #1 0x000000000042925a in main_loop (argc=<value optimized out>, argv=<value optimized out>) at main.c:984
    i = <value optimized out>
    pid = <value optimized out>
    si = <value optimized out>
    startup_done = 0x0
    chd_rank = 19
    rc = <value optimized out>
    load_p = 0x7fb56e08ca10
    #2 main (argc=<value optimized out>, argv=<value optimized out>) at main.c:1490
    cfg_log_stderr = <value optimized out>
    cfg_stream = <value optimized out>
    c = <value optimized out>
    r = <value optimized out>
    tmp = 0x1 <Address 0x1 out of bounds>
    tmp_len = <value optimized out>
    port = <value optimized out>
    proto = <value optimized out>
    options = 0x4e5e70 "f:cCm:b:l:n:N:rRvdDETSVhw:t:u:g:P:G:W:o:"
    ret = -1
    seed = 2309946016
    rfd = <value optimized out>
    __FUNCTION__ = "main"

    And mem_dump.log that opensips produced on stop. (Sent to private mail).
    Hope this helps. I'll try to dump it again and I hope to reproduce 100% CPU load.

     
  • Hi,

    Please apply the attached patch and let me know if the issue still persists.

    Regards,
    Vlad

     
  • Patch for STUN memory leaks

     
    Attachments
  • Nick Altmann
    Nick Altmann
    2012-04-10

    it seems it helped. No crashes yet.

     
  • Hi Nick,

    Thanks for testing.
    I have committed the fix on trunk, 1.8 and 1.7.

    Regards,
    Vlad

     
    • status: open --> closed-fixed
     
1 2 > >> (Page 1 of 2)