Menu

#5 Branches disappear when branches are expanded or collapsed

open
nobody
None
5
2014-10-27
2011-02-23
Amadon
No

I'm working with a database of 5217 nodes, imported from BGP routing tables and cisco configs (or CSVs created from Cisco configs, with subnet and host level entries). When expanding or collapsing branches on the tree a branch being expanded will sometimes disappear. For branches collapsed the observed behavior is more varied. Sometimes the branch which is collapsed will disappear, but sometimes it will disappear and take with it all of the branches at that same level of that root which are lower in the tree (collapsing 10.10.10.0/24 will get rid of all 10.10.11.x and higher branches, if the 10.10.10.0/24 is itself a branch of a 10.0.0.0/8 branch), but might leave 172.x.x.x branches and sub-branches). And sometimes collapsing 10.10.10.0/24 will cause all branches below it to disappear, including branches which are attached directly to the root.

Private Root
+10.0.0.0/8
-172.16.0.0/12
+172.16.0.0/16 Region 1
-172.17.0.0/16 Region 1 Site 1
-172.17.10.0/24 Region 1 Site 1 Subnet 1
172.17.10.1/32 Router interface
172.17.10.5/32 DHCP server
172.17.10.6/32 DNS server
+172.17.11.0/24
+192.168.0.0/16

With a tree like above collapsing 172.17.10.0/24 might cause 172.17.10.0/24 to disappear, or it might cause 172.17.11.0/24 to disappear, or it might cause both 172.17.10.0/24 and 172.17.11.0/24 to disappear, or it might cause all of that PLUS 192.168.0.0/16 will disappear. If the page is reloaded/refreshed these nodes/leafs/branches GENERALLY come back, but occasionally a logout/login is required.

Discussion

  • Amadon

    Amadon - 2011-02-23

    Apache Version & build info

     
  • Amadon

    Amadon - 2011-02-23

    Database counts (per root) and min-max network ID

     
  • Amadon

    Amadon - 2011-02-23

    Import and root "distribution" info

     
  • Amadon

    Amadon - 2011-02-23

    perl and mysql versions

     
  • Amadon

    Amadon - 2011-02-23

    OS & hardware info

     
  • Amadon

    Amadon - 2011-02-23

    Locale support is disabled to try to get around a similar problem I was seeing on a debian 1 Ghz machine w/ 1GB memory and MOST of the network entries (<4000 networks, but more deletes/adds so max net ID was 14000+). On the debian machine it eventually started logging me out anything I tried to do anything other than expand/contract the tree. The database from the Debian machine was exported to an Ubuntu VM where expanding/contracting nodes would give an error:

    Software error: Can't call method "param" on an undefined value at /var/HaCi/modules/HaCi/GUI/init.pm line 34. For help, please send mail to the webmaster ([no address given] (mailto:[no%20address%20given])), giving this error message and the time and date of the error.

    disabling locale support (not needed for me) got rid of that error, and most of the problem support until the database was reimported from the source files on the CentOS box.

     
  • larsux

    larsux - 2011-02-25

    Hi Amadon,

    thanks for the detailed description of your problem. Sadly, it's an ugly issue, because it's hard to provoke that "disappear error". I'll do my best to investigate that issue.

    Your "locale" error seems to be a session error. In line 34 of "/var/HaCi/modules/HaCi/GUI/init.pm" it tries to get a parameter of the established session. But the session is undefined. That should not happen. Are there any other errors or hints in the error logfile?

    Greets Lars

     
  • Amadon

    Amadon - 2011-03-01

    I've just gotten another Param error, in line 51 of init.pm, while adding and editing a network. Here are the last several UNIQUE lines of var/log/httpd/error_log

    [Mon Feb 28 22:57:41 2011] -e: Use of uninitialized value in subroutine entry at /var/HaCi/modules/HaCi/GUI/gettext.pm line 32.
    [Mon Feb 28 22:57:53 2011] -e: flush(): couldn't remove session data: remove(): couldn't unlink '/tmp/cgisess_40b5760432c83f93a95bd4ad8c8eb5e2': No such file or directory at /var/HaCi/modules/HaCi/HaCi.pm line 714.
    [Mon Feb 28 22:57:53 2011] [error] [Mon Feb 28 22:57:53 2011] -e: Can't call method "param" on an undefined value at /var/HaCi/modules/HaCi/GUI/init.pm line 51.\n
    [Mon Feb 28 22:57:57 2011] -e: Use of uninitialized value in subroutine entry at /var/HaCi/modules/HaCi/GUI/gettext.pm line 32.
    [Mon Feb 28 22:59:26 2011] -e: flush(): couldn't remove session data: remove(): couldn't unlink '/tmp/cgisess_40b5760432c83f93a95bd4ad8c8eb5e2': No such file or directory at /var/HaCi/modules/HaCi/HaCi.pm line 714.
    [Mon Feb 28 22:59:26 2011] [error] [Mon Feb 28 22:59:26 2011] -e: Can't call method "param" on an undefined value at /var/HaCi/modules/HaCi/GUI/init.pm line 51.\n
    [Mon Feb 28 22:59:27 2011] -e: Use of uninitialized value in subroutine entry at /var/HaCi/modules/HaCi/GUI/gettext.pm line 32.

    Next time I have a branches disappear I'll check that same log again.

     
  • larsux

    larsux - 2011-03-01

    Okay, those uninitialized warnings appear because you disabled locale support. But those errors shouldn't be the reason for your disappear errors. I've attached a patch for the file 'modules/HaCi/GUI/init.pm' that should fix those warnings.

    Those session errors are not okay:
    - flush(): couldn't remove session data: remove(): couldn't unlink '/tmp/cgisess_40b5760432c83f93a95bd4ad8c8eb5e2': No such file or directory
    - Can't call method "param" on an undefined value
    Has the executing user (www-data?) enough permissions to write into /tmp?
    If you don't want the user write into /tmp, I can make you a patch so that you can define a session directory.

    Greets, Lars

     
  • larsux

    larsux - 2011-03-01

    Patch for locale warnings (modules/HaCi/GUI/init.pm)

     
  • Amadon

    Amadon - 2011-03-01

    ls -l / | grep tmp
    drwxrwxrwt 6 root root 4096 Mar 1 04:02 tmp

    plus there are a bunch of session files...
    cgisess_2b546c0542c4949f6b338f4923f44f30
    cgisess_2d67956350355da7503f056aa764138a
    cgisess_3222dd76bd2bc3d96dfd4838aa46b260
    cgisess_379660bc303df5eec781bef122cfd55c
    cgisess_426bb52d32e551d27322adc3dc6c746d
    cgisess_455633f874db22b4580d8ccc1e2f28a0
    cgisess_538465621485bbe1e2f75821bfd55efa
    cgisess_5c2e0994357636c7323f06b7b3ab1170
    cgisess_6dc42ff68d15dd8ccf21831dbdf7d91e
    cgisess_703ce8a1f7ceb2cc030e8c2f7a51befd
    cgisess_70ea0bba388f2563b26b0619826a86d8
    cgisess_791cb030cd78096a4183db53cc770bd7
    cgisess_8c36c1f6f1c7cde3e697faf1d9c5959b
    cgisess_8dabfebd29a08fbd94665926e92ee529
    cgisess_9eacfe2efd8f3deeb8505b2675d3ec3b
    cgisess_a77b6b2aa1e558a76580385b8db715e3
    cgisess_d881577948cefc33d71b6813e1b53e52
    cgisess_d963bed76707cb5ff3e3b335eb9583f5
    cgisess_e026e268bc0b927bd48711a427ca1ae2
    cgisess_e9ac30faa955cea6f6af8879fc35ff9b
    cgisess_f70208d19c09e2fe4479af7760379bd1
    cgisess_f94c37c897481e6021fd51b57d9852ae
    cgisess_testname

     
  • Amadon

    Amadon - 2011-03-02

    Over the last couple of days the branches of the tree only disappear when the error on line 51 is generated. I haven't had them disappear without throwing an error since last week.

    Here are the printable bytes from the request which generated an error on line 51, followed by the printable bytes from the request on reload/refresh which worked without error (copied from wireshark). At the end we have the Apache error log at the time of the error. Hostnames have been removed. Several times before the error which was captured a similar error occured. 2 times the same /tmp/cgisess_40b5760432c83f93a95bd4ad8c8eb5e2 file was referenced, and at least once /tmp/cgisess_835e2b024aa7f18527e51e547fbab877 was the cause of the error. This was all during a single login session, so I would expect the session IDs to be the same. I'd also expect them to match the CGISESS mentioned in the cookies from the capture, but they don't.

    Caused error:
    POST /cgi-bin/HaCi.cgi HTTP/1.1
    Host: ****.****.com
    User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.14) Gecko/20110218 Firefox/3.6.14 ( .NET CLR 3.5.30729; .NET4.0C)
    Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
    Accept-Language: en-us,en;q=0.5
    Accept-Encoding: gzip,deflate
    Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
    Keep-Alive: 115
    Connection: keep-alive
    Referer: http://\****.****.com/cgi-bin/HaCi.cgi
    Cookie: a62f1d92f4f620b73820dadb1c52e9c0; ipplanCustomer=36; ipplanGroup=WANAdmin; CGISESSID=d13ec1b7cc8f1582374f9336b49fd6d0
    Content-Type: application/x-www-form-urlencoded
    Content-Length: 101

    buttonTmp=&rootIDJump=2&jumpTo=%3CIP+address%3E&editNet=1&netID=12405&rootID=0&networkDec=0&fillNet=0

    Didn't cause error:
    POST /cgi-bin/HaCi.cgi HTTP/1.1
    Host: ****.****.com
    User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.14) Gecko/20110218 Firefox/3.6.14 ( .NET CLR 3.5.30729; .NET4.0C)
    Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
    Accept-Language: en-us,en;q=0.5
    Accept-Encoding: gzip,deflate
    Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
    Keep-Alive: 115
    Connection: keep-alive
    Referer: http://\****.****.com/cgi-bin/HaCi.cgi
    Cookie: a62f1d92f4f620b73820dadb1c52e9c0; ipplanCustomer=36; ipplanGroup=WANAdmin; CGISESSID=d13ec1b7cc8f1582374f9336b49fd6d0
    Cache-Control: max-age=0
    Content-Type: application/x-www-form-urlencoded
    Content-Length: 101

    buttonTmp=&rootIDJump=2&jumpTo=%3CIP+address%3E&editNet=1&netID=12405&rootID=0&networkDec=0&fillNet=0

    [Wed Mar 2 12:26:32 2011] -e: flush(): couldn't remove session data: remove(): couldn't unlink '/tmp/cgisess_40b5760432c83f93a95bd4ad8c8eb5e2': No such file or directory at /var/HaCi/modules/HaCi/HaCi.pm line 714.
    [Wed Mar 02 12:26:32 2011] [error] [Wed Mar 2 12:26:32 2011] -e: Can't call method "param" on an undefined value at /var/HaCi/modules/HaCi/GUI/init.pm line 51.\n

     
  • larsux

    larsux - 2014-10-27

    The CGI::Session errors seem to be a bug in the perl module. I also get these errors, but very unpredictable. I'll check, if I can substitute this module by another.

    I hope the disappearing issue is already fixed by the newest version.

    Kind regards,
    Lars