I'm working with a database of 5217 nodes, imported from BGP routing tables and cisco configs (or CSVs created from Cisco configs, with subnet and host level entries). When expanding or collapsing branches on the tree a branch being expanded will sometimes disappear. For branches collapsed the observed behavior is more varied. Sometimes the branch which is collapsed will disappear, but sometimes it will disappear and take with it all of the branches at that same level of that root which are lower in the tree (collapsing 10.10.10.0/24 will get rid of all 10.10.11.x and higher branches, if the 10.10.10.0/24 is itself a branch of a 10.0.0.0/8 branch), but might leave 172.x.x.x branches and sub-branches). And sometimes collapsing 10.10.10.0/24 will cause all branches below it to disappear, including branches which are attached directly to the root.
Private Root
+10.0.0.0/8
-172.16.0.0/12
+172.16.0.0/16 Region 1
-172.17.0.0/16 Region 1 Site 1
-172.17.10.0/24 Region 1 Site 1 Subnet 1
172.17.10.1/32 Router interface
172.17.10.5/32 DHCP server
172.17.10.6/32 DNS server
+172.17.11.0/24
+192.168.0.0/16
With a tree like above collapsing 172.17.10.0/24 might cause 172.17.10.0/24 to disappear, or it might cause 172.17.11.0/24 to disappear, or it might cause both 172.17.10.0/24 and 172.17.11.0/24 to disappear, or it might cause all of that PLUS 192.168.0.0/16 will disappear. If the page is reloaded/refreshed these nodes/leafs/branches GENERALLY come back, but occasionally a logout/login is required.
Apache Version & build info
Database counts (per root) and min-max network ID
Import and root "distribution" info
perl and mysql versions
OS & hardware info
Locale support is disabled to try to get around a similar problem I was seeing on a debian 1 Ghz machine w/ 1GB memory and MOST of the network entries (<4000 networks, but more deletes/adds so max net ID was 14000+). On the debian machine it eventually started logging me out anything I tried to do anything other than expand/contract the tree. The database from the Debian machine was exported to an Ubuntu VM where expanding/contracting nodes would give an error:
Software error: Can't call method "param" on an undefined value at /var/HaCi/modules/HaCi/GUI/init.pm line 34. For help, please send mail to the webmaster ([no address given] (mailto:[no%20address%20given])), giving this error message and the time and date of the error.
disabling locale support (not needed for me) got rid of that error, and most of the problem support until the database was reimported from the source files on the CentOS box.
Hi Amadon,
thanks for the detailed description of your problem. Sadly, it's an ugly issue, because it's hard to provoke that "disappear error". I'll do my best to investigate that issue.
Your "locale" error seems to be a session error. In line 34 of "/var/HaCi/modules/HaCi/GUI/init.pm" it tries to get a parameter of the established session. But the session is undefined. That should not happen. Are there any other errors or hints in the error logfile?
Greets Lars
I've just gotten another Param error, in line 51 of init.pm, while adding and editing a network. Here are the last several UNIQUE lines of var/log/httpd/error_log
[Mon Feb 28 22:57:41 2011] -e: Use of uninitialized value in subroutine entry at /var/HaCi/modules/HaCi/GUI/gettext.pm line 32.
[Mon Feb 28 22:57:53 2011] -e: flush(): couldn't remove session data: remove(): couldn't unlink '/tmp/cgisess_40b5760432c83f93a95bd4ad8c8eb5e2': No such file or directory at /var/HaCi/modules/HaCi/HaCi.pm line 714.
[Mon Feb 28 22:57:53 2011] [error] [Mon Feb 28 22:57:53 2011] -e: Can't call method "param" on an undefined value at /var/HaCi/modules/HaCi/GUI/init.pm line 51.\n
[Mon Feb 28 22:57:57 2011] -e: Use of uninitialized value in subroutine entry at /var/HaCi/modules/HaCi/GUI/gettext.pm line 32.
[Mon Feb 28 22:59:26 2011] -e: flush(): couldn't remove session data: remove(): couldn't unlink '/tmp/cgisess_40b5760432c83f93a95bd4ad8c8eb5e2': No such file or directory at /var/HaCi/modules/HaCi/HaCi.pm line 714.
[Mon Feb 28 22:59:26 2011] [error] [Mon Feb 28 22:59:26 2011] -e: Can't call method "param" on an undefined value at /var/HaCi/modules/HaCi/GUI/init.pm line 51.\n
[Mon Feb 28 22:59:27 2011] -e: Use of uninitialized value in subroutine entry at /var/HaCi/modules/HaCi/GUI/gettext.pm line 32.
Next time I have a branches disappear I'll check that same log again.
Okay, those uninitialized warnings appear because you disabled locale support. But those errors shouldn't be the reason for your disappear errors. I've attached a patch for the file 'modules/HaCi/GUI/init.pm' that should fix those warnings.
Those session errors are not okay:
- flush(): couldn't remove session data: remove(): couldn't unlink '/tmp/cgisess_40b5760432c83f93a95bd4ad8c8eb5e2': No such file or directory
- Can't call method "param" on an undefined value
Has the executing user (www-data?) enough permissions to write into /tmp?
If you don't want the user write into /tmp, I can make you a patch so that you can define a session directory.
Greets, Lars
Patch for locale warnings (modules/HaCi/GUI/init.pm)
ls -l / | grep tmp
drwxrwxrwt 6 root root 4096 Mar 1 04:02 tmp
plus there are a bunch of session files...
cgisess_2b546c0542c4949f6b338f4923f44f30
cgisess_2d67956350355da7503f056aa764138a
cgisess_3222dd76bd2bc3d96dfd4838aa46b260
cgisess_379660bc303df5eec781bef122cfd55c
cgisess_426bb52d32e551d27322adc3dc6c746d
cgisess_455633f874db22b4580d8ccc1e2f28a0
cgisess_538465621485bbe1e2f75821bfd55efa
cgisess_5c2e0994357636c7323f06b7b3ab1170
cgisess_6dc42ff68d15dd8ccf21831dbdf7d91e
cgisess_703ce8a1f7ceb2cc030e8c2f7a51befd
cgisess_70ea0bba388f2563b26b0619826a86d8
cgisess_791cb030cd78096a4183db53cc770bd7
cgisess_8c36c1f6f1c7cde3e697faf1d9c5959b
cgisess_8dabfebd29a08fbd94665926e92ee529
cgisess_9eacfe2efd8f3deeb8505b2675d3ec3b
cgisess_a77b6b2aa1e558a76580385b8db715e3
cgisess_d881577948cefc33d71b6813e1b53e52
cgisess_d963bed76707cb5ff3e3b335eb9583f5
cgisess_e026e268bc0b927bd48711a427ca1ae2
cgisess_e9ac30faa955cea6f6af8879fc35ff9b
cgisess_f70208d19c09e2fe4479af7760379bd1
cgisess_f94c37c897481e6021fd51b57d9852ae
cgisess_testname
Over the last couple of days the branches of the tree only disappear when the error on line 51 is generated. I haven't had them disappear without throwing an error since last week.
Here are the printable bytes from the request which generated an error on line 51, followed by the printable bytes from the request on reload/refresh which worked without error (copied from wireshark). At the end we have the Apache error log at the time of the error. Hostnames have been removed. Several times before the error which was captured a similar error occured. 2 times the same /tmp/cgisess_40b5760432c83f93a95bd4ad8c8eb5e2 file was referenced, and at least once /tmp/cgisess_835e2b024aa7f18527e51e547fbab877 was the cause of the error. This was all during a single login session, so I would expect the session IDs to be the same. I'd also expect them to match the CGISESS mentioned in the cookies from the capture, but they don't.
Caused error:
POST /cgi-bin/HaCi.cgi HTTP/1.1
Host: ****.****.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.14) Gecko/20110218 Firefox/3.6.14 ( .NET CLR 3.5.30729; .NET4.0C)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
Referer: http://\****.****.com/cgi-bin/HaCi.cgi
Cookie: a62f1d92f4f620b73820dadb1c52e9c0; ipplanCustomer=36; ipplanGroup=WANAdmin; CGISESSID=d13ec1b7cc8f1582374f9336b49fd6d0
Content-Type: application/x-www-form-urlencoded
Content-Length: 101
buttonTmp=&rootIDJump=2&jumpTo=%3CIP+address%3E&editNet=1&netID=12405&rootID=0&networkDec=0&fillNet=0
Didn't cause error:
POST /cgi-bin/HaCi.cgi HTTP/1.1
Host: ****.****.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.14) Gecko/20110218 Firefox/3.6.14 ( .NET CLR 3.5.30729; .NET4.0C)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
Referer: http://\****.****.com/cgi-bin/HaCi.cgi
Cookie: a62f1d92f4f620b73820dadb1c52e9c0; ipplanCustomer=36; ipplanGroup=WANAdmin; CGISESSID=d13ec1b7cc8f1582374f9336b49fd6d0
Cache-Control: max-age=0
Content-Type: application/x-www-form-urlencoded
Content-Length: 101
buttonTmp=&rootIDJump=2&jumpTo=%3CIP+address%3E&editNet=1&netID=12405&rootID=0&networkDec=0&fillNet=0
[Wed Mar 2 12:26:32 2011] -e: flush(): couldn't remove session data: remove(): couldn't unlink '/tmp/cgisess_40b5760432c83f93a95bd4ad8c8eb5e2': No such file or directory at /var/HaCi/modules/HaCi/HaCi.pm line 714.
[Wed Mar 02 12:26:32 2011] [error] [Wed Mar 2 12:26:32 2011] -e: Can't call method "param" on an undefined value at /var/HaCi/modules/HaCi/GUI/init.pm line 51.\n
The CGI::Session errors seem to be a bug in the perl module. I also get these errors, but very unpredictable. I'll check, if I can substitute this module by another.
I hope the disappearing issue is already fixed by the newest version.
Kind regards,
Lars