I've been trying to get one of our switches added in to netdb and I'm running into a bit of a hitch. The switches don't show up when I try to query based on a name and when I query one of the MACs I found in data/arptable.txt the switch name shows up under "controller" and the switch port shows up under "access point". It's like it's detecting that the switch is a wireless access point or something. (These are Juniper devices if that makes any difference.)
Any ideas...?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Also, what scraper are you using? I think I see the bug and it depends on the scraper itself, whether it's pulling the vlan id from the switch or not. Let's see how that goes but I think I see the larger issue as well.
Thanks
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I made the changes in my /usr/lib/cgi-bin/netdb.pl file and it didn't seem to make a difference.
Edit: I'm assuming I don't have to run netdbctl.pl first to see a change since it's in the cgi-bin folder. I'll try that and see if it makes a difference.
Last edit: mbrinkho 2015-03-25
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
OK, so lets see what's in the database next. Try logging in as the mysql root user or the netdbadmin user from the /etc/netdb.conf file:
mysql -u netdbadmin -p
use netdb;
select * from switchports where type = 'wifi';
See if you get results from that or not. If not, then I'm somehow flagging the wifi flag outside of the database data. If you're getting wifi data in the database, then it's a scraper/import issue.
Thanks
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hmm, so that is confusing. I would try removing this section altogether then in netdb.cgi.pl, and make sure you are editing the right copy that is linked to your cgi-bin/. That’s the only place that variable is set.
if ( $ptext[0]{type} eq "wifi" ) {
$wifi = 1;
}
See how that works and let me know if somehow the wrong file was edited. Otherwise I’m fairly confused.
netdb@netdb:/opt/netdb$ ls -l /usr/lib/cgi-bin/netdb.pl
lrwxrwxrwx 1 root root 23 Aug 7 2012 /usr/lib/cgi-bin/netdb.pl -> /opt/netdb/netdb.cgi.pl
And here is the code
Check for whether data is standard switchport data or wifi data
if ( $ptext[0]{type} eq "wifi" ) {
$wifi = 1;
}
Any other db queries I can run to help diagnose it?
I think we are making progress. The names of the new switches I added that are being problematic all start with "BF".
In the NetDB web UI I enter "bf*" into the search field and select "Switch Report on a Name" and it returns one line. When I run
SELECT * FROM netdb.switchports where switch like 'bf%' order by switch;
It returns 396 rows. If I take one of the MAC addresses that that query returns and copy/paste it into the NetDB search field and choose "Device by IP, MAC or hostname" it now does show up with Switch and Port columns. If I put the "|| $ptext[0]{wifi}" back into the line of the netdb.pl file it goes back to showing the Controller and Access Point columns.
So the wifi thing may be solved but I still don't get why the new switches aren't showing up on the switch report.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
OK, so this is getting interesting. When you do that select * on the bf% above, is the type field null or perhaps showing wifi? I don't understand how {type} wifi is being flagged, when that should come directly from the database. I may be manipulating it somewhere but I don't see where that could be happening. Getting rid of the wifi capability does solve that issue for you.
As for the switch search, have you tried it case sensitive, BF*? That shouldn't matter but just wondering. What's a full name of one of those switches? Perhaps there's a special character in there causing many of the issues.
Thanks
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The 'type' field from the switchports table is null in all of the results. I tried to avoid characters that might cause weird results when I named the switches, one example would be 'bf227-10-34-3-252' or 'bf274-10-33-9-252'.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
We just added some more switches to our NetDB and we are getting some of them showing up as switches now but not all. Are there any scraper logs that maybe I could send your way to help out with this?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Wed Apr 1 17:20:12 2015: netdbctl(11140): SSHProcessError The ssh process was terminated. at /usr/lib/perl5/NetDBHelper.pm line 736
Wed Apr 1 17:20:14 2015: netdbctl(11140): SSHProcessError The ssh process was terminated. at /usr/lib/perl5/NetDBHelper.pm line 736
Wed Apr 1 17:20:19 2015: netdbctl(11140): junosscraper.pl(11984): |Warning|: No mac-address table data received from BF000-10-34-71-2: Use netdbc
tl -debug 2 for more info, or disable mac-address tables on BF000-10-34-71-2 in the devicelist.csv with nomac if mac table unsupported on this dev
ice.
Wed Apr 1 17:20:24 2015: netdbctl(11140): SSHProcessError The ssh process was terminated. at /usr/lib/perl5/NetDBHelper.pm line 736
Wed Apr 1 17:20:35 2015: netdbctl(11140): SSHProcessError The ssh process was terminated. at /usr/lib/perl5/NetDBHelper.pm line 736
Wed Apr 1 17:22:13 2015: netdbctl(11140): Can't call method "read_all" on an undefined value at /usr/lib/perl5/NetDBHelper.pm line 734.
Wed Apr 1 17:22:14 2015: netdbctl(11140): Can't call method "read_all" on an undefined value at /usr/lib/perl5/NetDBHelper.pm line 734.
Wed Apr 1 17:22:36 2015: netdbctl(11140): Can't call method "read_all" on an undefined value at /usr/lib/perl5/NetDBHelper.pm line 734.
Wed Apr 1 17:24:02 2015: netdbctl(11140): Can't call method "read_all" on an undefined value at /usr/lib/perl5/NetDBHelper.pm line 734.
Wed Apr 1 17:24:04 2015: netdbctl(11140): Can't call method "read_all" on an undefined value at /usr/lib/perl5/NetDBHelper.pm line 734.
Wed Apr 1 17:24:05 2015: netdbctl(11140): Can't call method "read_all" on an undefined value at /usr/lib/perl5/NetDBHelper.pm line 734.
Wed Apr 1 17:24:06 2015: netdbctl(11140): Can't call method "read_all" on an undefined value at /usr/lib/perl5/NetDBHelper.pm line 734.
Wed Apr 1 17:24:06 2015: netdbctl(11140): Importing 18221 intstatus entries in to switchstatus table
Wed Apr 1 17:40:15 2015: netdbctl(11140): Importing 11800 MAC entries in to switchports table
Wed Apr 1 17:54:36 2015: netdbctl(11140): Importing 1 neighbor discovery entries in to neighbor table
Wed Apr 1 17:54:37 2015: netdbctl(11140): Can't use an undefined value as a HASH reference at /usr/lib/perl5/NetDB.pm line 2198.
Wed Apr 1 17:54:37 2015: netdbctl(11140): Importing 9957 ARP Entries in to ipmac table
Wed Apr 1 18:06:55 2015: netdbctl(11140): NetDB update complete
Wed Apr 1 19:12:01 2015: netdbctl(14456): Running netdbscraper on devices
Wed Apr 1 19:14:02 2015: netdbctl(14456): Can't call method "read_all" on an undefined value at /usr/lib/perl5/NetDBHelper.pm line 734.
Wed Apr 1 19:14:03 2015: netdbctl(14456): Can't call method "read_all" on an undefined value at /usr/lib/perl5/NetDBHelper.pm line 734.
Wed Apr 1 19:14:13 2015: netdbctl(14456): Can't call method "read_all" on an undefined value at /usr/lib/perl5/NetDBHelper.pm line 734.
Wed Apr 1 19:14:25 2015: netdbctl(14456): Can't call method "read_all" on an undefined value at /usr/lib/perl5/NetDBHelper.pm line 734.
Wed Apr 1 19:14:33 2015: netdbctl(14456): Can't call method "read_all" on an undefined value at /usr/lib/perl5/NetDBHelper.pm line 734.
Wed Apr 1 19:14:34 2015: netdbctl(14456): Can't call method "read_all" on an undefined value at /usr/lib/perl5/NetDBHelper.pm line 734.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I've been trying to get one of our switches added in to netdb and I'm running into a bit of a hitch. The switches don't show up when I try to query based on a name and when I query one of the MACs I found in data/arptable.txt the switch name shows up under "controller" and the switch port shows up under "access point". It's like it's detecting that the switch is a wireless access point or something. (These are Juniper devices if that makes any difference.)
Any ideas...?
Hey, I think this is related to the wifi data string being either empty or null. Try changing this line in the cgi netdb.pl from this:
To this:
Let me know the results of that and I will go from there.
Jonathan
Also, what scraper are you using? I think I see the bug and it depends on the scraper itself, whether it's pulling the vlan id from the switch or not. Let's see how that goes but I think I see the larger issue as well.
Thanks
I'll give that a try. I'm using the Junos scraper.
I made the changes in my /usr/lib/cgi-bin/netdb.pl file and it didn't seem to make a difference.
Edit: I'm assuming I don't have to run netdbctl.pl first to see a change since it's in the cgi-bin folder. I'll try that and see if it makes a difference.
Last edit: mbrinkho 2015-03-25
OK, so lets see what's in the database next. Try logging in as the mysql root user or the netdbadmin user from the /etc/netdb.conf file:
mysql -u netdbadmin -p
use netdb;
select * from switchports where type = 'wifi';
See if you get results from that or not. If not, then I'm somehow flagging the wifi flag outside of the database data. If you're getting wifi data in the database, then it's a scraper/import issue.
Thanks
Here's what I get running as mysql root (copied and pasted just in case I got the syntax wrong.)
mysql> select * from switchports where type = 'wifi';
Empty set (0.01 sec)
Hmm, so that is confusing. I would try removing this section altogether then in netdb.cgi.pl, and make sure you are editing the right copy that is linked to your cgi-bin/. That’s the only place that variable is set.
if ( $ptext[0]{type} eq "wifi" ) {
$wifi = 1;
}
See how that works and let me know if somehow the wrong file was edited. Otherwise I’m fairly confused.
Jonathan
Yeah that's the file I edited..
And here is the code
Any other db queries I can run to help diagnose it?
Comment out that section altogether or set $wifi = 0 and see if it goes away. If it does then I’ve got a mystery to solve.
Thanks
I think we are making progress. The names of the new switches I added that are being problematic all start with "BF".
In the NetDB web UI I enter "bf*" into the search field and select "Switch Report on a Name" and it returns one line. When I run
It returns 396 rows. If I take one of the MAC addresses that that query returns and copy/paste it into the NetDB search field and choose "Device by IP, MAC or hostname" it now does show up with Switch and Port columns. If I put the "|| $ptext[0]{wifi}" back into the line of the netdb.pl file it goes back to showing the Controller and Access Point columns.
So the wifi thing may be solved but I still don't get why the new switches aren't showing up on the switch report.
OK, so this is getting interesting. When you do that select * on the bf% above, is the type field null or perhaps showing wifi? I don't understand how {type} wifi is being flagged, when that should come directly from the database. I may be manipulating it somewhere but I don't see where that could be happening. Getting rid of the wifi capability does solve that issue for you.
As for the switch search, have you tried it case sensitive, BF*? That shouldn't matter but just wondering. What's a full name of one of those switches? Perhaps there's a special character in there causing many of the issues.
Thanks
The 'type' field from the switchports table is null in all of the results. I tried to avoid characters that might cause weird results when I named the switches, one example would be 'bf227-10-34-3-252' or 'bf274-10-33-9-252'.
We just added some more switches to our NetDB and we are getting some of them showing up as switches now but not all. Are there any scraper logs that maybe I could send your way to help out with this?
Looking at the log file I'm seeing a lot of this:
Wed Apr 1 17:20:12 2015: netdbctl(11140): SSHProcessError The ssh process was terminated. at /usr/lib/perl5/NetDBHelper.pm line 736
Wed Apr 1 17:20:14 2015: netdbctl(11140): SSHProcessError The ssh process was terminated. at /usr/lib/perl5/NetDBHelper.pm line 736
Wed Apr 1 17:20:19 2015: netdbctl(11140): junosscraper.pl(11984): |Warning|: No mac-address table data received from BF000-10-34-71-2: Use netdbc
tl -debug 2 for more info, or disable mac-address tables on BF000-10-34-71-2 in the devicelist.csv with nomac if mac table unsupported on this dev
ice.
Wed Apr 1 17:20:24 2015: netdbctl(11140): SSHProcessError The ssh process was terminated. at /usr/lib/perl5/NetDBHelper.pm line 736
Wed Apr 1 17:20:35 2015: netdbctl(11140): SSHProcessError The ssh process was terminated. at /usr/lib/perl5/NetDBHelper.pm line 736
Wed Apr 1 17:22:13 2015: netdbctl(11140): Can't call method "read_all" on an undefined value at /usr/lib/perl5/NetDBHelper.pm line 734.
Wed Apr 1 17:22:14 2015: netdbctl(11140): Can't call method "read_all" on an undefined value at /usr/lib/perl5/NetDBHelper.pm line 734.
Wed Apr 1 17:22:36 2015: netdbctl(11140): Can't call method "read_all" on an undefined value at /usr/lib/perl5/NetDBHelper.pm line 734.
Wed Apr 1 17:24:02 2015: netdbctl(11140): Can't call method "read_all" on an undefined value at /usr/lib/perl5/NetDBHelper.pm line 734.
Wed Apr 1 17:24:04 2015: netdbctl(11140): Can't call method "read_all" on an undefined value at /usr/lib/perl5/NetDBHelper.pm line 734.
Wed Apr 1 17:24:05 2015: netdbctl(11140): Can't call method "read_all" on an undefined value at /usr/lib/perl5/NetDBHelper.pm line 734.
Wed Apr 1 17:24:06 2015: netdbctl(11140): Can't call method "read_all" on an undefined value at /usr/lib/perl5/NetDBHelper.pm line 734.
Wed Apr 1 17:24:06 2015: netdbctl(11140): Importing 18221 intstatus entries in to switchstatus table
Wed Apr 1 17:40:15 2015: netdbctl(11140): Importing 11800 MAC entries in to switchports table
Wed Apr 1 17:54:36 2015: netdbctl(11140): Importing 1 neighbor discovery entries in to neighbor table
Wed Apr 1 17:54:37 2015: netdbctl(11140): Can't use an undefined value as a HASH reference at /usr/lib/perl5/NetDB.pm line 2198.
Wed Apr 1 17:54:37 2015: netdbctl(11140): Importing 9957 ARP Entries in to ipmac table
Wed Apr 1 18:06:55 2015: netdbctl(11140): NetDB update complete
Wed Apr 1 19:12:01 2015: netdbctl(14456): Running netdbscraper on devices
Wed Apr 1 19:14:02 2015: netdbctl(14456): Can't call method "read_all" on an undefined value at /usr/lib/perl5/NetDBHelper.pm line 734.
Wed Apr 1 19:14:03 2015: netdbctl(14456): Can't call method "read_all" on an undefined value at /usr/lib/perl5/NetDBHelper.pm line 734.
Wed Apr 1 19:14:13 2015: netdbctl(14456): Can't call method "read_all" on an undefined value at /usr/lib/perl5/NetDBHelper.pm line 734.
Wed Apr 1 19:14:25 2015: netdbctl(14456): Can't call method "read_all" on an undefined value at /usr/lib/perl5/NetDBHelper.pm line 734.
Wed Apr 1 19:14:33 2015: netdbctl(14456): Can't call method "read_all" on an undefined value at /usr/lib/perl5/NetDBHelper.pm line 734.
Wed Apr 1 19:14:34 2015: netdbctl(14456): Can't call method "read_all" on an undefined value at /usr/lib/perl5/NetDBHelper.pm line 734.