Re: [PacketFence-users] pfsetvlan high cpu, innodb deadlocks

[Muller, D. (HEALTH) <dan...@he...>]

> Allow me to ask a seemingly dumb question: 
>
> Did you reboot after restoring the snapshot?

Not dumb at all; I'm currently using a snapshot without memory so when reverted the system boots.

Troubleshooting it this morning I rebooted it about twice before I had the network configuration sorted out, (there is an interaction between VMware MAC auto-assignment and linux UDEV).



> And another:
> How many processes do you have reading from snmptrapd.log?
> I.e.: 
> # lsof /usr/local/pf/logs/snmptrapd.log
>
>
>
> This looks to me like a case of multiple (stale) processes trying to read and write from the same resources.
> 
> Also check the output of ?ps? for multiple instances of pfmon and pfdhcplistener.

I think I see the correct number, do not think this is the case;  

I did try stopping packetfence and mysql, rotating the logs and then starting up with empty logs because I was concerned about old entries in snmptrapd.log ...

However I do continue to see the problem even after starting up mysql, then packetfence with empty logs.

---
[root@pkfn pf]# lsof /usr/local/pf/logs/snmptrapd.log
COMMAND    PID USER   FD   TYPE DEVICE SIZE/OFF   NODE NAME
snmptrapd 3900 root    3w   REG  253,1  1607325 393053 /usr/local/pf/logs/snmptrapd.log
pfsetvlan 3906 root    4r   REG  253,1  1607325 393053 /usr/local/pf/logs/snmptrapd.log
[root@pkfn pf]# uptime
 13:39:07 up  4:11,  3 users,  load average: 1.43, 1.21, 1.12
[root@pkfn pf]# ps -ef |egrep "pfmon|pfdhcplistener"
root      3874     1  0 11:27 ?        00:00:00 pfdhcplistener_eth0.110
root      3875     1  0 11:27 ?        00:00:00 pfdhcplistener_eth0.111
root      3876     1  0 11:27 ?        00:00:06 pfdhcplistener_eth1
root      3887     1  0 11:27 ?        00:00:00 pfmon
root      3888  3887  0 11:27 ?        00:00:00 pfmon - iplog cleanup
root      3889  3887  0 11:27 ?        00:00:00 pfmon - locationlog cleanup
root      3890  3887  0 11:27 ?        00:00:00 pfmon - node cleanup
root      3891  3887  0 11:27 ?        00:00:00 pfmon - traplog cleanup
root      3892  3887  0 11:27 ?        00:00:01 pfmon - Expire httpd.admin sessions
root      3893  3887  0 11:27 ?        00:00:01 pfmon - Expire httpd.portal sessions
root      3894  3887  0 11:27 ?        00:00:00 pfmon - checking registered nodes for expiration
root      3895  3887  0 11:27 ?        00:00:00 pfmon - checking violations for expiration
root      3896  3887  0 11:27 ?        00:00:00 pfmon - checking accounting data for potential bandwidth abuse
root      3897  3887  0 11:27 ?        00:00:00 pfmon - Provisioner polling enforcement
root      3898  3887  0 11:27 ?        00:00:00 pfmon - Account maintenance
root      4229  4196  0 12:01 pts/2    00:00:00 tail -f pfmon.log
root      4230  4196  0 12:01 pts/2    00:00:00 tail -f pfdhcplistener.log
root      4234  4196  0 12:01 pts/2    00:00:00 tail -f pfmon.log
root      5066  2099  0 13:41 pts/0    00:00:00 egrep pfmon|pfdhcplistener