Re: [Netdisco] scheduled jobs stop

[Ricardo S. <st...@ri...>]

Also happening here. I also had exported the DB in order to install on a
new VM with new OS. Had a couple of problems that I posted but had this
same error on the logs.

Noticed all jobs queued for a couple of days and nothing running.

Last message on logs was:

Argument "" isn't numeric in read at
/home/netdisco/perl5/lib/perl5/MCE/Queue.pm line 1439, <__ANONIO__> line 1.
Sereal: Error: Bad Sereal header: Not a valid Sereal document. at offset 1
of input at srl_decoder.c line 580 at
/home/netdisco/perl5/lib/perl5/MCE/Queue.pm line 1445, <__ANONIO__> line 1.

Restarting it seems to get the jobs running again.



On Mon, Jan 27, 2020 at 10:54 AM marco via netdisco-users <
net...@li...> wrote:

> Hi there
>
> I had set up a new ND2 host on debian buster some weeks ago
> for experimental purpose
> I have another ND2 host up and running since years
>
> Software        Version
> App::Netdisco   2.44.4
> SNMP::Info      3.70
> DB Schema       61
> PostgreSQL      12.00.1
> Perl    5.28.1
>
> I restore db from another ND2
> and copy deployment.yml
> It worked
>
> But I noticed that it stops running the scheduled jobs after some times
> (days)
> I had to restart netdisco-backend,
>
>
> here some info I collect
>
>     from netdisco-backend.log
>     ...
>     [392] 2020-01-24 15:15:18 debug mgr (2): getting potential jobs for 1
> workers
>     [2700] 2020-01-24 15:15:18 debug  [172.17.185.50] arpnip - processed
> 373 ARP Cache entries
>     [2700] 2020-01-24 15:15:18 debug  [172.17.185.50] arpnip - processed 0
> IPv6 Neighbor Cache entries
>     [2700] 2020-01-24 15:15:18  info pol (3): wrapping up arpnip
> job(22423168) - status done at Fri Jan 24 16:15:18 2020
>     [392] 2020-01-24 15:15:18 debug getsome: cancelled 0E0 duplicate(s) of
> job 22423235
>     [392] 2020-01-24 15:15:18  info mgr (2): job 22423235 booked out for
> this processing node
>     Argument "PID_2700" isn't numeric in read at
> /home/netdisco/perl5/lib/perl5/MCE/Queue.pm line 477, <__ANONIO__> line
> 31470.
>     Sereal: Error: Bad Sereal header: Not a valid Sereal document. at
> offset 1 of input at srl_decoder.c line 580 at
> /home/netdisco/perl5/lib/perl5/MCE/Queue.pm line 480, <__ANONIO__> line
> 31470.
>
>     root@deb-netdisco:~# systemctl status netdisco-backend.service
>     ● netdisco-backend.service - Netdisco Backend Service
>     Loaded: loaded (/etc/systemd/system/netdisco-backend.service; enabled;
> vendor preset: enabled)
>     Active: active (running) since Fri 2020-01-24 09:53:03 CET; 3 days ago
>     Process: 110 ExecStart=/home/netdisco/bin/netdisco-backend start
> (code=exited, status=0/SUCCESS)
>     Main PID: 216 (netdisco-backen)
>         Tasks: 2 (limit: 4915)
>     Memory: 143.0M
>     CGroup: /system.slice/netdisco-backend.service
>             └─216 netdisco-backend
>
>     gen 24 09:53:02 deb-netdisco systemd[1]: Starting Netdisco Backend
> Service...
>     gen 24 09:53:03 deb-netdisco netdisco-backend[110]: Netdisco Backend
>                                             [Started]
>     gen 24 09:53:03 deb-netdisco netdisco-backend[110]: config watcher:
> watching /home/netdisco/environments for updates.
>     gen 24 09:53:03 deb-netdisco systemd[1]: Started Netdisco Backend
> Service.
>     gen 24 10:01:48 deb-netdisco netdisco-backend[110]: --
> /home/netdisco/environments/deployment.yml updated.
>     gen 24 10:01:48 deb-netdisco netdisco-backend[110]: config watcher:
> sending TERM to the server (pid:217)...
>
>     root@deb-netdisco:~# ps aux | grep netd
>     netdisco   216  0.0  0.3  22840 16008 ?        S    gen24   6:19
> netdisco-backend
>     netdisco   281  0.0  0.3  20744 13680 ?        S    gen24   0:00 perl
> /home/netdisco/bin/netdisco-web start
>     netdisco   282  0.0  0.3  22152 16696 ?        S    gen24   0:47
> starman master --disable-keepalive --user 1001 --group 1001
> /home/netdisco/perl5/bin/netdisco-web-fg
>     netdisco   372  0.0  0.0      0     0 ?        Z    gen24   0:16 [nd2:
> master] <defunct>
>     netdisco   373  0.0  2.7 135148 117200 ?       S    gen24   0:06
> starman worker --disable-keepalive --user 1001 --group 1001
> /home/netdisco/perl5/bin/netdisco-web-fg
>     netdisco   374  0.0  2.8 136000 118000 ?       S    gen24   0:06
> starman worker --disable-keepalive --user 1001 --group 1001
> /home/netdisco/perl5/bin/netdisco-web-fg
>     netdisco   375  0.0  2.7 133744 115940 ?       S    gen24   0:06
> starman worker --disable-keepalive --user 1001 --group 1001
> /home/netdisco/perl5/bin/netdisco-web-fg
>     netdisco   376  0.0  2.8 137420 119504 ?       S    gen24   0:06
> starman worker --disable-keepalive --user 1001 --group 1001
> /home/netdisco/perl5/bin/netdisco-web-fg
>     netdisco   377  0.0  2.7 133792 115996 ?       S    gen24   0:05
> starman worker --disable-keepalive --user 1001 --group 1001
> /home/netdisco/perl5/bin/netdisco-web-fg
>     root      3405  0.0  0.0   6096   824 pts/0    S+   10:59   0:00 grep
> netd
>
>
> after stop and start
>     root@deb-netdisco:~# systemctl start netdisco-backend.service
>
> it seems to work again
>     [392] 2020-01-24 15:15:18  info mgr (2): job 22423235 booked out for
> this processing node
>     Argument "PID_2700" isn't numeric in read at
> /home/netdisco/perl5/lib/perl5/MCE/Queue.pm line 477, <__ANONIO__> line
> 31470.
>     Sereal: Error: Bad Sereal header: Not a valid Sereal document. at
> offset 1 of input at srl_decoder.c line 580 at
> /home/netdisco/perl5/lib/perl5/MCE/Queue.pm line 480, <__ANONIO__> line
> 31470.
>     [3429] 2020-01-27 10:10:08  warn App::Netdisco 2.044004 backend
>     [3429] 2020-01-27 10:10:08  info resolving backend hostname...
>     [3433] 2020-01-27 10:10:08  info applying role Scheduler to worker 1
>     [3436] 2020-01-27 10:10:08  info applying role Poller to worker 4
>     ...
>
>
> _______________________________________________
> Netdisco mailing list
> net...@li...
> https://sourceforge.net/p/netdisco/mailman/netdisco-users/



-- 
°(((=((===°°°(((================================================