Re: [munin-users] munin-async update issue

[Renaud M. <ren...@gm...>]

Hello
Le 26/05/2015 à 15:42, Fatma Lejmi a écrit:
> I'm using a linux system CentOS release 6.4. I started using munin-asyncd
> 2.0.25-2.el6.
> As soon as I start the asycnd service, some rrd files are no more updated.
> I tried to fetch the information manually using "spoolfetch <timestamp>"
> and everything works fine.
> Does anyone have an idea please?

I think it is a kind of race condition.
Your servers are probably synchronized using ntp.
The package you use come from EPEL, that release -2 backported a patch from
the 2.1 branch:
https://bugzilla.redhat.com/show_bug.cgi?id=1149949
which cause munin-asyncd to synchronize on wall clock (to execute exactly
at :00 :05 :10 :15 ...), so if your servers are synchronized, that is
almost exactly when the master will connect to fetch the data, so it will
spoolfetch while munin-asyncd is updating the files.
The plugins not getting updated are those that are not yet updated when
master fetches the data.

It seems the problem (but I am not an expert) is that munin-asyncd takes the
timestamp when it starts fetching the plugins and will write that timestamp
for each fetched data, even if it is a few seconds after, so the data
actually fetched by munin-asyncd after the master spoolfetched the data
will be written with a timestamp that is before the spoolfetch time, so the
next time they will not be eligible for the spoolfetch.

The problem can happen with a vanilla munin 2.0 if munin-asyncd happen to
be started at the moment the master connect.

My workaround for the few CentOS servers I have was to patch munin-asyncd
and change the line 157 from

		my $should_have_been = $when - ($when % $plugin_rate);

to

		my $should_have_been = $when - ($when % $plugin_rate)+210;

which will make munin-asyncd run 210s after, or 90s before, the master
connect to fetch the data, which leave plenty of time for the plugins to
run before the master connects.


I should have filed a bug report for this, but didn't have time to do so
and forgot about it.

regards
-- 
Renaud MICHEL
defimedia S.A.