Hi,

 

maybe the Nagios livestatus module is more robust. I took your script and it crashed, but only when I had hosts without a comment. I rewrote it a little bit and it works now (CentOS5.6,64,Python 2.4.3)

 

def main_function():

        host_table = send_query("GET hosts\nFilter: state = 0\nColumns: name comments\nOutputFormat: csv\n")

        for host in host_table:

                host_name, host_comment_ids = host

 

                try:

                    host_last_comment_id = host_comment_ids.split(',')[-1] #use only last comment

                    host_last_comment_id = str(int(host_last_comment_id))  # if it is not a number (or empty), then it raises an exception

                    comment = send_query("GET comments\nColumns: comment_id comment\nFilter: comment_id = "+host_last_comment_id+"\n")

                    print comment

                except Exception:

                    print host_name + " has no comments"

 

OMD[denis]:~$ python denis.py

[['16', 'kaas']]

[['2', 'schmarrn']]

omd11 has no comments

[['4', 'schmarrn']]

[['5', 'schmarrn']]

[['6', 'schmarrn']]

[['7', 'schmarrn']]

[['17', 'kaas-omd-2']]

[['9', 'schmarrn']]

[['10', 'schmarrn']]

[['11', 'schmarrn']]

[['12', 'schmarrn']]

[['13', 'schmarrn']]

[['14', 'schmarrn']]

[['15', 'schmarrn']]

 

Even if I add lots of main_function() calls to the script, I get valid output.

Can you please try it with my modifications?

 

Gerhard

 

Von: Denis GERMAIN [mailto:dt.germain@gmail.com]
Gesendet: Mittwoch, 2. November 2011 18:08
An: shinken-devel@lists.sourceforge.net
Betreff: Re: [Shinken-devel] Slight problem with livestatus module with direct queries

 

Hi again,

I might have been a little optimistic, because my prod crashed again.

This time I used python code which has been adapted from the website from Livestatus conceptor, so I guess that this time the queries are correctly done

This error is really strange, because it seems to be realted to the number of request I do in a short period of time. I managed to reproduce the error on my dev environment *relief*. Here, everything works fine until the broker crashes.

I might be totally wrong here as I don't know much of Livestatus coding in Shinken, but is it possible that the maximum number of socket has been reached or something like that?

Attached, you can see the scenario I used to reproduce the error :
I first added comments on running hosts (in order to have a certain amount of hosts with comments on them)
Then I launched my script
Livestatus answered correctly for some hosts, then crashed, like on prod env

If this is indeed a problem with max number of sockets in livestatus module, I could recode my script in order to avoid making too many queries (getting all hosts, then getting all comments, only 2 queries here), but I'd like to be sure it's really that in order to avoid further issues on prod environnement

Regards

Denis GERMAIN

2011/10/27 Olivier Hanesse <olivier.hanesse@gmail.com>

hi,

 

I used to work with netcat too, and I got some issues in the past with some netcat version.

On debian for example, you have 2 versions : 

 

netcat-openbsd - TCP/IP swiss army knife

netcat-traditional - TCP/IP swiss army knife

 

One was working, but not the other one.

Maybe you could give a try and install the other version.

 

Regards

 

Olivier

 

2011/10/27 Denis GERMAIN <dt.germain@gmail.com>

Hi,

Thanks for the answer, I had forgotten that we used netcat instead of unixcat. I have still to confirm this, but i'm think we have a problem because we're not correctly closing the connection with netcat.

I'm going to code it in python in order to do this in a proper way.

I'll keep you posted if this is not the issue but I'm pretty sure that this is our problem.

Thanks!

Denis

2011/10/26 Gerhard Lausser <Gerhard.Lausser@consol.de>

Hi,

 

which version of Shinken do you use?

I have here a 0.8 and can’t reproduce your error. There’s one host (omd-disco) which I made DOWN by submitting a passive command and which has two comments attached.

 

OMD[disco]:~$  printf "GET hosts\nFilter: state = 1\nColumns: name comments\n"|unixcat /omd/sites/disco/tmp/run/live

omd-disco;3,4

 

OMD[disco]:~$ printf 'GET comments\nColumns: comment_id comment\n' | unixcat /omd/sites/disco/tmp/run/live

3;i bi hi

4;i kos ned oft gnou sogn, i bi hi

 

OMD[disco]:~$ printf 'GET comments\nColumns: comment_id comment\nFilter: comment_id = 3\n' | unixcat /omd/sites/disco/tmp/run/live

3;i bi hi

 

Gerhard

 

 

Von: Denis GERMAIN [mailto:dt.germain@gmail.com]
Gesendet: Mittwoch, 26. Oktober 2011 16:40
An: shinken-devel@lists.sourceforge.net
Betreff: [Shinken-devel] Slight problem with livestatus module with direct queries

 

Hello list,

For historic reporting purpose, we have a developper in our team that directly execute queries to the livestatus module via commands like
printf 'GET hosts\nFilter: state = 1' | nc 10.197.64.97 50000 > /data/ftptrace/nagios/thruk.txt
printf 'GET comments\nFilter comment_id = '$commid | nc 10.197.64.97 50000 | tail -1 | awk -F";" '{print $2}'

This gives us informations about hosts down which are then reformatted in a dusty old report sent to our client.
At the beginning we used only the first request for our report, as comments were directly available from the host information given by the query. After an update, we didn't realised that the comments were not directly in the query result, but only the comment ID (which I grant you is much more "clean")

1) Problem is that now, we need to query for the comments (selecting good comment ID) and that it constantly break livestatus module (which doesn't always manage to relaunch itself)

2) We also noted that when we use the Column statement (to select only the columns we need), the livestatus module instantly crash

I realise that our problem is a little exotic, as the vast majority of livestatus users only use it through Thruk, but if something could be done (or as already been done), we would be grateful (our developper mostly :-p)

Denis GERMAIN

PS: An example traceback. I don't have much more yet, we are running Shinken on production (I already see admins frowning eyes, but it's stable enough if you don't try strange things like us) and my manager don't really like us crashing it on purpose. We'll try to reproduce on preproduction ASAP, but if this can help pinpointing the issue...

[1319536549] [broker-1] Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/shinken/modules/livestatus_broker/livestatus_broker.py", line 828, in main
    self.do_main()
  File "/usr/lib/python2.6/site-packages/shinken/modules/livestatus_broker/livestatus_broker.py", line 1052, in do_main
    response, keepalive = self.livestatus.handle_request(open_connections[socketid]['buffer'])
  File "/usr/lib/python2.6/site-packages/shinken/modules/livestatus_broker/livestatus.py", line 129, in handle_request
    request.parse_input(data)
  File "/usr/lib/python2.6/site-packages/shinken/modules/livestatus_broker/livestatus_request.py", line 96, in parse_input
    query.parse_input('\n'.join(query_cmds))
  File "/usr/lib/python2.6/site-packages/shinken/modules/livestatus_broker/livestatus_query.py", line 259, in parse_input
    self.filter_stack.put(self.make_filter(operator, attribute, reference))
  File "/usr/lib/python2.6/site-packages/shinken/modules/livestatus_broker/livestatus_query.py", line 786, in make_filter
    converter = self.find_converter(attribute)
  File "/usr/lib/python2.6/site-packages/shinken/modules/livestatus_broker/livestatus_query.py", line 102, in find_converter
    out_map = LSout_map[self.out_map_name]
KeyError: 'hosts'

 

------------------------------------------------------------------------------
The demand for IT networking professionals continues to grow, and the
demand for specialized networking skills is growing even more rapidly.
Take a complimentary Learning@Cisco Self-Assessment and learn
about Cisco certifications, training, and career opportunities.
http://p.sf.net/sfu/cisco-dev2dev
_______________________________________________
Shinken-devel mailing list
Shinken-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/shinken-devel



------------------------------------------------------------------------------
The demand for IT networking professionals continues to grow, and the
demand for specialized networking skills is growing even more rapidly.
Take a complimentary Learning@Cisco Self-Assessment and learn
about Cisco certifications, training, and career opportunities.
http://p.sf.net/sfu/cisco-dev2dev
_______________________________________________
Shinken-devel mailing list
Shinken-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/shinken-devel

 


------------------------------------------------------------------------------
The demand for IT networking professionals continues to grow, and the
demand for specialized networking skills is growing even more rapidly.
Take a complimentary Learning@Cisco Self-Assessment and learn
about Cisco certifications, training, and career opportunities.
http://p.sf.net/sfu/cisco-dev2dev
_______________________________________________
Shinken-devel mailing list
Shinken-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/shinken-devel