Many thanks once again Gerhard
I can't test it in production environnement yet, but for developpement, I
pulled your modifications from the git, and it works like a charm.
It's way better than my ".encore('ascii', 'replace')" workaround :-p
Regards,
Denis GERMAIN
2011/12/9 Gerhard Lausser <Gerhard.Lausser@...>
> Hi,
>
> the hudson tests failed, so i committed another modification.
> Now i get special characters on the command line
>
> OMD[stest]:~$ printf 'GET comments\nColumns: comment_id
> comment\nOutputFormat: csv\n' | unixcat tmp/run/live
> 66;îcî un cômméntère
> 67;This host has been scheduled for fixed downtime from 2011-12-08 21:44:01
> to 2011-12-08 23:44:01. Notifications for the host will not be sent out
> during that time period.
>
> and the unittests look good, too.
>
> Gerhard
>
> > -----Ursprüngliche Nachricht-----
> > Von: Gerhard Lausser [mailto:Gerhard.Lausser@...]
> > Gesendet: Donnerstag, 8. Dezember 2011 23:14
> > An: shinken-devel@...
> > Betreff: Re: [Shinken-devel] Slight problem with livestatus
> > modulewithdirect queries
> >
> > Hi,
> >
> > as you can see from the screenshot, accents are no problem in
> > the Thruk gui.
> > That is because it uses OutputFormat:json in the livestatus
> > queries. And the
> > json module seems to handle these special characters automagically.
> > Now i made this:
> >
> > stdout_encoding = sys.getdefaultencoding()
> > ...
> > elif isinstance(x, basestring):
> >
> > l.append(x.encode(stdout_encoding, 'replace'))
> > else:
> > l.append(str(x))
> >
> > when i run the broker as a daemon, stdout_encoding is
> > "ascii", so this code
> > is practically the same as your "dirty" workaround. You
> > should see your
> > accents.
> >
> > Gerhard
> >
> >
> > > -----Ursprüngliche Nachricht-----
> > > Von: Denis GERMAIN [mailto:dt.germain@...]
> > > Gesendet: Donnerstag, 8. Dezember 2011 14:56
> > > An: shinken-devel@...
> > > Betreff: Re: [Shinken-devel] Slight problem with livestatus
> > > module withdirect queries
> > >
> > > For now i used the following workaround, which is really
> > > dirty but seems to work. I post it in case someone uses it
> > > also (even though it's highly improbable)
> > >
> > >
> > > cd
> > /usr/lib/python2.6/site-packages/shinken/modules/livestatus_broker/
> > > cp livestatus_response.py livestatus_response.py.old
> > > vi livestatus_response.py
> > > line 92
> > > -- l.append(str(x))
> > > ++ if type(x) == int:
> > > ++ l.append(str(x))
> > > ++ else:
> > > ++ x = x.encode('ascii', 'replace')
> > > ++ l.append(str(x))
> > > I had to make a difference between int which don't have the
> > > .encode() method and string types
> > >
> > > Denis GERMAIN
> > >
> > >
> > > 2011/12/7 Denis GERMAIN <dt.germain@...>
> > >
> > >
> > > Hi,
> > >
> > > I have a new issue, partly related to this script. The
> > > script in itself is now working fine, thanks to your
> > > modifications, but we are getting a Traceback on Livestatus
> > > broker because we have special French caracters in our
> > > comments for hosts down (é, è, à, etc...)
> > >
> > > This didn't happen before because we couldn't add these
> > > caracters at all in Livestatus until our last Shinken upgrade
> > > (there has been some additions to prevent these to create
> > > bugs in the latest version of Shinken)
> > >
> > > I don't know what kind of "protection" has been added
> > > in the code to prevent these special caracters from crashing
> > > the broker, but if I knew it I could add it here also, in
> > > order to avoid the error I have here (see attached traceback)
> > >
> > > Regards
> > >
> > > Denis GERMAIN
> > >
> > >
> > > 2011/11/3 Denis GERMAIN <dt.germain@...>
> > >
> > >
> > > Hi
> > >
> > > This indeed seems to be a valid solution, I
> > > launched it many times, with no more issue on the broker. I'm
> > > going to put back my script in production, thanks to your help.
> > >
> > > Many thanks Gerhard
> > >
> > > Denis
> > >
> > >
> > > 2011/11/2 Gerhard Lausser <Gerhard.Lausser@...>
> > >
> > >
> > > Hi,
> > >
> > >
> > >
> > > maybe the Nagios livestatus module is
> > > more robust. I took your script and it crashed, but only when
> > > I had hosts without a comment. I rewrote it a little bit and
> > > it works now (CentOS5.6,64,Python 2.4.3)
> > >
> > >
> > >
> > > def main_function():
> > >
> > > host_table = send_query("GET
> > > hosts\nFilter: state = 0\nColumns: name
> > > comments\nOutputFormat: csv\n")
> > >
> > > for host in host_table:
> > >
> > > host_name,
> > > host_comment_ids = host
> > >
> > >
> > >
> > > try:
> > >
> > >
> > > host_last_comment_id = host_comment_ids.split(',')[-1] #use
> > > only last comment
> > >
> > >
> > > host_last_comment_id = str(int(host_last_comment_id)) # if
> > > it is not a number (or empty), then it raises an exception
> > >
> > > comment =
> > > send_query("GET comments\nColumns: comment_id
> > > comment\nFilter: comment_id = "+host_last_comment_id+"\n")
> > >
> > > print comment
> > >
> > > except Exception:
> > >
> > > print host_name + "
> > > has no comments"
> > >
> > >
> > >
> > > OMD[denis]:~$ python denis.py
> > >
> > > [['16', 'kaas']]
> > >
> > > [['2', 'schmarrn']]
> > >
> > > omd11 has no comments
> > >
> > > [['4', 'schmarrn']]
> > >
> > > [['5', 'schmarrn']]
> > >
> > > [['6', 'schmarrn']]
> > >
> > > [['7', 'schmarrn']]
> > >
> > > [['17', 'kaas-omd-2']]
> > >
> > > [['9', 'schmarrn']]
> > >
> > > [['10', 'schmarrn']]
> > >
> > > [['11', 'schmarrn']]
> > >
> > > [['12', 'schmarrn']]
> > >
> > > [['13', 'schmarrn']]
> > >
> > > [['14', 'schmarrn']]
> > >
> > > [['15', 'schmarrn']]
> > >
> > >
> > >
> > > Even if I add lots of main_function()
> > > calls to the script, I get valid output.
> > >
> > > Can you please try it with my modifications?
> > >
> > >
> > >
> > > Gerhard
> > >
> > >
> > >
> > > Von: Denis GERMAIN
> > > [mailto:dt.germain@...]
> > > Gesendet: Mittwoch, 2. November 2011 18:08
> > > An: shinken-devel@...
> > > Betreff: Re: [Shinken-devel] Slight
> > > problem with livestatus module with direct queries
> > >
> > >
> > >
> > > Hi again,
> > >
> > > I might have been a little optimistic,
> > > because my prod crashed again.
> > >
> > > This time I used python code which has
> > > been adapted from the website from Livestatus conceptor, so I
> > > guess that this time the queries are correctly done
> > >
> > > This error is really strange, because
> > > it seems to be realted to the number of request I do in a
> > > short period of time. I managed to reproduce the error on my
> > > dev environment *relief*. Here, everything works fine until
> > > the broker crashes.
> > >
> > > I might be totally wrong here as I
> > > don't know much of Livestatus coding in Shinken, but is it
> > > possible that the maximum number of socket has been reached
> > > or something like that?
> > >
> > > Attached, you can see the scenario I
> > > used to reproduce the error :
> > > I first added comments on running hosts
> > > (in order to have a certain amount of hosts with comments on them)
> > > Then I launched my script
> > > Livestatus answered correctly for some
> > > hosts, then crashed, like on prod env
> > >
> > > If this is indeed a problem with max
> > > number of sockets in livestatus module, I could recode my
> > > script in order to avoid making too many queries (getting all
> > > hosts, then getting all comments, only 2 queries here), but
> > > I'd like to be sure it's really that in order to avoid
> > > further issues on prod environnement
> > >
> > > Regards
> > >
> > > Denis GERMAIN
> > >
> > > 2011/10/27 Olivier Hanesse
> > > <olivier.hanesse@...>
> > >
> > > hi,
> > >
> > >
> > >
> > > I used to work with netcat too, and I
> > > got some issues in the past with some netcat version.
> > >
> > > On debian for example, you have 2 versions :
> > >
> > >
> > >
> > > netcat-openbsd - TCP/IP swiss army knife
> > >
> > > netcat-traditional - TCP/IP swiss army knife
> > >
> > >
> > >
> > > One was working, but not the other one.
> > >
> > > Maybe you could give a try and install
> > > the other version.
> > >
> > >
> > >
> > > Regards
> > >
> > >
> > >
> > > Olivier
> > >
> > >
> > >
> > > 2011/10/27 Denis GERMAIN <dt.germain@...>
> > >
> > > Hi,
> > >
> > > Thanks for the answer, I had forgotten
> > > that we used netcat instead of unixcat. I have still to
> > > confirm this, but i'm think we have a problem because we're
> > > not correctly closing the connection with netcat.
> > >
> > > I'm going to code it in python in order
> > > to do this in a proper way.
> > >
> > > I'll keep you posted if this is not the
> > > issue but I'm pretty sure that this is our problem.
> > >
> > > Thanks!
> > >
> > > Denis
> > >
> > > 2011/10/26 Gerhard Lausser
> > > <Gerhard.Lausser@...>
> > >
> > > Hi,
> > >
> > >
> > >
> > > which version of Shinken do you use?
> > >
> > > I have here a 0.8 and can’t reproduce
> > > your error. There’s one host (omd-disco) which I made DOWN by
> > > submitting a passive command and which has two comments attached.
> > >
> > >
> > >
> > > OMD[disco]:~$ printf "GET
> > > hosts\nFilter: state = 1\nColumns: name comments\n"|unixcat
> > > /omd/sites/disco/tmp/run/live
> > >
> > > omd-disco;3,4
> > >
> > >
> > >
> > > OMD[disco]:~$ printf 'GET
> > > comments\nColumns: comment_id comment\n' | unixcat
> > > /omd/sites/disco/tmp/run/live
> > >
> > > 3;i bi hi
> > >
> > > 4;i kos ned oft gnou sogn, i bi hi
> > >
> > >
> > >
> > > OMD[disco]:~$ printf 'GET
> > > comments\nColumns: comment_id comment\nFilter: comment_id =
> > > 3\n' | unixcat /omd/sites/disco/tmp/run/live
> > >
> > > 3;i bi hi
> > >
> > >
> > >
> > > Gerhard
> > >
> > >
> > >
> > >
> > >
> > > Von: Denis GERMAIN
> > > [mailto:dt.germain@...]
> > > Gesendet: Mittwoch, 26. Oktober 2011 16:40
> > > An: shinken-devel@...
> > > Betreff: [Shinken-devel] Slight problem
> > > with livestatus module with direct queries
> > >
> > >
> > >
> > > Hello list,
> > >
> > > For historic reporting purpose, we have
> > > a developper in our team that directly execute queries to the
> > > livestatus module via commands like
> > > printf 'GET hosts\nFilter: state = 1' |
> > > nc 10.197.64.97 50000 > /data/ftptrace/nagios/thruk.txt
> > > printf 'GET comments\nFilter comment_id
> > > = '$commid | nc 10.197.64.97 50000 | tail -1 | awk -F";"
> > '{print $2}'
> > >
> > > This gives us informations about hosts
> > > down which are then reformatted in a dusty old report sent to
> > > our client.
> > > At the beginning we used only the first
> > > request for our report, as comments were directly available
> > > from the host information given by the query. After an
> > > update, we didn't realised that the comments were not
> > > directly in the query result, but only the comment ID (which
> > > I grant you is much more "clean")
> > >
> > > 1) Problem is that now, we need to
> > > query for the comments (selecting good comment ID) and that
> > > it constantly break livestatus module (which doesn't always
> > > manage to relaunch itself)
> > >
> > > 2) We also noted that when we use the
> > > Column statement (to select only the columns we need), the
> > > livestatus module instantly crash
> > >
> > > I realise that our problem is a little
> > > exotic, as the vast majority of livestatus users only use it
> > > through Thruk, but if something could be done (or as already
> > > been done), we would be grateful (our developper mostly :-p)
> > >
> > > Denis GERMAIN
> > >
> > > PS: An example traceback. I don't have
> > > much more yet, we are running Shinken on production (I
> > > already see admins frowning eyes, but it's stable enough if
> > > you don't try strange things like us) and my manager don't
> > > really like us crashing it on purpose. We'll try to reproduce
> > > on preproduction ASAP, but if this can help pinpointing the issue...
> > >
> > > [1319536549] [broker-1] Traceback (most
> > > recent call last):
> > > File
> > > "/usr/lib/python2.6/site-packages/shinken/modules/livestatus_b
> > roker/livestatus_broker.py", line 828, in main
> > > self.do_main()
> > > File
> > > "/usr/lib/python2.6/site-packages/shinken/modules/livestatus_b
> > roker/livestatus_broker.py", line 1052, in do_main
> > > response, keepalive =
> > > self.livestatus.handle_request(open_connections[socketid]['buffer'])
> > > File
> > > "/usr/lib/python2.6/site-packages/shinken/modules/livestatus_b
> > roker/livestatus.py", line 129, in handle_request
> > > request.parse_input(data)
> > > File
> > > "/usr/lib/python2.6/site-packages/shinken/modules/livestatus_b
> > roker/livestatus_request.py", line 96, in parse_input
> > > query.parse_input('\n'.join(query_cmds))
> > > File
> > > "/usr/lib/python2.6/site-packages/shinken/modules/livestatus_b
> > roker/livestatus_query.py", line 259, in parse_input
> > >
> > self.filter_stack.put(self.make_filter(operator, attribute, >
> > reference))
> > > File
> > > "/usr/lib/python2.6/site-packages/shinken/modules/livestatus_b
> > roker/livestatus_query.py", line 786, in make_filter
> > > converter = self.find_converter(attribute)
> > > File
> > > "/usr/lib/python2.6/site-packages/shinken/modules/livestatus_b
> > roker/livestatus_query.py", line 102, in find_converter
> > > out_map = LSout_map[self.out_map_name]
> > > KeyError: 'hosts'
> > >
> > >
> > >
> > >
> > > --------------------------------------------------------------
> > > ----------------
> > > The demand for IT networking
> > > professionals continues to grow, and the
> > > demand for specialized networking
> > > skills is growing even more rapidly.
> > > Take a complimentary Learning@...
> > > Self-Assessment and learn
> > > about Cisco certifications, training,
> > > and career opportunities.
> > > http://p.sf.net/sfu/cisco-dev2dev
> > > _______________________________________________
> > > Shinken-devel mailing list
> > > Shinken-devel@...
> > >
> > > https://lists.sourceforge.net/lists/listinfo/shinken-devel
> > >
> > >
> > >
> > >
> > > --------------------------------------------------------------
> > > ----------------
> > > The demand for IT networking
> > > professionals continues to grow, and the
> > > demand for specialized networking
> > > skills is growing even more rapidly.
> > > Take a complimentary Learning@...
> > > Self-Assessment and learn
> > > about Cisco certifications, training,
> > > and career opportunities.
> > > http://p.sf.net/sfu/cisco-dev2dev
> > > _______________________________________________
> > > Shinken-devel mailing list
> > > Shinken-devel@...
> > >
> > > https://lists.sourceforge.net/lists/listinfo/shinken-devel
> > >
> > >
> > >
> > >
> > >
> > > --------------------------------------------------------------
> > > ----------------
> > > The demand for IT networking
> > > professionals continues to grow, and the
> > > demand for specialized networking
> > > skills is growing even more rapidly.
> > > Take a complimentary Learning@...
> > > Self-Assessment and learn
> > > about Cisco certifications, training,
> > > and career opportunities.
> > > http://p.sf.net/sfu/cisco-dev2dev
> > > _______________________________________________
> > > Shinken-devel mailing list
> > > Shinken-devel@...
> > >
> > > https://lists.sourceforge.net/lists/listinfo/shinken-devel
> > >
> > >
> > >
> > >
> > >
> > > --------------------------------------------------------------
> > > ----------------
> > > RSA® Conference 2012
> > > Save $700 by Nov 18
> > > Register now!
> > > http://p.sf.net/sfu/rsa-sfdev2dev1
> > >
> > > _______________________________________________
> > > Shinken-devel mailing list
> > > Shinken-devel@...
> > >
> > > https://lists.sourceforge.net/lists/listinfo/shinken-devel
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > __________ Hinweis von ESET NOD32 Antivirus,
> > > Signaturdatenbank-Version 6695 (20111208) __________
> > >
> > >
> > > E-Mail wurde geprüft mit ESET NOD32 Antivirus.
> > >
> > >
> > > http://www.eset.com
> > >
> > >
> >
> >
> > __________ Hinweis von ESET NOD32 Antivirus,
> > Signaturdatenbank-Version 6695
> > (20111208) __________
> >
> > E-Mail wurde geprüft mit ESET NOD32 Antivirus.
> >
> > http://www.eset.com
> >
> >
> >
> > __________ Hinweis von ESET NOD32 Antivirus,
> > Signaturdatenbank-Version 6695 (20111208) __________
> >
> > E-Mail wurde geprüft mit ESET NOD32 Antivirus.
> >
> > http://www.eset.com
> >
> >
>
>
> __________ Hinweis von ESET NOD32 Antivirus, Signaturdatenbank-Version 6695
> (20111208) __________
>
> E-Mail wurde geprüft mit ESET NOD32 Antivirus.
>
> http://www.eset.com
>
>
>
>
> ------------------------------------------------------------------------------
> Cloud Services Checklist: Pricing and Packaging Optimization
> This white paper is intended to serve as a reference, checklist and point
> of
> discussion for anyone considering optimizing the pricing and packaging
> model
> of a cloud services business. Read Now!
> http://www.accelacomm.com/jaw/sfnl/114/51491232/
> _______________________________________________
> Shinken-devel mailing list
> Shinken-devel@...
> https://lists.sourceforge.net/lists/listinfo/shinken-devel
>
|