Re: [mod-security-users] Reading Concurrent Logs into Graylog
Brought to you by:
victorhora,
zimmerletw
Menu
▾
▴
Re: [mod-security-users] Reading Concurrent Logs into Graylog
|
From: Jose P. V. L. <pab...@gm...> - 2016-10-19 23:18:57
|
I didn't find anything about new lines except this: http://serverfault.com/questions/685613/passing-json-application-log-to-remote-logstash-via-nxlog-on-windows Regards El jue., 20 de octubre de 2016 1:11, Jason Mull <jm...@te...> escribió: > I think that might be the issue…I went ahead and enabled the multiline > module in nxlog.conf and all of the info instantly started coming through. > I’m assuming that the multiline module is adding the newline that GrayLog > wants to see. Any thoughts on whether this setup could cause me any issues > down the line? > > > > > > > > *From:* Robert Paprocki [mailto:rpa...@fe...] > *Sent:* Wednesday, October 19, 2016 4:45 PM > > > *To:* mod...@li... > *Subject:* Re: [mod-security-users] Reading Concurrent Logs into Graylog > > > > Does graylog require that data end in a newline? It appears that > concurrent JSON files are written and not appended with a newline, (so > doing something like `cat *` produces a single blob of data). > > > > Probably not a bad idea to patch sec log writer to append an additional > newline. > > > > On Wed, Oct 19, 2016 at 2:26 PM, Jason Mull <jm...@te...> wrote: > > Yes. I’ve run the sample lines from both concurrent and serial logging > through https://jsonformatter.curiousconcept.com/. In some cases they > might have a couple of different fields, but in all cases they come through > as valid JSON. > > > > *From:* Robert Paprocki [mailto:rpa...@fe...] > *Sent:* Wednesday, October 19, 2016 3:56 PM > > > *To:* mod...@li... > *Subject:* Re: [mod-security-users] Reading Concurrent Logs into Graylog > > > > Have you validated that this JSON is correct, valid JSON? Perhaps try > comparing a line of serial JSON from a line of concurrent JSON and compare > the two? > > > > On Wed, Oct 19, 2016 at 1:49 PM, Jason Mull <jm...@te...> wrote: > > For the sake of testing, I’ve given all users full access. NXLog and > Graylog are running under their own service accounts. As I mentioned > previously, I’m not thinking it’s a permissions issue as I can copy a line > of JSON from a serial log file on another server and insert it into a new > file in the concurrent logging structure and it works fine. If I copy a > line of JSON generated since I enabled concurrent logging and paste it into > a new file within the logging structure, that file will not read. > > > > *From:* Jose Pablo Valcárcel Lázaro [mailto:pab...@gm...] > > *Sent:* Wednesday, October 19, 2016 2:39 PM > *To:* mod...@li... > *Subject:* Re: [mod-security-users] Reading Concurrent Logs into Graylog > > > > Hi. Have you checked if directory where nxlog send files has x permission? > Under what user is running nxlog and graylog? In some applications you can > map users from one server to another like I think nfs service does. > > Kind regards > > > > El mié., 19 de octubre de 2016 21:25, Jason Mull <jm...@te...> > escribió: > > Hello, I’m hoping someone can assist me with this issue. I’m using > Graylog to take in all of my log data, including Modsecurity. I was > initially using nxlog to send serial logs in JSON format to Graylog and > everything worked great until I started running into performance issues > running modsecurity on a server running multiple websites and was informed > that concurrent logging was better for my needs. I switched to concurrent > logging, and the data is not reading into Graylog now. I feel confident > that I do not have a permissions issue as I do not see permission denied > errors in the nxlog error logs. Furthermore, if I copy a line of JSON logs > from the serial log on another server, it reads in just fine. The issue > appears to me to be with how the JSON is generated in concurrent mode. Has > anyone else run into any issues similar to this? > > > > Jason > > > ------------------------------------------------------------------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, SlashDot.org! http://sdm.link/slashdot > _______________________________________________ > mod-security-users mailing list > mod...@li... > https://lists.sourceforge.net/lists/listinfo/mod-security-users > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: > http://www.modsecurity.org/projects/commercial/rules/ > http://www.modsecurity.org/projects/commercial/support/ > > > > ------------------------------------------------------------------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, SlashDot.org! http://sdm.link/slashdot > _______________________________________________ > mod-security-users mailing list > mod...@li... > https://lists.sourceforge.net/lists/listinfo/mod-security-users > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: > http://www.modsecurity.org/projects/commercial/rules/ > http://www.modsecurity.org/projects/commercial/support/ > > > > > > ------------------------------------------------------------------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, SlashDot.org! http://sdm.link/slashdot > _______________________________________________ > mod-security-users mailing list > mod...@li... > https://lists.sourceforge.net/lists/listinfo/mod-security-users > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: > http://www.modsecurity.org/projects/commercial/rules/ > http://www.modsecurity.org/projects/commercial/support/ > > > > ------------------------------------------------------------------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, SlashDot.org! http://sdm.link/slashdot > _______________________________________________ > mod-security-users mailing list > mod...@li... > https://lists.sourceforge.net/lists/listinfo/mod-security-users > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs: > http://www.modsecurity.org/projects/commercial/rules/ > http://www.modsecurity.org/projects/commercial/support/ > |
