openfpc-users Mailing List for openfpc (Page 2)
Open Source Full Packet Capture
Brought to you by:
leonward
Menu
▾
▴
openfpc-users — OpenFPC Discussion
You can subscribe to this list here.
| 2015 |
Jan
|
Feb
|
Mar
|
Apr
|
May
(6) |
Jun
|
Jul
|
Aug
(18) |
Sep
(17) |
Oct
(1) |
Nov
(10) |
Dec
(2) |
|---|
|
From: Leon W. <le...@rm...> - 2015-09-10 18:41:04
|
Okay, so just tested it and it looked like it works for me... Can you please send the output of.... $ sudo openfpc -v -t openfpc-daemonlogger -a restart $ grep BPF_FILE /etc/openfpc/*.conf $ ps aux |grep daemonlogger There should also be a chunk of data about daemonlogger in your syslog... On Tue, Sep 8, 2015 at 4:14 PM, Igor Kaplan <igo...@gm...> wrote: > Leon, > > > > Thanks so much, really appreciate! > > > > *From:* leo...@gm... [mailto:leo...@gm...] *On Behalf > Of *Leon Ward > *Sent:* Tuesday, September 08, 2015 9:44 AM > *To:* Igor Kaplan > *Cc:* <ope...@li...> > *Subject:* Re: BPF filter > > > > Will take a look, can't from where I am right now. > > Its likely that I've broken something years back, It used to work as I > once used it a lot myself, but no longer. > > > > On Tue, Sep 8, 2015 at 2:42 PM, Igor Kaplan <igo...@gm...> wrote: > > Hi Leon, > > I am still having problems with the input bpf filter. I wonder, if you > could please answer my question below. > Not sure, if I am doing everything correctly, however as my best > understanding, I do. > > Many thanks. > > -Igor. > > -----Original Message----- > From: Igor Kaplan [mailto:igo...@gm...] > Sent: Friday, August 28, 2015 4:32 PM > To: 'Leon Ward' > Cc: ope...@li... > Subject: BPF filter > > Hi Leon, > > Not sure, if I am doing everything correctly, however my BPF filter in > openfpc-default.conf does not look to be working. > I have a line in my openfpc-default.conf: > BPF_File=/var/filter.bpf > > While starting openfpc -a start --verbose I am able to see, the proper bpf > file is found and loaded. > > My filter.bpf is very simple: > Port 502 > > So it should capture packets from port 502 only > > However after fetching the data I still see packets from other ports. > > I wonder, do I understand correctly, the BPF_FILE in the config file will > restrict, which packets are captured? So using the bpf file as above I > should not see any packets from other ports beside 502? Or it is something > different? > > Many thanks and have a nice weekend! > > -Igor > > -----Original Message----- > From: Leon Ward [mailto:leo...@gm...] On Behalf Of Leon Ward > Sent: Tuesday, August 25, 2015 4:59 PM > To: Igor Kaplan > Cc: <ope...@li...> > Subject: Re: [Openfpc-users] Openfpc-users Digest, Vol 2, Issue 4 > > That's where the pcaps should live, and they will grow to the max > percentage > that you allow in the node config. > > What does an openfpc-client -a status show? > > Also what's a df -h look like? > > The pcaps will auto-prune unless something has gone wrong along the way.... > > Thinking out loud, what's an ls of your pcaps directory look like? Have you > got multiple nodes running on one box? > > -L > > Sent from a mobile device. Apologies for any typos but they happen. > > > On 25 Aug 2015, at 16:25, Igor Kaplan <igo...@gm...> wrote: > > > > Hi Leon, > > > > Could you please help me with following. > > I am running openfpc for several days already and now I am out of > > space on my Ubuntu box which runs openfpc Under /var/tmp/openfpc I see > > directories, some of which contain number of large files: > > api-pcaps extracted pcap session > > > > I wonder, can I safely delete data under any of those directories above? > > Could you please let me know, which directory I can empty without > > breaking openfpc functionality? > > > > Is there any way to clean all captured data and start fresh? > > > > Many thanks. > > > > -Igor. > > > > > > -----Original Message----- > > From: ope...@li... > > [mailto:ope...@li...] > > Sent: Thursday, August 20, 2015 8:01 PM > > To: ope...@li... > > Subject: Openfpc-users Digest, Vol 2, Issue 4 > > > > Send Openfpc-users mailing list submissions to > > ope...@li... > > > > To subscribe or unsubscribe via the World Wide Web, visit > > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > or, via email, send a message with subject or body 'help' to > > ope...@li... > > > > You can reach the person managing the list at > > ope...@li... > > > > When replying, please edit your Subject line so it is more specific > > than > > "Re: Contents of Openfpc-users digest..." > > > > > > Today's Topics: > > > > 1. Re: Openfpc usage (Igor Kaplan) > > > > > > ---------------------------------------------------------------------- > > > > Message: 1 > > Date: Thu, 20 Aug 2015 20:01:08 -0400 > > From: "Igor Kaplan" <igo...@gm...> > > Subject: Re: [Openfpc-users] Openfpc usage > > To: <ope...@li...> > > Message-ID: <000c01d0dba4$7c3c9910$74b5cb30$@gmail.com> > > Content-Type: text/plain; charset="utf-8" > > > > Also sending my reply to the list, sorry, forgot to include it. > > > > > > > > > > > > From: Igor Kaplan [mailto:igo...@gm...] > > Sent: Thursday, August 20, 2015 6:44 PM > > To: 'Leon Ward' > > Subject: RE: [Openfpc-users] Openfpc usage > > > > > > > > Version of mergecap: > > > > Mergecap 1.10.6 (v1.10.6 from master-1.10) > > > > > > > > Linux is Ubuntu 14.04.2 LTS > > > > > > > > File list.pcap > > > > list.pcap: pcap-ng capture file - version 1.0 > > > > > > > > Thanks. > > > > > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf > > Of Leon Ward > > Sent: Thursday, August 20, 2015 6:23 PM > > To: Igor Kaplan > > Cc: ope...@li... > > Subject: Re: [Openfpc-users] Openfpc usage > > > > > > > > What's your platform, version of mergecap etc. > > > > Also, if you '$ file list.pcap' what does it say? > > > > > > > > -L > > > > > > > > > > > > On Thu, Aug 20, 2015 at 10:15 PM, Igor Kaplan <igo...@gm...> > wrote: > > > > Hi Leon, > > > > > > > > Need your help please once again. > > > > Got the following problem and spent several hours trying to solve it. > > > > > > > > When making the API call to fetch the pcap data I am getting the data > > in pcapng format. > > > > The OpenFPC is using the mergecap to merge pcap files and by default > > mergecap creates the output in pcapng format instead of pcap. > > > > > > > > I have changed the following line in openfpc-default.conf file > > > > MERGECAP=/usr/bin/mergecap -F pcap > > > > > > > > This helped when I use the openfpc-client command to create pcap > > files, however when I use curl to fetch the data I still receive the > > output in pcapng format. > > > > > > > > curl -k > > 192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F-C061B4C > > E8A48\ > > <http://192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F > > -C061B > > 4CE8A48%5C&stime=20150818%2010:00%5C&etime=20150818%2010:30%5C&dpt=22> > > &stime=20150818%2010:00\&etime=20150818%2010:30\&dpt=22 > list.pcap > > > > > > > > cat list.pcap|tshark -i- > > > > Capturing on 'Standard input' > > > > tshark: Unrecognized libpcap format > > > > > > > > Looks like in case of API call the mergecap utility is not used at all. > > And I was not able to find in the code how merging is done in this case. > > > > > > > > Could you please help me. Is it possible to make the fetch API call > > to return the data in pcap format? > > > > > > > > Thanks so much! > > > > > > > > Igor > > > > > > > > > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf > > Of Leon Ward > > Sent: Tuesday, August 18, 2015 1:29 PM > > > > > > To: Igor Kaplan > > Cc: ope...@li... > > Subject: Re: [Openfpc-users] Openfpc usage > > > > > > > > Actually it wont. It will only remove the oldest PCAP file. It's best > > to keep those PCAP files on their own partition. > > > > The old flow records in mysql actually get removed automatically based > > on the oldest packet in the store. So you won't have records that are > > older than the pcaps. > > > > > > > > -L > > > > > > > > > > > > On Tue, Aug 18, 2015 at 6:21 PM, Igor Kaplan <igo...@gm...> > wrote: > > > > Hi Leon, all, > > > > > > > > I have one more question please. > > > > > > > > Based on the documentation the following line in the openfpc config > > file restricts the space usage of captured data to 50 percent: > > > > PCAP_SPACE=50 > > > > > > > > So, if the data size exceeds 50 percent old files will be deleted > > automatically? > > > > Will openfpc also delete the old MySQL session tables? > > > > > > > > Many thanks and all the best! > > > > > > > > -Igor. > > > > > > > > > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf > > Of Leon Ward > > Sent: Monday, August 17, 2015 11:51 AM > > To: Igor Kaplan > > Cc: ope...@li... > > Subject: Re: [Openfpc-users] Openfpc usage > > > > > > > > Hi, > > > > > > > > Documentation is really one of the places that really needs some extra > > focus. > > > > > > > > The best docs I can point you to are in that folder, plus there is > > some out-of date info on my blog http://www.leonward.com. > > > > I actually delivered a presentation at Defcon last weekend all about > > OpenFPC. I have forwarded the slides separately. Hopefully that will > > help as well. > > > > > > > > As for your specific question about OpenFPC GUI. That's actually now > > been deprecated as it's no longer relevant for how it functions in a > > distributed manner. The OpenFPC-Chrome Extension will be the next best > > thing for interacting with the QueueDaemon remotely in a GUI-like way. > > > > > > > > Cheers, > > > > > > > > -L > > > > > > > > > > > > > > > > On Mon, Aug 17, 2015 at 4:25 PM, Igor Kaplan <igo...@gm...> > wrote: > > > > Hi All, > > > > > > > > My name is Igor. I just found the OpenFPC and evaluating it. Looks > > like it is very good tool. > > > > I successfully installed on Ubuntu 14.4 with Perl 5.18 > > > > I have installed the OpenFPC-master, so it is the latest code. > > > > > > > > Now I would like to find out if there is more documentation beside > > files which I could find under docs directory. > > > > For example the INSTALL.md refers to the USAGE document, however I > > was not able to find it anywhere > > > > > > > > I am looking for the usage other then basic, just to find out, what > > are my advanced options. > > > > > > > > For example the openfpc-dbmaint.sh script is also able to create the > > gui database, I wonder, what it is for? > > > > > > > > The OpenFPC looks to be very powerful, just would like to understand > > it as best as I can. > > > > > > > > Would so much appreciate any reply?s. > > > > > > > > Many thanks. > > > > > > > > Igor. > > > > > > > > > > ---------------------------------------------------------------------- > > ------ > > -- > > > > _______________________________________________ > > Openfpc-users mailing list > > Ope...@li... > > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > > > > > > > > ---------------------------------------------------------------------- > > ------ > > -- > > > > _______________________________________________ > > Openfpc-users mailing list > > Ope...@li... > > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > > > > > > > > > > -------------- next part -------------- An HTML attachment was > > scrubbed... > > > > ------------------------------ > > > > ---------------------------------------------------------------------- > > ------ > > -- > > > > > > ------------------------------ > > > > _______________________________________________ > > Openfpc-users mailing list > > Ope...@li... > > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > > > > End of Openfpc-users Digest, Vol 2, Issue 4 > > ******************************************* > > > > > > ---------------------------------------------------------------------- > > -------- _______________________________________________ > > Openfpc-users mailing list > > Ope...@li... > > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > |
|
From: Igor K. <igo...@gm...> - 2015-09-08 15:14:21
|
Leon, Thanks so much, really appreciate! From: leo...@gm... [mailto:leo...@gm...] On Behalf Of Leon Ward Sent: Tuesday, September 08, 2015 9:44 AM To: Igor Kaplan Cc: <ope...@li...> Subject: Re: BPF filter Will take a look, can't from where I am right now. Its likely that I've broken something years back, It used to work as I once used it a lot myself, but no longer. On Tue, Sep 8, 2015 at 2:42 PM, Igor Kaplan <igo...@gm...> wrote: Hi Leon, I am still having problems with the input bpf filter. I wonder, if you could please answer my question below. Not sure, if I am doing everything correctly, however as my best understanding, I do. Many thanks. -Igor. -----Original Message----- From: Igor Kaplan [mailto:igo...@gm...] Sent: Friday, August 28, 2015 4:32 PM To: 'Leon Ward' Cc: ope...@li... Subject: BPF filter Hi Leon, Not sure, if I am doing everything correctly, however my BPF filter in openfpc-default.conf does not look to be working. I have a line in my openfpc-default.conf: BPF_File=/var/filter.bpf While starting openfpc -a start --verbose I am able to see, the proper bpf file is found and loaded. My filter.bpf is very simple: Port 502 So it should capture packets from port 502 only However after fetching the data I still see packets from other ports. I wonder, do I understand correctly, the BPF_FILE in the config file will restrict, which packets are captured? So using the bpf file as above I should not see any packets from other ports beside 502? Or it is something different? Many thanks and have a nice weekend! -Igor -----Original Message----- From: Leon Ward [mailto:leo...@gm...] On Behalf Of Leon Ward Sent: Tuesday, August 25, 2015 4:59 PM To: Igor Kaplan Cc: <ope...@li...> Subject: Re: [Openfpc-users] Openfpc-users Digest, Vol 2, Issue 4 That's where the pcaps should live, and they will grow to the max percentage that you allow in the node config. What does an openfpc-client -a status show? Also what's a df -h look like? The pcaps will auto-prune unless something has gone wrong along the way.... Thinking out loud, what's an ls of your pcaps directory look like? Have you got multiple nodes running on one box? -L Sent from a mobile device. Apologies for any typos but they happen. > On 25 Aug 2015, at 16:25, Igor Kaplan <igo...@gm...> wrote: > > Hi Leon, > > Could you please help me with following. > I am running openfpc for several days already and now I am out of > space on my Ubuntu box which runs openfpc Under /var/tmp/openfpc I see > directories, some of which contain number of large files: > api-pcaps extracted pcap session > > I wonder, can I safely delete data under any of those directories above? > Could you please let me know, which directory I can empty without > breaking openfpc functionality? > > Is there any way to clean all captured data and start fresh? > > Many thanks. > > -Igor. > > > -----Original Message----- > From: ope...@li... > [mailto:ope...@li...] > Sent: Thursday, August 20, 2015 8:01 PM > To: ope...@li... > Subject: Openfpc-users Digest, Vol 2, Issue 4 > > Send Openfpc-users mailing list submissions to > ope...@li... > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.sourceforge.net/lists/listinfo/openfpc-users > or, via email, send a message with subject or body 'help' to > ope...@li... > > You can reach the person managing the list at > ope...@li... > > When replying, please edit your Subject line so it is more specific > than > "Re: Contents of Openfpc-users digest..." > > > Today's Topics: > > 1. Re: Openfpc usage (Igor Kaplan) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 20 Aug 2015 20:01:08 -0400 > From: "Igor Kaplan" <igo...@gm...> > Subject: Re: [Openfpc-users] Openfpc usage > To: <ope...@li...> > Message-ID: <000c01d0dba4$7c3c9910$74b5cb30$@gmail.com> > Content-Type: text/plain; charset="utf-8" > > Also sending my reply to the list, sorry, forgot to include it. > > > > > > From: Igor Kaplan [mailto:igo...@gm...] > Sent: Thursday, August 20, 2015 6:44 PM > To: 'Leon Ward' > Subject: RE: [Openfpc-users] Openfpc usage > > > > Version of mergecap: > > Mergecap 1.10.6 (v1.10.6 from master-1.10) > > > > Linux is Ubuntu 14.04.2 LTS > > > > File list.pcap > > list.pcap: pcap-ng capture file - version 1.0 > > > > Thanks. > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf > Of Leon Ward > Sent: Thursday, August 20, 2015 6:23 PM > To: Igor Kaplan > Cc: ope...@li... > Subject: Re: [Openfpc-users] Openfpc usage > > > > What's your platform, version of mergecap etc. > > Also, if you '$ file list.pcap' what does it say? > > > > -L > > > > > > On Thu, Aug 20, 2015 at 10:15 PM, Igor Kaplan <igo...@gm...> wrote: > > Hi Leon, > > > > Need your help please once again. > > Got the following problem and spent several hours trying to solve it. > > > > When making the API call to fetch the pcap data I am getting the data > in pcapng format. > > The OpenFPC is using the mergecap to merge pcap files and by default > mergecap creates the output in pcapng format instead of pcap. > > > > I have changed the following line in openfpc-default.conf file > > MERGECAP=/usr/bin/mergecap -F pcap > > > > This helped when I use the openfpc-client command to create pcap > files, however when I use curl to fetch the data I still receive the > output in pcapng format. > > > > curl -k > 192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F-C061B4C > E8A48\ > <http://192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F > -C061B > 4CE8A48%5C&stime=20150818%2010:00%5C&etime=20150818%2010:30%5C&dpt=22> > &stime=20150818%2010:00\&etime=20150818%2010:30\&dpt=22 > list.pcap > > > > cat list.pcap|tshark -i- > > Capturing on 'Standard input' > > tshark: Unrecognized libpcap format > > > > Looks like in case of API call the mergecap utility is not used at all. > And I was not able to find in the code how merging is done in this case. > > > > Could you please help me. Is it possible to make the fetch API call > to return the data in pcap format? > > > > Thanks so much! > > > > Igor > > > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf > Of Leon Ward > Sent: Tuesday, August 18, 2015 1:29 PM > > > To: Igor Kaplan > Cc: ope...@li... > Subject: Re: [Openfpc-users] Openfpc usage > > > > Actually it wont. It will only remove the oldest PCAP file. It's best > to keep those PCAP files on their own partition. > > The old flow records in mysql actually get removed automatically based > on the oldest packet in the store. So you won't have records that are > older than the pcaps. > > > > -L > > > > > > On Tue, Aug 18, 2015 at 6:21 PM, Igor Kaplan <igo...@gm...> wrote: > > Hi Leon, all, > > > > I have one more question please. > > > > Based on the documentation the following line in the openfpc config > file restricts the space usage of captured data to 50 percent: > > PCAP_SPACE=50 > > > > So, if the data size exceeds 50 percent old files will be deleted > automatically? > > Will openfpc also delete the old MySQL session tables? > > > > Many thanks and all the best! > > > > -Igor. > > > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf > Of Leon Ward > Sent: Monday, August 17, 2015 11:51 AM > To: Igor Kaplan > Cc: ope...@li... > Subject: Re: [Openfpc-users] Openfpc usage > > > > Hi, > > > > Documentation is really one of the places that really needs some extra > focus. > > > > The best docs I can point you to are in that folder, plus there is > some out-of date info on my blog http://www.leonward.com. > > I actually delivered a presentation at Defcon last weekend all about > OpenFPC. I have forwarded the slides separately. Hopefully that will > help as well. > > > > As for your specific question about OpenFPC GUI. That's actually now > been deprecated as it's no longer relevant for how it functions in a > distributed manner. The OpenFPC-Chrome Extension will be the next best > thing for interacting with the QueueDaemon remotely in a GUI-like way. > > > > Cheers, > > > > -L > > > > > > > > On Mon, Aug 17, 2015 at 4:25 PM, Igor Kaplan <igo...@gm...> wrote: > > Hi All, > > > > My name is Igor. I just found the OpenFPC and evaluating it. Looks > like it is very good tool. > > I successfully installed on Ubuntu 14.4 with Perl 5.18 > > I have installed the OpenFPC-master, so it is the latest code. > > > > Now I would like to find out if there is more documentation beside > files which I could find under docs directory. > > For example the INSTALL.md refers to the USAGE document, however I > was not able to find it anywhere > > > > I am looking for the usage other then basic, just to find out, what > are my advanced options. > > > > For example the openfpc-dbmaint.sh script is also able to create the > gui database, I wonder, what it is for? > > > > The OpenFPC looks to be very powerful, just would like to understand > it as best as I can. > > > > Would so much appreciate any reply?s. > > > > Many thanks. > > > > Igor. > > > > > ---------------------------------------------------------------------- > ------ > -- > > _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > > > ---------------------------------------------------------------------- > ------ > -- > > _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > > > > -------------- next part -------------- An HTML attachment was > scrubbed... > > ------------------------------ > > ---------------------------------------------------------------------- > ------ > -- > > > ------------------------------ > > _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > End of Openfpc-users Digest, Vol 2, Issue 4 > ******************************************* > > > ---------------------------------------------------------------------- > -------- _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users |
|
From: Leon W. <le...@rm...> - 2015-09-08 13:44:29
|
Will take a look, can't from where I am right now. Its likely that I've broken something years back, It used to work as I once used it a lot myself, but no longer. On Tue, Sep 8, 2015 at 2:42 PM, Igor Kaplan <igo...@gm...> wrote: > Hi Leon, > > I am still having problems with the input bpf filter. I wonder, if you > could please answer my question below. > Not sure, if I am doing everything correctly, however as my best > understanding, I do. > > Many thanks. > > -Igor. > > -----Original Message----- > From: Igor Kaplan [mailto:igo...@gm...] > Sent: Friday, August 28, 2015 4:32 PM > To: 'Leon Ward' > Cc: ope...@li... > Subject: BPF filter > > Hi Leon, > > Not sure, if I am doing everything correctly, however my BPF filter in > openfpc-default.conf does not look to be working. > I have a line in my openfpc-default.conf: > BPF_File=/var/filter.bpf > > While starting openfpc -a start --verbose I am able to see, the proper bpf > file is found and loaded. > > My filter.bpf is very simple: > Port 502 > > So it should capture packets from port 502 only > > However after fetching the data I still see packets from other ports. > > I wonder, do I understand correctly, the BPF_FILE in the config file will > restrict, which packets are captured? So using the bpf file as above I > should not see any packets from other ports beside 502? Or it is something > different? > > Many thanks and have a nice weekend! > > -Igor > > -----Original Message----- > From: Leon Ward [mailto:leo...@gm...] On Behalf Of Leon Ward > Sent: Tuesday, August 25, 2015 4:59 PM > To: Igor Kaplan > Cc: <ope...@li...> > Subject: Re: [Openfpc-users] Openfpc-users Digest, Vol 2, Issue 4 > > That's where the pcaps should live, and they will grow to the max > percentage > that you allow in the node config. > > What does an openfpc-client -a status show? > > Also what's a df -h look like? > > The pcaps will auto-prune unless something has gone wrong along the way.... > > Thinking out loud, what's an ls of your pcaps directory look like? Have you > got multiple nodes running on one box? > > -L > > Sent from a mobile device. Apologies for any typos but they happen. > > > On 25 Aug 2015, at 16:25, Igor Kaplan <igo...@gm...> wrote: > > > > Hi Leon, > > > > Could you please help me with following. > > I am running openfpc for several days already and now I am out of > > space on my Ubuntu box which runs openfpc Under /var/tmp/openfpc I see > > directories, some of which contain number of large files: > > api-pcaps extracted pcap session > > > > I wonder, can I safely delete data under any of those directories above? > > Could you please let me know, which directory I can empty without > > breaking openfpc functionality? > > > > Is there any way to clean all captured data and start fresh? > > > > Many thanks. > > > > -Igor. > > > > > > -----Original Message----- > > From: ope...@li... > > [mailto:ope...@li...] > > Sent: Thursday, August 20, 2015 8:01 PM > > To: ope...@li... > > Subject: Openfpc-users Digest, Vol 2, Issue 4 > > > > Send Openfpc-users mailing list submissions to > > ope...@li... > > > > To subscribe or unsubscribe via the World Wide Web, visit > > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > or, via email, send a message with subject or body 'help' to > > ope...@li... > > > > You can reach the person managing the list at > > ope...@li... > > > > When replying, please edit your Subject line so it is more specific > > than > > "Re: Contents of Openfpc-users digest..." > > > > > > Today's Topics: > > > > 1. Re: Openfpc usage (Igor Kaplan) > > > > > > ---------------------------------------------------------------------- > > > > Message: 1 > > Date: Thu, 20 Aug 2015 20:01:08 -0400 > > From: "Igor Kaplan" <igo...@gm...> > > Subject: Re: [Openfpc-users] Openfpc usage > > To: <ope...@li...> > > Message-ID: <000c01d0dba4$7c3c9910$74b5cb30$@gmail.com> > > Content-Type: text/plain; charset="utf-8" > > > > Also sending my reply to the list, sorry, forgot to include it. > > > > > > > > > > > > From: Igor Kaplan [mailto:igo...@gm...] > > Sent: Thursday, August 20, 2015 6:44 PM > > To: 'Leon Ward' > > Subject: RE: [Openfpc-users] Openfpc usage > > > > > > > > Version of mergecap: > > > > Mergecap 1.10.6 (v1.10.6 from master-1.10) > > > > > > > > Linux is Ubuntu 14.04.2 LTS > > > > > > > > File list.pcap > > > > list.pcap: pcap-ng capture file - version 1.0 > > > > > > > > Thanks. > > > > > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf > > Of Leon Ward > > Sent: Thursday, August 20, 2015 6:23 PM > > To: Igor Kaplan > > Cc: ope...@li... > > Subject: Re: [Openfpc-users] Openfpc usage > > > > > > > > What's your platform, version of mergecap etc. > > > > Also, if you '$ file list.pcap' what does it say? > > > > > > > > -L > > > > > > > > > > > > On Thu, Aug 20, 2015 at 10:15 PM, Igor Kaplan <igo...@gm...> > wrote: > > > > Hi Leon, > > > > > > > > Need your help please once again. > > > > Got the following problem and spent several hours trying to solve it. > > > > > > > > When making the API call to fetch the pcap data I am getting the data > > in pcapng format. > > > > The OpenFPC is using the mergecap to merge pcap files and by default > > mergecap creates the output in pcapng format instead of pcap. > > > > > > > > I have changed the following line in openfpc-default.conf file > > > > MERGECAP=/usr/bin/mergecap -F pcap > > > > > > > > This helped when I use the openfpc-client command to create pcap > > files, however when I use curl to fetch the data I still receive the > > output in pcapng format. > > > > > > > > curl -k > > 192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F-C061B4C > > E8A48\ > > <http://192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F > > -C061B > > 4CE8A48%5C&stime=20150818%2010:00%5C&etime=20150818%2010:30%5C&dpt=22> > > &stime=20150818%2010:00\&etime=20150818%2010:30\&dpt=22 > list.pcap > > > > > > > > cat list.pcap|tshark -i- > > > > Capturing on 'Standard input' > > > > tshark: Unrecognized libpcap format > > > > > > > > Looks like in case of API call the mergecap utility is not used at all. > > And I was not able to find in the code how merging is done in this case. > > > > > > > > Could you please help me. Is it possible to make the fetch API call > > to return the data in pcap format? > > > > > > > > Thanks so much! > > > > > > > > Igor > > > > > > > > > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf > > Of Leon Ward > > Sent: Tuesday, August 18, 2015 1:29 PM > > > > > > To: Igor Kaplan > > Cc: ope...@li... > > Subject: Re: [Openfpc-users] Openfpc usage > > > > > > > > Actually it wont. It will only remove the oldest PCAP file. It's best > > to keep those PCAP files on their own partition. > > > > The old flow records in mysql actually get removed automatically based > > on the oldest packet in the store. So you won't have records that are > > older than the pcaps. > > > > > > > > -L > > > > > > > > > > > > On Tue, Aug 18, 2015 at 6:21 PM, Igor Kaplan <igo...@gm...> > wrote: > > > > Hi Leon, all, > > > > > > > > I have one more question please. > > > > > > > > Based on the documentation the following line in the openfpc config > > file restricts the space usage of captured data to 50 percent: > > > > PCAP_SPACE=50 > > > > > > > > So, if the data size exceeds 50 percent old files will be deleted > > automatically? > > > > Will openfpc also delete the old MySQL session tables? > > > > > > > > Many thanks and all the best! > > > > > > > > -Igor. > > > > > > > > > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf > > Of Leon Ward > > Sent: Monday, August 17, 2015 11:51 AM > > To: Igor Kaplan > > Cc: ope...@li... > > Subject: Re: [Openfpc-users] Openfpc usage > > > > > > > > Hi, > > > > > > > > Documentation is really one of the places that really needs some extra > > focus. > > > > > > > > The best docs I can point you to are in that folder, plus there is > > some out-of date info on my blog http://www.leonward.com. > > > > I actually delivered a presentation at Defcon last weekend all about > > OpenFPC. I have forwarded the slides separately. Hopefully that will > > help as well. > > > > > > > > As for your specific question about OpenFPC GUI. That's actually now > > been deprecated as it's no longer relevant for how it functions in a > > distributed manner. The OpenFPC-Chrome Extension will be the next best > > thing for interacting with the QueueDaemon remotely in a GUI-like way. > > > > > > > > Cheers, > > > > > > > > -L > > > > > > > > > > > > > > > > On Mon, Aug 17, 2015 at 4:25 PM, Igor Kaplan <igo...@gm...> > wrote: > > > > Hi All, > > > > > > > > My name is Igor. I just found the OpenFPC and evaluating it. Looks > > like it is very good tool. > > > > I successfully installed on Ubuntu 14.4 with Perl 5.18 > > > > I have installed the OpenFPC-master, so it is the latest code. > > > > > > > > Now I would like to find out if there is more documentation beside > > files which I could find under docs directory. > > > > For example the INSTALL.md refers to the USAGE document, however I > > was not able to find it anywhere > > > > > > > > I am looking for the usage other then basic, just to find out, what > > are my advanced options. > > > > > > > > For example the openfpc-dbmaint.sh script is also able to create the > > gui database, I wonder, what it is for? > > > > > > > > The OpenFPC looks to be very powerful, just would like to understand > > it as best as I can. > > > > > > > > Would so much appreciate any reply?s. > > > > > > > > Many thanks. > > > > > > > > Igor. > > > > > > > > > > ---------------------------------------------------------------------- > > ------ > > -- > > > > _______________________________________________ > > Openfpc-users mailing list > > Ope...@li... > > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > > > > > > > > ---------------------------------------------------------------------- > > ------ > > -- > > > > _______________________________________________ > > Openfpc-users mailing list > > Ope...@li... > > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > > > > > > > > > > -------------- next part -------------- An HTML attachment was > > scrubbed... > > > > ------------------------------ > > > > ---------------------------------------------------------------------- > > ------ > > -- > > > > > > ------------------------------ > > > > _______________________________________________ > > Openfpc-users mailing list > > Ope...@li... > > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > > > > End of Openfpc-users Digest, Vol 2, Issue 4 > > ******************************************* > > > > > > ---------------------------------------------------------------------- > > -------- _______________________________________________ > > Openfpc-users mailing list > > Ope...@li... > > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > |
|
From: Igor K. <igo...@gm...> - 2015-09-08 13:42:31
|
Hi Leon, I am still having problems with the input bpf filter. I wonder, if you could please answer my question below. Not sure, if I am doing everything correctly, however as my best understanding, I do. Many thanks. -Igor. -----Original Message----- From: Igor Kaplan [mailto:igo...@gm...] Sent: Friday, August 28, 2015 4:32 PM To: 'Leon Ward' Cc: ope...@li... Subject: BPF filter Hi Leon, Not sure, if I am doing everything correctly, however my BPF filter in openfpc-default.conf does not look to be working. I have a line in my openfpc-default.conf: BPF_File=/var/filter.bpf While starting openfpc -a start --verbose I am able to see, the proper bpf file is found and loaded. My filter.bpf is very simple: Port 502 So it should capture packets from port 502 only However after fetching the data I still see packets from other ports. I wonder, do I understand correctly, the BPF_FILE in the config file will restrict, which packets are captured? So using the bpf file as above I should not see any packets from other ports beside 502? Or it is something different? Many thanks and have a nice weekend! -Igor -----Original Message----- From: Leon Ward [mailto:leo...@gm...] On Behalf Of Leon Ward Sent: Tuesday, August 25, 2015 4:59 PM To: Igor Kaplan Cc: <ope...@li...> Subject: Re: [Openfpc-users] Openfpc-users Digest, Vol 2, Issue 4 That's where the pcaps should live, and they will grow to the max percentage that you allow in the node config. What does an openfpc-client -a status show? Also what's a df -h look like? The pcaps will auto-prune unless something has gone wrong along the way.... Thinking out loud, what's an ls of your pcaps directory look like? Have you got multiple nodes running on one box? -L Sent from a mobile device. Apologies for any typos but they happen. > On 25 Aug 2015, at 16:25, Igor Kaplan <igo...@gm...> wrote: > > Hi Leon, > > Could you please help me with following. > I am running openfpc for several days already and now I am out of > space on my Ubuntu box which runs openfpc Under /var/tmp/openfpc I see > directories, some of which contain number of large files: > api-pcaps extracted pcap session > > I wonder, can I safely delete data under any of those directories above? > Could you please let me know, which directory I can empty without > breaking openfpc functionality? > > Is there any way to clean all captured data and start fresh? > > Many thanks. > > -Igor. > > > -----Original Message----- > From: ope...@li... > [mailto:ope...@li...] > Sent: Thursday, August 20, 2015 8:01 PM > To: ope...@li... > Subject: Openfpc-users Digest, Vol 2, Issue 4 > > Send Openfpc-users mailing list submissions to > ope...@li... > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.sourceforge.net/lists/listinfo/openfpc-users > or, via email, send a message with subject or body 'help' to > ope...@li... > > You can reach the person managing the list at > ope...@li... > > When replying, please edit your Subject line so it is more specific > than > "Re: Contents of Openfpc-users digest..." > > > Today's Topics: > > 1. Re: Openfpc usage (Igor Kaplan) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 20 Aug 2015 20:01:08 -0400 > From: "Igor Kaplan" <igo...@gm...> > Subject: Re: [Openfpc-users] Openfpc usage > To: <ope...@li...> > Message-ID: <000c01d0dba4$7c3c9910$74b5cb30$@gmail.com> > Content-Type: text/plain; charset="utf-8" > > Also sending my reply to the list, sorry, forgot to include it. > > > > > > From: Igor Kaplan [mailto:igo...@gm...] > Sent: Thursday, August 20, 2015 6:44 PM > To: 'Leon Ward' > Subject: RE: [Openfpc-users] Openfpc usage > > > > Version of mergecap: > > Mergecap 1.10.6 (v1.10.6 from master-1.10) > > > > Linux is Ubuntu 14.04.2 LTS > > > > File list.pcap > > list.pcap: pcap-ng capture file - version 1.0 > > > > Thanks. > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf > Of Leon Ward > Sent: Thursday, August 20, 2015 6:23 PM > To: Igor Kaplan > Cc: ope...@li... > Subject: Re: [Openfpc-users] Openfpc usage > > > > What's your platform, version of mergecap etc. > > Also, if you '$ file list.pcap' what does it say? > > > > -L > > > > > > On Thu, Aug 20, 2015 at 10:15 PM, Igor Kaplan <igo...@gm...> wrote: > > Hi Leon, > > > > Need your help please once again. > > Got the following problem and spent several hours trying to solve it. > > > > When making the API call to fetch the pcap data I am getting the data > in pcapng format. > > The OpenFPC is using the mergecap to merge pcap files and by default > mergecap creates the output in pcapng format instead of pcap. > > > > I have changed the following line in openfpc-default.conf file > > MERGECAP=/usr/bin/mergecap -F pcap > > > > This helped when I use the openfpc-client command to create pcap > files, however when I use curl to fetch the data I still receive the > output in pcapng format. > > > > curl -k > 192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F-C061B4C > E8A48\ > <http://192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F > -C061B > 4CE8A48%5C&stime=20150818%2010:00%5C&etime=20150818%2010:30%5C&dpt=22> > &stime=20150818%2010:00\&etime=20150818%2010:30\&dpt=22 > list.pcap > > > > cat list.pcap|tshark -i- > > Capturing on 'Standard input' > > tshark: Unrecognized libpcap format > > > > Looks like in case of API call the mergecap utility is not used at all. > And I was not able to find in the code how merging is done in this case. > > > > Could you please help me. Is it possible to make the fetch API call > to return the data in pcap format? > > > > Thanks so much! > > > > Igor > > > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf > Of Leon Ward > Sent: Tuesday, August 18, 2015 1:29 PM > > > To: Igor Kaplan > Cc: ope...@li... > Subject: Re: [Openfpc-users] Openfpc usage > > > > Actually it wont. It will only remove the oldest PCAP file. It's best > to keep those PCAP files on their own partition. > > The old flow records in mysql actually get removed automatically based > on the oldest packet in the store. So you won't have records that are > older than the pcaps. > > > > -L > > > > > > On Tue, Aug 18, 2015 at 6:21 PM, Igor Kaplan <igo...@gm...> wrote: > > Hi Leon, all, > > > > I have one more question please. > > > > Based on the documentation the following line in the openfpc config > file restricts the space usage of captured data to 50 percent: > > PCAP_SPACE=50 > > > > So, if the data size exceeds 50 percent old files will be deleted > automatically? > > Will openfpc also delete the old MySQL session tables? > > > > Many thanks and all the best! > > > > -Igor. > > > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf > Of Leon Ward > Sent: Monday, August 17, 2015 11:51 AM > To: Igor Kaplan > Cc: ope...@li... > Subject: Re: [Openfpc-users] Openfpc usage > > > > Hi, > > > > Documentation is really one of the places that really needs some extra > focus. > > > > The best docs I can point you to are in that folder, plus there is > some out-of date info on my blog http://www.leonward.com. > > I actually delivered a presentation at Defcon last weekend all about > OpenFPC. I have forwarded the slides separately. Hopefully that will > help as well. > > > > As for your specific question about OpenFPC GUI. That's actually now > been deprecated as it's no longer relevant for how it functions in a > distributed manner. The OpenFPC-Chrome Extension will be the next best > thing for interacting with the QueueDaemon remotely in a GUI-like way. > > > > Cheers, > > > > -L > > > > > > > > On Mon, Aug 17, 2015 at 4:25 PM, Igor Kaplan <igo...@gm...> wrote: > > Hi All, > > > > My name is Igor. I just found the OpenFPC and evaluating it. Looks > like it is very good tool. > > I successfully installed on Ubuntu 14.4 with Perl 5.18 > > I have installed the OpenFPC-master, so it is the latest code. > > > > Now I would like to find out if there is more documentation beside > files which I could find under docs directory. > > For example the INSTALL.md refers to the USAGE document, however I > was not able to find it anywhere > > > > I am looking for the usage other then basic, just to find out, what > are my advanced options. > > > > For example the openfpc-dbmaint.sh script is also able to create the > gui database, I wonder, what it is for? > > > > The OpenFPC looks to be very powerful, just would like to understand > it as best as I can. > > > > Would so much appreciate any reply?s. > > > > Many thanks. > > > > Igor. > > > > > ---------------------------------------------------------------------- > ------ > -- > > _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > > > ---------------------------------------------------------------------- > ------ > -- > > _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > > > > -------------- next part -------------- An HTML attachment was > scrubbed... > > ------------------------------ > > ---------------------------------------------------------------------- > ------ > -- > > > ------------------------------ > > _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > End of Openfpc-users Digest, Vol 2, Issue 4 > ******************************************* > > > ---------------------------------------------------------------------- > -------- _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users |
|
From: Leon W. <le...@rm...> - 2015-09-01 12:50:45
|
How long is a bit of string? Really hard to answer this question. I do know of people that have used dedicate network capture cards with daemonlogger and OpenFPC to grow its single device scale. Daemonlogger doesn't record dropped packet stats that I've noticed, if it did then it would be attractive to report these back to a user. I'll take a look at it. -L On Thu, Aug 27, 2015 at 3:33 PM, Igor Kaplan <igo...@gm...> wrote: > Hi Leon, > > Would like please to ask you the following question: > How much traffic, in Gbps or pps, can the openfpc keep up with on a > typical > server? Does the openfpc application provide any indication (in terms of > uncaptured packet statistics or log) when it fails to keep up with incoming > traffic? > > Many thanks. > > -Igor > > -----Original Message----- > From: Leon Ward [mailto:leo...@gm...] On Behalf Of Leon Ward > Sent: Tuesday, August 25, 2015 4:59 PM > To: Igor Kaplan > Cc: <ope...@li...> > Subject: Re: [Openfpc-users] Openfpc-users Digest, Vol 2, Issue 4 > > That's where the pcaps should live, and they will grow to the max > percentage > that you allow in the node config. > > What does an openfpc-client -a status show? > > Also what's a df -h look like? > > The pcaps will auto-prune unless something has gone wrong along the way.... > > Thinking out loud, what's an ls of your pcaps directory look like? Have you > got multiple nodes running on one box? > > -L > > Sent from a mobile device. Apologies for any typos but they happen. > > > On 25 Aug 2015, at 16:25, Igor Kaplan <igo...@gm...> wrote: > > > > Hi Leon, > > > > Could you please help me with following. > > I am running openfpc for several days already and now I am out of > > space on my Ubuntu box which runs openfpc Under /var/tmp/openfpc I see > > directories, some of which contain number of large files: > > api-pcaps extracted pcap session > > > > I wonder, can I safely delete data under any of those directories above? > > Could you please let me know, which directory I can empty without > > breaking openfpc functionality? > > > > Is there any way to clean all captured data and start fresh? > > > > Many thanks. > > > > -Igor. > > > > > > -----Original Message----- > > From: ope...@li... > > [mailto:ope...@li...] > > Sent: Thursday, August 20, 2015 8:01 PM > > To: ope...@li... > > Subject: Openfpc-users Digest, Vol 2, Issue 4 > > > > Send Openfpc-users mailing list submissions to > > ope...@li... > > > > To subscribe or unsubscribe via the World Wide Web, visit > > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > or, via email, send a message with subject or body 'help' to > > ope...@li... > > > > You can reach the person managing the list at > > ope...@li... > > > > When replying, please edit your Subject line so it is more specific > > than > > "Re: Contents of Openfpc-users digest..." > > > > > > Today's Topics: > > > > 1. Re: Openfpc usage (Igor Kaplan) > > > > > > ---------------------------------------------------------------------- > > > > Message: 1 > > Date: Thu, 20 Aug 2015 20:01:08 -0400 > > From: "Igor Kaplan" <igo...@gm...> > > Subject: Re: [Openfpc-users] Openfpc usage > > To: <ope...@li...> > > Message-ID: <000c01d0dba4$7c3c9910$74b5cb30$@gmail.com> > > Content-Type: text/plain; charset="utf-8" > > > > Also sending my reply to the list, sorry, forgot to include it. > > > > > > > > > > > > From: Igor Kaplan [mailto:igo...@gm...] > > Sent: Thursday, August 20, 2015 6:44 PM > > To: 'Leon Ward' > > Subject: RE: [Openfpc-users] Openfpc usage > > > > > > > > Version of mergecap: > > > > Mergecap 1.10.6 (v1.10.6 from master-1.10) > > > > > > > > Linux is Ubuntu 14.04.2 LTS > > > > > > > > File list.pcap > > > > list.pcap: pcap-ng capture file - version 1.0 > > > > > > > > Thanks. > > > > > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf > > Of Leon Ward > > Sent: Thursday, August 20, 2015 6:23 PM > > To: Igor Kaplan > > Cc: ope...@li... > > Subject: Re: [Openfpc-users] Openfpc usage > > > > > > > > What's your platform, version of mergecap etc. > > > > Also, if you '$ file list.pcap' what does it say? > > > > > > > > -L > > > > > > > > > > > > On Thu, Aug 20, 2015 at 10:15 PM, Igor Kaplan <igo...@gm...> > wrote: > > > > Hi Leon, > > > > > > > > Need your help please once again. > > > > Got the following problem and spent several hours trying to solve it. > > > > > > > > When making the API call to fetch the pcap data I am getting the data > > in pcapng format. > > > > The OpenFPC is using the mergecap to merge pcap files and by default > > mergecap creates the output in pcapng format instead of pcap. > > > > > > > > I have changed the following line in openfpc-default.conf file > > > > MERGECAP=/usr/bin/mergecap -F pcap > > > > > > > > This helped when I use the openfpc-client command to create pcap > > files, however when I use curl to fetch the data I still receive the > > output in pcapng format. > > > > > > > > curl -k > > 192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F-C061B4C > > E8A48\ > > <http://192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F > > -C061B > > 4CE8A48%5C&stime=20150818%2010:00%5C&etime=20150818%2010:30%5C&dpt=22> > > &stime=20150818%2010:00\&etime=20150818%2010:30\&dpt=22 > list.pcap > > > > > > > > cat list.pcap|tshark -i- > > > > Capturing on 'Standard input' > > > > tshark: Unrecognized libpcap format > > > > > > > > Looks like in case of API call the mergecap utility is not used at all. > > And I was not able to find in the code how merging is done in this case. > > > > > > > > Could you please help me. Is it possible to make the fetch API call > > to return the data in pcap format? > > > > > > > > Thanks so much! > > > > > > > > Igor > > > > > > > > > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf > > Of Leon Ward > > Sent: Tuesday, August 18, 2015 1:29 PM > > > > > > To: Igor Kaplan > > Cc: ope...@li... > > Subject: Re: [Openfpc-users] Openfpc usage > > > > > > > > Actually it wont. It will only remove the oldest PCAP file. It's best > > to keep those PCAP files on their own partition. > > > > The old flow records in mysql actually get removed automatically based > > on the oldest packet in the store. So you won't have records that are > > older than the pcaps. > > > > > > > > -L > > > > > > > > > > > > On Tue, Aug 18, 2015 at 6:21 PM, Igor Kaplan <igo...@gm...> > wrote: > > > > Hi Leon, all, > > > > > > > > I have one more question please. > > > > > > > > Based on the documentation the following line in the openfpc config > > file restricts the space usage of captured data to 50 percent: > > > > PCAP_SPACE=50 > > > > > > > > So, if the data size exceeds 50 percent old files will be deleted > > automatically? > > > > Will openfpc also delete the old MySQL session tables? > > > > > > > > Many thanks and all the best! > > > > > > > > -Igor. > > > > > > > > > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf > > Of Leon Ward > > Sent: Monday, August 17, 2015 11:51 AM > > To: Igor Kaplan > > Cc: ope...@li... > > Subject: Re: [Openfpc-users] Openfpc usage > > > > > > > > Hi, > > > > > > > > Documentation is really one of the places that really needs some extra > > focus. > > > > > > > > The best docs I can point you to are in that folder, plus there is > > some out-of date info on my blog http://www.leonward.com. > > > > I actually delivered a presentation at Defcon last weekend all about > > OpenFPC. I have forwarded the slides separately. Hopefully that will > > help as well. > > > > > > > > As for your specific question about OpenFPC GUI. That's actually now > > been deprecated as it's no longer relevant for how it functions in a > > distributed manner. The OpenFPC-Chrome Extension will be the next best > > thing for interacting with the QueueDaemon remotely in a GUI-like way. > > > > > > > > Cheers, > > > > > > > > -L > > > > > > > > > > > > > > > > On Mon, Aug 17, 2015 at 4:25 PM, Igor Kaplan <igo...@gm...> > wrote: > > > > Hi All, > > > > > > > > My name is Igor. I just found the OpenFPC and evaluating it. Looks > > like it is very good tool. > > > > I successfully installed on Ubuntu 14.4 with Perl 5.18 > > > > I have installed the OpenFPC-master, so it is the latest code. > > > > > > > > Now I would like to find out if there is more documentation beside > > files which I could find under docs directory. > > > > For example the INSTALL.md refers to the USAGE document, however I > > was not able to find it anywhere > > > > > > > > I am looking for the usage other then basic, just to find out, what > > are my advanced options. > > > > > > > > For example the openfpc-dbmaint.sh script is also able to create the > > gui database, I wonder, what it is for? > > > > > > > > The OpenFPC looks to be very powerful, just would like to understand > > it as best as I can. > > > > > > > > Would so much appreciate any reply?s. > > > > > > > > Many thanks. > > > > > > > > Igor. > > > > > > > > > > ---------------------------------------------------------------------- > > ------ > > -- > > > > _______________________________________________ > > Openfpc-users mailing list > > Ope...@li... > > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > > > > > > > > ---------------------------------------------------------------------- > > ------ > > -- > > > > _______________________________________________ > > Openfpc-users mailing list > > Ope...@li... > > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > > > > > > > > > > -------------- next part -------------- An HTML attachment was > > scrubbed... > > > > ------------------------------ > > > > ---------------------------------------------------------------------- > > ------ > > -- > > > > > > ------------------------------ > > > > _______________________________________________ > > Openfpc-users mailing list > > Ope...@li... > > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > > > > End of Openfpc-users Digest, Vol 2, Issue 4 > > ******************************************* > > > > > > ---------------------------------------------------------------------- > > -------- _______________________________________________ > > Openfpc-users mailing list > > Ope...@li... > > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > |
|
From: Igor K. <igo...@gm...> - 2015-08-28 20:32:12
|
Hi Leon, Not sure, if I am doing everything correctly, however my BPF filter in openfpc-default.conf does not look to be working. I have a line in my openfpc-default.conf: BPF_File=/var/filter.bpf While starting openfpc -a start --verbose I am able to see, the proper bpf file is found and loaded. My filter.bpf is very simple: Port 502 So it should capture packets from port 502 only However after fetching the data I still see packets from other ports. I wonder, do I understand correctly, the BPF_FILE in the config file will restrict, which packets are captured? So using the bpf file as above I should not see any packets from other ports beside 502? Or it is something different? Many thanks and have a nice weekend! -Igor -----Original Message----- From: Leon Ward [mailto:leo...@gm...] On Behalf Of Leon Ward Sent: Tuesday, August 25, 2015 4:59 PM To: Igor Kaplan Cc: <ope...@li...> Subject: Re: [Openfpc-users] Openfpc-users Digest, Vol 2, Issue 4 That's where the pcaps should live, and they will grow to the max percentage that you allow in the node config. What does an openfpc-client -a status show? Also what's a df -h look like? The pcaps will auto-prune unless something has gone wrong along the way.... Thinking out loud, what's an ls of your pcaps directory look like? Have you got multiple nodes running on one box? -L Sent from a mobile device. Apologies for any typos but they happen. > On 25 Aug 2015, at 16:25, Igor Kaplan <igo...@gm...> wrote: > > Hi Leon, > > Could you please help me with following. > I am running openfpc for several days already and now I am out of > space on my Ubuntu box which runs openfpc Under /var/tmp/openfpc I see > directories, some of which contain number of large files: > api-pcaps extracted pcap session > > I wonder, can I safely delete data under any of those directories above? > Could you please let me know, which directory I can empty without > breaking openfpc functionality? > > Is there any way to clean all captured data and start fresh? > > Many thanks. > > -Igor. > > > -----Original Message----- > From: ope...@li... > [mailto:ope...@li...] > Sent: Thursday, August 20, 2015 8:01 PM > To: ope...@li... > Subject: Openfpc-users Digest, Vol 2, Issue 4 > > Send Openfpc-users mailing list submissions to > ope...@li... > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.sourceforge.net/lists/listinfo/openfpc-users > or, via email, send a message with subject or body 'help' to > ope...@li... > > You can reach the person managing the list at > ope...@li... > > When replying, please edit your Subject line so it is more specific > than > "Re: Contents of Openfpc-users digest..." > > > Today's Topics: > > 1. Re: Openfpc usage (Igor Kaplan) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 20 Aug 2015 20:01:08 -0400 > From: "Igor Kaplan" <igo...@gm...> > Subject: Re: [Openfpc-users] Openfpc usage > To: <ope...@li...> > Message-ID: <000c01d0dba4$7c3c9910$74b5cb30$@gmail.com> > Content-Type: text/plain; charset="utf-8" > > Also sending my reply to the list, sorry, forgot to include it. > > > > > > From: Igor Kaplan [mailto:igo...@gm...] > Sent: Thursday, August 20, 2015 6:44 PM > To: 'Leon Ward' > Subject: RE: [Openfpc-users] Openfpc usage > > > > Version of mergecap: > > Mergecap 1.10.6 (v1.10.6 from master-1.10) > > > > Linux is Ubuntu 14.04.2 LTS > > > > File list.pcap > > list.pcap: pcap-ng capture file - version 1.0 > > > > Thanks. > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf > Of Leon Ward > Sent: Thursday, August 20, 2015 6:23 PM > To: Igor Kaplan > Cc: ope...@li... > Subject: Re: [Openfpc-users] Openfpc usage > > > > What's your platform, version of mergecap etc. > > Also, if you '$ file list.pcap' what does it say? > > > > -L > > > > > > On Thu, Aug 20, 2015 at 10:15 PM, Igor Kaplan <igo...@gm...> wrote: > > Hi Leon, > > > > Need your help please once again. > > Got the following problem and spent several hours trying to solve it. > > > > When making the API call to fetch the pcap data I am getting the data > in pcapng format. > > The OpenFPC is using the mergecap to merge pcap files and by default > mergecap creates the output in pcapng format instead of pcap. > > > > I have changed the following line in openfpc-default.conf file > > MERGECAP=/usr/bin/mergecap -F pcap > > > > This helped when I use the openfpc-client command to create pcap > files, however when I use curl to fetch the data I still receive the > output in pcapng format. > > > > curl -k > 192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F-C061B4C > E8A48\ > <http://192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F > -C061B > 4CE8A48%5C&stime=20150818%2010:00%5C&etime=20150818%2010:30%5C&dpt=22> > &stime=20150818%2010:00\&etime=20150818%2010:30\&dpt=22 > list.pcap > > > > cat list.pcap|tshark -i- > > Capturing on 'Standard input' > > tshark: Unrecognized libpcap format > > > > Looks like in case of API call the mergecap utility is not used at all. > And I was not able to find in the code how merging is done in this case. > > > > Could you please help me. Is it possible to make the fetch API call > to return the data in pcap format? > > > > Thanks so much! > > > > Igor > > > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf > Of Leon Ward > Sent: Tuesday, August 18, 2015 1:29 PM > > > To: Igor Kaplan > Cc: ope...@li... > Subject: Re: [Openfpc-users] Openfpc usage > > > > Actually it wont. It will only remove the oldest PCAP file. It's best > to keep those PCAP files on their own partition. > > The old flow records in mysql actually get removed automatically based > on the oldest packet in the store. So you won't have records that are > older than the pcaps. > > > > -L > > > > > > On Tue, Aug 18, 2015 at 6:21 PM, Igor Kaplan <igo...@gm...> wrote: > > Hi Leon, all, > > > > I have one more question please. > > > > Based on the documentation the following line in the openfpc config > file restricts the space usage of captured data to 50 percent: > > PCAP_SPACE=50 > > > > So, if the data size exceeds 50 percent old files will be deleted > automatically? > > Will openfpc also delete the old MySQL session tables? > > > > Many thanks and all the best! > > > > -Igor. > > > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf > Of Leon Ward > Sent: Monday, August 17, 2015 11:51 AM > To: Igor Kaplan > Cc: ope...@li... > Subject: Re: [Openfpc-users] Openfpc usage > > > > Hi, > > > > Documentation is really one of the places that really needs some extra > focus. > > > > The best docs I can point you to are in that folder, plus there is > some out-of date info on my blog http://www.leonward.com. > > I actually delivered a presentation at Defcon last weekend all about > OpenFPC. I have forwarded the slides separately. Hopefully that will > help as well. > > > > As for your specific question about OpenFPC GUI. That's actually now > been deprecated as it's no longer relevant for how it functions in a > distributed manner. The OpenFPC-Chrome Extension will be the next best > thing for interacting with the QueueDaemon remotely in a GUI-like way. > > > > Cheers, > > > > -L > > > > > > > > On Mon, Aug 17, 2015 at 4:25 PM, Igor Kaplan <igo...@gm...> wrote: > > Hi All, > > > > My name is Igor. I just found the OpenFPC and evaluating it. Looks > like it is very good tool. > > I successfully installed on Ubuntu 14.4 with Perl 5.18 > > I have installed the OpenFPC-master, so it is the latest code. > > > > Now I would like to find out if there is more documentation beside > files which I could find under docs directory. > > For example the INSTALL.md refers to the USAGE document, however I > was not able to find it anywhere > > > > I am looking for the usage other then basic, just to find out, what > are my advanced options. > > > > For example the openfpc-dbmaint.sh script is also able to create the > gui database, I wonder, what it is for? > > > > The OpenFPC looks to be very powerful, just would like to understand > it as best as I can. > > > > Would so much appreciate any reply?s. > > > > Many thanks. > > > > Igor. > > > > > ---------------------------------------------------------------------- > ------ > -- > > _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > > > ---------------------------------------------------------------------- > ------ > -- > > _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > > > > -------------- next part -------------- An HTML attachment was > scrubbed... > > ------------------------------ > > ---------------------------------------------------------------------- > ------ > -- > > > ------------------------------ > > _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > End of Openfpc-users Digest, Vol 2, Issue 4 > ******************************************* > > > ---------------------------------------------------------------------- > -------- _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users |
|
From: Igor K. <igo...@gm...> - 2015-08-27 14:34:00
|
Hi Leon, Would like please to ask you the following question: How much traffic, in Gbps or pps, can the openfpc keep up with on a typical server? Does the openfpc application provide any indication (in terms of uncaptured packet statistics or log) when it fails to keep up with incoming traffic? Many thanks. -Igor -----Original Message----- From: Leon Ward [mailto:leo...@gm...] On Behalf Of Leon Ward Sent: Tuesday, August 25, 2015 4:59 PM To: Igor Kaplan Cc: <ope...@li...> Subject: Re: [Openfpc-users] Openfpc-users Digest, Vol 2, Issue 4 That's where the pcaps should live, and they will grow to the max percentage that you allow in the node config. What does an openfpc-client -a status show? Also what's a df -h look like? The pcaps will auto-prune unless something has gone wrong along the way.... Thinking out loud, what's an ls of your pcaps directory look like? Have you got multiple nodes running on one box? -L Sent from a mobile device. Apologies for any typos but they happen. > On 25 Aug 2015, at 16:25, Igor Kaplan <igo...@gm...> wrote: > > Hi Leon, > > Could you please help me with following. > I am running openfpc for several days already and now I am out of > space on my Ubuntu box which runs openfpc Under /var/tmp/openfpc I see > directories, some of which contain number of large files: > api-pcaps extracted pcap session > > I wonder, can I safely delete data under any of those directories above? > Could you please let me know, which directory I can empty without > breaking openfpc functionality? > > Is there any way to clean all captured data and start fresh? > > Many thanks. > > -Igor. > > > -----Original Message----- > From: ope...@li... > [mailto:ope...@li...] > Sent: Thursday, August 20, 2015 8:01 PM > To: ope...@li... > Subject: Openfpc-users Digest, Vol 2, Issue 4 > > Send Openfpc-users mailing list submissions to > ope...@li... > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.sourceforge.net/lists/listinfo/openfpc-users > or, via email, send a message with subject or body 'help' to > ope...@li... > > You can reach the person managing the list at > ope...@li... > > When replying, please edit your Subject line so it is more specific > than > "Re: Contents of Openfpc-users digest..." > > > Today's Topics: > > 1. Re: Openfpc usage (Igor Kaplan) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 20 Aug 2015 20:01:08 -0400 > From: "Igor Kaplan" <igo...@gm...> > Subject: Re: [Openfpc-users] Openfpc usage > To: <ope...@li...> > Message-ID: <000c01d0dba4$7c3c9910$74b5cb30$@gmail.com> > Content-Type: text/plain; charset="utf-8" > > Also sending my reply to the list, sorry, forgot to include it. > > > > > > From: Igor Kaplan [mailto:igo...@gm...] > Sent: Thursday, August 20, 2015 6:44 PM > To: 'Leon Ward' > Subject: RE: [Openfpc-users] Openfpc usage > > > > Version of mergecap: > > Mergecap 1.10.6 (v1.10.6 from master-1.10) > > > > Linux is Ubuntu 14.04.2 LTS > > > > File list.pcap > > list.pcap: pcap-ng capture file - version 1.0 > > > > Thanks. > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf > Of Leon Ward > Sent: Thursday, August 20, 2015 6:23 PM > To: Igor Kaplan > Cc: ope...@li... > Subject: Re: [Openfpc-users] Openfpc usage > > > > What's your platform, version of mergecap etc. > > Also, if you '$ file list.pcap' what does it say? > > > > -L > > > > > > On Thu, Aug 20, 2015 at 10:15 PM, Igor Kaplan <igo...@gm...> wrote: > > Hi Leon, > > > > Need your help please once again. > > Got the following problem and spent several hours trying to solve it. > > > > When making the API call to fetch the pcap data I am getting the data > in pcapng format. > > The OpenFPC is using the mergecap to merge pcap files and by default > mergecap creates the output in pcapng format instead of pcap. > > > > I have changed the following line in openfpc-default.conf file > > MERGECAP=/usr/bin/mergecap -F pcap > > > > This helped when I use the openfpc-client command to create pcap > files, however when I use curl to fetch the data I still receive the > output in pcapng format. > > > > curl -k > 192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F-C061B4C > E8A48\ > <http://192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F > -C061B > 4CE8A48%5C&stime=20150818%2010:00%5C&etime=20150818%2010:30%5C&dpt=22> > &stime=20150818%2010:00\&etime=20150818%2010:30\&dpt=22 > list.pcap > > > > cat list.pcap|tshark -i- > > Capturing on 'Standard input' > > tshark: Unrecognized libpcap format > > > > Looks like in case of API call the mergecap utility is not used at all. > And I was not able to find in the code how merging is done in this case. > > > > Could you please help me. Is it possible to make the fetch API call > to return the data in pcap format? > > > > Thanks so much! > > > > Igor > > > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf > Of Leon Ward > Sent: Tuesday, August 18, 2015 1:29 PM > > > To: Igor Kaplan > Cc: ope...@li... > Subject: Re: [Openfpc-users] Openfpc usage > > > > Actually it wont. It will only remove the oldest PCAP file. It's best > to keep those PCAP files on their own partition. > > The old flow records in mysql actually get removed automatically based > on the oldest packet in the store. So you won't have records that are > older than the pcaps. > > > > -L > > > > > > On Tue, Aug 18, 2015 at 6:21 PM, Igor Kaplan <igo...@gm...> wrote: > > Hi Leon, all, > > > > I have one more question please. > > > > Based on the documentation the following line in the openfpc config > file restricts the space usage of captured data to 50 percent: > > PCAP_SPACE=50 > > > > So, if the data size exceeds 50 percent old files will be deleted > automatically? > > Will openfpc also delete the old MySQL session tables? > > > > Many thanks and all the best! > > > > -Igor. > > > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf > Of Leon Ward > Sent: Monday, August 17, 2015 11:51 AM > To: Igor Kaplan > Cc: ope...@li... > Subject: Re: [Openfpc-users] Openfpc usage > > > > Hi, > > > > Documentation is really one of the places that really needs some extra > focus. > > > > The best docs I can point you to are in that folder, plus there is > some out-of date info on my blog http://www.leonward.com. > > I actually delivered a presentation at Defcon last weekend all about > OpenFPC. I have forwarded the slides separately. Hopefully that will > help as well. > > > > As for your specific question about OpenFPC GUI. That's actually now > been deprecated as it's no longer relevant for how it functions in a > distributed manner. The OpenFPC-Chrome Extension will be the next best > thing for interacting with the QueueDaemon remotely in a GUI-like way. > > > > Cheers, > > > > -L > > > > > > > > On Mon, Aug 17, 2015 at 4:25 PM, Igor Kaplan <igo...@gm...> wrote: > > Hi All, > > > > My name is Igor. I just found the OpenFPC and evaluating it. Looks > like it is very good tool. > > I successfully installed on Ubuntu 14.4 with Perl 5.18 > > I have installed the OpenFPC-master, so it is the latest code. > > > > Now I would like to find out if there is more documentation beside > files which I could find under docs directory. > > For example the INSTALL.md refers to the USAGE document, however I > was not able to find it anywhere > > > > I am looking for the usage other then basic, just to find out, what > are my advanced options. > > > > For example the openfpc-dbmaint.sh script is also able to create the > gui database, I wonder, what it is for? > > > > The OpenFPC looks to be very powerful, just would like to understand > it as best as I can. > > > > Would so much appreciate any reply?s. > > > > Many thanks. > > > > Igor. > > > > > ---------------------------------------------------------------------- > ------ > -- > > _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > > > ---------------------------------------------------------------------- > ------ > -- > > _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > > > > -------------- next part -------------- An HTML attachment was > scrubbed... > > ------------------------------ > > ---------------------------------------------------------------------- > ------ > -- > > > ------------------------------ > > _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > End of Openfpc-users Digest, Vol 2, Issue 4 > ******************************************* > > > ---------------------------------------------------------------------- > -------- _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users |
|
From: Leon W. <le...@rm...> - 2015-08-26 08:15:47
|
Ahhhh, If when you start daemonlogger you've already got less than the miniumum amount of space left bad things can happen. IIRC I did create a warning on startup to let you know, but it may have been missed if it started at boot time. So if your VM has < 50% of space all the time, up the max space to something like 80%. Once again, its always best to put the pcaps on a second partition so you can handle them without worrying about other files on the filesystem. -L On Tue, Aug 25, 2015 at 10:28 PM, Igor Kaplan <igo...@gm...> wrote: > Hi Leon, > > Thanks so much for your reply. > Actually what happened. > The total size of my vm is 16 GB. > The maximum allowed size in my config is 50 percent > PCAP_SPACE=50 > > However the size of /var/tmp/openfpc/pcap was 14 GB, much more then 50 > percent. > > Unfortunately I already cleaned everything since vm was not operational, so > not able to send the listing of pcap directory. > I am also for now don't use any bpf filters, however still, as I > understand, the size should not have grown more then 50 percent. And I am > using just the single openfpc node on that host. > > I'll watch the size growth in nearest future and see,. > > All the best. > > -Igor > > -----Original Message----- > From: Leon Ward [mailto:leo...@gm...] On Behalf Of Leon Ward > Sent: Tuesday, August 25, 2015 4:59 PM > To: Igor Kaplan > Cc: <ope...@li...> > Subject: Re: [Openfpc-users] Openfpc-users Digest, Vol 2, Issue 4 > > That's where the pcaps should live, and they will grow to the max > percentage > that you allow in the node config. > > What does an openfpc-client -a status show? > > Also what's a df -h look like? > > The pcaps will auto-prune unless something has gone wrong along the way.... > > Thinking out loud, what's an ls of your pcaps directory look like? Have you > got multiple nodes running on one box? > > -L > > Sent from a mobile device. Apologies for any typos but they happen. > > > On 25 Aug 2015, at 16:25, Igor Kaplan <igo...@gm...> wrote: > > > > Hi Leon, > > > > Could you please help me with following. > > I am running openfpc for several days already and now I am out of > > space on my Ubuntu box which runs openfpc Under /var/tmp/openfpc I see > > directories, some of which contain number of large files: > > api-pcaps extracted pcap session > > > > I wonder, can I safely delete data under any of those directories above? > > Could you please let me know, which directory I can empty without > > breaking openfpc functionality? > > > > Is there any way to clean all captured data and start fresh? > > > > Many thanks. > > > > -Igor. > > > > > > -----Original Message----- > > From: ope...@li... > > [mailto:ope...@li...] > > Sent: Thursday, August 20, 2015 8:01 PM > > To: ope...@li... > > Subject: Openfpc-users Digest, Vol 2, Issue 4 > > > > Send Openfpc-users mailing list submissions to > > ope...@li... > > > > To subscribe or unsubscribe via the World Wide Web, visit > > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > or, via email, send a message with subject or body 'help' to > > ope...@li... > > > > You can reach the person managing the list at > > ope...@li... > > > > When replying, please edit your Subject line so it is more specific > > than > > "Re: Contents of Openfpc-users digest..." > > > > > > Today's Topics: > > > > 1. Re: Openfpc usage (Igor Kaplan) > > > > > > ---------------------------------------------------------------------- > > > > Message: 1 > > Date: Thu, 20 Aug 2015 20:01:08 -0400 > > From: "Igor Kaplan" <igo...@gm...> > > Subject: Re: [Openfpc-users] Openfpc usage > > To: <ope...@li...> > > Message-ID: <000c01d0dba4$7c3c9910$74b5cb30$@gmail.com> > > Content-Type: text/plain; charset="utf-8" > > > > Also sending my reply to the list, sorry, forgot to include it. > > > > > > > > > > > > From: Igor Kaplan [mailto:igo...@gm...] > > Sent: Thursday, August 20, 2015 6:44 PM > > To: 'Leon Ward' > > Subject: RE: [Openfpc-users] Openfpc usage > > > > > > > > Version of mergecap: > > > > Mergecap 1.10.6 (v1.10.6 from master-1.10) > > > > > > > > Linux is Ubuntu 14.04.2 LTS > > > > > > > > File list.pcap > > > > list.pcap: pcap-ng capture file - version 1.0 > > > > > > > > Thanks. > > > > > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf > > Of Leon Ward > > Sent: Thursday, August 20, 2015 6:23 PM > > To: Igor Kaplan > > Cc: ope...@li... > > Subject: Re: [Openfpc-users] Openfpc usage > > > > > > > > What's your platform, version of mergecap etc. > > > > Also, if you '$ file list.pcap' what does it say? > > > > > > > > -L > > > > > > > > > > > > On Thu, Aug 20, 2015 at 10:15 PM, Igor Kaplan <igo...@gm...> > wrote: > > > > Hi Leon, > > > > > > > > Need your help please once again. > > > > Got the following problem and spent several hours trying to solve it. > > > > > > > > When making the API call to fetch the pcap data I am getting the data > > in pcapng format. > > > > The OpenFPC is using the mergecap to merge pcap files and by default > > mergecap creates the output in pcapng format instead of pcap. > > > > > > > > I have changed the following line in openfpc-default.conf file > > > > MERGECAP=/usr/bin/mergecap -F pcap > > > > > > > > This helped when I use the openfpc-client command to create pcap > > files, however when I use curl to fetch the data I still receive the > > output in pcapng format. > > > > > > > > curl -k > > 192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F-C061B4C > > E8A48\ > > <http://192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F > > -C061B > > 4CE8A48%5C&stime=20150818%2010:00%5C&etime=20150818%2010:30%5C&dpt=22> > > &stime=20150818%2010:00\&etime=20150818%2010:30\&dpt=22 > list.pcap > > > > > > > > cat list.pcap|tshark -i- > > > > Capturing on 'Standard input' > > > > tshark: Unrecognized libpcap format > > > > > > > > Looks like in case of API call the mergecap utility is not used at all. > > And I was not able to find in the code how merging is done in this case. > > > > > > > > Could you please help me. Is it possible to make the fetch API call > > to return the data in pcap format? > > > > > > > > Thanks so much! > > > > > > > > Igor > > > > > > > > > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf > > Of Leon Ward > > Sent: Tuesday, August 18, 2015 1:29 PM > > > > > > To: Igor Kaplan > > Cc: ope...@li... > > Subject: Re: [Openfpc-users] Openfpc usage > > > > > > > > Actually it wont. It will only remove the oldest PCAP file. It's best > > to keep those PCAP files on their own partition. > > > > The old flow records in mysql actually get removed automatically based > > on the oldest packet in the store. So you won't have records that are > > older than the pcaps. > > > > > > > > -L > > > > > > > > > > > > On Tue, Aug 18, 2015 at 6:21 PM, Igor Kaplan <igo...@gm...> > wrote: > > > > Hi Leon, all, > > > > > > > > I have one more question please. > > > > > > > > Based on the documentation the following line in the openfpc config > > file restricts the space usage of captured data to 50 percent: > > > > PCAP_SPACE=50 > > > > > > > > So, if the data size exceeds 50 percent old files will be deleted > > automatically? > > > > Will openfpc also delete the old MySQL session tables? > > > > > > > > Many thanks and all the best! > > > > > > > > -Igor. > > > > > > > > > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf > > Of Leon Ward > > Sent: Monday, August 17, 2015 11:51 AM > > To: Igor Kaplan > > Cc: ope...@li... > > Subject: Re: [Openfpc-users] Openfpc usage > > > > > > > > Hi, > > > > > > > > Documentation is really one of the places that really needs some extra > > focus. > > > > > > > > The best docs I can point you to are in that folder, plus there is > > some out-of date info on my blog http://www.leonward.com. > > > > I actually delivered a presentation at Defcon last weekend all about > > OpenFPC. I have forwarded the slides separately. Hopefully that will > > help as well. > > > > > > > > As for your specific question about OpenFPC GUI. That's actually now > > been deprecated as it's no longer relevant for how it functions in a > > distributed manner. The OpenFPC-Chrome Extension will be the next best > > thing for interacting with the QueueDaemon remotely in a GUI-like way. > > > > > > > > Cheers, > > > > > > > > -L > > > > > > > > > > > > > > > > On Mon, Aug 17, 2015 at 4:25 PM, Igor Kaplan <igo...@gm...> > wrote: > > > > Hi All, > > > > > > > > My name is Igor. I just found the OpenFPC and evaluating it. Looks > > like it is very good tool. > > > > I successfully installed on Ubuntu 14.4 with Perl 5.18 > > > > I have installed the OpenFPC-master, so it is the latest code. > > > > > > > > Now I would like to find out if there is more documentation beside > > files which I could find under docs directory. > > > > For example the INSTALL.md refers to the USAGE document, however I > > was not able to find it anywhere > > > > > > > > I am looking for the usage other then basic, just to find out, what > > are my advanced options. > > > > > > > > For example the openfpc-dbmaint.sh script is also able to create the > > gui database, I wonder, what it is for? > > > > > > > > The OpenFPC looks to be very powerful, just would like to understand > > it as best as I can. > > > > > > > > Would so much appreciate any reply?s. > > > > > > > > Many thanks. > > > > > > > > Igor. > > > > > > > > > > ---------------------------------------------------------------------- > > ------ > > -- > > > > _______________________________________________ > > Openfpc-users mailing list > > Ope...@li... > > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > > > > > > > > ---------------------------------------------------------------------- > > ------ > > -- > > > > _______________________________________________ > > Openfpc-users mailing list > > Ope...@li... > > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > > > > > > > > > > -------------- next part -------------- An HTML attachment was > > scrubbed... > > > > ------------------------------ > > > > ---------------------------------------------------------------------- > > ------ > > -- > > > > > > ------------------------------ > > > > _______________________________________________ > > Openfpc-users mailing list > > Ope...@li... > > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > > > > End of Openfpc-users Digest, Vol 2, Issue 4 > > ******************************************* > > > > > > ---------------------------------------------------------------------- > > -------- _______________________________________________ > > Openfpc-users mailing list > > Ope...@li... > > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > |
|
From: Leon W. <le...@rm...> - 2015-08-26 08:13:19
|
Hi, It should sort itself out, however if it was a large DB it will use a delete action to cull the tables, that isn't ideal but will work. If it doesn't, let mw know and simply drop and recreate the db before restarting the openfpc. Check the size of the DB by using a status command, check the session count is shrinking to match the new size of the packets. Some of the time window calculations may be a little off while it's shrinking. The shrink will start ~10 minutes after the queue daemon starts up. IIRC it will write something to the logfile about waking up to trim. Something like.. $ sudo openfpc-dbmaint -a drop -t session -c /etc/openfpc/<nodename>.conf $ sudo openfpc-dbmaint -a create -t session -c /etc/openfpc/<nodename>.conf -L On Tue, Aug 25, 2015 at 10:17 PM, John York <Yo...@br...> wrote: > Related question: My /var/openfpc hard drive just failed. I've replace > the drive--do I need to do anything to the database? > > BTW--Love OpenFPC!!! > > John > > -----Original Message----- > From: Leon Ward [mailto:le...@rm...] > Sent: Tuesday, August 25, 2015 4:59 PM > To: Igor Kaplan <igo...@gm...> > Cc: <ope...@li...> < > ope...@li...> > Subject: Re: [Openfpc-users] Openfpc-users Digest, Vol 2, Issue 4 > > That's where the pcaps should live, and they will grow to the max > percentage that you allow in the node config. > > What does an openfpc-client -a status show? > > Also what's a df -h look like? > > The pcaps will auto-prune unless something has gone wrong along the way.... > > Thinking out loud, what's an ls of your pcaps directory look like? Have > you got multiple nodes running on one box? > > -L > > Sent from a mobile device. Apologies for any typos but they happen. > > > On 25 Aug 2015, at 16:25, Igor Kaplan <igo...@gm...> wrote: > > > > Hi Leon, > > > > Could you please help me with following. > > I am running openfpc for several days already and now I am out of > > space on my Ubuntu box which runs openfpc Under /var/tmp/openfpc I see > > directories, some of which contain number of large files: > > api-pcaps extracted pcap session > > > > I wonder, can I safely delete data under any of those directories above? > > Could you please let me know, which directory I can empty without > > breaking openfpc functionality? > > > > Is there any way to clean all captured data and start fresh? > > > > Many thanks. > > > > -Igor. > > > > > > -----Original Message----- > > From: ope...@li... > > [mailto:ope...@li...] > > Sent: Thursday, August 20, 2015 8:01 PM > > To: ope...@li... > > Subject: Openfpc-users Digest, Vol 2, Issue 4 > > > > Send Openfpc-users mailing list submissions to > > ope...@li... > > > > To subscribe or unsubscribe via the World Wide Web, visit > > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > or, via email, send a message with subject or body 'help' to > > ope...@li... > > > > You can reach the person managing the list at > > ope...@li... > > > > When replying, please edit your Subject line so it is more specific > > than > > "Re: Contents of Openfpc-users digest..." > > > > > > Today's Topics: > > > > 1. Re: Openfpc usage (Igor Kaplan) > > > > > > ---------------------------------------------------------------------- > > > > Message: 1 > > Date: Thu, 20 Aug 2015 20:01:08 -0400 > > From: "Igor Kaplan" <igo...@gm...> > > Subject: Re: [Openfpc-users] Openfpc usage > > To: <ope...@li...> > > Message-ID: <000c01d0dba4$7c3c9910$74b5cb30$@gmail.com> > > Content-Type: text/plain; charset="utf-8" > > > > Also sending my reply to the list, sorry, forgot to include it. > > > > > > > > > > > > From: Igor Kaplan [mailto:igo...@gm...] > > Sent: Thursday, August 20, 2015 6:44 PM > > To: 'Leon Ward' > > Subject: RE: [Openfpc-users] Openfpc usage > > > > > > > > Version of mergecap: > > > > Mergecap 1.10.6 (v1.10.6 from master-1.10) > > > > > > > > Linux is Ubuntu 14.04.2 LTS > > > > > > > > File list.pcap > > > > list.pcap: pcap-ng capture file - version 1.0 > > > > > > > > Thanks. > > > > > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf > > Of Leon Ward > > Sent: Thursday, August 20, 2015 6:23 PM > > To: Igor Kaplan > > Cc: ope...@li... > > Subject: Re: [Openfpc-users] Openfpc usage > > > > > > > > What's your platform, version of mergecap etc. > > > > Also, if you '$ file list.pcap' what does it say? > > > > > > > > -L > > > > > > > > > > > > On Thu, Aug 20, 2015 at 10:15 PM, Igor Kaplan <igo...@gm...> > wrote: > > > > Hi Leon, > > > > > > > > Need your help please once again. > > > > Got the following problem and spent several hours trying to solve it. > > > > > > > > When making the API call to fetch the pcap data I am getting the data > > in pcapng format. > > > > The OpenFPC is using the mergecap to merge pcap files and by default > > mergecap creates the output in pcapng format instead of pcap. > > > > > > > > I have changed the following line in openfpc-default.conf file > > > > MERGECAP=/usr/bin/mergecap -F pcap > > > > > > > > This helped when I use the openfpc-client command to create pcap > > files, however when I use curl to fetch the data I still receive the > > output in pcapng format. > > > > > > > > curl -k > > 192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F-C061B4C > > E8A48\ > > <http://192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F > > -C061B > > 4CE8A48%5C&stime=20150818%2010:00%5C&etime=20150818%2010:30%5C&dpt=22> > > &stime=20150818%2010:00\&etime=20150818%2010:30\&dpt=22 > list.pcap > > > > > > > > cat list.pcap|tshark -i- > > > > Capturing on 'Standard input' > > > > tshark: Unrecognized libpcap format > > > > > > > > Looks like in case of API call the mergecap utility is not used at all. > > And I was not able to find in the code how merging is done in this case. > > > > > > > > Could you please help me. Is it possible to make the fetch API call > > to return the data in pcap format? > > > > > > > > Thanks so much! > > > > > > > > Igor > > > > > > > > > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf > > Of Leon Ward > > Sent: Tuesday, August 18, 2015 1:29 PM > > > > > > To: Igor Kaplan > > Cc: ope...@li... > > Subject: Re: [Openfpc-users] Openfpc usage > > > > > > > > Actually it wont. It will only remove the oldest PCAP file. It's best > > to keep those PCAP files on their own partition. > > > > The old flow records in mysql actually get removed automatically based > > on the oldest packet in the store. So you won't have records that are > > older than the pcaps. > > > > > > > > -L > > > > > > > > > > > > On Tue, Aug 18, 2015 at 6:21 PM, Igor Kaplan <igo...@gm...> > wrote: > > > > Hi Leon, all, > > > > > > > > I have one more question please. > > > > > > > > Based on the documentation the following line in the openfpc config > > file restricts the space usage of captured data to 50 percent: > > > > PCAP_SPACE=50 > > > > > > > > So, if the data size exceeds 50 percent old files will be deleted > > automatically? > > > > Will openfpc also delete the old MySQL session tables? > > > > > > > > Many thanks and all the best! > > > > > > > > -Igor. > > > > > > > > > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf > > Of Leon Ward > > Sent: Monday, August 17, 2015 11:51 AM > > To: Igor Kaplan > > Cc: ope...@li... > > Subject: Re: [Openfpc-users] Openfpc usage > > > > > > > > Hi, > > > > > > > > Documentation is really one of the places that really needs some extra > > focus. > > > > > > > > The best docs I can point you to are in that folder, plus there is > > some out-of date info on my blog http://www.leonward.com. > > > > I actually delivered a presentation at Defcon last weekend all about > > OpenFPC. I have forwarded the slides separately. Hopefully that will > > help as well. > > > > > > > > As for your specific question about OpenFPC GUI. That's actually now > > been deprecated as it's no longer relevant for how it functions in a > > distributed manner. The OpenFPC-Chrome Extension will be the next best > > thing for interacting with the QueueDaemon remotely in a GUI-like way. > > > > > > > > Cheers, > > > > > > > > -L > > > > > > > > > > > > > > > > On Mon, Aug 17, 2015 at 4:25 PM, Igor Kaplan <igo...@gm...> > wrote: > > > > Hi All, > > > > > > > > My name is Igor. I just found the OpenFPC and evaluating it. Looks > > like it is very good tool. > > > > I successfully installed on Ubuntu 14.4 with Perl 5.18 > > > > I have installed the OpenFPC-master, so it is the latest code. > > > > > > > > Now I would like to find out if there is more documentation beside > > files which I could find under docs directory. > > > > For example the INSTALL.md refers to the USAGE document, however I > > was not able to find it anywhere > > > > > > > > I am looking for the usage other then basic, just to find out, what > > are my advanced options. > > > > > > > > For example the openfpc-dbmaint.sh script is also able to create the > > gui database, I wonder, what it is for? > > > > > > > > The OpenFPC looks to be very powerful, just would like to understand > > it as best as I can. > > > > > > > > Would so much appreciate any reply?s. > > > > > > > > Many thanks. > > > > > > > > Igor. > > > > > > > > > > ---------------------------------------------------------------------- > > ------ > > -- > > > > _______________________________________________ > > Openfpc-users mailing list > > Ope...@li... > > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > > > > > > > > ---------------------------------------------------------------------- > > ------ > > -- > > > > _______________________________________________ > > Openfpc-users mailing list > > Ope...@li... > > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > > > > > > > > > > -------------- next part -------------- An HTML attachment was > > scrubbed... > > > > ------------------------------ > > > > ---------------------------------------------------------------------- > > ------ > > -- > > > > > > ------------------------------ > > > > _______________________________________________ > > Openfpc-users mailing list > > Ope...@li... > > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > > > > End of Openfpc-users Digest, Vol 2, Issue 4 > > ******************************************* > > > > > > ---------------------------------------------------------------------- > > -------- _______________________________________________ > > Openfpc-users mailing list > > Ope...@li... > > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > ------------------------------------------------------------------------------ > _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users > |
|
From: John Y. <Yo...@br...> - 2015-08-25 21:33:09
|
Related question: My /var/openfpc hard drive just failed. I've replace the drive--do I need to do anything to the database? BTW--Love OpenFPC!!! John -----Original Message----- From: Leon Ward [mailto:le...@rm...] Sent: Tuesday, August 25, 2015 4:59 PM To: Igor Kaplan <igo...@gm...> Cc: <ope...@li...> <ope...@li...> Subject: Re: [Openfpc-users] Openfpc-users Digest, Vol 2, Issue 4 That's where the pcaps should live, and they will grow to the max percentage that you allow in the node config. What does an openfpc-client -a status show? Also what's a df -h look like? The pcaps will auto-prune unless something has gone wrong along the way.... Thinking out loud, what's an ls of your pcaps directory look like? Have you got multiple nodes running on one box? -L Sent from a mobile device. Apologies for any typos but they happen. > On 25 Aug 2015, at 16:25, Igor Kaplan <igo...@gm...> wrote: > > Hi Leon, > > Could you please help me with following. > I am running openfpc for several days already and now I am out of > space on my Ubuntu box which runs openfpc Under /var/tmp/openfpc I see > directories, some of which contain number of large files: > api-pcaps extracted pcap session > > I wonder, can I safely delete data under any of those directories above? > Could you please let me know, which directory I can empty without > breaking openfpc functionality? > > Is there any way to clean all captured data and start fresh? > > Many thanks. > > -Igor. > > > -----Original Message----- > From: ope...@li... > [mailto:ope...@li...] > Sent: Thursday, August 20, 2015 8:01 PM > To: ope...@li... > Subject: Openfpc-users Digest, Vol 2, Issue 4 > > Send Openfpc-users mailing list submissions to > ope...@li... > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.sourceforge.net/lists/listinfo/openfpc-users > or, via email, send a message with subject or body 'help' to > ope...@li... > > You can reach the person managing the list at > ope...@li... > > When replying, please edit your Subject line so it is more specific > than > "Re: Contents of Openfpc-users digest..." > > > Today's Topics: > > 1. Re: Openfpc usage (Igor Kaplan) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 20 Aug 2015 20:01:08 -0400 > From: "Igor Kaplan" <igo...@gm...> > Subject: Re: [Openfpc-users] Openfpc usage > To: <ope...@li...> > Message-ID: <000c01d0dba4$7c3c9910$74b5cb30$@gmail.com> > Content-Type: text/plain; charset="utf-8" > > Also sending my reply to the list, sorry, forgot to include it. > > > > > > From: Igor Kaplan [mailto:igo...@gm...] > Sent: Thursday, August 20, 2015 6:44 PM > To: 'Leon Ward' > Subject: RE: [Openfpc-users] Openfpc usage > > > > Version of mergecap: > > Mergecap 1.10.6 (v1.10.6 from master-1.10) > > > > Linux is Ubuntu 14.04.2 LTS > > > > File list.pcap > > list.pcap: pcap-ng capture file - version 1.0 > > > > Thanks. > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf > Of Leon Ward > Sent: Thursday, August 20, 2015 6:23 PM > To: Igor Kaplan > Cc: ope...@li... > Subject: Re: [Openfpc-users] Openfpc usage > > > > What's your platform, version of mergecap etc. > > Also, if you '$ file list.pcap' what does it say? > > > > -L > > > > > > On Thu, Aug 20, 2015 at 10:15 PM, Igor Kaplan <igo...@gm...> wrote: > > Hi Leon, > > > > Need your help please once again. > > Got the following problem and spent several hours trying to solve it. > > > > When making the API call to fetch the pcap data I am getting the data > in pcapng format. > > The OpenFPC is using the mergecap to merge pcap files and by default > mergecap creates the output in pcapng format instead of pcap. > > > > I have changed the following line in openfpc-default.conf file > > MERGECAP=/usr/bin/mergecap -F pcap > > > > This helped when I use the openfpc-client command to create pcap > files, however when I use curl to fetch the data I still receive the > output in pcapng format. > > > > curl -k > 192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F-C061B4C > E8A48\ > <http://192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F > -C061B > 4CE8A48%5C&stime=20150818%2010:00%5C&etime=20150818%2010:30%5C&dpt=22> > &stime=20150818%2010:00\&etime=20150818%2010:30\&dpt=22 > list.pcap > > > > cat list.pcap|tshark -i- > > Capturing on 'Standard input' > > tshark: Unrecognized libpcap format > > > > Looks like in case of API call the mergecap utility is not used at all. > And I was not able to find in the code how merging is done in this case. > > > > Could you please help me. Is it possible to make the fetch API call > to return the data in pcap format? > > > > Thanks so much! > > > > Igor > > > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf > Of Leon Ward > Sent: Tuesday, August 18, 2015 1:29 PM > > > To: Igor Kaplan > Cc: ope...@li... > Subject: Re: [Openfpc-users] Openfpc usage > > > > Actually it wont. It will only remove the oldest PCAP file. It's best > to keep those PCAP files on their own partition. > > The old flow records in mysql actually get removed automatically based > on the oldest packet in the store. So you won't have records that are > older than the pcaps. > > > > -L > > > > > > On Tue, Aug 18, 2015 at 6:21 PM, Igor Kaplan <igo...@gm...> wrote: > > Hi Leon, all, > > > > I have one more question please. > > > > Based on the documentation the following line in the openfpc config > file restricts the space usage of captured data to 50 percent: > > PCAP_SPACE=50 > > > > So, if the data size exceeds 50 percent old files will be deleted > automatically? > > Will openfpc also delete the old MySQL session tables? > > > > Many thanks and all the best! > > > > -Igor. > > > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf > Of Leon Ward > Sent: Monday, August 17, 2015 11:51 AM > To: Igor Kaplan > Cc: ope...@li... > Subject: Re: [Openfpc-users] Openfpc usage > > > > Hi, > > > > Documentation is really one of the places that really needs some extra > focus. > > > > The best docs I can point you to are in that folder, plus there is > some out-of date info on my blog http://www.leonward.com. > > I actually delivered a presentation at Defcon last weekend all about > OpenFPC. I have forwarded the slides separately. Hopefully that will > help as well. > > > > As for your specific question about OpenFPC GUI. That's actually now > been deprecated as it's no longer relevant for how it functions in a > distributed manner. The OpenFPC-Chrome Extension will be the next best > thing for interacting with the QueueDaemon remotely in a GUI-like way. > > > > Cheers, > > > > -L > > > > > > > > On Mon, Aug 17, 2015 at 4:25 PM, Igor Kaplan <igo...@gm...> wrote: > > Hi All, > > > > My name is Igor. I just found the OpenFPC and evaluating it. Looks > like it is very good tool. > > I successfully installed on Ubuntu 14.4 with Perl 5.18 > > I have installed the OpenFPC-master, so it is the latest code. > > > > Now I would like to find out if there is more documentation beside > files which I could find under docs directory. > > For example the INSTALL.md refers to the USAGE document, however I > was not able to find it anywhere > > > > I am looking for the usage other then basic, just to find out, what > are my advanced options. > > > > For example the openfpc-dbmaint.sh script is also able to create the > gui database, I wonder, what it is for? > > > > The OpenFPC looks to be very powerful, just would like to understand > it as best as I can. > > > > Would so much appreciate any reply?s. > > > > Many thanks. > > > > Igor. > > > > > ---------------------------------------------------------------------- > ------ > -- > > _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > > > ---------------------------------------------------------------------- > ------ > -- > > _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > > > > -------------- next part -------------- An HTML attachment was > scrubbed... > > ------------------------------ > > ---------------------------------------------------------------------- > ------ > -- > > > ------------------------------ > > _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > End of Openfpc-users Digest, Vol 2, Issue 4 > ******************************************* > > > ---------------------------------------------------------------------- > -------- _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users ------------------------------------------------------------------------------ _______________________________________________ Openfpc-users mailing list Ope...@li... https://lists.sourceforge.net/lists/listinfo/openfpc-users |
|
From: Igor K. <igo...@gm...> - 2015-08-25 21:28:14
|
Hi Leon, Thanks so much for your reply. Actually what happened. The total size of my vm is 16 GB. The maximum allowed size in my config is 50 percent PCAP_SPACE=50 However the size of /var/tmp/openfpc/pcap was 14 GB, much more then 50 percent. Unfortunately I already cleaned everything since vm was not operational, so not able to send the listing of pcap directory. I am also for now don't use any bpf filters, however still, as I understand, the size should not have grown more then 50 percent. And I am using just the single openfpc node on that host. I'll watch the size growth in nearest future and see,. All the best. -Igor -----Original Message----- From: Leon Ward [mailto:leo...@gm...] On Behalf Of Leon Ward Sent: Tuesday, August 25, 2015 4:59 PM To: Igor Kaplan Cc: <ope...@li...> Subject: Re: [Openfpc-users] Openfpc-users Digest, Vol 2, Issue 4 That's where the pcaps should live, and they will grow to the max percentage that you allow in the node config. What does an openfpc-client -a status show? Also what's a df -h look like? The pcaps will auto-prune unless something has gone wrong along the way.... Thinking out loud, what's an ls of your pcaps directory look like? Have you got multiple nodes running on one box? -L Sent from a mobile device. Apologies for any typos but they happen. > On 25 Aug 2015, at 16:25, Igor Kaplan <igo...@gm...> wrote: > > Hi Leon, > > Could you please help me with following. > I am running openfpc for several days already and now I am out of > space on my Ubuntu box which runs openfpc Under /var/tmp/openfpc I see > directories, some of which contain number of large files: > api-pcaps extracted pcap session > > I wonder, can I safely delete data under any of those directories above? > Could you please let me know, which directory I can empty without > breaking openfpc functionality? > > Is there any way to clean all captured data and start fresh? > > Many thanks. > > -Igor. > > > -----Original Message----- > From: ope...@li... > [mailto:ope...@li...] > Sent: Thursday, August 20, 2015 8:01 PM > To: ope...@li... > Subject: Openfpc-users Digest, Vol 2, Issue 4 > > Send Openfpc-users mailing list submissions to > ope...@li... > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.sourceforge.net/lists/listinfo/openfpc-users > or, via email, send a message with subject or body 'help' to > ope...@li... > > You can reach the person managing the list at > ope...@li... > > When replying, please edit your Subject line so it is more specific > than > "Re: Contents of Openfpc-users digest..." > > > Today's Topics: > > 1. Re: Openfpc usage (Igor Kaplan) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 20 Aug 2015 20:01:08 -0400 > From: "Igor Kaplan" <igo...@gm...> > Subject: Re: [Openfpc-users] Openfpc usage > To: <ope...@li...> > Message-ID: <000c01d0dba4$7c3c9910$74b5cb30$@gmail.com> > Content-Type: text/plain; charset="utf-8" > > Also sending my reply to the list, sorry, forgot to include it. > > > > > > From: Igor Kaplan [mailto:igo...@gm...] > Sent: Thursday, August 20, 2015 6:44 PM > To: 'Leon Ward' > Subject: RE: [Openfpc-users] Openfpc usage > > > > Version of mergecap: > > Mergecap 1.10.6 (v1.10.6 from master-1.10) > > > > Linux is Ubuntu 14.04.2 LTS > > > > File list.pcap > > list.pcap: pcap-ng capture file - version 1.0 > > > > Thanks. > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf > Of Leon Ward > Sent: Thursday, August 20, 2015 6:23 PM > To: Igor Kaplan > Cc: ope...@li... > Subject: Re: [Openfpc-users] Openfpc usage > > > > What's your platform, version of mergecap etc. > > Also, if you '$ file list.pcap' what does it say? > > > > -L > > > > > > On Thu, Aug 20, 2015 at 10:15 PM, Igor Kaplan <igo...@gm...> wrote: > > Hi Leon, > > > > Need your help please once again. > > Got the following problem and spent several hours trying to solve it. > > > > When making the API call to fetch the pcap data I am getting the data > in pcapng format. > > The OpenFPC is using the mergecap to merge pcap files and by default > mergecap creates the output in pcapng format instead of pcap. > > > > I have changed the following line in openfpc-default.conf file > > MERGECAP=/usr/bin/mergecap -F pcap > > > > This helped when I use the openfpc-client command to create pcap > files, however when I use curl to fetch the data I still receive the > output in pcapng format. > > > > curl -k > 192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F-C061B4C > E8A48\ > <http://192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F > -C061B > 4CE8A48%5C&stime=20150818%2010:00%5C&etime=20150818%2010:30%5C&dpt=22> > &stime=20150818%2010:00\&etime=20150818%2010:30\&dpt=22 > list.pcap > > > > cat list.pcap|tshark -i- > > Capturing on 'Standard input' > > tshark: Unrecognized libpcap format > > > > Looks like in case of API call the mergecap utility is not used at all. > And I was not able to find in the code how merging is done in this case. > > > > Could you please help me. Is it possible to make the fetch API call > to return the data in pcap format? > > > > Thanks so much! > > > > Igor > > > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf > Of Leon Ward > Sent: Tuesday, August 18, 2015 1:29 PM > > > To: Igor Kaplan > Cc: ope...@li... > Subject: Re: [Openfpc-users] Openfpc usage > > > > Actually it wont. It will only remove the oldest PCAP file. It's best > to keep those PCAP files on their own partition. > > The old flow records in mysql actually get removed automatically based > on the oldest packet in the store. So you won't have records that are > older than the pcaps. > > > > -L > > > > > > On Tue, Aug 18, 2015 at 6:21 PM, Igor Kaplan <igo...@gm...> wrote: > > Hi Leon, all, > > > > I have one more question please. > > > > Based on the documentation the following line in the openfpc config > file restricts the space usage of captured data to 50 percent: > > PCAP_SPACE=50 > > > > So, if the data size exceeds 50 percent old files will be deleted > automatically? > > Will openfpc also delete the old MySQL session tables? > > > > Many thanks and all the best! > > > > -Igor. > > > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf > Of Leon Ward > Sent: Monday, August 17, 2015 11:51 AM > To: Igor Kaplan > Cc: ope...@li... > Subject: Re: [Openfpc-users] Openfpc usage > > > > Hi, > > > > Documentation is really one of the places that really needs some extra > focus. > > > > The best docs I can point you to are in that folder, plus there is > some out-of date info on my blog http://www.leonward.com. > > I actually delivered a presentation at Defcon last weekend all about > OpenFPC. I have forwarded the slides separately. Hopefully that will > help as well. > > > > As for your specific question about OpenFPC GUI. That's actually now > been deprecated as it's no longer relevant for how it functions in a > distributed manner. The OpenFPC-Chrome Extension will be the next best > thing for interacting with the QueueDaemon remotely in a GUI-like way. > > > > Cheers, > > > > -L > > > > > > > > On Mon, Aug 17, 2015 at 4:25 PM, Igor Kaplan <igo...@gm...> wrote: > > Hi All, > > > > My name is Igor. I just found the OpenFPC and evaluating it. Looks > like it is very good tool. > > I successfully installed on Ubuntu 14.4 with Perl 5.18 > > I have installed the OpenFPC-master, so it is the latest code. > > > > Now I would like to find out if there is more documentation beside > files which I could find under docs directory. > > For example the INSTALL.md refers to the USAGE document, however I > was not able to find it anywhere > > > > I am looking for the usage other then basic, just to find out, what > are my advanced options. > > > > For example the openfpc-dbmaint.sh script is also able to create the > gui database, I wonder, what it is for? > > > > The OpenFPC looks to be very powerful, just would like to understand > it as best as I can. > > > > Would so much appreciate any reply?s. > > > > Many thanks. > > > > Igor. > > > > > ---------------------------------------------------------------------- > ------ > -- > > _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > > > ---------------------------------------------------------------------- > ------ > -- > > _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > > > > -------------- next part -------------- An HTML attachment was > scrubbed... > > ------------------------------ > > ---------------------------------------------------------------------- > ------ > -- > > > ------------------------------ > > _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > End of Openfpc-users Digest, Vol 2, Issue 4 > ******************************************* > > > ---------------------------------------------------------------------- > -------- _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users |
|
From: Leon W. <le...@rm...> - 2015-08-25 20:58:55
|
That's where the pcaps should live, and they will grow to the max percentage that you allow in the node config. What does an openfpc-client -a status show? Also what's a df -h look like? The pcaps will auto-prune unless something has gone wrong along the way.... Thinking out loud, what's an ls of your pcaps directory look like? Have you got multiple nodes running on one box? -L Sent from a mobile device. Apologies for any typos but they happen. > On 25 Aug 2015, at 16:25, Igor Kaplan <igo...@gm...> wrote: > > Hi Leon, > > Could you please help me with following. > I am running openfpc for several days already and now I am out of space on > my Ubuntu box which runs openfpc > Under /var/tmp/openfpc I see directories, some of which contain number of > large files: > api-pcaps extracted pcap session > > I wonder, can I safely delete data under any of those directories above? > Could you please let me know, which directory I can empty without breaking > openfpc functionality? > > Is there any way to clean all captured data and start fresh? > > Many thanks. > > -Igor. > > > -----Original Message----- > From: ope...@li... > [mailto:ope...@li...] > Sent: Thursday, August 20, 2015 8:01 PM > To: ope...@li... > Subject: Openfpc-users Digest, Vol 2, Issue 4 > > Send Openfpc-users mailing list submissions to > ope...@li... > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.sourceforge.net/lists/listinfo/openfpc-users > or, via email, send a message with subject or body 'help' to > ope...@li... > > You can reach the person managing the list at > ope...@li... > > When replying, please edit your Subject line so it is more specific than > "Re: Contents of Openfpc-users digest..." > > > Today's Topics: > > 1. Re: Openfpc usage (Igor Kaplan) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 20 Aug 2015 20:01:08 -0400 > From: "Igor Kaplan" <igo...@gm...> > Subject: Re: [Openfpc-users] Openfpc usage > To: <ope...@li...> > Message-ID: <000c01d0dba4$7c3c9910$74b5cb30$@gmail.com> > Content-Type: text/plain; charset="utf-8" > > Also sending my reply to the list, sorry, forgot to include it. > > > > > > From: Igor Kaplan [mailto:igo...@gm...] > Sent: Thursday, August 20, 2015 6:44 PM > To: 'Leon Ward' > Subject: RE: [Openfpc-users] Openfpc usage > > > > Version of mergecap: > > Mergecap 1.10.6 (v1.10.6 from master-1.10) > > > > Linux is Ubuntu 14.04.2 LTS > > > > File list.pcap > > list.pcap: pcap-ng capture file - version 1.0 > > > > Thanks. > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf Of Leon > Ward > Sent: Thursday, August 20, 2015 6:23 PM > To: Igor Kaplan > Cc: ope...@li... > Subject: Re: [Openfpc-users] Openfpc usage > > > > What's your platform, version of mergecap etc. > > Also, if you '$ file list.pcap' what does it say? > > > > -L > > > > > > On Thu, Aug 20, 2015 at 10:15 PM, Igor Kaplan <igo...@gm...> wrote: > > Hi Leon, > > > > Need your help please once again. > > Got the following problem and spent several hours trying to solve it. > > > > When making the API call to fetch the pcap data I am getting the data in > pcapng format. > > The OpenFPC is using the mergecap to merge pcap files and by default > mergecap creates the output in pcapng format instead of pcap. > > > > I have changed the following line in openfpc-default.conf file > > MERGECAP=/usr/bin/mergecap -F pcap > > > > This helped when I use the openfpc-client command to create pcap files, > however when I use curl to fetch the data I still receive the output in > pcapng format. > > > > curl -k > 192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F-C061B4CE8A48\ > <http://192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F-C061B > 4CE8A48%5C&stime=20150818%2010:00%5C&etime=20150818%2010:30%5C&dpt=22> > &stime=20150818%2010:00\&etime=20150818%2010:30\&dpt=22 > list.pcap > > > > cat list.pcap|tshark -i- > > Capturing on 'Standard input' > > tshark: Unrecognized libpcap format > > > > Looks like in case of API call the mergecap utility is not used at all. > And I was not able to find in the code how merging is done in this case. > > > > Could you please help me. Is it possible to make the fetch API call to > return the data in pcap format? > > > > Thanks so much! > > > > Igor > > > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf Of Leon > Ward > Sent: Tuesday, August 18, 2015 1:29 PM > > > To: Igor Kaplan > Cc: ope...@li... > Subject: Re: [Openfpc-users] Openfpc usage > > > > Actually it wont. It will only remove the oldest PCAP file. It's best to > keep those PCAP files on their own partition. > > The old flow records in mysql actually get removed automatically based on > the oldest packet in the store. So you won't have records that are older > than the pcaps. > > > > -L > > > > > > On Tue, Aug 18, 2015 at 6:21 PM, Igor Kaplan <igo...@gm...> wrote: > > Hi Leon, all, > > > > I have one more question please. > > > > Based on the documentation the following line in the openfpc config file > restricts the space usage of captured data to 50 percent: > > PCAP_SPACE=50 > > > > So, if the data size exceeds 50 percent old files will be deleted > automatically? > > Will openfpc also delete the old MySQL session tables? > > > > Many thanks and all the best! > > > > -Igor. > > > > > > From: leo...@gm... [mailto:leo...@gm...] On Behalf Of Leon > Ward > Sent: Monday, August 17, 2015 11:51 AM > To: Igor Kaplan > Cc: ope...@li... > Subject: Re: [Openfpc-users] Openfpc usage > > > > Hi, > > > > Documentation is really one of the places that really needs some extra > focus. > > > > The best docs I can point you to are in that folder, plus there is some > out-of date info on my blog http://www.leonward.com. > > I actually delivered a presentation at Defcon last weekend all about > OpenFPC. I have forwarded the slides separately. Hopefully that will help as > well. > > > > As for your specific question about OpenFPC GUI. That's actually now been > deprecated as it's no longer relevant for how it functions in a distributed > manner. The OpenFPC-Chrome Extension will be the next best thing for > interacting with the QueueDaemon remotely in a GUI-like way. > > > > Cheers, > > > > -L > > > > > > > > On Mon, Aug 17, 2015 at 4:25 PM, Igor Kaplan <igo...@gm...> wrote: > > Hi All, > > > > My name is Igor. I just found the OpenFPC and evaluating it. Looks like it > is very good tool. > > I successfully installed on Ubuntu 14.4 with Perl 5.18 > > I have installed the OpenFPC-master, so it is the latest code. > > > > Now I would like to find out if there is more documentation beside files > which I could find under docs directory. > > For example the INSTALL.md refers to the USAGE document, however I was not > able to find it anywhere > > > > I am looking for the usage other then basic, just to find out, what are my > advanced options. > > > > For example the openfpc-dbmaint.sh script is also able to create the gui > database, I wonder, what it is for? > > > > The OpenFPC looks to be very powerful, just would like to understand it as > best as I can. > > > > Would so much appreciate any reply?s. > > > > Many thanks. > > > > Igor. > > > > > ---------------------------------------------------------------------------- > -- > > _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > > > ---------------------------------------------------------------------------- > -- > > _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > > > > -------------- next part -------------- > An HTML attachment was scrubbed... > > ------------------------------ > > ---------------------------------------------------------------------------- > -- > > > ------------------------------ > > _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > End of Openfpc-users Digest, Vol 2, Issue 4 > ******************************************* > > > ------------------------------------------------------------------------------ > _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users |
|
From: Igor K. <igo...@gm...> - 2015-08-25 15:26:06
|
Hi Leon, Could you please help me with following. I am running openfpc for several days already and now I am out of space on my Ubuntu box which runs openfpc Under /var/tmp/openfpc I see directories, some of which contain number of large files: api-pcaps extracted pcap session I wonder, can I safely delete data under any of those directories above? Could you please let me know, which directory I can empty without breaking openfpc functionality? Is there any way to clean all captured data and start fresh? Many thanks. -Igor. -----Original Message----- From: ope...@li... [mailto:ope...@li...] Sent: Thursday, August 20, 2015 8:01 PM To: ope...@li... Subject: Openfpc-users Digest, Vol 2, Issue 4 Send Openfpc-users mailing list submissions to ope...@li... To subscribe or unsubscribe via the World Wide Web, visit https://lists.sourceforge.net/lists/listinfo/openfpc-users or, via email, send a message with subject or body 'help' to ope...@li... You can reach the person managing the list at ope...@li... When replying, please edit your Subject line so it is more specific than "Re: Contents of Openfpc-users digest..." Today's Topics: 1. Re: Openfpc usage (Igor Kaplan) ---------------------------------------------------------------------- Message: 1 Date: Thu, 20 Aug 2015 20:01:08 -0400 From: "Igor Kaplan" <igo...@gm...> Subject: Re: [Openfpc-users] Openfpc usage To: <ope...@li...> Message-ID: <000c01d0dba4$7c3c9910$74b5cb30$@gmail.com> Content-Type: text/plain; charset="utf-8" Also sending my reply to the list, sorry, forgot to include it. From: Igor Kaplan [mailto:igo...@gm...] Sent: Thursday, August 20, 2015 6:44 PM To: 'Leon Ward' Subject: RE: [Openfpc-users] Openfpc usage Version of mergecap: Mergecap 1.10.6 (v1.10.6 from master-1.10) Linux is Ubuntu 14.04.2 LTS File list.pcap list.pcap: pcap-ng capture file - version 1.0 Thanks. From: leo...@gm... [mailto:leo...@gm...] On Behalf Of Leon Ward Sent: Thursday, August 20, 2015 6:23 PM To: Igor Kaplan Cc: ope...@li... Subject: Re: [Openfpc-users] Openfpc usage What's your platform, version of mergecap etc. Also, if you '$ file list.pcap' what does it say? -L On Thu, Aug 20, 2015 at 10:15 PM, Igor Kaplan <igo...@gm...> wrote: Hi Leon, Need your help please once again. Got the following problem and spent several hours trying to solve it. When making the API call to fetch the pcap data I am getting the data in pcapng format. The OpenFPC is using the mergecap to merge pcap files and by default mergecap creates the output in pcapng format instead of pcap. I have changed the following line in openfpc-default.conf file MERGECAP=/usr/bin/mergecap -F pcap This helped when I use the openfpc-client command to create pcap files, however when I use curl to fetch the data I still receive the output in pcapng format. curl -k 192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F-C061B4CE8A48\ <http://192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F-C061B 4CE8A48%5C&stime=20150818%2010:00%5C&etime=20150818%2010:30%5C&dpt=22> &stime=20150818%2010:00\&etime=20150818%2010:30\&dpt=22 > list.pcap cat list.pcap|tshark -i- Capturing on 'Standard input' tshark: Unrecognized libpcap format Looks like in case of API call the mergecap utility is not used at all. And I was not able to find in the code how merging is done in this case. Could you please help me. Is it possible to make the fetch API call to return the data in pcap format? Thanks so much! Igor From: leo...@gm... [mailto:leo...@gm...] On Behalf Of Leon Ward Sent: Tuesday, August 18, 2015 1:29 PM To: Igor Kaplan Cc: ope...@li... Subject: Re: [Openfpc-users] Openfpc usage Actually it wont. It will only remove the oldest PCAP file. It's best to keep those PCAP files on their own partition. The old flow records in mysql actually get removed automatically based on the oldest packet in the store. So you won't have records that are older than the pcaps. -L On Tue, Aug 18, 2015 at 6:21 PM, Igor Kaplan <igo...@gm...> wrote: Hi Leon, all, I have one more question please. Based on the documentation the following line in the openfpc config file restricts the space usage of captured data to 50 percent: PCAP_SPACE=50 So, if the data size exceeds 50 percent old files will be deleted automatically? Will openfpc also delete the old MySQL session tables? Many thanks and all the best! -Igor. From: leo...@gm... [mailto:leo...@gm...] On Behalf Of Leon Ward Sent: Monday, August 17, 2015 11:51 AM To: Igor Kaplan Cc: ope...@li... Subject: Re: [Openfpc-users] Openfpc usage Hi, Documentation is really one of the places that really needs some extra focus. The best docs I can point you to are in that folder, plus there is some out-of date info on my blog http://www.leonward.com. I actually delivered a presentation at Defcon last weekend all about OpenFPC. I have forwarded the slides separately. Hopefully that will help as well. As for your specific question about OpenFPC GUI. That's actually now been deprecated as it's no longer relevant for how it functions in a distributed manner. The OpenFPC-Chrome Extension will be the next best thing for interacting with the QueueDaemon remotely in a GUI-like way. Cheers, -L On Mon, Aug 17, 2015 at 4:25 PM, Igor Kaplan <igo...@gm...> wrote: Hi All, My name is Igor. I just found the OpenFPC and evaluating it. Looks like it is very good tool. I successfully installed on Ubuntu 14.4 with Perl 5.18 I have installed the OpenFPC-master, so it is the latest code. Now I would like to find out if there is more documentation beside files which I could find under docs directory. For example the INSTALL.md refers to the USAGE document, however I was not able to find it anywhere I am looking for the usage other then basic, just to find out, what are my advanced options. For example the openfpc-dbmaint.sh script is also able to create the gui database, I wonder, what it is for? The OpenFPC looks to be very powerful, just would like to understand it as best as I can. Would so much appreciate any reply?s. Many thanks. Igor. ---------------------------------------------------------------------------- -- _______________________________________________ Openfpc-users mailing list Ope...@li... https://lists.sourceforge.net/lists/listinfo/openfpc-users ---------------------------------------------------------------------------- -- _______________________________________________ Openfpc-users mailing list Ope...@li... https://lists.sourceforge.net/lists/listinfo/openfpc-users -------------- next part -------------- An HTML attachment was scrubbed... ------------------------------ ---------------------------------------------------------------------------- -- ------------------------------ _______________________________________________ Openfpc-users mailing list Ope...@li... https://lists.sourceforge.net/lists/listinfo/openfpc-users End of Openfpc-users Digest, Vol 2, Issue 4 ******************************************* |
|
From: Igor K. <igo...@gm...> - 2015-08-21 13:05:39
|
Hi Leon, Yes, I also noticed that tshark will accept the pcap_ng file with –r However it is more convenient for me to pipe the output of curl to the tshark directly, something like Curl …|tshark –i- I have tried it with tshark 1.10, the default one which is installed on Ubuntu 14.04. While being absolutely fine as tshark –r file_name it did not want to process pcap_ng files as a stdin interface. I am glad, that the simple config file change was able to fix it for me. Many thanks once again. -Igor From: leo...@gm... [mailto:leo...@gm...] On Behalf Of Leon Ward Sent: Friday, August 21, 2015 6:23 AM To: Igor Kaplan Cc: ope...@li... Subject: Re: [Openfpc-users] Openfpc usage Hi, I found this interesting and I hadn't noticed that mergecap had moved to default to pcap-ng. It's clear that Snort, and tcpdump both work with it fine (at least on the systems I use here). Tshark is able to read the pcapng file, just not from STDIN. I've never tried to pipe something to tshark like this before so I don't know the limitations. pcapng is also the new standard format for all wireshark (as of 1.8, according to the Internets) including tshark. Is it tshark you're specifically trying to use, and is there a reason why you can't open the file via -r? lward@openfpc:~/openfpc$ tshark -h |grep stdin -r <infile> set the filename to read from (no pipes or stdin!) Glad that there is a simple fix for you, but I'm asking all of these questions to work out of I should make this change by default. On Fri, Aug 21, 2015 at 1:17 AM, Igor Kaplan <igo...@gm...> wrote: Hi all, Sorry for my last message. Everything is fine now. It was completely my folt. Was connecting to wrong ip address! The –F switch for the mergecap utility fixed the problem. Thanks. From: Igor Kaplan [mailto:igo...@gm...] Sent: Thursday, August 20, 2015 5:16 PM To: 'Leon Ward' Cc: ope...@li... Subject: RE: [Openfpc-users] Openfpc usage Hi Leon, Need your help please once again. Got the following problem and spent several hours trying to solve it. When making the API call to fetch the pcap data I am getting the data in pcapng format. The OpenFPC is using the mergecap to merge pcap files and by default mergecap creates the output in pcapng format instead of pcap. I have changed the following line in openfpc-default.conf file MERGECAP=/usr/bin/mergecap -F pcap This helped when I use the openfpc-client command to create pcap files, however when I use curl to fetch the data I still receive the output in pcapng format. curl -k 192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F-C061B4CE8A48\ <http://192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F-C061B4CE8A48%5C&stime=20150818%2010:00%5C&etime=20150818%2010:30%5C&dpt=22> &stime=20150818%2010:00\&etime=20150818%2010:30\&dpt=22 > list.pcap cat list.pcap|tshark -i- Capturing on 'Standard input' tshark: Unrecognized libpcap format Looks like in case of API call the mergecap utility is not used at all. And I was not able to find in the code how merging is done in this case. Could you please help me. Is it possible to make the fetch API call to return the data in pcap format? Thanks so much! Igor From: leo...@gm... [mailto:leo...@gm...] On Behalf Of Leon Ward Sent: Tuesday, August 18, 2015 1:29 PM To: Igor Kaplan Cc: ope...@li... Subject: Re: [Openfpc-users] Openfpc usage Actually it wont. It will only remove the oldest PCAP file. It's best to keep those PCAP files on their own partition. The old flow records in mysql actually get removed automatically based on the oldest packet in the store. So you won't have records that are older than the pcaps. -L On Tue, Aug 18, 2015 at 6:21 PM, Igor Kaplan <igo...@gm...> wrote: Hi Leon, all, I have one more question please. Based on the documentation the following line in the openfpc config file restricts the space usage of captured data to 50 percent: PCAP_SPACE=50 So, if the data size exceeds 50 percent old files will be deleted automatically? Will openfpc also delete the old MySQL session tables? Many thanks and all the best! -Igor. From: leo...@gm... [mailto:leo...@gm...] On Behalf Of Leon Ward Sent: Monday, August 17, 2015 11:51 AM To: Igor Kaplan Cc: ope...@li... Subject: Re: [Openfpc-users] Openfpc usage Hi, Documentation is really one of the places that really needs some extra focus. The best docs I can point you to are in that folder, plus there is some out-of date info on my blog http://www.leonward.com. I actually delivered a presentation at Defcon last weekend all about OpenFPC. I have forwarded the slides separately. Hopefully that will help as well. As for your specific question about OpenFPC GUI. That's actually now been deprecated as it's no longer relevant for how it functions in a distributed manner. The OpenFPC-Chrome Extension will be the next best thing for interacting with the QueueDaemon remotely in a GUI-like way. Cheers, -L On Mon, Aug 17, 2015 at 4:25 PM, Igor Kaplan <igo...@gm...> wrote: Hi All, My name is Igor. I just found the OpenFPC and evaluating it. Looks like it is very good tool. I successfully installed on Ubuntu 14.4 with Perl 5.18 I have installed the OpenFPC-master, so it is the latest code. Now I would like to find out if there is more documentation beside files which I could find under docs directory. For example the INSTALL.md refers to the USAGE document, however I was not able to find it anywhere I am looking for the usage other then basic, just to find out, what are my advanced options. For example the openfpc-dbmaint.sh script is also able to create the gui database, I wonder, what it is for? The OpenFPC looks to be very powerful, just would like to understand it as best as I can. Would so much appreciate any reply’s. Many thanks. Igor. ------------------------------------------------------------------------------ _______________________________________________ Openfpc-users mailing list Ope...@li... https://lists.sourceforge.net/lists/listinfo/openfpc-users ------------------------------------------------------------------------------ _______________________________________________ Openfpc-users mailing list Ope...@li... https://lists.sourceforge.net/lists/listinfo/openfpc-users |
|
From: Leon W. <le...@le...> - 2015-08-21 10:22:40
|
Hi, I found this interesting and I hadn't noticed that mergecap had moved to default to pcap-ng. It's clear that Snort, and tcpdump both work with it fine (at least on the systems I use here). Tshark is able to read the pcapng file, just not from STDIN. I've never tried to pipe something to tshark like this before so I don't know the limitations. pcapng is also the new standard format for all wireshark (as of 1.8, according to the Internets) including tshark. Is it tshark you're specifically trying to use, and is there a reason why you can't open the file via -r? lward@openfpc:~/openfpc$ tshark -h |grep stdin -r <infile> set the filename to read from (no pipes or stdin!) Glad that there is a simple fix for you, but I'm asking all of these questions to work out of I should make this change by default. On Fri, Aug 21, 2015 at 1:17 AM, Igor Kaplan <igo...@gm...> wrote: > Hi all, > > > > Sorry for my last message. Everything is fine now. It was completely my > folt. Was connecting to wrong ip address! > > The –F switch for the mergecap utility fixed the problem. > > > > Thanks. > > > > *From:* Igor Kaplan [mailto:igo...@gm...] > *Sent:* Thursday, August 20, 2015 5:16 PM > *To:* 'Leon Ward' > *Cc:* ope...@li... > *Subject:* RE: [Openfpc-users] Openfpc usage > > > > Hi Leon, > > > > Need your help please once again. > > Got the following problem and spent several hours trying to solve it. > > > > When making the API call to fetch the pcap data I am getting the data in > pcapng format. > > The OpenFPC is using the mergecap to merge pcap files and by default > mergecap creates the output in pcapng format instead of pcap. > > > > I have changed the following line in openfpc-default.conf file > > MERGECAP=/usr/bin/mergecap -F pcap > > > > This helped when I use the openfpc-client command to create pcap files, > however when I use curl to fetch the data I still receive the output in > pcapng format. > > > > curl -k > 192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F-C061B4CE8A48\&stime=20150818%2010:00\&etime=20150818%2010:30\&dpt=22 > <http://192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F-C061B4CE8A48%5C&stime=20150818%2010:00%5C&etime=20150818%2010:30%5C&dpt=22> > > list.pcap > > > > cat list.pcap|tshark -i- > > Capturing on 'Standard input' > > tshark: Unrecognized libpcap format > > > > Looks like in case of API call the mergecap utility is not used at all. > And I was not able to find in the code how merging is done in this case. > > > > Could you please help me. Is it possible to make the fetch API call to > return the data in pcap format? > > > > Thanks so much! > > > > Igor > > > > > > *From:* leo...@gm... [mailto:leo...@gm...] *On Behalf > Of *Leon Ward > *Sent:* Tuesday, August 18, 2015 1:29 PM > *To:* Igor Kaplan > *Cc:* ope...@li... > *Subject:* Re: [Openfpc-users] Openfpc usage > > > > Actually it wont. It will only remove the oldest PCAP file. It's best to > keep those PCAP files on their own partition. > > The old flow records in mysql actually get removed automatically based on > the oldest packet in the store. So you won't have records that are older > than the pcaps. > > > > -L > > > > > > On Tue, Aug 18, 2015 at 6:21 PM, Igor Kaplan <igo...@gm...> wrote: > > Hi Leon, all, > > > > I have one more question please. > > > > Based on the documentation the following line in the openfpc config file > restricts the space usage of captured data to 50 percent: > > PCAP_SPACE=50 > > > > So, if the data size exceeds 50 percent old files will be deleted > automatically? > > Will openfpc also delete the old MySQL session tables? > > > > Many thanks and all the best! > > > > -Igor. > > > > > > *From:* leo...@gm... [mailto:leo...@gm...] *On Behalf > Of *Leon Ward > *Sent:* Monday, August 17, 2015 11:51 AM > *To:* Igor Kaplan > *Cc:* ope...@li... > *Subject:* Re: [Openfpc-users] Openfpc usage > > > > Hi, > > > > Documentation is really one of the places that really needs some extra > focus. > > > > The best docs I can point you to are in that folder, plus there is some > out-of date info on my blog http://www.leonward.com. > > I actually delivered a presentation at Defcon last weekend all about > OpenFPC. I have forwarded the slides separately. Hopefully that will help > as well. > > > > As for your specific question about OpenFPC GUI. That's actually now been > deprecated as it's no longer relevant for how it functions in a distributed > manner. The OpenFPC-Chrome Extension will be the next best thing for > interacting with the QueueDaemon remotely in a GUI-like way. > > > > Cheers, > > > > -L > > > > > > > > On Mon, Aug 17, 2015 at 4:25 PM, Igor Kaplan <igo...@gm...> wrote: > > Hi All, > > > > My name is Igor. I just found the OpenFPC and evaluating it. Looks like > it is very good tool. > > I successfully installed on Ubuntu 14.4 with Perl 5.18 > > I have installed the OpenFPC-master, so it is the latest code. > > > > Now I would like to find out if there is more documentation beside files > which I could find under docs directory. > > For example the INSTALL.md refers to the USAGE document, however I was > not able to find it anywhere > > > > I am looking for the usage other then basic, just to find out, what are > my advanced options. > > > > For example the openfpc-dbmaint.sh script is also able to create the gui > database, I wonder, what it is for? > > > > The OpenFPC looks to be very powerful, just would like to understand it as > best as I can. > > > > Would so much appreciate any reply’s. > > > > Many thanks. > > > > Igor. > > > > > > ------------------------------------------------------------------------------ > > _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > > > > ------------------------------------------------------------------------------ > > _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > |
|
From: Igor K. <igo...@gm...> - 2015-08-21 00:18:09
|
Hi all, Sorry for my last message. Everything is fine now. It was completely my folt. Was connecting to wrong ip address! The –F switch for the mergecap utility fixed the problem. Thanks. From: Igor Kaplan [mailto:igo...@gm...] Sent: Thursday, August 20, 2015 5:16 PM To: 'Leon Ward' Cc: ope...@li... Subject: RE: [Openfpc-users] Openfpc usage Hi Leon, Need your help please once again. Got the following problem and spent several hours trying to solve it. When making the API call to fetch the pcap data I am getting the data in pcapng format. The OpenFPC is using the mergecap to merge pcap files and by default mergecap creates the output in pcapng format instead of pcap. I have changed the following line in openfpc-default.conf file MERGECAP=/usr/bin/mergecap -F pcap This helped when I use the openfpc-client command to create pcap files, however when I use curl to fetch the data I still receive the output in pcapng format. curl -k 192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F-C061B4CE8A48\&stime=20150818%2010:00\&etime=20150818%2010:30\&dpt=22 > list.pcap cat list.pcap|tshark -i- Capturing on 'Standard input' tshark: Unrecognized libpcap format Looks like in case of API call the mergecap utility is not used at all. And I was not able to find in the code how merging is done in this case. Could you please help me. Is it possible to make the fetch API call to return the data in pcap format? Thanks so much! Igor From: leo...@gm... [mailto:leo...@gm...] On Behalf Of Leon Ward Sent: Tuesday, August 18, 2015 1:29 PM To: Igor Kaplan Cc: ope...@li... Subject: Re: [Openfpc-users] Openfpc usage Actually it wont. It will only remove the oldest PCAP file. It's best to keep those PCAP files on their own partition. The old flow records in mysql actually get removed automatically based on the oldest packet in the store. So you won't have records that are older than the pcaps. -L On Tue, Aug 18, 2015 at 6:21 PM, Igor Kaplan <igo...@gm...> wrote: Hi Leon, all, I have one more question please. Based on the documentation the following line in the openfpc config file restricts the space usage of captured data to 50 percent: PCAP_SPACE=50 So, if the data size exceeds 50 percent old files will be deleted automatically? Will openfpc also delete the old MySQL session tables? Many thanks and all the best! -Igor. From: leo...@gm... [mailto:leo...@gm...] On Behalf Of Leon Ward Sent: Monday, August 17, 2015 11:51 AM To: Igor Kaplan Cc: ope...@li... Subject: Re: [Openfpc-users] Openfpc usage Hi, Documentation is really one of the places that really needs some extra focus. The best docs I can point you to are in that folder, plus there is some out-of date info on my blog http://www.leonward.com. I actually delivered a presentation at Defcon last weekend all about OpenFPC. I have forwarded the slides separately. Hopefully that will help as well. As for your specific question about OpenFPC GUI. That's actually now been deprecated as it's no longer relevant for how it functions in a distributed manner. The OpenFPC-Chrome Extension will be the next best thing for interacting with the QueueDaemon remotely in a GUI-like way. Cheers, -L On Mon, Aug 17, 2015 at 4:25 PM, Igor Kaplan <igo...@gm...> wrote: Hi All, My name is Igor. I just found the OpenFPC and evaluating it. Looks like it is very good tool. I successfully installed on Ubuntu 14.4 with Perl 5.18 I have installed the OpenFPC-master, so it is the latest code. Now I would like to find out if there is more documentation beside files which I could find under docs directory. For example the INSTALL.md refers to the USAGE document, however I was not able to find it anywhere I am looking for the usage other then basic, just to find out, what are my advanced options. For example the openfpc-dbmaint.sh script is also able to create the gui database, I wonder, what it is for? The OpenFPC looks to be very powerful, just would like to understand it as best as I can. Would so much appreciate any reply’s. Many thanks. Igor. ------------------------------------------------------------------------------ _______________________________________________ Openfpc-users mailing list Ope...@li... https://lists.sourceforge.net/lists/listinfo/openfpc-users ------------------------------------------------------------------------------ _______________________________________________ Openfpc-users mailing list Ope...@li... https://lists.sourceforge.net/lists/listinfo/openfpc-users |
|
From: Igor K. <igo...@gm...> - 2015-08-21 00:01:25
|
Also sending my reply to the list, sorry, forgot to include it. From: Igor Kaplan [mailto:igo...@gm...] Sent: Thursday, August 20, 2015 6:44 PM To: 'Leon Ward' Subject: RE: [Openfpc-users] Openfpc usage Version of mergecap: Mergecap 1.10.6 (v1.10.6 from master-1.10) Linux is Ubuntu 14.04.2 LTS File list.pcap list.pcap: pcap-ng capture file - version 1.0 Thanks. From: leo...@gm... [mailto:leo...@gm...] On Behalf Of Leon Ward Sent: Thursday, August 20, 2015 6:23 PM To: Igor Kaplan Cc: ope...@li... Subject: Re: [Openfpc-users] Openfpc usage What's your platform, version of mergecap etc. Also, if you '$ file list.pcap' what does it say? -L On Thu, Aug 20, 2015 at 10:15 PM, Igor Kaplan <igo...@gm...> wrote: Hi Leon, Need your help please once again. Got the following problem and spent several hours trying to solve it. When making the API call to fetch the pcap data I am getting the data in pcapng format. The OpenFPC is using the mergecap to merge pcap files and by default mergecap creates the output in pcapng format instead of pcap. I have changed the following line in openfpc-default.conf file MERGECAP=/usr/bin/mergecap -F pcap This helped when I use the openfpc-client command to create pcap files, however when I use curl to fetch the data I still receive the output in pcapng format. curl -k 192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F-C061B4CE8A48\ <http://192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F-C061B4CE8A48%5C&stime=20150818%2010:00%5C&etime=20150818%2010:30%5C&dpt=22> &stime=20150818%2010:00\&etime=20150818%2010:30\&dpt=22 > list.pcap cat list.pcap|tshark -i- Capturing on 'Standard input' tshark: Unrecognized libpcap format Looks like in case of API call the mergecap utility is not used at all. And I was not able to find in the code how merging is done in this case. Could you please help me. Is it possible to make the fetch API call to return the data in pcap format? Thanks so much! Igor From: leo...@gm... [mailto:leo...@gm...] On Behalf Of Leon Ward Sent: Tuesday, August 18, 2015 1:29 PM To: Igor Kaplan Cc: ope...@li... Subject: Re: [Openfpc-users] Openfpc usage Actually it wont. It will only remove the oldest PCAP file. It's best to keep those PCAP files on their own partition. The old flow records in mysql actually get removed automatically based on the oldest packet in the store. So you won't have records that are older than the pcaps. -L On Tue, Aug 18, 2015 at 6:21 PM, Igor Kaplan <igo...@gm...> wrote: Hi Leon, all, I have one more question please. Based on the documentation the following line in the openfpc config file restricts the space usage of captured data to 50 percent: PCAP_SPACE=50 So, if the data size exceeds 50 percent old files will be deleted automatically? Will openfpc also delete the old MySQL session tables? Many thanks and all the best! -Igor. From: leo...@gm... [mailto:leo...@gm...] On Behalf Of Leon Ward Sent: Monday, August 17, 2015 11:51 AM To: Igor Kaplan Cc: ope...@li... Subject: Re: [Openfpc-users] Openfpc usage Hi, Documentation is really one of the places that really needs some extra focus. The best docs I can point you to are in that folder, plus there is some out-of date info on my blog http://www.leonward.com. I actually delivered a presentation at Defcon last weekend all about OpenFPC. I have forwarded the slides separately. Hopefully that will help as well. As for your specific question about OpenFPC GUI. That's actually now been deprecated as it's no longer relevant for how it functions in a distributed manner. The OpenFPC-Chrome Extension will be the next best thing for interacting with the QueueDaemon remotely in a GUI-like way. Cheers, -L On Mon, Aug 17, 2015 at 4:25 PM, Igor Kaplan <igo...@gm...> wrote: Hi All, My name is Igor. I just found the OpenFPC and evaluating it. Looks like it is very good tool. I successfully installed on Ubuntu 14.4 with Perl 5.18 I have installed the OpenFPC-master, so it is the latest code. Now I would like to find out if there is more documentation beside files which I could find under docs directory. For example the INSTALL.md refers to the USAGE document, however I was not able to find it anywhere I am looking for the usage other then basic, just to find out, what are my advanced options. For example the openfpc-dbmaint.sh script is also able to create the gui database, I wonder, what it is for? The OpenFPC looks to be very powerful, just would like to understand it as best as I can. Would so much appreciate any reply’s. Many thanks. Igor. ------------------------------------------------------------------------------ _______________________________________________ Openfpc-users mailing list Ope...@li... https://lists.sourceforge.net/lists/listinfo/openfpc-users ------------------------------------------------------------------------------ _______________________________________________ Openfpc-users mailing list Ope...@li... https://lists.sourceforge.net/lists/listinfo/openfpc-users |
|
From: Leon W. <le...@le...> - 2015-08-20 22:23:33
|
What's your platform, version of mergecap etc. Also, if you '$ file list.pcap' what does it say? -L On Thu, Aug 20, 2015 at 10:15 PM, Igor Kaplan <igo...@gm...> wrote: > Hi Leon, > > > > Need your help please once again. > > Got the following problem and spent several hours trying to solve it. > > > > When making the API call to fetch the pcap data I am getting the data in > pcapng format. > > The OpenFPC is using the mergecap to merge pcap files and by default > mergecap creates the output in pcapng format instead of pcap. > > > > I have changed the following line in openfpc-default.conf file > > MERGECAP=/usr/bin/mergecap -F pcap > > > > This helped when I use the openfpc-client command to create pcap files, > however when I use curl to fetch the data I still receive the output in > pcapng format. > > > > curl -k > 192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F-C061B4CE8A48\&stime=20150818%2010:00\&etime=20150818%2010:30\&dpt=22 > <http://192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F-C061B4CE8A48%5C&stime=20150818%2010:00%5C&etime=20150818%2010:30%5C&dpt=22> > > list.pcap > > > > cat list.pcap|tshark -i- > > Capturing on 'Standard input' > > tshark: Unrecognized libpcap format > > > > Looks like in case of API call the mergecap utility is not used at all. > And I was not able to find in the code how merging is done in this case. > > > > Could you please help me. Is it possible to make the fetch API call to > return the data in pcap format? > > > > Thanks so much! > > > > Igor > > > > > > *From:* leo...@gm... [mailto:leo...@gm...] *On Behalf > Of *Leon Ward > *Sent:* Tuesday, August 18, 2015 1:29 PM > > *To:* Igor Kaplan > *Cc:* ope...@li... > *Subject:* Re: [Openfpc-users] Openfpc usage > > > > Actually it wont. It will only remove the oldest PCAP file. It's best to > keep those PCAP files on their own partition. > > The old flow records in mysql actually get removed automatically based on > the oldest packet in the store. So you won't have records that are older > than the pcaps. > > > > -L > > > > > > On Tue, Aug 18, 2015 at 6:21 PM, Igor Kaplan <igo...@gm...> wrote: > > Hi Leon, all, > > > > I have one more question please. > > > > Based on the documentation the following line in the openfpc config file > restricts the space usage of captured data to 50 percent: > > PCAP_SPACE=50 > > > > So, if the data size exceeds 50 percent old files will be deleted > automatically? > > Will openfpc also delete the old MySQL session tables? > > > > Many thanks and all the best! > > > > -Igor. > > > > > > *From:* leo...@gm... [mailto:leo...@gm...] *On Behalf > Of *Leon Ward > *Sent:* Monday, August 17, 2015 11:51 AM > *To:* Igor Kaplan > *Cc:* ope...@li... > *Subject:* Re: [Openfpc-users] Openfpc usage > > > > Hi, > > > > Documentation is really one of the places that really needs some extra > focus. > > > > The best docs I can point you to are in that folder, plus there is some > out-of date info on my blog http://www.leonward.com. > > I actually delivered a presentation at Defcon last weekend all about > OpenFPC. I have forwarded the slides separately. Hopefully that will help > as well. > > > > As for your specific question about OpenFPC GUI. That's actually now been > deprecated as it's no longer relevant for how it functions in a distributed > manner. The OpenFPC-Chrome Extension will be the next best thing for > interacting with the QueueDaemon remotely in a GUI-like way. > > > > Cheers, > > > > -L > > > > > > > > On Mon, Aug 17, 2015 at 4:25 PM, Igor Kaplan <igo...@gm...> wrote: > > Hi All, > > > > My name is Igor. I just found the OpenFPC and evaluating it. Looks like > it is very good tool. > > I successfully installed on Ubuntu 14.4 with Perl 5.18 > > I have installed the OpenFPC-master, so it is the latest code. > > > > Now I would like to find out if there is more documentation beside files > which I could find under docs directory. > > For example the INSTALL.md refers to the USAGE document, however I was > not able to find it anywhere > > > > I am looking for the usage other then basic, just to find out, what are > my advanced options. > > > > For example the openfpc-dbmaint.sh script is also able to create the gui > database, I wonder, what it is for? > > > > The OpenFPC looks to be very powerful, just would like to understand it as > best as I can. > > > > Would so much appreciate any reply’s. > > > > Many thanks. > > > > Igor. > > > > > > ------------------------------------------------------------------------------ > > _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > > > > ------------------------------------------------------------------------------ > > _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > |
|
From: Igor K. <igo...@gm...> - 2015-08-20 21:16:08
|
Hi Leon, Need your help please once again. Got the following problem and spent several hours trying to solve it. When making the API call to fetch the pcap data I am getting the data in pcapng format. The OpenFPC is using the mergecap to merge pcap files and by default mergecap creates the output in pcapng format instead of pcap. I have changed the following line in openfpc-default.conf file MERGECAP=/usr/bin/mergecap -F pcap This helped when I use the openfpc-client command to create pcap files, however when I use curl to fetch the data I still receive the output in pcapng format. curl -k 192.168.145.20:4222/api/1/fetch?apikey=38A69684-41F6-11E5-B47F-C061B4CE8A48\&stime=20150818%2010:00\&etime=20150818%2010:30\&dpt=22 > list.pcap cat list.pcap|tshark -i- Capturing on 'Standard input' tshark: Unrecognized libpcap format Looks like in case of API call the mergecap utility is not used at all. And I was not able to find in the code how merging is done in this case. Could you please help me. Is it possible to make the fetch API call to return the data in pcap format? Thanks so much! Igor From: leo...@gm... [mailto:leo...@gm...] On Behalf Of Leon Ward Sent: Tuesday, August 18, 2015 1:29 PM To: Igor Kaplan Cc: ope...@li... Subject: Re: [Openfpc-users] Openfpc usage Actually it wont. It will only remove the oldest PCAP file. It's best to keep those PCAP files on their own partition. The old flow records in mysql actually get removed automatically based on the oldest packet in the store. So you won't have records that are older than the pcaps. -L On Tue, Aug 18, 2015 at 6:21 PM, Igor Kaplan <igo...@gm...> wrote: Hi Leon, all, I have one more question please. Based on the documentation the following line in the openfpc config file restricts the space usage of captured data to 50 percent: PCAP_SPACE=50 So, if the data size exceeds 50 percent old files will be deleted automatically? Will openfpc also delete the old MySQL session tables? Many thanks and all the best! -Igor. From: leo...@gm... [mailto:leo...@gm...] On Behalf Of Leon Ward Sent: Monday, August 17, 2015 11:51 AM To: Igor Kaplan Cc: ope...@li... Subject: Re: [Openfpc-users] Openfpc usage Hi, Documentation is really one of the places that really needs some extra focus. The best docs I can point you to are in that folder, plus there is some out-of date info on my blog http://www.leonward.com. I actually delivered a presentation at Defcon last weekend all about OpenFPC. I have forwarded the slides separately. Hopefully that will help as well. As for your specific question about OpenFPC GUI. That's actually now been deprecated as it's no longer relevant for how it functions in a distributed manner. The OpenFPC-Chrome Extension will be the next best thing for interacting with the QueueDaemon remotely in a GUI-like way. Cheers, -L On Mon, Aug 17, 2015 at 4:25 PM, Igor Kaplan <igo...@gm...> wrote: Hi All, My name is Igor. I just found the OpenFPC and evaluating it. Looks like it is very good tool. I successfully installed on Ubuntu 14.4 with Perl 5.18 I have installed the OpenFPC-master, so it is the latest code. Now I would like to find out if there is more documentation beside files which I could find under docs directory. For example the INSTALL.md refers to the USAGE document, however I was not able to find it anywhere I am looking for the usage other then basic, just to find out, what are my advanced options. For example the openfpc-dbmaint.sh script is also able to create the gui database, I wonder, what it is for? The OpenFPC looks to be very powerful, just would like to understand it as best as I can. Would so much appreciate any reply’s. Many thanks. Igor. ------------------------------------------------------------------------------ _______________________________________________ Openfpc-users mailing list Ope...@li... https://lists.sourceforge.net/lists/listinfo/openfpc-users ------------------------------------------------------------------------------ _______________________________________________ Openfpc-users mailing list Ope...@li... https://lists.sourceforge.net/lists/listinfo/openfpc-users |
|
From: Leon W. <le...@le...> - 2015-08-18 17:29:17
|
Actually it wont. It will only remove the oldest PCAP file. It's best to keep those PCAP files on their own partition. The old flow records in mysql actually get removed automatically based on the oldest packet in the store. So you won't have records that are older than the pcaps. -L On Tue, Aug 18, 2015 at 6:21 PM, Igor Kaplan <igo...@gm...> wrote: > Hi Leon, all, > > > > I have one more question please. > > > > Based on the documentation the following line in the openfpc config file > restricts the space usage of captured data to 50 percent: > > PCAP_SPACE=50 > > > > So, if the data size exceeds 50 percent old files will be deleted > automatically? > > Will openfpc also delete the old MySQL session tables? > > > > Many thanks and all the best! > > > > -Igor. > > > > > > *From:* leo...@gm... [mailto:leo...@gm...] *On Behalf > Of *Leon Ward > *Sent:* Monday, August 17, 2015 11:51 AM > *To:* Igor Kaplan > *Cc:* ope...@li... > *Subject:* Re: [Openfpc-users] Openfpc usage > > > > Hi, > > > > Documentation is really one of the places that really needs some extra > focus. > > > > The best docs I can point you to are in that folder, plus there is some > out-of date info on my blog http://www.leonward.com. > > I actually delivered a presentation at Defcon last weekend all about > OpenFPC. I have forwarded the slides separately. Hopefully that will help > as well. > > > > As for your specific question about OpenFPC GUI. That's actually now been > deprecated as it's no longer relevant for how it functions in a distributed > manner. The OpenFPC-Chrome Extension will be the next best thing for > interacting with the QueueDaemon remotely in a GUI-like way. > > > > Cheers, > > > > -L > > > > > > > > On Mon, Aug 17, 2015 at 4:25 PM, Igor Kaplan <igo...@gm...> wrote: > > Hi All, > > > > My name is Igor. I just found the OpenFPC and evaluating it. Looks like > it is very good tool. > > I successfully installed on Ubuntu 14.4 with Perl 5.18 > > I have installed the OpenFPC-master, so it is the latest code. > > > > Now I would like to find out if there is more documentation beside files > which I could find under docs directory. > > For example the INSTALL.md refers to the USAGE document, however I was > not able to find it anywhere > > > > I am looking for the usage other then basic, just to find out, what are > my advanced options. > > > > For example the openfpc-dbmaint.sh script is also able to create the gui > database, I wonder, what it is for? > > > > The OpenFPC looks to be very powerful, just would like to understand it as > best as I can. > > > > Would so much appreciate any reply’s. > > > > Many thanks. > > > > Igor. > > > > > > ------------------------------------------------------------------------------ > > _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > > > > ------------------------------------------------------------------------------ > > _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > |
|
From: Igor K. <igo...@gm...> - 2015-08-18 17:21:22
|
Hi Leon, all, I have one more question please. Based on the documentation the following line in the openfpc config file restricts the space usage of captured data to 50 percent: PCAP_SPACE=50 So, if the data size exceeds 50 percent old files will be deleted automatically? Will openfpc also delete the old MySQL session tables? Many thanks and all the best! -Igor. From: leo...@gm... [mailto:leo...@gm...] On Behalf Of Leon Ward Sent: Monday, August 17, 2015 11:51 AM To: Igor Kaplan Cc: ope...@li... Subject: Re: [Openfpc-users] Openfpc usage Hi, Documentation is really one of the places that really needs some extra focus. The best docs I can point you to are in that folder, plus there is some out-of date info on my blog http://www.leonward.com. I actually delivered a presentation at Defcon last weekend all about OpenFPC. I have forwarded the slides separately. Hopefully that will help as well. As for your specific question about OpenFPC GUI. That's actually now been deprecated as it's no longer relevant for how it functions in a distributed manner. The OpenFPC-Chrome Extension will be the next best thing for interacting with the QueueDaemon remotely in a GUI-like way. Cheers, -L On Mon, Aug 17, 2015 at 4:25 PM, Igor Kaplan <igo...@gm...> wrote: Hi All, My name is Igor. I just found the OpenFPC and evaluating it. Looks like it is very good tool. I successfully installed on Ubuntu 14.4 with Perl 5.18 I have installed the OpenFPC-master, so it is the latest code. Now I would like to find out if there is more documentation beside files which I could find under docs directory. For example the INSTALL.md refers to the USAGE document, however I was not able to find it anywhere I am looking for the usage other then basic, just to find out, what are my advanced options. For example the openfpc-dbmaint.sh script is also able to create the gui database, I wonder, what it is for? The OpenFPC looks to be very powerful, just would like to understand it as best as I can. Would so much appreciate any reply’s. Many thanks. Igor. ------------------------------------------------------------------------------ _______________________________________________ Openfpc-users mailing list Ope...@li... https://lists.sourceforge.net/lists/listinfo/openfpc-users |
|
From: Leon W. <le...@le...> - 2015-08-17 15:51:20
|
Hi, Documentation is really one of the places that really needs some extra focus. The best docs I can point you to are in that folder, plus there is some out-of date info on my blog http://www.leonward.com. I actually delivered a presentation at Defcon last weekend all about OpenFPC. I have forwarded the slides separately. Hopefully that will help as well. As for your specific question about OpenFPC GUI. That's actually now been deprecated as it's no longer relevant for how it functions in a distributed manner. The OpenFPC-Chrome Extension will be the next best thing for interacting with the QueueDaemon remotely in a GUI-like way. Cheers, -L On Mon, Aug 17, 2015 at 4:25 PM, Igor Kaplan <igo...@gm...> wrote: > Hi All, > > > > My name is Igor. I just found the OpenFPC and evaluating it. Looks like > it is very good tool. > > I successfully installed on Ubuntu 14.4 with Perl 5.18 > > I have installed the OpenFPC-master, so it is the latest code. > > > > Now I would like to find out if there is more documentation beside files > which I could find under docs directory. > > For example the INSTALL.md refers to the USAGE document, however I was > not able to find it anywhere > > > > I am looking for the usage other then basic, just to find out, what are > my advanced options. > > > > For example the openfpc-dbmaint.sh script is also able to create the gui > database, I wonder, what it is for? > > > > The OpenFPC looks to be very powerful, just would like to understand it as > best as I can. > > > > Would so much appreciate any reply’s. > > > > Many thanks. > > > > Igor. > > > > > ------------------------------------------------------------------------------ > > _______________________________________________ > Openfpc-users mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openfpc-users > > |
|
From: Igor K. <igo...@gm...> - 2015-08-17 15:25:26
|
Hi All, My name is Igor. I just found the OpenFPC and evaluating it. Looks like it is very good tool. I successfully installed on Ubuntu 14.4 with Perl 5.18 I have installed the OpenFPC-master, so it is the latest code. Now I would like to find out if there is more documentation beside files which I could find under docs directory. For example the INSTALL.md refers to the USAGE document, however I was not able to find it anywhere I am looking for the usage other then basic, just to find out, what are my advanced options. For example the openfpc-dbmaint.sh script is also able to create the gui database, I wonder, what it is for? The OpenFPC looks to be very powerful, just would like to understand it as best as I can. Would so much appreciate any reply's. Many thanks. Igor. |
|
From: Leon W. <le...@rm...> - 2015-05-21 11:16:21
|
Made the change and pushed it. However looking at where the user is created
as part of the install, I'm thinking you're not using a debian-like OS
(Ubuntu).
I can only support the platform that I know. This is how I check for and
create the user using the adduser command, but it's only relevant to some
distros.
if ! getent passwd openfpc >/dev/null
then
echo -e "[*] Adding user openfpc"
* adduser --quiet --system --group --no-create-home
--shell /usr/sbin/nologin openfpc* fi
done
On Thu, May 21, 2015 at 11:33 AM, Leon Ward <le...@rm...> wrote:
> Yeah, this is a part time 'passion' thing.
>
> It's clear that the forceinstall option is where all the problems have
> come from, I forgot that even existed and it's not working as it should.
> I'll give it a quick update, shouldn't take more than 10 mins to 'fix' and
> check it in.
>
> When I'm done, update your checkout, then use the "reinstall" option.
>
> -L
>
>
> On Thu, May 21, 2015 at 12:07 AM, Le CON <mat...@ho...> wrote:
>
>> haha real life.. so is this a side project for you?
>>
>> ok your absolutely right, there is no 'openfpc' system user, how do i
>> create a system/hidden one with the correct attributes?? I did
>> an openfpc-install.sh forceinstall because it wasnt recognising that the
>> packages were installed when they were so i forced it.
>>
>> pretty sure cxtracker is installed... ok it was, just hadnt copied to
>> /usr/bin .... my bad
>>
>> I think SELinux was causing me problems ......
>>
>>
>>
>>
>>
>>
>> got this now -
>>
>> sudo openfpc -a start
>> Starting Daemonlogger (DC1_fw)...
>> unable to map openfpc to a uid, ownership not changed: at /usr/bin/openfpc
>> line 577
>> unable to map openfpc to a gid, group ownership not changed: at
>> /usr/bin/openfpc line 577
>> unable to map openfpc to a uid, ownership not changed: at
>> /usr/bin/openfpc line 595
>> unable to map openfpc to a gid, group ownership not changed: at
>> /usr/bin/openfpc line 595
>> Done
>> Starting OpenFPC Queue Daemon (DC1_fw)...
>> Could not find uid and gid user openfpc at /usr/bin/openfpc-queued line 126
>> Failed
>> - Check syslog for details
>> Starting OpenFPC cxtracker
>> (DC1_fw)... Done
>> Starting OpenFPC Connection Uploader (DC1_fw)
>> ... Done
>>
>>
>> but still halted -
>>
>> sudo openfpc -a status
>> Daemonlogger (DC1_fw) :
>> Stopped
>> OpenFPC Queue Daemon (DC1_fw):
>> Stopped
>> OpenFPC Connection Tracker (DC1_fw) :
>> Stopped
>> OpenFPC Connection Uploader (DC1_fw) :
>> Stopped
>>
>>
>>
>>
>>
>> ------------------------------
>> From: mat...@ho...
>> Subject: Fwd: OpenFPC GUI
>> Date: Wed, 20 May 2015 21:45:03 +1200
>> To: mat...@nz...
>>
>>
>>
>> Sent from my iPad
>>
>> Begin forwarded message:
>>
>> *From:* Leon Ward <le...@rm...>
>> *Date:* 20 May 2015 8:03:08 pm NZST
>> *To:* Le CON <mat...@ho...>, ope...@so...
>> *Subject:* *Re: FW: OpenFPC GUI*
>>
>> Hi,
>>
>> Your 1st problem is interesting. I can see why you're getting the error,
>> but I'm not sure why it's working for me as both with and without %. Hrmmm
>> Perl weirdness.
>>
>> What version of Perl are you using and what linux distro?
>>
>> The 2nd problem(s) are different. You don't have cxtracker installed, and
>> doens't look like you have the openfpc user. How did you perform install? I
>> assume using openfpc-install? At startup OpenFPC drops root permissions for
>> security reasons.
>>
>> "Starting Daemonlogger (DC1_fw)... unable to map openfpc to a uid,
>> ownership not changed: at /usr/bin/openfpc line 577"
>>
>> This makes me think that you've not got a user called openfpc. This
>> should have been made as part of the install, does the user exist?
>>
>> "Starting OpenFPC Queue Daemon (DC1_fw)...
>> Could not find uid and gid user openfpc at /usr/bin/openfpc-queued line 126"
>>
>> Same as above.
>>
>> Was there any errors when running openfpc-install.sh install?
>>
>> "[!] cxtracker not found on this system. Can't start it"
>>
>> You need to install cxtracker to use the connection tracking/searching
>> capabilities.
>> You'll find a .deb for amd64 in the repo to save you compiling your own.
>> At some point I'm going to set up an Ubuntu PPA so it can all be apt-get
>> installed, but real life keeps getting in the way.
>>
>> -L
>>
>>
>>
>>
>> On Wed, May 20, 2015 at 3:59 AM, Le CON <mat...@ho...> wrote:
>>
>> Before you write back,
>>
>> I have sorted out that initial issue, typo for 'for each my $file (keys
>> $c) {' - I put an '%' in front of "$c" to make
>>
>> for each my $file (keys %$c) {
>>
>> and it seemed to work.
>>
>>
>>
>> but now I have some other issues, looks like permissions but it has
>> managed to create a 'failed' directory in the buffer/data dir.
>>
>> is it ok to have the buffer/data dir the same as the session dir?
>>
>>
>>
>>
>>
>>
>>
>> $ sudo openfpc -a start
>>
>> Starting Daemonlogger (DC1_fw)...
>> unable to map openfpc to a uid, ownership not changed: at /usr/bin/openfpc
>> line 577
>> unable to map openfpc to a gid, group ownership not changed: at
>> /usr/bin/openfpc line 577
>> unable to map openfpc to a uid, ownership not changed: at
>> /usr/bin/openfpc line 595
>> unable to map openfpc to a gid, group ownership not changed: at
>> /usr/bin/openfpc line 595
>> Done
>> Starting OpenFPC Queue Daemon (DC1_fw)...
>> Could not find uid and gid user openfpc at /usr/bin/openfpc-queued line 126
>> Failed
>> - Check syslog for details
>> Starting OpenFPC cxtracker (DC1_fw)...
>> Failed
>> [!] cxtracker not found on this system. Can't start it
>> Starting OpenFPC Connection Uploader (DC1_fw)
>> ... Done
>>
>>
>> ------------------------------
>> From: mat...@ho...
>> To: le...@rm...
>> Subject: RE: OpenFPC GUI
>> Date: Wed, 20 May 2015 13:06:27 +1200
>>
>>
>> Hey Leon,
>>
>> hope you had a good holidays, havent talked to you in a while but im back
>> about to release a prod box and I want OFPC on it for possible action.
>>
>> running into a problem here with your latest gitclone -
>>
>> $ ./openfpc -a status
>> Type of arg 1 to keys must be hash (not private variable) at ./openfpc
>> line 80, near "$c) "
>> Execution of ./openfpc aborted due to compilation errors.
>>
>>
>> inspecting the /usr/bin/openfpc file around line 80 -
>>
>>
>>
>> for each my $file (keys $c) {
>> my $bad=0;
>> if ($c->{$file}{NODENAME} =~ /[-\?\\\/\.;]/ ) {
>> my $err="Invalid characters found in NODENAME
>> \"$c->{$f$
>> push(@errs, $err);
>> $bad=1;
>> }
>>
>> if ($c->{$file}{'OFPC_ENABLED'} =~ /[yY1]/ ) {
>> if ( grep (/$c->{$file}{'OFPC_PORT'}/,
>> @ports)) {
>> #if ( $c->{$file}{'OFPC_PORT'} ~~
>> @ports) {
>> my $err="OFPC Port
>> $c->{$file}{'OFPC_PORT'} alr$
>> push(@errs, $err);
>> $bad=1;
>> }
>>
>>
>>
>>
>> .... any help ? :-)
>>
>>
>> ------------------------------
>> From: mat...@ho...
>> To: le...@rm...
>> Subject: RE: OpenFPC GUI
>> Date: Tue, 3 Feb 2015 15:57:35 +1300
>>
>> also the attached files are in /tmp/ along with the pcaps under arbitrary
>> charstrg folder names which should be passed to
>> '/var/tmp/openfpc/extracted' or to where specified by '-w'
>>
>> directory listing of '/tmp' below
>>
>> ./yrS6Iyy3VA
>> ./yrS6Iyy3VA/AB51CD9A-AB4F-11E4-BC85-ABA9F27951B6.pcap-1422930496.pcap
>> ./yrS6Iyy3VA/AB51CD9A-AB4F-11E4-BC85-ABA9F27951B6.pcap-1422927447.pcap
>> ./LVarjPtZeD
>>
>> ./LVarjPtZeD/1422931871-82A48734-AB4F-11E4-8C8C-C2F0232310D0.pcap-1422930496.pcap
>>
>> ./LVarjPtZeD/1422931871-82A48734-AB4F-11E4-8C8C-C2F0232310D0.pcap-1422927447.pcap
>> ./tZD9uK6zBQ
>>
>> ./tZD9uK6zBQ/1422931856-79E21E04-AB4F-11E4-BF16-AAB72E8C5B43.pcap-1422930496.pcap
>>
>> ./tZD9uK6zBQ/1422931856-79E21E04-AB4F-11E4-BF16-AAB72E8C5B43.pcap-1422927447.pcap
>> ./Fsm6jpXfZ1
>> ./Fsm6jpXfZ1/70CF2DAC-AB4F-11E4-AB1B-927F580AA963.pcap-1422930496.pcap
>> ./Fsm6jpXfZ1/70CF2DAC-AB4F-11E4-AB1B-927F580AA963.pcap-1422927447.pcap
>> ./yd7MXT3N1C
>> ./yd7MXT3N1C/620FCBD2-AB4F-11E4-9D95-B9F6FDE379F2.pcap-1422930496.pcap
>> ./yd7MXT3N1C/620FCBD2-AB4F-11E4-9D95-B9F6FDE379F2.pcap-1422927447.pcap
>> ./vWCfzuHlAT
>> ./vWCfzuHlAT/25C97BF2-AB4D-11E4-831B-E90AE8CCE601.pcap-1422927447.pcap
>> ./vWCfzuHlAT/25C97BF2-AB4D-11E4-831B-E90AE8CCE601.pcap-1422930496.pcap
>> ./zIf6QdvWMD
>> ./zIf6QdvWMD/20A9BCB8-AB4D-11E4-BB8F-BC6F0D6DDDF9.pcap-1422927447.pcap
>> ./zIf6QdvWMD/20A9BCB8-AB4D-11E4-BB8F-BC6F0D6DDDF9.pcap-1422930496.pcap
>> ./9rXWKGy7k8
>>
>> ./9rXWKGy7k8/1422930542-6AB765D6-AB4C-11E4-8DD8-FCDB2C1C0BFB.pcap-1422927447.pcap
>>
>> ./9rXWKGy7k8/1422930542-6AB765D6-AB4C-11E4-8DD8-FCDB2C1C0BFB.pcap-1422930496.pcap
>> ./yJPlb8V5ck
>> ./yJPlb8V5ck/64C17CC0-AB4C-11E4-96E2-F65304730EEE.pcap-1422927447.pcap
>> ./yJPlb8V5ck/64C17CC0-AB4C-11E4-96E2-F65304730EEE.pcap-1422930496.pcap
>> ./GnZBweu_98
>>
>> ./GnZBweu_98/1422929046-EF330512-AB48-11E4-8C90-B00FB10C42E7.pcap-1422921283.pcap
>>
>> ./GnZBweu_98/1422929046-EF330512-AB48-11E4-8C90-B00FB10C42E7.pcap-1422927447.pcap
>> ./axVvFb9BGi
>> ./axVvFb9BGi/E49D4A36-AB48-11E4-BF6A-CFCF0FC3CBD4.pcap-1422921283.pcap
>> ./axVvFb9BGi/E49D4A36-AB48-11E4-BF6A-CFCF0FC3CBD4.pcap-1422927447.pcap
>> ./0nUUI4fSL9
>>
>> ./0nUUI4fSL9/1422928573-D5281E10-AB47-11E4-BD4E-D91F209C3CB7.pcap-1422921283.pcap
>>
>> ./0nUUI4fSL9/1422928573-D5281E10-AB47-11E4-BD4E-D91F209C3CB7.pcap-1422927447.pcap
>> ./7tobw5Q5AF
>> ./7tobw5Q5AF/CBBF3EEE-AB47-11E4-86CD-ADAAB097DFA2.pcap-1422921283.pcap
>> ./7tobw5Q5AF/CBBF3EEE-AB47-11E4-86CD-ADAAB097DFA2.pcap-1422927447.pcap
>> ./V7eE9tudIL
>> ./V7eE9tudIL/8DA5FC66-AB46-11E4-9277-CC6BA90FB7B0.pcap-1422921283.pcap
>> ./V7eE9tudIL/8DA5FC66-AB46-11E4-9277-CC6BA90FB7B0.pcap-1422927447.pcap
>> ./mWHjnLKQiU
>>
>> ./mWHjnLKQiU/1422927926-5397C37E-AB46-11E4-AEB6-9917720B0A8C.pcap-1422921283.pcap
>>
>> ./mWHjnLKQiU/1422927926-5397C37E-AB46-11E4-AEB6-9917720B0A8C.pcap-1422927447.pcap
>> ./LNLXfNtoyp
>>
>> ./LNLXfNtoyp/1422927906-479E6F0A-AB46-11E4-A9EF-82998480152D.pcap-1422921283.pcap
>>
>> ./LNLXfNtoyp/1422927906-479E6F0A-AB46-11E4-A9EF-82998480152D.pcap-1422927447.pcap
>> ./systemd-mysqld.service-XIzh6fa
>> find: ‘./systemd-mysqld.service-XIzh6fa’: Permission denied
>> ./.UUID_STATE
>> ./.UUID_NODEID
>> ./cxtracker.start.log
>> ./openfpc-untitled.log
>> ./systemd-cups.service-X2TaKgN
>> find: ‘./systemd-cups.service-X2TaKgN’: Permission denied
>> ./.esd-1000
>> ./.esd-1000/socket
>> ./systemd-colord.service-XzrLIJU
>> find: ‘./systemd-colord.service-XzrLIJU’: Permission denied
>> ./hogsuspend
>> ./.X0-lock
>> ./systemd-rtkit-daemon.service-XIrHD2J
>> find: ‘./systemd-rtkit-daemon.service-XIrHD2J’: Permission denied
>> ./.Test-unix
>> ./.font-unix
>> ./.XIM-unix
>> ./.ICE-unix
>> ./.ICE-unix/1471
>> ./.ICE-unix/1040
>> ./.X11-unix
>> ./.X11-unix/X0
>>
>>
>>
>>
>> ------------------------------
>> From: mat...@ho...
>> To: le...@rm...
>> Subject: RE: OpenFPC GUI
>> Date: Tue, 3 Feb 2015 15:30:34 +1300
>>
>> Hey Leon hope you had a good break :-)
>>
>> so im back running into the same problem - can search fine but cant
>> 'fetch' or 'store' the pcaps.
>>
>> 'fetch' gives -
>> OFPC Request Failed: 0
>>
>>
>>
>> 'store' gives
>>
>> #####################################
>> Queue Position: 0
>> Remote File : E49D4A36-AB48-11E4-BF6A-CFCF0FC3CBD4.pcap
>> Result : In Queue
>>
>>
>>
>>
>> interestingly when running a search, the pcaps seem to be being extracted
>> to some extent, but dont make it to the specififed
>> /var/tmp/openfpc/extracted folder.
>>
>>
>> >>
>>
>> sudo find / -iname "*.pcap"
>>
>> /tmp/0nUUI4fSL9/1422928573-D5281E10-AB47-11E4-BD4E-D91F209C3CB7.pcap-1422921283.pcap
>>
>> /tmp/0nUUI4fSL9/1422928573-D5281E10-AB47-11E4-BD4E-D91F209C3CB7.pcap-1422927447.pcap
>> /tmp/7tobw5Q5AF/CBBF3EEE-AB47-11E4-86CD-ADAAB097DFA2.pcap-1422921283.pcap
>> /tmp/7tobw5Q5AF/CBBF3EEE-AB47-11E4-86CD-ADAAB097DFA2.pcap-1422927447.pcap
>> /tmp/V7eE9tudIL/8DA5FC66-AB46-11E4-9277-CC6BA90FB7B0.pcap-1422921283.pcap
>> /tmp/V7eE9tudIL/8DA5FC66-AB46-11E4-9277-CC6BA90FB7B0.pcap-1422927447.pcap
>>
>> /tmp/mWHjnLKQiU/1422927926-5397C37E-AB46-11E4-AEB6-9917720B0A8C.pcap-1422921283.pcap
>>
>> /tmp/mWHjnLKQiU/1422927926-5397C37E-AB46-11E4-AEB6-9917720B0A8C.pcap-1422927447.pcap
>>
>> /tmp/LNLXfNtoyp/1422927906-479E6F0A-AB46-11E4-A9EF-82998480152D.pcap-1422921283.pcap
>>
>> /tmp/LNLXfNtoyp/1422927906-479E6F0A-AB46-11E4-A9EF-82998480152D.pcap-1422927447.pcap
>>
>>
>> if run debug it looks fine
>>
>> the only thing in the /var/tmp/openfpc/extracted folder is '0.txt'
>>
>> looks like the pcaps are writing properly to the pcaps folder but when
>> attempting to store or fetch its getting stuck??
>>
>>
>>
>
|
|
From: Leon W. <le...@rm...> - 2015-05-21 10:36:26
|
Yeah, this is a part time 'passion' thing.
It's clear that the forceinstall option is where all the problems have come
from, I forgot that even existed and it's not working as it should.
I'll give it a quick update, shouldn't take more than 10 mins to 'fix' and
check it in.
When I'm done, update your checkout, then use the "reinstall" option.
-L
On Thu, May 21, 2015 at 12:07 AM, Le CON <mat...@ho...> wrote:
> haha real life.. so is this a side project for you?
>
> ok your absolutely right, there is no 'openfpc' system user, how do i
> create a system/hidden one with the correct attributes?? I did
> an openfpc-install.sh forceinstall because it wasnt recognising that the
> packages were installed when they were so i forced it.
>
> pretty sure cxtracker is installed... ok it was, just hadnt copied to
> /usr/bin .... my bad
>
> I think SELinux was causing me problems ......
>
>
>
>
>
>
> got this now -
>
> sudo openfpc -a start
> Starting Daemonlogger (DC1_fw)...
> unable to map openfpc to a uid, ownership not changed: at /usr/bin/openfpc
> line 577
> unable to map openfpc to a gid, group ownership not changed: at
> /usr/bin/openfpc line 577
> unable to map openfpc to a uid, ownership not changed: at
> /usr/bin/openfpc line 595
> unable to map openfpc to a gid, group ownership not changed: at
> /usr/bin/openfpc line 595
> Done
> Starting OpenFPC Queue Daemon (DC1_fw)...
> Could not find uid and gid user openfpc at /usr/bin/openfpc-queued line 126
> Failed
> - Check syslog for details
> Starting OpenFPC cxtracker (DC1_fw)...
> Done
> Starting OpenFPC Connection Uploader (DC1_fw) ...
> Done
>
>
> but still halted -
>
> sudo openfpc -a status
> Daemonlogger (DC1_fw) :
> Stopped
> OpenFPC Queue Daemon (DC1_fw):
> Stopped
> OpenFPC Connection Tracker (DC1_fw) :
> Stopped
> OpenFPC Connection Uploader (DC1_fw) :
> Stopped
>
>
>
>
>
> ------------------------------
> From: mat...@ho...
> Subject: Fwd: OpenFPC GUI
> Date: Wed, 20 May 2015 21:45:03 +1200
> To: mat...@nz...
>
>
>
> Sent from my iPad
>
> Begin forwarded message:
>
> *From:* Leon Ward <le...@rm...>
> *Date:* 20 May 2015 8:03:08 pm NZST
> *To:* Le CON <mat...@ho...>, ope...@so...
> *Subject:* *Re: FW: OpenFPC GUI*
>
> Hi,
>
> Your 1st problem is interesting. I can see why you're getting the error,
> but I'm not sure why it's working for me as both with and without %. Hrmmm
> Perl weirdness.
>
> What version of Perl are you using and what linux distro?
>
> The 2nd problem(s) are different. You don't have cxtracker installed, and
> doens't look like you have the openfpc user. How did you perform install? I
> assume using openfpc-install? At startup OpenFPC drops root permissions for
> security reasons.
>
> "Starting Daemonlogger (DC1_fw)... unable to map openfpc to a uid,
> ownership not changed: at /usr/bin/openfpc line 577"
>
> This makes me think that you've not got a user called openfpc. This should
> have been made as part of the install, does the user exist?
>
> "Starting OpenFPC Queue Daemon (DC1_fw)...
> Could not find uid and gid user openfpc at /usr/bin/openfpc-queued line 126"
>
> Same as above.
>
> Was there any errors when running openfpc-install.sh install?
>
> "[!] cxtracker not found on this system. Can't start it"
>
> You need to install cxtracker to use the connection tracking/searching
> capabilities.
> You'll find a .deb for amd64 in the repo to save you compiling your own.
> At some point I'm going to set up an Ubuntu PPA so it can all be apt-get
> installed, but real life keeps getting in the way.
>
> -L
>
>
>
>
> On Wed, May 20, 2015 at 3:59 AM, Le CON <mat...@ho...> wrote:
>
> Before you write back,
>
> I have sorted out that initial issue, typo for 'for each my $file (keys
> $c) {' - I put an '%' in front of "$c" to make
>
> for each my $file (keys %$c) {
>
> and it seemed to work.
>
>
>
> but now I have some other issues, looks like permissions but it has
> managed to create a 'failed' directory in the buffer/data dir.
>
> is it ok to have the buffer/data dir the same as the session dir?
>
>
>
>
>
>
>
> $ sudo openfpc -a start
>
> Starting Daemonlogger (DC1_fw)...
> unable to map openfpc to a uid, ownership not changed: at /usr/bin/openfpc
> line 577
> unable to map openfpc to a gid, group ownership not changed: at
> /usr/bin/openfpc line 577
> unable to map openfpc to a uid, ownership not changed: at
> /usr/bin/openfpc line 595
> unable to map openfpc to a gid, group ownership not changed: at
> /usr/bin/openfpc line 595
> Done
> Starting OpenFPC Queue Daemon (DC1_fw)...
> Could not find uid and gid user openfpc at /usr/bin/openfpc-queued line 126
> Failed
> - Check syslog for details
> Starting OpenFPC cxtracker (DC1_fw)...
> Failed
> [!] cxtracker not found on this system. Can't start it
> Starting OpenFPC Connection Uploader (DC1_fw) ...
> Done
>
>
> ------------------------------
> From: mat...@ho...
> To: le...@rm...
> Subject: RE: OpenFPC GUI
> Date: Wed, 20 May 2015 13:06:27 +1200
>
>
> Hey Leon,
>
> hope you had a good holidays, havent talked to you in a while but im back
> about to release a prod box and I want OFPC on it for possible action.
>
> running into a problem here with your latest gitclone -
>
> $ ./openfpc -a status
> Type of arg 1 to keys must be hash (not private variable) at ./openfpc
> line 80, near "$c) "
> Execution of ./openfpc aborted due to compilation errors.
>
>
> inspecting the /usr/bin/openfpc file around line 80 -
>
>
>
> for each my $file (keys $c) {
> my $bad=0;
> if ($c->{$file}{NODENAME} =~ /[-\?\\\/\.;]/ ) {
> my $err="Invalid characters found in NODENAME
> \"$c->{$f$
> push(@errs, $err);
> $bad=1;
> }
>
> if ($c->{$file}{'OFPC_ENABLED'} =~ /[yY1]/ ) {
> if ( grep (/$c->{$file}{'OFPC_PORT'}/, @ports))
> {
> #if ( $c->{$file}{'OFPC_PORT'} ~~
> @ports) {
> my $err="OFPC Port
> $c->{$file}{'OFPC_PORT'} alr$
> push(@errs, $err);
> $bad=1;
> }
>
>
>
>
> .... any help ? :-)
>
>
> ------------------------------
> From: mat...@ho...
> To: le...@rm...
> Subject: RE: OpenFPC GUI
> Date: Tue, 3 Feb 2015 15:57:35 +1300
>
> also the attached files are in /tmp/ along with the pcaps under arbitrary
> charstrg folder names which should be passed to
> '/var/tmp/openfpc/extracted' or to where specified by '-w'
>
> directory listing of '/tmp' below
>
> ./yrS6Iyy3VA
> ./yrS6Iyy3VA/AB51CD9A-AB4F-11E4-BC85-ABA9F27951B6.pcap-1422930496.pcap
> ./yrS6Iyy3VA/AB51CD9A-AB4F-11E4-BC85-ABA9F27951B6.pcap-1422927447.pcap
> ./LVarjPtZeD
>
> ./LVarjPtZeD/1422931871-82A48734-AB4F-11E4-8C8C-C2F0232310D0.pcap-1422930496.pcap
>
> ./LVarjPtZeD/1422931871-82A48734-AB4F-11E4-8C8C-C2F0232310D0.pcap-1422927447.pcap
> ./tZD9uK6zBQ
>
> ./tZD9uK6zBQ/1422931856-79E21E04-AB4F-11E4-BF16-AAB72E8C5B43.pcap-1422930496.pcap
>
> ./tZD9uK6zBQ/1422931856-79E21E04-AB4F-11E4-BF16-AAB72E8C5B43.pcap-1422927447.pcap
> ./Fsm6jpXfZ1
> ./Fsm6jpXfZ1/70CF2DAC-AB4F-11E4-AB1B-927F580AA963.pcap-1422930496.pcap
> ./Fsm6jpXfZ1/70CF2DAC-AB4F-11E4-AB1B-927F580AA963.pcap-1422927447.pcap
> ./yd7MXT3N1C
> ./yd7MXT3N1C/620FCBD2-AB4F-11E4-9D95-B9F6FDE379F2.pcap-1422930496.pcap
> ./yd7MXT3N1C/620FCBD2-AB4F-11E4-9D95-B9F6FDE379F2.pcap-1422927447.pcap
> ./vWCfzuHlAT
> ./vWCfzuHlAT/25C97BF2-AB4D-11E4-831B-E90AE8CCE601.pcap-1422927447.pcap
> ./vWCfzuHlAT/25C97BF2-AB4D-11E4-831B-E90AE8CCE601.pcap-1422930496.pcap
> ./zIf6QdvWMD
> ./zIf6QdvWMD/20A9BCB8-AB4D-11E4-BB8F-BC6F0D6DDDF9.pcap-1422927447.pcap
> ./zIf6QdvWMD/20A9BCB8-AB4D-11E4-BB8F-BC6F0D6DDDF9.pcap-1422930496.pcap
> ./9rXWKGy7k8
>
> ./9rXWKGy7k8/1422930542-6AB765D6-AB4C-11E4-8DD8-FCDB2C1C0BFB.pcap-1422927447.pcap
>
> ./9rXWKGy7k8/1422930542-6AB765D6-AB4C-11E4-8DD8-FCDB2C1C0BFB.pcap-1422930496.pcap
> ./yJPlb8V5ck
> ./yJPlb8V5ck/64C17CC0-AB4C-11E4-96E2-F65304730EEE.pcap-1422927447.pcap
> ./yJPlb8V5ck/64C17CC0-AB4C-11E4-96E2-F65304730EEE.pcap-1422930496.pcap
> ./GnZBweu_98
>
> ./GnZBweu_98/1422929046-EF330512-AB48-11E4-8C90-B00FB10C42E7.pcap-1422921283.pcap
>
> ./GnZBweu_98/1422929046-EF330512-AB48-11E4-8C90-B00FB10C42E7.pcap-1422927447.pcap
> ./axVvFb9BGi
> ./axVvFb9BGi/E49D4A36-AB48-11E4-BF6A-CFCF0FC3CBD4.pcap-1422921283.pcap
> ./axVvFb9BGi/E49D4A36-AB48-11E4-BF6A-CFCF0FC3CBD4.pcap-1422927447.pcap
> ./0nUUI4fSL9
>
> ./0nUUI4fSL9/1422928573-D5281E10-AB47-11E4-BD4E-D91F209C3CB7.pcap-1422921283.pcap
>
> ./0nUUI4fSL9/1422928573-D5281E10-AB47-11E4-BD4E-D91F209C3CB7.pcap-1422927447.pcap
> ./7tobw5Q5AF
> ./7tobw5Q5AF/CBBF3EEE-AB47-11E4-86CD-ADAAB097DFA2.pcap-1422921283.pcap
> ./7tobw5Q5AF/CBBF3EEE-AB47-11E4-86CD-ADAAB097DFA2.pcap-1422927447.pcap
> ./V7eE9tudIL
> ./V7eE9tudIL/8DA5FC66-AB46-11E4-9277-CC6BA90FB7B0.pcap-1422921283.pcap
> ./V7eE9tudIL/8DA5FC66-AB46-11E4-9277-CC6BA90FB7B0.pcap-1422927447.pcap
> ./mWHjnLKQiU
>
> ./mWHjnLKQiU/1422927926-5397C37E-AB46-11E4-AEB6-9917720B0A8C.pcap-1422921283.pcap
>
> ./mWHjnLKQiU/1422927926-5397C37E-AB46-11E4-AEB6-9917720B0A8C.pcap-1422927447.pcap
> ./LNLXfNtoyp
>
> ./LNLXfNtoyp/1422927906-479E6F0A-AB46-11E4-A9EF-82998480152D.pcap-1422921283.pcap
>
> ./LNLXfNtoyp/1422927906-479E6F0A-AB46-11E4-A9EF-82998480152D.pcap-1422927447.pcap
> ./systemd-mysqld.service-XIzh6fa
> find: ‘./systemd-mysqld.service-XIzh6fa’: Permission denied
> ./.UUID_STATE
> ./.UUID_NODEID
> ./cxtracker.start.log
> ./openfpc-untitled.log
> ./systemd-cups.service-X2TaKgN
> find: ‘./systemd-cups.service-X2TaKgN’: Permission denied
> ./.esd-1000
> ./.esd-1000/socket
> ./systemd-colord.service-XzrLIJU
> find: ‘./systemd-colord.service-XzrLIJU’: Permission denied
> ./hogsuspend
> ./.X0-lock
> ./systemd-rtkit-daemon.service-XIrHD2J
> find: ‘./systemd-rtkit-daemon.service-XIrHD2J’: Permission denied
> ./.Test-unix
> ./.font-unix
> ./.XIM-unix
> ./.ICE-unix
> ./.ICE-unix/1471
> ./.ICE-unix/1040
> ./.X11-unix
> ./.X11-unix/X0
>
>
>
>
> ------------------------------
> From: mat...@ho...
> To: le...@rm...
> Subject: RE: OpenFPC GUI
> Date: Tue, 3 Feb 2015 15:30:34 +1300
>
> Hey Leon hope you had a good break :-)
>
> so im back running into the same problem - can search fine but cant
> 'fetch' or 'store' the pcaps.
>
> 'fetch' gives -
> OFPC Request Failed: 0
>
>
>
> 'store' gives
>
> #####################################
> Queue Position: 0
> Remote File : E49D4A36-AB48-11E4-BF6A-CFCF0FC3CBD4.pcap
> Result : In Queue
>
>
>
>
> interestingly when running a search, the pcaps seem to be being extracted
> to some extent, but dont make it to the specififed
> /var/tmp/openfpc/extracted folder.
>
>
> >>
>
> sudo find / -iname "*.pcap"
>
> /tmp/0nUUI4fSL9/1422928573-D5281E10-AB47-11E4-BD4E-D91F209C3CB7.pcap-1422921283.pcap
>
> /tmp/0nUUI4fSL9/1422928573-D5281E10-AB47-11E4-BD4E-D91F209C3CB7.pcap-1422927447.pcap
> /tmp/7tobw5Q5AF/CBBF3EEE-AB47-11E4-86CD-ADAAB097DFA2.pcap-1422921283.pcap
> /tmp/7tobw5Q5AF/CBBF3EEE-AB47-11E4-86CD-ADAAB097DFA2.pcap-1422927447.pcap
> /tmp/V7eE9tudIL/8DA5FC66-AB46-11E4-9277-CC6BA90FB7B0.pcap-1422921283.pcap
> /tmp/V7eE9tudIL/8DA5FC66-AB46-11E4-9277-CC6BA90FB7B0.pcap-1422927447.pcap
>
> /tmp/mWHjnLKQiU/1422927926-5397C37E-AB46-11E4-AEB6-9917720B0A8C.pcap-1422921283.pcap
>
> /tmp/mWHjnLKQiU/1422927926-5397C37E-AB46-11E4-AEB6-9917720B0A8C.pcap-1422927447.pcap
>
> /tmp/LNLXfNtoyp/1422927906-479E6F0A-AB46-11E4-A9EF-82998480152D.pcap-1422921283.pcap
>
> /tmp/LNLXfNtoyp/1422927906-479E6F0A-AB46-11E4-A9EF-82998480152D.pcap-1422927447.pcap
>
>
> if run debug it looks fine
>
> the only thing in the /var/tmp/openfpc/extracted folder is '0.txt'
>
> looks like the pcaps are writing properly to the pcaps folder but when
> attempting to store or fetch its getting stuck??
>
>
>
|
