NetworkMiner packet analyzer

hjelmvik, can you assist me with NetworkMiner? I love and depend on it greatly for summarizing network packet capture data. However I have problem that has stumped me. I've got multiple XP SP3 systems (fully patched/updated) and I have been unable to run any version of NetworkMiner over 0.88. That is the last "stable" version that will run on our systems. I've tried using Microsoft's Process Monitor, but it doesn't provide any useful clues for once! Nor do I experience a "crash" that would generate a dump file for analysis. The scenario is this: I download Network Miner (say 0.92) and unzip it. I then run it. So far so good. Launches perfectly. Then I load a .pcap file (typically 250-450 MB though exits on smaller sizes as well). It loads fine and you can see both the loading window progressing as well as the data filling in the Network Miner window below. However, about 1/3 to 1/2 way through the load the program suddenly terminates. No errors, no lockups, nothing. This happens on .pcap files from both the stable and RC version of Wireshark as well as Network Monitor captures (.cap) that were converted to .pcap using "editcap -F" I'm stumped why I can't get a full .pcap load into Network Miner on any of our XP systems past v 0.88. I love a good mystery and would deeply value your insight into this awesome app! I really want to move up to the latest versions of Network Miner but just can't yet. Any troubleshooting tips or methods to generate debugging info for you would be great! I don't think there are missing dependencies as it launches and runs fine, for a while. I can't imagine that our systems are the only ones having this issue and I would gratefully help you to the best of my technical ability to trace it out. Cheers! --Claus Valca

Hi Claus, Thanks for pointing out this bug. I have been running NetworkMiner on XP SP3 machines before without any problems, so I suspect that this problem does not only depend on the OS. The best way to find out what is happening is to download the NetworkMiner source code from SourceForge and run it from within Visual Studio. That will surely output some info about the crash. One guess is that there could be some resource exhaustion problem, do your XP machines typically have less RAM than the others for example? Or have you been able to trigger this crash even with smaller files (such as <100 MB)? It would also be interesting to know if NetworkMiner always crashes on the same files, regardless which XP machine you load the pcap on. Or are these crashes intermittent, i.e. randomly occuring?

Erik, Thank you kindly for responding! I agree that I don't think it is an OS dependent issue either. Especially since I can execute the older (working) version of NM against the same pcap files with no sudden-exit issues. It's just on all the "newer" versions of NM this behavior happens on. It occurs on both desktop and laptop units with system RAM from 2 ~ 4 GB (x32 bit XP Pro systems.) And I'm not doing any other resource intensive routines while running NetworkMiner. Because we see a pretty wide range of file types getting reassembled I was (initially) starting to think it could be a newer/updated parsing file issue tripping over a particular file reassembly but I've not done a comparison to see that it halts in the same place each and every time. Per your suggestion I will also throw some smaller pcap files at it as well. I never considered loading/running the code in Visual Studio. That's an excellent suggestion! I think I will also build a simple/clean XP Pro Virtual PC VHD to execute it within as well....in case any of our standard image configuration applications are interfering as well for some reason. I might also run a screen-cast recording of it while loaded up in Process Explorer in the thread stack view. So when it exits I could go back and see where in the stack it was processing when killed. That might help generate a clue as well. I suppose one other thing I haven't tried yet is to actual run a longer term capture with NetworkMiner itself. I've teased with short-term captures but really only use it for reassembly of packets captured trough Wireshark or Network Monitor (after converting them to pcap from cap format). I've got some new areas to focus in now and will pass on any findings. NetworkMiner is an awesome utility that really enhances our network traffic analysis. Thank you for all the time you invest in maintaining it! Cheers! --Claus V.

A simple test you could perform before launching Visual Studio is to load the large pcap file with NetworkMiner from the command line. Just start CMD and run "NetworkMiner.exe somelargepcapfile.pcap". NetworkMiner will most probably crash also when you load the pcap this way, but the chances of getting a proper crash dump describing the triggered exception are much better. As for using NetworkMiner to capture live traffic I only recommend to use it as you've done so far, i.e. for short trouble shooting tasks. You run a much higher risk of dropping packets when sniffing with NetworkMiner since it will try to do all sorts of other stuff in parallell (such as reassembling files etc.). My recommendation is to use some simple capturing tool, such as dumpcap, to sniff reliably and then load the pcap into NetworkMiner, Wireshark or some other similar application as I've described here: http://networkminer.sourceforge.net/documents/Network_Forensics_Workshop_with_NetworkMiner.pdf I also generally recommend to split the capture files into segments of approx. 100MB per file for leaner handling. /erik

Erik, I actually prefer to use Microsoft's NetMon 3.4 (nmcap.exe) via the command line for my captures. Works perfectly. then I use editcap.exe in Wireshark to convert the cap to pcap (libpcap) format. Works like a charm. Wireshark is my backup capture. (Note: it doesn't seem to be an issue with the capture/conversion method as I can crash NetworkMiner 0.92 with either a native Wireshark pcap file over a certain size now, or a NetMon nmcap.exe capture I then converted from cap -> pcap.) I do feel a bit silly! Your tips jogged my brain and I suddenly recalled a few other ways to reliably get crash dump data on a process when Windows doesn't do it automatically: Process Explorer - http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx - new versions support attaching to a process for crash dump generations (full/mini). ProcDump - http://technet.microsoft.com/en-us/sysinternals/dd996900.aspx - Perfect CLI tool to do so from sysinternals\ ProcessActivityView - http://www.nirsoft.net/utils/process_activity_view.html - Nirsoft tool to monitor and export process activity logs. I've successfully done so now multiple times. ProcDump seems to give the most reliable output kicking off a Windows Crash Report dialog window every time. The error signature it generates seems to be consistently as follows: AppName: networkminer.exe AppVer: 0.92.0.0 AppStamp:4bfed6e2 ModName:P mscorwks.dll ModVer: 2.0.50727.3607 ModStamp:4add5446 fDebug: 0 Offset: 00082bcc I can now confirm that I am able to fully load (no crash exits) files that are up to ~ 100 MB in size in NetworkMiner 0.92 for the most part. Smaller size files, 10 ~ 80 MB load with no crashes at all. 100 ~ 135 MB captures are hit and miss, though it seems to work better/stabler (?) if I "pre-load" a smaller cap file (10-15 MB), dump the assembled files, then reload the larger one. That doesn't really make sense so it could be something else causing that observed behavior. Anything bigger than ~ 140 MB and I seem to always get the crash/exit. At least ProcDump is kicking off good dump files (procdump.exe -T networkminer.exe c:\dump.dmp) This is on a XP Pro (x86) (SP3) dual-core Intel Dell E6400 laptop, fully OS patched. 4 GB RAM, 320 GB HDD, lots of free space, NTFS formatted. Monitoring with Process Explorer, the CPU load for NetworkMiner never seemed to get over 10% and there was lots of available memory in the Process Explorer charts showing. I've got multiple DMP files now. Would you be interested in looking at them and if so, how can I get you a zip file of them? I'm not at all a programmer and though I can do basic crash look-ups and analysis with a fair bit of Googling, they might be very easy for you to analyze. It's not much but if it might be a way that I in some small way can contribute to your project, I'd be happy to help. As you mentioned, I always can split captures (our are in the 450 MB max range) into smaller ones, but I can just as easily load them up entirely in NetworkMiner 0.88 with none of the crash issues that 0.92 seems to have for me. I really was looking forward to moving up to the latest version. Cheers! --Claus V.

Claus, It would be nice if you could email one of those DMP files my way. I'm sure you'll be able to find my gmail address on the Internet or in NetworkMiner (I prefer not to post it here though). By the way, did you try to load the pcap from the command line as I suggested in my previous response? I would really like to see what Exception NetworkMiner will throw back at you when it crashes! Looking back at the features I introduced into NetworkMiner between version 0.88 and 0.89 the thing that most likely could be causing this behaviour is the "Sessions" tab. NetworkMiner might run into problems if there are just too many sessions to track..,. It would be interesting to investigate if that would be the case, for example by counting the number of sessions in the files that crash NM and compare those counts to pcap files of equal sizes that don't cause NM to crash.

Greetings Erik, Files have been so sent with some information on how they were generated (under what conditions). Specifically, yes, one of the dmp files was while loading a pcap from the command line as suggested. That was via attaching ProcessExplorer configured to do a mini-dmp to the process once running. If that didn't come out good let me know and I will replicate via ProcDmp instead which I think executes "cleaner". Several dmp files using that tool (but not via a CLI launch of NetworkMiner) were also attached. Again, I publicly wish to thank you for the time you are taking looking into this for me. I hope it benefits NetworkMiner and the larger community that depends on it! Sincerely, --Claus V.