Menu
▾
▴
Re: [Capstone-users] Memory Leak?
|
From: Jurriaan B. <jur...@gm...> - 2014-12-03 23:33:00
|
The 'yield' keyword allows one to fetch new records/rows (in this case disassembled instructions) on-demand. Doing an early exit (e.g., quitting from a for-loop iterating over a function that yield's after only, say, 2 out of 10 items) will prematurely exit the function as well. Because, of course, why would Python calculate the latter 8 results when they're not used in the first place? This is also useful for never-ending functions - I suppose calculating digits of Pi would be a 'good' example. Anyway, so your cs_free() call is never reached in this case - you should switch to cs_free()'ing every row after each iteration through the for loop. Regards, Jurriaan On 12/04/2014 12:27 AM, Nguyen Anh Quynh wrote: > > > On Thu, Dec 4, 2014 at 4:42 AM, Jan Newger <jan...@ne... > <mailto:jan...@ne...>> wrote: > > This is the python implementation of the disasm function (starting at > line 791): > > def disasm(self, code, offset, count=0): > all_insn = ctypes.POINTER(_cs_insn)() > '''if not _python2: > print(code) > code = code.encode() > print(code)''' > res = _cs.cs_disasm(self.csh, code, len(code), offset, count, > ctypes.byref(all_insn)) > if res > 0: > for i in range(res): > yield CsInsn(self, all_insn[i]) > _cs.cs_free(all_insn, res) > else: > status = _cs.cs_errno(self.csh) > if status != CS_ERR_OK: > raise CsError(status) > return > yield > > I'm really no python expert, but from what I see you apparently need to > free the instruction instances manually. However, if client code stops > enumeration over the instructions prematurely, then _cs.cs_free() is > never invoked, and thus memory is leaked, right? > > > yes the problem must be with Python binding but not the core. > however, in the above code, cs_free() is called after the "for" loop, > so i dont see how memleak can happen "prematurely". > > > thanks. > > > > > > On 03.12.2014 22:25, Jan Newger wrote: > > It seems the equivalent C implementation is not affected by the mem > > leak, which is to be expected, since the memory is explicitly freed > > anyways, and the group checking boils down to comparing an integer > value. > > > > If I had to guess, I'd suspect that in the python case the group > > checking code introduces a spurious reference to the instruction > > instance(?) which cannot be claimed by the GC. > > > > On 03.12.2014 16:57, Jan Newger wrote: > >> No I haven't tried to reproduce the mem leak with C. > >> It already took me a considerable amount of time to come up with this > >> minimal example. > >> > >> On 12/03/2014 04:53 PM, Capstone Engine wrote: > >>> > >>> > >>> On Wed, Dec 3, 2014 at 10:32 PM, Jan Newger > <jan...@ne... <mailto:jan...@ne...> > >>> <mailto:jan...@ne... <mailto:jan...@ne...>>> > wrote: > >>> > >>> > >>> Yes, it's using the latest version. > >>> OS was win7 x64 running python 2.7 with 32bit libraries. > >>> > >>> > >>> this is interesting. have you tried to code the same program in > C to see > >>> if the mem leak issue still happens? > >>> > >>> > >>> thanks. > >>> > >>> > >>> > >>> > >>> On 12/03/2014 04:31 PM, Nguyen Anh Quynh wrote: > >>> > > >>> > > >>> > On Wed, Dec 3, 2014 at 4:57 PM, Jan Newger > <jan...@ne... <mailto:jan...@ne...> > <mailto:jan...@ne... <mailto:jan...@ne...>> > >>> > <mailto:jan...@ne... > <mailto:jan...@ne...> <mailto:jan...@ne... > <mailto:jan...@ne...>>>> wrote: > >>> > > >>> > Hey, > >>> > > >>> > I was playing around with a few python scripts (using > >>> capstone among > >>> > other things) and always ran out of memory - and I > have no > >>> freaking idea > >>> > why. > >>> > > >>> > > >>> > is this with the latest 3.0 version? > >>> > > >>> > thanks, > >>> > Q > >>> > > >>> > > >>> > > >>> > > >>> > The code is really short: > >>> > > >>> > > >>> > from capstone import Cs > >>> > from capstone import CS_ARCH_X86 > >>> > from capstone import CS_MODE_32 > >>> > from capstone import CS_GRP_JUMP > >>> > from capstone import CS_GRP_CALL > >>> > from capstone import CS_GRP_RET > >>> > from capstone.x86_const import X86_INS_JNE, X86_INS_JMP > >>> > > >>> > """ > >>> > 0x401000: push ecx > >>> > 0x401001: pop ecx > >>> > 0x401002: mov eax, dword ptr [esp + 0x18] > >>> > 0x401006: mov eax, dword ptr [eax] > >>> > 0x401008: sar eax, 0 > >>> > 0x40100b: xor edi, eax > >>> > 0x40100d: nop > >>> > 0x40100e: add dword ptr [esp + 0x18], 4 > >>> > 0x401013: nop > >>> > 0x401014: dec word ptr [esp + 0x14] > >>> > 0x401019: shld edi, ecx, 0 > >>> > 0x40101d: jne 0x401000 > >>> > """ > >>> > def get_code(): > >>> > CODE = > >>> > > >>> > "\x51\x59\x8B\x44\x24\x18\x8B\x00\xC1\xF8\x00\x33\xF8\x90\x83\x44\x24\x18\x04\x90\x66\xFF\x4C\x24\x14\x0F\xA4\xCF\x00\x75\xE1" > >>> > > >>> > return CODE > >>> > > >>> > def is_branch(instr): > >>> > for group in branch_groups: > >>> > if group in instr.groups: > >>> > return True > >>> > return False > >>> > #return False > >>> > > >>> > # Disassemble until we hit basic block end. > >>> > def disasm(code): > >>> > disasm = Cs(CS_ARCH_X86, CS_MODE_32) > >>> > disasm.detail = True > >>> > address = 0x401000 > >>> > for instr in disasm.disasm(code, address): > >>> > print "0x%x:\t%s\t%s" % (instr.address, > instr.mnemonic, > >>> > instr.op_str) > >>> > if is_branch(instr): > >>> > break > >>> > > >>> > branch_groups = [CS_GRP_JUMP, CS_GRP_CALL, CS_GRP_RET] > >>> > code = get_code() > >>> > while True: > >>> > disasm(code) > >>> > > >>> > > >>> > That code goes out of memory after a few seconds. The > super > >>> weird thing > >>> > is, that if I change the implementation of > "is_branch(instr)" > >>> to simply > >>> > return False all the time, then the program does not > go out > >>> of memory! > >>> > Does anyone have an idea what's going on? > >>> > > >>> > Best > >>> > Jan > >>> > > >>> > > >>> > ------------------------------------------------------------------------------ > >>> > Download BIRT iHub F-Type - The Free Enterprise-Grade > BIRT Server > >>> > from Actuate! Instantly Supercharge Your Business > Reports and > >>> Dashboards > >>> > with Interactivity, Sharing, Native Excel Exports, App > >>> Integration & > >>> > more > >>> > Get technology previously reserved for billion-dollar > >>> corporations, FREE > >>> > > >>> > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > >>> > _______________________________________________ > >>> > Capstone-users mailing list > >>> > Cap...@li... > <mailto:Cap...@li...> > >>> <mailto:Cap...@li... > <mailto:Cap...@li...>> > >>> > <mailto:Cap...@li... > <mailto:Cap...@li...> > >>> <mailto:Cap...@li... > <mailto:Cap...@li...>>> > >>> > https://lists.sourceforge.net/lists/listinfo/capstone-users > >>> > > >>> > > >>> > > >>> > > >>> > > >>> > ------------------------------------------------------------------------------ > >>> > Download BIRT iHub F-Type - The Free Enterprise-Grade > BIRT Server > >>> > from Actuate! Instantly Supercharge Your Business Reports and > >>> Dashboards > >>> > with Interactivity, Sharing, Native Excel Exports, App > >>> Integration & more > >>> > Get technology previously reserved for billion-dollar > >>> corporations, FREE > >>> > > >>> > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > >>> > > >>> > > >>> > > >>> > _______________________________________________ > >>> > Capstone-users mailing list > >>> > Cap...@li... > <mailto:Cap...@li...> > >>> <mailto:Cap...@li... > <mailto:Cap...@li...>> > >>> > https://lists.sourceforge.net/lists/listinfo/capstone-users > >>> > > >>> > >>> > >>> > ------------------------------------------------------------------------------ > >>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT > Server > >>> from Actuate! Instantly Supercharge Your Business Reports > and Dashboards > >>> with Interactivity, Sharing, Native Excel Exports, App > Integration & > >>> more > >>> Get technology previously reserved for billion-dollar > corporations, FREE > >>> > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > >>> _______________________________________________ > >>> Capstone-users mailing list > >>> Cap...@li... > <mailto:Cap...@li...> > >>> <mailto:Cap...@li... > <mailto:Cap...@li...>> > >>> https://lists.sourceforge.net/lists/listinfo/capstone-users > >>> > >>> > >>> > >>> > >>> > ------------------------------------------------------------------------------ > >>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > >>> from Actuate! Instantly Supercharge Your Business Reports and > Dashboards > >>> with Interactivity, Sharing, Native Excel Exports, App > Integration & more > >>> Get technology previously reserved for billion-dollar > corporations, FREE > >>> > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > >>> > >>> > >>> > >>> _______________________________________________ > >>> Capstone-users mailing list > >>> Cap...@li... > <mailto:Cap...@li...> > >>> https://lists.sourceforge.net/lists/listinfo/capstone-users > >>> > >> > >> > >> > ------------------------------------------------------------------------------ > >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > >> from Actuate! Instantly Supercharge Your Business Reports and > Dashboards > >> with Interactivity, Sharing, Native Excel Exports, App > Integration & more > >> Get technology previously reserved for billion-dollar > corporations, FREE > >> > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > >> _______________________________________________ > >> Capstone-users mailing list > >> Cap...@li... > <mailto:Cap...@li...> > >> https://lists.sourceforge.net/lists/listinfo/capstone-users > >> > > > > > > > ------------------------------------------------------------------------------ > > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > > from Actuate! Instantly Supercharge Your Business Reports and > Dashboards > > with Interactivity, Sharing, Native Excel Exports, App Integration > & more > > Get technology previously reserved for billion-dollar > corporations, FREE > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > _______________________________________________ > > Capstone-users mailing list > > Cap...@li... > <mailto:Cap...@li...> > > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & > more > Get technology previously reserved for billion-dollar corporations, FREE > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > _______________________________________________ > Capstone-users mailing list > Cap...@li... > <mailto:Cap...@li...> > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > > > _______________________________________________ > Capstone-users mailing list > Cap...@li... > https://lists.sourceforge.net/lists/listinfo/capstone-users > |
