Menu
▾
▴
capstone-users
|
From: Jan N. <jan...@ne...> - 2014-12-03 09:57:59
|
Hey,
I was playing around with a few python scripts (using capstone among
other things) and always ran out of memory - and I have no freaking idea
why.
The code is really short:
from capstone import Cs
from capstone import CS_ARCH_X86
from capstone import CS_MODE_32
from capstone import CS_GRP_JUMP
from capstone import CS_GRP_CALL
from capstone import CS_GRP_RET
from capstone.x86_const import X86_INS_JNE, X86_INS_JMP
"""
0x401000: push ecx
0x401001: pop ecx
0x401002: mov eax, dword ptr [esp + 0x18]
0x401006: mov eax, dword ptr [eax]
0x401008: sar eax, 0
0x40100b: xor edi, eax
0x40100d: nop
0x40100e: add dword ptr [esp + 0x18], 4
0x401013: nop
0x401014: dec word ptr [esp + 0x14]
0x401019: shld edi, ecx, 0
0x40101d: jne 0x401000
"""
def get_code():
CODE =
"\x51\x59\x8B\x44\x24\x18\x8B\x00\xC1\xF8\x00\x33\xF8\x90\x83\x44\x24\x18\x04\x90\x66\xFF\x4C\x24\x14\x0F\xA4\xCF\x00\x75\xE1"
return CODE
def is_branch(instr):
for group in branch_groups:
if group in instr.groups:
return True
return False
#return False
# Disassemble until we hit basic block end.
def disasm(code):
disasm = Cs(CS_ARCH_X86, CS_MODE_32)
disasm.detail = True
address = 0x401000
for instr in disasm.disasm(code, address):
print "0x%x:\t%s\t%s" % (instr.address, instr.mnemonic, instr.op_str)
if is_branch(instr):
break
branch_groups = [CS_GRP_JUMP, CS_GRP_CALL, CS_GRP_RET]
code = get_code()
while True:
disasm(code)
That code goes out of memory after a few seconds. The super weird thing
is, that if I change the implementation of "is_branch(instr)" to simply
return False all the time, then the program does not go out of memory!
Does anyone have an idea what's going on?
Best
Jan
|
|
From: Nguyen A. Q. <aq...@gm...> - 2014-12-03 15:31:43
|
On Wed, Dec 3, 2014 at 4:57 PM, Jan Newger <jan...@ne...> wrote: > Hey, > > I was playing around with a few python scripts (using capstone among > other things) and always ran out of memory - and I have no freaking idea > why. > is this with the latest 3.0 version? thanks, Q > The code is really short: > > > from capstone import Cs > from capstone import CS_ARCH_X86 > from capstone import CS_MODE_32 > from capstone import CS_GRP_JUMP > from capstone import CS_GRP_CALL > from capstone import CS_GRP_RET > from capstone.x86_const import X86_INS_JNE, X86_INS_JMP > > """ > 0x401000: push ecx > 0x401001: pop ecx > 0x401002: mov eax, dword ptr [esp + 0x18] > 0x401006: mov eax, dword ptr [eax] > 0x401008: sar eax, 0 > 0x40100b: xor edi, eax > 0x40100d: nop > 0x40100e: add dword ptr [esp + 0x18], 4 > 0x401013: nop > 0x401014: dec word ptr [esp + 0x14] > 0x401019: shld edi, ecx, 0 > 0x40101d: jne 0x401000 > """ > def get_code(): > CODE = > > "\x51\x59\x8B\x44\x24\x18\x8B\x00\xC1\xF8\x00\x33\xF8\x90\x83\x44\x24\x18\x04\x90\x66\xFF\x4C\x24\x14\x0F\xA4\xCF\x00\x75\xE1" > > return CODE > > def is_branch(instr): > for group in branch_groups: > if group in instr.groups: > return True > return False > #return False > > # Disassemble until we hit basic block end. > def disasm(code): > disasm = Cs(CS_ARCH_X86, CS_MODE_32) > disasm.detail = True > address = 0x401000 > for instr in disasm.disasm(code, address): > print "0x%x:\t%s\t%s" % (instr.address, instr.mnemonic, instr.op_str) > if is_branch(instr): > break > > branch_groups = [CS_GRP_JUMP, CS_GRP_CALL, CS_GRP_RET] > code = get_code() > while True: > disasm(code) > > > That code goes out of memory after a few seconds. The super weird thing > is, that if I change the implementation of "is_branch(instr)" to simply > return False all the time, then the program does not go out of memory! > Does anyone have an idea what's going on? > > Best > Jan > > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > _______________________________________________ > Capstone-users mailing list > Cap...@li... > https://lists.sourceforge.net/lists/listinfo/capstone-users > |
|
From: Jan N. <jan...@ne...> - 2014-12-03 15:33:17
|
Yes, it's using the latest version. OS was win7 x64 running python 2.7 with 32bit libraries. On 12/03/2014 04:31 PM, Nguyen Anh Quynh wrote: > > > On Wed, Dec 3, 2014 at 4:57 PM, Jan Newger <jan...@ne... > <mailto:jan...@ne...>> wrote: > > Hey, > > I was playing around with a few python scripts (using capstone among > other things) and always ran out of memory - and I have no freaking idea > why. > > > is this with the latest 3.0 version? > > thanks, > Q > > > > > The code is really short: > > > from capstone import Cs > from capstone import CS_ARCH_X86 > from capstone import CS_MODE_32 > from capstone import CS_GRP_JUMP > from capstone import CS_GRP_CALL > from capstone import CS_GRP_RET > from capstone.x86_const import X86_INS_JNE, X86_INS_JMP > > """ > 0x401000: push ecx > 0x401001: pop ecx > 0x401002: mov eax, dword ptr [esp + 0x18] > 0x401006: mov eax, dword ptr [eax] > 0x401008: sar eax, 0 > 0x40100b: xor edi, eax > 0x40100d: nop > 0x40100e: add dword ptr [esp + 0x18], 4 > 0x401013: nop > 0x401014: dec word ptr [esp + 0x14] > 0x401019: shld edi, ecx, 0 > 0x40101d: jne 0x401000 > """ > def get_code(): > CODE = > "\x51\x59\x8B\x44\x24\x18\x8B\x00\xC1\xF8\x00\x33\xF8\x90\x83\x44\x24\x18\x04\x90\x66\xFF\x4C\x24\x14\x0F\xA4\xCF\x00\x75\xE1" > > return CODE > > def is_branch(instr): > for group in branch_groups: > if group in instr.groups: > return True > return False > #return False > > # Disassemble until we hit basic block end. > def disasm(code): > disasm = Cs(CS_ARCH_X86, CS_MODE_32) > disasm.detail = True > address = 0x401000 > for instr in disasm.disasm(code, address): > print "0x%x:\t%s\t%s" % (instr.address, instr.mnemonic, > instr.op_str) > if is_branch(instr): > break > > branch_groups = [CS_GRP_JUMP, CS_GRP_CALL, CS_GRP_RET] > code = get_code() > while True: > disasm(code) > > > That code goes out of memory after a few seconds. The super weird thing > is, that if I change the implementation of "is_branch(instr)" to simply > return False all the time, then the program does not go out of memory! > Does anyone have an idea what's going on? > > Best > Jan > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & > more > Get technology previously reserved for billion-dollar corporations, FREE > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > _______________________________________________ > Capstone-users mailing list > Cap...@li... > <mailto:Cap...@li...> > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > > > _______________________________________________ > Capstone-users mailing list > Cap...@li... > https://lists.sourceforge.net/lists/listinfo/capstone-users > |
|
From: Capstone E. <cap...@gm...> - 2014-12-03 15:54:03
|
On Wed, Dec 3, 2014 at 10:32 PM, Jan Newger <jan...@ne...> wrote: > > Yes, it's using the latest version. > OS was win7 x64 running python 2.7 with 32bit libraries. > this is interesting. have you tried to code the same program in C to see if the mem leak issue still happens? thanks. > > > On 12/03/2014 04:31 PM, Nguyen Anh Quynh wrote: > > > > > > On Wed, Dec 3, 2014 at 4:57 PM, Jan Newger <jan...@ne... > > <mailto:jan...@ne...>> wrote: > > > > Hey, > > > > I was playing around with a few python scripts (using capstone among > > other things) and always ran out of memory - and I have no freaking > idea > > why. > > > > > > is this with the latest 3.0 version? > > > > thanks, > > Q > > > > > > > > > > The code is really short: > > > > > > from capstone import Cs > > from capstone import CS_ARCH_X86 > > from capstone import CS_MODE_32 > > from capstone import CS_GRP_JUMP > > from capstone import CS_GRP_CALL > > from capstone import CS_GRP_RET > > from capstone.x86_const import X86_INS_JNE, X86_INS_JMP > > > > """ > > 0x401000: push ecx > > 0x401001: pop ecx > > 0x401002: mov eax, dword ptr [esp + 0x18] > > 0x401006: mov eax, dword ptr [eax] > > 0x401008: sar eax, 0 > > 0x40100b: xor edi, eax > > 0x40100d: nop > > 0x40100e: add dword ptr [esp + 0x18], 4 > > 0x401013: nop > > 0x401014: dec word ptr [esp + 0x14] > > 0x401019: shld edi, ecx, 0 > > 0x40101d: jne 0x401000 > > """ > > def get_code(): > > CODE = > > > "\x51\x59\x8B\x44\x24\x18\x8B\x00\xC1\xF8\x00\x33\xF8\x90\x83\x44\x24\x18\x04\x90\x66\xFF\x4C\x24\x14\x0F\xA4\xCF\x00\x75\xE1" > > > > return CODE > > > > def is_branch(instr): > > for group in branch_groups: > > if group in instr.groups: > > return True > > return False > > #return False > > > > # Disassemble until we hit basic block end. > > def disasm(code): > > disasm = Cs(CS_ARCH_X86, CS_MODE_32) > > disasm.detail = True > > address = 0x401000 > > for instr in disasm.disasm(code, address): > > print "0x%x:\t%s\t%s" % (instr.address, instr.mnemonic, > > instr.op_str) > > if is_branch(instr): > > break > > > > branch_groups = [CS_GRP_JUMP, CS_GRP_CALL, CS_GRP_RET] > > code = get_code() > > while True: > > disasm(code) > > > > > > That code goes out of memory after a few seconds. The super weird > thing > > is, that if I change the implementation of "is_branch(instr)" to > simply > > return False all the time, then the program does not go out of > memory! > > Does anyone have an idea what's going on? > > > > Best > > Jan > > > > > ------------------------------------------------------------------------------ > > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > > from Actuate! Instantly Supercharge Your Business Reports and > Dashboards > > with Interactivity, Sharing, Native Excel Exports, App Integration & > > more > > Get technology previously reserved for billion-dollar corporations, > FREE > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > _______________________________________________ > > Capstone-users mailing list > > Cap...@li... > > <mailto:Cap...@li...> > > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > > > > > > > > > ------------------------------------------------------------------------------ > > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > > with Interactivity, Sharing, Native Excel Exports, App Integration & more > > Get technology previously reserved for billion-dollar corporations, FREE > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > > > > > > > _______________________________________________ > > Capstone-users mailing list > > Cap...@li... > > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > > > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > _______________________________________________ > Capstone-users mailing list > Cap...@li... > https://lists.sourceforge.net/lists/listinfo/capstone-users > |
|
From: Jan N. <jan...@ne...> - 2014-12-03 15:57:54
|
No I haven't tried to reproduce the mem leak with C. It already took me a considerable amount of time to come up with this minimal example. On 12/03/2014 04:53 PM, Capstone Engine wrote: > > > On Wed, Dec 3, 2014 at 10:32 PM, Jan Newger <jan...@ne... > <mailto:jan...@ne...>> wrote: > > > Yes, it's using the latest version. > OS was win7 x64 running python 2.7 with 32bit libraries. > > > this is interesting. have you tried to code the same program in C to see > if the mem leak issue still happens? > > > thanks. > > > > > On 12/03/2014 04:31 PM, Nguyen Anh Quynh wrote: > > > > > > On Wed, Dec 3, 2014 at 4:57 PM, Jan Newger <jan...@ne... <mailto:jan...@ne...> > > <mailto:jan...@ne... <mailto:jan...@ne...>>> wrote: > > > > Hey, > > > > I was playing around with a few python scripts (using > capstone among > > other things) and always ran out of memory - and I have no > freaking idea > > why. > > > > > > is this with the latest 3.0 version? > > > > thanks, > > Q > > > > > > > > > > The code is really short: > > > > > > from capstone import Cs > > from capstone import CS_ARCH_X86 > > from capstone import CS_MODE_32 > > from capstone import CS_GRP_JUMP > > from capstone import CS_GRP_CALL > > from capstone import CS_GRP_RET > > from capstone.x86_const import X86_INS_JNE, X86_INS_JMP > > > > """ > > 0x401000: push ecx > > 0x401001: pop ecx > > 0x401002: mov eax, dword ptr [esp + 0x18] > > 0x401006: mov eax, dword ptr [eax] > > 0x401008: sar eax, 0 > > 0x40100b: xor edi, eax > > 0x40100d: nop > > 0x40100e: add dword ptr [esp + 0x18], 4 > > 0x401013: nop > > 0x401014: dec word ptr [esp + 0x14] > > 0x401019: shld edi, ecx, 0 > > 0x40101d: jne 0x401000 > > """ > > def get_code(): > > CODE = > > > "\x51\x59\x8B\x44\x24\x18\x8B\x00\xC1\xF8\x00\x33\xF8\x90\x83\x44\x24\x18\x04\x90\x66\xFF\x4C\x24\x14\x0F\xA4\xCF\x00\x75\xE1" > > > > return CODE > > > > def is_branch(instr): > > for group in branch_groups: > > if group in instr.groups: > > return True > > return False > > #return False > > > > # Disassemble until we hit basic block end. > > def disasm(code): > > disasm = Cs(CS_ARCH_X86, CS_MODE_32) > > disasm.detail = True > > address = 0x401000 > > for instr in disasm.disasm(code, address): > > print "0x%x:\t%s\t%s" % (instr.address, instr.mnemonic, > > instr.op_str) > > if is_branch(instr): > > break > > > > branch_groups = [CS_GRP_JUMP, CS_GRP_CALL, CS_GRP_RET] > > code = get_code() > > while True: > > disasm(code) > > > > > > That code goes out of memory after a few seconds. The super > weird thing > > is, that if I change the implementation of "is_branch(instr)" > to simply > > return False all the time, then the program does not go out > of memory! > > Does anyone have an idea what's going on? > > > > Best > > Jan > > > > > ------------------------------------------------------------------------------ > > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > > from Actuate! Instantly Supercharge Your Business Reports and > Dashboards > > with Interactivity, Sharing, Native Excel Exports, App > Integration & > > more > > Get technology previously reserved for billion-dollar > corporations, FREE > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > _______________________________________________ > > Capstone-users mailing list > > Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>> > > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > > > > > > > > > ------------------------------------------------------------------------------ > > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > > from Actuate! Instantly Supercharge Your Business Reports and > Dashboards > > with Interactivity, Sharing, Native Excel Exports, App > Integration & more > > Get technology previously reserved for billion-dollar > corporations, FREE > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > > > > > > > _______________________________________________ > > Capstone-users mailing list > > Cap...@li... > <mailto:Cap...@li...> > > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & > more > Get technology previously reserved for billion-dollar corporations, FREE > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > _______________________________________________ > Capstone-users mailing list > Cap...@li... > <mailto:Cap...@li...> > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > > > _______________________________________________ > Capstone-users mailing list > Cap...@li... > https://lists.sourceforge.net/lists/listinfo/capstone-users > |
|
From: Jan N. <jan...@ne...> - 2014-12-03 21:25:33
|
It seems the equivalent C implementation is not affected by the mem leak, which is to be expected, since the memory is explicitly freed anyways, and the group checking boils down to comparing an integer value. If I had to guess, I'd suspect that in the python case the group checking code introduces a spurious reference to the instruction instance(?) which cannot be claimed by the GC. On 03.12.2014 16:57, Jan Newger wrote: > No I haven't tried to reproduce the mem leak with C. > It already took me a considerable amount of time to come up with this > minimal example. > > On 12/03/2014 04:53 PM, Capstone Engine wrote: >> >> >> On Wed, Dec 3, 2014 at 10:32 PM, Jan Newger <jan...@ne... >> <mailto:jan...@ne...>> wrote: >> >> >> Yes, it's using the latest version. >> OS was win7 x64 running python 2.7 with 32bit libraries. >> >> >> this is interesting. have you tried to code the same program in C to see >> if the mem leak issue still happens? >> >> >> thanks. >> >> >> >> >> On 12/03/2014 04:31 PM, Nguyen Anh Quynh wrote: >> > >> > >> > On Wed, Dec 3, 2014 at 4:57 PM, Jan Newger <jan...@ne... <mailto:jan...@ne...> >> > <mailto:jan...@ne... <mailto:jan...@ne...>>> wrote: >> > >> > Hey, >> > >> > I was playing around with a few python scripts (using >> capstone among >> > other things) and always ran out of memory - and I have no >> freaking idea >> > why. >> > >> > >> > is this with the latest 3.0 version? >> > >> > thanks, >> > Q >> > >> > >> > >> > >> > The code is really short: >> > >> > >> > from capstone import Cs >> > from capstone import CS_ARCH_X86 >> > from capstone import CS_MODE_32 >> > from capstone import CS_GRP_JUMP >> > from capstone import CS_GRP_CALL >> > from capstone import CS_GRP_RET >> > from capstone.x86_const import X86_INS_JNE, X86_INS_JMP >> > >> > """ >> > 0x401000: push ecx >> > 0x401001: pop ecx >> > 0x401002: mov eax, dword ptr [esp + 0x18] >> > 0x401006: mov eax, dword ptr [eax] >> > 0x401008: sar eax, 0 >> > 0x40100b: xor edi, eax >> > 0x40100d: nop >> > 0x40100e: add dword ptr [esp + 0x18], 4 >> > 0x401013: nop >> > 0x401014: dec word ptr [esp + 0x14] >> > 0x401019: shld edi, ecx, 0 >> > 0x40101d: jne 0x401000 >> > """ >> > def get_code(): >> > CODE = >> > >> "\x51\x59\x8B\x44\x24\x18\x8B\x00\xC1\xF8\x00\x33\xF8\x90\x83\x44\x24\x18\x04\x90\x66\xFF\x4C\x24\x14\x0F\xA4\xCF\x00\x75\xE1" >> > >> > return CODE >> > >> > def is_branch(instr): >> > for group in branch_groups: >> > if group in instr.groups: >> > return True >> > return False >> > #return False >> > >> > # Disassemble until we hit basic block end. >> > def disasm(code): >> > disasm = Cs(CS_ARCH_X86, CS_MODE_32) >> > disasm.detail = True >> > address = 0x401000 >> > for instr in disasm.disasm(code, address): >> > print "0x%x:\t%s\t%s" % (instr.address, instr.mnemonic, >> > instr.op_str) >> > if is_branch(instr): >> > break >> > >> > branch_groups = [CS_GRP_JUMP, CS_GRP_CALL, CS_GRP_RET] >> > code = get_code() >> > while True: >> > disasm(code) >> > >> > >> > That code goes out of memory after a few seconds. The super >> weird thing >> > is, that if I change the implementation of "is_branch(instr)" >> to simply >> > return False all the time, then the program does not go out >> of memory! >> > Does anyone have an idea what's going on? >> > >> > Best >> > Jan >> > >> > >> ------------------------------------------------------------------------------ >> > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >> > from Actuate! Instantly Supercharge Your Business Reports and >> Dashboards >> > with Interactivity, Sharing, Native Excel Exports, App >> Integration & >> > more >> > Get technology previously reserved for billion-dollar >> corporations, FREE >> > >> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >> > _______________________________________________ >> > Capstone-users mailing list >> > Cap...@li... >> <mailto:Cap...@li...> >> > <mailto:Cap...@li... >> <mailto:Cap...@li...>> >> > https://lists.sourceforge.net/lists/listinfo/capstone-users >> > >> > >> > >> > >> > >> ------------------------------------------------------------------------------ >> > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >> > from Actuate! Instantly Supercharge Your Business Reports and >> Dashboards >> > with Interactivity, Sharing, Native Excel Exports, App >> Integration & more >> > Get technology previously reserved for billion-dollar >> corporations, FREE >> > >> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >> > >> > >> > >> > _______________________________________________ >> > Capstone-users mailing list >> > Cap...@li... >> <mailto:Cap...@li...> >> > https://lists.sourceforge.net/lists/listinfo/capstone-users >> > >> >> >> ------------------------------------------------------------------------------ >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >> from Actuate! Instantly Supercharge Your Business Reports and Dashboards >> with Interactivity, Sharing, Native Excel Exports, App Integration & >> more >> Get technology previously reserved for billion-dollar corporations, FREE >> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >> _______________________________________________ >> Capstone-users mailing list >> Cap...@li... >> <mailto:Cap...@li...> >> https://lists.sourceforge.net/lists/listinfo/capstone-users >> >> >> >> >> ------------------------------------------------------------------------------ >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >> from Actuate! Instantly Supercharge Your Business Reports and Dashboards >> with Interactivity, Sharing, Native Excel Exports, App Integration & more >> Get technology previously reserved for billion-dollar corporations, FREE >> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >> >> >> >> _______________________________________________ >> Capstone-users mailing list >> Cap...@li... >> https://lists.sourceforge.net/lists/listinfo/capstone-users >> > > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > _______________________________________________ > Capstone-users mailing list > Cap...@li... > https://lists.sourceforge.net/lists/listinfo/capstone-users > |
|
From: Jan N. <jan...@ne...> - 2014-12-03 21:43:11
|
This is the python implementation of the disasm function (starting at
line 791):
def disasm(self, code, offset, count=0):
all_insn = ctypes.POINTER(_cs_insn)()
'''if not _python2:
print(code)
code = code.encode()
print(code)'''
res = _cs.cs_disasm(self.csh, code, len(code), offset, count,
ctypes.byref(all_insn))
if res > 0:
for i in range(res):
yield CsInsn(self, all_insn[i])
_cs.cs_free(all_insn, res)
else:
status = _cs.cs_errno(self.csh)
if status != CS_ERR_OK:
raise CsError(status)
return
yield
I'm really no python expert, but from what I see you apparently need to
free the instruction instances manually. However, if client code stops
enumeration over the instructions prematurely, then _cs.cs_free() is
never invoked, and thus memory is leaked, right?
On 03.12.2014 22:25, Jan Newger wrote:
> It seems the equivalent C implementation is not affected by the mem
> leak, which is to be expected, since the memory is explicitly freed
> anyways, and the group checking boils down to comparing an integer value.
>
> If I had to guess, I'd suspect that in the python case the group
> checking code introduces a spurious reference to the instruction
> instance(?) which cannot be claimed by the GC.
>
> On 03.12.2014 16:57, Jan Newger wrote:
>> No I haven't tried to reproduce the mem leak with C.
>> It already took me a considerable amount of time to come up with this
>> minimal example.
>>
>> On 12/03/2014 04:53 PM, Capstone Engine wrote:
>>>
>>>
>>> On Wed, Dec 3, 2014 at 10:32 PM, Jan Newger <jan...@ne...
>>> <mailto:jan...@ne...>> wrote:
>>>
>>>
>>> Yes, it's using the latest version.
>>> OS was win7 x64 running python 2.7 with 32bit libraries.
>>>
>>>
>>> this is interesting. have you tried to code the same program in C to see
>>> if the mem leak issue still happens?
>>>
>>>
>>> thanks.
>>>
>>>
>>>
>>>
>>> On 12/03/2014 04:31 PM, Nguyen Anh Quynh wrote:
>>> >
>>> >
>>> > On Wed, Dec 3, 2014 at 4:57 PM, Jan Newger <jan...@ne... <mailto:jan...@ne...>
>>> > <mailto:jan...@ne... <mailto:jan...@ne...>>> wrote:
>>> >
>>> > Hey,
>>> >
>>> > I was playing around with a few python scripts (using
>>> capstone among
>>> > other things) and always ran out of memory - and I have no
>>> freaking idea
>>> > why.
>>> >
>>> >
>>> > is this with the latest 3.0 version?
>>> >
>>> > thanks,
>>> > Q
>>> >
>>> >
>>> >
>>> >
>>> > The code is really short:
>>> >
>>> >
>>> > from capstone import Cs
>>> > from capstone import CS_ARCH_X86
>>> > from capstone import CS_MODE_32
>>> > from capstone import CS_GRP_JUMP
>>> > from capstone import CS_GRP_CALL
>>> > from capstone import CS_GRP_RET
>>> > from capstone.x86_const import X86_INS_JNE, X86_INS_JMP
>>> >
>>> > """
>>> > 0x401000: push ecx
>>> > 0x401001: pop ecx
>>> > 0x401002: mov eax, dword ptr [esp + 0x18]
>>> > 0x401006: mov eax, dword ptr [eax]
>>> > 0x401008: sar eax, 0
>>> > 0x40100b: xor edi, eax
>>> > 0x40100d: nop
>>> > 0x40100e: add dword ptr [esp + 0x18], 4
>>> > 0x401013: nop
>>> > 0x401014: dec word ptr [esp + 0x14]
>>> > 0x401019: shld edi, ecx, 0
>>> > 0x40101d: jne 0x401000
>>> > """
>>> > def get_code():
>>> > CODE =
>>> >
>>> "\x51\x59\x8B\x44\x24\x18\x8B\x00\xC1\xF8\x00\x33\xF8\x90\x83\x44\x24\x18\x04\x90\x66\xFF\x4C\x24\x14\x0F\xA4\xCF\x00\x75\xE1"
>>> >
>>> > return CODE
>>> >
>>> > def is_branch(instr):
>>> > for group in branch_groups:
>>> > if group in instr.groups:
>>> > return True
>>> > return False
>>> > #return False
>>> >
>>> > # Disassemble until we hit basic block end.
>>> > def disasm(code):
>>> > disasm = Cs(CS_ARCH_X86, CS_MODE_32)
>>> > disasm.detail = True
>>> > address = 0x401000
>>> > for instr in disasm.disasm(code, address):
>>> > print "0x%x:\t%s\t%s" % (instr.address, instr.mnemonic,
>>> > instr.op_str)
>>> > if is_branch(instr):
>>> > break
>>> >
>>> > branch_groups = [CS_GRP_JUMP, CS_GRP_CALL, CS_GRP_RET]
>>> > code = get_code()
>>> > while True:
>>> > disasm(code)
>>> >
>>> >
>>> > That code goes out of memory after a few seconds. The super
>>> weird thing
>>> > is, that if I change the implementation of "is_branch(instr)"
>>> to simply
>>> > return False all the time, then the program does not go out
>>> of memory!
>>> > Does anyone have an idea what's going on?
>>> >
>>> > Best
>>> > Jan
>>> >
>>> >
>>> ------------------------------------------------------------------------------
>>> > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
>>> > from Actuate! Instantly Supercharge Your Business Reports and
>>> Dashboards
>>> > with Interactivity, Sharing, Native Excel Exports, App
>>> Integration &
>>> > more
>>> > Get technology previously reserved for billion-dollar
>>> corporations, FREE
>>> >
>>> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
>>> > _______________________________________________
>>> > Capstone-users mailing list
>>> > Cap...@li...
>>> <mailto:Cap...@li...>
>>> > <mailto:Cap...@li...
>>> <mailto:Cap...@li...>>
>>> > https://lists.sourceforge.net/lists/listinfo/capstone-users
>>> >
>>> >
>>> >
>>> >
>>> >
>>> ------------------------------------------------------------------------------
>>> > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
>>> > from Actuate! Instantly Supercharge Your Business Reports and
>>> Dashboards
>>> > with Interactivity, Sharing, Native Excel Exports, App
>>> Integration & more
>>> > Get technology previously reserved for billion-dollar
>>> corporations, FREE
>>> >
>>> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
>>> >
>>> >
>>> >
>>> > _______________________________________________
>>> > Capstone-users mailing list
>>> > Cap...@li...
>>> <mailto:Cap...@li...>
>>> > https://lists.sourceforge.net/lists/listinfo/capstone-users
>>> >
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
>>> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
>>> with Interactivity, Sharing, Native Excel Exports, App Integration &
>>> more
>>> Get technology previously reserved for billion-dollar corporations, FREE
>>> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
>>> _______________________________________________
>>> Capstone-users mailing list
>>> Cap...@li...
>>> <mailto:Cap...@li...>
>>> https://lists.sourceforge.net/lists/listinfo/capstone-users
>>>
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
>>> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
>>> with Interactivity, Sharing, Native Excel Exports, App Integration & more
>>> Get technology previously reserved for billion-dollar corporations, FREE
>>> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
>>>
>>>
>>>
>>> _______________________________________________
>>> Capstone-users mailing list
>>> Cap...@li...
>>> https://lists.sourceforge.net/lists/listinfo/capstone-users
>>>
>>
>>
>> ------------------------------------------------------------------------------
>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
>> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
>> with Interactivity, Sharing, Native Excel Exports, App Integration & more
>> Get technology previously reserved for billion-dollar corporations, FREE
>> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Capstone-users mailing list
>> Cap...@li...
>> https://lists.sourceforge.net/lists/listinfo/capstone-users
>>
>
>
> ------------------------------------------------------------------------------
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> Get technology previously reserved for billion-dollar corporations, FREE
> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
> _______________________________________________
> Capstone-users mailing list
> Cap...@li...
> https://lists.sourceforge.net/lists/listinfo/capstone-users
>
|
|
From: Jay O. <ja...@ko...> - 2014-12-03 22:41:25
|
The _cs.cs_free call can be made within the special __del__ method (a destructor), but this is only safe in a reference-counting garbage collector (like Python) when there are no circular references. See documentation for more info: https://docs.python.org/2/reference/datamodel.html#object.__del__ This will fix the leak by freeing memory when the generator goes out of scope. Which, in your particular case, is an acceptable solution. I would recommend the GC not be relied upon for freeing memory, but use it instead as a safety net. On Wed, Dec 3, 2014 at 1:42 PM, Jan Newger <jan...@ne...> wrote: > This is the python implementation of the disasm function (starting at > line 791): > > def disasm(self, code, offset, count=0): > all_insn = ctypes.POINTER(_cs_insn)() > '''if not _python2: > print(code) > code = code.encode() > print(code)''' > res = _cs.cs_disasm(self.csh, code, len(code), offset, count, > ctypes.byref(all_insn)) > if res > 0: > for i in range(res): > yield CsInsn(self, all_insn[i]) > _cs.cs_free(all_insn, res) > else: > status = _cs.cs_errno(self.csh) > if status != CS_ERR_OK: > raise CsError(status) > return > yield > > I'm really no python expert, but from what I see you apparently need to > free the instruction instances manually. However, if client code stops > enumeration over the instructions prematurely, then _cs.cs_free() is > never invoked, and thus memory is leaked, right? > > On 03.12.2014 22:25, Jan Newger wrote: > > It seems the equivalent C implementation is not affected by the mem > > leak, which is to be expected, since the memory is explicitly freed > > anyways, and the group checking boils down to comparing an integer value. > > > > If I had to guess, I'd suspect that in the python case the group > > checking code introduces a spurious reference to the instruction > > instance(?) which cannot be claimed by the GC. > > > > On 03.12.2014 16:57, Jan Newger wrote: > >> No I haven't tried to reproduce the mem leak with C. > >> It already took me a considerable amount of time to come up with this > >> minimal example. > >> > >> On 12/03/2014 04:53 PM, Capstone Engine wrote: > >>> > >>> > >>> On Wed, Dec 3, 2014 at 10:32 PM, Jan Newger <jan...@ne... > >>> <mailto:jan...@ne...>> wrote: > >>> > >>> > >>> Yes, it's using the latest version. > >>> OS was win7 x64 running python 2.7 with 32bit libraries. > >>> > >>> > >>> this is interesting. have you tried to code the same program in C to > see > >>> if the mem leak issue still happens? > >>> > >>> > >>> thanks. > >>> > >>> > >>> > >>> > >>> On 12/03/2014 04:31 PM, Nguyen Anh Quynh wrote: > >>> > > >>> > > >>> > On Wed, Dec 3, 2014 at 4:57 PM, Jan Newger < > jan...@ne... <mailto:jan...@ne...> > >>> > <mailto:jan...@ne... <mailto:jan...@ne...>>> > wrote: > >>> > > >>> > Hey, > >>> > > >>> > I was playing around with a few python scripts (using > >>> capstone among > >>> > other things) and always ran out of memory - and I have no > >>> freaking idea > >>> > why. > >>> > > >>> > > >>> > is this with the latest 3.0 version? > >>> > > >>> > thanks, > >>> > Q > >>> > > >>> > > >>> > > >>> > > >>> > The code is really short: > >>> > > >>> > > >>> > from capstone import Cs > >>> > from capstone import CS_ARCH_X86 > >>> > from capstone import CS_MODE_32 > >>> > from capstone import CS_GRP_JUMP > >>> > from capstone import CS_GRP_CALL > >>> > from capstone import CS_GRP_RET > >>> > from capstone.x86_const import X86_INS_JNE, X86_INS_JMP > >>> > > >>> > """ > >>> > 0x401000: push ecx > >>> > 0x401001: pop ecx > >>> > 0x401002: mov eax, dword ptr [esp + 0x18] > >>> > 0x401006: mov eax, dword ptr [eax] > >>> > 0x401008: sar eax, 0 > >>> > 0x40100b: xor edi, eax > >>> > 0x40100d: nop > >>> > 0x40100e: add dword ptr [esp + 0x18], 4 > >>> > 0x401013: nop > >>> > 0x401014: dec word ptr [esp + 0x14] > >>> > 0x401019: shld edi, ecx, 0 > >>> > 0x40101d: jne 0x401000 > >>> > """ > >>> > def get_code(): > >>> > CODE = > >>> > > >>> > "\x51\x59\x8B\x44\x24\x18\x8B\x00\xC1\xF8\x00\x33\xF8\x90\x83\x44\x24\x18\x04\x90\x66\xFF\x4C\x24\x14\x0F\xA4\xCF\x00\x75\xE1" > >>> > > >>> > return CODE > >>> > > >>> > def is_branch(instr): > >>> > for group in branch_groups: > >>> > if group in instr.groups: > >>> > return True > >>> > return False > >>> > #return False > >>> > > >>> > # Disassemble until we hit basic block end. > >>> > def disasm(code): > >>> > disasm = Cs(CS_ARCH_X86, CS_MODE_32) > >>> > disasm.detail = True > >>> > address = 0x401000 > >>> > for instr in disasm.disasm(code, address): > >>> > print "0x%x:\t%s\t%s" % (instr.address, > instr.mnemonic, > >>> > instr.op_str) > >>> > if is_branch(instr): > >>> > break > >>> > > >>> > branch_groups = [CS_GRP_JUMP, CS_GRP_CALL, CS_GRP_RET] > >>> > code = get_code() > >>> > while True: > >>> > disasm(code) > >>> > > >>> > > >>> > That code goes out of memory after a few seconds. The super > >>> weird thing > >>> > is, that if I change the implementation of > "is_branch(instr)" > >>> to simply > >>> > return False all the time, then the program does not go out > >>> of memory! > >>> > Does anyone have an idea what's going on? > >>> > > >>> > Best > >>> > Jan > >>> > > >>> > > >>> > ------------------------------------------------------------------------------ > >>> > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT > Server > >>> > from Actuate! Instantly Supercharge Your Business Reports > and > >>> Dashboards > >>> > with Interactivity, Sharing, Native Excel Exports, App > >>> Integration & > >>> > more > >>> > Get technology previously reserved for billion-dollar > >>> corporations, FREE > >>> > > >>> > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > >>> > _______________________________________________ > >>> > Capstone-users mailing list > >>> > Cap...@li... > >>> <mailto:Cap...@li...> > >>> > <mailto:Cap...@li... > >>> <mailto:Cap...@li...>> > >>> > https://lists.sourceforge.net/lists/listinfo/capstone-users > >>> > > >>> > > >>> > > >>> > > >>> > > >>> > ------------------------------------------------------------------------------ > >>> > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT > Server > >>> > from Actuate! Instantly Supercharge Your Business Reports and > >>> Dashboards > >>> > with Interactivity, Sharing, Native Excel Exports, App > >>> Integration & more > >>> > Get technology previously reserved for billion-dollar > >>> corporations, FREE > >>> > > >>> > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > >>> > > >>> > > >>> > > >>> > _______________________________________________ > >>> > Capstone-users mailing list > >>> > Cap...@li... > >>> <mailto:Cap...@li...> > >>> > https://lists.sourceforge.net/lists/listinfo/capstone-users > >>> > > >>> > >>> > >>> > ------------------------------------------------------------------------------ > >>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > >>> from Actuate! Instantly Supercharge Your Business Reports and > Dashboards > >>> with Interactivity, Sharing, Native Excel Exports, App Integration > & > >>> more > >>> Get technology previously reserved for billion-dollar > corporations, FREE > >>> > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > >>> _______________________________________________ > >>> Capstone-users mailing list > >>> Cap...@li... > >>> <mailto:Cap...@li...> > >>> https://lists.sourceforge.net/lists/listinfo/capstone-users > >>> > >>> > >>> > >>> > >>> > ------------------------------------------------------------------------------ > >>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > >>> from Actuate! Instantly Supercharge Your Business Reports and > Dashboards > >>> with Interactivity, Sharing, Native Excel Exports, App Integration & > more > >>> Get technology previously reserved for billion-dollar corporations, > FREE > >>> > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > >>> > >>> > >>> > >>> _______________________________________________ > >>> Capstone-users mailing list > >>> Cap...@li... > >>> https://lists.sourceforge.net/lists/listinfo/capstone-users > >>> > >> > >> > >> > ------------------------------------------------------------------------------ > >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > >> from Actuate! Instantly Supercharge Your Business Reports and Dashboards > >> with Interactivity, Sharing, Native Excel Exports, App Integration & > more > >> Get technology previously reserved for billion-dollar corporations, FREE > >> > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > >> _______________________________________________ > >> Capstone-users mailing list > >> Cap...@li... > >> https://lists.sourceforge.net/lists/listinfo/capstone-users > >> > > > > > > > ------------------------------------------------------------------------------ > > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > > with Interactivity, Sharing, Native Excel Exports, App Integration & more > > Get technology previously reserved for billion-dollar corporations, FREE > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > _______________________________________________ > > Capstone-users mailing list > > Cap...@li... > > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > > > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > _______________________________________________ > Capstone-users mailing list > Cap...@li... > https://lists.sourceforge.net/lists/listinfo/capstone-users > |
|
From: Nguyen A. Q. <aq...@gm...> - 2014-12-03 23:28:16
|
On Thu, Dec 4, 2014 at 4:42 AM, Jan Newger <jan...@ne...> wrote: > This is the python implementation of the disasm function (starting at > line 791): > > def disasm(self, code, offset, count=0): > all_insn = ctypes.POINTER(_cs_insn)() > '''if not _python2: > print(code) > code = code.encode() > print(code)''' > res = _cs.cs_disasm(self.csh, code, len(code), offset, count, > ctypes.byref(all_insn)) > if res > 0: > for i in range(res): > yield CsInsn(self, all_insn[i]) > _cs.cs_free(all_insn, res) > else: > status = _cs.cs_errno(self.csh) > if status != CS_ERR_OK: > raise CsError(status) > return > yield > > I'm really no python expert, but from what I see you apparently need to > free the instruction instances manually. However, if client code stops > enumeration over the instructions prematurely, then _cs.cs_free() is > never invoked, and thus memory is leaked, right? > yes the problem must be with Python binding but not the core. however, in the above code, cs_free() is called after the "for" loop, so i dont see how memleak can happen "prematurely". thanks. > On 03.12.2014 22:25, Jan Newger wrote: > > It seems the equivalent C implementation is not affected by the mem > > leak, which is to be expected, since the memory is explicitly freed > > anyways, and the group checking boils down to comparing an integer value. > > > > If I had to guess, I'd suspect that in the python case the group > > checking code introduces a spurious reference to the instruction > > instance(?) which cannot be claimed by the GC. > > > > On 03.12.2014 16:57, Jan Newger wrote: > >> No I haven't tried to reproduce the mem leak with C. > >> It already took me a considerable amount of time to come up with this > >> minimal example. > >> > >> On 12/03/2014 04:53 PM, Capstone Engine wrote: > >>> > >>> > >>> On Wed, Dec 3, 2014 at 10:32 PM, Jan Newger <jan...@ne... > >>> <mailto:jan...@ne...>> wrote: > >>> > >>> > >>> Yes, it's using the latest version. > >>> OS was win7 x64 running python 2.7 with 32bit libraries. > >>> > >>> > >>> this is interesting. have you tried to code the same program in C to > see > >>> if the mem leak issue still happens? > >>> > >>> > >>> thanks. > >>> > >>> > >>> > >>> > >>> On 12/03/2014 04:31 PM, Nguyen Anh Quynh wrote: > >>> > > >>> > > >>> > On Wed, Dec 3, 2014 at 4:57 PM, Jan Newger < > jan...@ne... <mailto:jan...@ne...> > >>> > <mailto:jan...@ne... <mailto:jan...@ne...>>> > wrote: > >>> > > >>> > Hey, > >>> > > >>> > I was playing around with a few python scripts (using > >>> capstone among > >>> > other things) and always ran out of memory - and I have no > >>> freaking idea > >>> > why. > >>> > > >>> > > >>> > is this with the latest 3.0 version? > >>> > > >>> > thanks, > >>> > Q > >>> > > >>> > > >>> > > >>> > > >>> > The code is really short: > >>> > > >>> > > >>> > from capstone import Cs > >>> > from capstone import CS_ARCH_X86 > >>> > from capstone import CS_MODE_32 > >>> > from capstone import CS_GRP_JUMP > >>> > from capstone import CS_GRP_CALL > >>> > from capstone import CS_GRP_RET > >>> > from capstone.x86_const import X86_INS_JNE, X86_INS_JMP > >>> > > >>> > """ > >>> > 0x401000: push ecx > >>> > 0x401001: pop ecx > >>> > 0x401002: mov eax, dword ptr [esp + 0x18] > >>> > 0x401006: mov eax, dword ptr [eax] > >>> > 0x401008: sar eax, 0 > >>> > 0x40100b: xor edi, eax > >>> > 0x40100d: nop > >>> > 0x40100e: add dword ptr [esp + 0x18], 4 > >>> > 0x401013: nop > >>> > 0x401014: dec word ptr [esp + 0x14] > >>> > 0x401019: shld edi, ecx, 0 > >>> > 0x40101d: jne 0x401000 > >>> > """ > >>> > def get_code(): > >>> > CODE = > >>> > > >>> > "\x51\x59\x8B\x44\x24\x18\x8B\x00\xC1\xF8\x00\x33\xF8\x90\x83\x44\x24\x18\x04\x90\x66\xFF\x4C\x24\x14\x0F\xA4\xCF\x00\x75\xE1" > >>> > > >>> > return CODE > >>> > > >>> > def is_branch(instr): > >>> > for group in branch_groups: > >>> > if group in instr.groups: > >>> > return True > >>> > return False > >>> > #return False > >>> > > >>> > # Disassemble until we hit basic block end. > >>> > def disasm(code): > >>> > disasm = Cs(CS_ARCH_X86, CS_MODE_32) > >>> > disasm.detail = True > >>> > address = 0x401000 > >>> > for instr in disasm.disasm(code, address): > >>> > print "0x%x:\t%s\t%s" % (instr.address, > instr.mnemonic, > >>> > instr.op_str) > >>> > if is_branch(instr): > >>> > break > >>> > > >>> > branch_groups = [CS_GRP_JUMP, CS_GRP_CALL, CS_GRP_RET] > >>> > code = get_code() > >>> > while True: > >>> > disasm(code) > >>> > > >>> > > >>> > That code goes out of memory after a few seconds. The super > >>> weird thing > >>> > is, that if I change the implementation of > "is_branch(instr)" > >>> to simply > >>> > return False all the time, then the program does not go out > >>> of memory! > >>> > Does anyone have an idea what's going on? > >>> > > >>> > Best > >>> > Jan > >>> > > >>> > > >>> > ------------------------------------------------------------------------------ > >>> > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT > Server > >>> > from Actuate! Instantly Supercharge Your Business Reports > and > >>> Dashboards > >>> > with Interactivity, Sharing, Native Excel Exports, App > >>> Integration & > >>> > more > >>> > Get technology previously reserved for billion-dollar > >>> corporations, FREE > >>> > > >>> > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > >>> > _______________________________________________ > >>> > Capstone-users mailing list > >>> > Cap...@li... > >>> <mailto:Cap...@li...> > >>> > <mailto:Cap...@li... > >>> <mailto:Cap...@li...>> > >>> > https://lists.sourceforge.net/lists/listinfo/capstone-users > >>> > > >>> > > >>> > > >>> > > >>> > > >>> > ------------------------------------------------------------------------------ > >>> > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT > Server > >>> > from Actuate! Instantly Supercharge Your Business Reports and > >>> Dashboards > >>> > with Interactivity, Sharing, Native Excel Exports, App > >>> Integration & more > >>> > Get technology previously reserved for billion-dollar > >>> corporations, FREE > >>> > > >>> > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > >>> > > >>> > > >>> > > >>> > _______________________________________________ > >>> > Capstone-users mailing list > >>> > Cap...@li... > >>> <mailto:Cap...@li...> > >>> > https://lists.sourceforge.net/lists/listinfo/capstone-users > >>> > > >>> > >>> > >>> > ------------------------------------------------------------------------------ > >>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > >>> from Actuate! Instantly Supercharge Your Business Reports and > Dashboards > >>> with Interactivity, Sharing, Native Excel Exports, App Integration > & > >>> more > >>> Get technology previously reserved for billion-dollar > corporations, FREE > >>> > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > >>> _______________________________________________ > >>> Capstone-users mailing list > >>> Cap...@li... > >>> <mailto:Cap...@li...> > >>> https://lists.sourceforge.net/lists/listinfo/capstone-users > >>> > >>> > >>> > >>> > >>> > ------------------------------------------------------------------------------ > >>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > >>> from Actuate! Instantly Supercharge Your Business Reports and > Dashboards > >>> with Interactivity, Sharing, Native Excel Exports, App Integration & > more > >>> Get technology previously reserved for billion-dollar corporations, > FREE > >>> > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > >>> > >>> > >>> > >>> _______________________________________________ > >>> Capstone-users mailing list > >>> Cap...@li... > >>> https://lists.sourceforge.net/lists/listinfo/capstone-users > >>> > >> > >> > >> > ------------------------------------------------------------------------------ > >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > >> from Actuate! Instantly Supercharge Your Business Reports and Dashboards > >> with Interactivity, Sharing, Native Excel Exports, App Integration & > more > >> Get technology previously reserved for billion-dollar corporations, FREE > >> > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > >> _______________________________________________ > >> Capstone-users mailing list > >> Cap...@li... > >> https://lists.sourceforge.net/lists/listinfo/capstone-users > >> > > > > > > > ------------------------------------------------------------------------------ > > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > > with Interactivity, Sharing, Native Excel Exports, App Integration & more > > Get technology previously reserved for billion-dollar corporations, FREE > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > _______________________________________________ > > Capstone-users mailing list > > Cap...@li... > > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > > > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > _______________________________________________ > Capstone-users mailing list > Cap...@li... > https://lists.sourceforge.net/lists/listinfo/capstone-users > |
|
From: Jurriaan B. <jur...@gm...> - 2014-12-03 23:33:00
|
The 'yield' keyword allows one to fetch new records/rows (in this case disassembled instructions) on-demand. Doing an early exit (e.g., quitting from a for-loop iterating over a function that yield's after only, say, 2 out of 10 items) will prematurely exit the function as well. Because, of course, why would Python calculate the latter 8 results when they're not used in the first place? This is also useful for never-ending functions - I suppose calculating digits of Pi would be a 'good' example. Anyway, so your cs_free() call is never reached in this case - you should switch to cs_free()'ing every row after each iteration through the for loop. Regards, Jurriaan On 12/04/2014 12:27 AM, Nguyen Anh Quynh wrote: > > > On Thu, Dec 4, 2014 at 4:42 AM, Jan Newger <jan...@ne... > <mailto:jan...@ne...>> wrote: > > This is the python implementation of the disasm function (starting at > line 791): > > def disasm(self, code, offset, count=0): > all_insn = ctypes.POINTER(_cs_insn)() > '''if not _python2: > print(code) > code = code.encode() > print(code)''' > res = _cs.cs_disasm(self.csh, code, len(code), offset, count, > ctypes.byref(all_insn)) > if res > 0: > for i in range(res): > yield CsInsn(self, all_insn[i]) > _cs.cs_free(all_insn, res) > else: > status = _cs.cs_errno(self.csh) > if status != CS_ERR_OK: > raise CsError(status) > return > yield > > I'm really no python expert, but from what I see you apparently need to > free the instruction instances manually. However, if client code stops > enumeration over the instructions prematurely, then _cs.cs_free() is > never invoked, and thus memory is leaked, right? > > > yes the problem must be with Python binding but not the core. > however, in the above code, cs_free() is called after the "for" loop, > so i dont see how memleak can happen "prematurely". > > > thanks. > > > > > > On 03.12.2014 22:25, Jan Newger wrote: > > It seems the equivalent C implementation is not affected by the mem > > leak, which is to be expected, since the memory is explicitly freed > > anyways, and the group checking boils down to comparing an integer > value. > > > > If I had to guess, I'd suspect that in the python case the group > > checking code introduces a spurious reference to the instruction > > instance(?) which cannot be claimed by the GC. > > > > On 03.12.2014 16:57, Jan Newger wrote: > >> No I haven't tried to reproduce the mem leak with C. > >> It already took me a considerable amount of time to come up with this > >> minimal example. > >> > >> On 12/03/2014 04:53 PM, Capstone Engine wrote: > >>> > >>> > >>> On Wed, Dec 3, 2014 at 10:32 PM, Jan Newger > <jan...@ne... <mailto:jan...@ne...> > >>> <mailto:jan...@ne... <mailto:jan...@ne...>>> > wrote: > >>> > >>> > >>> Yes, it's using the latest version. > >>> OS was win7 x64 running python 2.7 with 32bit libraries. > >>> > >>> > >>> this is interesting. have you tried to code the same program in > C to see > >>> if the mem leak issue still happens? > >>> > >>> > >>> thanks. > >>> > >>> > >>> > >>> > >>> On 12/03/2014 04:31 PM, Nguyen Anh Quynh wrote: > >>> > > >>> > > >>> > On Wed, Dec 3, 2014 at 4:57 PM, Jan Newger > <jan...@ne... <mailto:jan...@ne...> > <mailto:jan...@ne... <mailto:jan...@ne...>> > >>> > <mailto:jan...@ne... > <mailto:jan...@ne...> <mailto:jan...@ne... > <mailto:jan...@ne...>>>> wrote: > >>> > > >>> > Hey, > >>> > > >>> > I was playing around with a few python scripts (using > >>> capstone among > >>> > other things) and always ran out of memory - and I > have no > >>> freaking idea > >>> > why. > >>> > > >>> > > >>> > is this with the latest 3.0 version? > >>> > > >>> > thanks, > >>> > Q > >>> > > >>> > > >>> > > >>> > > >>> > The code is really short: > >>> > > >>> > > >>> > from capstone import Cs > >>> > from capstone import CS_ARCH_X86 > >>> > from capstone import CS_MODE_32 > >>> > from capstone import CS_GRP_JUMP > >>> > from capstone import CS_GRP_CALL > >>> > from capstone import CS_GRP_RET > >>> > from capstone.x86_const import X86_INS_JNE, X86_INS_JMP > >>> > > >>> > """ > >>> > 0x401000: push ecx > >>> > 0x401001: pop ecx > >>> > 0x401002: mov eax, dword ptr [esp + 0x18] > >>> > 0x401006: mov eax, dword ptr [eax] > >>> > 0x401008: sar eax, 0 > >>> > 0x40100b: xor edi, eax > >>> > 0x40100d: nop > >>> > 0x40100e: add dword ptr [esp + 0x18], 4 > >>> > 0x401013: nop > >>> > 0x401014: dec word ptr [esp + 0x14] > >>> > 0x401019: shld edi, ecx, 0 > >>> > 0x40101d: jne 0x401000 > >>> > """ > >>> > def get_code(): > >>> > CODE = > >>> > > >>> > "\x51\x59\x8B\x44\x24\x18\x8B\x00\xC1\xF8\x00\x33\xF8\x90\x83\x44\x24\x18\x04\x90\x66\xFF\x4C\x24\x14\x0F\xA4\xCF\x00\x75\xE1" > >>> > > >>> > return CODE > >>> > > >>> > def is_branch(instr): > >>> > for group in branch_groups: > >>> > if group in instr.groups: > >>> > return True > >>> > return False > >>> > #return False > >>> > > >>> > # Disassemble until we hit basic block end. > >>> > def disasm(code): > >>> > disasm = Cs(CS_ARCH_X86, CS_MODE_32) > >>> > disasm.detail = True > >>> > address = 0x401000 > >>> > for instr in disasm.disasm(code, address): > >>> > print "0x%x:\t%s\t%s" % (instr.address, > instr.mnemonic, > >>> > instr.op_str) > >>> > if is_branch(instr): > >>> > break > >>> > > >>> > branch_groups = [CS_GRP_JUMP, CS_GRP_CALL, CS_GRP_RET] > >>> > code = get_code() > >>> > while True: > >>> > disasm(code) > >>> > > >>> > > >>> > That code goes out of memory after a few seconds. The > super > >>> weird thing > >>> > is, that if I change the implementation of > "is_branch(instr)" > >>> to simply > >>> > return False all the time, then the program does not > go out > >>> of memory! > >>> > Does anyone have an idea what's going on? > >>> > > >>> > Best > >>> > Jan > >>> > > >>> > > >>> > ------------------------------------------------------------------------------ > >>> > Download BIRT iHub F-Type - The Free Enterprise-Grade > BIRT Server > >>> > from Actuate! Instantly Supercharge Your Business > Reports and > >>> Dashboards > >>> > with Interactivity, Sharing, Native Excel Exports, App > >>> Integration & > >>> > more > >>> > Get technology previously reserved for billion-dollar > >>> corporations, FREE > >>> > > >>> > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > >>> > _______________________________________________ > >>> > Capstone-users mailing list > >>> > Cap...@li... > <mailto:Cap...@li...> > >>> <mailto:Cap...@li... > <mailto:Cap...@li...>> > >>> > <mailto:Cap...@li... > <mailto:Cap...@li...> > >>> <mailto:Cap...@li... > <mailto:Cap...@li...>>> > >>> > https://lists.sourceforge.net/lists/listinfo/capstone-users > >>> > > >>> > > >>> > > >>> > > >>> > > >>> > ------------------------------------------------------------------------------ > >>> > Download BIRT iHub F-Type - The Free Enterprise-Grade > BIRT Server > >>> > from Actuate! Instantly Supercharge Your Business Reports and > >>> Dashboards > >>> > with Interactivity, Sharing, Native Excel Exports, App > >>> Integration & more > >>> > Get technology previously reserved for billion-dollar > >>> corporations, FREE > >>> > > >>> > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > >>> > > >>> > > >>> > > >>> > _______________________________________________ > >>> > Capstone-users mailing list > >>> > Cap...@li... > <mailto:Cap...@li...> > >>> <mailto:Cap...@li... > <mailto:Cap...@li...>> > >>> > https://lists.sourceforge.net/lists/listinfo/capstone-users > >>> > > >>> > >>> > >>> > ------------------------------------------------------------------------------ > >>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT > Server > >>> from Actuate! Instantly Supercharge Your Business Reports > and Dashboards > >>> with Interactivity, Sharing, Native Excel Exports, App > Integration & > >>> more > >>> Get technology previously reserved for billion-dollar > corporations, FREE > >>> > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > >>> _______________________________________________ > >>> Capstone-users mailing list > >>> Cap...@li... > <mailto:Cap...@li...> > >>> <mailto:Cap...@li... > <mailto:Cap...@li...>> > >>> https://lists.sourceforge.net/lists/listinfo/capstone-users > >>> > >>> > >>> > >>> > >>> > ------------------------------------------------------------------------------ > >>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > >>> from Actuate! Instantly Supercharge Your Business Reports and > Dashboards > >>> with Interactivity, Sharing, Native Excel Exports, App > Integration & more > >>> Get technology previously reserved for billion-dollar > corporations, FREE > >>> > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > >>> > >>> > >>> > >>> _______________________________________________ > >>> Capstone-users mailing list > >>> Cap...@li... > <mailto:Cap...@li...> > >>> https://lists.sourceforge.net/lists/listinfo/capstone-users > >>> > >> > >> > >> > ------------------------------------------------------------------------------ > >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > >> from Actuate! Instantly Supercharge Your Business Reports and > Dashboards > >> with Interactivity, Sharing, Native Excel Exports, App > Integration & more > >> Get technology previously reserved for billion-dollar > corporations, FREE > >> > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > >> _______________________________________________ > >> Capstone-users mailing list > >> Cap...@li... > <mailto:Cap...@li...> > >> https://lists.sourceforge.net/lists/listinfo/capstone-users > >> > > > > > > > ------------------------------------------------------------------------------ > > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > > from Actuate! Instantly Supercharge Your Business Reports and > Dashboards > > with Interactivity, Sharing, Native Excel Exports, App Integration > & more > > Get technology previously reserved for billion-dollar > corporations, FREE > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > _______________________________________________ > > Capstone-users mailing list > > Cap...@li... > <mailto:Cap...@li...> > > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & > more > Get technology previously reserved for billion-dollar corporations, FREE > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > _______________________________________________ > Capstone-users mailing list > Cap...@li... > <mailto:Cap...@li...> > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > > > _______________________________________________ > Capstone-users mailing list > Cap...@li... > https://lists.sourceforge.net/lists/listinfo/capstone-users > |
|
From: Nguyen A. Q. <aq...@gm...> - 2014-12-03 23:50:22
|
On Thu, Dec 4, 2014 at 6:32 AM, Jurriaan Bremer <jur...@gm...> wrote: > The 'yield' keyword allows one to fetch new records/rows (in this case > disassembled instructions) on-demand. Doing an early exit (e.g., > quitting from a for-loop iterating over a function that yield's after > only, say, 2 out of 10 items) will prematurely exit the function as > well. Because, of course, why would Python calculate the latter 8 > results when they're not used in the first place? This is also useful > for never-ending functions - I suppose calculating digits of Pi would be > a 'good' example. > > Anyway, so your cs_free() call is never reached in this case - you > should switch to cs_free()'ing every row after each iteration through > the for loop. > > but the loop is called upon the number of successfully disassembled instructions, so there is no where in the code that might possibly quit the loop prematurely. thanks, Q On 12/04/2014 12:27 AM, Nguyen Anh Quynh wrote: > > > On Thu, Dec 4, 2014 at 4:42 AM, Jan Newger <jan...@ne... > <mailto:jan...@ne...>> wrote: > > This is the python implementation of the disasm function (starting at > line 791): > > def disasm(self, code, offset, count=0): > all_insn = ctypes.POINTER(_cs_insn)() > '''if not _python2: > print(code) > code = code.encode() > print(code)''' > res = _cs.cs_disasm(self.csh, code, len(code), offset, count, > ctypes.byref(all_insn)) > if res > 0: > for i in range(res): > yield CsInsn(self, all_insn[i]) > _cs.cs_free(all_insn, res) > else: > status = _cs.cs_errno(self.csh) > if status != CS_ERR_OK: > raise CsError(status) > return > yield > > I'm really no python expert, but from what I see you apparently need to > free the instruction instances manually. However, if client code stops > enumeration over the instructions prematurely, then _cs.cs_free() is > never invoked, and thus memory is leaked, right? > > > yes the problem must be with Python binding but not the core. > however, in the above code, cs_free() is called after the "for" loop, > so i dont see how memleak can happen "prematurely". > > > thanks. > > > > > > On 03.12.2014 22:25, Jan Newger wrote: > > It seems the equivalent C implementation is not affected by the mem > > leak, which is to be expected, since the memory is explicitly freed > > anyways, and the group checking boils down to comparing an integer > value. > > > > If I had to guess, I'd suspect that in the python case the group > > checking code introduces a spurious reference to the instruction > > instance(?) which cannot be claimed by the GC. > > > > On 03.12.2014 16:57, Jan Newger wrote: > >> No I haven't tried to reproduce the mem leak with C. > >> It already took me a considerable amount of time to come up with this > >> minimal example. > >> > >> On 12/03/2014 04:53 PM, Capstone Engine wrote: > >>> > >>> > >>> On Wed, Dec 3, 2014 at 10:32 PM, Jan Newger > <jan...@ne... <mailto:jan...@ne...> > >>> <mailto:jan...@ne... <mailto:jan...@ne...>>> > wrote: > >>> > >>> > >>> Yes, it's using the latest version. > >>> OS was win7 x64 running python 2.7 with 32bit libraries. > >>> > >>> > >>> this is interesting. have you tried to code the same program in > C to see > >>> if the mem leak issue still happens? > >>> > >>> > >>> thanks. > >>> > >>> > >>> > >>> > >>> On 12/03/2014 04:31 PM, Nguyen Anh Quynh wrote: > >>> > > >>> > > >>> > On Wed, Dec 3, 2014 at 4:57 PM, Jan Newger > <jan...@ne... <mailto:jan...@ne...> > <mailto:jan...@ne... <mailto:jan...@ne...>> > >>> > <mailto:jan...@ne... > <mailto:jan...@ne...> <mailto:jan...@ne... > <mailto:jan...@ne...>>>> wrote: > >>> > > >>> > Hey, > >>> > > >>> > I was playing around with a few python scripts (using > >>> capstone among > >>> > other things) and always ran out of memory - and I > have no > >>> freaking idea > >>> > why. > >>> > > >>> > > >>> > is this with the latest 3.0 version? > >>> > > >>> > thanks, > >>> > Q > >>> > > >>> > > >>> > > >>> > > >>> > The code is really short: > >>> > > >>> > > >>> > from capstone import Cs > >>> > from capstone import CS_ARCH_X86 > >>> > from capstone import CS_MODE_32 > >>> > from capstone import CS_GRP_JUMP > >>> > from capstone import CS_GRP_CALL > >>> > from capstone import CS_GRP_RET > >>> > from capstone.x86_const import X86_INS_JNE, X86_INS_JMP > >>> > > >>> > """ > >>> > 0x401000: push ecx > >>> > 0x401001: pop ecx > >>> > 0x401002: mov eax, dword ptr [esp + 0x18] > >>> > 0x401006: mov eax, dword ptr [eax] > >>> > 0x401008: sar eax, 0 > >>> > 0x40100b: xor edi, eax > >>> > 0x40100d: nop > >>> > 0x40100e: add dword ptr [esp + 0x18], 4 > >>> > 0x401013: nop > >>> > 0x401014: dec word ptr [esp + 0x14] > >>> > 0x401019: shld edi, ecx, 0 > >>> > 0x40101d: jne 0x401000 > >>> > """ > >>> > def get_code(): > >>> > CODE = > >>> > > >>> > "\x51\x59\x8B\x44\x24\x18\x8B\x00\xC1\xF8\x00\x33\xF8\x90\x83\x44\x24\x18\x04\x90\x66\xFF\x4C\x24\x14\x0F\xA4\xCF\x00\x75\xE1" > >>> > > >>> > return CODE > >>> > > >>> > def is_branch(instr): > >>> > for group in branch_groups: > >>> > if group in instr.groups: > >>> > return True > >>> > return False > >>> > #return False > >>> > > >>> > # Disassemble until we hit basic block end. > >>> > def disasm(code): > >>> > disasm = Cs(CS_ARCH_X86, CS_MODE_32) > >>> > disasm.detail = True > >>> > address = 0x401000 > >>> > for instr in disasm.disasm(code, address): > >>> > print "0x%x:\t%s\t%s" % (instr.address, > instr.mnemonic, > >>> > instr.op_str) > >>> > if is_branch(instr): > >>> > break > >>> > > >>> > branch_groups = [CS_GRP_JUMP, CS_GRP_CALL, CS_GRP_RET] > >>> > code = get_code() > >>> > while True: > >>> > disasm(code) > >>> > > >>> > > >>> > That code goes out of memory after a few seconds. The > super > >>> weird thing > >>> > is, that if I change the implementation of > "is_branch(instr)" > >>> to simply > >>> > return False all the time, then the program does not > go out > >>> of memory! > >>> > Does anyone have an idea what's going on? > >>> > > >>> > Best > >>> > Jan > >>> > > >>> > > >>> > ------------------------------------------------------------------------------ > >>> > Download BIRT iHub F-Type - The Free Enterprise-Grade > BIRT Server > >>> > from Actuate! Instantly Supercharge Your Business > Reports and > >>> Dashboards > >>> > with Interactivity, Sharing, Native Excel Exports, App > >>> Integration & > >>> > more > >>> > Get technology previously reserved for billion-dollar > >>> corporations, FREE > >>> > > >>> > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > >>> > _______________________________________________ > >>> > Capstone-users mailing list > >>> > Cap...@li... > <mailto:Cap...@li...> > >>> <mailto:Cap...@li... > <mailto:Cap...@li...>> > >>> > <mailto:Cap...@li... > <mailto:Cap...@li...> > >>> <mailto:Cap...@li... > <mailto:Cap...@li...>>> > >>> > https://lists.sourceforge.net/lists/listinfo/capstone-users > >>> > > >>> > > >>> > > >>> > > >>> > > >>> > ------------------------------------------------------------------------------ > >>> > Download BIRT iHub F-Type - The Free Enterprise-Grade > BIRT Server > >>> > from Actuate! Instantly Supercharge Your Business Reports and > >>> Dashboards > >>> > with Interactivity, Sharing, Native Excel Exports, App > >>> Integration & more > >>> > Get technology previously reserved for billion-dollar > >>> corporations, FREE > >>> > > >>> > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > >>> > > >>> > > >>> > > >>> > _______________________________________________ > >>> > Capstone-users mailing list > >>> > Cap...@li... > <mailto:Cap...@li...> > >>> <mailto:Cap...@li... > <mailto:Cap...@li...>> > >>> > https://lists.sourceforge.net/lists/listinfo/capstone-users > >>> > > >>> > >>> > >>> > ------------------------------------------------------------------------------ > >>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT > Server > >>> from Actuate! Instantly Supercharge Your Business Reports > and Dashboards > >>> with Interactivity, Sharing, Native Excel Exports, App > Integration & > >>> more > >>> Get technology previously reserved for billion-dollar > corporations, FREE > >>> > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > >>> _______________________________________________ > >>> Capstone-users mailing list > >>> Cap...@li... > <mailto:Cap...@li...> > >>> <mailto:Cap...@li... > <mailto:Cap...@li...>> > >>> https://lists.sourceforge.net/lists/listinfo/capstone-users > >>> > >>> > >>> > >>> > >>> > ------------------------------------------------------------------------------ > >>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > >>> from Actuate! Instantly Supercharge Your Business Reports and > Dashboards > >>> with Interactivity, Sharing, Native Excel Exports, App > Integration & more > >>> Get technology previously reserved for billion-dollar > corporations, FREE > >>> > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > >>> > >>> > >>> > >>> _______________________________________________ > >>> Capstone-users mailing list > >>> Cap...@li... > <mailto:Cap...@li...> > >>> https://lists.sourceforge.net/lists/listinfo/capstone-users > >>> > >> > >> > >> > ------------------------------------------------------------------------------ > >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > >> from Actuate! Instantly Supercharge Your Business Reports and > Dashboards > >> with Interactivity, Sharing, Native Excel Exports, App > Integration & more > >> Get technology previously reserved for billion-dollar > corporations, FREE > >> > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > >> _______________________________________________ > >> Capstone-users mailing list > >> Cap...@li... > <mailto:Cap...@li...> > >> https://lists.sourceforge.net/lists/listinfo/capstone-users > >> > > > > > > > ------------------------------------------------------------------------------ > > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > > from Actuate! Instantly Supercharge Your Business Reports and > Dashboards > > with Interactivity, Sharing, Native Excel Exports, App Integration > & more > > Get technology previously reserved for billion-dollar > corporations, FREE > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > _______________________________________________ > > Capstone-users mailing list > > Cap...@li... > <mailto:Cap...@li...> > > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & > more > Get technology previously reserved for billion-dollar corporations, FREE > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > _______________________________________________ > Capstone-users mailing list > Cap...@li... > <mailto:Cap...@li...> > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > > > _______________________________________________ > Capstone-users mailing list > Cap...@li... > https://lists.sourceforge.net/lists/listinfo/capstone-users > ------------------------------------------------------------------------------ Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server from Actuate! Instantly Supercharge Your Business Reports and Dashboards with Interactivity, Sharing, Native Excel Exports, App Integration & more Get technology previously reserved for billion-dollar corporations, FREE http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk _______________________________________________ Capstone-users mailing list Cap...@li... https://lists.sourceforge.net/lists/listinfo/capstone-users |
|
From: Jasiel S. <jas...@gm...> - 2014-12-04 00:08:04
|
Nguyen,
Here is example code that hopefully better illustrates Juriaan's point/what
Jan is seeing:
def gen_example():
print '[gen_example] START'
for i in xrange(2):
print '[gen_example] before yield: ', i
yield i
print '[gen_example] after yield: ', i
print '[gen_example] END'
print "Typical case"
for i in gen_example():
print
print "Jan's case"
for i in gen_example():
if i == 1:
break
print
Typical case
[gen_example] START
[gen_example] before yield: 0
[gen_example] after yield: 0
[gen_example] before yield: 1
[gen_example] after yield: 1
[gen_example] END
Jan's case
[gen_example] START
[gen_example] before yield: 0
[gen_example] after yield: 0
[gen_example] before yield: 1
On Wed, Dec 3, 2014 at 5:49 PM, Nguyen Anh Quynh <aq...@gm...> wrote:
>
>
> On Thu, Dec 4, 2014 at 6:32 AM, Jurriaan Bremer <jur...@gm...>
> wrote:
>
>> The 'yield' keyword allows one to fetch new records/rows (in this case
>> disassembled instructions) on-demand. Doing an early exit (e.g.,
>> quitting from a for-loop iterating over a function that yield's after
>> only, say, 2 out of 10 items) will prematurely exit the function as
>> well. Because, of course, why would Python calculate the latter 8
>> results when they're not used in the first place? This is also useful
>> for never-ending functions - I suppose calculating digits of Pi would be
>> a 'good' example.
>>
>> Anyway, so your cs_free() call is never reached in this case - you
>> should switch to cs_free()'ing every row after each iteration through
>> the for loop.
>>
>>
> but the loop is called upon the number of successfully disassembled
> instructions,
> so there is no where in the code that might possibly quit the loop
> prematurely.
>
> thanks,
>
> Q
>
> On 12/04/2014 12:27 AM, Nguyen Anh Quynh wrote:
> >
> >
> > On Thu, Dec 4, 2014 at 4:42 AM, Jan Newger <jan...@ne...
> > <mailto:jan...@ne...>> wrote:
> >
> > This is the python implementation of the disasm function (starting at
> > line 791):
> >
> > def disasm(self, code, offset, count=0):
> > all_insn = ctypes.POINTER(_cs_insn)()
> > '''if not _python2:
> > print(code)
> > code = code.encode()
> > print(code)'''
> > res = _cs.cs_disasm(self.csh, code, len(code), offset, count,
> > ctypes.byref(all_insn))
> > if res > 0:
> > for i in range(res):
> > yield CsInsn(self, all_insn[i])
> > _cs.cs_free(all_insn, res)
> > else:
> > status = _cs.cs_errno(self.csh)
> > if status != CS_ERR_OK:
> > raise CsError(status)
> > return
> > yield
> >
> > I'm really no python expert, but from what I see you apparently need
> to
> > free the instruction instances manually. However, if client code
> stops
> > enumeration over the instructions prematurely, then _cs.cs_free() is
> > never invoked, and thus memory is leaked, right?
> >
> >
> > yes the problem must be with Python binding but not the core.
> > however, in the above code, cs_free() is called after the "for" loop,
> > so i dont see how memleak can happen "prematurely".
> >
> >
> > thanks.
> >
> >
> >
> >
> >
> > On 03.12.2014 22:25, Jan Newger wrote:
> > > It seems the equivalent C implementation is not affected by the mem
> > > leak, which is to be expected, since the memory is explicitly freed
> > > anyways, and the group checking boils down to comparing an integer
> > value.
> > >
> > > If I had to guess, I'd suspect that in the python case the group
> > > checking code introduces a spurious reference to the instruction
> > > instance(?) which cannot be claimed by the GC.
> > >
> > > On 03.12.2014 16:57, Jan Newger wrote:
> > >> No I haven't tried to reproduce the mem leak with C.
> > >> It already took me a considerable amount of time to come up with
> this
> > >> minimal example.
> > >>
> > >> On 12/03/2014 04:53 PM, Capstone Engine wrote:
> > >>>
> > >>>
> > >>> On Wed, Dec 3, 2014 at 10:32 PM, Jan Newger
> > <jan...@ne... <mailto:jan...@ne...>
> > >>> <mailto:jan...@ne... <mailto:jan...@ne...>>>
> > wrote:
> > >>>
> > >>>
> > >>> Yes, it's using the latest version.
> > >>> OS was win7 x64 running python 2.7 with 32bit libraries.
> > >>>
> > >>>
> > >>> this is interesting. have you tried to code the same program in
> > C to see
> > >>> if the mem leak issue still happens?
> > >>>
> > >>>
> > >>> thanks.
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> On 12/03/2014 04:31 PM, Nguyen Anh Quynh wrote:
> > >>> >
> > >>> >
> > >>> > On Wed, Dec 3, 2014 at 4:57 PM, Jan Newger
> > <jan...@ne... <mailto:jan...@ne...>
> > <mailto:jan...@ne... <mailto:jan...@ne...>>
> > >>> > <mailto:jan...@ne...
> > <mailto:jan...@ne...> <mailto:jan...@ne...
> > <mailto:jan...@ne...>>>> wrote:
> > >>> >
> > >>> > Hey,
> > >>> >
> > >>> > I was playing around with a few python scripts (using
> > >>> capstone among
> > >>> > other things) and always ran out of memory - and I
> > have no
> > >>> freaking idea
> > >>> > why.
> > >>> >
> > >>> >
> > >>> > is this with the latest 3.0 version?
> > >>> >
> > >>> > thanks,
> > >>> > Q
> > >>> >
> > >>> >
> > >>> >
> > >>> >
> > >>> > The code is really short:
> > >>> >
> > >>> >
> > >>> > from capstone import Cs
> > >>> > from capstone import CS_ARCH_X86
> > >>> > from capstone import CS_MODE_32
> > >>> > from capstone import CS_GRP_JUMP
> > >>> > from capstone import CS_GRP_CALL
> > >>> > from capstone import CS_GRP_RET
> > >>> > from capstone.x86_const import X86_INS_JNE,
> X86_INS_JMP
> > >>> >
> > >>> > """
> > >>> > 0x401000: push ecx
> > >>> > 0x401001: pop ecx
> > >>> > 0x401002: mov eax, dword ptr [esp + 0x18]
> > >>> > 0x401006: mov eax, dword ptr [eax]
> > >>> > 0x401008: sar eax, 0
> > >>> > 0x40100b: xor edi, eax
> > >>> > 0x40100d: nop
> > >>> > 0x40100e: add dword ptr [esp + 0x18], 4
> > >>> > 0x401013: nop
> > >>> > 0x401014: dec word ptr [esp + 0x14]
> > >>> > 0x401019: shld edi, ecx, 0
> > >>> > 0x40101d: jne 0x401000
> > >>> > """
> > >>> > def get_code():
> > >>> > CODE =
> > >>> >
> > >>>
> >
> "\x51\x59\x8B\x44\x24\x18\x8B\x00\xC1\xF8\x00\x33\xF8\x90\x83\x44\x24\x18\x04\x90\x66\xFF\x4C\x24\x14\x0F\xA4\xCF\x00\x75\xE1"
> > >>> >
> > >>> > return CODE
> > >>> >
> > >>> > def is_branch(instr):
> > >>> > for group in branch_groups:
> > >>> > if group in instr.groups:
> > >>> > return True
> > >>> > return False
> > >>> > #return False
> > >>> >
> > >>> > # Disassemble until we hit basic block end.
> > >>> > def disasm(code):
> > >>> > disasm = Cs(CS_ARCH_X86, CS_MODE_32)
> > >>> > disasm.detail = True
> > >>> > address = 0x401000
> > >>> > for instr in disasm.disasm(code, address):
> > >>> > print "0x%x:\t%s\t%s" % (instr.address,
> > instr.mnemonic,
> > >>> > instr.op_str)
> > >>> > if is_branch(instr):
> > >>> > break
> > >>> >
> > >>> > branch_groups = [CS_GRP_JUMP, CS_GRP_CALL, CS_GRP_RET]
> > >>> > code = get_code()
> > >>> > while True:
> > >>> > disasm(code)
> > >>> >
> > >>> >
> > >>> > That code goes out of memory after a few seconds. The
> > super
> > >>> weird thing
> > >>> > is, that if I change the implementation of
> > "is_branch(instr)"
> > >>> to simply
> > >>> > return False all the time, then the program does not
> > go out
> > >>> of memory!
> > >>> > Does anyone have an idea what's going on?
> > >>> >
> > >>> > Best
> > >>> > Jan
> > >>> >
> > >>> >
> > >>>
> >
> ------------------------------------------------------------------------------
> > >>> > Download BIRT iHub F-Type - The Free Enterprise-Grade
> > BIRT Server
> > >>> > from Actuate! Instantly Supercharge Your Business
> > Reports and
> > >>> Dashboards
> > >>> > with Interactivity, Sharing, Native Excel Exports, App
> > >>> Integration &
> > >>> > more
> > >>> > Get technology previously reserved for billion-dollar
> > >>> corporations, FREE
> > >>> >
> > >>>
> >
> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
> > >>> > _______________________________________________
> > >>> > Capstone-users mailing list
> > >>> > Cap...@li...
> > <mailto:Cap...@li...>
> > >>> <mailto:Cap...@li...
> > <mailto:Cap...@li...>>
> > >>> > <mailto:Cap...@li...
> > <mailto:Cap...@li...>
> > >>> <mailto:Cap...@li...
> > <mailto:Cap...@li...>>>
> > >>> >
> https://lists.sourceforge.net/lists/listinfo/capstone-users
> > >>> >
> > >>> >
> > >>> >
> > >>> >
> > >>> >
> > >>>
> >
> ------------------------------------------------------------------------------
> > >>> > Download BIRT iHub F-Type - The Free Enterprise-Grade
> > BIRT Server
> > >>> > from Actuate! Instantly Supercharge Your Business Reports
> and
> > >>> Dashboards
> > >>> > with Interactivity, Sharing, Native Excel Exports, App
> > >>> Integration & more
> > >>> > Get technology previously reserved for billion-dollar
> > >>> corporations, FREE
> > >>> >
> > >>>
> >
> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
> > >>> >
> > >>> >
> > >>> >
> > >>> > _______________________________________________
> > >>> > Capstone-users mailing list
> > >>> > Cap...@li...
> > <mailto:Cap...@li...>
> > >>> <mailto:Cap...@li...
> > <mailto:Cap...@li...>>
> > >>> >
> https://lists.sourceforge.net/lists/listinfo/capstone-users
> > >>> >
> > >>>
> > >>>
> > >>>
> >
> ------------------------------------------------------------------------------
> > >>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT
> > Server
> > >>> from Actuate! Instantly Supercharge Your Business Reports
> > and Dashboards
> > >>> with Interactivity, Sharing, Native Excel Exports, App
> > Integration &
> > >>> more
> > >>> Get technology previously reserved for billion-dollar
> > corporations, FREE
> > >>>
> >
> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
> > >>> _______________________________________________
> > >>> Capstone-users mailing list
> > >>> Cap...@li...
> > <mailto:Cap...@li...>
> > >>> <mailto:Cap...@li...
> > <mailto:Cap...@li...>>
> > >>> https://lists.sourceforge.net/lists/listinfo/capstone-users
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> >
> ------------------------------------------------------------------------------
> > >>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> > >>> from Actuate! Instantly Supercharge Your Business Reports and
> > Dashboards
> > >>> with Interactivity, Sharing, Native Excel Exports, App
> > Integration & more
> > >>> Get technology previously reserved for billion-dollar
> > corporations, FREE
> > >>>
> >
> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
> > >>>
> > >>>
> > >>>
> > >>> _______________________________________________
> > >>> Capstone-users mailing list
> > >>> Cap...@li...
> > <mailto:Cap...@li...>
> > >>> https://lists.sourceforge.net/lists/listinfo/capstone-users
> > >>>
> > >>
> > >>
> > >>
> >
> ------------------------------------------------------------------------------
> > >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> > >> from Actuate! Instantly Supercharge Your Business Reports and
> > Dashboards
> > >> with Interactivity, Sharing, Native Excel Exports, App
> > Integration & more
> > >> Get technology previously reserved for billion-dollar
> > corporations, FREE
> > >>
> >
> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
> > >> _______________________________________________
> > >> Capstone-users mailing list
> > >> Cap...@li...
> > <mailto:Cap...@li...>
> > >> https://lists.sourceforge.net/lists/listinfo/capstone-users
> > >>
> > >
> > >
> > >
> >
> ------------------------------------------------------------------------------
> > > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> > > from Actuate! Instantly Supercharge Your Business Reports and
> > Dashboards
> > > with Interactivity, Sharing, Native Excel Exports, App Integration
> > & more
> > > Get technology previously reserved for billion-dollar
> > corporations, FREE
> > >
> >
> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
> > > _______________________________________________
> > > Capstone-users mailing list
> > > Cap...@li...
> > <mailto:Cap...@li...>
> > > https://lists.sourceforge.net/lists/listinfo/capstone-users
> > >
> >
> >
> >
> ------------------------------------------------------------------------------
> > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> > from Actuate! Instantly Supercharge Your Business Reports and
> Dashboards
> > with Interactivity, Sharing, Native Excel Exports, App Integration &
> > more
> > Get technology previously reserved for billion-dollar corporations,
> FREE
> >
> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
> > _______________________________________________
> > Capstone-users mailing list
> > Cap...@li...
> > <mailto:Cap...@li...>
> > https://lists.sourceforge.net/lists/listinfo/capstone-users
> >
> >
> >
> >
> >
> ------------------------------------------------------------------------------
> > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> > from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> > with Interactivity, Sharing, Native Excel Exports, App Integration & more
> > Get technology previously reserved for billion-dollar corporations, FREE
> >
> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
> >
> >
> >
> > _______________________________________________
> > Capstone-users mailing list
> > Cap...@li...
> > https://lists.sourceforge.net/lists/listinfo/capstone-users
> >
>
>
> ------------------------------------------------------------------------------
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> Get technology previously reserved for billion-dollar corporations, FREE
>
> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
> _______________________________________________
> Capstone-users mailing list
> Cap...@li...
> https://lists.sourceforge.net/lists/listinfo/capstone-users
>
>
>
> ------------------------------------------------------------------------------
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> Get technology previously reserved for billion-dollar corporations, FREE
>
> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
> _______________________________________________
> Capstone-users mailing list
> Cap...@li...
> https://lists.sourceforge.net/lists/listinfo/capstone-users
>
>
|
|
From: Jan N. <jan...@ne...> - 2014-12-04 13:01:53
|
I think the easiest fix would be to free the memory from a finally clause, as described here: https://docs.python.org/2/reference/simple_stmts.html#the-yield-statement Like this (untested): def disasm(self, code, offset, count=0): all_insn = ctypes.POINTER(_cs_insn)() '''if not _python2: print(code) code = code.encode() print(code)''' res = _cs.cs_disasm(self.csh, code, len(code), offset, count, ctypes.byref(all_insn)) if res > 0: try: for i in range(res): yield CsInsn(self, all_insn[i]) finally: _cs.cs_free(all_insn, res) else: status = _cs.cs_errno(self.csh) if status != CS_ERR_OK: raise CsError(status) return yield On 12/04/2014 01:07 AM, Jasiel Spelman wrote: > Nguyen, > > Here is example code that hopefully better illustrates Juriaan's > point/what Jan is seeing: > > def gen_example(): > print '[gen_example] START' > for i in xrange(2): > print '[gen_example] before yield: ', i > yield i > print '[gen_example] after yield: ', i > print '[gen_example] END' > > print "Typical case" > for i in gen_example(): > print > > print "Jan's case" > for i in gen_example(): > if i == 1: > break > print > > > Typical case > [gen_example] START > [gen_example] before yield: 0 > > [gen_example] after yield: 0 > [gen_example] before yield: 1 > > [gen_example] after yield: 1 > [gen_example] END > > > Jan's case > [gen_example] START > [gen_example] before yield: 0 > > [gen_example] after yield: 0 > [gen_example] before yield: 1 > > > On Wed, Dec 3, 2014 at 5:49 PM, Nguyen Anh Quynh <aq...@gm... > <mailto:aq...@gm...>> wrote: > > > > On Thu, Dec 4, 2014 at 6:32 AM, Jurriaan Bremer > <jur...@gm... <mailto:jur...@gm...>> wrote: > > The 'yield' keyword allows one to fetch new records/rows (in > this case > disassembled instructions) on-demand. Doing an early exit (e.g., > quitting from a for-loop iterating over a function that yield's > after > only, say, 2 out of 10 items) will prematurely exit the function as > well. Because, of course, why would Python calculate the latter 8 > results when they're not used in the first place? This is also > useful > for never-ending functions - I suppose calculating digits of Pi > would be > a 'good' example. > > Anyway, so your cs_free() call is never reached in this case - you > should switch to cs_free()'ing every row after each iteration > through > the for loop. > > > but the loop is called upon the number of successfully disassembled > instructions, > so there is no where in the code that might possibly quit the loop > prematurely. > > thanks, > > Q > > On 12/04/2014 12:27 AM, Nguyen Anh Quynh wrote: > > > > > > On Thu, Dec 4, 2014 at 4:42 AM, Jan Newger <jan...@ne... <mailto:jan...@ne...> > > <mailto:jan...@ne... <mailto:jan...@ne...>>> wrote: > > > > This is the python implementation of the disasm function > (starting at > > line 791): > > > > def disasm(self, code, offset, count=0): > > all_insn = ctypes.POINTER(_cs_insn)() > > '''if not _python2: > > print(code) > > code = code.encode() > > print(code)''' > > res = _cs.cs_disasm(self.csh, code, len(code), > offset, count, > > ctypes.byref(all_insn)) > > if res > 0: > > for i in range(res): > > yield CsInsn(self, all_insn[i]) > > _cs.cs_free(all_insn, res) > > else: > > status = _cs.cs_errno(self.csh) > > if status != CS_ERR_OK: > > raise CsError(status) > > return > > yield > > > > I'm really no python expert, but from what I see you > apparently need to > > free the instruction instances manually. However, if client > code stops > > enumeration over the instructions prematurely, then > _cs.cs_free() is > > never invoked, and thus memory is leaked, right? > > > > > > yes the problem must be with Python binding but not the core. > > however, in the above code, cs_free() is called after the "for" loop, > > so i dont see how memleak can happen "prematurely". > > > > > > thanks. > > > > > > > > > > > > On 03.12.2014 22:25, Jan Newger wrote: > > > It seems the equivalent C implementation is not affected by > the mem > > > leak, which is to be expected, since the memory is > explicitly freed > > > anyways, and the group checking boils down to comparing an > integer > > value. > > > > > > If I had to guess, I'd suspect that in the python case the > group > > > checking code introduces a spurious reference to the > instruction > > > instance(?) which cannot be claimed by the GC. > > > > > > On 03.12.2014 16:57, Jan Newger wrote: > > >> No I haven't tried to reproduce the mem leak with C. > > >> It already took me a considerable amount of time to come > up with this > > >> minimal example. > > >> > > >> On 12/03/2014 04:53 PM, Capstone Engine wrote: > > >>> > > >>> > > >>> On Wed, Dec 3, 2014 at 10:32 PM, Jan Newger > > <jan...@ne... <mailto:jan...@ne...> > <mailto:jan...@ne... <mailto:jan...@ne...>> > > >>> <mailto:jan...@ne... > <mailto:jan...@ne...> <mailto:jan...@ne... > <mailto:jan...@ne...>>>> > > wrote: > > >>> > > >>> > > >>> Yes, it's using the latest version. > > >>> OS was win7 x64 running python 2.7 with 32bit libraries. > > >>> > > >>> > > >>> this is interesting. have you tried to code the same > program in > > C to see > > >>> if the mem leak issue still happens? > > >>> > > >>> > > >>> thanks. > > >>> > > >>> > > >>> > > >>> > > >>> On 12/03/2014 04:31 PM, Nguyen Anh Quynh wrote: > > >>> > > > >>> > > > >>> > On Wed, Dec 3, 2014 at 4:57 PM, Jan Newger > > <jan...@ne... <mailto:jan...@ne...> > <mailto:jan...@ne... <mailto:jan...@ne...>> > > <mailto:jan...@ne... <mailto:jan...@ne...> > <mailto:jan...@ne... <mailto:jan...@ne...>>> > > >>> > <mailto:jan...@ne... > <mailto:jan...@ne...> > > <mailto:jan...@ne... <mailto:jan...@ne...>> > <mailto:jan...@ne... <mailto:jan...@ne...> > > <mailto:jan...@ne... > <mailto:jan...@ne...>>>>> wrote: > > >>> > > > >>> > Hey, > > >>> > > > >>> > I was playing around with a few python scripts > (using > > >>> capstone among > > >>> > other things) and always ran out of memory - and I > > have no > > >>> freaking idea > > >>> > why. > > >>> > > > >>> > > > >>> > is this with the latest 3.0 version? > > >>> > > > >>> > thanks, > > >>> > Q > > >>> > > > >>> > > > >>> > > > >>> > > > >>> > The code is really short: > > >>> > > > >>> > > > >>> > from capstone import Cs > > >>> > from capstone import CS_ARCH_X86 > > >>> > from capstone import CS_MODE_32 > > >>> > from capstone import CS_GRP_JUMP > > >>> > from capstone import CS_GRP_CALL > > >>> > from capstone import CS_GRP_RET > > >>> > from capstone.x86_const import X86_INS_JNE, > X86_INS_JMP > > >>> > > > >>> > """ > > >>> > 0x401000: push ecx > > >>> > 0x401001: pop ecx > > >>> > 0x401002: mov eax, dword ptr [esp + > 0x18] > > >>> > 0x401006: mov eax, dword ptr [eax] > > >>> > 0x401008: sar eax, 0 > > >>> > 0x40100b: xor edi, eax > > >>> > 0x40100d: nop > > >>> > 0x40100e: add dword ptr [esp + 0x18], 4 > > >>> > 0x401013: nop > > >>> > 0x401014: dec word ptr [esp + 0x14] > > >>> > 0x401019: shld edi, ecx, 0 > > >>> > 0x40101d: jne 0x401000 > > >>> > """ > > >>> > def get_code(): > > >>> > CODE = > > >>> > > > >>> > > > "\x51\x59\x8B\x44\x24\x18\x8B\x00\xC1\xF8\x00\x33\xF8\x90\x83\x44\x24\x18\x04\x90\x66\xFF\x4C\x24\x14\x0F\xA4\xCF\x00\x75\xE1" > > >>> > > > >>> > return CODE > > >>> > > > >>> > def is_branch(instr): > > >>> > for group in branch_groups: > > >>> > if group in instr.groups: > > >>> > return True > > >>> > return False > > >>> > #return False > > >>> > > > >>> > # Disassemble until we hit basic block end. > > >>> > def disasm(code): > > >>> > disasm = Cs(CS_ARCH_X86, CS_MODE_32) > > >>> > disasm.detail = True > > >>> > address = 0x401000 > > >>> > for instr in disasm.disasm(code, address): > > >>> > print "0x%x:\t%s\t%s" % (instr.address, > > instr.mnemonic, > > >>> > instr.op_str) > > >>> > if is_branch(instr): > > >>> > break > > >>> > > > >>> > branch_groups = [CS_GRP_JUMP, CS_GRP_CALL, > CS_GRP_RET] > > >>> > code = get_code() > > >>> > while True: > > >>> > disasm(code) > > >>> > > > >>> > > > >>> > That code goes out of memory after a few > seconds. The > > super > > >>> weird thing > > >>> > is, that if I change the implementation of > > "is_branch(instr)" > > >>> to simply > > >>> > return False all the time, then the program > does not > > go out > > >>> of memory! > > >>> > Does anyone have an idea what's going on? > > >>> > > > >>> > Best > > >>> > Jan > > >>> > > > >>> > > > >>> > > > ------------------------------------------------------------------------------ > > >>> > Download BIRT iHub F-Type - The Free > Enterprise-Grade > > BIRT Server > > >>> > from Actuate! Instantly Supercharge Your Business > > Reports and > > >>> Dashboards > > >>> > with Interactivity, Sharing, Native Excel > Exports, App > > >>> Integration & > > >>> > more > > >>> > Get technology previously reserved for > billion-dollar > > >>> corporations, FREE > > >>> > > > >>> > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > >>> > _______________________________________________ > > >>> > Capstone-users mailing list > > >>> > Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>> > > >>> <mailto:Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>>> > > >>> > <mailto:Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>> > > >>> <mailto:Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>>>> > > >>> > > https://lists.sourceforge.net/lists/listinfo/capstone-users > > >>> > > > >>> > > > >>> > > > >>> > > > >>> > > > >>> > > > ------------------------------------------------------------------------------ > > >>> > Download BIRT iHub F-Type - The Free Enterprise-Grade > > BIRT Server > > >>> > from Actuate! Instantly Supercharge Your Business > Reports and > > >>> Dashboards > > >>> > with Interactivity, Sharing, Native Excel Exports, App > > >>> Integration & more > > >>> > Get technology previously reserved for billion-dollar > > >>> corporations, FREE > > >>> > > > >>> > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > >>> > > > >>> > > > >>> > > > >>> > _______________________________________________ > > >>> > Capstone-users mailing list > > >>> > Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>> > > >>> <mailto:Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>>> > > >>> > > https://lists.sourceforge.net/lists/listinfo/capstone-users > > >>> > > > >>> > > >>> > > >>> > > > ------------------------------------------------------------------------------ > > >>> Download BIRT iHub F-Type - The Free Enterprise-Grade > BIRT > > Server > > >>> from Actuate! Instantly Supercharge Your Business Reports > > and Dashboards > > >>> with Interactivity, Sharing, Native Excel Exports, App > > Integration & > > >>> more > > >>> Get technology previously reserved for billion-dollar > > corporations, FREE > > >>> > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > >>> _______________________________________________ > > >>> Capstone-users mailing list > > >>> Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>> > > >>> <mailto:Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>>> > > >>> https://lists.sourceforge.net/lists/listinfo/capstone-users > > >>> > > >>> > > >>> > > >>> > > >>> > > > ------------------------------------------------------------------------------ > > >>> Download BIRT iHub F-Type - The Free Enterprise-Grade > BIRT Server > > >>> from Actuate! Instantly Supercharge Your Business Reports and > > Dashboards > > >>> with Interactivity, Sharing, Native Excel Exports, App > > Integration & more > > >>> Get technology previously reserved for billion-dollar > > corporations, FREE > > >>> > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > >>> > > >>> > > >>> > > >>> _______________________________________________ > > >>> Capstone-users mailing list > > >>> Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>> > > >>> https://lists.sourceforge.net/lists/listinfo/capstone-users > > >>> > > >> > > >> > > >> > > > ------------------------------------------------------------------------------ > > >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT > Server > > >> from Actuate! Instantly Supercharge Your Business Reports and > > Dashboards > > >> with Interactivity, Sharing, Native Excel Exports, App > > Integration & more > > >> Get technology previously reserved for billion-dollar > > corporations, FREE > > >> > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > >> _______________________________________________ > > >> Capstone-users mailing list > > >> Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>> > > >> https://lists.sourceforge.net/lists/listinfo/capstone-users > > >> > > > > > > > > > > > > ------------------------------------------------------------------------------ > > > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT > Server > > > from Actuate! Instantly Supercharge Your Business Reports and > > Dashboards > > > with Interactivity, Sharing, Native Excel Exports, App > Integration > > & more > > > Get technology previously reserved for billion-dollar > > corporations, FREE > > > > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > > _______________________________________________ > > > Capstone-users mailing list > > > Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>> > > > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > > > > > > > > ------------------------------------------------------------------------------ > > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > > from Actuate! Instantly Supercharge Your Business Reports and > Dashboards > > with Interactivity, Sharing, Native Excel Exports, App > Integration & > > more > > Get technology previously reserved for billion-dollar > corporations, FREE > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > _______________________________________________ > > Capstone-users mailing list > > Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>> > > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > > > > > > > > > ------------------------------------------------------------------------------ > > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > > from Actuate! Instantly Supercharge Your Business Reports and > Dashboards > > with Interactivity, Sharing, Native Excel Exports, App > Integration & more > > Get technology previously reserved for billion-dollar > corporations, FREE > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > > > > > > > _______________________________________________ > > Capstone-users mailing list > > Cap...@li... > <mailto:Cap...@li...> > > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & > more > Get technology previously reserved for billion-dollar corporations, FREE > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > _______________________________________________ > Capstone-users mailing list > Cap...@li... > <mailto:Cap...@li...> > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & > more > Get technology previously reserved for billion-dollar corporations, FREE > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > _______________________________________________ > Capstone-users mailing list > Cap...@li... > <mailto:Cap...@li...> > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > > > _______________________________________________ > Capstone-users mailing list > Cap...@li... > https://lists.sourceforge.net/lists/listinfo/capstone-users > |
|
From: Jurriaan B. <jur...@gm...> - 2014-12-04 13:04:16
|
I was unaware of this approach - nice! +1 for using this as a patch ;) Jurriaan On 12/04/2014 02:01 PM, Jan Newger wrote: > I think the easiest fix would be to free the memory from a finally > clause, as described here: > > https://docs.python.org/2/reference/simple_stmts.html#the-yield-statement > > Like this (untested): > > def disasm(self, code, offset, count=0): > all_insn = ctypes.POINTER(_cs_insn)() > '''if not _python2: > print(code) > code = code.encode() > print(code)''' > res = _cs.cs_disasm(self.csh, code, len(code), offset, count, > ctypes.byref(all_insn)) > if res > 0: > try: > for i in range(res): > yield CsInsn(self, all_insn[i]) > finally: > _cs.cs_free(all_insn, res) > else: > status = _cs.cs_errno(self.csh) > if status != CS_ERR_OK: > raise CsError(status) > return > yield > > On 12/04/2014 01:07 AM, Jasiel Spelman wrote: >> Nguyen, >> >> Here is example code that hopefully better illustrates Juriaan's >> point/what Jan is seeing: >> >> def gen_example(): >> print '[gen_example] START' >> for i in xrange(2): >> print '[gen_example] before yield: ', i >> yield i >> print '[gen_example] after yield: ', i >> print '[gen_example] END' >> >> print "Typical case" >> for i in gen_example(): >> print >> >> print "Jan's case" >> for i in gen_example(): >> if i == 1: >> break >> print >> >> >> Typical case >> [gen_example] START >> [gen_example] before yield: 0 >> >> [gen_example] after yield: 0 >> [gen_example] before yield: 1 >> >> [gen_example] after yield: 1 >> [gen_example] END >> >> >> Jan's case >> [gen_example] START >> [gen_example] before yield: 0 >> >> [gen_example] after yield: 0 >> [gen_example] before yield: 1 >> >> >> On Wed, Dec 3, 2014 at 5:49 PM, Nguyen Anh Quynh <aq...@gm... >> <mailto:aq...@gm...>> wrote: >> >> >> >> On Thu, Dec 4, 2014 at 6:32 AM, Jurriaan Bremer >> <jur...@gm... <mailto:jur...@gm...>> wrote: >> >> The 'yield' keyword allows one to fetch new records/rows (in >> this case >> disassembled instructions) on-demand. Doing an early exit (e.g., >> quitting from a for-loop iterating over a function that yield's >> after >> only, say, 2 out of 10 items) will prematurely exit the function as >> well. Because, of course, why would Python calculate the latter 8 >> results when they're not used in the first place? This is also >> useful >> for never-ending functions - I suppose calculating digits of Pi >> would be >> a 'good' example. >> >> Anyway, so your cs_free() call is never reached in this case - you >> should switch to cs_free()'ing every row after each iteration >> through >> the for loop. >> >> >> but the loop is called upon the number of successfully disassembled >> instructions, >> so there is no where in the code that might possibly quit the loop >> prematurely. >> >> thanks, >> >> Q >> >> On 12/04/2014 12:27 AM, Nguyen Anh Quynh wrote: >> > >> > >> > On Thu, Dec 4, 2014 at 4:42 AM, Jan Newger <jan...@ne... <mailto:jan...@ne...> >> > <mailto:jan...@ne... <mailto:jan...@ne...>>> wrote: >> > >> > This is the python implementation of the disasm function >> (starting at >> > line 791): >> > >> > def disasm(self, code, offset, count=0): >> > all_insn = ctypes.POINTER(_cs_insn)() >> > '''if not _python2: >> > print(code) >> > code = code.encode() >> > print(code)''' >> > res = _cs.cs_disasm(self.csh, code, len(code), >> offset, count, >> > ctypes.byref(all_insn)) >> > if res > 0: >> > for i in range(res): >> > yield CsInsn(self, all_insn[i]) >> > _cs.cs_free(all_insn, res) >> > else: >> > status = _cs.cs_errno(self.csh) >> > if status != CS_ERR_OK: >> > raise CsError(status) >> > return >> > yield >> > >> > I'm really no python expert, but from what I see you >> apparently need to >> > free the instruction instances manually. However, if client >> code stops >> > enumeration over the instructions prematurely, then >> _cs.cs_free() is >> > never invoked, and thus memory is leaked, right? >> > >> > >> > yes the problem must be with Python binding but not the core. >> > however, in the above code, cs_free() is called after the "for" loop, >> > so i dont see how memleak can happen "prematurely". >> > >> > >> > thanks. >> > >> > >> > >> > >> > >> > On 03.12.2014 22:25, Jan Newger wrote: >> > > It seems the equivalent C implementation is not affected by >> the mem >> > > leak, which is to be expected, since the memory is >> explicitly freed >> > > anyways, and the group checking boils down to comparing an >> integer >> > value. >> > > >> > > If I had to guess, I'd suspect that in the python case the >> group >> > > checking code introduces a spurious reference to the >> instruction >> > > instance(?) which cannot be claimed by the GC. >> > > >> > > On 03.12.2014 16:57, Jan Newger wrote: >> > >> No I haven't tried to reproduce the mem leak with C. >> > >> It already took me a considerable amount of time to come >> up with this >> > >> minimal example. >> > >> >> > >> On 12/03/2014 04:53 PM, Capstone Engine wrote: >> > >>> >> > >>> >> > >>> On Wed, Dec 3, 2014 at 10:32 PM, Jan Newger >> > <jan...@ne... <mailto:jan...@ne...> >> <mailto:jan...@ne... <mailto:jan...@ne...>> >> > >>> <mailto:jan...@ne... >> <mailto:jan...@ne...> <mailto:jan...@ne... >> <mailto:jan...@ne...>>>> >> > wrote: >> > >>> >> > >>> >> > >>> Yes, it's using the latest version. >> > >>> OS was win7 x64 running python 2.7 with 32bit libraries. >> > >>> >> > >>> >> > >>> this is interesting. have you tried to code the same >> program in >> > C to see >> > >>> if the mem leak issue still happens? >> > >>> >> > >>> >> > >>> thanks. >> > >>> >> > >>> >> > >>> >> > >>> >> > >>> On 12/03/2014 04:31 PM, Nguyen Anh Quynh wrote: >> > >>> > >> > >>> > >> > >>> > On Wed, Dec 3, 2014 at 4:57 PM, Jan Newger >> > <jan...@ne... <mailto:jan...@ne...> >> <mailto:jan...@ne... <mailto:jan...@ne...>> >> > <mailto:jan...@ne... <mailto:jan...@ne...> >> <mailto:jan...@ne... <mailto:jan...@ne...>>> >> > >>> > <mailto:jan...@ne... >> <mailto:jan...@ne...> >> > <mailto:jan...@ne... <mailto:jan...@ne...>> >> <mailto:jan...@ne... <mailto:jan...@ne...> >> > <mailto:jan...@ne... >> <mailto:jan...@ne...>>>>> wrote: >> > >>> > >> > >>> > Hey, >> > >>> > >> > >>> > I was playing around with a few python scripts >> (using >> > >>> capstone among >> > >>> > other things) and always ran out of memory - and I >> > have no >> > >>> freaking idea >> > >>> > why. >> > >>> > >> > >>> > >> > >>> > is this with the latest 3.0 version? >> > >>> > >> > >>> > thanks, >> > >>> > Q >> > >>> > >> > >>> > >> > >>> > >> > >>> > >> > >>> > The code is really short: >> > >>> > >> > >>> > >> > >>> > from capstone import Cs >> > >>> > from capstone import CS_ARCH_X86 >> > >>> > from capstone import CS_MODE_32 >> > >>> > from capstone import CS_GRP_JUMP >> > >>> > from capstone import CS_GRP_CALL >> > >>> > from capstone import CS_GRP_RET >> > >>> > from capstone.x86_const import X86_INS_JNE, >> X86_INS_JMP >> > >>> > >> > >>> > """ >> > >>> > 0x401000: push ecx >> > >>> > 0x401001: pop ecx >> > >>> > 0x401002: mov eax, dword ptr [esp + >> 0x18] >> > >>> > 0x401006: mov eax, dword ptr [eax] >> > >>> > 0x401008: sar eax, 0 >> > >>> > 0x40100b: xor edi, eax >> > >>> > 0x40100d: nop >> > >>> > 0x40100e: add dword ptr [esp + 0x18], 4 >> > >>> > 0x401013: nop >> > >>> > 0x401014: dec word ptr [esp + 0x14] >> > >>> > 0x401019: shld edi, ecx, 0 >> > >>> > 0x40101d: jne 0x401000 >> > >>> > """ >> > >>> > def get_code(): >> > >>> > CODE = >> > >>> > >> > >>> >> > >> "\x51\x59\x8B\x44\x24\x18\x8B\x00\xC1\xF8\x00\x33\xF8\x90\x83\x44\x24\x18\x04\x90\x66\xFF\x4C\x24\x14\x0F\xA4\xCF\x00\x75\xE1" >> > >>> > >> > >>> > return CODE >> > >>> > >> > >>> > def is_branch(instr): >> > >>> > for group in branch_groups: >> > >>> > if group in instr.groups: >> > >>> > return True >> > >>> > return False >> > >>> > #return False >> > >>> > >> > >>> > # Disassemble until we hit basic block end. >> > >>> > def disasm(code): >> > >>> > disasm = Cs(CS_ARCH_X86, CS_MODE_32) >> > >>> > disasm.detail = True >> > >>> > address = 0x401000 >> > >>> > for instr in disasm.disasm(code, address): >> > >>> > print "0x%x:\t%s\t%s" % (instr.address, >> > instr.mnemonic, >> > >>> > instr.op_str) >> > >>> > if is_branch(instr): >> > >>> > break >> > >>> > >> > >>> > branch_groups = [CS_GRP_JUMP, CS_GRP_CALL, >> CS_GRP_RET] >> > >>> > code = get_code() >> > >>> > while True: >> > >>> > disasm(code) >> > >>> > >> > >>> > >> > >>> > That code goes out of memory after a few >> seconds. The >> > super >> > >>> weird thing >> > >>> > is, that if I change the implementation of >> > "is_branch(instr)" >> > >>> to simply >> > >>> > return False all the time, then the program >> does not >> > go out >> > >>> of memory! >> > >>> > Does anyone have an idea what's going on? >> > >>> > >> > >>> > Best >> > >>> > Jan >> > >>> > >> > >>> > >> > >>> >> > >> ------------------------------------------------------------------------------ >> > >>> > Download BIRT iHub F-Type - The Free >> Enterprise-Grade >> > BIRT Server >> > >>> > from Actuate! Instantly Supercharge Your Business >> > Reports and >> > >>> Dashboards >> > >>> > with Interactivity, Sharing, Native Excel >> Exports, App >> > >>> Integration & >> > >>> > more >> > >>> > Get technology previously reserved for >> billion-dollar >> > >>> corporations, FREE >> > >>> > >> > >>> >> > >> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >> > >>> > _______________________________________________ >> > >>> > Capstone-users mailing list >> > >>> > Cap...@li... >> <mailto:Cap...@li...> >> > <mailto:Cap...@li... >> <mailto:Cap...@li...>> >> > >>> <mailto:Cap...@li... >> <mailto:Cap...@li...> >> > <mailto:Cap...@li... >> <mailto:Cap...@li...>>> >> > >>> > <mailto:Cap...@li... >> <mailto:Cap...@li...> >> > <mailto:Cap...@li... >> <mailto:Cap...@li...>> >> > >>> <mailto:Cap...@li... >> <mailto:Cap...@li...> >> > <mailto:Cap...@li... >> <mailto:Cap...@li...>>>> >> > >>> > >> https://lists.sourceforge.net/lists/listinfo/capstone-users >> > >>> > >> > >>> > >> > >>> > >> > >>> > >> > >>> > >> > >>> >> > >> ------------------------------------------------------------------------------ >> > >>> > Download BIRT iHub F-Type - The Free Enterprise-Grade >> > BIRT Server >> > >>> > from Actuate! Instantly Supercharge Your Business >> Reports and >> > >>> Dashboards >> > >>> > with Interactivity, Sharing, Native Excel Exports, App >> > >>> Integration & more >> > >>> > Get technology previously reserved for billion-dollar >> > >>> corporations, FREE >> > >>> > >> > >>> >> > >> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >> > >>> > >> > >>> > >> > >>> > >> > >>> > _______________________________________________ >> > >>> > Capstone-users mailing list >> > >>> > Cap...@li... >> <mailto:Cap...@li...> >> > <mailto:Cap...@li... >> <mailto:Cap...@li...>> >> > >>> <mailto:Cap...@li... >> <mailto:Cap...@li...> >> > <mailto:Cap...@li... >> <mailto:Cap...@li...>>> >> > >>> > >> https://lists.sourceforge.net/lists/listinfo/capstone-users >> > >>> > >> > >>> >> > >>> >> > >>> >> > >> ------------------------------------------------------------------------------ >> > >>> Download BIRT iHub F-Type - The Free Enterprise-Grade >> BIRT >> > Server >> > >>> from Actuate! Instantly Supercharge Your Business Reports >> > and Dashboards >> > >>> with Interactivity, Sharing, Native Excel Exports, App >> > Integration & >> > >>> more >> > >>> Get technology previously reserved for billion-dollar >> > corporations, FREE >> > >>> >> > >> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >> > >>> _______________________________________________ >> > >>> Capstone-users mailing list >> > >>> Cap...@li... >> <mailto:Cap...@li...> >> > <mailto:Cap...@li... >> <mailto:Cap...@li...>> >> > >>> <mailto:Cap...@li... >> <mailto:Cap...@li...> >> > <mailto:Cap...@li... >> <mailto:Cap...@li...>>> >> > >>> https://lists.sourceforge.net/lists/listinfo/capstone-users >> > >>> >> > >>> >> > >>> >> > >>> >> > >>> >> > >> ------------------------------------------------------------------------------ >> > >>> Download BIRT iHub F-Type - The Free Enterprise-Grade >> BIRT Server >> > >>> from Actuate! Instantly Supercharge Your Business Reports and >> > Dashboards >> > >>> with Interactivity, Sharing, Native Excel Exports, App >> > Integration & more >> > >>> Get technology previously reserved for billion-dollar >> > corporations, FREE >> > >>> >> > >> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >> > >>> >> > >>> >> > >>> >> > >>> _______________________________________________ >> > >>> Capstone-users mailing list >> > >>> Cap...@li... >> <mailto:Cap...@li...> >> > <mailto:Cap...@li... >> <mailto:Cap...@li...>> >> > >>> https://lists.sourceforge.net/lists/listinfo/capstone-users >> > >>> >> > >> >> > >> >> > >> >> > >> ------------------------------------------------------------------------------ >> > >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT >> Server >> > >> from Actuate! Instantly Supercharge Your Business Reports and >> > Dashboards >> > >> with Interactivity, Sharing, Native Excel Exports, App >> > Integration & more >> > >> Get technology previously reserved for billion-dollar >> > corporations, FREE >> > >> >> > >> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >> > >> _______________________________________________ >> > >> Capstone-users mailing list >> > >> Cap...@li... >> <mailto:Cap...@li...> >> > <mailto:Cap...@li... >> <mailto:Cap...@li...>> >> > >> https://lists.sourceforge.net/lists/listinfo/capstone-users >> > >> >> > > >> > > >> > > >> > >> ------------------------------------------------------------------------------ >> > > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT >> Server >> > > from Actuate! Instantly Supercharge Your Business Reports and >> > Dashboards >> > > with Interactivity, Sharing, Native Excel Exports, App >> Integration >> > & more >> > > Get technology previously reserved for billion-dollar >> > corporations, FREE >> > > >> > >> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >> > > _______________________________________________ >> > > Capstone-users mailing list >> > > Cap...@li... >> <mailto:Cap...@li...> >> > <mailto:Cap...@li... >> <mailto:Cap...@li...>> >> > > https://lists.sourceforge.net/lists/listinfo/capstone-users >> > > >> > >> > >> > >> ------------------------------------------------------------------------------ >> > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >> > from Actuate! Instantly Supercharge Your Business Reports and >> Dashboards >> > with Interactivity, Sharing, Native Excel Exports, App >> Integration & >> > more >> > Get technology previously reserved for billion-dollar >> corporations, FREE >> > >> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >> > _______________________________________________ >> > Capstone-users mailing list >> > Cap...@li... >> <mailto:Cap...@li...> >> > <mailto:Cap...@li... >> <mailto:Cap...@li...>> >> > https://lists.sourceforge.net/lists/listinfo/capstone-users >> > >> > >> > >> > >> > >> ------------------------------------------------------------------------------ >> > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >> > from Actuate! Instantly Supercharge Your Business Reports and >> Dashboards >> > with Interactivity, Sharing, Native Excel Exports, App >> Integration & more >> > Get technology previously reserved for billion-dollar >> corporations, FREE >> > >> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >> > >> > >> > >> > _______________________________________________ >> > Capstone-users mailing list >> > Cap...@li... >> <mailto:Cap...@li...> >> > https://lists.sourceforge.net/lists/listinfo/capstone-users >> > >> >> ------------------------------------------------------------------------------ >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >> from Actuate! Instantly Supercharge Your Business Reports and Dashboards >> with Interactivity, Sharing, Native Excel Exports, App Integration & >> more >> Get technology previously reserved for billion-dollar corporations, FREE >> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >> _______________________________________________ >> Capstone-users mailing list >> Cap...@li... >> <mailto:Cap...@li...> >> https://lists.sourceforge.net/lists/listinfo/capstone-users >> >> >> ------------------------------------------------------------------------------ >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >> from Actuate! Instantly Supercharge Your Business Reports and Dashboards >> with Interactivity, Sharing, Native Excel Exports, App Integration & >> more >> Get technology previously reserved for billion-dollar corporations, FREE >> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >> _______________________________________________ >> Capstone-users mailing list >> Cap...@li... >> <mailto:Cap...@li...> >> https://lists.sourceforge.net/lists/listinfo/capstone-users >> >> >> >> >> ------------------------------------------------------------------------------ >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >> from Actuate! Instantly Supercharge Your Business Reports and Dashboards >> with Interactivity, Sharing, Native Excel Exports, App Integration & more >> Get technology previously reserved for billion-dollar corporations, FREE >> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >> >> >> >> _______________________________________________ >> Capstone-users mailing list >> Cap...@li... >> https://lists.sourceforge.net/lists/listinfo/capstone-users >> > > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > _______________________________________________ > Capstone-users mailing list > Cap...@li... > https://lists.sourceforge.net/lists/listinfo/capstone-users > |
|
From: Nguyen A. Q. <aq...@gm...> - 2014-12-04 13:55:29
|
On Thu, Dec 4, 2014 at 8:01 PM, Jan Newger <jan...@ne...> wrote: > I think the easiest fix would be to free the memory from a finally > clause, as described here: > > https://docs.python.org/2/reference/simple_stmts.html#the-yield-statement > > Like this (untested): > > def disasm(self, code, offset, count=0): > all_insn = ctypes.POINTER(_cs_insn)() > '''if not _python2: > print(code) > code = code.encode() > print(code)''' > res = _cs.cs_disasm(self.csh, code, len(code), offset, count, > ctypes.byref(all_insn)) > if res > 0: > try: > for i in range(res): > yield CsInsn(self, all_insn[i]) > finally: > _cs.cs_free(all_insn, res) > else: > status = _cs.cs_errno(self.csh) > if status != CS_ERR_OK: > raise CsError(status) > return > yield > > can you submit a pull request on Github? thanks, Q On 12/04/2014 01:07 AM, Jasiel Spelman wrote: > > Nguyen, > > > > Here is example code that hopefully better illustrates Juriaan's > > point/what Jan is seeing: > > > > def gen_example(): > > print '[gen_example] START' > > for i in xrange(2): > > print '[gen_example] before yield: ', i > > yield i > > print '[gen_example] after yield: ', i > > print '[gen_example] END' > > > > print "Typical case" > > for i in gen_example(): > > print > > > > print "Jan's case" > > for i in gen_example(): > > if i == 1: > > break > > print > > > > > > Typical case > > [gen_example] START > > [gen_example] before yield: 0 > > > > [gen_example] after yield: 0 > > [gen_example] before yield: 1 > > > > [gen_example] after yield: 1 > > [gen_example] END > > > > > > Jan's case > > [gen_example] START > > [gen_example] before yield: 0 > > > > [gen_example] after yield: 0 > > [gen_example] before yield: 1 > > > > > > On Wed, Dec 3, 2014 at 5:49 PM, Nguyen Anh Quynh <aq...@gm... > > <mailto:aq...@gm...>> wrote: > > > > > > > > On Thu, Dec 4, 2014 at 6:32 AM, Jurriaan Bremer > > <jur...@gm... <mailto:jur...@gm...>> wrote: > > > > The 'yield' keyword allows one to fetch new records/rows (in > > this case > > disassembled instructions) on-demand. Doing an early exit (e.g., > > quitting from a for-loop iterating over a function that yield's > > after > > only, say, 2 out of 10 items) will prematurely exit the function > as > > well. Because, of course, why would Python calculate the latter 8 > > results when they're not used in the first place? This is also > > useful > > for never-ending functions - I suppose calculating digits of Pi > > would be > > a 'good' example. > > > > Anyway, so your cs_free() call is never reached in this case - > you > > should switch to cs_free()'ing every row after each iteration > > through > > the for loop. > > > > > > but the loop is called upon the number of successfully disassembled > > instructions, > > so there is no where in the code that might possibly quit the loop > > prematurely. > > > > thanks, > > > > Q > > > > On 12/04/2014 12:27 AM, Nguyen Anh Quynh wrote: > > > > > > > > > On Thu, Dec 4, 2014 at 4:42 AM, Jan Newger <jan...@ne... > <mailto:jan...@ne...> > > > <mailto:jan...@ne... <mailto:jan...@ne...>>> > wrote: > > > > > > This is the python implementation of the disasm function > > (starting at > > > line 791): > > > > > > def disasm(self, code, offset, count=0): > > > all_insn = ctypes.POINTER(_cs_insn)() > > > '''if not _python2: > > > print(code) > > > code = code.encode() > > > print(code)''' > > > res = _cs.cs_disasm(self.csh, code, len(code), > > offset, count, > > > ctypes.byref(all_insn)) > > > if res > 0: > > > for i in range(res): > > > yield CsInsn(self, all_insn[i]) > > > _cs.cs_free(all_insn, res) > > > else: > > > status = _cs.cs_errno(self.csh) > > > if status != CS_ERR_OK: > > > raise CsError(status) > > > return > > > yield > > > > > > I'm really no python expert, but from what I see you > > apparently need to > > > free the instruction instances manually. However, if client > > code stops > > > enumeration over the instructions prematurely, then > > _cs.cs_free() is > > > never invoked, and thus memory is leaked, right? > > > > > > > > > yes the problem must be with Python binding but not the core. > > > however, in the above code, cs_free() is called after the "for" > loop, > > > so i dont see how memleak can happen "prematurely". > > > > > > > > > thanks. > > > > > > > > > > > > > > > > > > On 03.12.2014 22:25, Jan Newger wrote: > > > > It seems the equivalent C implementation is not affected by > > the mem > > > > leak, which is to be expected, since the memory is > > explicitly freed > > > > anyways, and the group checking boils down to comparing an > > integer > > > value. > > > > > > > > If I had to guess, I'd suspect that in the python case the > > group > > > > checking code introduces a spurious reference to the > > instruction > > > > instance(?) which cannot be claimed by the GC. > > > > > > > > On 03.12.2014 16:57, Jan Newger wrote: > > > >> No I haven't tried to reproduce the mem leak with C. > > > >> It already took me a considerable amount of time to come > > up with this > > > >> minimal example. > > > >> > > > >> On 12/03/2014 04:53 PM, Capstone Engine wrote: > > > >>> > > > >>> > > > >>> On Wed, Dec 3, 2014 at 10:32 PM, Jan Newger > > > <jan...@ne... <mailto:jan...@ne...> > > <mailto:jan...@ne... <mailto:jan...@ne...>> > > > >>> <mailto:jan...@ne... > > <mailto:jan...@ne...> <mailto:jan...@ne... > > <mailto:jan...@ne...>>>> > > > wrote: > > > >>> > > > >>> > > > >>> Yes, it's using the latest version. > > > >>> OS was win7 x64 running python 2.7 with 32bit > libraries. > > > >>> > > > >>> > > > >>> this is interesting. have you tried to code the same > > program in > > > C to see > > > >>> if the mem leak issue still happens? > > > >>> > > > >>> > > > >>> thanks. > > > >>> > > > >>> > > > >>> > > > >>> > > > >>> On 12/03/2014 04:31 PM, Nguyen Anh Quynh wrote: > > > >>> > > > > >>> > > > > >>> > On Wed, Dec 3, 2014 at 4:57 PM, Jan Newger > > > <jan...@ne... <mailto:jan...@ne...> > > <mailto:jan...@ne... <mailto:jan...@ne...>> > > > <mailto:jan...@ne... <mailto:jan...@ne...> > > <mailto:jan...@ne... <mailto:jan...@ne...>>> > > > >>> > <mailto:jan...@ne... > > <mailto:jan...@ne...> > > > <mailto:jan...@ne... <mailto:jan...@ne...>> > > <mailto:jan...@ne... <mailto:jan...@ne...> > > > <mailto:jan...@ne... > > <mailto:jan...@ne...>>>>> wrote: > > > >>> > > > > >>> > Hey, > > > >>> > > > > >>> > I was playing around with a few python scripts > > (using > > > >>> capstone among > > > >>> > other things) and always ran out of memory - > and I > > > have no > > > >>> freaking idea > > > >>> > why. > > > >>> > > > > >>> > > > > >>> > is this with the latest 3.0 version? > > > >>> > > > > >>> > thanks, > > > >>> > Q > > > >>> > > > > >>> > > > > >>> > > > > >>> > > > > >>> > The code is really short: > > > >>> > > > > >>> > > > > >>> > from capstone import Cs > > > >>> > from capstone import CS_ARCH_X86 > > > >>> > from capstone import CS_MODE_32 > > > >>> > from capstone import CS_GRP_JUMP > > > >>> > from capstone import CS_GRP_CALL > > > >>> > from capstone import CS_GRP_RET > > > >>> > from capstone.x86_const import X86_INS_JNE, > > X86_INS_JMP > > > >>> > > > > >>> > """ > > > >>> > 0x401000: push ecx > > > >>> > 0x401001: pop ecx > > > >>> > 0x401002: mov eax, dword ptr [esp + > > 0x18] > > > >>> > 0x401006: mov eax, dword ptr [eax] > > > >>> > 0x401008: sar eax, 0 > > > >>> > 0x40100b: xor edi, eax > > > >>> > 0x40100d: nop > > > >>> > 0x40100e: add dword ptr [esp + > 0x18], 4 > > > >>> > 0x401013: nop > > > >>> > 0x401014: dec word ptr [esp + 0x14] > > > >>> > 0x401019: shld edi, ecx, 0 > > > >>> > 0x40101d: jne 0x401000 > > > >>> > """ > > > >>> > def get_code(): > > > >>> > CODE = > > > >>> > > > > >>> > > > > > > "\x51\x59\x8B\x44\x24\x18\x8B\x00\xC1\xF8\x00\x33\xF8\x90\x83\x44\x24\x18\x04\x90\x66\xFF\x4C\x24\x14\x0F\xA4\xCF\x00\x75\xE1" > > > >>> > > > > >>> > return CODE > > > >>> > > > > >>> > def is_branch(instr): > > > >>> > for group in branch_groups: > > > >>> > if group in instr.groups: > > > >>> > return True > > > >>> > return False > > > >>> > #return False > > > >>> > > > > >>> > # Disassemble until we hit basic block end. > > > >>> > def disasm(code): > > > >>> > disasm = Cs(CS_ARCH_X86, CS_MODE_32) > > > >>> > disasm.detail = True > > > >>> > address = 0x401000 > > > >>> > for instr in disasm.disasm(code, address): > > > >>> > print "0x%x:\t%s\t%s" % (instr.address, > > > instr.mnemonic, > > > >>> > instr.op_str) > > > >>> > if is_branch(instr): > > > >>> > break > > > >>> > > > > >>> > branch_groups = [CS_GRP_JUMP, CS_GRP_CALL, > > CS_GRP_RET] > > > >>> > code = get_code() > > > >>> > while True: > > > >>> > disasm(code) > > > >>> > > > > >>> > > > > >>> > That code goes out of memory after a few > > seconds. The > > > super > > > >>> weird thing > > > >>> > is, that if I change the implementation of > > > "is_branch(instr)" > > > >>> to simply > > > >>> > return False all the time, then the program > > does not > > > go out > > > >>> of memory! > > > >>> > Does anyone have an idea what's going on? > > > >>> > > > > >>> > Best > > > >>> > Jan > > > >>> > > > > >>> > > > > >>> > > > > > > ------------------------------------------------------------------------------ > > > >>> > Download BIRT iHub F-Type - The Free > > Enterprise-Grade > > > BIRT Server > > > >>> > from Actuate! Instantly Supercharge Your > Business > > > Reports and > > > >>> Dashboards > > > >>> > with Interactivity, Sharing, Native Excel > > Exports, App > > > >>> Integration & > > > >>> > more > > > >>> > Get technology previously reserved for > > billion-dollar > > > >>> corporations, FREE > > > >>> > > > > >>> > > > > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > > >>> > _______________________________________________ > > > >>> > Capstone-users mailing list > > > >>> > Cap...@li... > > <mailto:Cap...@li...> > > > <mailto:Cap...@li... > > <mailto:Cap...@li...>> > > > >>> <mailto:Cap...@li... > > <mailto:Cap...@li...> > > > <mailto:Cap...@li... > > <mailto:Cap...@li...>>> > > > >>> > <mailto:Cap...@li... > > <mailto:Cap...@li...> > > > <mailto:Cap...@li... > > <mailto:Cap...@li...>> > > > >>> <mailto:Cap...@li... > > <mailto:Cap...@li...> > > > <mailto:Cap...@li... > > <mailto:Cap...@li...>>>> > > > >>> > > > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > >>> > > > > >>> > > > > >>> > > > > >>> > > > > >>> > > > > >>> > > > > > > ------------------------------------------------------------------------------ > > > >>> > Download BIRT iHub F-Type - The Free > Enterprise-Grade > > > BIRT Server > > > >>> > from Actuate! Instantly Supercharge Your Business > > Reports and > > > >>> Dashboards > > > >>> > with Interactivity, Sharing, Native Excel Exports, > App > > > >>> Integration & more > > > >>> > Get technology previously reserved for > billion-dollar > > > >>> corporations, FREE > > > >>> > > > > >>> > > > > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > > >>> > > > > >>> > > > > >>> > > > > >>> > _______________________________________________ > > > >>> > Capstone-users mailing list > > > >>> > Cap...@li... > > <mailto:Cap...@li...> > > > <mailto:Cap...@li... > > <mailto:Cap...@li...>> > > > >>> <mailto:Cap...@li... > > <mailto:Cap...@li...> > > > <mailto:Cap...@li... > > <mailto:Cap...@li...>>> > > > >>> > > > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > >>> > > > > >>> > > > >>> > > > >>> > > > > > > ------------------------------------------------------------------------------ > > > >>> Download BIRT iHub F-Type - The Free Enterprise-Grade > > BIRT > > > Server > > > >>> from Actuate! Instantly Supercharge Your Business > Reports > > > and Dashboards > > > >>> with Interactivity, Sharing, Native Excel Exports, App > > > Integration & > > > >>> more > > > >>> Get technology previously reserved for billion-dollar > > > corporations, FREE > > > >>> > > > > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > > >>> _______________________________________________ > > > >>> Capstone-users mailing list > > > >>> Cap...@li... > > <mailto:Cap...@li...> > > > <mailto:Cap...@li... > > <mailto:Cap...@li...>> > > > >>> <mailto:Cap...@li... > > <mailto:Cap...@li...> > > > <mailto:Cap...@li... > > <mailto:Cap...@li...>>> > > > >>> > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > >>> > > > >>> > > > >>> > > > >>> > > > >>> > > > > > > ------------------------------------------------------------------------------ > > > >>> Download BIRT iHub F-Type - The Free Enterprise-Grade > > BIRT Server > > > >>> from Actuate! Instantly Supercharge Your Business Reports > and > > > Dashboards > > > >>> with Interactivity, Sharing, Native Excel Exports, App > > > Integration & more > > > >>> Get technology previously reserved for billion-dollar > > > corporations, FREE > > > >>> > > > > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > > >>> > > > >>> > > > >>> > > > >>> _______________________________________________ > > > >>> Capstone-users mailing list > > > >>> Cap...@li... > > <mailto:Cap...@li...> > > > <mailto:Cap...@li... > > <mailto:Cap...@li...>> > > > >>> > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > >>> > > > >> > > > >> > > > >> > > > > > > ------------------------------------------------------------------------------ > > > >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT > > Server > > > >> from Actuate! Instantly Supercharge Your Business Reports > and > > > Dashboards > > > >> with Interactivity, Sharing, Native Excel Exports, App > > > Integration & more > > > >> Get technology previously reserved for billion-dollar > > > corporations, FREE > > > >> > > > > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > > >> _______________________________________________ > > > >> Capstone-users mailing list > > > >> Cap...@li... > > <mailto:Cap...@li...> > > > <mailto:Cap...@li... > > <mailto:Cap...@li...>> > > > >> > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > >> > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------------------ > > > > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT > > Server > > > > from Actuate! Instantly Supercharge Your Business Reports > and > > > Dashboards > > > > with Interactivity, Sharing, Native Excel Exports, App > > Integration > > > & more > > > > Get technology previously reserved for billion-dollar > > > corporations, FREE > > > > > > > > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > > > _______________________________________________ > > > > Capstone-users mailing list > > > > Cap...@li... > > <mailto:Cap...@li...> > > > <mailto:Cap...@li... > > <mailto:Cap...@li...>> > > > > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > > > > > > > > > > > > > > ------------------------------------------------------------------------------ > > > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT > Server > > > from Actuate! Instantly Supercharge Your Business Reports and > > Dashboards > > > with Interactivity, Sharing, Native Excel Exports, App > > Integration & > > > more > > > Get technology previously reserved for billion-dollar > > corporations, FREE > > > > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > > _______________________________________________ > > > Capstone-users mailing list > > > Cap...@li... > > <mailto:Cap...@li...> > > > <mailto:Cap...@li... > > <mailto:Cap...@li...>> > > > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------------------ > > > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > > > from Actuate! Instantly Supercharge Your Business Reports and > > Dashboards > > > with Interactivity, Sharing, Native Excel Exports, App > > Integration & more > > > Get technology previously reserved for billion-dollar > > corporations, FREE > > > > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > > > > > > > > > > > _______________________________________________ > > > Capstone-users mailing list > > > Cap...@li... > > <mailto:Cap...@li...> > > > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > > > > > > ------------------------------------------------------------------------------ > > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > > from Actuate! Instantly Supercharge Your Business Reports and > Dashboards > > with Interactivity, Sharing, Native Excel Exports, App Integration & > > more > > Get technology previously reserved for billion-dollar corporations, > FREE > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > _______________________________________________ > > Capstone-users mailing list > > Cap...@li... > > <mailto:Cap...@li...> > > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > > > > > ------------------------------------------------------------------------------ > > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > > from Actuate! Instantly Supercharge Your Business Reports and > Dashboards > > with Interactivity, Sharing, Native Excel Exports, App Integration & > > more > > Get technology previously reserved for billion-dollar corporations, > FREE > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > _______________________________________________ > > Capstone-users mailing list > > Cap...@li... > > <mailto:Cap...@li...> > > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > > > > > > > > > ------------------------------------------------------------------------------ > > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > > with Interactivity, Sharing, Native Excel Exports, App Integration & more > > Get technology previously reserved for billion-dollar corporations, FREE > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > > > > > > > _______________________________________________ > > Capstone-users mailing list > > Cap...@li... > > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > > > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > _______________________________________________ > Capstone-users mailing list > Cap...@li... > https://lists.sourceforge.net/lists/listinfo/capstone-users > |
|
From: Jan N. <jan...@ne...> - 2014-12-04 14:09:16
|
I don't have the dev environment for this in place, it would mean additional time for me to set this up. On 12/04/2014 02:54 PM, Nguyen Anh Quynh wrote: > > > On Thu, Dec 4, 2014 at 8:01 PM, Jan Newger <jan...@ne... > <mailto:jan...@ne...>> wrote: > > I think the easiest fix would be to free the memory from a finally > clause, as described here: > > https://docs.python.org/2/reference/simple_stmts.html#the-yield-statement > > Like this (untested): > > def disasm(self, code, offset, count=0): > all_insn = ctypes.POINTER(_cs_insn)() > '''if not _python2: > print(code) > code = code.encode() > print(code)''' > res = _cs.cs_disasm(self.csh, code, len(code), offset, count, > ctypes.byref(all_insn)) > if res > 0: > try: > for i in range(res): > yield CsInsn(self, all_insn[i]) > finally: > _cs.cs_free(all_insn, res) > else: > status = _cs.cs_errno(self.csh) > if status != CS_ERR_OK: > raise CsError(status) > return > yield > > > can you submit a pull request on Github? > > thanks, > Q > > > > > > > On 12/04/2014 01:07 AM, Jasiel Spelman wrote: > > Nguyen, > > > > Here is example code that hopefully better illustrates Juriaan's > > point/what Jan is seeing: > > > > def gen_example(): > > print '[gen_example] START' > > for i in xrange(2): > > print '[gen_example] before yield: ', i > > yield i > > print '[gen_example] after yield: ', i > > print '[gen_example] END' > > > > print "Typical case" > > for i in gen_example(): > > print > > > > print "Jan's case" > > for i in gen_example(): > > if i == 1: > > break > > print > > > > > > Typical case > > [gen_example] START > > [gen_example] before yield: 0 > > > > [gen_example] after yield: 0 > > [gen_example] before yield: 1 > > > > [gen_example] after yield: 1 > > [gen_example] END > > > > > > Jan's case > > [gen_example] START > > [gen_example] before yield: 0 > > > > [gen_example] after yield: 0 > > [gen_example] before yield: 1 > > > > > > On Wed, Dec 3, 2014 at 5:49 PM, Nguyen Anh Quynh > <aq...@gm... <mailto:aq...@gm...> > > <mailto:aq...@gm... <mailto:aq...@gm...>>> wrote: > > > > > > > > On Thu, Dec 4, 2014 at 6:32 AM, Jurriaan Bremer > > <jur...@gm... <mailto:jur...@gm...> > <mailto:jur...@gm... <mailto:jur...@gm...>>> > wrote: > > > > The 'yield' keyword allows one to fetch new records/rows (in > > this case > > disassembled instructions) on-demand. Doing an early exit > (e.g., > > quitting from a for-loop iterating over a function that > yield's > > after > > only, say, 2 out of 10 items) will prematurely exit the > function as > > well. Because, of course, why would Python calculate the > latter 8 > > results when they're not used in the first place? This is > also > > useful > > for never-ending functions - I suppose calculating digits > of Pi > > would be > > a 'good' example. > > > > Anyway, so your cs_free() call is never reached in this > case - you > > should switch to cs_free()'ing every row after each iteration > > through > > the for loop. > > > > > > but the loop is called upon the number of successfully > disassembled > > instructions, > > so there is no where in the code that might possibly quit the > loop > > prematurely. > > > > thanks, > > > > Q > > > > On 12/04/2014 12:27 AM, Nguyen Anh Quynh wrote: > > > > > > > > > On Thu, Dec 4, 2014 at 4:42 AM, Jan Newger > <jan...@ne... <mailto:jan...@ne...> > <mailto:jan...@ne... <mailto:jan...@ne...>> > > > <mailto:jan...@ne... > <mailto:jan...@ne...> <mailto:jan...@ne... > <mailto:jan...@ne...>>>> wrote: > > > > > > This is the python implementation of the disasm function > > (starting at > > > line 791): > > > > > > def disasm(self, code, offset, count=0): > > > all_insn = ctypes.POINTER(_cs_insn)() > > > '''if not _python2: > > > print(code) > > > code = code.encode() > > > print(code)''' > > > res = _cs.cs_disasm(self.csh, code, len(code), > > offset, count, > > > ctypes.byref(all_insn)) > > > if res > 0: > > > for i in range(res): > > > yield CsInsn(self, all_insn[i]) > > > _cs.cs_free(all_insn, res) > > > else: > > > status = _cs.cs_errno(self.csh) > > > if status != CS_ERR_OK: > > > raise CsError(status) > > > return > > > yield > > > > > > I'm really no python expert, but from what I see you > > apparently need to > > > free the instruction instances manually. However, if > client > > code stops > > > enumeration over the instructions prematurely, then > > _cs.cs_free() is > > > never invoked, and thus memory is leaked, right? > > > > > > > > > yes the problem must be with Python binding but not the core. > > > however, in the above code, cs_free() is called after the > "for" loop, > > > so i dont see how memleak can happen "prematurely". > > > > > > > > > thanks. > > > > > > > > > > > > > > > > > > On 03.12.2014 22:25, Jan Newger wrote: > > > > It seems the equivalent C implementation is not > affected by > > the mem > > > > leak, which is to be expected, since the memory is > > explicitly freed > > > > anyways, and the group checking boils down to > comparing an > > integer > > > value. > > > > > > > > If I had to guess, I'd suspect that in the python > case the > > group > > > > checking code introduces a spurious reference to the > > instruction > > > > instance(?) which cannot be claimed by the GC. > > > > > > > > On 03.12.2014 16:57, Jan Newger wrote: > > > >> No I haven't tried to reproduce the mem leak with C. > > > >> It already took me a considerable amount of time to > come > > up with this > > > >> minimal example. > > > >> > > > >> On 12/03/2014 04:53 PM, Capstone Engine wrote: > > > >>> > > > >>> > > > >>> On Wed, Dec 3, 2014 at 10:32 PM, Jan Newger > > > <jan...@ne... <mailto:jan...@ne...> > <mailto:jan...@ne... <mailto:jan...@ne...>> > > <mailto:jan...@ne... <mailto:jan...@ne...> > <mailto:jan...@ne... <mailto:jan...@ne...>>> > > > >>> <mailto:jan...@ne... > <mailto:jan...@ne...> > > <mailto:jan...@ne... <mailto:jan...@ne...>> > <mailto:jan...@ne... <mailto:jan...@ne...> > > <mailto:jan...@ne... <mailto:jan...@ne...>>>>> > > > wrote: > > > >>> > > > >>> > > > >>> Yes, it's using the latest version. > > > >>> OS was win7 x64 running python 2.7 with 32bit > libraries. > > > >>> > > > >>> > > > >>> this is interesting. have you tried to code the same > > program in > > > C to see > > > >>> if the mem leak issue still happens? > > > >>> > > > >>> > > > >>> thanks. > > > >>> > > > >>> > > > >>> > > > >>> > > > >>> On 12/03/2014 04:31 PM, Nguyen Anh Quynh wrote: > > > >>> > > > > >>> > > > > >>> > On Wed, Dec 3, 2014 at 4:57 PM, Jan Newger > > > <jan...@ne... <mailto:jan...@ne...> > <mailto:jan...@ne... <mailto:jan...@ne...>> > > <mailto:jan...@ne... <mailto:jan...@ne...> > <mailto:jan...@ne... <mailto:jan...@ne...>>> > > > <mailto:jan...@ne... > <mailto:jan...@ne...> <mailto:jan...@ne... > <mailto:jan...@ne...>> > > <mailto:jan...@ne... <mailto:jan...@ne...> > <mailto:jan...@ne... <mailto:jan...@ne...>>>> > > > >>> > <mailto:jan...@ne... > <mailto:jan...@ne...> > > <mailto:jan...@ne... <mailto:jan...@ne...>> > > > <mailto:jan...@ne... > <mailto:jan...@ne...> <mailto:jan...@ne... > <mailto:jan...@ne...>>> > > <mailto:jan...@ne... <mailto:jan...@ne...> > <mailto:jan...@ne... <mailto:jan...@ne...>> > > > <mailto:jan...@ne... > <mailto:jan...@ne...> > > <mailto:jan...@ne... > <mailto:jan...@ne...>>>>>> wrote: > > > >>> > > > > >>> > Hey, > > > >>> > > > > >>> > I was playing around with a few python > scripts > > (using > > > >>> capstone among > > > >>> > other things) and always ran out of > memory - and I > > > have no > > > >>> freaking idea > > > >>> > why. > > > >>> > > > > >>> > > > > >>> > is this with the latest 3.0 version? > > > >>> > > > > >>> > thanks, > > > >>> > Q > > > >>> > > > > >>> > > > > >>> > > > > >>> > > > > >>> > The code is really short: > > > >>> > > > > >>> > > > > >>> > from capstone import Cs > > > >>> > from capstone import CS_ARCH_X86 > > > >>> > from capstone import CS_MODE_32 > > > >>> > from capstone import CS_GRP_JUMP > > > >>> > from capstone import CS_GRP_CALL > > > >>> > from capstone import CS_GRP_RET > > > >>> > from capstone.x86_const import X86_INS_JNE, > > X86_INS_JMP > > > >>> > > > > >>> > """ > > > >>> > 0x401000: push ecx > > > >>> > 0x401001: pop ecx > > > >>> > 0x401002: mov eax, dword ptr > [esp + > > 0x18] > > > >>> > 0x401006: mov eax, dword ptr > [eax] > > > >>> > 0x401008: sar eax, 0 > > > >>> > 0x40100b: xor edi, eax > > > >>> > 0x40100d: nop > > > >>> > 0x40100e: add dword ptr [esp > + 0x18], 4 > > > >>> > 0x401013: nop > > > >>> > 0x401014: dec word ptr [esp + > 0x14] > > > >>> > 0x401019: shld edi, ecx, 0 > > > >>> > 0x40101d: jne 0x401000 > > > >>> > """ > > > >>> > def get_code(): > > > >>> > CODE = > > > >>> > > > > >>> > > > > > > "\x51\x59\x8B\x44\x24\x18\x8B\x00\xC1\xF8\x00\x33\xF8\x90\x83\x44\x24\x18\x04\x90\x66\xFF\x4C\x24\x14\x0F\xA4\xCF\x00\x75\xE1" > > > >>> > > > > >>> > return CODE > > > >>> > > > > >>> > def is_branch(instr): > > > >>> > for group in branch_groups: > > > >>> > if group in instr.groups: > > > >>> > return True > > > >>> > return False > > > >>> > #return False > > > >>> > > > > >>> > # Disassemble until we hit basic block end. > > > >>> > def disasm(code): > > > >>> > disasm = Cs(CS_ARCH_X86, CS_MODE_32) > > > >>> > disasm.detail = True > > > >>> > address = 0x401000 > > > >>> > for instr in disasm.disasm(code, > address): > > > >>> > print "0x%x:\t%s\t%s" % > (instr.address, > > > instr.mnemonic, > > > >>> > instr.op_str) > > > >>> > if is_branch(instr): > > > >>> > break > > > >>> > > > > >>> > branch_groups = [CS_GRP_JUMP, CS_GRP_CALL, > > CS_GRP_RET] > > > >>> > code = get_code() > > > >>> > while True: > > > >>> > disasm(code) > > > >>> > > > > >>> > > > > >>> > That code goes out of memory after a few > > seconds. The > > > super > > > >>> weird thing > > > >>> > is, that if I change the implementation of > > > "is_branch(instr)" > > > >>> to simply > > > >>> > return False all the time, then the program > > does not > > > go out > > > >>> of memory! > > > >>> > Does anyone have an idea what's going on? > > > >>> > > > > >>> > Best > > > >>> > Jan > > > >>> > > > > >>> > > > > >>> > > > > > > ------------------------------------------------------------------------------ > > > >>> > Download BIRT iHub F-Type - The Free > > Enterprise-Grade > > > BIRT Server > > > >>> > from Actuate! Instantly Supercharge > Your Business > > > Reports and > > > >>> Dashboards > > > >>> > with Interactivity, Sharing, Native Excel > > Exports, App > > > >>> Integration & > > > >>> > more > > > >>> > Get technology previously reserved for > > billion-dollar > > > >>> corporations, FREE > > > >>> > > > > >>> > > > > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > > >>> > > _______________________________________________ > > > >>> > Capstone-users mailing list > > > >>> > Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>> > > > <mailto:Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>>> > > > >>> <mailto:Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>> > > > <mailto:Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>>>> > > > >>> > > <mailto:Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>> > > > <mailto:Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>>> > > > >>> <mailto:Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>> > > > <mailto:Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>>>>> > > > >>> > > > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > >>> > > > > >>> > > > > >>> > > > > >>> > > > > >>> > > > > >>> > > > > > > ------------------------------------------------------------------------------ > > > >>> > Download BIRT iHub F-Type - The Free > Enterprise-Grade > > > BIRT Server > > > >>> > from Actuate! Instantly Supercharge Your > Business > > Reports and > > > >>> Dashboards > > > >>> > with Interactivity, Sharing, Native Excel > Exports, App > > > >>> Integration & more > > > >>> > Get technology previously reserved for > billion-dollar > > > >>> corporations, FREE > > > >>> > > > > >>> > > > > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > > >>> > > > > >>> > > > > >>> > > > > >>> > _______________________________________________ > > > >>> > Capstone-users mailing list > > > >>> > Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>> > > > <mailto:Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>>> > > > >>> <mailto:Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>> > > > <mailto:Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>>>> > > > >>> > > > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > >>> > > > > >>> > > > >>> > > > >>> > > > > > > ------------------------------------------------------------------------------ > > > >>> Download BIRT iHub F-Type - The Free > Enterprise-Grade > > BIRT > > > Server > > > >>> from Actuate! Instantly Supercharge Your > Business Reports > > > and Dashboards > > > >>> with Interactivity, Sharing, Native Excel > Exports, App > > > Integration & > > > >>> more > > > >>> Get technology previously reserved for > billion-dollar > > > corporations, FREE > > > >>> > > > > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > > >>> _______________________________________________ > > > >>> Capstone-users mailing list > > > >>> Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>> > > > <mailto:Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>>> > > > >>> <mailto:Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>> > > > <mailto:Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>>>> > > > >>> > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > >>> > > > >>> > > > >>> > > > >>> > > > >>> > > > > > > ------------------------------------------------------------------------------ > > > >>> Download BIRT iHub F-Type - The Free Enterprise-Grade > > BIRT Server > > > >>> from Actuate! Instantly Supercharge Your Business > Reports and > > > Dashboards > > > >>> with Interactivity, Sharing, Native Excel Exports, App > > > Integration & more > > > >>> Get technology previously reserved for billion-dollar > > > corporations, FREE > > > >>> > > > > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > > >>> > > > >>> > > > >>> > > > >>> _______________________________________________ > > > >>> Capstone-users mailing list > > > >>> Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>> > > > <mailto:Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>>> > > > >>> > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > >>> > > > >> > > > >> > > > >> > > > > > > ------------------------------------------------------------------------------ > > > >> Download BIRT iHub F-Type - The Free > Enterprise-Grade BIRT > > Server > > > >> from Actuate! Instantly Supercharge Your Business > Reports and > > > Dashboards > > > >> with Interactivity, Sharing, Native Excel Exports, App > > > Integration & more > > > >> Get technology previously reserved for billion-dollar > > > corporations, FREE > > > >> > > > > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > > >> _______________________________________________ > > > >> Capstone-users mailing list > > > >> Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>> > > > <mailto:Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>>> > > > >> > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > >> > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------------------ > > > > Download BIRT iHub F-Type - The Free > Enterprise-Grade BIRT > > Server > > > > from Actuate! Instantly Supercharge Your Business > Reports and > > > Dashboards > > > > with Interactivity, Sharing, Native Excel Exports, App > > Integration > > > & more > > > > Get technology previously reserved for billion-dollar > > > corporations, FREE > > > > > > > > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > > > _______________________________________________ > > > > Capstone-users mailing list > > > > Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>> > > > <mailto:Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>>> > > > > > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > > > > > > > > > > > > > > ------------------------------------------------------------------------------ > > > Download BIRT iHub F-Type - The Free Enterprise-Grade > BIRT Server > > > from Actuate! Instantly Supercharge Your Business > Reports and > > Dashboards > > > with Interactivity, Sharing, Native Excel Exports, App > > Integration & > > > more > > > Get technology previously reserved for billion-dollar > > corporations, FREE > > > > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > > _______________________________________________ > > > Capstone-users mailing list > > > Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>> > > > <mailto:Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>>> > > > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------------------ > > > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT > Server > > > from Actuate! Instantly Supercharge Your Business Reports and > > Dashboards > > > with Interactivity, Sharing, Native Excel Exports, App > > Integration & more > > > Get technology previously reserved for billion-dollar > > corporations, FREE > > > > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > > > > > > > > > > > _______________________________________________ > > > Capstone-users mailing list > > > Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>> > > > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > > > > > > ------------------------------------------------------------------------------ > > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > > from Actuate! Instantly Supercharge Your Business Reports and > Dashboards > > with Interactivity, Sharing, Native Excel Exports, App > Integration & > > more > > Get technology previously reserved for billion-dollar > corporations, FREE > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > _______________________________________________ > > Capstone-users mailing list > > Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>> > > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > > > > > ------------------------------------------------------------------------------ > > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > > from Actuate! Instantly Supercharge Your Business Reports and > Dashboards > > with Interactivity, Sharing, Native Excel Exports, App > Integration & > > more > > Get technology previously reserved for billion-dollar > corporations, FREE > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > _______________________________________________ > > Capstone-users mailing list > > Cap...@li... > <mailto:Cap...@li...> > > <mailto:Cap...@li... > <mailto:Cap...@li...>> > > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > > > > > > > > > ------------------------------------------------------------------------------ > > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > > from Actuate! Instantly Supercharge Your Business Reports and > Dashboards > > with Interactivity, Sharing, Native Excel Exports, App > Integration & more > > Get technology previously reserved for billion-dollar > corporations, FREE > > > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > > > > > > > _______________________________________________ > > Capstone-users mailing list > > Cap...@li... > <mailto:Cap...@li...> > > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & > more > Get technology previously reserved for billion-dollar corporations, FREE > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > _______________________________________________ > Capstone-users mailing list > Cap...@li... > <mailto:Cap...@li...> > https://lists.sourceforge.net/lists/listinfo/capstone-users > > > > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > > > > _______________________________________________ > Capstone-users mailing list > Cap...@li... > https://lists.sourceforge.net/lists/listinfo/capstone-users > |
|
From: Nguyen A. Q. <aq...@gm...> - 2014-12-04 14:15:14
|
On Thu, Dec 4, 2014 at 9:08 PM, Jan Newger <jan...@ne...> wrote: > I don't have the dev environment for this in place, it would mean > additional time for me to set this up. > sure, there is no hurry. thanks. Q |
|
From: Jason O. <ja...@ko...> - 2014-12-04 17:38:54
|
The try:finally solution is a great tip. And it entirely avoids the circular reference issue with __del__! I love it. > On Dec 4, 2014, at 06:14, Nguyen Anh Quynh <aq...@gm...> wrote: > > > >> On Thu, Dec 4, 2014 at 9:08 PM, Jan Newger <jan...@ne...> wrote: >> I don't have the dev environment for this in place, it would mean >> additional time for me to set this up. > > > sure, there is no hurry. > > thanks. > Q > > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > _______________________________________________ > Capstone-users mailing list > Cap...@li... > https://lists.sourceforge.net/lists/listinfo/capstone-users |
|
From: Capstone E. <cap...@gm...> - 2014-12-14 20:44:59
|
On Thu, Dec 4, 2014 at 10:14 PM, Nguyen Anh Quynh <aq...@gm...> wrote: > > > > On Thu, Dec 4, 2014 at 9:08 PM, Jan Newger <jan...@ne...> wrote: > >> I don't have the dev environment for this in place, it would mean >> additional time for me to set this up. >> > > > sure, there is no hurry. > Jan, do you have any progress on this? thanks. |
|
From: Jan N. <jan...@ne...> - 2014-12-14 20:59:53
Attachments:
python_mem_leak_fix.diff
|
On 14.12.2014 21:44, Capstone Engine wrote: > > > On Thu, Dec 4, 2014 at 10:14 PM, Nguyen Anh Quynh <aq...@gm... > <mailto:aq...@gm...>> wrote: > > > > On Thu, Dec 4, 2014 at 9:08 PM, Jan Newger <jan...@ne... > <mailto:jan...@ne...>> wrote: > > I don't have the dev environment for this in place, it would mean > additional time for me to set this up. > > > > sure, there is no hurry. > > > Jan, do you have any progress on this? > > > thanks. > Patch is attached. |
|
From: Capstone E. <cap...@gm...> - 2014-12-15 03:48:48
|
On Mon, Dec 15, 2014 at 4:59 AM, Jan Newger <jan...@ne...> wrote: > > On 14.12.2014 21:44, Capstone Engine wrote: > > > > > > On Thu, Dec 4, 2014 at 10:14 PM, Nguyen Anh Quynh <aq...@gm... > > <mailto:aq...@gm...>> wrote: > > > > > > > > On Thu, Dec 4, 2014 at 9:08 PM, Jan Newger <jan...@ne... > > <mailto:jan...@ne...>> wrote: > > > > I don't have the dev environment for this in place, it would mean > > additional time for me to set this up. > > > > > > > > sure, there is no hurry. > > > > > > Jan, do you have any progress on this? > > > > > > thanks. > > > Patch is attached. > for the record, do you want to send a PR on Github? thanks, Q |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X