Version 0.4.2.0 uploaded

[John Källén]

Some highlights:
. Added support for PS3 and System V ELF files.

. For fun: a C64 BASIC "decompiler"! Give it a D64 Commodore disk image, navigate to a program, and load it. It will translate the BASIC instructions into intermediate code, and then generates pseudo-C from it.

. Fixed the following submitted bugs
- Can not load ELF or Amiga Hunk binaries
- Can not run the decompiler from the command line
. Implemented more x86 and PowerPC instructions
. Improvements in SSA transformation code in preparation for
move to new decompilation model
. ImageSegmentRenders allow viewing of the structured data
in image segments.

[Raimar 'second' Falke]

Thanks for the release. I wrote a small script to summarize the problems
the code has with the various subjects. I'm not sure about the .net
exception format so it may be wrong a little bit.

Hope this helps.

Regards,

Raimar

../decompiler-subjects> cat scripts/summarize_decompiler.sf.net.py
#!/usr/bin/python

import sys,re

content=sys.stdin.read()
m={}
for key in [" ; ".join(x) for x in re.findall("^    ([^ ].*)\n       (at .*)\n   (at .*)", content, re.M)]:
  m[key]=m.get(key,0)+1
keys=sorted(m.keys(), lambda x,y:cmp(m[y], m[x]))
for key in keys:
  print "%4d %s"%(m[key],key)
../decompiler-subjects> find . -name by_decompiler.sf.net.out  | xargs cat | python scripts/summarize_decompiler.sf.net.py 2>/dev/null| head -n 20
 354 The program's Architecture property must be set before accessing the Globals property. ; at Decompiler.Core.Program.get_Globals() ; at Decompiler.Core.Output.GlobalDataWriter.WriteGlobals(Formatter formatter)
 354 Object reference not set to an instance of an object. ; at Decompiler.Core.Platform.get_PointerType() ; at Decompiler.Typing.TypeAnalyzer.RewriteProgram(Program prog)
 295 The given key was not present in the dictionary. ; at System.Collections.Generic.Dictionary`2.get_Item(TKey key) ; at Decompiler.Analysis.ExpressionPropagator.GetStackDepthBeforeCall()
 292 The given key was not present in the dictionary. ; at System.Collections.Generic.Dictionary`2.get_Item(TKey key) ; at Decompiler.Core.Lib.DominatorGraph`1.BuildDominanceFrontiers(DirectedGraph`1 graph, Dictionary`2 idoms)
 245 Expected stack depth to be negative. ; at Decompiler.Analysis.ExpressionPropagator.GetStackDepthBeforeCall() ; at Decompiler.Analysis.ExpressionPropagator.VisitCallInstruction(CallInstruction ci)
 208 The method or operation is not implemented. ; at Decompiler.Core.Output.GlobalDataWriter.VisitUnion(UnionType ut) ; at Decompiler.Core.Output.GlobalDataWriter.VisitEquivalenceClass(EquivalenceClass eq)
 176 Object reference not set to an instance of an object. ; at Decompiler.Typing.TypedExpressionRewriter.RewriteFormals(ProcedureSignature sig) ; at Decompiler.Typing.TypedExpressionRewriter.RewriteProgram(Program prog)
  71 The method or operation is not implemented. ; at Decompiler.ImageLoaders.MachO.MachOLoader.get_PreferredBaseAddress() ; at Decompiler.Loading.Loader.LoadExecutable(String filename, Byte[] image, Address addrLoad)
  67 Unable to cast object of type 'Decompiler.Core.Code.ReturnInstruction' to type 'Decompiler.Core.Code.Branch'. ; at Decompiler.Structure.AbsynCodeGenerator.BranchCondition(StructureNode node) ; at Decompiler.Structure.IfConditional.GenerateCode(AbsynCodeGenerator codeGen, StructureNode node, StructureNode latchNode, AbsynStatementEmitter emitter)
  66 Drill into struct ; at Decompiler.Core.Output.GlobalDataWriter.VisitPointer(Pointer ptr) ; at Decompiler.Core.Output.GlobalDataWriter.WriteGlobals(Formatter formatter)
  65 Unable to cast object of type 'Decompiler.Core.Types.StructureType' to type 'Decompiler.Core.Types.EquivalenceClass'. ; at Decompiler.Core.Output.GlobalDataWriter.WriteGlobals(Formatter formatter) ; at Decompiler.DecompilerDriver.WriteGlobals(Program program, TextWriter w)
  48 Relocation type 5 not handled yet ; at Decompiler.ImageLoaders.Elf.ElfImageLoader.RelocateI386() ; at Decompiler.ImageLoaders.Elf.ElfImageLoader.Relocate(Address addrLoad)
  46 Object reference not set to an instance of an object. ; at Decompiler.Core.Configuration.DecompilerConfiguration.GetArchitecture(String archLabel) ; at Decompiler.ImageLoaders.Elf.ElfImageLoader.CreateArchitecture(UInt16 machineType)
  40 The method or operation is not implemented. ; at Decompiler.Core.Output.GlobalDataWriter.VisitCode(CodeType c) ; at Decompiler.Core.Output.GlobalDataWriter.WriteGlobals(Formatter formatter)
  38 Processor format 62 is not supported. ; at Decompiler.ImageLoaders.Elf.ElfImageLoader.CreateArchitecture(UInt16 machineType) ; at Decompiler.ImageLoaders.Elf.ElfImageLoader.LoadHeader()
  33 Unable to cast object of type 'Decompiler.Core.Types.PrimitiveType' to type 'Decompiler.Core.Types.EquivalenceClass'. ; at Decompiler.Core.Output.GlobalDataWriter.WriteGlobals(Formatter formatter) ; at Decompiler.DecompilerDriver.WriteGlobals(Program program, TextWriter w)
  32 An item with the same key has already been added. ; at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add) ; at Decompiler.ImageLoaders.Elf.ElfImageLoader.RelocateI386()
  22 The method or operation is not implemented. ; at Decompiler.Structure.AbsynStatementEmitter.Decompiler.Core.Code.InstructionVisitor.VisitDefInstruction(DefInstruction def) ; at Decompiler.Structure.AbsynCodeGenerator.EmitLinearBlockStatements(StructureNode node, AbsynStatementEmitter emitter)
  17 The given key was not present in the dictionary. ; at System.Collections.Generic.Dictionary`2.get_Item(TKey key) ; at Decompiler.Analysis.DeadCode.VisitIdentifier(Identifier id)
  14 Drill into struct ; at Decompiler.Core.Output.GlobalDataWriter.VisitPointer(Pointer ptr) ; at Decompiler.Core.Output.GlobalDataWriter.VisitStructure(StructureType str)
../decompiler-subjects> 

[John Källén]

Thanks for the script Raimar, it will prove handy.

I did some analysis of the programs that were causing the errors. It turns out that a large percentage of the programs that cause the "The program's Architecture property..." error message are the MS-DOS COM programs in the subdirectory from_pouet. Because a COM program has no identifiable executable magic number or header, like say EXE or ELF files, the Decompiler needs some additional hints in order to know what to do with it. As Decompiler targets many processor architectures, it can't simply blindly assume that a "header-less" file is an x86 COM file like some other decompilers do.

Today, I checked in some enhancements to the command-line interpreter. It now accepts a few options, which (once I've made a release) you can look at by typing decompile --help:

usage: decompile [options] <filename>
    <filename> can be either an executable file or a project file.

Options:
 --version               Show version number and exit
 -h, --help              Show this message and exit
 --arch <architecture>   Use an architecture from the following:
    arm                  ARM family
    m6502                MOS 6502
    m68k                 Motorola 68K family
    ppc32                PowerPC 32-bit
    ppc64                PowerPC 64-bit
    sparc                Sun SPARC family
    x86-protected-32     Intel x86 32-bit Protected Mode
    x86-real-16          Intel x86 16-bit Real Mode
    z80                  Zilog Z-80
 --env <environment>     Use an operating environment from the following:
    amigaOS              AmigaOS
    elf-cell-lv2         Playstation 3
    elf-neutral          Unspecified ELF ABI
    macOs                Mac OS Classic
    ms-dos               MS-DOS
    riscOS               RISC OS
    win32                Win32
    zx81                 ZX81
 --base <address>        Use <address> as the base address of the program
 --entry <address>       Use <address> as an entry point to the program
 --reg <regInit>         Set register to value, where regInit is formatted as
                          reg_name:value, e.g. sp:FF00

To get rid of many of those errors I mentioned, you can modify the script so that when you are decompiling MS-DOS COM programs, you use the following command line:

decompile --arch x86-real-16 --env ms-dos --base 0C00:0100 --reg ax:0 sample.exe

This tells the decompiler to force the architecture to 16-bit real mode x86, use conventions and interrupt lists from the MS-DOS operating environment, and loads the COM program image at address 0C00:0100 with the AX register set to 0.

I will be making a new release 0.4.3.0 later in the week. In the meantime, if you have the time, would you consider adapting your decompiler scripts so that this command is executed when dealing with COM programs? Naturally, if the executable in question has a valid executable header, it should be enough to just write

decompile sample.exe

as before.

[Raimar 'second' Falke]

Yes MS-DOS COM files are hard/impossible to detect. This is the reason holdec
has a command line option

--treat-unknown-files-as-ms-dos-com

which is a bit more user friendly compared to

--arch x86-real-16 --env ms-dos --base 0C00:0100 --reg ax:0