[Pythoncad-developer] PythonCAD performance

[Gertwin G. <ger...@gm...>]

Hi Matteo,

I think I have found the part that is (for a large part) responsible
for the bad performance.

The file imageio.py contain a function:

def load_image(image, filehandle):
    ...
    _doc = xml.dom.minidom.parse(filehandle)
    ...

Processing large xml files with a dom parser gives a very bad
performance and costs lost of memory.
It is better to use the expat parser (or an sax parser) for large files.

At work I have a very good experience with the expat parser.
I have written an arx (C++) program under AutoCAD which processes a
200+ mb xml file containing over 200.000 entities in a few (3 to 4)
minutes.

Before and after the call to _doc = xml.dom.minidom.parse(filehandle):
Memory before: ~ 30 mb.
Memory after: ~535 mb.

This was done when opening the 850 kb "8th floor furniture.xml.gz"
file (translated from dxf).
I think this call alone is responsible for more than half the total
memory usage.

At this moment I think that the chosen dom parser and the complex way
entities are stored in the image/layer are responsible for the bad
performance.

So question is what are we going to do:
1. Replace the dom parser by the expat parser.
2. Simplify the entity storage in the image/layer objects.
3. Other?

What are your or anyone else thoughts on this?

Regards,
Gertwin