pythoncad-developer

[Pythoncad-developer] Pushed many changes to the R38 branch

[<ger...@gm...>]

Hello,

I just pushed many changed into the R38 repository.

This means the old drawing methods are mostly replaced by the new cairo  
ones.
All draw functions (entity draw and rubberbanding and future selection  
draw) are now in the <entity>draw.py modules.

Not all entity draw functions are replaced by new ones, I will convert them  
one by one the next days.
There are issues with drawings when a very large part of the entity is off  
screen (maybe need some clipping).
There are issues with rubberbanding which can be slow at some times (after  
dynamic zoom with mouse wheel while drawing).
Mouse wheel zoom is pointer position sensitive, zooming is done around the  
mouse pointer and not screen center.

At this stage the cairo drawing should give a good indication of the  
performance which can be achieved.
I have not done any optimizations but I don't think it will be more than  
1.5x faster as it is now.
At least the panning is faster now (but after pan a redraw it needed which  
can be slow for large drawings).
Also each dynamic zoom with the wheel mouse button requires a redraw.
When zoomed into a small part of the drawing less entities are drawn than  
redraw is faster, whole drawing redraw is still slow.

Please look at the drawing performance and share your feelings (is it fast  
enough, does it need changes).
I think for really good performance you need display lists as used in  
opengl.

Regards,
Gertwin

Re: [Pythoncad-developer] Pushed many changes to the R38 branch

[Matteo B. <mat...@bo...>]

Hi Gertwin,
I've done some test and is seems to go faster then the previous version.

It's hard to make some test without some drawing functionality like
lines circles ....

I've done some tests importing the dxf files under doc folder and it
seems faster then the older system but not as much as I expected.

In my installation the pan mouse functionality dose not work.
And i get some strange message when i close PythonCad

__imageRemovedChild
remaining connections for obj: <PythonCAD.Generic.layer.Layer object at
0x97a1d6c>
message: added_child
connected to obj: <PythonCAD.Interface.Gtk.gtkimage.GTKImage object at
0x97ab68c>
message: removed_child
connected to obj: <PythonCAD.Interface.Gtk.gtkimage.GTKImage object at
0x97ab68c>


>At least the panning is faster now (but after pan a redraw it needed
>which can be slow for large drawings).

Could be use Tread for beach Redraw ? so the user can work and in the
batch mode we get the redraw.
I made some Test with treading while importing dxf file but i get some
bad error and i stop developing it.
I can make some test on it to get it work ..

>I think for really good performance you need display lists as used in
>opengl.

We can think of it .. i like to do e good job and performance in
zooming/redrawing are very important in a cad program.
So if you think that the display list will be the right choice.. we can
work on it we can share the work.

Regards,
Matteo

 

On Thu, 2010-01-07 at 22:26 +0000, ger...@gm... wrote:
> Hello,
> 
> I just pushed many changed into the R38 repository.
> 
> This means the old drawing methods are mostly replaced by the new
> cairo ones.
> All draw functions (entity draw and rubberbanding and future selection
> draw) are now in the <entity>draw.py modules.
> 
> Not all entity draw functions are replaced by new ones, I will convert
> them one by one the next days.
> There are issues with drawings when a very large part of the entity is
> off screen (maybe need some clipping).
> There are issues with rubberbanding which can be slow at some times
> (after dynamic zoom with mouse wheel while drawing).
> Mouse wheel zoom is pointer position sensitive, zooming is done around
> the mouse pointer and not screen center. 
> 
> At this stage the cairo drawing should give a good indication of the
> performance which can be achieved.
> I have not done any optimizations but I don't think it will be more
> than 1.5x faster as it is now.
> At least the panning is faster now (but after pan a redraw it needed
> which can be slow for large drawings).
> Also each dynamic zoom with the wheel mouse button requires a redraw.
> When zoomed into a small part of the drawing less entities are drawn
> than redraw is faster, whole drawing redraw is still slow.
> 
> Please look at the drawing performance and share your feelings (is it
> fast enough, does it need changes).
> I think for really good performance you need display lists as used in
> opengl.
> 
> Regards,
> Gertwin

Re: [Pythoncad-developer] Pushed many changes to the R38 branch

[<ger...@gm...>]

Hi Matteo,

Draw functionality like segment, circle, polyline etc. should work, text  
and dimension not.

At this point I did not expect a performance improvement for redraw.
The method of drawing is basically the same as it was except that is now  
done by a few functions in viewport.py.

Panning works only from the menu at this point, not with the mouse buttons.
The rendering is now done to a pixmap and not directly to the screen.
When panning the pixmap is moved around the screen without the need of a  
redraw.
When panning stops a redraw is needed for the visible area of the drawing.

The only thing I did until now was a rewrite of the rendering, all it is  
now all done by cairo.
I did not expect the dxf import was faster (except maybe for rendering at  
the end).

The strange messages when closing PythonCAD I don't know where the came  
from.
I am not aware of any changes in that area.
I think they are from the image class disposing entities from layers, this  
cost a very long time with large drawings.

Last night I did some more thinking on how display performance can be  
improved.
One thing is that when a redraw is done all entites are queried from the  
image class and rendered on the pixmap canvas.
I think getting the coordinates from the entities and translate them to  
cairo paths cost some time.
Also PythonCAD itself does the mapping from world to screen coordinates  
(without the use of the cairo matrix).
Maybe a large performance gain can be achieved if we once query the  
entities and store the cairo rendering paths in lists.
If we make use of the cairo transformation matrix for zooming and  
translation we only have to replay the paths in the lists without accessing  
the individual entities.
This is a little like the opengl display lists work.

At this point I am most concerned about the large memory consumption and  
the time needed to unload (garbage collection?) a drawing from PytonCAD.
I posted this a week ago on the pythoncad-developer list, a small dxf file  
leads to a memory use of almost 1GB.

I know PythonCAD is now a single threaded application which uses only one  
of my the available cpu cores.
But don't you think stuff will be much more difficult when we go multi  
threaded?

I think first we must get the core (generic + rendering) right before  
adding new functionality.
Maybe we can first look at this and decide what to do to get this right.

We can always rewrite the package Generic + rendering in C/C++ as an  
dynamic python extension library (just a thought).
The Interface package and the dxf/dwg import/export can remain in python.

If you find some time the next days please look the the performance and  
memory consumption of loading/unloading drawings.
I will do the same, we can discuss these things on the python-developer  
mailing list and hopefully also get some input from others.


Regards,
Gertwin










Op schreef Matteo Boscolo <mat...@bo...>:
> Hi Gertwin,

> I've done some test and is seems to go faster then the previous version.



> It's hard to make some test without some drawing functionality like

> lines circles ....



> I've done some tests importing the dxf files under doc folder and it

> seems faster then the older system but not as much as I expected.



> In my installation the pan mouse functionality dose not work.

> And i get some strange message when i close PythonCad



> __imageRemovedChild

> remaining connections for obj:
> 0x97a1d6c>

> message: added_child

> connected to obj:
> 0x97ab68c>

> message: removed_child

> connected to obj:
> 0x97ab68c>





> >At least the panning is faster now (but after pan a redraw it needed

> >which can be slow for large drawings).



> Could be use Tread for beach Redraw ? so the user can work and in the

> batch mode we get the redraw.

> I made some Test with treading while importing dxf file but i get some

> bad error and i stop developing it.

> I can make some test on it to get it work ..



> >I think for really good performance you need display lists as used in

> >opengl.



> We can think of it .. i like to do e good job and performance in

> zooming/redrawing are very important in a cad program.

> So if you think that the display list will be the right choice.. we can

> work on it we can share the work.



> Regards,

> Matteo







> On Thu, 2010-01-07 at 22:26 +0000, ger...@gm... wrote:

> > Hello,

> >

> > I just pushed many changed into the R38 repository.

> >

> > This means the old drawing methods are mostly replaced by the new

> > cairo ones.

> > All draw functions (entity draw and rubberbanding and future selection

> > draw) are now in the draw.py modules.

> >

> > Not all entity draw functions are replaced by new ones, I will convert

> > them one by one the next days.

> > There are issues with drawings when a very large part of the entity is

> > off screen (maybe need some clipping).

> > There are issues with rubberbanding which can be slow at some times

> > (after dynamic zoom with mouse wheel while drawing).

> > Mouse wheel zoom is pointer position sensitive, zooming is done around

> > the mouse pointer and not screen center.

> >

> > At this stage the cairo drawing should give a good indication of the

> > performance which can be achieved.

> > I have not done any optimizations but I don't think it will be more

> > than 1.5x faster as it is now.

> > At least the panning is faster now (but after pan a redraw it needed

> > which can be slow for large drawings).

> > Also each dynamic zoom with the wheel mouse button requires a redraw.

> > When zoomed into a small part of the drawing less entities are drawn

> > than redraw is faster, whole drawing redraw is still slow.

> >

> > Please look at the drawing performance and share your feelings (is it

> > fast enough, does it need changes).

> > I think for really good performance you need display lists as used in

> > opengl.

> >

> > Regards,

> > Gertwin

Re: [Pythoncad-developer] Pushed many changes to the R38 branch

[Matteo B. <mat...@bo...>]

Hi Gertwin,

> 
> Draw functionality like segment, circle, polyline etc. should work,
> text and dimension not.
> 
I found an error in the gtkImage:
the init collection is defined as 
__inittool = {
        Tools.PasteTool : cmd.paste_mode_init,
this is not right, 
the method that set the tool:

    def __imageToolChanged(self, obj, *args):
        _tool = self.__image.getTool()
        if _tool is not None:
            _init = GTKImage.__inittool.get(type(_tool))
            #here the _init is none because the tool is 
            #Tools.paste_tool.PasteTool
     if _init is not None:
                _init(self)

__inittool = {
        Tools.paste_tool.PasteTool : cmd.paste_mode_init,

I have done this fix in my local repository and now it works well.
let me know witch is the difference in your  code ... 
I wold like to commit this change but i do not understand why your code
works fine en i get this errors !?!?

> 
> At this point I did not expect a performance improvement for redraw.
> The method of drawing is basically the same as it was except that is
> now done by a few functions in viewport.py.
ok me too


> Panning works only from the menu at this point, not with the mouse
> buttons.
After some tests it look very fast even if with a lot of entity in the
drawing.

> The rendering is now done to a pixmap and not directly to the screen.
> When panning the pixmap is moved around the screen without the need of
> a redraw.
> When panning stops a redraw is needed for the visible area of the
> drawing.
> 
> The only thing I did until now was a rewrite of the rendering, all it
> is now all done by cairo.
> I did not expect the dxf import was faster (except maybe for rendering
> at the end).
me to .. 

> The strange messages when closing PythonCAD I don't know where the
> came from.
I still have some strange message even if i draw entity and zooming ...
may be we have to make some fix on it ..

> I am not aware of any changes in that area.
> I think they are from the image class disposing entities from layers,
> this cost a very long time with large drawings.
> 
> Last night I did some more thinking on how display performance can be
> improved.
> One thing is that when a redraw is done all entites are queried from
> the image class and rendered on the pixmap canvas.
> I think getting the coordinates from the entities and translate them
> to cairo paths cost some time.
> Also PythonCAD itself does the mapping from world to screen
> coordinates (without the use of the cairo matrix).
> Maybe a large performance gain can be achieved if we once query the
> entities and store the cairo rendering paths in lists.
> If we make use of the cairo transformation matrix for zooming and
> translation we only have to replay the paths in the lists without
> accessing the individual entities.
> This is a little like the opengl display lists work.
It seems an clever idea we can do some tests on it ..
I can look in deep  to the pytoncad entity and try to find a solution.
may be we can optimize the way haw pythoncad get the entity.
Using a db will be a solution !?!?!

> 
> At this point I am most concerned about the large memory consumption
> and the time needed to unload (garbage collection?) a drawing from
> PytonCAD.
> I posted this a week ago on the pythoncad-developer list, a small dxf
> file leads to a memory use of almost 1GB.
Witch dxf file ? could you please commit it in the doc folder on git ?
I would like to make some tests.

During dxf import I create an array to store all the error that we find
in the dxf file. may be we do not destroy in a well manner the obj that
read the dxf..


> 
> I know PythonCAD is now a single threaded application which uses only
> one of my the available cpu cores.
> But don't you think stuff will be much more difficult when we go multi
> threaded?
> 
Of course but all the new cad system works with multi tread ..
so it will be nice to have such a functionality .. so we can have batch
working ..es (When Solidworks create the drawing view or open a big 3d
assembly it use the multitread system for generete the view or the
hystory tree of the assembly).
I'm not so skilled in multi thread application, but i can do some test..


> I think first we must get the core (generic + rendering) right before
> adding new functionality.
right
> Maybe we can first look at this and decide what to do to get this
> right.
ok.

I think that the pan is very fast .
I did some test importing more then one dxf file(it means a lot of
entity) and the pan working fast.
The well zoom is still working slow may be there is to match redraw (one
for each step?).
It's possible to scale the pixmap insted of redraw the entire drawing
area?


> We can always rewrite the package Generic + rendering in C/C++ as an
> dynamic python extension library (just a thought).
I agree with you, but we need to have all the basic feature ready ..
and a well defined structure of the core application before translate
the code in c/c++.

> The Interface package and the dxf/dwg import/export can remain in
> python.
> 
> If you find some time the next days please look the the performance
> and memory consumption of loading/unloading drawings.
ok i will do some test an let you know
> I will do the same, we can discuss these things on the
> python-developer mailing list and hopefully also get some input from
> others.

Regards,
Matteo

Re: [Pythoncad-developer] Pushed many changes to the R38 branch

[<ger...@gm...>]

Hi Matteo,

I did not do a thorough test on my submitted code, so there are bugs in it.
If you find a bug and repaired it please push the changes back to the  
repository.

The file which gives me performance problems is "8th floor furniture.dxf",  
it is already in git.
I don't think the import of dxf is the problem, the problem remains even  
after saving to the PythonCAD xml format.
In native PythonCAD format the file "8th floor furniture.xml.gz" has a  
filesize of 850 kb (and still the same memory usage).
I will also push a larger dxf file (9 mb) to the repository, it contain  
xdata which gives an exception while loading.
Maybe you can use this file to test dxf import.

We can start with two threads, one separate for background processes like  
loading drawings/dxf and rendering and one for the user interface.
This will keep the user interface "alive" during long processes.

Maybe scaling of the image is a solution for mouse wheel zoom.
But a redraw is needed at the end just like with pan.

I looked into the way the entites are stored within the image/layer, this  
is very complex with many callback functions, relations and undo objects  
bound to every entity.
I have to investigate this more but maybe this is the source of the memory  
consumption.
If I have a clear view of how the image/layer/entity structures are  
organized I make a document of it and push it to the repository.
After that we can discuss where we think the bottle neck is.

Your remark about using a db is interesting.
A sqlite db can be used in memory and we have to write/maintain less code.

Regards,
Gertwin








On Jan 9, 2010 9:29am, Matteo Boscolo <mat...@bo...> wrote:
> Hi Gertwin,



> >

> > Draw functionality like segment, circle, polyline etc. should work,

> > text and dimension not.

> >

> I found an error in the gtkImage:

> the init collection is defined as

> __inittool = {

> Tools.PasteTool : cmd.paste_mode_init,

> this is not right,

> the method that set the tool:



> def __imageToolChanged(self, obj, *args):

> _tool = self.__image.getTool()

> if _tool is not None:

> _init = GTKImage.__inittool.get(type(_tool))

> #here the _init is none because the tool is

> #Tools.paste_tool.PasteTool

> if _init is not None:

> _init(self)



> __inittool = {

> Tools.paste_tool.PasteTool : cmd.paste_mode_init,



> I have done this fix in my local repository and now it works well.

> let me know witch is the difference in your code ...

> I wold like to commit this change but i do not understand why your code

> works fine en i get this errors !?!?



> >

> > At this point I did not expect a performance improvement for redraw.

> > The method of drawing is basically the same as it was except that is

> > now done by a few functions in viewport.py.

> ok me too





> > Panning works only from the menu at this point, not with the mouse

> > buttons.

> After some tests it look very fast even if with a lot of entity in the

> drawing.



> > The rendering is now done to a pixmap and not directly to the screen.

> > When panning the pixmap is moved around the screen without the need of

> > a redraw.

> > When panning stops a redraw is needed for the visible area of the

> > drawing.

> >

> > The only thing I did until now was a rewrite of the rendering, all it

> > is now all done by cairo.

> > I did not expect the dxf import was faster (except maybe for rendering

> > at the end).

> me to ..



> > The strange messages when closing PythonCAD I don't know where the

> > came from.

> I still have some strange message even if i draw entity and zooming ...

> may be we have to make some fix on it ..



> > I am not aware of any changes in that area.

> > I think they are from the image class disposing entities from layers,

> > this cost a very long time with large drawings.

> >

> > Last night I did some more thinking on how display performance can be

> > improved.

> > One thing is that when a redraw is done all entites are queried from

> > the image class and rendered on the pixmap canvas.

> > I think getting the coordinates from the entities and translate them

> > to cairo paths cost some time.

> > Also PythonCAD itself does the mapping from world to screen

> > coordinates (without the use of the cairo matrix).

> > Maybe a large performance gain can be achieved if we once query the

> > entities and store the cairo rendering paths in lists.

> > If we make use of the cairo transformation matrix for zooming and

> > translation we only have to replay the paths in the lists without

> > accessing the individual entities.

> > This is a little like the opengl display lists work.

> It seems an clever idea we can do some tests on it ..

> I can look in deep to the pytoncad entity and try to find a solution.

> may be we can optimize the way haw pythoncad get the entity.

> Using a db will be a solution !?!?!



> >

> > At this point I am most concerned about the large memory consumption

> > and the time needed to unload (garbage collection?) a drawing from

> > PytonCAD.

> > I posted this a week ago on the pythoncad-developer list, a small dxf

> > file leads to a memory use of almost 1GB.

> Witch dxf file ? could you please commit it in the doc folder on git ?

> I would like to make some tests.



> During dxf import I create an array to store all the error that we find

> in the dxf file. may be we do not destroy in a well manner the obj that

> read the dxf..





> >

> > I know PythonCAD is now a single threaded application which uses only

> > one of my the available cpu cores.

> > But don't you think stuff will be much more difficult when we go multi

> > threaded?

> >

> Of course but all the new cad system works with multi tread ..

> so it will be nice to have such a functionality .. so we can have batch

> working ..es (When Solidworks create the drawing view or open a big 3d

> assembly it use the multitread system for generete the view or the

> hystory tree of the assembly).

> I'm not so skilled in multi thread application, but i can do some test..





> > I think first we must get the core (generic + rendering) right before

> > adding new functionality.

> right

> > Maybe we can first look at this and decide what to do to get this

> > right.

> ok.



> I think that the pan is very fast .

> I did some test importing more then one dxf file(it means a lot of

> entity) and the pan working fast.

> The well zoom is still working slow may be there is to match redraw (one

> for each step?).

> It's possible to scale the pixmap insted of redraw the entire drawing

> area?





> > We can always rewrite the package Generic + rendering in C/C++ as an

> > dynamic python extension library (just a thought).

> I agree with you, but we need to have all the basic feature ready ..

> and a well defined structure of the core application before translate

> the code in c/c++.



> > The Interface package and the dxf/dwg import/export can remain in

> > python.

> >

> > If you find some time the next days please look the the performance

> > and memory consumption of loading/unloading drawings.

> ok i will do some test an let you know

> > I will do the same, we can discuss these things on the

> > python-developer mailing list and hopefully also get some input from

> > others.



> Regards,

> Matteo