Re: [Pythoncad-developer] New kernel

[Matteo B. <mat...@bo...>]

Hi Gertwin,


You have produced a lot of code, I am still behind, no the interface code
does not display anything yet.
Take your time, it's not a job this . so we do what we can and when we can .


I think also that the pickle of larger objects will take more time.
Maybe with polylines containing a large number of points it will take longer
to load into the database.
I think the some ..

We can create a entityPoint table as you suggest ..

So when we create a new entity instead of picling the array of point we
will insert in a separate table ..like this ..

Id | idEnt | x | y | z

I Will do some test to see what is fast . 


I am satisfied with the performance so far, I still think it is a good
decision to use a database.
Me too .

But I would like to see it in action J


I don't think insertion of entities in a database containing 2 million
entities will take more time than an empty database.
Also query a large database must not be slow otherwise the indexes are not
on the right columns.
You are right but I do not know the performance of sqlite ..

 

At this moment I am writing the code to populate the quadtrees from the
PyCadDbKernel.
I made a copy of the quadtree Art has implemented, for the time being these
are in memory, maybe at a later stage we can put them in the db.
How do you need the quad tree structure ?

It's not so clear for me the use that you want do to of  the quad tree .

If It's used for intersection or to retrieve an entity from the user
interface I think we do not need it ..

I add the bbox (Need to finish it) for each entity so with a simple select
we can get all the entity near a picked point in the user interface ..


For my construction of the display lists I need a function that retrieves
all layers from the database.

You can use  pyKadKernel.getEntityFromType('LAYER') this retrieve all the
layer in the db (I need to improve this function . actually takes all the
revision of all the layer .. I need to improve this select with more
constraint).

 

I would like to discuss with you regarding the undo and versioning of the
entity. 

I will sent to you an e mail with this thread.


I did not see such a function, maybe I overlooked it, otherwise I try to
implement it tomorrow evening in the PyCadDbKernel.
It's olready there use  pyKadKernel.getEntityFromType('LAYER')


I have some remarks on your code, please don't take this personally.

I think that we can grow each other .. so please make all the remarks that
you want


I know this is still test code and maybe you already aware of it.
For instance this member from PyCadDbKernel:

def getEntLayer(self,layerName):
"""
get the pycadent of type layer
"""
self.__logger.debug('getEntLayer')
_settingsObjs=self.getEntityFromType('LAYER')
if len(_settingsObjs)<=0:
raise EntityMissing,"No Layer object in the db"
else:
>> for sto in _settingsObjs:
>> _setts=sto.getConstructionElements()
>> for i in _setts:
>> if _setts[i].name==self.__settings.layerName:
>> return sto
else:
raise EntityMissing,"Layer name %s missing"%str(layerName) 

After the function getEntityFromType that queries the database there is a
nested loop that picks the right entity.
This implies that there are multiple records selected from the database.
If we want to support MySQL in the future this means all those selected
records are transported from the database server to the client computer.
To get the best possible performance if we need one record from the
database, build a query that selects only one record.
I don't' know how to improve this code ..

The name of the layer is inside the pickle entity .. so I need to get all
the layer and then ask to the unpickle elemente the name ..

 

We can store the entity type with the name like this 'LAYER_MAIN' or
"LAYER_PIPE" but I do not like this sort of naming conversion for the entity
type.

Any idea?


Also please use more understandable variable names so I can see what they
mean.
For me it is hard to understand what _setts and sto stands for.
Now I have to look into the members getEntityFromType and
getConstructionElements to get an idea.
Yes you are right that part of code is very bad ..:-) .. I did it very fast
. I will take more attention on naming ..

 

Regards,

Matteo


Op schreef Matteo Boscolo <mat...@bo...>:
> Hi Gertwin,
> 
> 
> 
> I had implemented the layer creation and now the entity are written in
> 
> the entity table, undo table and layer table ..
> 
> here you have some performance test
> 
> 
> 
> Create pycad object
> 
> Points:
> 
> Create n: 1 entity in : 0.0s
> 
> Create n: 10 entity in : 0.0s
> 
> Create n: 100 entity in : 0.02s
> 
> Create n: 1000 entity in : 0.18s
> 
> Create n: 10000 entity in : 1.77s
> 
> Create n: 100000 entity in : 18.09s
> 
> Segments:
> 
> Create n: 1 entity in : 0.0s
> 
> Create n: 10 entity in : 0.0s
> 
> Create n: 100 entity in : 0.02s
> 
> Create n: 1000 entity in : 0.22s
> 
> Create n: 10000 entity in : 2.03s
> 
> Create n: 100000 entity in : 21.91s
> 
> Script terminated.
> 
> 
> 
> Create pycad object
> 
> Points:
> 
> Create n: 1 entity in : 0.0s
> 
> Create n: 10 entity in : 0.01s
> 
> Create n: 100 entity in : 0.03s
> 
> Create n: 1000 entity in : 0.17s
> 
> Create n: 10000 entity in : 1.75s
> 
> Create n: 100000 entity in : 18.58s
> 
> Segments:
> 
> Create n: 1 entity in : 0.0s
> 
> Create n: 10 entity in : 0.0s
> 
> Create n: 100 entity in : 0.02s
> 
> Create n: 1000 entity in : 0.23s
> 
> Create n: 10000 entity in : 2.09s
> 
> Create n: 100000 entity in : 22.44s
> 
> Script terminated.
> 
> 
> 
> At the end of the script we have the following entity in the db
> 
> pycadent 691117
> 
> pycadrel 691115
> 
> pycadundo 56 
> for a single bulk operation.
> 
> 
> 
> As you can see the performance are stable even if we already have more
> 
> that 690000 entity...
> 
> 
> 
> I think for the normal operation we do not have any problem for speed
> 
> storing entity.
> 
> 
> 
> I would like to do more test and see what append if we have more that
> 
> 2.000.000 entity .. that could be the case with all the undo and so an.
> 
> 
> 
> as you can see the time for storing segment is more than storing
> 
> points ...
> 
> may be this time is due to the pickling ...
> 
> 
> 
> now you can get the entire child entity id from the kernel using the
> 
> 
> 
> pycadkernel.getLayerChild(LayerName)
> 
> def getChildrenIds(self,entityParentId):
> 
>        """
> 
>            Get the children id of a relation
> 
>        """
> 
>        _outObj=[]
> 
>        _sqlGet="""SELECT pycad_child_id
> 
>                FROM pycadrel
> 
>                WHERE pycad_parent_id=%s"""%str(entityParentId)
> 
>        _rows=self.makeSelect(_sqlGet)
> 
>        _dbEntRow=self.makeSelect(_sqlGet)
> 
>        if _dbEntRow is not None:
> 
>            for _row in _dbEntRow:
> 
>                _outObj.append(_row[0])
> 
>        return _outObj
> 
> 
> 
> Get n: 691115 layer entity in : 1.33s
> 
> this is the time that it need to do such an operation.
> 
> 
> 
> Fill free to add function to my class if you need it ..
> 
> 
> 
> This is the first version .. so please add comment and improvements on
> 
> my work..
> 
> 
> 
> Regards,
> 
> Matteo
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
>