sqlobject-cvs — CVS notification

[SQL-CVS] [ sqlobject-Support Requests-3366899 ] The speed of SQLObject is slower on the time

[SourceForge.net <no...@so...>]

Support Requests item #3366899, was opened at 2011-07-14 11:40
Message generated for change (Comment added) made by phd
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=540673&aid=3366899&group_id=74338

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Priority: 5
Private: No
Submitted By: Rosa Robles (marobles)
Assigned to: Oleg Broytman (phd)
Summary: The speed of SQLObject is slower on the time

Initial Comment:
I had a test of a SQLObject and I had found a strange behaviour of SQLObject. On the time, the respond time is lesser.

Here, I put a example of test code :

from sqlobject.sqlbuilder import *
connection = connectionForURI('postgresql://test:test@localhost/test_sql_object')
sqlhub.processConnection = connection

class Person(SQLObject):
    firstName = StringCol()
    middleInitial = StringCol(length=1, default=None)
    lastName = StringCol()

iteration = 0
while 1:
    start = datetime.now()
    person_list = []
    for key in range(1000):
        person = Person.selectBy(firstName='name%s%s' % (iteration,key))[0]
        person_list.append(person)
   
    for person in person_list:
        person.expire()
        del person
    del person_list
    sqlhub.processConnection.cache.clear()
   
    end = datetime.now()
       
    print "TIME ",iteration,':' , end - start
    iteration = iteration + 1

In the result I can check that in each iteration the time is bigger. The first iteration has time of 1.37s, then the time increases and the 19 iteration has time of 5.83 s!

TIME  0 : 0:00:01.376392
TIME  1 : 0:00:01.663250
TIME  2 : 0:00:01.978639
TIME  3 : 0:00:02.319939
TIME  4 : 0:00:02.618213
TIME  5 : 0:00:02.723846
TIME  6 : 0:00:02.584007
TIME  7 : 0:00:02.868877
TIME  8 : 0:00:03.051937
TIME  9 : 0:00:03.389757
TIME  10 : 0:00:03.503036
TIME  11 : 0:00:03.563154
TIME  12 : 0:00:04.191539
TIME  13 : 0:00:04.274464
TIME  14 : 0:00:04.813206
TIME  15 : 0:00:04.677002
TIME  16 : 0:00:04.995708
TIME  17 : 0:00:05.126811
TIME  18 : 0:00:05.571279
TIME  19 : 0:00:05.832840

Why is happend this? 
Thanks in advance for your attention.
Cheers Rosa

----------------------------------------------------------------------

>Comment By: Oleg Broytman (phd)
Date: 2011-07-19 19:21

Message:
To stop cache culling I added

connection.cache = CacheSet(cullFrequency=10**6)

to your test program. It didn't help. So it seems cache culling is not the
culprit. There is a need to do a deeper investigation. Can you run a python
profiler over your test(s)?

----------------------------------------------------------------------

Comment By: Rosa Robles (marobles)
Date: 2011-07-19 12:09

Message:
if we review the results,  after read a lots of records selectBy takes more
times, in the first iteration is almost zero but in 20 iteration is bigger.


If you do the example, and you takes the times without to call to
expireAll()  or clear(), you can check that the time c

    for iteration in range(20):
        start = datetime.now()
        for key in range(1000):
            person = Person.selectBy(firstName='name%s%s' %
(iteration,key))[0]
        end = datetime.now()
   
       print "ITERATION ",iteration, "TIME", end-start

       Person.sqlmeta.expireAll()
       sqlhub.getConnection().expireAll()
       connection.cache.clear()

----------------------------------------------------------------------

Comment By: Oleg Broytman (phd)
Date: 2011-07-18 21:50

Message:
time_end-time_start are consistently almost zero, so selectBy doesn't take
time, right?

----------------------------------------------------------------------

Comment By: Rosa Robles (marobles)
Date: 2011-07-18 10:36

Message:
The program takes time only from the selectby operation
for key in range(1000):
    time_start = datetime.now()
    person = Person.selectBy(firstName='name%s%s' %(iteration,key))[0]
    time_end = datetime.now()
    times.append(time_end-time_start)

I attach the example to you can read better

If we review the results, the selectBy method takes times random in
iteration, but also each iteration's time grows up. 
I think the problem is in the execution  of cull process when delete
expiredCache values, what do you think?

In my real program do not use select in mass but it can acumulate a lots
operations of select in the time, 
then  we have problems of memory and we must restart the application.
May be this situation is the source of our problem with the memory.

----------------------------------------------------------------------

Comment By: Oleg Broytman (phd)
Date: 2011-07-15 21:14

Message:
Min/max/avg times are almost zero, so it seems the only place the program
spends time is calling .expireAll() and cache.clear(), right?

----------------------------------------------------------------------

Comment By: Rosa Robles (marobles)
Date: 2011-07-15 10:52

Message:
Thanks Oleg.
I'm using SQLObject 1.1.1, Python2.6, Postgres 8.4

Ok, I change the code to takes more times.

    for iteration in range(20):
        start = datetime.now()
        acum = timedelta(microseconds=0)
        times = []
        
        for key in range(1000):
            time_start = datetime.now()
            person = Person.selectBy(firstName='name%s%s' %
(iteration,key))[0]
            time_end = datetime.now()
            times.append(time_end-time_start)
            acum = acum + (time_end-time_start)
            
        end = datetime.now()
        
        print "ITERATION ",iteration,"time",end-start
        print "   First Key  ->", times[0] 
        print "   Last Key   ->",times[999] 
        print "   Key        ->",times.index(max(times)), "   Max Time 
->", max(times)
        print "   Key        ->",times.index(min(times)), "   Min Time
->", min(times)
        print "   avg        ->",acum/999
        
        Person.sqlmeta.expireAll()
        for k in connection.cache.caches.keys():
            connection.cache.caches[k].expireAll()
        sqlhub.getConnection().expireAll()
        connection.cache.clear()
        gc.collect()


And the result is:


ITERATION  0 time 0:00:01.302290
   First Key  -> 0:00:00.001555
   Last Key   -> 0:00:00.001445
   Key        -> 11    Max Time  -> 0:00:00.003216
   Key        -> 36    Min Time -> 0:00:00.001115
   avg        -> 0:00:00.001287
ITERATION  1 time 0:00:01.621580
   First Key  -> 0:00:00.001879
   Last Key   -> 0:00:00.001734
   Key        -> 924    Max Time  -> 0:00:00.009572
   Key        -> 22    Min Time -> 0:00:00.001076
   avg        -> 0:00:00.001606
ITERATION  2 time 0:00:01.851719
   First Key  -> 0:00:00.002328
   Last Key   -> 0:00:00.002064
   Key        -> 970    Max Time  -> 0:00:00.002544
   Key        -> 22    Min Time -> 0:00:00.001306
   avg        -> 0:00:00.001836
ITERATION  3 time 0:00:02.040802
   First Key  -> 0:00:00.002435
   Last Key   -> 0:00:00.002376
   Key        -> 557    Max Time  -> 0:00:00.009674
   Key        -> 25    Min Time -> 0:00:00.001538
   avg        -> 0:00:00.002026
ITERATION  4 time 0:00:02.427911
   First Key  -> 0:00:00.002906
   Last Key   -> 0:00:00.002065
   Key        -> 774    Max Time  -> 0:00:00.003263
   Key        -> 7    Min Time -> 0:00:00.001763
   avg        -> 0:00:00.002413
ITERATION  5 time 0:00:02.768819
   First Key  -> 0:00:00.003051
   Last Key   -> 0:00:00.002319
   Key        -> 920    Max Time  -> 0:00:00.007694
   Key        -> 11    Min Time -> 0:00:00.002007
   avg        -> 0:00:00.002753
ITERATION  6 time 0:00:02.537283
   First Key  -> 0:00:00.002676
   Last Key   -> 0:00:00.002549
   Key        -> 2    Max Time  -> 0:00:00.003512
   Key        -> 132    Min Time -> 0:00:00.002318
   avg        -> 0:00:00.002523
ITERATION  7 time 0:00:02.723405
   First Key  -> 0:00:00.005272
   Last Key   -> 0:00:00.002788
   Key        -> 79    Max Time  -> 0:00:00.007543
   Key        -> 12    Min Time -> 0:00:00.002457
   avg        -> 0:00:00.002709
ITERATION  8 time 0:00:02.947676
   First Key  -> 0:00:00.003161
   Last Key   -> 0:00:00.003028
   Key        -> 1    Max Time  -> 0:00:00.006068
   Key        -> 31    Min Time -> 0:00:00.002683
   avg        -> 0:00:00.002933
ITERATION  9 time 0:00:03.781268
   First Key  -> 0:00:00.004271
   Last Key   -> 0:00:00.003416
   Key        -> 930    Max Time  -> 0:00:00.011791
   Key        -> 5    Min Time -> 0:00:00.003029
   avg        -> 0:00:00.003765
ITERATION  10 time 0:00:03.842353
   First Key  -> 0:00:00.006544
   Last Key   -> 0:00:00.003394
   Key        -> 0    Max Time  -> 0:00:00.006544
   Key        -> 4    Min Time -> 0:00:00.003171
   avg        -> 0:00:00.003826
ITERATION  11 time 0:00:03.361368
   First Key  -> 0:00:00.001509
   Last Key   -> 0:00:00.003626
   Key        -> 602    Max Time  -> 0:00:00.009093
   Key        -> 9    Min Time -> 0:00:00.001089
   avg        -> 0:00:00.003346
ITERATION  12 time 0:00:03.581802
   First Key  -> 0:00:00.001511
   Last Key   -> 0:00:00.003912
   Key        -> 474    Max Time  -> 0:00:00.009027
   Key        -> 1    Min Time -> 0:00:00.001117
   avg        -> 0:00:00.003566
ITERATION  13 time 0:00:03.797965
   First Key  -> 0:00:00.001833
   Last Key   -> 0:00:00.004118
   Key        -> 100    Max Time  -> 0:00:00.005115
   Key        -> 9    Min Time -> 0:00:00.001126
   avg        -> 0:00:00.003782
ITERATION  14 time 0:00:04.147961
   First Key  -> 0:00:00.001851
   Last Key   -> 0:00:00.004362
   Key        -> 782    Max Time  -> 0:00:00.013857
   Key        -> 16    Min Time -> 0:00:00.001186
   avg        -> 0:00:00.004132
ITERATION  15 time 0:00:04.251830
   First Key  -> 0:00:00.001538
   Last Key   -> 0:00:00.004611
   Key        -> 518    Max Time  -> 0:00:00.009511
   Key        -> 9    Min Time -> 0:00:00.001114
   avg        -> 0:00:00.004236
ITERATION  16 time 0:00:04.487687
   First Key  -> 0:00:00.001541
   Last Key   -> 0:00:00.004994
   Key        -> 858    Max Time  -> 0:00:00.009368
   Key        -> 9    Min Time -> 0:00:00.001120
   avg        -> 0:00:00.004472
ITERATION  17 time 0:00:04.703588
   First Key  -> 0:00:00.003440
   Last Key   -> 0:00:00.005104
   Key        -> 529    Max Time  -> 0:00:00.008179
   Key        -> 8    Min Time -> 0:00:00.001105
   avg        -> 0:00:00.004688
ITERATION  18 time 0:00:05.300920
   First Key  -> 0:00:00.001851
   Last Key   -> 0:00:00.006987
   Key        -> 854    Max Time  -> 0:00:00.007299
   Key        -> 9    Min Time -> 0:00:00.001181
   avg        -> 0:00:00.005285
ITERATION  19 time 0:00:06.168681
   First Key  -> 0:00:00.001880
   Last Key   -> 0:00:00.005577
   Key        -> 911    Max Time  -> 0:00:00.009503
   Key        -> 9    Min Time -> 0:00:00.001119
   avg        -> 0:00:00.006152








----------------------------------------------------------------------

Comment By: Oleg Broytman (phd)
Date: 2011-07-14 20:56

Message:
No idea what is going on. Let's investigate a little further. What version
of SQLObject are you using? Can you add more prints inside the loop to see
what part of the program takes time?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=540673&aid=3366899&group_id=74338

[SQL-CVS] [ sqlobject-Support Requests-3366899 ] The speed of SQLObject is slower on the time

[SourceForge.net <no...@so...>]

Support Requests item #3366899, was opened at 2011-07-14 09:40
Message generated for change (Comment added) made by marobles
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=540673&aid=3366899&group_id=74338

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Priority: 5
Private: No
Submitted By: Rosa Robles (marobles)
Assigned to: Oleg Broytman (phd)
Summary: The speed of SQLObject is slower on the time

Initial Comment:
I had a test of a SQLObject and I had found a strange behaviour of SQLObject. On the time, the respond time is lesser.

Here, I put a example of test code :

from sqlobject.sqlbuilder import *
connection = connectionForURI('postgresql://test:test@localhost/test_sql_object')
sqlhub.processConnection = connection

class Person(SQLObject):
    firstName = StringCol()
    middleInitial = StringCol(length=1, default=None)
    lastName = StringCol()

iteration = 0
while 1:
    start = datetime.now()
    person_list = []
    for key in range(1000):
        person = Person.selectBy(firstName='name%s%s' % (iteration,key))[0]
        person_list.append(person)
   
    for person in person_list:
        person.expire()
        del person
    del person_list
    sqlhub.processConnection.cache.clear()
   
    end = datetime.now()
       
    print "TIME ",iteration,':' , end - start
    iteration = iteration + 1

In the result I can check that in each iteration the time is bigger. The first iteration has time of 1.37s, then the time increases and the 19 iteration has time of 5.83 s!

TIME  0 : 0:00:01.376392
TIME  1 : 0:00:01.663250
TIME  2 : 0:00:01.978639
TIME  3 : 0:00:02.319939
TIME  4 : 0:00:02.618213
TIME  5 : 0:00:02.723846
TIME  6 : 0:00:02.584007
TIME  7 : 0:00:02.868877
TIME  8 : 0:00:03.051937
TIME  9 : 0:00:03.389757
TIME  10 : 0:00:03.503036
TIME  11 : 0:00:03.563154
TIME  12 : 0:00:04.191539
TIME  13 : 0:00:04.274464
TIME  14 : 0:00:04.813206
TIME  15 : 0:00:04.677002
TIME  16 : 0:00:04.995708
TIME  17 : 0:00:05.126811
TIME  18 : 0:00:05.571279
TIME  19 : 0:00:05.832840

Why is happend this? 
Thanks in advance for your attention.
Cheers Rosa

----------------------------------------------------------------------

Comment By: Rosa Robles (marobles)
Date: 2011-07-19 10:09

Message:
if we review the results,  after read a lots of records selectBy takes more
times, in the first iteration is almost zero but in 20 iteration is bigger.


If you do the example, and you takes the times without to call to
expireAll()  or clear(), you can check that the time c

    for iteration in range(20):
        start = datetime.now()
        for key in range(1000):
            person = Person.selectBy(firstName='name%s%s' %
(iteration,key))[0]
        end = datetime.now()
   
       print "ITERATION ",iteration, "TIME", end-start

       Person.sqlmeta.expireAll()
       sqlhub.getConnection().expireAll()
       connection.cache.clear()

----------------------------------------------------------------------

Comment By: Oleg Broytman (phd)
Date: 2011-07-18 19:50

Message:
time_end-time_start are consistently almost zero, so selectBy doesn't take
time, right?

----------------------------------------------------------------------

Comment By: Rosa Robles (marobles)
Date: 2011-07-18 08:36

Message:
The program takes time only from the selectby operation
for key in range(1000):
    time_start = datetime.now()
    person = Person.selectBy(firstName='name%s%s' %(iteration,key))[0]
    time_end = datetime.now()
    times.append(time_end-time_start)

I attach the example to you can read better

If we review the results, the selectBy method takes times random in
iteration, but also each iteration's time grows up. 
I think the problem is in the execution  of cull process when delete
expiredCache values, what do you think?

In my real program do not use select in mass but it can acumulate a lots
operations of select in the time, 
then  we have problems of memory and we must restart the application.
May be this situation is the source of our problem with the memory.

----------------------------------------------------------------------

Comment By: Oleg Broytman (phd)
Date: 2011-07-15 19:14

Message:
Min/max/avg times are almost zero, so it seems the only place the program
spends time is calling .expireAll() and cache.clear(), right?

----------------------------------------------------------------------

Comment By: Rosa Robles (marobles)
Date: 2011-07-15 08:52

Message:
Thanks Oleg.
I'm using SQLObject 1.1.1, Python2.6, Postgres 8.4

Ok, I change the code to takes more times.

    for iteration in range(20):
        start = datetime.now()
        acum = timedelta(microseconds=0)
        times = []
        
        for key in range(1000):
            time_start = datetime.now()
            person = Person.selectBy(firstName='name%s%s' %
(iteration,key))[0]
            time_end = datetime.now()
            times.append(time_end-time_start)
            acum = acum + (time_end-time_start)
            
        end = datetime.now()
        
        print "ITERATION ",iteration,"time",end-start
        print "   First Key  ->", times[0] 
        print "   Last Key   ->",times[999] 
        print "   Key        ->",times.index(max(times)), "   Max Time 
->", max(times)
        print "   Key        ->",times.index(min(times)), "   Min Time
->", min(times)
        print "   avg        ->",acum/999
        
        Person.sqlmeta.expireAll()
        for k in connection.cache.caches.keys():
            connection.cache.caches[k].expireAll()
        sqlhub.getConnection().expireAll()
        connection.cache.clear()
        gc.collect()


And the result is:


ITERATION  0 time 0:00:01.302290
   First Key  -> 0:00:00.001555
   Last Key   -> 0:00:00.001445
   Key        -> 11    Max Time  -> 0:00:00.003216
   Key        -> 36    Min Time -> 0:00:00.001115
   avg        -> 0:00:00.001287
ITERATION  1 time 0:00:01.621580
   First Key  -> 0:00:00.001879
   Last Key   -> 0:00:00.001734
   Key        -> 924    Max Time  -> 0:00:00.009572
   Key        -> 22    Min Time -> 0:00:00.001076
   avg        -> 0:00:00.001606
ITERATION  2 time 0:00:01.851719
   First Key  -> 0:00:00.002328
   Last Key   -> 0:00:00.002064
   Key        -> 970    Max Time  -> 0:00:00.002544
   Key        -> 22    Min Time -> 0:00:00.001306
   avg        -> 0:00:00.001836
ITERATION  3 time 0:00:02.040802
   First Key  -> 0:00:00.002435
   Last Key   -> 0:00:00.002376
   Key        -> 557    Max Time  -> 0:00:00.009674
   Key        -> 25    Min Time -> 0:00:00.001538
   avg        -> 0:00:00.002026
ITERATION  4 time 0:00:02.427911
   First Key  -> 0:00:00.002906
   Last Key   -> 0:00:00.002065
   Key        -> 774    Max Time  -> 0:00:00.003263
   Key        -> 7    Min Time -> 0:00:00.001763
   avg        -> 0:00:00.002413
ITERATION  5 time 0:00:02.768819
   First Key  -> 0:00:00.003051
   Last Key   -> 0:00:00.002319
   Key        -> 920    Max Time  -> 0:00:00.007694
   Key        -> 11    Min Time -> 0:00:00.002007
   avg        -> 0:00:00.002753
ITERATION  6 time 0:00:02.537283
   First Key  -> 0:00:00.002676
   Last Key   -> 0:00:00.002549
   Key        -> 2    Max Time  -> 0:00:00.003512
   Key        -> 132    Min Time -> 0:00:00.002318
   avg        -> 0:00:00.002523
ITERATION  7 time 0:00:02.723405
   First Key  -> 0:00:00.005272
   Last Key   -> 0:00:00.002788
   Key        -> 79    Max Time  -> 0:00:00.007543
   Key        -> 12    Min Time -> 0:00:00.002457
   avg        -> 0:00:00.002709
ITERATION  8 time 0:00:02.947676
   First Key  -> 0:00:00.003161
   Last Key   -> 0:00:00.003028
   Key        -> 1    Max Time  -> 0:00:00.006068
   Key        -> 31    Min Time -> 0:00:00.002683
   avg        -> 0:00:00.002933
ITERATION  9 time 0:00:03.781268
   First Key  -> 0:00:00.004271
   Last Key   -> 0:00:00.003416
   Key        -> 930    Max Time  -> 0:00:00.011791
   Key        -> 5    Min Time -> 0:00:00.003029
   avg        -> 0:00:00.003765
ITERATION  10 time 0:00:03.842353
   First Key  -> 0:00:00.006544
   Last Key   -> 0:00:00.003394
   Key        -> 0    Max Time  -> 0:00:00.006544
   Key        -> 4    Min Time -> 0:00:00.003171
   avg        -> 0:00:00.003826
ITERATION  11 time 0:00:03.361368
   First Key  -> 0:00:00.001509
   Last Key   -> 0:00:00.003626
   Key        -> 602    Max Time  -> 0:00:00.009093
   Key        -> 9    Min Time -> 0:00:00.001089
   avg        -> 0:00:00.003346
ITERATION  12 time 0:00:03.581802
   First Key  -> 0:00:00.001511
   Last Key   -> 0:00:00.003912
   Key        -> 474    Max Time  -> 0:00:00.009027
   Key        -> 1    Min Time -> 0:00:00.001117
   avg        -> 0:00:00.003566
ITERATION  13 time 0:00:03.797965
   First Key  -> 0:00:00.001833
   Last Key   -> 0:00:00.004118
   Key        -> 100    Max Time  -> 0:00:00.005115
   Key        -> 9    Min Time -> 0:00:00.001126
   avg        -> 0:00:00.003782
ITERATION  14 time 0:00:04.147961
   First Key  -> 0:00:00.001851
   Last Key   -> 0:00:00.004362
   Key        -> 782    Max Time  -> 0:00:00.013857
   Key        -> 16    Min Time -> 0:00:00.001186
   avg        -> 0:00:00.004132
ITERATION  15 time 0:00:04.251830
   First Key  -> 0:00:00.001538
   Last Key   -> 0:00:00.004611
   Key        -> 518    Max Time  -> 0:00:00.009511
   Key        -> 9    Min Time -> 0:00:00.001114
   avg        -> 0:00:00.004236
ITERATION  16 time 0:00:04.487687
   First Key  -> 0:00:00.001541
   Last Key   -> 0:00:00.004994
   Key        -> 858    Max Time  -> 0:00:00.009368
   Key        -> 9    Min Time -> 0:00:00.001120
   avg        -> 0:00:00.004472
ITERATION  17 time 0:00:04.703588
   First Key  -> 0:00:00.003440
   Last Key   -> 0:00:00.005104
   Key        -> 529    Max Time  -> 0:00:00.008179
   Key        -> 8    Min Time -> 0:00:00.001105
   avg        -> 0:00:00.004688
ITERATION  18 time 0:00:05.300920
   First Key  -> 0:00:00.001851
   Last Key   -> 0:00:00.006987
   Key        -> 854    Max Time  -> 0:00:00.007299
   Key        -> 9    Min Time -> 0:00:00.001181
   avg        -> 0:00:00.005285
ITERATION  19 time 0:00:06.168681
   First Key  -> 0:00:00.001880
   Last Key   -> 0:00:00.005577
   Key        -> 911    Max Time  -> 0:00:00.009503
   Key        -> 9    Min Time -> 0:00:00.001119
   avg        -> 0:00:00.006152








----------------------------------------------------------------------

Comment By: Oleg Broytman (phd)
Date: 2011-07-14 18:56

Message:
No idea what is going on. Let's investigate a little further. What version
of SQLObject are you using? Can you add more prints inside the loop to see
what part of the program takes time?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=540673&aid=3366899&group_id=74338

[SQL-CVS] [ sqlobject-Bugs-3370094 ] reslicing SelectResults object leads to error

[SourceForge.net <no...@so...>]

Bugs item #3370094, was opened at 2011-07-18 14:01
Message generated for change (Comment added) made by phd
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=540672&aid=3370094&group_id=74338

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
>Status: Closed
>Resolution: Accepted
Priority: 5
Private: No
Submitted By: mexicanferret (mferret)
Assigned to: Oleg Broytman (phd)
Summary: reslicing SelectResults object leads to error

Initial Comment:
reslicing SelectResults object sometimes leads to error
Ex:
    my_selection[0:10][0:20] will result in a "slice object is unsuscriptable" error

This is because of a typo in the sresults.py file line 156.

[code]if self.ops.get('end', None) is not None and value['end'] < end:[/code]

should be replaced by

[code]if self.ops.get('end', None) is not None  and self.ops['end'] < end:[/code]

value is a slice object and is indeed unsuscriptable.

I attach the corrected file.


----------------------------------------------------------------------

>Comment By: Oleg Broytman (phd)
Date: 2011-07-19 00:38

Message:
Yes, you are right. Your fix is applied and committed in the revisions
4429:4431 (branches 1.0, 1.1 and the trunk). Will be in the next round of
releases. Thank you!

----------------------------------------------------------------------

Comment By: mexicanferret (mferret)
Date: 2011-07-18 22:45

Message:
OK it works when the slice starts with 0 but try this:
my_selection[1:10][1:20]

You will get:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/sqlobject/sresults.py", line 156,
in __getitem__
    and value['end'] < end:
TypeError: 'slice' object is not subscriptable

----------------------------------------------------------------------

Comment By: Oleg Broytman (phd)
Date: 2011-07-18 22:05

Message:
I cannot reproduce the problem. This program works:

class Test(SQLObject):
    test = StringCol()

Test.createTable()

Test(test='1')
Test(test='2')
Test(test='3')
Test(test='4')
Test(test='5')

print list(Test.select()[0:3][0:5])

It prints

[<Test 1 test='1'>, <Test 2 test='2'>, <Test 3 test='3'>]

Looks good, right? I use SQLObject 1.1. What version do you have problems
with?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=540672&aid=3370094&group_id=74338

[SQL-CVS] r4431 - in SQLObject/trunk: docs sqlobject sqlobject/tests

[<sub...@co...>]

Author: phd
Date: Mon Jul 18 14:37:08 2011
New Revision: 4431

Log:
Merged revision 4430 from branch 1.1: a bug was fixed in SelectResults slicing
that prevented to slice a slice (my_results[:20][1:5]).

Modified:
   SQLObject/trunk/docs/News.txt
   SQLObject/trunk/sqlobject/sresults.py
   SQLObject/trunk/sqlobject/tests/test_slice.py

Modified: SQLObject/trunk/docs/News.txt
==============================================================================
--- SQLObject/trunk/docs/News.txt	Mon Jul 18 14:35:52 2011	(r4430)
+++ SQLObject/trunk/docs/News.txt	Mon Jul 18 14:37:08 2011	(r4431)
@@ -54,6 +54,12 @@
 
 * All 'mydict.has_key(name)' checks were replaced with 'name in mydict'.
 
+SQLObject 1.0.2
+===============
+
+* A bug was fixed in SelectResults slicing that prevented to slice a
+  slice (my_results[:20][1:5]).
+
 SQLObject 1.0.1
 ===============
 

Modified: SQLObject/trunk/sqlobject/sresults.py
==============================================================================
--- SQLObject/trunk/sqlobject/sresults.py	Mon Jul 18 14:35:52 2011	(r4430)
+++ SQLObject/trunk/sqlobject/sresults.py	Mon Jul 18 14:37:08 2011	(r4431)
@@ -1,7 +1,9 @@
-import sqlbuilder
 import dbconnection
-import main
 import joins
+import main
+import sqlbuilder
+
+__all__ = ['SelectResults']
 
 class SelectResults(object):
     IterationClass = dbconnection.Iteration
@@ -152,8 +154,8 @@
                         end = start
                     else:
                         end = value.stop + self.ops.get('start', 0)
-                        if self.ops.get('end', None) is not None \
-                           and value['end'] < end:
+                        if self.ops.get('end', None) is not None and \
+                                self.ops['end'] < end:
                             # truncated by previous slice:
                             end = self.ops['end']
                 else:
@@ -342,5 +344,3 @@
         clause = sqlbuilder.AND(otherClass.q.id == getattr(intTable, join.otherColumn),
                      getattr(intTable, colName) == query.q.id)
         return otherClass, clause
-
-__all__ = ['SelectResults']

Modified: SQLObject/trunk/sqlobject/tests/test_slice.py
==============================================================================
--- SQLObject/trunk/sqlobject/tests/test_slice.py	Mon Jul 18 14:35:52 2011	(r4430)
+++ SQLObject/trunk/sqlobject/tests/test_slice.py	Mon Jul 18 14:37:08 2011	(r4431)
@@ -21,31 +21,30 @@
             return
         assert [c.number for c in counters] == value
 
-    def test_1(self):
+    def test_slice(self):
         self.counterEqual(
             Counter.select(None, orderBy='number'), range(100))
 
-    def test_2(self):
         self.counterEqual(
             Counter.select(None, orderBy='number')[10:20],
             range(10, 20))
 
-    def test_3(self):
         self.counterEqual(
             Counter.select(None, orderBy='number')[20:30][:5],
             range(20, 25))
 
-    def test_4(self):
+        self.counterEqual(
+            Counter.select(None, orderBy='number')[20:30][1:5],
+            range(21, 25))
+
         self.counterEqual(
             Counter.select(None, orderBy='number')[:-10],
             range(0, 90))
 
-    def test_5(self):
         self.counterEqual(
             Counter.select(None, orderBy='number', reversed=True),
             range(99, -1, -1))
 
-    def test_6(self):
         self.counterEqual(
             Counter.select(None, orderBy='-number'),
             range(99, -1, -1))

[SQL-CVS] r4430 - in SQLObject/branches/1.1: docs sqlobject sqlobject/tests

[<sub...@co...>]

Author: phd
Date: Mon Jul 18 14:35:52 2011
New Revision: 4430

Log:
Merged revision 4429 from branch 1.0: a bug was fixed in SelectResults slicing
that prevented to slice a slice (my_results[:20][1:5]).

Modified:
   SQLObject/branches/1.1/docs/News.txt
   SQLObject/branches/1.1/sqlobject/sresults.py
   SQLObject/branches/1.1/sqlobject/tests/test_slice.py

Modified: SQLObject/branches/1.1/docs/News.txt
==============================================================================
--- SQLObject/branches/1.1/docs/News.txt	Mon Jul 18 14:32:40 2011	(r4429)
+++ SQLObject/branches/1.1/docs/News.txt	Mon Jul 18 14:35:52 2011	(r4430)
@@ -45,6 +45,12 @@
 
 * All 'mydict.has_key(name)' checks were replaced with 'name in mydict'.
 
+SQLObject 1.0.2
+===============
+
+* A bug was fixed in SelectResults slicing that prevented to slice a
+  slice (my_results[:20][1:5]).
+
 SQLObject 1.0.1
 ===============
 

Modified: SQLObject/branches/1.1/sqlobject/sresults.py
==============================================================================
--- SQLObject/branches/1.1/sqlobject/sresults.py	Mon Jul 18 14:32:40 2011	(r4429)
+++ SQLObject/branches/1.1/sqlobject/sresults.py	Mon Jul 18 14:35:52 2011	(r4430)
@@ -1,7 +1,9 @@
-import sqlbuilder
 import dbconnection
-import main
 import joins
+import main
+import sqlbuilder
+
+__all__ = ['SelectResults']
 
 class SelectResults(object):
     IterationClass = dbconnection.Iteration
@@ -152,8 +154,8 @@
                         end = start
                     else:
                         end = value.stop + self.ops.get('start', 0)
-                        if self.ops.get('end', None) is not None \
-                           and value['end'] < end:
+                        if self.ops.get('end', None) is not None and \
+                                self.ops['end'] < end:
                             # truncated by previous slice:
                             end = self.ops['end']
                 else:
@@ -342,5 +344,3 @@
         clause = sqlbuilder.AND(otherClass.q.id == getattr(intTable, join.otherColumn),
                      getattr(intTable, colName) == query.q.id)
         return otherClass, clause
-
-__all__ = ['SelectResults']

Modified: SQLObject/branches/1.1/sqlobject/tests/test_slice.py
==============================================================================
--- SQLObject/branches/1.1/sqlobject/tests/test_slice.py	Mon Jul 18 14:32:40 2011	(r4429)
+++ SQLObject/branches/1.1/sqlobject/tests/test_slice.py	Mon Jul 18 14:35:52 2011	(r4430)
@@ -21,31 +21,30 @@
             return
         assert [c.number for c in counters] == value
 
-    def test_1(self):
+    def test_slice(self):
         self.counterEqual(
             Counter.select(None, orderBy='number'), range(100))
 
-    def test_2(self):
         self.counterEqual(
             Counter.select(None, orderBy='number')[10:20],
             range(10, 20))
 
-    def test_3(self):
         self.counterEqual(
             Counter.select(None, orderBy='number')[20:30][:5],
             range(20, 25))
 
-    def test_4(self):
+        self.counterEqual(
+            Counter.select(None, orderBy='number')[20:30][1:5],
+            range(21, 25))
+
         self.counterEqual(
             Counter.select(None, orderBy='number')[:-10],
             range(0, 90))
 
-    def test_5(self):
         self.counterEqual(
             Counter.select(None, orderBy='number', reversed=True),
             range(99, -1, -1))
 
-    def test_6(self):
         self.counterEqual(
             Counter.select(None, orderBy='-number'),
             range(99, -1, -1))

[SQL-CVS] r4429 - in SQLObject/branches/1.0: docs sqlobject sqlobject/tests

[<sub...@co...>]

Author: phd
Date: Mon Jul 18 14:32:40 2011
New Revision: 4429

Log:
A bug was fixed in SelectResults slicing that prevented to slice a slice
(my_results[:20][1:5]).

Modified:
   SQLObject/branches/1.0/docs/News.txt
   SQLObject/branches/1.0/sqlobject/sresults.py
   SQLObject/branches/1.0/sqlobject/tests/test_slice.py

Modified: SQLObject/branches/1.0/docs/News.txt
==============================================================================
--- SQLObject/branches/1.0/docs/News.txt	Thu Jul 14 11:20:56 2011	(r4428)
+++ SQLObject/branches/1.0/docs/News.txt	Mon Jul 18 14:32:40 2011	(r4429)
@@ -7,6 +7,12 @@
 
 .. _start:
 
+SQLObject 1.0.2
+===============
+
+* A bug was fixed in SelectResults slicing that prevented to slice a
+  slice (my_results[:20][1:5]).
+
 SQLObject 1.0.1
 ===============
 

Modified: SQLObject/branches/1.0/sqlobject/sresults.py
==============================================================================
--- SQLObject/branches/1.0/sqlobject/sresults.py	Thu Jul 14 11:20:56 2011	(r4428)
+++ SQLObject/branches/1.0/sqlobject/sresults.py	Mon Jul 18 14:32:40 2011	(r4429)
@@ -1,7 +1,9 @@
-import sqlbuilder
 import dbconnection
-import main
 import joins
+import main
+import sqlbuilder
+
+__all__ = ['SelectResults']
 
 class SelectResults(object):
     IterationClass = dbconnection.Iteration
@@ -152,8 +154,8 @@
                         end = start
                     else:
                         end = value.stop + self.ops.get('start', 0)
-                        if self.ops.get('end', None) is not None \
-                           and value['end'] < end:
+                        if self.ops.get('end', None) is not None and \
+                                self.ops['end'] < end:
                             # truncated by previous slice:
                             end = self.ops['end']
                 else:
@@ -342,5 +344,3 @@
         clause = sqlbuilder.AND(otherClass.q.id == getattr(intTable, join.otherColumn),
                      getattr(intTable, colName) == query.q.id)
         return otherClass, clause
-
-__all__ = ['SelectResults']

Modified: SQLObject/branches/1.0/sqlobject/tests/test_slice.py
==============================================================================
--- SQLObject/branches/1.0/sqlobject/tests/test_slice.py	Thu Jul 14 11:20:56 2011	(r4428)
+++ SQLObject/branches/1.0/sqlobject/tests/test_slice.py	Mon Jul 18 14:32:40 2011	(r4429)
@@ -21,31 +21,30 @@
             return
         assert [c.number for c in counters] == value
 
-    def test_1(self):
+    def test_slice(self):
         self.counterEqual(
             Counter.select(None, orderBy='number'), range(100))
 
-    def test_2(self):
         self.counterEqual(
             Counter.select(None, orderBy='number')[10:20],
             range(10, 20))
 
-    def test_3(self):
         self.counterEqual(
             Counter.select(None, orderBy='number')[20:30][:5],
             range(20, 25))
 
-    def test_4(self):
+        self.counterEqual(
+            Counter.select(None, orderBy='number')[20:30][1:5],
+            range(21, 25))
+
         self.counterEqual(
             Counter.select(None, orderBy='number')[:-10],
             range(0, 90))
 
-    def test_5(self):
         self.counterEqual(
             Counter.select(None, orderBy='number', reversed=True),
             range(99, -1, -1))
 
-    def test_6(self):
         self.counterEqual(
             Counter.select(None, orderBy='-number'),
             range(99, -1, -1))

[SQL-CVS] [ sqlobject-Bugs-3370094 ] reslicing SelectResults object leads to error

[SourceForge.net <no...@so...>]

Bugs item #3370094, was opened at 2011-07-18 12:01
Message generated for change (Comment added) made by mferret
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=540672&aid=3370094&group_id=74338

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: Works For Me
Priority: 5
Private: No
Submitted By: mexicanferret (mferret)
Assigned to: Oleg Broytman (phd)
Summary: reslicing SelectResults object leads to error

Initial Comment:
reslicing SelectResults object sometimes leads to error
Ex:
    my_selection[0:10][0:20] will result in a "slice object is unsuscriptable" error

This is because of a typo in the sresults.py file line 156.

[code]if self.ops.get('end', None) is not None and value['end'] < end:[/code]

should be replaced by

[code]if self.ops.get('end', None) is not None  and self.ops['end'] < end:[/code]

value is a slice object and is indeed unsuscriptable.

I attach the corrected file.


----------------------------------------------------------------------

>Comment By: mexicanferret (mferret)
Date: 2011-07-18 20:45

Message:
OK it works when the slice starts with 0 but try this:
my_selection[1:10][1:20]

You will get:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/sqlobject/sresults.py", line 156,
in __getitem__
    and value['end'] < end:
TypeError: 'slice' object is not subscriptable

----------------------------------------------------------------------

Comment By: Oleg Broytman (phd)
Date: 2011-07-18 20:05

Message:
I cannot reproduce the problem. This program works:

class Test(SQLObject):
    test = StringCol()

Test.createTable()

Test(test='1')
Test(test='2')
Test(test='3')
Test(test='4')
Test(test='5')

print list(Test.select()[0:3][0:5])

It prints

[<Test 1 test='1'>, <Test 2 test='2'>, <Test 3 test='3'>]

Looks good, right? I use SQLObject 1.1. What version do you have problems
with?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=540672&aid=3370094&group_id=74338

[SQL-CVS] [ sqlobject-Bugs-3370094 ] reslicing SelectResults object leads to error

[SourceForge.net <no...@so...>]

Bugs item #3370094, was opened at 2011-07-18 14:01
Message generated for change (Comment added) made by phd
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=540672&aid=3370094&group_id=74338

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
>Resolution: Works For Me
Priority: 5
Private: No
Submitted By: mexicanferret (mferret)
>Assigned to: Oleg Broytman (phd)
Summary: reslicing SelectResults object leads to error

Initial Comment:
reslicing SelectResults object sometimes leads to error
Ex:
    my_selection[0:10][0:20] will result in a "slice object is unsuscriptable" error

This is because of a typo in the sresults.py file line 156.

[code]if self.ops.get('end', None) is not None and value['end'] < end:[/code]

should be replaced by

[code]if self.ops.get('end', None) is not None  and self.ops['end'] < end:[/code]

value is a slice object and is indeed unsuscriptable.

I attach the corrected file.


----------------------------------------------------------------------

>Comment By: Oleg Broytman (phd)
Date: 2011-07-18 22:05

Message:
I cannot reproduce the problem. This program works:

class Test(SQLObject):
    test = StringCol()

Test.createTable()

Test(test='1')
Test(test='2')
Test(test='3')
Test(test='4')
Test(test='5')

print list(Test.select()[0:3][0:5])

It prints

[<Test 1 test='1'>, <Test 2 test='2'>, <Test 3 test='3'>]

Looks good, right? I use SQLObject 1.1. What version do you have problems
with?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=540672&aid=3370094&group_id=74338

[SQL-CVS] [ sqlobject-Support Requests-3366899 ] The speed of SQLObject is slower on the time

[SourceForge.net <no...@so...>]

Support Requests item #3366899, was opened at 2011-07-14 11:40
Message generated for change (Comment added) made by phd
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=540673&aid=3366899&group_id=74338

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Priority: 5
Private: No
Submitted By: Rosa Robles (marobles)
Assigned to: Oleg Broytman (phd)
Summary: The speed of SQLObject is slower on the time

Initial Comment:
I had a test of a SQLObject and I had found a strange behaviour of SQLObject. On the time, the respond time is lesser.

Here, I put a example of test code :

from sqlobject.sqlbuilder import *
connection = connectionForURI('postgresql://test:test@localhost/test_sql_object')
sqlhub.processConnection = connection

class Person(SQLObject):
    firstName = StringCol()
    middleInitial = StringCol(length=1, default=None)
    lastName = StringCol()

iteration = 0
while 1:
    start = datetime.now()
    person_list = []
    for key in range(1000):
        person = Person.selectBy(firstName='name%s%s' % (iteration,key))[0]
        person_list.append(person)
   
    for person in person_list:
        person.expire()
        del person
    del person_list
    sqlhub.processConnection.cache.clear()
   
    end = datetime.now()
       
    print "TIME ",iteration,':' , end - start
    iteration = iteration + 1

In the result I can check that in each iteration the time is bigger. The first iteration has time of 1.37s, then the time increases and the 19 iteration has time of 5.83 s!

TIME  0 : 0:00:01.376392
TIME  1 : 0:00:01.663250
TIME  2 : 0:00:01.978639
TIME  3 : 0:00:02.319939
TIME  4 : 0:00:02.618213
TIME  5 : 0:00:02.723846
TIME  6 : 0:00:02.584007
TIME  7 : 0:00:02.868877
TIME  8 : 0:00:03.051937
TIME  9 : 0:00:03.389757
TIME  10 : 0:00:03.503036
TIME  11 : 0:00:03.563154
TIME  12 : 0:00:04.191539
TIME  13 : 0:00:04.274464
TIME  14 : 0:00:04.813206
TIME  15 : 0:00:04.677002
TIME  16 : 0:00:04.995708
TIME  17 : 0:00:05.126811
TIME  18 : 0:00:05.571279
TIME  19 : 0:00:05.832840

Why is happend this? 
Thanks in advance for your attention.
Cheers Rosa

----------------------------------------------------------------------

>Comment By: Oleg Broytman (phd)
Date: 2011-07-18 21:50

Message:
time_end-time_start are consistently almost zero, so selectBy doesn't take
time, right?

----------------------------------------------------------------------

Comment By: Rosa Robles (marobles)
Date: 2011-07-18 10:36

Message:
The program takes time only from the selectby operation
for key in range(1000):
    time_start = datetime.now()
    person = Person.selectBy(firstName='name%s%s' %(iteration,key))[0]
    time_end = datetime.now()
    times.append(time_end-time_start)

I attach the example to you can read better

If we review the results, the selectBy method takes times random in
iteration, but also each iteration's time grows up. 
I think the problem is in the execution  of cull process when delete
expiredCache values, what do you think?

In my real program do not use select in mass but it can acumulate a lots
operations of select in the time, 
then  we have problems of memory and we must restart the application.
May be this situation is the source of our problem with the memory.

----------------------------------------------------------------------

Comment By: Oleg Broytman (phd)
Date: 2011-07-15 21:14

Message:
Min/max/avg times are almost zero, so it seems the only place the program
spends time is calling .expireAll() and cache.clear(), right?

----------------------------------------------------------------------

Comment By: Rosa Robles (marobles)
Date: 2011-07-15 10:52

Message:
Thanks Oleg.
I'm using SQLObject 1.1.1, Python2.6, Postgres 8.4

Ok, I change the code to takes more times.

    for iteration in range(20):
        start = datetime.now()
        acum = timedelta(microseconds=0)
        times = []
        
        for key in range(1000):
            time_start = datetime.now()
            person = Person.selectBy(firstName='name%s%s' %
(iteration,key))[0]
            time_end = datetime.now()
            times.append(time_end-time_start)
            acum = acum + (time_end-time_start)
            
        end = datetime.now()
        
        print "ITERATION ",iteration,"time",end-start
        print "   First Key  ->", times[0] 
        print "   Last Key   ->",times[999] 
        print "   Key        ->",times.index(max(times)), "   Max Time 
->", max(times)
        print "   Key        ->",times.index(min(times)), "   Min Time
->", min(times)
        print "   avg        ->",acum/999
        
        Person.sqlmeta.expireAll()
        for k in connection.cache.caches.keys():
            connection.cache.caches[k].expireAll()
        sqlhub.getConnection().expireAll()
        connection.cache.clear()
        gc.collect()


And the result is:


ITERATION  0 time 0:00:01.302290
   First Key  -> 0:00:00.001555
   Last Key   -> 0:00:00.001445
   Key        -> 11    Max Time  -> 0:00:00.003216
   Key        -> 36    Min Time -> 0:00:00.001115
   avg        -> 0:00:00.001287
ITERATION  1 time 0:00:01.621580
   First Key  -> 0:00:00.001879
   Last Key   -> 0:00:00.001734
   Key        -> 924    Max Time  -> 0:00:00.009572
   Key        -> 22    Min Time -> 0:00:00.001076
   avg        -> 0:00:00.001606
ITERATION  2 time 0:00:01.851719
   First Key  -> 0:00:00.002328
   Last Key   -> 0:00:00.002064
   Key        -> 970    Max Time  -> 0:00:00.002544
   Key        -> 22    Min Time -> 0:00:00.001306
   avg        -> 0:00:00.001836
ITERATION  3 time 0:00:02.040802
   First Key  -> 0:00:00.002435
   Last Key   -> 0:00:00.002376
   Key        -> 557    Max Time  -> 0:00:00.009674
   Key        -> 25    Min Time -> 0:00:00.001538
   avg        -> 0:00:00.002026
ITERATION  4 time 0:00:02.427911
   First Key  -> 0:00:00.002906
   Last Key   -> 0:00:00.002065
   Key        -> 774    Max Time  -> 0:00:00.003263
   Key        -> 7    Min Time -> 0:00:00.001763
   avg        -> 0:00:00.002413
ITERATION  5 time 0:00:02.768819
   First Key  -> 0:00:00.003051
   Last Key   -> 0:00:00.002319
   Key        -> 920    Max Time  -> 0:00:00.007694
   Key        -> 11    Min Time -> 0:00:00.002007
   avg        -> 0:00:00.002753
ITERATION  6 time 0:00:02.537283
   First Key  -> 0:00:00.002676
   Last Key   -> 0:00:00.002549
   Key        -> 2    Max Time  -> 0:00:00.003512
   Key        -> 132    Min Time -> 0:00:00.002318
   avg        -> 0:00:00.002523
ITERATION  7 time 0:00:02.723405
   First Key  -> 0:00:00.005272
   Last Key   -> 0:00:00.002788
   Key        -> 79    Max Time  -> 0:00:00.007543
   Key        -> 12    Min Time -> 0:00:00.002457
   avg        -> 0:00:00.002709
ITERATION  8 time 0:00:02.947676
   First Key  -> 0:00:00.003161
   Last Key   -> 0:00:00.003028
   Key        -> 1    Max Time  -> 0:00:00.006068
   Key        -> 31    Min Time -> 0:00:00.002683
   avg        -> 0:00:00.002933
ITERATION  9 time 0:00:03.781268
   First Key  -> 0:00:00.004271
   Last Key   -> 0:00:00.003416
   Key        -> 930    Max Time  -> 0:00:00.011791
   Key        -> 5    Min Time -> 0:00:00.003029
   avg        -> 0:00:00.003765
ITERATION  10 time 0:00:03.842353
   First Key  -> 0:00:00.006544
   Last Key   -> 0:00:00.003394
   Key        -> 0    Max Time  -> 0:00:00.006544
   Key        -> 4    Min Time -> 0:00:00.003171
   avg        -> 0:00:00.003826
ITERATION  11 time 0:00:03.361368
   First Key  -> 0:00:00.001509
   Last Key   -> 0:00:00.003626
   Key        -> 602    Max Time  -> 0:00:00.009093
   Key        -> 9    Min Time -> 0:00:00.001089
   avg        -> 0:00:00.003346
ITERATION  12 time 0:00:03.581802
   First Key  -> 0:00:00.001511
   Last Key   -> 0:00:00.003912
   Key        -> 474    Max Time  -> 0:00:00.009027
   Key        -> 1    Min Time -> 0:00:00.001117
   avg        -> 0:00:00.003566
ITERATION  13 time 0:00:03.797965
   First Key  -> 0:00:00.001833
   Last Key   -> 0:00:00.004118
   Key        -> 100    Max Time  -> 0:00:00.005115
   Key        -> 9    Min Time -> 0:00:00.001126
   avg        -> 0:00:00.003782
ITERATION  14 time 0:00:04.147961
   First Key  -> 0:00:00.001851
   Last Key   -> 0:00:00.004362
   Key        -> 782    Max Time  -> 0:00:00.013857
   Key        -> 16    Min Time -> 0:00:00.001186
   avg        -> 0:00:00.004132
ITERATION  15 time 0:00:04.251830
   First Key  -> 0:00:00.001538
   Last Key   -> 0:00:00.004611
   Key        -> 518    Max Time  -> 0:00:00.009511
   Key        -> 9    Min Time -> 0:00:00.001114
   avg        -> 0:00:00.004236
ITERATION  16 time 0:00:04.487687
   First Key  -> 0:00:00.001541
   Last Key   -> 0:00:00.004994
   Key        -> 858    Max Time  -> 0:00:00.009368
   Key        -> 9    Min Time -> 0:00:00.001120
   avg        -> 0:00:00.004472
ITERATION  17 time 0:00:04.703588
   First Key  -> 0:00:00.003440
   Last Key   -> 0:00:00.005104
   Key        -> 529    Max Time  -> 0:00:00.008179
   Key        -> 8    Min Time -> 0:00:00.001105
   avg        -> 0:00:00.004688
ITERATION  18 time 0:00:05.300920
   First Key  -> 0:00:00.001851
   Last Key   -> 0:00:00.006987
   Key        -> 854    Max Time  -> 0:00:00.007299
   Key        -> 9    Min Time -> 0:00:00.001181
   avg        -> 0:00:00.005285
ITERATION  19 time 0:00:06.168681
   First Key  -> 0:00:00.001880
   Last Key   -> 0:00:00.005577
   Key        -> 911    Max Time  -> 0:00:00.009503
   Key        -> 9    Min Time -> 0:00:00.001119
   avg        -> 0:00:00.006152








----------------------------------------------------------------------

Comment By: Oleg Broytman (phd)
Date: 2011-07-14 20:56

Message:
No idea what is going on. Let's investigate a little further. What version
of SQLObject are you using? Can you add more prints inside the loop to see
what part of the program takes time?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=540673&aid=3366899&group_id=74338

[SQL-CVS] [ sqlobject-Bugs-3370094 ] reslicing SelectResults object leads to error

[SourceForge.net <no...@so...>]

Bugs item #3370094, was opened at 2011-07-18 12:01
Message generated for change (Tracker Item Submitted) made by mferret
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=540672&aid=3370094&group_id=74338

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: mexicanferret (mferret)
Assigned to: Nobody/Anonymous (nobody)
Summary: reslicing SelectResults object leads to error

Initial Comment:
reslicing SelectResults object sometimes leads to error
Ex:
    my_selection[0:10][0:20] will result in a "slice object is unsuscriptable" error

This is because of a typo in the sresults.py file line 156.

[code]if self.ops.get('end', None) is not None and value['end'] < end:[/code]

should be replaced by

[code]if self.ops.get('end', None) is not None  and self.ops['end'] < end:[/code]

value is a slice object and is indeed unsuscriptable.

I attach the corrected file.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=540672&aid=3370094&group_id=74338

[SQL-CVS] [ sqlobject-Support Requests-3366899 ] The speed of SQLObject is slower on the time

[SourceForge.net <no...@so...>]

Support Requests item #3366899, was opened at 2011-07-14 09:40
Message generated for change (Comment added) made by marobles
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=540673&aid=3366899&group_id=74338

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Priority: 5
Private: No
Submitted By: Rosa Robles (marobles)
Assigned to: Oleg Broytman (phd)
Summary: The speed of SQLObject is slower on the time

Initial Comment:
I had a test of a SQLObject and I had found a strange behaviour of SQLObject. On the time, the respond time is lesser.

Here, I put a example of test code :

from sqlobject.sqlbuilder import *
connection = connectionForURI('postgresql://test:test@localhost/test_sql_object')
sqlhub.processConnection = connection

class Person(SQLObject):
    firstName = StringCol()
    middleInitial = StringCol(length=1, default=None)
    lastName = StringCol()

iteration = 0
while 1:
    start = datetime.now()
    person_list = []
    for key in range(1000):
        person = Person.selectBy(firstName='name%s%s' % (iteration,key))[0]
        person_list.append(person)
   
    for person in person_list:
        person.expire()
        del person
    del person_list
    sqlhub.processConnection.cache.clear()
   
    end = datetime.now()
       
    print "TIME ",iteration,':' , end - start
    iteration = iteration + 1

In the result I can check that in each iteration the time is bigger. The first iteration has time of 1.37s, then the time increases and the 19 iteration has time of 5.83 s!

TIME  0 : 0:00:01.376392
TIME  1 : 0:00:01.663250
TIME  2 : 0:00:01.978639
TIME  3 : 0:00:02.319939
TIME  4 : 0:00:02.618213
TIME  5 : 0:00:02.723846
TIME  6 : 0:00:02.584007
TIME  7 : 0:00:02.868877
TIME  8 : 0:00:03.051937
TIME  9 : 0:00:03.389757
TIME  10 : 0:00:03.503036
TIME  11 : 0:00:03.563154
TIME  12 : 0:00:04.191539
TIME  13 : 0:00:04.274464
TIME  14 : 0:00:04.813206
TIME  15 : 0:00:04.677002
TIME  16 : 0:00:04.995708
TIME  17 : 0:00:05.126811
TIME  18 : 0:00:05.571279
TIME  19 : 0:00:05.832840

Why is happend this? 
Thanks in advance for your attention.
Cheers Rosa

----------------------------------------------------------------------

>Comment By: Rosa Robles (marobles)
Date: 2011-07-18 08:36

Message:
The program takes time only from the selectby operation
for key in range(1000):
    time_start = datetime.now()
    person = Person.selectBy(firstName='name%s%s' %(iteration,key))[0]
    time_end = datetime.now()
    times.append(time_end-time_start)

I attach the example to you can read better

If we review the results, the selectBy method takes times random in
iteration, but also each iteration's time grows up. 
I think the problem is in the execution  of cull process when delete
expiredCache values, what do you think?

In my real program do not use select in mass but it can acumulate a lots
operations of select in the time, 
then  we have problems of memory and we must restart the application.
May be this situation is the source of our problem with the memory.

----------------------------------------------------------------------

Comment By: Oleg Broytman (phd)
Date: 2011-07-15 19:14

Message:
Min/max/avg times are almost zero, so it seems the only place the program
spends time is calling .expireAll() and cache.clear(), right?

----------------------------------------------------------------------

Comment By: Rosa Robles (marobles)
Date: 2011-07-15 08:52

Message:
Thanks Oleg.
I'm using SQLObject 1.1.1, Python2.6, Postgres 8.4

Ok, I change the code to takes more times.

    for iteration in range(20):
        start = datetime.now()
        acum = timedelta(microseconds=0)
        times = []
        
        for key in range(1000):
            time_start = datetime.now()
            person = Person.selectBy(firstName='name%s%s' %
(iteration,key))[0]
            time_end = datetime.now()
            times.append(time_end-time_start)
            acum = acum + (time_end-time_start)
            
        end = datetime.now()
        
        print "ITERATION ",iteration,"time",end-start
        print "   First Key  ->", times[0] 
        print "   Last Key   ->",times[999] 
        print "   Key        ->",times.index(max(times)), "   Max Time 
->", max(times)
        print "   Key        ->",times.index(min(times)), "   Min Time
->", min(times)
        print "   avg        ->",acum/999
        
        Person.sqlmeta.expireAll()
        for k in connection.cache.caches.keys():
            connection.cache.caches[k].expireAll()
        sqlhub.getConnection().expireAll()
        connection.cache.clear()
        gc.collect()


And the result is:


ITERATION  0 time 0:00:01.302290
   First Key  -> 0:00:00.001555
   Last Key   -> 0:00:00.001445
   Key        -> 11    Max Time  -> 0:00:00.003216
   Key        -> 36    Min Time -> 0:00:00.001115
   avg        -> 0:00:00.001287
ITERATION  1 time 0:00:01.621580
   First Key  -> 0:00:00.001879
   Last Key   -> 0:00:00.001734
   Key        -> 924    Max Time  -> 0:00:00.009572
   Key        -> 22    Min Time -> 0:00:00.001076
   avg        -> 0:00:00.001606
ITERATION  2 time 0:00:01.851719
   First Key  -> 0:00:00.002328
   Last Key   -> 0:00:00.002064
   Key        -> 970    Max Time  -> 0:00:00.002544
   Key        -> 22    Min Time -> 0:00:00.001306
   avg        -> 0:00:00.001836
ITERATION  3 time 0:00:02.040802
   First Key  -> 0:00:00.002435
   Last Key   -> 0:00:00.002376
   Key        -> 557    Max Time  -> 0:00:00.009674
   Key        -> 25    Min Time -> 0:00:00.001538
   avg        -> 0:00:00.002026
ITERATION  4 time 0:00:02.427911
   First Key  -> 0:00:00.002906
   Last Key   -> 0:00:00.002065
   Key        -> 774    Max Time  -> 0:00:00.003263
   Key        -> 7    Min Time -> 0:00:00.001763
   avg        -> 0:00:00.002413
ITERATION  5 time 0:00:02.768819
   First Key  -> 0:00:00.003051
   Last Key   -> 0:00:00.002319
   Key        -> 920    Max Time  -> 0:00:00.007694
   Key        -> 11    Min Time -> 0:00:00.002007
   avg        -> 0:00:00.002753
ITERATION  6 time 0:00:02.537283
   First Key  -> 0:00:00.002676
   Last Key   -> 0:00:00.002549
   Key        -> 2    Max Time  -> 0:00:00.003512
   Key        -> 132    Min Time -> 0:00:00.002318
   avg        -> 0:00:00.002523
ITERATION  7 time 0:00:02.723405
   First Key  -> 0:00:00.005272
   Last Key   -> 0:00:00.002788
   Key        -> 79    Max Time  -> 0:00:00.007543
   Key        -> 12    Min Time -> 0:00:00.002457
   avg        -> 0:00:00.002709
ITERATION  8 time 0:00:02.947676
   First Key  -> 0:00:00.003161
   Last Key   -> 0:00:00.003028
   Key        -> 1    Max Time  -> 0:00:00.006068
   Key        -> 31    Min Time -> 0:00:00.002683
   avg        -> 0:00:00.002933
ITERATION  9 time 0:00:03.781268
   First Key  -> 0:00:00.004271
   Last Key   -> 0:00:00.003416
   Key        -> 930    Max Time  -> 0:00:00.011791
   Key        -> 5    Min Time -> 0:00:00.003029
   avg        -> 0:00:00.003765
ITERATION  10 time 0:00:03.842353
   First Key  -> 0:00:00.006544
   Last Key   -> 0:00:00.003394
   Key        -> 0    Max Time  -> 0:00:00.006544
   Key        -> 4    Min Time -> 0:00:00.003171
   avg        -> 0:00:00.003826
ITERATION  11 time 0:00:03.361368
   First Key  -> 0:00:00.001509
   Last Key   -> 0:00:00.003626
   Key        -> 602    Max Time  -> 0:00:00.009093
   Key        -> 9    Min Time -> 0:00:00.001089
   avg        -> 0:00:00.003346
ITERATION  12 time 0:00:03.581802
   First Key  -> 0:00:00.001511
   Last Key   -> 0:00:00.003912
   Key        -> 474    Max Time  -> 0:00:00.009027
   Key        -> 1    Min Time -> 0:00:00.001117
   avg        -> 0:00:00.003566
ITERATION  13 time 0:00:03.797965
   First Key  -> 0:00:00.001833
   Last Key   -> 0:00:00.004118
   Key        -> 100    Max Time  -> 0:00:00.005115
   Key        -> 9    Min Time -> 0:00:00.001126
   avg        -> 0:00:00.003782
ITERATION  14 time 0:00:04.147961
   First Key  -> 0:00:00.001851
   Last Key   -> 0:00:00.004362
   Key        -> 782    Max Time  -> 0:00:00.013857
   Key        -> 16    Min Time -> 0:00:00.001186
   avg        -> 0:00:00.004132
ITERATION  15 time 0:00:04.251830
   First Key  -> 0:00:00.001538
   Last Key   -> 0:00:00.004611
   Key        -> 518    Max Time  -> 0:00:00.009511
   Key        -> 9    Min Time -> 0:00:00.001114
   avg        -> 0:00:00.004236
ITERATION  16 time 0:00:04.487687
   First Key  -> 0:00:00.001541
   Last Key   -> 0:00:00.004994
   Key        -> 858    Max Time  -> 0:00:00.009368
   Key        -> 9    Min Time -> 0:00:00.001120
   avg        -> 0:00:00.004472
ITERATION  17 time 0:00:04.703588
   First Key  -> 0:00:00.003440
   Last Key   -> 0:00:00.005104
   Key        -> 529    Max Time  -> 0:00:00.008179
   Key        -> 8    Min Time -> 0:00:00.001105
   avg        -> 0:00:00.004688
ITERATION  18 time 0:00:05.300920
   First Key  -> 0:00:00.001851
   Last Key   -> 0:00:00.006987
   Key        -> 854    Max Time  -> 0:00:00.007299
   Key        -> 9    Min Time -> 0:00:00.001181
   avg        -> 0:00:00.005285
ITERATION  19 time 0:00:06.168681
   First Key  -> 0:00:00.001880
   Last Key   -> 0:00:00.005577
   Key        -> 911    Max Time  -> 0:00:00.009503
   Key        -> 9    Min Time -> 0:00:00.001119
   avg        -> 0:00:00.006152








----------------------------------------------------------------------

Comment By: Oleg Broytman (phd)
Date: 2011-07-14 18:56

Message:
No idea what is going on. Let's investigate a little further. What version
of SQLObject are you using? Can you add more prints inside the loop to see
what part of the program takes time?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=540673&aid=3366899&group_id=74338

[SQL-CVS] [ sqlobject-Support Requests-3366899 ] The speed of SQLObject is slower on the time

[SourceForge.net <no...@so...>]

Support Requests item #3366899, was opened at 2011-07-14 11:40
Message generated for change (Comment added) made by phd
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=540673&aid=3366899&group_id=74338

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Priority: 5
Private: No
Submitted By: Rosa Robles (marobles)
Assigned to: Oleg Broytman (phd)
Summary: The speed of SQLObject is slower on the time

Initial Comment:
I had a test of a SQLObject and I had found a strange behaviour of SQLObject. On the time, the respond time is lesser.

Here, I put a example of test code :

from sqlobject.sqlbuilder import *
connection = connectionForURI('postgresql://test:test@localhost/test_sql_object')
sqlhub.processConnection = connection

class Person(SQLObject):
    firstName = StringCol()
    middleInitial = StringCol(length=1, default=None)
    lastName = StringCol()

iteration = 0
while 1:
    start = datetime.now()
    person_list = []
    for key in range(1000):
        person = Person.selectBy(firstName='name%s%s' % (iteration,key))[0]
        person_list.append(person)
   
    for person in person_list:
        person.expire()
        del person
    del person_list
    sqlhub.processConnection.cache.clear()
   
    end = datetime.now()
       
    print "TIME ",iteration,':' , end - start
    iteration = iteration + 1

In the result I can check that in each iteration the time is bigger. The first iteration has time of 1.37s, then the time increases and the 19 iteration has time of 5.83 s!

TIME  0 : 0:00:01.376392
TIME  1 : 0:00:01.663250
TIME  2 : 0:00:01.978639
TIME  3 : 0:00:02.319939
TIME  4 : 0:00:02.618213
TIME  5 : 0:00:02.723846
TIME  6 : 0:00:02.584007
TIME  7 : 0:00:02.868877
TIME  8 : 0:00:03.051937
TIME  9 : 0:00:03.389757
TIME  10 : 0:00:03.503036
TIME  11 : 0:00:03.563154
TIME  12 : 0:00:04.191539
TIME  13 : 0:00:04.274464
TIME  14 : 0:00:04.813206
TIME  15 : 0:00:04.677002
TIME  16 : 0:00:04.995708
TIME  17 : 0:00:05.126811
TIME  18 : 0:00:05.571279
TIME  19 : 0:00:05.832840

Why is happend this? 
Thanks in advance for your attention.
Cheers Rosa

----------------------------------------------------------------------

>Comment By: Oleg Broytman (phd)
Date: 2011-07-15 21:14

Message:
Min/max/avg times are almost zero, so it seems the only place the program
spends time is calling .expireAll() and cache.clear(), right?

----------------------------------------------------------------------

Comment By: Rosa Robles (marobles)
Date: 2011-07-15 10:52

Message:
Thanks Oleg.
I'm using SQLObject 1.1.1, Python2.6, Postgres 8.4

Ok, I change the code to takes more times.

    for iteration in range(20):
        start = datetime.now()
        acum = timedelta(microseconds=0)
        times = []
        
        for key in range(1000):
            time_start = datetime.now()
            person = Person.selectBy(firstName='name%s%s' %
(iteration,key))[0]
            time_end = datetime.now()
            times.append(time_end-time_start)
            acum = acum + (time_end-time_start)
            
        end = datetime.now()
        
        print "ITERATION ",iteration,"time",end-start
        print "   First Key  ->", times[0] 
        print "   Last Key   ->",times[999] 
        print "   Key        ->",times.index(max(times)), "   Max Time 
->", max(times)
        print "   Key        ->",times.index(min(times)), "   Min Time
->", min(times)
        print "   avg        ->",acum/999
        
        Person.sqlmeta.expireAll()
        for k in connection.cache.caches.keys():
            connection.cache.caches[k].expireAll()
        sqlhub.getConnection().expireAll()
        connection.cache.clear()
        gc.collect()


And the result is:


ITERATION  0 time 0:00:01.302290
   First Key  -> 0:00:00.001555
   Last Key   -> 0:00:00.001445
   Key        -> 11    Max Time  -> 0:00:00.003216
   Key        -> 36    Min Time -> 0:00:00.001115
   avg        -> 0:00:00.001287
ITERATION  1 time 0:00:01.621580
   First Key  -> 0:00:00.001879
   Last Key   -> 0:00:00.001734
   Key        -> 924    Max Time  -> 0:00:00.009572
   Key        -> 22    Min Time -> 0:00:00.001076
   avg        -> 0:00:00.001606
ITERATION  2 time 0:00:01.851719
   First Key  -> 0:00:00.002328
   Last Key   -> 0:00:00.002064
   Key        -> 970    Max Time  -> 0:00:00.002544
   Key        -> 22    Min Time -> 0:00:00.001306
   avg        -> 0:00:00.001836
ITERATION  3 time 0:00:02.040802
   First Key  -> 0:00:00.002435
   Last Key   -> 0:00:00.002376
   Key        -> 557    Max Time  -> 0:00:00.009674
   Key        -> 25    Min Time -> 0:00:00.001538
   avg        -> 0:00:00.002026
ITERATION  4 time 0:00:02.427911
   First Key  -> 0:00:00.002906
   Last Key   -> 0:00:00.002065
   Key        -> 774    Max Time  -> 0:00:00.003263
   Key        -> 7    Min Time -> 0:00:00.001763
   avg        -> 0:00:00.002413
ITERATION  5 time 0:00:02.768819
   First Key  -> 0:00:00.003051
   Last Key   -> 0:00:00.002319
   Key        -> 920    Max Time  -> 0:00:00.007694
   Key        -> 11    Min Time -> 0:00:00.002007
   avg        -> 0:00:00.002753
ITERATION  6 time 0:00:02.537283
   First Key  -> 0:00:00.002676
   Last Key   -> 0:00:00.002549
   Key        -> 2    Max Time  -> 0:00:00.003512
   Key        -> 132    Min Time -> 0:00:00.002318
   avg        -> 0:00:00.002523
ITERATION  7 time 0:00:02.723405
   First Key  -> 0:00:00.005272
   Last Key   -> 0:00:00.002788
   Key        -> 79    Max Time  -> 0:00:00.007543
   Key        -> 12    Min Time -> 0:00:00.002457
   avg        -> 0:00:00.002709
ITERATION  8 time 0:00:02.947676
   First Key  -> 0:00:00.003161
   Last Key   -> 0:00:00.003028
   Key        -> 1    Max Time  -> 0:00:00.006068
   Key        -> 31    Min Time -> 0:00:00.002683
   avg        -> 0:00:00.002933
ITERATION  9 time 0:00:03.781268
   First Key  -> 0:00:00.004271
   Last Key   -> 0:00:00.003416
   Key        -> 930    Max Time  -> 0:00:00.011791
   Key        -> 5    Min Time -> 0:00:00.003029
   avg        -> 0:00:00.003765
ITERATION  10 time 0:00:03.842353
   First Key  -> 0:00:00.006544
   Last Key   -> 0:00:00.003394
   Key        -> 0    Max Time  -> 0:00:00.006544
   Key        -> 4    Min Time -> 0:00:00.003171
   avg        -> 0:00:00.003826
ITERATION  11 time 0:00:03.361368
   First Key  -> 0:00:00.001509
   Last Key   -> 0:00:00.003626
   Key        -> 602    Max Time  -> 0:00:00.009093
   Key        -> 9    Min Time -> 0:00:00.001089
   avg        -> 0:00:00.003346
ITERATION  12 time 0:00:03.581802
   First Key  -> 0:00:00.001511
   Last Key   -> 0:00:00.003912
   Key        -> 474    Max Time  -> 0:00:00.009027
   Key        -> 1    Min Time -> 0:00:00.001117
   avg        -> 0:00:00.003566
ITERATION  13 time 0:00:03.797965
   First Key  -> 0:00:00.001833
   Last Key   -> 0:00:00.004118
   Key        -> 100    Max Time  -> 0:00:00.005115
   Key        -> 9    Min Time -> 0:00:00.001126
   avg        -> 0:00:00.003782
ITERATION  14 time 0:00:04.147961
   First Key  -> 0:00:00.001851
   Last Key   -> 0:00:00.004362
   Key        -> 782    Max Time  -> 0:00:00.013857
   Key        -> 16    Min Time -> 0:00:00.001186
   avg        -> 0:00:00.004132
ITERATION  15 time 0:00:04.251830
   First Key  -> 0:00:00.001538
   Last Key   -> 0:00:00.004611
   Key        -> 518    Max Time  -> 0:00:00.009511
   Key        -> 9    Min Time -> 0:00:00.001114
   avg        -> 0:00:00.004236
ITERATION  16 time 0:00:04.487687
   First Key  -> 0:00:00.001541
   Last Key   -> 0:00:00.004994
   Key        -> 858    Max Time  -> 0:00:00.009368
   Key        -> 9    Min Time -> 0:00:00.001120
   avg        -> 0:00:00.004472
ITERATION  17 time 0:00:04.703588
   First Key  -> 0:00:00.003440
   Last Key   -> 0:00:00.005104
   Key        -> 529    Max Time  -> 0:00:00.008179
   Key        -> 8    Min Time -> 0:00:00.001105
   avg        -> 0:00:00.004688
ITERATION  18 time 0:00:05.300920
   First Key  -> 0:00:00.001851
   Last Key   -> 0:00:00.006987
   Key        -> 854    Max Time  -> 0:00:00.007299
   Key        -> 9    Min Time -> 0:00:00.001181
   avg        -> 0:00:00.005285
ITERATION  19 time 0:00:06.168681
   First Key  -> 0:00:00.001880
   Last Key   -> 0:00:00.005577
   Key        -> 911    Max Time  -> 0:00:00.009503
   Key        -> 9    Min Time -> 0:00:00.001119
   avg        -> 0:00:00.006152








----------------------------------------------------------------------

Comment By: Oleg Broytman (phd)
Date: 2011-07-14 20:56

Message:
No idea what is going on. Let's investigate a little further. What version
of SQLObject are you using? Can you add more prints inside the loop to see
what part of the program takes time?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=540673&aid=3366899&group_id=74338

[SQL-CVS] [ sqlobject-Support Requests-3366899 ] The speed of SQLObject is slower on the time

[SourceForge.net <no...@so...>]

Support Requests item #3366899, was opened at 2011-07-14 09:40
Message generated for change (Comment added) made by marobles
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=540673&aid=3366899&group_id=74338

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Priority: 5
Private: No
Submitted By: Rosa Robles (marobles)
Assigned to: Oleg Broytman (phd)
Summary: The speed of SQLObject is slower on the time

Initial Comment:
I had a test of a SQLObject and I had found a strange behaviour of SQLObject. On the time, the respond time is lesser.

Here, I put a example of test code :

from sqlobject.sqlbuilder import *
connection = connectionForURI('postgresql://test:test@localhost/test_sql_object')
sqlhub.processConnection = connection

class Person(SQLObject):
    firstName = StringCol()
    middleInitial = StringCol(length=1, default=None)
    lastName = StringCol()

iteration = 0
while 1:
    start = datetime.now()
    person_list = []
    for key in range(1000):
        person = Person.selectBy(firstName='name%s%s' % (iteration,key))[0]
        person_list.append(person)
   
    for person in person_list:
        person.expire()
        del person
    del person_list
    sqlhub.processConnection.cache.clear()
   
    end = datetime.now()
       
    print "TIME ",iteration,':' , end - start
    iteration = iteration + 1

In the result I can check that in each iteration the time is bigger. The first iteration has time of 1.37s, then the time increases and the 19 iteration has time of 5.83 s!

TIME  0 : 0:00:01.376392
TIME  1 : 0:00:01.663250
TIME  2 : 0:00:01.978639
TIME  3 : 0:00:02.319939
TIME  4 : 0:00:02.618213
TIME  5 : 0:00:02.723846
TIME  6 : 0:00:02.584007
TIME  7 : 0:00:02.868877
TIME  8 : 0:00:03.051937
TIME  9 : 0:00:03.389757
TIME  10 : 0:00:03.503036
TIME  11 : 0:00:03.563154
TIME  12 : 0:00:04.191539
TIME  13 : 0:00:04.274464
TIME  14 : 0:00:04.813206
TIME  15 : 0:00:04.677002
TIME  16 : 0:00:04.995708
TIME  17 : 0:00:05.126811
TIME  18 : 0:00:05.571279
TIME  19 : 0:00:05.832840

Why is happend this? 
Thanks in advance for your attention.
Cheers Rosa

----------------------------------------------------------------------

>Comment By: Rosa Robles (marobles)
Date: 2011-07-15 08:52

Message:
Thanks Oleg.
I'm using SQLObject 1.1.1, Python2.6, Postgres 8.4

Ok, I change the code to takes more times.

    for iteration in range(20):
        start = datetime.now()
        acum = timedelta(microseconds=0)
        times = []
        
        for key in range(1000):
            time_start = datetime.now()
            person = Person.selectBy(firstName='name%s%s' %
(iteration,key))[0]
            time_end = datetime.now()
            times.append(time_end-time_start)
            acum = acum + (time_end-time_start)
            
        end = datetime.now()
        
        print "ITERATION ",iteration,"time",end-start
        print "   First Key  ->", times[0] 
        print "   Last Key   ->",times[999] 
        print "   Key        ->",times.index(max(times)), "   Max Time 
->", max(times)
        print "   Key        ->",times.index(min(times)), "   Min Time
->", min(times)
        print "   avg        ->",acum/999
        
        Person.sqlmeta.expireAll()
        for k in connection.cache.caches.keys():
            connection.cache.caches[k].expireAll()
        sqlhub.getConnection().expireAll()
        connection.cache.clear()
        gc.collect()


And the result is:


ITERATION  0 time 0:00:01.302290
   First Key  -> 0:00:00.001555
   Last Key   -> 0:00:00.001445
   Key        -> 11    Max Time  -> 0:00:00.003216
   Key        -> 36    Min Time -> 0:00:00.001115
   avg        -> 0:00:00.001287
ITERATION  1 time 0:00:01.621580
   First Key  -> 0:00:00.001879
   Last Key   -> 0:00:00.001734
   Key        -> 924    Max Time  -> 0:00:00.009572
   Key        -> 22    Min Time -> 0:00:00.001076
   avg        -> 0:00:00.001606
ITERATION  2 time 0:00:01.851719
   First Key  -> 0:00:00.002328
   Last Key   -> 0:00:00.002064
   Key        -> 970    Max Time  -> 0:00:00.002544
   Key        -> 22    Min Time -> 0:00:00.001306
   avg        -> 0:00:00.001836
ITERATION  3 time 0:00:02.040802
   First Key  -> 0:00:00.002435
   Last Key   -> 0:00:00.002376
   Key        -> 557    Max Time  -> 0:00:00.009674
   Key        -> 25    Min Time -> 0:00:00.001538
   avg        -> 0:00:00.002026
ITERATION  4 time 0:00:02.427911
   First Key  -> 0:00:00.002906
   Last Key   -> 0:00:00.002065
   Key        -> 774    Max Time  -> 0:00:00.003263
   Key        -> 7    Min Time -> 0:00:00.001763
   avg        -> 0:00:00.002413
ITERATION  5 time 0:00:02.768819
   First Key  -> 0:00:00.003051
   Last Key   -> 0:00:00.002319
   Key        -> 920    Max Time  -> 0:00:00.007694
   Key        -> 11    Min Time -> 0:00:00.002007
   avg        -> 0:00:00.002753
ITERATION  6 time 0:00:02.537283
   First Key  -> 0:00:00.002676
   Last Key   -> 0:00:00.002549
   Key        -> 2    Max Time  -> 0:00:00.003512
   Key        -> 132    Min Time -> 0:00:00.002318
   avg        -> 0:00:00.002523
ITERATION  7 time 0:00:02.723405
   First Key  -> 0:00:00.005272
   Last Key   -> 0:00:00.002788
   Key        -> 79    Max Time  -> 0:00:00.007543
   Key        -> 12    Min Time -> 0:00:00.002457
   avg        -> 0:00:00.002709
ITERATION  8 time 0:00:02.947676
   First Key  -> 0:00:00.003161
   Last Key   -> 0:00:00.003028
   Key        -> 1    Max Time  -> 0:00:00.006068
   Key        -> 31    Min Time -> 0:00:00.002683
   avg        -> 0:00:00.002933
ITERATION  9 time 0:00:03.781268
   First Key  -> 0:00:00.004271
   Last Key   -> 0:00:00.003416
   Key        -> 930    Max Time  -> 0:00:00.011791
   Key        -> 5    Min Time -> 0:00:00.003029
   avg        -> 0:00:00.003765
ITERATION  10 time 0:00:03.842353
   First Key  -> 0:00:00.006544
   Last Key   -> 0:00:00.003394
   Key        -> 0    Max Time  -> 0:00:00.006544
   Key        -> 4    Min Time -> 0:00:00.003171
   avg        -> 0:00:00.003826
ITERATION  11 time 0:00:03.361368
   First Key  -> 0:00:00.001509
   Last Key   -> 0:00:00.003626
   Key        -> 602    Max Time  -> 0:00:00.009093
   Key        -> 9    Min Time -> 0:00:00.001089
   avg        -> 0:00:00.003346
ITERATION  12 time 0:00:03.581802
   First Key  -> 0:00:00.001511
   Last Key   -> 0:00:00.003912
   Key        -> 474    Max Time  -> 0:00:00.009027
   Key        -> 1    Min Time -> 0:00:00.001117
   avg        -> 0:00:00.003566
ITERATION  13 time 0:00:03.797965
   First Key  -> 0:00:00.001833
   Last Key   -> 0:00:00.004118
   Key        -> 100    Max Time  -> 0:00:00.005115
   Key        -> 9    Min Time -> 0:00:00.001126
   avg        -> 0:00:00.003782
ITERATION  14 time 0:00:04.147961
   First Key  -> 0:00:00.001851
   Last Key   -> 0:00:00.004362
   Key        -> 782    Max Time  -> 0:00:00.013857
   Key        -> 16    Min Time -> 0:00:00.001186
   avg        -> 0:00:00.004132
ITERATION  15 time 0:00:04.251830
   First Key  -> 0:00:00.001538
   Last Key   -> 0:00:00.004611
   Key        -> 518    Max Time  -> 0:00:00.009511
   Key        -> 9    Min Time -> 0:00:00.001114
   avg        -> 0:00:00.004236
ITERATION  16 time 0:00:04.487687
   First Key  -> 0:00:00.001541
   Last Key   -> 0:00:00.004994
   Key        -> 858    Max Time  -> 0:00:00.009368
   Key        -> 9    Min Time -> 0:00:00.001120
   avg        -> 0:00:00.004472
ITERATION  17 time 0:00:04.703588
   First Key  -> 0:00:00.003440
   Last Key   -> 0:00:00.005104
   Key        -> 529    Max Time  -> 0:00:00.008179
   Key        -> 8    Min Time -> 0:00:00.001105
   avg        -> 0:00:00.004688
ITERATION  18 time 0:00:05.300920
   First Key  -> 0:00:00.001851
   Last Key   -> 0:00:00.006987
   Key        -> 854    Max Time  -> 0:00:00.007299
   Key        -> 9    Min Time -> 0:00:00.001181
   avg        -> 0:00:00.005285
ITERATION  19 time 0:00:06.168681
   First Key  -> 0:00:00.001880
   Last Key   -> 0:00:00.005577
   Key        -> 911    Max Time  -> 0:00:00.009503
   Key        -> 9    Min Time -> 0:00:00.001119
   avg        -> 0:00:00.006152








----------------------------------------------------------------------

Comment By: Oleg Broytman (phd)
Date: 2011-07-14 18:56

Message:
No idea what is going on. Let's investigate a little further. What version
of SQLObject are you using? Can you add more prints inside the loop to see
what part of the program takes time?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=540673&aid=3366899&group_id=74338

[SQL-CVS] [ sqlobject-Patches-3367100 ] psycopg2 - autocommit

[SourceForge.net <no...@so...>]

Patches item #3367100, was opened at 2011-07-14 14:06
Message generated for change (Comment added) made by phd
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=540674&aid=3367100&group_id=74338

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
>Status: Closed
>Resolution: Accepted
Priority: 5
Private: No
Submitted By: El_Queen (pacoqueen)
>Assigned to: Oleg Broytman (phd)
Summary: psycopg2 - autocommit

Initial Comment:
Recently psycopg2 has added a new autcommit attribute (bool type). SQLObject raises an exception trying to call the autocommit function in pgconnection.py. This patch catch the exception and set autocommit the correct value if so, so it's backward compatible.

----------------------------------------------------------------------

>Comment By: Oleg Broytman (phd)
Date: 2011-07-14 21:22

Message:
Applied, refactored and committed in the revision 4428 in the trunk. Will
be in the next release. Thank you very much!

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=540674&aid=3367100&group_id=74338

[SQL-CVS] r4428 - in SQLObject/trunk: docs sqlobject/postgres

[<sub...@co...>]

Author: phd
Date: Thu Jul 14 11:20:56 2011
New Revision: 4428

Log:
A bug caused by psycopg2 recently added a new boolean not callable autocommit attribute was fixed.

Modified:
   SQLObject/trunk/docs/News.txt
   SQLObject/trunk/sqlobject/postgres/pgconnection.py

Modified: SQLObject/trunk/docs/News.txt
==============================================================================
--- SQLObject/trunk/docs/News.txt	Mon Jul 11 11:42:54 2011	(r4427)
+++ SQLObject/trunk/docs/News.txt	Thu Jul 14 11:20:56 2011	(r4428)
@@ -13,6 +13,9 @@
 Features & Interface
 --------------------
 
+* A bug caused by psycopg2 recently added a new boolean not callable
+  autocommit attribute was fixed.
+
 SQLObject 1.1.1
 ===============
 

Modified: SQLObject/trunk/sqlobject/postgres/pgconnection.py
==============================================================================
--- SQLObject/trunk/sqlobject/postgres/pgconnection.py	Mon Jul 11 11:42:54 2011	(r4427)
+++ SQLObject/trunk/sqlobject/postgres/pgconnection.py	Thu Jul 14 11:20:56 2011	(r4428)
@@ -118,7 +118,10 @@
     def _setAutoCommit(self, conn, auto):
         # psycopg2 does not have an autocommit method.
         if hasattr(conn, 'autocommit'):
-            conn.autocommit(auto)
+            try:
+                conn.autocommit(auto)
+            except TypeError:
+                conn.autocommit = auto
 
     def makeConnection(self):
         try:
@@ -128,10 +131,7 @@
                 conn = self.module.connect(**self.dsn_dict)
         except self.module.OperationalError, e:
             raise self.module.OperationalError("%s; used connection string %r" % (e, self.dsn))
-        if self.autoCommit:
-            # psycopg2 does not have an autocommit method.
-            if hasattr(conn, 'autocommit'):
-                conn.autocommit(1)
+        if self.autoCommit: self._setAutoCommit(conn, 1)
         c = conn.cursor()
         if self.schema:
             c.execute("SET search_path TO " + self.schema)

[SQL-CVS] [ sqlobject-Support Requests-3366899 ] The speed of SQLObject is slower on the time

[SourceForge.net <no...@so...>]

Support Requests item #3366899, was opened at 2011-07-14 11:40
Message generated for change (Comment added) made by phd
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=540673&aid=3366899&group_id=74338

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Priority: 5
Private: No
Submitted By: Rosa Robles (marobles)
>Assigned to: Oleg Broytman (phd)
Summary: The speed of SQLObject is slower on the time

Initial Comment:
I had a test of a SQLObject and I had found a strange behaviour of SQLObject. On the time, the respond time is lesser.

Here, I put a example of test code :

from sqlobject.sqlbuilder import *
connection = connectionForURI('postgresql://test:test@localhost/test_sql_object')
sqlhub.processConnection = connection

class Person(SQLObject):
    firstName = StringCol()
    middleInitial = StringCol(length=1, default=None)
    lastName = StringCol()

iteration = 0
while 1:
    start = datetime.now()
    person_list = []
    for key in range(1000):
        person = Person.selectBy(firstName='name%s%s' % (iteration,key))[0]
        person_list.append(person)
   
    for person in person_list:
        person.expire()
        del person
    del person_list
    sqlhub.processConnection.cache.clear()
   
    end = datetime.now()
       
    print "TIME ",iteration,':' , end - start
    iteration = iteration + 1

In the result I can check that in each iteration the time is bigger. The first iteration has time of 1.37s, then the time increases and the 19 iteration has time of 5.83 s!

TIME  0 : 0:00:01.376392
TIME  1 : 0:00:01.663250
TIME  2 : 0:00:01.978639
TIME  3 : 0:00:02.319939
TIME  4 : 0:00:02.618213
TIME  5 : 0:00:02.723846
TIME  6 : 0:00:02.584007
TIME  7 : 0:00:02.868877
TIME  8 : 0:00:03.051937
TIME  9 : 0:00:03.389757
TIME  10 : 0:00:03.503036
TIME  11 : 0:00:03.563154
TIME  12 : 0:00:04.191539
TIME  13 : 0:00:04.274464
TIME  14 : 0:00:04.813206
TIME  15 : 0:00:04.677002
TIME  16 : 0:00:04.995708
TIME  17 : 0:00:05.126811
TIME  18 : 0:00:05.571279
TIME  19 : 0:00:05.832840

Why is happend this? 
Thanks in advance for your attention.
Cheers Rosa

----------------------------------------------------------------------

>Comment By: Oleg Broytman (phd)
Date: 2011-07-14 20:56

Message:
No idea what is going on. Let's investigate a little further. What version
of SQLObject are you using? Can you add more prints inside the loop to see
what part of the program takes time?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=540673&aid=3366899&group_id=74338

[SQL-CVS] [ sqlobject-Patches-3367100 ] psycopg2 - autocommit

[SourceForge.net <no...@so...>]

Patches item #3367100, was opened at 2011-07-14 12:06
Message generated for change (Tracker Item Submitted) made by pacoqueen
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=540674&aid=3367100&group_id=74338

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: El_Queen (pacoqueen)
Assigned to: Nobody/Anonymous (nobody)
Summary: psycopg2 - autocommit

Initial Comment:
Recently psycopg2 has added a new autcommit attribute (bool type). SQLObject raises an exception trying to call the autocommit function in pgconnection.py. This patch catch the exception and set autocommit the correct value if so, so it's backward compatible.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=540674&aid=3367100&group_id=74338

[SQL-CVS] [ sqlobject-Support Requests-3366899 ] The speed of SQLObject is slower on the time

[SourceForge.net <no...@so...>]

Support Requests item #3366899, was opened at 2011-07-14 09:40
Message generated for change (Tracker Item Submitted) made by marobles
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=540673&aid=3366899&group_id=74338

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Priority: 5
Private: No
Submitted By: Rosa Robles (marobles)
Assigned to: Nobody/Anonymous (nobody)
Summary: The speed of SQLObject is slower on the time

Initial Comment:
I had a test of a SQLObject and I had found a strange behaviour of SQLObject. On the time, the respond time is lesser.

Here, I put a example of test code :

from sqlobject.sqlbuilder import *
connection = connectionForURI('postgresql://test:test@localhost/test_sql_object')
sqlhub.processConnection = connection

class Person(SQLObject):
    firstName = StringCol()
    middleInitial = StringCol(length=1, default=None)
    lastName = StringCol()

iteration = 0
while 1:
    start = datetime.now()
    person_list = []
    for key in range(1000):
        person = Person.selectBy(firstName='name%s%s' % (iteration,key))[0]
        person_list.append(person)
   
    for person in person_list:
        person.expire()
        del person
    del person_list
    sqlhub.processConnection.cache.clear()
   
    end = datetime.now()
       
    print "TIME ",iteration,':' , end - start
    iteration = iteration + 1

In the result I can check that in each iteration the time is bigger. The first iteration has time of 1.37s, then the time increases and the 19 iteration has time of 5.83 s!

TIME  0 : 0:00:01.376392
TIME  1 : 0:00:01.663250
TIME  2 : 0:00:01.978639
TIME  3 : 0:00:02.319939
TIME  4 : 0:00:02.618213
TIME  5 : 0:00:02.723846
TIME  6 : 0:00:02.584007
TIME  7 : 0:00:02.868877
TIME  8 : 0:00:03.051937
TIME  9 : 0:00:03.389757
TIME  10 : 0:00:03.503036
TIME  11 : 0:00:03.563154
TIME  12 : 0:00:04.191539
TIME  13 : 0:00:04.274464
TIME  14 : 0:00:04.813206
TIME  15 : 0:00:04.677002
TIME  16 : 0:00:04.995708
TIME  17 : 0:00:05.126811
TIME  18 : 0:00:05.571279
TIME  19 : 0:00:05.832840

Why is happend this? 
Thanks in advance for your attention.
Cheers Rosa

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=540673&aid=3366899&group_id=74338

[SQL-CVS] r4427 - in SQLObject/trunk: . docs

[<sub...@co...>]

Author: phd
Date: Mon Jul 11 11:42:54 2011
New Revision: 4427

Log:
CheeseShop started redirecting to PyPI.

Modified:
   SQLObject/trunk/docs/DeveloperGuide.txt
   SQLObject/trunk/docs/download.txt
   SQLObject/trunk/setup.py

Modified: SQLObject/trunk/docs/DeveloperGuide.txt
==============================================================================
--- SQLObject/trunk/docs/DeveloperGuide.txt	Mon Jul 11 11:42:12 2011	(r4426)
+++ SQLObject/trunk/docs/DeveloperGuide.txt	Mon Jul 11 11:42:54 2011	(r4427)
@@ -189,7 +189,7 @@
 Please write documentation.  Documentation should live in the docs/
 directory.  Pudge converts documentation from Restructured Text to
 HTML.  It presently requires kid 0.9.3, which must be obtained
-separately (for instance, from http://cheeseshop.python.org/pypi/kid/0.9.3)
+separately (for instance, from http://pypi.python.org/pypi/kid/0.9.3)
 
 .. image:: http://sflogo.sourceforge.net/sflogo.php?group_id=74338&type=10
    :target: http://sourceforge.net/projects/sqlobject

Modified: SQLObject/trunk/docs/download.txt
==============================================================================
--- SQLObject/trunk/docs/download.txt	Mon Jul 11 11:42:12 2011	(r4426)
+++ SQLObject/trunk/docs/download.txt	Mon Jul 11 11:42:54 2011	(r4427)
@@ -1,8 +1,8 @@
 Download SQLObject
 ++++++++++++++++++
 
-The latest releases are always available on the `Python Cheese Shop
-<http://cheeseshop.python.org/pypi/SQLObject>`_, and is installable
+The latest releases are always available on the `Python Package Index
+<http://pypi.python.org/pypi/SQLObject>`_, and is installable
 with `easy_install
 <http://peak.telecommunity.com/DevCenter/EasyInstall>`_.
 

Modified: SQLObject/trunk/setup.py
==============================================================================
--- SQLObject/trunk/setup.py	Mon Jul 11 11:42:12 2011	(r4426)
+++ SQLObject/trunk/setup.py	Mon Jul 11 11:42:54 2011	(r4427)
@@ -51,7 +51,7 @@
       author="Ian Bicking",
       author_email="ia...@co...",
       url="http://sqlobject.org/devel/",
-      download_url="http://cheeseshop.python.org/pypi/SQLObject/%s" % version,
+      download_url="http://pypi.python.org/pypi/SQLObject/%s" % version,
       license="LGPL",
       packages=["sqlobject"] + ['sqlobject.%s' % package for package in subpackages],
       scripts=["scripts/sqlobject-admin", "scripts/sqlobject-convertOldURI"],
@@ -102,7 +102,7 @@
 http://news.gmane.org/gmane.comp.python.sqlobject
 
 Download:
-http://cheeseshop.python.org/pypi/SQLObject/@@
+http://pypi.python.org/pypi/SQLObject/@@
 
 News and changes:
 http://sqlobject.org/docs/News.html

[SQL-CVS] r4426 - SQLObject/trunk/docs

[<sub...@co...>]

Author: phd
Date: Mon Jul 11 11:42:12 2011
New Revision: 4426

Log:
Reordered items. Removed some duplicates.

Modified:
   SQLObject/trunk/docs/TODO.txt

Modified: SQLObject/trunk/docs/TODO.txt
==============================================================================
--- SQLObject/trunk/docs/TODO.txt	Tue Jul  5 09:20:49 2011	(r4425)
+++ SQLObject/trunk/docs/TODO.txt	Mon Jul 11 11:42:12 2011	(r4426)
@@ -1,8 +1,6 @@
 TODO
 ----
 
-* Deprecate parsing sqlobject.__doc__ to find out version.
-
 * Allow to override ConsoleWriter/LogWriter classes and makeDebugWriter
   function.
 
@@ -20,6 +18,8 @@
           tableParamSQL = 'ENGINE InnoDB'
           tableParamSQL = {'mysql': 'ENGINE InnoDB'}
 
+* SQLObject.fastInsert().
+
 * List tables in the DB. The query in MySQL is ``SHOW TABLES``; in SQLite it is
 
     SELECT name FROM sqlite_master WHERE type='table' ORDER BY name
@@ -38,8 +38,6 @@
 * Invert tests isinstance(obj, (tuple, list)) to not isinstance(obj, basestr)
   to allow any iterable.
 
-* SQLObject.fastInsert().
-
 * Always use .lazyIter().
 
 * Optimize Iteration.next() - use cursor.fetchmany().
@@ -60,11 +58,13 @@
         function1 = ExpressionCol(func.my_function(MyClass.q.col1))
         function2 = ExpressionCol('sum(col2)')
 
-* Catch all DB API exceptions and convert them to sqlobject.dberrors.
+* A hierarchy of exceptions. SQLObject should translate exceptions from
+  low-level drivers to a consistent set of high-level exceptions.
 
 * Memcache.
 
-* Parametrized queries.
+* Refactor ``DBConnection`` to use parameterized queries instead of
+  generating query strings.
 
 * PREPARE/EXECUTE
 
@@ -126,12 +126,6 @@
 * More kinds of joins, and more powerful join results (closer to how
   `select` works).
 
-* Refactor ``DBConnection`` to use parameterized queries instead of
-  generating query strings.
-
-* A hierarchy of exceptions. SQLObject should translate exceptions from
-  low-level drivers to a consistent set of high-level exceptions.
-
 * Merge SQLObject.create*, .create*SQL methods with DBPI.create* methods.
 
 * Made SQLObject unicode-based instead of just unicode-aware. All internal

[SQL-CVS] r4425 - in SQLObject/trunk/sqlobject: . tests

[<sub...@co...>]

Author: phd
Date: Tue Jul  5 09:20:49 2011
New Revision: 4425

Log:
Neil Muller made the test suite runs with both py.test 1.3 and 2.0.

Modified:
   SQLObject/trunk/sqlobject/conftest.py
   SQLObject/trunk/sqlobject/tests/test_boundattributes.py
   SQLObject/trunk/sqlobject/tests/test_paste.py

Modified: SQLObject/trunk/sqlobject/conftest.py
==============================================================================
--- SQLObject/trunk/sqlobject/conftest.py	Tue Jul  5 09:19:27 2011	(r4424)
+++ SQLObject/trunk/sqlobject/conftest.py	Tue Jul  5 09:20:49 2011	(r4425)
@@ -31,27 +31,32 @@
     'mssql': 'mssql://sa:@127.0.0.1/test'
     }
 
-Option = py.test.config.Option
-option = py.test.config.addoptions(
-    "SQLObject options",
-    Option('-D', '--Database',
+def pytest_addoption(parser):
+    """Add the SQLObject options"""
+    parser.addoption('-D', '--Database',
            action="store", dest="Database", default='sqlite',
            help="The database to run the tests under (default sqlite).  "
            "Can also use an alias from: %s"
-           % (', '.join(connectionShortcuts.keys()))),
-    Option('-S', '--SQL',
+           % (', '.join(connectionShortcuts.keys())))
+    parser.addoption('-S', '--SQL',
            action="store_true", dest="show_sql", default=False,
            help="Show SQL from statements (when capturing stdout the "
-           "SQL is only displayed when a test fails)"),
-    Option('-O', '--SQL-output',
+           "SQL is only displayed when a test fails)")
+    parser.addoption('-O', '--SQL-output',
            action="store_true", dest="show_sql_output", default=False,
            help="Show output from SQL statements (when capturing "
-           "stdout the output is only displayed when a test fails)"),
-    Option('-E', '--events',
+           "stdout the output is only displayed when a test fails)")
+    parser.addoption('-E', '--events',
            action="store_true", dest="debug_events", default=False,
            help="Debug events (print information about events as they are "
-           "sent)"),
-    )
+           "sent)")
+
+option = None
+
+def pytest_configure(config):
+    """Make cmdline arguments available to dbtest"""
+    global option
+    option = config.option
 
 class SQLObjectClass(py.test.collect.Class):
     def run(self):

Modified: SQLObject/trunk/sqlobject/tests/test_boundattributes.py
==============================================================================
--- SQLObject/trunk/sqlobject/tests/test_boundattributes.py	Tue Jul  5 09:19:27 2011	(r4424)
+++ SQLObject/trunk/sqlobject/tests/test_boundattributes.py	Tue Jul  5 09:20:49 2011	(r4425)
@@ -1,7 +1,8 @@
 from sqlobject import declarative
 from sqlobject import boundattributes
+import py.test
 
-disabled = True
+pytestmark = py.test.mark.skipif('True')
 
 class TestMe(object):
 

Modified: SQLObject/trunk/sqlobject/tests/test_paste.py
==============================================================================
--- SQLObject/trunk/sqlobject/tests/test_paste.py	Tue Jul  5 09:19:27 2011	(r4424)
+++ SQLObject/trunk/sqlobject/tests/test_paste.py	Tue Jul  5 09:20:49 2011	(r4425)
@@ -1,9 +1,11 @@
 from dbtest import *
 from sqlobject import sqlhub, SQLObject, StringCol
+import py.test
 try:
     from sqlobject.wsgi_middleware import make_middleware
 except ImportError:
-    disabled = True
+    pytestmark = py.test.mark.skipif('True')
+
 
 class NameOnly(SQLObject):
     name = StringCol()

[SQL-CVS] r4424 - in SQLObject/trunk: docs sqlobject/include/pydispatch

[<sub...@co...>]

Author: phd
Date: Tue Jul  5 09:19:27 2011
New Revision: 4424

Log:
Neil Muller fixed a bug in pydispatcher that prevented to install it under pypy.

Modified:
   SQLObject/trunk/docs/Authors.txt
   SQLObject/trunk/sqlobject/include/pydispatch/robustapply.py

Modified: SQLObject/trunk/docs/Authors.txt
==============================================================================
--- SQLObject/trunk/docs/Authors.txt	Fri Jul  1 08:59:32 2011	(r4423)
+++ SQLObject/trunk/docs/Authors.txt	Tue Jul  5 09:19:27 2011	(r4424)
@@ -24,6 +24,7 @@
 * Christopher Singley <csingley at gmail.com>
 * David Keeney <dkeeney at rdbhost.com>
 * Daniel Fetchinson <fetchinson at googlemail.com>
+* Neil Muller <drnlmuller+sqlobject at gmail.com>
 * Oleg Broytman <ph...@ph...>
 
 .. image:: http://sflogo.sourceforge.net/sflogo.php?group_id=74338&type=10

Modified: SQLObject/trunk/sqlobject/include/pydispatch/robustapply.py
==============================================================================
--- SQLObject/trunk/sqlobject/include/pydispatch/robustapply.py	Fri Jul  1 08:59:32 2011	(r4423)
+++ SQLObject/trunk/sqlobject/include/pydispatch/robustapply.py	Tue Jul  5 09:19:27 2011	(r4424)
@@ -14,17 +14,14 @@
 	If fromMethod is true, then the callable already
 	has its first argument bound
 	"""
-	if hasattr(receiver, '__call__'):
-		# receiver is a class instance; assume it is callable.
-		# Reassign receiver to the actual method that will be called.
-		if hasattr( receiver.__call__, 'im_func') or hasattr( receiver.__call__, 'im_code'):
-			receiver = receiver.__call__
-	if hasattr( receiver, 'im_func' ):
-		# an instance-method...
+	if hasattr(receiver, 'im_func'):
 		return receiver, receiver.im_func.func_code, 1
-	elif not hasattr( receiver, 'func_code'):
+	elif hasattr(receiver, 'func_code'):
+		return receiver, receiver.func_code, 0
+	elif hasattr(receiver, '__call__'):
+		return function(receiver.__call__)
+	else:
 		raise ValueError('unknown reciever type %s %s'%(receiver, type(receiver)))
-	return receiver, receiver.func_code, 0
 
 def robustApply(receiver, *arguments, **named):
 	"""Call receiver with arguments and an appropriate subset of named

[SQL-CVS] r4423 - in SQLObject/tags/1.1.1: . sqlobject

[<sub...@co...>]

Author: phd
Date: Fri Jul  1 08:59:32 2011
New Revision: 4423

Log:
Stable version 1.1.1.

Modified:
   SQLObject/tags/1.1.1/README.txt
   SQLObject/tags/1.1.1/setup.cfg
   SQLObject/tags/1.1.1/sqlobject/__init__.py
   SQLObject/tags/1.1.1/sqlobject/__version__.py
   SQLObject/tags/1.1.1/sqlobject/main.py

Modified: SQLObject/tags/1.1.1/README.txt
==============================================================================
--- SQLObject/tags/1.1.1/README.txt	Fri Jul  1 08:57:05 2011	(r4422)
+++ SQLObject/tags/1.1.1/README.txt	Fri Jul  1 08:59:32 2011	(r4423)
@@ -1,5 +1,5 @@
-SQLObject 1.1
-=============
+SQLObject 1.1.1
+===============
 
 Thanks for looking at SQLObject.  SQLObject is an object-relational
 mapper, i.e., a library that will wrap your database tables in Python

Modified: SQLObject/tags/1.1.1/setup.cfg
==============================================================================
--- SQLObject/tags/1.1.1/setup.cfg	Fri Jul  1 08:57:05 2011	(r4422)
+++ SQLObject/tags/1.1.1/setup.cfg	Fri Jul  1 08:59:32 2011	(r4423)
@@ -5,10 +5,6 @@
 [easy_install]
 #find_links = http://svn.pythonpaste.org/package_index.html
 
-[egg_info]
-tag_build = dev
-tag_svn_revision = true
-
 [pudge]
 theme = pythonpaste.org
 docs = docs/index.txt docs/Authors.txt docs/DeveloperGuide.txt docs/FAQ.txt

Modified: SQLObject/tags/1.1.1/sqlobject/__init__.py
==============================================================================
--- SQLObject/tags/1.1.1/sqlobject/__init__.py	Fri Jul  1 08:57:05 2011	(r4422)
+++ SQLObject/tags/1.1.1/sqlobject/__init__.py	Fri Jul  1 08:59:32 2011	(r4423)
@@ -1,5 +1,5 @@
 """
-SQLObject 1.1
+SQLObject 1.1.1
 """
 from __version__ import version, version_info
 

Modified: SQLObject/tags/1.1.1/sqlobject/__version__.py
==============================================================================
--- SQLObject/tags/1.1.1/sqlobject/__version__.py	Fri Jul  1 08:57:05 2011	(r4422)
+++ SQLObject/tags/1.1.1/sqlobject/__version__.py	Fri Jul  1 08:59:32 2011	(r4423)
@@ -1,8 +1,8 @@
 
-version = '1.1'
+version = '1.1.1'
 major   = 1
 minor   = 1
-micro   = 0
-release_level = 'branch'
+micro   = 1
+release_level = 'final'
 serial  = 0
 version_info = (major, minor, micro, release_level, serial)

Modified: SQLObject/tags/1.1.1/sqlobject/main.py
==============================================================================
--- SQLObject/tags/1.1.1/sqlobject/main.py	Fri Jul  1 08:57:05 2011	(r4422)
+++ SQLObject/tags/1.1.1/sqlobject/main.py	Fri Jul  1 08:59:32 2011	(r4423)
@@ -1,6 +1,6 @@
 """
-SQLObject 1.1
--------------
+SQLObject 1.1.1
+---------------
 
 :author: Ian Bicking <ia...@co...>
 

[SQL-CVS] r4422 - SQLObject/tags/1.1.1

[<sub...@co...>]

Author: phd
Date: Fri Jul  1 08:57:05 2011
New Revision: 4422

Log:
Tagging 1.1.1

Added:
   SQLObject/tags/1.1.1/
      - copied from r4421, SQLObject/branches/1.1/

[SQL-CVS] r4421 - SQLObject/trunk/docs

[<sub...@co...>]

Author: phd
Date: Fri Jul  1 08:55:24 2011
New Revision: 4421

Log:
Merged revision 4420 from branch 1.1: releasing 1.1.1.

Modified:
   SQLObject/trunk/docs/News.txt

Modified: SQLObject/trunk/docs/News.txt
==============================================================================
--- SQLObject/trunk/docs/News.txt	Fri Jul  1 08:54:21 2011	(r4420)
+++ SQLObject/trunk/docs/News.txt	Fri Jul  1 08:55:24 2011	(r4421)
@@ -16,6 +16,8 @@
 SQLObject 1.1.1
 ===============
 
+Released 1 Jul 2011.
+
 * Parsing sqlobject.__doc__ for version number is declared obsolete. Use
   sqlobject.version or version_info.