sqlobject-discuss Mailing List for SQLObject (Page 53)
SQLObject is a Python ORM.
Brought to you by:
ianbicking,
phd
You can subscribe to this list here.
2003 |
Jan
|
Feb
(2) |
Mar
(43) |
Apr
(204) |
May
(208) |
Jun
(102) |
Jul
(113) |
Aug
(63) |
Sep
(88) |
Oct
(85) |
Nov
(95) |
Dec
(62) |
---|---|---|---|---|---|---|---|---|---|---|---|---|
2004 |
Jan
(38) |
Feb
(93) |
Mar
(125) |
Apr
(89) |
May
(66) |
Jun
(65) |
Jul
(53) |
Aug
(65) |
Sep
(79) |
Oct
(60) |
Nov
(171) |
Dec
(176) |
2005 |
Jan
(264) |
Feb
(260) |
Mar
(145) |
Apr
(153) |
May
(192) |
Jun
(166) |
Jul
(265) |
Aug
(340) |
Sep
(300) |
Oct
(469) |
Nov
(316) |
Dec
(235) |
2006 |
Jan
(236) |
Feb
(156) |
Mar
(229) |
Apr
(221) |
May
(257) |
Jun
(161) |
Jul
(97) |
Aug
(169) |
Sep
(159) |
Oct
(400) |
Nov
(136) |
Dec
(134) |
2007 |
Jan
(152) |
Feb
(101) |
Mar
(115) |
Apr
(120) |
May
(129) |
Jun
(82) |
Jul
(118) |
Aug
(82) |
Sep
(30) |
Oct
(101) |
Nov
(137) |
Dec
(53) |
2008 |
Jan
(83) |
Feb
(139) |
Mar
(55) |
Apr
(69) |
May
(82) |
Jun
(31) |
Jul
(66) |
Aug
(30) |
Sep
(21) |
Oct
(37) |
Nov
(41) |
Dec
(65) |
2009 |
Jan
(69) |
Feb
(46) |
Mar
(22) |
Apr
(20) |
May
(39) |
Jun
(30) |
Jul
(36) |
Aug
(58) |
Sep
(38) |
Oct
(20) |
Nov
(10) |
Dec
(11) |
2010 |
Jan
(24) |
Feb
(63) |
Mar
(22) |
Apr
(72) |
May
(8) |
Jun
(13) |
Jul
(35) |
Aug
(23) |
Sep
(12) |
Oct
(26) |
Nov
(11) |
Dec
(30) |
2011 |
Jan
(15) |
Feb
(44) |
Mar
(36) |
Apr
(26) |
May
(27) |
Jun
(10) |
Jul
(28) |
Aug
(12) |
Sep
|
Oct
|
Nov
(17) |
Dec
(16) |
2012 |
Jan
(12) |
Feb
(31) |
Mar
(23) |
Apr
(14) |
May
(10) |
Jun
(26) |
Jul
|
Aug
(2) |
Sep
(2) |
Oct
(1) |
Nov
|
Dec
(6) |
2013 |
Jan
(4) |
Feb
(5) |
Mar
|
Apr
(4) |
May
(13) |
Jun
(7) |
Jul
(5) |
Aug
(15) |
Sep
(25) |
Oct
(18) |
Nov
(7) |
Dec
(3) |
2014 |
Jan
(1) |
Feb
(5) |
Mar
|
Apr
(3) |
May
(3) |
Jun
(2) |
Jul
(4) |
Aug
(5) |
Sep
|
Oct
(11) |
Nov
|
Dec
(62) |
2015 |
Jan
(8) |
Feb
(3) |
Mar
(15) |
Apr
|
May
|
Jun
(6) |
Jul
|
Aug
(6) |
Sep
|
Oct
|
Nov
|
Dec
(19) |
2016 |
Jan
(2) |
Feb
|
Mar
(2) |
Apr
(4) |
May
(3) |
Jun
(7) |
Jul
(14) |
Aug
(13) |
Sep
(6) |
Oct
(2) |
Nov
(3) |
Dec
|
2017 |
Jan
(6) |
Feb
(14) |
Mar
(2) |
Apr
|
May
(1) |
Jun
|
Jul
|
Aug
(1) |
Sep
|
Oct
(4) |
Nov
(3) |
Dec
|
2018 |
Jan
|
Feb
(1) |
Mar
|
Apr
|
May
|
Jun
(1) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
2019 |
Jan
|
Feb
(1) |
Mar
|
Apr
(44) |
May
(1) |
Jun
|
Jul
|
Aug
|
Sep
(1) |
Oct
|
Nov
|
Dec
(1) |
2020 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
(1) |
Nov
|
Dec
(1) |
2021 |
Jan
|
Feb
(1) |
Mar
|
Apr
|
May
|
Jun
(3) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
2022 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
(1) |
Oct
|
Nov
|
Dec
(1) |
2023 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
(1) |
Sep
|
Oct
(1) |
Nov
(2) |
Dec
|
2024 |
Jan
|
Feb
|
Mar
|
Apr
(4) |
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
(1) |
2025 |
Jan
|
Feb
(1) |
Mar
(1) |
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
From: Dan P. <da...@ag...> - 2009-06-11 09:31:04
|
On 11 Jun 2009, at 11:23, jonhattan wrote: > Sam's Lists escribió: >> So I keep humming along on this upgrade of someone else's code from >> old versions of SQLObject and Mysql to the current SQLObject and >> MySQL >> 5... >> >> The current problem is that I have a table with the column name >> 'condition' which apparently is not a reserved word in MySQL 4.0 but >> is a reserved word in 5.0. >> >> When SQLObject translates its stuff to sql it does not put quotes >> around the column names. If it only would everything would work >> fine. >> >> I could rename the column, but the word is used a lot in the program >> in different ways, so it will get confusing to make sure I only >> change >> the right ones . >> >> Is there a way to force SQLObject to quote it properly? >> > you can force the name of the column in the database this way: > > condition = StringCol(dbName='_condition') Or simply quote it and you do not need to change anything: condition = StringCol(dbName='`condition`') -- Dan |
From: Christian S. <chr...@un...> - 2009-06-11 09:30:53
|
Dear sqlobject users, Is it possible to call functions with func() in a postgresql schema other than the public one? I am using SQLObject 0.10. Thanks a lot for your help! Regards, Christian. |
From: jonhattan <jon...@fa...> - 2009-06-11 08:41:56
|
Sam's Lists escribió: > So I keep humming along on this upgrade of someone else's code from > old versions of SQLObject and Mysql to the current SQLObject and MySQL > 5... > > The current problem is that I have a table with the column name > 'condition' which apparently is not a reserved word in MySQL 4.0 but > is a reserved word in 5.0. > > When SQLObject translates its stuff to sql it does not put quotes > around the column names. If it only would everything would work fine. > > I could rename the column, but the word is used a lot in the program > in different ways, so it will get confusing to make sure I only change > the right ones . > > Is there a way to force SQLObject to quote it properly? > you can force the name of the column in the database this way: condition = StringCol(dbName='_condition') > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------------ > Crystal Reports - New Free Runtime and 30 Day Trial > Check out the new simplified licensing option that enables unlimited > royalty-free distribution of the report engine for externally facing > server and web deployment. > http://p.sf.net/sfu/businessobjects > ------------------------------------------------------------------------ > > _______________________________________________ > sqlobject-discuss mailing list > sql...@li... > https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss > |
From: Simon C. <hod...@gm...> - 2009-06-11 08:27:48
|
On Thu, Jun 11, 2009 at 3:06 AM, Sam's Lists<sam...@gm...> wrote: > How can I make sqlObject seemlessly let me refer to columns named in > the long style just using id? I think this is more of a Python question than a SQLObject question, although it may be a little trickier in the SQLObject case because SQLObject already uses a lot of attribute magic. One approach might be to just add .id properties to all your classes: >>> class EventQueueID(object): ... def __init__(self): ... self.EventQueueID = 5 ... >>> e = EventQueueID() >>> e.id Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'EventQueueID' object has no attribute 'id' >>> def getid(self): ... return getattr(self, self.__class__.__name__) ... >>> def setid(self, x): ... return setattr(self, self.__class__.__name__, x) ... >>> EventQueueID.id = property(fget=getid, fset=setid) >>> e.id 5 >>> e.id = 7 >>> e.id 7 I don't know if SQLObject has any magic that would break this, but I think it should work. |
From: Sam's L. <sam...@gm...> - 2009-06-11 04:40:45
|
So I keep humming along on this upgrade of someone else's code from old versions of SQLObject and Mysql to the current SQLObject and MySQL 5... The current problem is that I have a table with the column name 'condition' which apparently is not a reserved word in MySQL 4.0 but is a reserved word in 5.0. When SQLObject translates its stuff to sql it does not put quotes around the column names. If it only would everything would work fine. I could rename the column, but the word is used a lot in the program in different ways, so it will get confusing to make sure I only change the right ones . Is there a way to force SQLObject to quote it properly? |
From: Sam's L. <sam...@gm...> - 2009-06-11 01:06:19
|
I posted this on the TurboGears mailing list but now I'm posting it here in hopes for more help. Essentially I am upgrading from SQLObject 0.7.1dev_r1653 and MySQL 4.x something to MySQL 5 something and SQLObject 0.10.6 I have this very complicated legacy TurboGears application that I'm tasked with upgrading to 1.0.8 which has laid stagnant since the days of early .99 series. The original author used the following a lot in his table definitions: class sqlmeta: style = MixedCaseStyle(longID=True) What this means is that instead of the id column being named 'id' it takes on the name of the table...so if the table is called EventQueue, the id column name is EventQueueID. So far so good....but here's where it gets weird. It appears that under the really old versions of TurboGears one could refer to this EventQueueID as just .id and everything would work the way you want it to. (This appears to only be done in the app under kid templates). But now, under TG 1.0.8/the latest version of SQLObject that doesn't work. Either I have to rename the column to id or I have to refer to it as EventQueueID. How can I make sqlObject seemlessly let me refer to columns named in the long style just using id? Thanks |
From: Daniel F. <fet...@go...> - 2009-06-09 01:16:44
|
> Hello! I'm leaving the town for a vacation. Will be back at June 20. Please > help each other. > > Oleg. Have fun! Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown |
From: Herwig H. <hho...@gm...> - 2009-06-06 10:20:55
|
Thanks for your answer I messed up the code snippet; forgot the commit. Please check updated attachment. > AFAIU, what is going on there is: > > -- the code opens the first connection (without a transaction); > -- a new table is created via connection 1; > -- a new row is inserted via connection 1; > -- the code opens another connection, and BEGINs a transaction ; > -- the code draws two copies of the row via connection1 and connection2; > -- the code changes the attribute .foo of the FIRST copy; > -- the code prints the attribute .foo of the SECOND copy that was drawn via > another transaction, and the value is still 'init' because it isn't > changed - the code hasn't changed trans_dummy.foo yet. > I intended to illustrate following behavior: Two instances of the same class with same id; one pulled via a connection (with cache=False); one pulled via a transaction createt from that same connection; --> Commits to the transaction _do_ invalidate the instance pulled via connection (thanks to cache=False); --> Changes to the connection instance _don't_ invalidate the transaction instance; This asymmetric behavior seems strange to me, since (as far as i understood the code), the transaction object even uses the same cache; it's using the same DBConnection instance, after all. I use SQLObject for a GUI app, so I need long living instances; transactionally separated. Can I evade manually syncing all the transactions. If not, what's the best way to invalidate a transaction's cache for a given instance/all instances of given class. Are there any docs on this? regards Herwig Hochleitner |
From: Oleg B. <ph...@ph...> - 2009-06-05 22:46:44
|
I'll try to answer briefly and then will leave the computer... On Fri, Jun 05, 2009 at 11:21:49PM +0200, Herwig Hochleitner wrote: > Writing 'connection_val' to connection foo > Value of transaction foo: 'init' > Writing 'transaction_val' to transaction foo > Value of connection foo: 'connection_val' > import sqlobject > # Turn off caching so that multiple connections (ie transactions) sync properly > conn = sqlobject.connectionForURI("sqlite:/:memory:", debug=True, logger="database.query", loglevel="debug", cache=False) > class Dummy(sqlobject.SQLObject): > foo = sqlobject.StringCol() > Dummy.createTable(connection=conn) > Dummy(foo="init", connection=conn) > tr = conn.transaction() > conn_dummy=Dummy.get(1,conn) > trans_dummy=Dummy.get(1,tr) > print "Writing 'connection_val' to connection foo" > conn_dummy.foo = "connection_val" > print "Value of transaction foo: '%s'" % trans_dummy.foo AFAIU, what is going on there is: -- the code opens the first connection (without a transaction); -- a new table is created via connection 1; -- a new row is inserted via connection 1; -- the code opens another connection, and BEGINs a transaction ; -- the code draws two copies of the row via connection1 and connection2; -- the code changes the attribute .foo of the FIRST copy; -- the code prints the attribute .foo of the SECOND copy that was drawn via another transaction, and the value is still 'init' because it isn't changed - the code hasn't changed trans_dummy.foo yet. I consider this behaviour correct, there is nothing to fix. Two different objects have two different values. They are two different objects because they have been drawn via two different connections; every connection has its own cache, and they are not synced. If you want to "fix" this in your program - change the order of actions: -- draw the first row, change it and COMMIT the transaction; -- draw the second copy of the row. Or: -- draw the first row; -- draw the second copy of the row. -- change the first copy and COMMIT the transaction; -- clear the cache of second connection and re-draw the second copy of the row. Oleg. -- Oleg Broytmann http://phd.pp.ru/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |
From: Oleg B. <ph...@ph...> - 2009-06-05 22:25:04
|
Hello! I'm leaving the town for a vacation. Will be back at June 20. Please help each other. Oleg. -- Oleg Broytmann http://phd.pp.ru/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |
From: Herwig H. <hho...@gm...> - 2009-06-05 21:22:58
|
Hi! Attached is a small code snippet highlighting some strange behavior. It outputs: Writing 'connection_val' to connection foo Value of transaction foo: 'init' Writing 'transaction_val' to transaction foo Value of connection foo: 'connection_val' Instead of the expected: Writing 'connection_val' to connection foo Value of transaction foo: 'connection_val' Writing 'transaction_val' to transaction foo Value of connection foo: 'connection_val' Is this behavior intentional? (ie can I rely on it) Is is sqlite3 related? (right now I can't test it on another DB) How should I fix this? .expire()? .sync()? Invalidate the transaction's cache? Caching behavior in sqlobject looks like some can worms to me. regards Herwig Hochleitner |
From: Daniel F. <fet...@go...> - 2009-05-28 16:49:40
|
> Hello! Does anybody use SQLObject with mod_python? I have a strange bug > report: > https://sourceforge.net/tracker/?func=detail&aid=2794073&group_id=74338&atid=540672 I don't use mod_python but since it's a dead project I also discourage anyone from using it. A better choice these days is mod_wsgi, the maintainer of mod_python actually stopped maintaining mod_python and rather is working on mod_wsgi. Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown |
From: Oleg B. <ph...@ph...> - 2009-05-23 17:08:07
|
Hello! Does anybody use SQLObject with mod_python? I have a strange bug report: https://sourceforge.net/tracker/?func=detail&aid=2794073&group_id=74338&atid=540672 Oleg. -- Oleg Broytmann http://phd.pp.ru/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |
From: Stef T. <st...@um...> - 2009-05-22 21:04:08
|
On 05/22/2009 02:21 PM, Matthew Wilson wrote: > <snip> > I find myself using SQLBuilder tools *a lot*. Maybe I should read the > code and see if I can help at least write some documentation for others, > because I've had to figure out most of this stuff through trial and > error. > > Matt > Hello Matt, Everyone, I am wondering.. do you notice any speed issue's when doing instantiations of objects (complex or otherwise) ? I have a page that instantiates about 7000 booking objects. When I replace them with mocks (or pypo's) then I see the page render in about 0.3 seconds.. when I use SQLObject it goes upto about 80-90 seconds ... curiously, replacing the SQLObjects with SQLAlchemy objects I got the page rending in 2 seconds. This is why I was thinking it was object instantiation rather than mass inserts that was 'slow' .. perhaps my test doesn't display the slowness I am seeing, although I maybe hard pressed to come up with a test case that adequately replicates the slowdown :( Regards Stef (ps. I am not 'knocking' SO in anyway, nor saying SA is "great".. merely wondering if it's only me being "special" :D |
From: Matthew W. <ma...@tp...> - 2009-05-22 18:22:23
|
On Fri 22 May 2009 05:44:24 AM EDT, Oleg Broytmann wrote: > On Fri, May 22, 2009 at 01:11:51AM -0400, Stef Telford wrote: >> SQLObject development should -really- focus on is speed. > > There are many areas to improve SQLObject - code, tests, documentation, > development process, communication skills of the developers. Let's add > speed to the list. > But well, it is open source, developers scratch their itches, and speed > is not our biggest itch. > SQLObject certainly isn't optimized for mass insertion, this is a well > known fact. SQLObject does a lot of thing behind the scene. For mass > insertion it is better to use SQLBuilder lower-level API (underdocumented, > yes). > You are welcome to start a project to improve SQLObject. Or at least you > can help by explaining how SA optimizes mass insertion. On an INSERT > SQLObject does: > -- creates an instance; > -- sends INSERT query; > -- sends SELECT query to get the row back - this is necessary to get > back autogenerated fields - autoincremented IDs, timestamps, system > variables and such; this is what slows down SQLObject so much; > -- caches the row in its internal cache; this is what makes SQLObject to > eat memory; SQLObject periodically clears the cache thus spending > even more time. > What SA does and does not do? The original comparison isn't fair. There are two differences between the SQLAlchemy (SA) approach and the SQLObject (SO) approach. 1. The SA code does one single insert statement and the SQLObject code does a separate query for each object. 2. The SO code instantiates a class after inserting each row. The SA code doesn't instantiate anything. It is possible to do get SO to act like SA, but like Oleg says, it is underdocumented. Here's how I do a big insert with SO: def bulky_insert(cls, insert_these): """ Does a single insert, cramming everything from insert_these, which must be a list of dictionaries, and each dictionary must be like: d = dict(shift_template_id=sr.shift_templateID, shift_time_id=new_st.id, shift_type_id=sr.shift_typeID, location_id=sr.locationID, build_id=bld.id, required_staff=sr.required_staff) """ cls._connection.query(cls.sqlrepr( Insert(cls.sqlmeta.table, insert_these))) The cls is a subclass of the typical SQLObject class. I build a list of dictionaries, where each dictionary has a key named with the column name and the value is (duhh...) the value for that column name. This does a single big insert and doesn't return anything. No classes are instantiated. This is *really* fast. I find myself using SQLBuilder tools *a lot*. Maybe I should read the code and see if I can help at least write some documentation for others, because I've had to figure out most of this stuff through trial and error. Matt |
From: Oleg B. <ph...@ph...> - 2009-05-22 17:32:35
|
On Fri, May 22, 2009 at 12:52:52PM -0400, Stef Telford wrote: > I am merely wondering where things 'slow > down' so much and if there is anything we can do about it :( Can you run your test program under the python profiler and find out the slow spot(s)? Oleg. -- Oleg Broytmann http://phd.pp.ru/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |
From: Stef T. <st...@um...> - 2009-05-22 16:53:50
|
Hello again Oleg On 05/22/2009 05:44 AM, Oleg Broytmann wrote: > On Fri, May 22, 2009 at 01:11:51AM -0400, Stef Telford wrote: > >> SQLObject development should -really- focus on is speed. >> > There are many areas to improve SQLObject - code, tests, documentation, > development process, communication skills of the developers. Let's add > speed to the list. > But well, it is open source, developers scratch their itches, and speed > is not our biggest itch. > I know that everyone always see's -their- problem as the biggest. This is human nature, although I would argue that people will notice speed before anything else (except documentation perhaps), but I understand your points. A lot could be done, it merely needs the manpower. > SQLObject certainly isn't optimized for mass insertion, this is a well > known fact. SQLObject does a lot of thing behind the scene. For mass > insertion it is better to use SQLBuilder lower-level API (underdocumented, > yes). > actually, I am not entirely sure that the speed problem is due to not being optimised for mass inserts. I notice on most webpages which have a lot of objects (over 1k) that object instantiation is the 'slow' part. I do notice a fair amount of evals of lambda's that are going on in the guts of the system. Those are -never- fast. I am wondering if there is perhaps something else going on during an init that you could think may slow things down ? For what it's worth, I am unsure how a select from a database could be slowing things down when the database is entirely in memory. I would expect that to be the -last- thing to slow the system down. Perhaps that's my guessing but.. yes. Again, I know tone is often hard to convey, but I am not blaming nor attacking anyone. I -do- use SQLObject on a daily basis and thank you and Ian for all the work done. I am merely wondering where things 'slow down' so much and if there is anything we can do about it :( Regards Stef |
From: Oleg B. <ph...@ph...> - 2009-05-22 09:46:47
|
On Fri, May 22, 2009 at 01:11:51AM -0400, Stef Telford wrote: > SQLObject development should -really- focus on is speed. There are many areas to improve SQLObject - code, tests, documentation, development process, communication skills of the developers. Let's add speed to the list. But well, it is open source, developers scratch their itches, and speed is not our biggest itch. SQLObject certainly isn't optimized for mass insertion, this is a well known fact. SQLObject does a lot of thing behind the scene. For mass insertion it is better to use SQLBuilder lower-level API (underdocumented, yes). You are welcome to start a project to improve SQLObject. Or at least you can help by explaining how SA optimizes mass insertion. On an INSERT SQLObject does: -- creates an instance; -- sends INSERT query; -- sends SELECT query to get the row back - this is necessary to get back autogenerated fields - autoincremented IDs, timestamps, system variables and such; this is what slows down SQLObject so much; -- caches the row in its internal cache; this is what makes SQLObject to eat memory; SQLObject periodically clears the cache thus spending even more time. What SA does and does not do? Oleg. -- Oleg Broytmann http://phd.pp.ru/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |
From: Sean D. <hal...@gm...> - 2009-05-22 05:58:32
|
Why are you subscribed to the list then?? What is it about SQLObject that makes you want the developers to speed it up? I'm a nobody that uses SQLObject for production work. What are you looking for? See below to learn something. Pronunciation: \ˈtakt\ Function:* noun*Etymology: French, sense of touch, from Latin *tactus,* from *tangere* to touch — more at tangent<http://www.merriam-webster.com/dictionary/tangent>Date: 17972 *:* a keen sense of what to do or say in order to maintain good relations with others or avoid offense On Thu, May 21, 2009 at 10:11 PM, Stef Telford <st...@um...> wrote: > Hey Oleg, Everyone, > Please don't take this as 'rude' but, I think that the thing SQLObject > development should -really- focus on is speed. Perhaps I am doing something > fundamentally wrong with the following code but, for the life of me, I can't > figure out why. I know your going to hate me saying this but.. compare the > SQLObject vs SQLAlchemy > > > Starting benchmarking with 100000 records (inserting and selecting) > . > Inserting 100000 records into SQLite(memory) with SQLAlchemy > Number of records selected: 100000 > > Time for SQLAlchemy with SQlite db in Memory: 2.71 seconds > > Inserting 100000 records into SQLite(Disk) with SQLAlchemy > Number of records selected: 100000 > Time for SQLAlchemy with SQlite db on Disk: 2.64 seconds > > > Versus > > > Starting benchmarking with 100000 records (inserting and selecting) > . > Inserting 100000 records into SQLite(Memory) with SQLObject > Number of records selected: 100000 > Time for SQLObject with db in memory: 38.53 seconds > > and when I change it to put the sqlite db onto the -exact- same disk as > SQLAlchemy used > > Time for SQLObject with db on disk: 79.74 seconds > > > > So, that's roughly 20 times slower for memory, and 40 times slower for > disk. > > As I said, if I am doing something wrong in the example programs, please do > feel free to correct me, but, otherwise, the speed .. well .. 'sucks'. You > may ask how 'likely' is 100k inserts, but the figures are even -worse- for > object instantiation from a database .. I mean a LOT worse :( now, perhaps > SQLObject does things like provide sqlmeta or .. some such.. but.. 20 > -times- slower is the trade off ? seems like a bad idea to me :\ > > Ideas ? Thoughts ? Am I crazy ? > > Regards > Stef > > > ------------------------------------------------------------------------------ > Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT > is a gathering of tech-side developers & brand creativity professionals. > Meet > the minds behind Google Creative Lab, Visual Complexity, Processing, & > iPhoneDevCamp asthey present alongside digital heavyweights like Barbarian > Group, R/GA, & Big Spaceship. http://www.creativitycat.com > _______________________________________________ > sqlobject-discuss mailing list > sql...@li... > https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss > > |
From: Stef T. <st...@um...> - 2009-05-22 05:39:29
|
Hey Oleg, Everyone, Please don't take this as 'rude' but, I think that the thing SQLObject development should -really- focus on is speed. Perhaps I am doing something fundamentally wrong with the following code but, for the life of me, I can't figure out why. I know your going to hate me saying this but.. compare the SQLObject vs SQLAlchemy Starting benchmarking with 100000 records (inserting and selecting) . Inserting 100000 records into SQLite(memory) with SQLAlchemy Number of records selected: 100000 Time for SQLAlchemy with SQlite db in Memory: 2.71 seconds Inserting 100000 records into SQLite(Disk) with SQLAlchemy Number of records selected: 100000 Time for SQLAlchemy with SQlite db on Disk: 2.64 seconds Versus Starting benchmarking with 100000 records (inserting and selecting) . Inserting 100000 records into SQLite(Memory) with SQLObject Number of records selected: 100000 Time for SQLObject with db in memory: 38.53 seconds and when I change it to put the sqlite db onto the -exact- same disk as SQLAlchemy used Time for SQLObject with db on disk: 79.74 seconds So, that's roughly 20 times slower for memory, and 40 times slower for disk. As I said, if I am doing something wrong in the example programs, please do feel free to correct me, but, otherwise, the speed .. well .. 'sucks'. You may ask how 'likely' is 100k inserts, but the figures are even -worse- for object instantiation from a database .. I mean a LOT worse :( now, perhaps SQLObject does things like provide sqlmeta or .. some such.. but.. 20 -times- slower is the trade off ? seems like a bad idea to me :\ Ideas ? Thoughts ? Am I crazy ? Regards Stef |
From: Daniel F. <fet...@go...> - 2009-05-18 21:31:02
|
>> Actually, I think TimedeltaCol could be implemented for sqlite as >> well. Although it doesn't have an INTERVAL data type it could use a >> plain TEXT instead and python would do the conversion. As far as I >> know sqlite only knows TEXT anyway so IntCol, StringCol, etc, all >> these things are implemented as TEXT and the data conversion is done >> by python. The TimedeltaCol could be similar. > > Well, not exactly - SQLite understands and produces some data types - > pysqlite passes them unmodified AFAIU. But anyway I did a similar trick > with DecimalStringCol - it stores decimals as strings to work around > SQLite's "type affinity" feature. I might be wrong, but from http://www.sqlite.org/datatype3.html I concluded that sqlite only knows about * TEXT * NUMERIC * INTEGER * REAL * NONE so things like date, datetime and anything else in sqlobject that is not one of the above 5 types will have to be stored as TEXT. So this trick of storing complicated python data types as TEXT is already happening in sqlobject (for example the one you mentioned, DecimalStringCol). The same thing can be done for the OP's proposed TimedeltaCol too. Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown |
From: Oleg B. <ph...@ph...> - 2009-05-18 18:14:13
|
Hello! I'm pleased to announce version 0.10.6, a minor bugfix release of 0.10 branch of SQLObject. What is SQLObject ================= SQLObject is an object-relational mapper. Your database tables are described as classes, and rows are instances of those classes. SQLObject is meant to be easy to use and quick to get started with. SQLObject supports a number of backends: MySQL, PostgreSQL, SQLite, Firebird, Sybase, MSSQL and MaxDB (also known as SAPDB). Where is SQLObject ================== Site: http://sqlobject.org Development: http://sqlobject.org/devel/ Mailing list: https://lists.sourceforge.net/mailman/listinfo/sqlobject-discuss Archives: http://news.gmane.org/gmane.comp.python.sqlobject Download: http://cheeseshop.python.org/pypi/SQLObject/0.10.6 News and changes: http://sqlobject.org/News.html What's New ========== News since 0.10.5 ----------------- * Better support for Python 2.6: do not import the deprecated sets module. * Two bugs in SQLiteConnection.columnsFromSchema() were fixed: use sqlmeta.idName instead of 'id'; convert default 'NULL' to None. * Use sqlmeta.idName instead of 'id' in all connection classes. * Fixed a bug that prevented to override per class _connection if there is sqlhub.processConnection. For a more complete list, please see the news: http://sqlobject.org/News.html Oleg. -- Oleg Broytmann http://phd.pp.ru/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |
From: Oleg B. <ph...@ph...> - 2009-05-18 18:11:53
|
Hello! I'm pleased to announce version 0.9.11, a minor bugfix release of 0.9 branch of SQLObject. What is SQLObject ================= SQLObject is an object-relational mapper. Your database tables are described as classes, and rows are instances of those classes. SQLObject is meant to be easy to use and quick to get started with. SQLObject supports a number of backends: MySQL, PostgreSQL, SQLite, Firebird, Sybase, MSSQL and MaxDB (also known as SAPDB). Where is SQLObject ================== Site: http://sqlobject.org Development: http://sqlobject.org/devel/ Mailing list: https://lists.sourceforge.net/mailman/listinfo/sqlobject-discuss Archives: http://news.gmane.org/gmane.comp.python.sqlobject Download: http://cheeseshop.python.org/pypi/SQLObject/0.9.11 News and changes: http://sqlobject.org/News.html What's New ========== News since 0.9.10 ---------------- * Two bugs in SQLiteConnection.columnsFromSchema() were fixed: use sqlmeta.idName instead of 'id'; convert default 'NULL' to None. * Use sqlmeta.idName instead of 'id' in all connection classes. * Fixed a bug that prevented to override per class _connection if there is sqlhub.processConnection. For a more complete list, please see the news: http://sqlobject.org/News.html Oleg. -- Oleg Broytmann http://phd.pp.ru/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |
From: Oleg B. <ph...@ph...> - 2009-05-17 01:22:49
|
On Sat, May 16, 2009 at 03:52:28PM -0400, inhahe wrote: > Hi, i have a question about limit/offset. i read something that says if > you're doing something like pages of results with next/previous buttons, you > should do the query once and then cache the results for a certain period of > time, but then my friend says you can do limit(x,y) where, i guess, one > variable is the starting point and the other is the number of records. then > i found out about the "offset" keyword on the web, so i guess the proper way > is to use limit and offset. either way, this seems more efficient than > caching the whole query--is it? and how do i do this in sqlobject? or > should i just store a query and use slicing on it? or just do the same > query each time and use slicing on it since sqlobject caches certain things > anyway? thanks. In SQLObject, MyTable.select() doesn't return a list of rows - the call returns an instance of SelectResult class which maps slicing to limit/offset. This is how you should do it: for row in MyTable.select(query)[offset:offset+limit]: process(row) Oleg. -- Oleg Broytmann http://phd.pp.ru/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |
From: inhahe <in...@gm...> - 2009-05-16 19:52:39
|
Hi, i have a question about limit/offset. i read something that says if you're doing something like pages of results with next/previous buttons, you should do the query once and then cache the results for a certain period of time, but then my friend says you can do limit(x,y) where, i guess, one variable is the starting point and the other is the number of records. then i found out about the "offset" keyword on the web, so i guess the proper way is to use limit and offset. either way, this seems more efficient than caching the whole query--is it? and how do i do this in sqlobject? or should i just store a query and use slicing on it? or just do the same query each time and use slicing on it since sqlobject caches certain things anyway? thanks. |