Thread: [SQLObject] Decimal type support
SQLObject is a Python ORM.
Brought to you by:
ianbicking,
phd
Menu
▾
▴
sqlobject-discuss
|
From: Carlos R. <car...@gm...> - 2004-12-06 15:59:31
|
Hello all, I just downloaded & installed the decimal module (the same from Python 2.4) for Python 2.3. I know that SQLObject has a DecimalCol type. I've tried to check the code that actually fetches decimal values from the DB, but I got a little bit lost on SQLObject internals at this point. I trie to grep for Decimal and found only a few references, so I assume it's not full implemented yet. What I want to do is to make sure that the get & set methods for the decimal columns returns the appropriate Decimal type, if it's installed on the system, instead of a float. I assume that this is to be done somewhere with the fromPython/toPython validators, but I'm still trying to figure out where to put my code into. Any help (and experiences with Decimal) will be great. -- Carlos Ribeiro Consultoria em Projetos blog: http://rascunhosrotos.blogspot.com blog: http://pythonnotes.blogspot.com mail: car...@gm... mail: car...@ya... |
|
From: Oleg B. <ph...@ma...> - 2004-12-06 16:09:51
|
On Mon, Dec 06, 2004 at 01:59:27PM -0200, Carlos Ribeiro wrote: > somewhere with the fromPython/toPython validators, but I'm > still trying to figure out where to put my code into This how I do it for Unicode columns: http://sourceforge.net/mailarchive/message.php?msg_id=10177526 Decimal should not be any harder. Oleg. -- Oleg Broytmann http://phd.pp.ru/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |
|
From: Carlos R. <car...@gm...> - 2004-12-06 16:25:32
|
On Mon, 6 Dec 2004 19:09:46 +0300, Oleg Broytmann <ph...@ma...> wrote: > On Mon, Dec 06, 2004 at 01:59:27PM -0200, Carlos Ribeiro wrote: > > somewhere with the fromPython/toPython validators, but I'm > > still trying to figure out where to put my code into > > This how I do it for Unicode columns: > http://sourceforge.net/mailarchive/message.php?msg_id=10177526 > Decimal should not be any harder. > > Oleg. Thanks for the pointer. I've read that message when first posted, and that was exactly what I was trying to do for Decimal. After some code reading, I think I began to understand how to make it work. I'll try it a little bit here, and if I make it work (with all tests, etc) I'll post a patch. But don't hold your breath, please :-) -- Carlos Ribeiro Consultoria em Projetos blog: http://rascunhosrotos.blogspot.com blog: http://pythonnotes.blogspot.com mail: car...@gm... mail: car...@ya... |
|
From: Ian B. <ia...@co...> - 2004-12-06 16:19:18
|
Carlos Ribeiro wrote: > I just downloaded & installed the decimal module (the same from Python > 2.4) for Python 2.3. I know that SQLObject has a DecimalCol type. I've > tried to check the code that actually fetches decimal values from the > DB, but I got a little bit lost on SQLObject internals at this point. > I trie to grep for Decimal and found only a few references, so I > assume it's not full implemented yet. One way would be to handle the conversion through the driver. This should be doable with psycopg. Some way you have to get the value from the database, probably as a string, then convert it to decimal. Obviously it's not very helpful if you get it from the database as a float, then convert it to a decimal. Right now there's no good hooks for coercing something before it is returned. E.g., in Postgres I think that'd be something like "SELECT (CAST dec_column AS TEXT) AS dec_column", which would keep psycopg from converting to a float. -- Ian Bicking / ia...@co... / http://blog.ianbicking.org |
|
From: Carlos R. <car...@gm...> - 2004-12-06 17:00:06
|
On Mon, 06 Dec 2004 10:15:34 -0600, Ian Bicking <ia...@co...> wrote: > Carlos Ribeiro wrote: > > I just downloaded & installed the decimal module (the same from Python > > 2.4) for Python 2.3. I know that SQLObject has a DecimalCol type. I've > > tried to check the code that actually fetches decimal values from the > > DB, but I got a little bit lost on SQLObject internals at this point. > > I trie to grep for Decimal and found only a few references, so I > > assume it's not full implemented yet. > > One way would be to handle the conversion through the driver. This > should be doable with psycopg. Some way you have to get the value from > the database, probably as a string, then convert it to decimal. > Obviously it's not very helpful if you get it from the database as a > float, then convert it to a decimal. > > Right now there's no good hooks for coercing something before it is > returned. E.g., in Postgres I think that'd be something like "SELECT > (CAST dec_column AS TEXT) AS dec_column", which would keep psycopg from > converting to a float. Humm. Decimals should always be passed around either as native decimals or as strings; the Decimal constructor takes a string as a parameter, and that's the only way to preserve precision. There are a few ways out of this problem: 1) Rely on the DB driver working with native decimals (perhaps with some patches, as you suggested, to trick the DB to use a string as an intermediate value). That's a definitive solution but I'm not sure even how to start it. 2) Forget decimals on the DB -- store everything as strings, and convert to native Decimals on reading & writing. This works, but in this case I can't rely on the DB itself for any arithmetic manipulation. That will cause problems with aggregate SQL clauses (SUM, MAX, etc.). I really don't know what to do. Implementing Decimal support in SQLObject would be nice, but I fear that it will take a lot of time and require a lot of knowledge about individual database drivers. What a problem... -- Carlos Ribeiro Consultoria em Projetos blog: http://rascunhosrotos.blogspot.com blog: http://pythonnotes.blogspot.com mail: car...@gm... mail: car...@ya... |
|
From: Oleg B. <ph...@ma...> - 2004-12-06 17:09:57
|
On Mon, Dec 06, 2004 at 03:00:02PM -0200, Carlos Ribeiro wrote:
> I really don't know what to do.
Wait until DB API 3 implements Decimal in drivers. To speed things up
start discussing DB API 3 in the db-sig, and send patches for your
driver.
Yes, that's hard...
Oleg.
--
Oleg Broytmann http://phd.pp.ru/ ph...@ph...
Programmers don't die, they just GOSUB without RETURN.
|
|
From: Carlos R. <car...@gm...> - 2004-12-06 17:23:33
|
On Mon, 6 Dec 2004 20:09:53 +0300, Oleg Broytmann <ph...@ma...> wrote: > On Mon, Dec 06, 2004 at 03:00:02PM -0200, Carlos Ribeiro wrote: > > I really don't know what to do. > > Wait until DB API 3 implements Decimal in drivers. To speed things up > start discussing DB API 3 in the db-sig, and send patches for your > driver. > Yes, that's hard... I remember posting a comment about the state of the DB API on c.l.py a couple of months ago (soon after the famous AMK blog post on the std library), and being quickly flamed by some well know DB-SIG members. So that's *really* hard. BTW, one of the reasons I've chosen to use SQLObject is that it allowed me to abstract out the database implementation details much more effectively than the DB API itself does. At the time I'd thought that I would rather wash my hands in boiling water than start another discussion on the DB-SIG. Oh dear, own naive :-) -- Carlos Ribeiro Consultoria em Projetos blog: http://rascunhosrotos.blogspot.com blog: http://pythonnotes.blogspot.com mail: car...@gm... mail: car...@ya... |
|
From: Oleg B. <ph...@ma...> - 2004-12-06 17:31:05
|
On Mon, Dec 06, 2004 at 03:23:26PM -0200, Carlos Ribeiro wrote:
> I remember posting a comment about the state of the DB API on c.l.py a
> couple of months ago (soon after the famous AMK blog post on the std
> library), and being quickly flamed by some well know DB-SIG members.
> So that's *really* hard.
<Wry grin>
> BTW, one of the reasons I've chosen to use SQLObject is that it
> allowed me to abstract out the database implementation details much
> more effectively than the DB API itself does.
But now you want to extend something. You can do it in your program,
or try to extend SQLObject, or do it in DB API level...
Oleg.
--
Oleg Broytmann http://phd.pp.ru/ ph...@ph...
Programmers don't die, they just GOSUB without RETURN.
|
|
From: Ian B. <ia...@co...> - 2004-12-06 17:10:15
|
Carlos Ribeiro wrote: > On Mon, 06 Dec 2004 10:15:34 -0600, Ian Bicking <ia...@co...> wrote: > >>Carlos Ribeiro wrote: >> >>>I just downloaded & installed the decimal module (the same from Python >>>2.4) for Python 2.3. I know that SQLObject has a DecimalCol type. I've >>>tried to check the code that actually fetches decimal values from the >>>DB, but I got a little bit lost on SQLObject internals at this point. >>>I trie to grep for Decimal and found only a few references, so I >>>assume it's not full implemented yet. >> >>One way would be to handle the conversion through the driver. This >>should be doable with psycopg. Some way you have to get the value from >>the database, probably as a string, then convert it to decimal. >>Obviously it's not very helpful if you get it from the database as a >>float, then convert it to a decimal. >> >>Right now there's no good hooks for coercing something before it is >>returned. E.g., in Postgres I think that'd be something like "SELECT >>(CAST dec_column AS TEXT) AS dec_column", which would keep psycopg from >>converting to a float. > > > Humm. Decimals should always be passed around either as native > decimals or as strings; the Decimal constructor takes a string as a > parameter, and that's the only way to preserve precision. There are a > few ways out of this problem: > > 1) Rely on the DB driver working with native decimals (perhaps with > some patches, as you suggested, to trick the DB to use a string as an > intermediate value). That's a definitive solution but I'm not sure > even how to start it. It depends what backends you are targetting. If Postgres/psycopg, you should be able to modify its type coercion fairly easily. For other database drivers, I'm not sure. There's no standard DB API way to manage type conversion, but I imagine there are interested people in each case, and workaround solutions. This seems like the easiest course, and the end result (getting drivers to return decimal types natively) will benefit everyone. -- Ian Bicking / ia...@co... / http://blog.ianbicking.org |
|
From: Carlos R. <car...@gm...> - 2004-12-07 03:35:32
|
On Mon, 06 Dec 2004 11:06:33 -0600, Ian Bicking <ia...@co...> wrote: > It depends what backends you are targetting. If Postgres/psycopg, you > should be able to modify its type coercion fairly easily. For other > database drivers, I'm not sure. There's no standard DB API way to > manage type conversion, but I imagine there are interested people in > each case, and workaround solutions. This seems like the easiest > course, and the end result (getting drivers to return decimal types > natively) will benefit everyone. If I only could find a way to do it in SQLObject without having to resort to database driver patching, that would be perfect :-) But unfortunately, things are not that easy, unless I store all my money values in text columns - but then I'll lose big on SQL aggregation functions & stuff like that. But I'm really tempted to do it, just to avoid messing with db drivers for now. -- Carlos Ribeiro Consultoria em Projetos blog: http://rascunhosrotos.blogspot.com blog: http://pythonnotes.blogspot.com mail: car...@gm... mail: car...@ya... |
|
From: Ian B. <ia...@co...> - 2004-12-06 18:08:18
|
Carlos Ribeiro wrote: > On Mon, 6 Dec 2004 20:09:53 +0300, Oleg Broytmann <ph...@ma...> wrote: > >>On Mon, Dec 06, 2004 at 03:00:02PM -0200, Carlos Ribeiro wrote: >> >>>I really don't know what to do. >> >> Wait until DB API 3 implements Decimal in drivers. To speed things up >>start discussing DB API 3 in the db-sig, and send patches for your >>driver. >> Yes, that's hard... > > > I remember posting a comment about the state of the DB API on c.l.py a > couple of months ago (soon after the famous AMK blog post on the std > library), and being quickly flamed by some well know DB-SIG members. > So that's *really* hard. It won't be hard for the decimal type -- the decimal type is an issue of correctness. It might be hard to make it actually *happen* given the development process for any particular driver, but everyone knows returning a Python decimal for a DB decimal column is the Right Thing. All you have to ask is, "how do I do it?" and if there's not a way, "how can we change the driver to allow me to do it?" -- Ian Bicking / ia...@co... / http://blog.ianbicking.org |
|
From: Carlos R. <car...@gm...> - 2004-12-06 18:17:10
|
On Mon, 06 Dec 2004 12:04:21 -0600, Ian Bicking <ia...@co...> wrote: > Carlos Ribeiro wrote: > > I remember posting a comment about the state of the DB API on c.l.py a > > couple of months ago (soon after the famous AMK blog post on the std > > library), and being quickly flamed by some well know DB-SIG members. > > So that's *really* hard. > > It won't be hard for the decimal type -- the decimal type is an issue of > correctness. It might be hard to make it actually *happen* given the > development process for any particular driver, but everyone knows > returning a Python decimal for a DB decimal column is the Right Thing. > All you have to ask is, "how do I do it?" and if there's not a way, "how > can we change the driver to allow me to do it?" Well, you're probably right. But that's realy not for a short timeframe though. I think I'll be using floats for now -- they did just fine up to this point, if used with proper care at the right locations, and I should not be in a hurry to replace it. I may be able to come up with a Decimals patch in the next few weeks (surely it's not going to be my top priority). I'll follow yours (Ian and Oleg) feedback, and subscribe to the DB-SIG, to check how do they feel about it. I'm lucky enough (I guess) to have three platforms installed & working here: PostgreSQL, MySQL and sqlite, so I think I can test with these. Let's see what I can bake here... Thanks for the info and for the encouragement :-) -- Carlos Ribeiro Consultoria em Projetos blog: http://rascunhosrotos.blogspot.com blog: http://pythonnotes.blogspot.com mail: car...@gm... mail: car...@ya... |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X