SQLObject

On Sat,  8 Jan 2005 15:38:42 +0100 (CET), Philippe Normand
<phil@...> wrote:
> 
> Hi
> 
> I've got a little issue with sqlo svn trunk. There's a module
> AlineaArticle storing one SQLObject declaration and i have to import it
> from multiple places. When i do so, sqlo raises a ValueError:
> 
> ValueError: class AlineaArticle is already in the registry (other class
> is <class 'Alinea.Articles.AlineaArticle.AlineaArticle'>, from the
> module Alinea.Articles.AlineaArticle in
> /home/phil/devel/alinea/Alinea/Articles/AlineaArticle.py; attempted new
> class is <class 'Articles.AlineaArticle.AlineaArticle'>, from the
> module Articles.AlineaArticle in
> /home/phil/devel/alinea/Alinea/Articles/AlineaArticle.pyc)
> 
> Seems like the .py and the .pyc try to register the SQLObject in their
> corner. That's weird, has anyboday ever had the same problem ?

I had the same issue. I don't think that this is a problem with
SQLObject itself; it depends a lot on the way you write your code.
Loading the same module twice with a simple "import" should not be a
problem, but anything that runs the code more than once will make it
fail. I believe that this can be triggered in some situations:

- by a reload() call, which makes the module code to be re-executed.

- by importing the same module from different locations, in such a way
that the internal module cache used by Python isn't able to tell that
the module was actually imported before. In this case the module code
is effectively run twice, which may also trigger the problem.

I solved the issue in my own code by putting all my SQLObject
declarations inside a class that is guaranteed to be instantiated only
once, as a singleton. But this solution will not work for everyone; it
was actually more of a design choice in my own case, dicted not only
by SQLObject, but also because of other architectural concerns. You
could try something along these lines, too, if it suites your needs.

-- 
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: carribeiro@...
mail: carribeiro@...

> ValueError: class AlineaArticle is already in the registry (other class
> is <class 'Alinea.Articles.AlineaArticle.AlineaArticle'>, from the
> module Alinea.Articles.AlineaArticle in
> /home/phil/devel/alinea/Alinea/Articles/AlineaArticle.py; attempted new
> class is <class 'Articles.AlineaArticle.AlineaArticle'>, from the
> module Articles.AlineaArticle in
> /home/phil/devel/alinea/Alinea/Articles/AlineaArticle.pyc)

I had a similar problem, it happened when I was importing module from
a python file in a symbolic link directory and did not provide
absolute import path:

from mymodule import MyClass

The solution was to use absolute import path:

from mypackage.mymodule import MyClass

HTH
-- 
Ksenia

Le 8/1/2005, "Carlos Ribeiro" <carribeiro@...> a =E9crit:

>
>I had the same issue. I don't think that this is a problem with
>SQLObject itself; it depends a lot on the way you write your code.
>Loading the same module twice with a simple "import" should not be a
>problem, but anything that runs the code more than once will make it
>fail. I believe that this can be triggered in some situations:
>
>- by a reload() call, which makes the module code to be re-executed.
>
>- by importing the same module from different locations, in such a way
>that the internal module cache used by Python isn't able to tell that
>the module was actually imported before. In this case the module code
>is effectively run twice, which may also trigger the problem.
>

But it's exactly what i am doing. The problem is that the
classregistry's addClass hook is too restrictive. If the class is
already in the registry it should not raise an error (in my opinion).
Instead it should simply exit from the hook.

Can other sqlo dev explain why addClass raises a ValueError such a way ?

Philippe

after 4 years of python hacking, i'm still shocked at how fast it is
to work with it.  in a few minutes i had my existing connection going,
with a few lines of code (there is a little hack for resetting the
registry):


from sqlobject.postgres.pgconnection import PostgresConnection

#=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D
# LOCAL DECLARATIONS
#=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D

# Reset the class registry to allow reloading.
classregistry.MasterRegistry =3D classregistry._MasterRegistry()
classregistry.registry =3D classregistry.MasterRegistry.registry

#--------------------------------------------------------------------------=
-----
#
class ExistingPgConnection(PostgresConnection):
    """
    Provide a connection object for SQLObject that reuses the existing
    connection.
    """

    def __init__(self, connection):
        """
        Initialize with an existing, opened connection from psycopg.
        """
        PostgresConnection.__init__(self, autoCommit=3DFalse)
        self.existing_connection =3D connection

    def makeConnection(self):
        # Note: support for autocommit member has been removed in psycopg2.
        return self.existing_connection



then in my code i do:

from sqlobject import *

__connection__ =3D ExistingPgConnection(database.m_connection)

class Person(SQLObject):
...


voila!

1. now, to be fair, it only "seems" to work, but i would really need
to dig into that class registry issue to find out if the hack is
enough of a reset.  i need to reset the registry because my framework
has an auto reload feature which make it really fast to develop and
i'm not giving that up...

2. also, i'm not sure that returning the same single connection from
makeConnection() is kosher with the connection pool code.  i'm not
very familiar with sql databases... why do we need connection pools?

any issue that might help make this clean would be appreciated
(anything off thetop of your head)... thanks

cheers,



On Sat,  8 Jan 2005 22:33:42 +0100 (CET), Philippe Normand
<phil@...> wrote:
>=20
> Le 8/1/2005, "Carlos Ribeiro" <carribeiro@...> a =E9crit:
>=20
> >
> >I had the same issue. I don't think that this is a problem with
> >SQLObject itself; it depends a lot on the way you write your code.
> >Loading the same module twice with a simple "import" should not be a
> >problem, but anything that runs the code more than once will make it
> >fail. I believe that this can be triggered in some situations:
> >
> >- by a reload() call, which makes the module code to be re-executed.
> >
> >- by importing the same module from different locations, in such a way
> >that the internal module cache used by Python isn't able to tell that
> >the module was actually imported before. In this case the module code
> >is effectively run twice, which may also trigger the problem.
> >
>=20
> But it's exactly what i am doing. The problem is that the
> classregistry's addClass hook is too restrictive. If the class is
> already in the registry it should not raise an error (in my opinion).
> Instead it should simply exit from the hook.
>=20
> Can other sqlo dev explain why addClass raises a ValueError such a way ?
>=20
> Philippe
>=20
> -------------------------------------------------------
> The SF.Net email is sponsored by: Beat the post-holiday blues
> Get a FREE limited edition SourceForge.net t-shirt from ThinkGeek.
> It's fun and FREE -- well, almost....http://www.thinkgeek.com/sfshirt
> _______________________________________________
> sqlobject-discuss mailing list
> sqlobject-discuss@...
> https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss
>

On Sat,  8 Jan 2005 22:33:42 +0100 (CET), Philippe Normand
<phil@...> wrote:
>=20
> Le 8/1/2005, "Carlos Ribeiro" <carribeiro@...> a =E9crit:
>=20
> >
> >I had the same issue. I don't think that this is a problem with
> >SQLObject itself; it depends a lot on the way you write your code.
> >Loading the same module twice with a simple "import" should not be a
> >problem, but anything that runs the code more than once will make it
> >fail. I believe that this can be triggered in some situations:
> >
> >- by a reload() call, which makes the module code to be re-executed.
> >
> >- by importing the same module from different locations, in such a way
> >that the internal module cache used by Python isn't able to tell that
> >the module was actually imported before. In this case the module code
> >is effectively run twice, which may also trigger the problem.
> >
>=20
> But it's exactly what i am doing. The problem is that the
> classregistry's addClass hook is too restrictive. If the class is
> already in the registry it should not raise an error (in my opinion).
> Instead it should simply exit from the hook.

If it simply exits from the hook, you end up with a dangerous
situation. There will be two classes with the same name. For all
purposes, they are not equivalent. If you add something at runtime to
one of them, the other one will not be updated accordingly. Any code
that depends on common class attributes to work will break. There is a
possible solution for this problem but it's not recommended anyway
(see below).
=20
> Can other sqlo dev explain why addClass raises a ValueError such a way ?

To avoid having two *different* declarations of the same class to be
inserted into the registry?

Note that the registry is necessary first place to avoid problems with
forward references to still "undeclared" tables - it may happen with
any of the columns that create relationships (ForeignKey, for
instance). So it allows classes to be referenced by name; it in turn
allows circular references to be created.

One possible solution that could be attempted is to use the metaclass
to make this check.  It could create the class, and check if it is
*equivalent* to the previously declared one. If it is found to be
equivalent (it contains the same columns with the same attributes for
each one of them), the metaclass could return the *old* reference
instead of the newly created one. It's not easy, I must say. Too much
work for little gain, so it's better to keep it as it is.

--=20
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: carribeiro@...
mail: carribeiro@...

>If it simply exits from the hook, you end up with a dangerous
>situation. There will be two classes with the same name. For all
>purposes, they are not equivalent. If you add something at runtime to
>one of them, the other one will not be updated accordingly. Any code
>that depends on common class attributes to work will break. There is a
>possible solution for this problem but it's not recommended anyway
>(see below).
>=20

Oops my mistake ;) addClass is not a hook at all. It's called when you
want to register a class. At the end it effectively calls some
pre-defined hooks. So if we get back to "import from multiple places"
issue, i propose following:

if class in registry:
   return

addIt to the registry
call some callbacks


instead of:

if class in registry:
   raise ValueError()


Isn't it more friendly ?
Philippe

On Sun,  9 Jan 2005 11:52:26 +0100 (CET), Philippe Normand
<phil@...> wrote:
> 
> >If it simply exits from the hook, you end up with a dangerous
> >situation. There will be two classes with the same name. For all
> >purposes, they are not equivalent. If you add something at runtime to
> >one of them, the other one will not be updated accordingly. Any code
> >that depends on common class attributes to work will break. There is a
> >possible solution for this problem but it's not recommended anyway
> >(see below).
> >
> 
> Oops my mistake ;) addClass is not a hook at all. It's called when you
> want to register a class. At the end it effectively calls some
> pre-defined hooks. So if we get back to "import from multiple places"
> issue, i propose following:
> 
> if class in registry:
>    return
> 
> addIt to the registry
> call some callbacks
> 
> instead of:
> 
> if class in registry:
>    raise ValueError()
> 
> Isn't it more friendly ?
> Philippe

The catch is the "if class in registry" statement. How does it know
that the class in the registry? By the name alone? And what would
happen if by mistake your program has two *different* declarations
with the same name but in different modules? I still prefer the
ValueError... whenever it got me, it was because I did run the module
twice inadvertently (in my case, due to unit test code).

-- 
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: carribeiro@...
mail: carribeiro@...

>The catch is the "if class in registry" statement. How does it know
>that the class in the registry? By the name alone? And what would
>happen if by mistake your program has two *different* declarations
>with the same name but in different modules? I still prefer the
>ValueError... whenever it got me, it was because I did run the module
>twice inadvertently (in my case, due to unit test code).
>

What about "if cls in self.allClasses()" ? Thus we don't rely on class
name vs module name, but directly on class objects. I'm not sure it
would work on all cases though but i believe it's a good compromise.

Philippe

Le 9/1/2005, "Philippe Normand" <phil@...> a =E9crit:

>
>>The catch is the "if class in registry" statement. How does it know
>>that the class in the registry? By the name alone? And what would
>>happen if by mistake your program has two *different* declarations
>>with the same name but in different modules? I still prefer the
>>ValueError... whenever it got me, it was because I did run the module
>>twice inadvertently (in my case, due to unit test code).
>>
>
>What about "if cls in self.allClasses()" ? Thus we don't rely on class
>name vs module name, but directly on class objects. I'm not sure it
>would work on all cases though but i believe it's a good compromise.
>

That's ugly. I finally found one good fix, which doesn't break
ClassRegistryTest. I can commit, but prefer submit the patch here before.

Philippe

On Sun,  9 Jan 2005 15:19:26 +0100 (CET), Philippe Normand
<phil@...> wrote:
> 
> >The catch is the "if class in registry" statement. How does it know
> >that the class in the registry? By the name alone? And what would
> >happen if by mistake your program has two *different* declarations
> >with the same name but in different modules? I still prefer the
> >ValueError... whenever it got me, it was because I did run the module
> >twice inadvertently (in my case, due to unit test code).
> >
> 
> What about "if cls in self.allClasses()" ? Thus we don't rely on class
> name vs module name, but directly on class objects. I'm not sure it
> would work on all cases though but i believe it's a good compromise.

Sorry. Either I misunderstand you idea, or you do misunderstand how do
class statements work in Python. At some risk I'll assume the second,
and try to clarify some things. A class is created when the statement
is executed. For example, it is possible (but not really that useful)
to do something like this:

x = []
for i in range(10):
    class C:
        def someMethod(self):
            pass
    x.append(C)

Run this code in interactive mode, and check the results:

>>> print x[0].__name__
C
>>> print x[1].__name__
C
>>> x[0] is x[1]
False
>>> x[0] == x[1]
False

You'll see that both classes give back the same result, and have the
same name stored in but are *different* - although you can argue that
the classes are in fact equivalent, as far as their internal structure
is concerned. It's very difficult to test for it though.

In short, that's the reason why it isn't possible to test for
inclusion as you proposed. The newly declared class is *different*
from the original one, and will not be found in the registry - even if
it is the result from the same declaration... run twice! The search by
name, however, will be able to find another previously defined class
with the same name, but there's no guarantee that both are equivalent.
That's why an error message is the best that can be done.


-- 
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: carribeiro@...
mail: carribeiro@...

Philippe Normand wrote:
> Hi
> 
> I've got a little issue with sqlo svn trunk. There's a module
> AlineaArticle storing one SQLObject declaration and i have to import it
> from multiple places. When i do so, sqlo raises a ValueError:
> 
> ValueError: class AlineaArticle is already in the registry (other class
> is <class 'Alinea.Articles.AlineaArticle.AlineaArticle'>, from the
> module Alinea.Articles.AlineaArticle in
> /home/phil/devel/alinea/Alinea/Articles/AlineaArticle.py; attempted new
> class is <class 'Articles.AlineaArticle.AlineaArticle'>, from the
> module Articles.AlineaArticle in
> /home/phil/devel/alinea/Alinea/Articles/AlineaArticle.pyc)
> 
> Seems like the .py and the .pyc try to register the SQLObject in their
> corner. That's weird, has anyboday ever had the same problem ?

It sounds like you have an abiguous sys.path.  This can lead to 
horrible, horrible bugs!  Bugs where everything looks right, but the 
data is just wrong.  SQLObject in this case is saving you from yourself.

This is because you have both /home/phil/devel/alinea and 
/home/phil/devel/alinea/Alinea in your sys.path, and you sometimes 
import from one path and sometimes from the other.  You get two distinct 
modules with identical code.  It's very bad.

-- 
Ian Bicking  /  ianb@...  /  http://blog.ianbicking.org

Le 10/1/2005, "Ian Bicking" <ianb@...> a =E9crit:

>
>It sounds like you have an abiguous sys.path.  This can lead to
>horrible, horrible bugs!  Bugs where everything looks right, but the
>data is just wrong.  SQLObject in this case is saving you from yourself.
>
>This is because you have both /home/phil/devel/alinea and
>/home/phil/devel/alinea/Alinea in your sys.path, and you sometimes
>import from one path and sometimes from the other.  You get two distinct
>modules with identical code.  It's very bad.
>

Indeed as usual you were right :)
I did some summer cleanup in all my import's and sys.path, now it works
like a charm.

Thx
Philippe