jython-users

[Jython-users] Issue with None and Serialization

[chuck c. <cc...@zi...>]

I'm using Jython for my project and today we tried adding the
functionality where you can save out a Config object and then when you
restart you can load a Config if you need it.  I'm using the java.io
libraries to do this because I can't assume pickle or any of the other
python libraries are on the machine.

However I ran into an issue where after reading the serialized object in
some statements which checked to see if an attribute on the object is
None were failing.  I've recreated the problem in the code snippet below
as succinctly as I can.  Does anyone have any ideas on what I'm doing
wrong?  I have a feeling it must be something really simple or could
this be a bug in Jython?

TIA
chuck

-------code-------
from java.io import *
from org.python.util import *

class Test(Serializable):
    def __init__(self):
        self.attr = None
    def test(self):
        print "attr is None:", self.attr is None
        print "attr type is:", type(self.attr)
        print "attr == None:", self.attr == None

def load(path):
    file = File(path)
    fileIn = FileInputStream(file)
    pyIn = PythonObjectInputStream(fileIn)
    pyObj = pyIn.readObject()
    pyIn.close()
    return pyObj

def save(obj, path):
    fileOut = FileOutputStream(path)
    objOut = ObjectOutputStream(fileOut)
    objOut.writeObject(obj)
    objOut.flush()
    objOut.close()

print "Testing initial object..."
a = Test()
a.test()
save(a, "test.out")
b = load("test.out")
print "Testing deserialized object..."
b.test()

-----results----
Testing initial object...
attr is None: 1
attr type is: org.python.core.PyNone
attr == None: 1
Testing deserialized object...
attr is None: 0
attr type is: org.python.core.PyNone
attr == None: 0

I would expect that if the attribute is of Type PyNone then it should
return 1 to
attr is None
and
attr == None

I thought the problem might be that between serializing/deserializing I
might somehow be getting a different instance of PyNone but this isn't
the case as evidenced below when I inspect the objects in the
interpreter after running th code above
>>> type(a.attr)
<jclass org.python.core.PyNone at 334527779>
>>> type(b.attr)
<jclass org.python.core.PyNone at 334527779>

[Jython-users] RE: Issue with None and Serialization

[brian z. <bz...@zi...>]

Chuck,

Why can't you assume cPickle is available?  It's standard in the Jython
dist as a Java-implemented module.  I have modified your example a bit
to use it and it seems to behave properly.  Add the following two
methods:

--- code ---

import cPickle

def load_with_pickle(path):
	fp = open(path, "rb")
	try:
		data = cPickle.load(fp)
	finally:
		fp.close()
	return data

def save_with_pickle(obj, path):
	fp = open(path, "wb")
	try:
		cPickle.dump(obj, fp)
	finally:
		fp.close()

 ... 

print "Testing initial object..."
a = Test()
a.test()
save_with_pickle(a, "test.out")
b = load_with_pickle("test.out")
print "Testing deserialized object..."
b.test()

--- results ---

$ jython cc.py
Testing initial object...
attr is None: 1
attr type is: org.python.core.PyNone
attr == None: 1
Testing deserialized object...
attr is None: 1
attr type is: org.python.core.PyNone
attr == None: 1

Nonetheless, I see the same problem you do with the code snippet.

brian

> -----Original Message-----
> From: ch...@er... [mailto:ch...@er...] On 
> Behalf Of chuck clark
> Sent: Tuesday, March 12, 2002 7:32 PM
> To: jyt...@li...
> Cc: bu...@cu...
> Subject: Issue with None and Serialization
> 
> 
> I'm using Jython for my project and today we tried adding the
> functionality where you can save out a Config object and then when you
> restart you can load a Config if you need it.  I'm using the java.io
> libraries to do this because I can't assume pickle or any of the other
> python libraries are on the machine.
> 
> However I ran into an issue where after reading the 
> serialized object in
> some statements which checked to see if an attribute on the object is
> None were failing.  I've recreated the problem in the code 
> snippet below
> as succinctly as I can.  Does anyone have any ideas on what I'm doing
> wrong?  I have a feeling it must be something really simple or could
> this be a bug in Jython?
> 
> TIA
> chuck
> 
> -------code-------
> from java.io import *
> from org.python.util import *
> 
> class Test(Serializable):
>     def __init__(self):
>         self.attr = None
>     def test(self):
>         print "attr is None:", self.attr is None
>         print "attr type is:", type(self.attr)
>         print "attr == None:", self.attr == None
> 
> def load(path):
>     file = File(path)
>     fileIn = FileInputStream(file)
>     pyIn = PythonObjectInputStream(fileIn)
>     pyObj = pyIn.readObject()
>     pyIn.close()
>     return pyObj
> 
> def save(obj, path):
>     fileOut = FileOutputStream(path)
>     objOut = ObjectOutputStream(fileOut)
>     objOut.writeObject(obj)
>     objOut.flush()
>     objOut.close()
> 
> print "Testing initial object..."
> a = Test()
> a.test()
> save(a, "test.out")
> b = load("test.out")
> print "Testing deserialized object..."
> b.test()
> 
> -----results----
> Testing initial object...
> attr is None: 1
> attr type is: org.python.core.PyNone
> attr == None: 1
> Testing deserialized object...
> attr is None: 0
> attr type is: org.python.core.PyNone
> attr == None: 0
> 
> I would expect that if the attribute is of Type PyNone then it should
> return 1 to
> attr is None
> and
> attr == None
> 
> I thought the problem might be that between 
> serializing/deserializing I
> might somehow be getting a different instance of PyNone but this isn't
> the case as evidenced below when I inspect the objects in the
> interpreter after running th code above
> >>> type(a.attr)
> <jclass org.python.core.PyNone at 334527779>
> >>> type(b.attr)
> <jclass org.python.core.PyNone at 334527779>
> 

Re: [Jython-users] RE: Issue with None and Serialization

[chuck c. <cc...@zi...>]

thanks brian...
on one front it was really simple...i tried import pickle and got an
ImportError and I moved on to writing my own serialization not even thinking
to try importing cPickle

i've switched to using the cPickle code now and everything works as expected

it looks like samuel has tracked down the bug...so all ends well

thanks
chuck

----- Original Message -----
From: "brian zimmer" <bz...@zi...>
To: <cc...@zi...>; <jyt...@li...>
Cc: <bu...@cu...>
Sent: Tuesday, March 12, 2002 7:50 PM
Subject: [Jython-users] RE: Issue with None and Serialization


> Chuck,
>
> Why can't you assume cPickle is available?  It's standard in the Jython
> dist as a Java-implemented module.  I have modified your example a bit
> to use it and it seems to behave properly.  Add the following two
> methods:
>
> --- code ---
>
> import cPickle
>
> def load_with_pickle(path):
> fp = open(path, "rb")
> try:
> data = cPickle.load(fp)
> finally:
> fp.close()
> return data
>
> def save_with_pickle(obj, path):
> fp = open(path, "wb")
> try:
> cPickle.dump(obj, fp)
> finally:
> fp.close()
>

Re: [Jython-users] Issue with None and Serialization

[Samuele P. <pe...@in...>]

From: chuck clark <cc...@zi...>
> I thought the problem might be that between serializing/deserializing I
> might somehow be getting a different instance of PyNone but this isn't
> the case as evidenced below when I inspect the objects in the
> interpreter after running th code above
> >>> type(a.attr)
> <jclass org.python.core.PyNone at 334527779>
> >>> type(b.attr)
> <jclass org.python.core.PyNone at 334527779>
> 

No, you're right it is a *bug* and your hypothesis is right,
only the above give you the "address" of the PyNone class.
The following is more revealing:
>>> execfile("test.py")
Testing initial object...
attr is None: 1
attr type is: org.python.core.PyNone
attr == None: 1
Testing deserialized object...
attr is None: 0
attr type is: org.python.core.PyNone
attr == None: 0
>>> id(None)
6503157
>>> id(a.attr)
6503157
>>> id(b.attr)
8302342

It is a problem with singletons serialization, it has gone
unnoticed until your report, it affects None, Ellipsis etc

Thanks, Samuele Pedroni.