How to use saveObjects and loadObjects

[Maurizio De Pitta']

Dear all,
I am trying to run a complex continuation problem for which I want to save intermediate steps of computation.

I am trying to use methods in PyDSTool such as saveObjects and loadObjects to handle my objects but I cannot make it work.

Following instructions on the user manual I have a bunch of objects such as: DSargs for a system specification, a 'traj', 'pts' and 'ode' objects related to a trajectory, set of points and an instance of a CVODE generator, and finally a PyCont (ContClass) object PC that I obtain to compute continuation curves like EC1, LC1 (i.e. PC has PC['EC1'], PC['LC1']).

I then wanna save in myfile.sav and use:

import PyDSTool as dst
dst.saveObjects('myfile.save',[DSargs,traj,pts,ode,PC])

But I get the following error:

TypeError                                 Traceback (most recent call last)
<ipython-input-48-7aecd6dfb4a2> in <module>()
      1 #dst.saveObjects('chain3.sav',['DSargs','traj','pts','ode','PC'])
----> 2 dst.saveObjects('chain3.sav',[DSargs,traj,pts,ode,PC])
      3 
      4 
      5 #import save_utils as svu

/home/maurizio/Downloads/PyDSTool/utils.pyc in saveObjects(objlist, filename, force)
    612     # protocol (e.g. binary format)
    613     if not force:
--> 614         if os.path.isfile(filename):
    615             raise ValueError("File '" + filename + "' already exists")
    616     pklfile = open(filename, 'wb')

/usr/lib64/python2.7/genericpath.pyc in isfile(path)
     27     """Test whether a path is a regular file"""
     28     try:
---> 29         st = os.stat(path)
     30     except os.error:
     31         return False

TypeError: coercing to Unicode: need string or buffer, list found

Thanks for your help.

M

[Rob Clewley]

You need to reverse the parameter ordering. Did you try help(saveObjects)? :)

[Maurizio De Pitta']

No I didn't... I am still missing some basic knowledge on Python introspection.

However after I reverse the order according to the help, it seems that ContClass objects are not handled by this method still.

Indeed I get the following (where 'PC' is a ContClass object obtain by PC = dst.ContClass(ode)):

dst.saveObjects([DSargs,ode,traj,pts,PC],'chain3.sav',force=True)

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-65-aa7a1dfec388> in <module>()
----> 1 dst.saveObjects([DSargs,ode,traj,pts,PC],'chain3.sav',force=True)

/home/maurizio/Downloads/PyDSTool/utils.pyc in saveObjects(objlist, filename, force)
    628     for obj in objlist:
    629         try:
--> 630             pickle.dump(obj, pklfile, opt)
    631         except:
    632             if hasattr(obj, 'name'):

TypeError: expected string or Unicode object, NoneType found

Failed to save object 'ContClass of model chain3'

---

If I leave out the PC object from my list instead, everything seem to work fine.

[Rob Clewley]

Could you open an issue ticket on github about PyCont not being picklable? It should not be super-hard to fix that when I have time.

[Florian Aspart]

Hi,

I actually had the same problem when trying to save my curves for plotting them later.
I'm not sure but I think the problem comes from the fact that PyCont curves hold a reference to some binary code ( Auto module or the C-integrator).

My current workaround for saving my work is to save the "sol" attributes of each curve separately:
dst.saveObjects([PC['EQ_1'].sol,PC['EQ_2'].sol],'chain3.sav',force=True)

To plot them afterwards, I just loaded the sol objects, created a dummy PyCont curve of the same type as the one I saved and replaced the "sol" attribute of the newly created dummy curve with the one I loaded.

It worked for me for plotting the curve in another session and might also work for computing continuation your curve in another session.
Maybe this could help you, as long as the bug is fixed. Note that it would not save your continuation parameter ( StepSize, , tolerance, etc...) though.

Best,
Florian

[Maurizio De Pitta']

Hi Florian,
thanks for the suggestion. I will try that out.

I shall point out that my original post was motivated by the fact that I am interactively building my bifurcation diagrams by ipython notebook (iPyN), and sometimes, in order to optimally tune the continuation, I want to recompute the curve. I was not originally able to recompute a curve or a branch after the first instance as the error message was telling me that the 'curve XX already existed'.

However I found out (on the User manual) that to recompute the curve it is sufficient to specify 'force=True' in the definition of args.

Example:

You are about to continue and equilibrium point along a curve EP-C. Then on the initialization of the curve (called EQ1) make sure to use force=True:

PCargs = dst.args(name='EQ1', type='EP-C',force=True)

In this fashion I can run my iPyN cell again and again till I fine tuned my solution.

[Maurizio De Pitta']

Hi Florian,
just a quick update to confirm that the workaround that you suggested indeed works. Just you need to make sure to reconstruct exactly the PC ContClass object and then assign what you previously saved to it, i.e.

Save it as:
dst.saveObjects([PC['EQ_1'].sol,PC['EQ_2'].sol],'chain3.sav',force=True)

Reconstruct it as:

PC = dst.ContClass(<my original ode generator or model>)
PCargs = args(name='EQ1',type='EP-C',force=True,<other original args>)
PC.newCurve(PCargs)
PCargs = args(name='LC1',type='LC-C',force=True,<other original args>)
PC.newCurve(PCargs)
PC['EQ1'].sol,PC['LC1'].sol = dst.loadObjects('chain3.sav')

Then I confirm that once picked one of the special points in the curves and set it as initial point to extend the curve, everything works.

Thanks.

M

[Florian Aspart]

Hi Maurizio,

I'm glad it helped you.

By the way, I'm not sure you need to "pick one of the special points and set it as initial point to extend the curve".
As far as I understand it, the only information about the existing curve is in the "sol" attribute.
Just setting this again should be sufficient.

Best,
Florian

[Maurizio De Pitta']

Un update on this issue. If you use dill module now, you may pickle any PyDSTool Object and reload them in a new session to perform any operation, i.e. integration or continuation.

[Rob Clewley]

Can you confirm that dill will serialize the PyCont classes? Because I didn't think they used to. Has dill undergone an update that might have improved its capabilities?

[Maurizio De Pitta']

Hi Rob,
I can confirm it.

Here are my two save/load routines:

import dill as pickle

def savedata(data,filename='./data.pkl'):
"""
Save list of variables in 'data' in file 'filename
Use:
savedata(data, filename='./data.pkl')

Input:
- data     : [var1,var2,...] list of variables names (NO strings! but real names)
- filename : string for file name (usually with extension 'pkl')

Maurizio De Pitta', The University of Chicago, August 18th, 2014.
"""
with open(filename, "wb") as f:
    pickle.dump(len(data), f)
    for value in data:
        pickle.dump(value, f)

def loaddata(filename,verbose=False):
    """
    load list of variables in 'data' in file 'filename
    Use:
    data = loaddata(filename)

    Input:
    - filename : string for file name (usually with extension 'pkl')

    Output:
    - data : list of objects

    Maurizio De Pitta', The University of Chicago, August 18th, 2014.
    """

    data = []
    with open(filename, "rb") as f:
        for _ in range(pickle.load(f)):
            data.append(pickle.load(f))
    if verbose : print "loaded: ", data
    return data

So for example, I want to save DS integrator object, traj object and PC PyCont class object out of my run:

# Save data
savedata([DS,traj,PC],'myds.pkl')

# Load data
import PyDSTool as ds # Might need to import module in the same way it was imported in the "save" scope (but not checked)

DS,traj,PC = loaddata('myds.pkl')

M