Re: [Modeling-users] Customizing autogenerated classes

[Sebastien B. <sbi...@us...>]

        Hi Brad & all,

Mario and soif did already comment, so I get directly to the point
Thanks Brad for raising the problem, it definitely should be answered!

Brad Bollenbach <br...@bb...> wrote:
> Now, onto my question:
>=20
> How do I customize the business logic validation routines without my
> customizations getting blown away the next time I autogenerate the
> Python package (e.g. when I decide two months down the line that the
> app needs to be extended and five more tables need to be added)?

As it was previously said, this is an interesting topic that should be
addressed and documented quickly. Most people face this problem once
they begin to develop real projects.

Mario> I also have the same question, and so I suggest that a recommendation
Mario> of how this is best handled should be added to the users guide...

  At least a recommendation, yes! I'll also expose another solution at
  the end of the message: we could propose more than a recommendation: a
  new generation scheme making life easier in these situations. However
  in the meantime, I agree this should be integrated in the User's
  Guide. I'll try to be as precise as possible, so that we can perhaps
  use it as a raw material for a coming section in the doc.

>=20
> I've read through many of your unit tests, and a few relevant bits of
> the code and come up with what I see being the four realistic options:
>=20
> 1. The obvious one: add code directly to autogenerated Python
> code. The problem here though, is that it's difficult to do the whole
> backup-and-merge dance when I autogenerate again in the future. It's
> also tedious to take the other route of only exporting the XML model
> every time, and maintaining the classes (and/or adding new ones, if
> applicable) to keep up with it.

It's the obvious one, for which cvs can help you a lot (wrt merging the
changes). This was the technique I once used, however this quickly
becomes tedious, indeed.

Soif also said that:
soif> 2) How would you stay update w/ the modeling if you change the
soif>    "bricks" (template used to generate code)

I must make it clear here that the templates/bricks used to generate the
code only contains the basics for the classes to be integrated into the
framework (i.e.: entityname() plus getters & setters calling
willRead()/willChange()). A real big change in the templates/generated
code would imply that the framework changed the way it integrates the
classes. This may happen in the future --I'm thinking ZODB's Persistent
class-- but this would be advertised and some migration scheme would be
proposed at that time.

  Until then, changes in the templates will not impact existing code.

soif> But i think this is a piece of the tool. I think you can use this
soif> "bricks" to generate some custom class

Sure, but the framework needs refactoring as far as the code-generation
is concerned. The way the ModelMasons are built is unclear and indeed
ugly, the API is unclear as well, badly documented, and part of it is
even useless. This is on the TODO list, with no ETA.


> 2. Write an adoption utility function, e.g. def adoptByClass(adopter,
> entities) that will iterate over a list of entities (e.g. those
> fetched from an editing context) and map them to the class specified
> as the "adopter". The "adopter" could be a class that I've created
> that inherits from one of the auto-generated classes and overrides its
> validation methods with my custom logic. It's good because I can
> autogenerate again and again, but it's bad because then I'm always
> making calls to adoptByClass.
>=20
> 3. Manually validate (e.g. using MySQLdb directly). This would ignore
> a significant piece of functionality that's already been created as
> part of the Modeling framework, so this isn't reasonable.

I don't like these two approaches and wouldn't recommend them, for the
same reasons you and soif gave.


> 4. Change an Entity's description so that it can have, perhaps both a
> baseclass *and* class attribute, where the baseclass attribute is the
> one that will get autogenerated and the class attribute is the only
> that I will code myself, overriding and extending the baseclass's
> methods as necessary.
>=20
> >From the point of view of the person that designed this framework,
> what do you think of each of these options? Have I missed any?


Following Mario's arguments, I don't think exposing a base class in the
description of entities is a good idea; but that's a detail however,
because the general idea of generating only baseclasses is the good one,
and the one I use and recommend. Here are the details of the technique I
currently use:

  Suppose we design a model AuthorBooks with entities Author and Books.

  1. design your model, name it 'AuthorBooksGeneration'

  2. design your entities, name them AuthorBase and BookBase

  3. generate the code: you get the package with the classes, plus
     model_AuthorBooksGeneration.xml

  4. in the same package, write class Author in module Author.py:

------------------------------------------------------------------------
from AuthorBase import AuthorBase
class Author(AuthorBase):
  def __init__(self):
    AuthorBase.__init__(self)
    #...
------------------------------------------------------------------------

     Same for Book

  5. drop the compile_model_AuthorBooks.py included at the end of the
     message in the package and execute it (adapt it to your own needs)
     It produces a new model, AuthorBooks, pointing to the subclasses
     Author and Book.

  6. Modify __init__.py so that the loaded model is AuthorBooks, not
     AuthorBooksGeneration


  Now you have classes Author and Book that won't be overriden when
  generating your classes again, using the original model
  AuthorBooksGeneration. The only thing you'll have to do after each
  re-generation is step 5. & 6. --obviously you still need to backup
  __init__.py if you added code to it.

=20=20=20=20=20
*PLEASE* backup your work before giving it a try, I've no time by now to
double-check that there is no risk at all and I do not want you to lose
your work just because I forgot some points in the procedure...

  This technique demonstrates that the model is really used at runtime
  to find out which classes it should use, here: not the base classes,
  but the subclasses. This is somehow a derivation of the 1st FAQ entry
  (section 2.6).

I hope this is clear enough; hopefully you'll find that this approach
can be an answer to your needs. Mario's and soif's comments, just like
yours, were exactly pointing to this direction.


Further comments on this:
soif> I think i will modify the bricks to handle this, during the week i
soif> hope.

  Now I agree that this could/should be handled at generation time,
  because this problem is a generic one. We should be able to offer the
  users an example of such an architecture, as soif suggested it, with a
  different python "brick". Soif, if you're going to study this maybe we
  could discuss that a bit --I'll suggest however not to modify the
  existing python brick, but to derive a new one for this purpose. For
  example, the following code-generation process could be implemented:

  - take an existing model (no 'Generation', no 'Base'), say AuthorBooks,

  - modify it to generate AuthorBooks.Base.AuthorBase and BookBase,
    where package AuthorBooks.Base is where live all the files that will
    be overriden unconditionally when re-generating the model (including
    the model) --we could simply use the existing python brick for that
    purpose.

  - Last, generate the modules/subclasses Author & Book in package
    AuthorBooks if and only if the modules do not already exists.


  What do you think?


-- S=E9bastien.






------------------------------------------------------------------------
#! /usr/bin/env python
"""
compile_model_AuthorBooks

  ``Compile'' an xml_model into two python modules, model_<modelName>.py and
  model_<modelName>_pickled.py:

    - model_<model_name>.py: contains the whole xml-file in a single module
        attribute: model_src
=20=20
    - model_<modelName>_pickled.py: the second one contains a single attrib=
ute
        as well: model_pickle, which is a raw string to be loaded by
        cPickle.loads()
=20=20=20=20=20=20

  Specific processing added:

    This script is dedicated to the model stored in
    'model_AuthorBooksGeneration.xml'

    The model within the xml file is loaded and changed (see fixModel()) th=
en
    written back to the filesystem, with the name 'AuthorBooks', under the
    three following forms:

      - xml file 'model_AuthorBooks.xml',

      - python file 'model_AuthorBooks.py',

      - python file 'model_AuthorBooks_pickle.py'


  Written by: Sebastien Bigaret
"""

import sys, getopt

def loadModel(xmlFilePath):
  """
  Loads the model stored in the xml file
  """
  from Modeling import ModelSet
  ms=3DModelSet.ModelSet()
  ms.addModelFromXML({'file': xmlFilePath})
  model=3Dms.models()[0]
  model._modelSet=3DNone
  return model

def fixModel(model):
  """
  Sets the model's name to 'AuthorBooks'; removes 'Base' from the
  entities' module name and class name.

  Returns: nothing
  """
  model.setName('AuthorBooks')
  for e in model.entities():
    name=3De.moduleName()
    if name[-4:]=3D=3D'Base':
      e.setModuleName(name[:-4]) # remove Base
    name=3De.className()
    if name[-4:]=3D=3D'Base':
      e.setClassName(name[:-4]) # remove Base


def write_xml_model(model):
  """
  Writes the model into the xml-file 'model_<modelName>.xml' ; creates a
  python module (file: 'model_<modelName>.py') and dumps the content of the
  xml-file in a string attribute named 'model_src'
  """
  # write xml file
  xml_file_path=3D'model_%s.xml'%model.name()
  model.saveModelAsXMLFile(xml_file_path)
  # write python file
  xml=3Dopen(xml_file_path, 'r')
  file=3Dopen('model_%s.py'%model.name(), 'w')
  file.write('model_src=3D"""')
  s=3Dxml.readline()
  while s!=3D'':
    file.write(s)
    s=3Dxml.readline()
  file.write('"""\n')
  xml.close()
  file.close()
=20=20
def write_pickled_model(model):
  """
  Creates a python module (file: model_<modelName>_pickle.py) and dumps a
  pickled version of the supplied model in module's attribute 'model_pickle=
'.
  """
  import cPickle
  file=3Dopen('model_%s_pickle.py'%model.name(), 'w')
  file.write('model_pickle=3Dr"""')
  pickled=3DcPickle.dump(model, file)
  file.write('"""\n')
  file.close()

def usage(prgName, exitStatus=3DNone):
  print """%s <model.xml>
``Compile'' an xml_model into two python modules, model_<modelName>.py and
model_<modelName>_pickled.py.

  - model_<model_name>.py: contains the whole xml-file in a single module
      attribute: model_src

  - model_<modelName>_pickled.py: the second one contains a single attribute
      as well: model_pickle, which is a raw string to be loaded by
      cPickle.loads()


NB: this adaptation is dedicated to the processing of
    'model_AuthorBooksGeneration.xml'. See code and its documentation for
    details.

Options
-------
  -h           Prints this message
""" % prgName
  if exitStatus:
    sys.exit(exitStatus)

def main(argv):
  me=3Dargv[0]
  try:
    options, args =3D getopt.getopt(argv[1:], 'h')
  except getopt.GetoptError,msg:
    usage(me, 1)
=20=20=20=20
  verbose=3D0
=20=20
  for k, v in options:
    if k=3D=3D'-h': usage(me, 0)

  if len(args)!=3D1:
    usage(me,1)
=20=20=20=20
  model=3DloadModel(args[0])
  fixModel(model)
  write_xml_model(model)
  write_pickled_model(model)

=20
if __name__ =3D=3D "__main__":
  main(sys.argv)
------------------------------------------------------------------------