RE: [Modus-devs] Collections of content objects and performance

[Nathan D. <na...@ch...>]

So, the question before us is whether "child" objects or "joined" objects
should be a separate kind of attribute of a contentObject from a field.
Hmm.

One question I've been thinking about is whether this is about a
parent-child relationship, in which case it makes more sense to use the way
you've been working on (and I'd consider calling it getChildren("threads")
or something like that.  Or,  is it more that a contentObject can have some
"joins" -- and really we need a way to define a join (as being either
one-to-many or many-to-many, for instance).

Seems the first issue is whether a field should be able to be of a type that
is actually just another contentObject.  An interesting question.  I think
the answer is yes, but I can also see how it causes a lot of problems for
the API and the cleanliness of the code base.  One idea I floated was that
there should be a way for a field to be an array of values, independent of
whether it's contentObjects or not.  Then, have a way for the value of the
field to be another contentObject.  A field would then, perhaps, have
methods like isArray() and isContentObject() -- which would help a user who
might want to build some abstracted interfaces.  Both arrays of values and
contentObjects as values creates some nastiness in the interface, though,
potentially.  For instance, renderSimpleForm() on a contentObject -- could
get ugly.

Right now I'm thinking that moving towards the XML descriptor is the first
step to any of this.   I am increasingly convinced that the CFPROPERTY route
is too limiting from an programming standpoint and apparently is also
potentially a big problem when it comes to performance (per an off-line
discussion with Sean Corfield).

Once we're using XML I think a lot of the other issues will be easier to
tackle.  So, for instance, the pressRelease.cfc in the modustest app might
look like:

<contentObjectType label="Press Release">
 <fields>
  <field
   name="title"
   label="Title"
   type="org.bacfug.modus.fields.text">
   <rules>
    <rule
     name="org.bacfug.modus.validation.full" />
   </rules>
  </field>
  <field
   name="body"
   label="Main Body"
   type="org.bacfug.modus.fields.longText"/>
  <field
   name="image"
   label="Photo"
   type="org.bacfug.modus.fields.webImage"/>
  <field
   name="featured"
   label="Featured?"
   type="org.bacfug.modus.fields.yesno"
   default="no">
   <rules>
    <rule
     name="org.bacfug.modus.validation.boolean"/>
   </rules>
  </field>
  <field
   name="category"
   label="Category"
   type="modustest.contentObjects.categoryPicker"/>
 </fields>
<contentObjectType>

First question: does that make sense?

So, now the issue is if I were building an app with child components (or
joined components) it might look like:

<contentObject label="Forum">
<field
name="threads"
isArray="yes"
type="myApp.threadComponent">
</contentObject>

Or, it might look like:

<contentObject label="Forum">
<childObjects>
    <objectType type="myApp.threadComponent">
</childObjects>
</contentObject>

Or, perhaps:

<contentObject label="Forum">
<joins>
    <join type="one2many" objectType="myApp.threadComponent">
</joins>
</contentObject>

Any thoughts from the rest of you about which would be best?

In that last one, would we think that the thread than also has a join in its
definition?  Would we need a separate place where a join is defined outside
of this config file?

Tangent: if we have fields that have arrays of values, do we want to expose
the CF array, or would we want to create methods to traverse it (using some
kind of iterator) to maintain the object-based API?

Hope that's enough to continue the conversation . . .

 - Nathan










-----Original Message-----
From: mod...@li...
[mailto:mod...@li...]On Behalf Of Jeremy
Firsenbaum
Sent: Tuesday, October 29, 2002 2:51 PM
To: modus devs
Subject: [Modus-devs] Collections of content objects and performance


Thought this thread might be of interest to others on the list. Nathan and I
have been talking about a couple of issues I've raised, namely,
contentObjects containing collections of other contentObjects and the
performance impact of instantiating so many objects.

Note that this reads from the bottom up:


Nathan - certainly not wed to the interface. But unless, as you suggest, the
definition of what a field is changes I don't see a way of using basefield
for content objects. Back to the forum example I've been using - retrieving
a collection of threads:

The way I suggested:

<cfset forum= createObject("component","forum").init(forumID)>
<cfset threads= category.getObjectArray('threads')>

<cfloop from="1" to="#arrayLen(thread)#" index="i">
    <!--- doing stuff with each thread, like: --->
    #thread[i].getField('thread').getValue()#
</cfloop>

One possible way of doing what you suggest:

<cfset forum= createObject("component","forum").init(forumID)>
<cfset threads= category.getField('threads').getValue()>

<cfloop from="1" to="#arrayLen(thread)#" index="i">
    #thread[i].getField('thread').getValue()#
</cfloop>

As you can see the only difference is the method call for the threads array.
And of course this gets back to the descriptor for the content object. In
the first case thread would have a property type of 'objectArray' or
'subcomponent' or something like that, and in the latter it has the type
'field' with a particular fieldType.

If the latter seems cleaner to you, fine. I just did it this way to avoid
reworking basefield -  sort of a proof of concept. The developer using the
API already needs to know the 'type' for various fields. Most are simple
strings, but anything that extends basefile will certainly be handled
differently, or at least make several new methods available. So making
'contentObject' be a possible fieldType should be doable - the
basefieldValue idea is probably the way to go. Let me think about that a
bit.

As for the performance issue, it comes in precisely when you want to
instantiate a whole array of contentObjects. If I call load() on a 40 page
book object, a new instance has to be created for each of these pages. The
only way to avoid that is to reference the cached object, which opens up a
whole other can of worms - locking the server scope, concurrent access to
writable data... This may be possible, but is this a route worth going down.
Maybe others have thoughts.

- Jeremy



----- Original Message -----
From: Nathan Dintenfass
To: Jeremy Firsenbaum
Sent: Tuesday, October 29, 2002 12:27 PM
Subject: RE:


Ah, I see what you mean.  Yes, get() does that, but only if you wish to
populate from another instance -- which you pass in -- that is, if you have
an existing contentObject that you want to "load()", then you need to do
this.  But, in this case you still never need to recreate a whole new
instance as you already have the one you want to load and the one you
retrieve from the cache (except on the first hit) -- but, what you are
saying is that is a performance bottleneck for you, right?  Hmm.

On the "interface" I was talking about -- yes, the API exposed to the end
user.  I figured you were not wed to it.  My instinct says that it would be
better to do an overhaul of what a "field" is.  First, I'd make it possible
to have an array of values, even if not other contentObject.  Then, I'd work
on making it possible to have the value of a field be a contentObject.
Perhaps baseFieldValue.cfc would be one way to get towards that??

I totally agree that one way or another there needs to be a way to have a
"book" and its "pages" (not to mention many-to-many relationships) and have
"book" and "page" each be instances of a contentObject.

 - Nathan



-----Original Message-----
From: Jeremy Firsenbaum [mailto:jfi...@ma...]
Sent: Monday, October 28, 2002 5:33 PM
To: na...@ch...
Subject: Re:


"By the way, the only place I am using the makeClone() is when I put objects
into the cache, not when I take them out of the cache"

I'm sorry if I was misleading. "Cloning" takes place going in as well as
coming out, although the makeClone method is only called one way. Going in,
makeclone calls contentObjectPopulateFromInstance, which does all of the
actually duplicating. Coming out, get() calls
contentObjectPopulateFromInstance as well:

line 41 from simplefilesystempersister.cfc:

if(instance.cache.isObjectCached(arguments.id)){
    objectRetrieved = instance.cache.getObject(arguments.id);
    if(structCount(arguments) GT 1)
     return contentObjectPopulateFromInstance(arguments[2],objectRetrieved);
    else
     return objectRetrieved;
}

I found the duplicating from a cached instance in
contentObjectPopulateFromInstance to be the bottleneck.

"I'm not sure I'm personally satisfied with the interface you are using in
your forums app."

I'm all up for criticism - but not sure what you're referring to by
interface. Do you mean the contentObjectArray within basecontentobject, or
the way in which the API is used externally to construct the forum.
Obviously this was just an example I was using to work with the idea of
content objects within content objects.

"Are you waiting for a signal from me as to what my alternative might be?"

Well - kinda, yeah. I mean I would like to know if this approach to
"composition" even makes sense, or whether there's another way of doing
"parts of" relationships within modus. Many systems, content management or
otherwise, have this kind of structure. An artifact, say a book, has many
pages, each one of which has been scanned in. At one level, you want to deal
with the book as a whole (when presenting a library of books), on another
level, each one of the pages is a content object unto itself (so that the
book is browsable). I would like to be able to load the book, for a single
request or in session scope, and have the pages come along for the ride,
just as each thread is available within the forum content object. Does this
even make sense within the context of what modus was and is being designed
to do?

----- Original Message -----
From: Nathan Dintenfass
To: Jeremy Firsenbaum
Sent: Monday, October 28, 2002 7:56 PM
Subject: RE:


Ah, yes.  Well, right now you are at the cutting edge of that question.  I'm
not sure I'm personally satisfied with the interface you are using in your
forums app.  Doesn't feel clean to me, but I have a hard time justifying
that feeling.  Are you waiting for a signal from me as to what my
alternative might be?

By the way, the only place I am using the makeClone() is when I put objects
into the cache, not when I take them out of the cache -- are you
experiencing serious performance problems with that?  My main issue with
performance before the caching was with the getAll().  Load() is still a bit
slower than I'd like, but I figure that will be rarely called in volume
since getAll would be generally be faster.  That is an area to work on,
though.

 - n




-----Original Message-----
From: Jeremy Firsenbaum [mailto:jfi...@ma...]
Sent: Monday, October 28, 2002 4:45 PM
To: na...@ch...
Subject: Re:


Sorry for being obtuse - I mean being able to declare other content objects
as being contained within another content object: forums are composed of
threads, which are composed of posts...
----- Original Message -----
From: Nathan Dintenfass
To: Jeremy Firsenbaum
Sent: Monday, October 28, 2002 7:37 PM
Subject: RE:


When you say "composition of content object" what do you mean?

 - n
-----Original Message-----
From: Jeremy Firsenbaum [mailto:jfi...@ma...]
Sent: Monday, October 28, 2002 4:33 PM
To: na...@ch...
Subject: Re:


Fair enough. Maybe it's all of the nested subcomponents that accounts for
the slowness in cloning from the cache.

I'd love to do some more work on modus, maybe sink my teeth in the XML
descriptor thing, but I feel like I can't do much with it until I know
whether composition of content objects is feasible. This seems to be the
most important feature needed to model more complex systems. Looking forward
to seeing what you think.

-Jeremy
----- Original Message -----
From: Nathan Dintenfass
To: Jeremy Firsenbaum
Sent: Monday, October 28, 2002 7:13 PM
Subject: RE:


Jeremy:

I hadn't yet gotten to look in depth at your changes.  I agree the clone()
method is far from optimal (as I mentioned in that posting).

My comment was based on the dramatic performance enhancement I witnessed on
the modustest app after implementing the caching.  My calls to getAll() went
from over 400ms with a handful of "press releases" to 20ms.

 - n

-----Original Message-----
From: Jeremy Firsenbaum [mailto:jfi...@ma...]
Sent: Monday, October 28, 2002 3:52 PM
To: Nathan Dintenfass
Subject:


Hey Nathan

I saw your post to CFCDev today:

"I have taken pages that were slow because they had to instantiate dozens of
components and made them very fast by just grabbing from the cache in the
server scope."

This got me wondering if you have looked into the performance issue with
clone() that I mentioned last week. It seems like it wouldn't make a
difference whether the subcomponents were instantiated within the
contentObject, as I had tried to do, or if this were left to the developer
to do externally: forums = createObject (forum.cfc), loop for threads =
createObject(thread.cfc). If you need "dozens of components" the clone
technique doesn't seem to be optimal.

Were my performance numbers similar to yours (if you've had time to look
into this) or am I totally off base here?

Jeremy