Re: [Cppcms-users] template plugins

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

Hi, 

Thanks for your detailed reply.

I apologise for the length of this email, but things are so confused to me at 
this stage, that I though too much information and redundant information may 
be better than too little.

And I thank you for your time spend reading and replying!

On Tuesday, January 03, 2012 06:12:02 PM Artyom Beilis wrote:
>    - Full rewrite of templates loading API (0.99.11)
>      which by the way would make modules generation much
>      easier.

Oh, up to today I have been using 0.99.10.
I see you released 0.99.11 ten days ago:
http://art-blog.no-ip.info/cppcms/blog/post/90
In your blog, you don't give any details on how / why the views have been 
refactored, and the implication, if any, for existing code 
(e.g. what is now possible that wasn't before?)

> So once 1.0.0 is released there would be immediately created a new
> beta version 1.1 that would receive new features and once it is stabilized
> it would be released as stable version 1.2 while 1.0.0 would receive
> only bug fix updates (1.0.1, 1.0.2 etc).
> 
> So the release frequency would be high and plugin support would
> be scheduled according to its priority (which is high BTW)

Ok, thanks. I understand the reasons for such a long release schedule to 
1.0.0. (and I certainly wasn't complaining about it! ;) )

So, back to my original question: can you guesstimate the release dates for 
1.0 and 1.1?
From yesterday's post, I guesstimate that  1.0.0 RC1 will be released this 
month, and the final 1.0.0 in February at the latest
When would the first beta 1.1 be released? Early spring?
And the first 1.1.0 RC? 
I don't mind developing against a beta release. I guess it'll be stable enough 
for a live release of my own application late this year.....

> > There seems to be a fairly large demand for a plugin framework. Several
> > people have already mentioned it on this list. Fortunately, we can roll
> > out our own without waiting for CPPCMS 1.1. However, I counted at least
> > three different people, maybe four, who brought the same issue over the
> > last 12~18 months, and I would be the fifth person to do so.
> 
> Small notes. Today CppCMS provides all the basics for support of plugins.
> 
> Templates are already fully pluggable, load shared object and you
> get your template in place.
> 
> Also there is a new booster::shared_object class that allows to load
> dynamic libraries easily.

OK. As I told you earlier, I have already implemented my own plugin framework, 
but since I was following unrelated tutorials found on the web, it's very 
possible I made things more complicated than they needed to be. On the other 
hand, I learned a lot that way! ;)

> On the other hand if I spend now lots of time today on plugin framework
> and tomorrow on other very important feature I'll never release a stable
> version and this is not good for the project. It was in beta stage for
> too long time.

Well, I was not asking for you to implement the whole plugin framework right 
now, before the release 1.0.0.
However, it seems that I am missing a single method in order to complete my 
own plugin framework: how to attach new template includes. And again, I am not 
the first one to ask about it. See below.
So, what I was asking is, in terms of additional code before the code freeze, 
can we have that extra method to complete our own frameworks? (unless, again, 
I am doing things the wrong way, and what I am asking is possible another 
way...)

--

I started replying to the rest of your comments, but I guess the discussion 
will be clearer if I do my best to state as clearly as possible what I am 
after.

Assume I only want to implement one single theme for my web site, i.e. one 
'skin' (one single layout, one colour scheme, etc.). Let's call this unique 
skin: my_skin.

I want the HTML output to be very simple, as follows:

<!DOCTYPE etc... >
<html>
<head>
	title, metas, links here.
</head>
<body>
	<div id="top_menu"> home | blog | etc </div>
	<div id="page_content"> (insert content here) </div>
</body>
</html>

Let's say the core of the application defines the general layout and a core 
.tmpl file declares the headers, the top_menu div like this:

<% c++ #include "content.hpp" %>
<% xhtml %>
<% skin my_skin %>
<% view page uses content::page %>
<% template render() %>
<html>
<head>
	title, metas, links here.
</head>
<body>
	<div id="top_menu"> home | blog | etc </div>
       <% include main_content() %>
</body>
</html>
<% end template %>
<% end view %>
<% end skin %>

Now, the content of <div id="page_content"> is defined in each plugin. 
When the application starts, the revelant plugins are loaded (no problem).
The plugins each register their own paths to the router (no problem).
Depending on the user request, the corresponding methods in the relevant 
plugin is called (no problem).

Each plugin implement its own .tmpl file as follows:

<% skin my_skin %>
<%view example_page uses content::example_page extends main %>
<% template main_content() %>
        <p>This is the content output by this plugin.</p>
        <%= content | raw %>
<% end template %>
<% end view %>
<% end skin %>

We'll discuss below the details of the implementation. Right now, the 
important is that you understand what I am after. 
The core of the application is defining the general layout of the template, and 
tells where the content from different plugins shall be inserted.

The above example is at its simplest: only one plugin at a time is affecting 
the output. It starts to get trickier when the whole html page is divided into 
a random number regions and different plugins are responsible for the content 
of each region.

> > It seems that independently from each other, we all were confronted with
> > the same issue, namely: how does a plugin register a single part of a
> > larger template which is implemented by the core application?
> 
> Simple:
> 
> Each template shared object has global constructor and destructor.
> Once shared object is loaded, constructor is called and it registers
> itself in cppcms::views::pool singleton and become visible for everybody.
> When it is unloaded it is unregistered via global destructor and can't
> be used any more. It is simple and transparent semantics.
> 
> > Let's say we have a core application which implements a common template
> > (skin). And we have several plugins, each implementing a functionality,
> > e.g. a block of content with its own specific HTML markup
> > (sub-template). The core of the application does not know in advance
> > what additional sub-templates the plugins will implement, and plugins do
> > not know what other plugins need. Basically, what each of us were trying
> > to do is to 'attach' a
> > sub-template to the core main template. And we all hit the problem that
> > you don't offer an API to do so.
> 
> If the plugin is linked with the template code it would loads its own
> template:
> 
> libmy_plugin.so
>    my_application.cpp
>       // my code
>       {
>          ...
>          render("my_plugin_skin","some_view",some_content);
>       }
>    my_view.cpp <- my_view.tmpl
>    // this code is auto-generated
>      namespace {
>        struct loader {
>          loader() {
> cppcms::views::pool::instance().add(my_skin_generator()); } ~loader() {
> cppcms::views::pool::instance().remove(my_skin_generator()); } }
> loader_instance;
>      }
>    
> So once you load shared object loader::loader() is executed
> and registers a view
> and when you unload the shared object loader::~loader()
> unregisters it.

Hmmmm... I've read all of the above a dozen times, been pondering its 
relevance, and I still can't decide whether it's you who don't understand my 
question or if it's me who don't understand the answer. If hope my simple case 
scenario above is helping...

> Only thing you should care is to make sure each as its own skin name and
> not collide with its other.

(=> with *each* other)

Err... This is a major departure from what I was trying to do. Maybe I don't 
understand the way you use the term 'skin'. For me, 'skin' refers to the 
general look and appearance and colour scheme of the web site. A plugin 
implementing a block of content would output this content in a way that's 
congruent with the site's skin/theme.

So, in my plugins, their templates use the same skin name as the core 
templates. It compiles, but it's useless! Semantically, it's the same skin. 
And I fail how using different skin names will solve my problem.

The most likely explanation is that I don't yet fully grasp the the proper 
uses of 'skins' and 'views' or their respective meaning, but it seems to me 
that you're telling me that each plugin should implement a whole skins.

http://art-blog.no-ip.info/wikipp/en/page/cppcms_1x_templates_gen
I quote from above:
	"Each skin should implement the virtual function render(),
	 unless it is already implemented in its parent."
Do you mean that a skin can have parents??
In the wiki page above, you mention a views hierarchy (which, I think, I 
understand) but not a skin hierarchy!

Again, for me skin == theme. Only one is operative for any page request.
Or do you mean that a single page can be assembled from the output of different 
skins?

For reference, my problem appears to me to be very, very similar to that of 
Julian Pietron, in the second half of the thread "Plugins and templates", from 
August 19th~23rd 2010.

He wrote:

> > Would it be possible to register a view at runtime? As I see in the code
> > generated of cppcms_tmpl_cc, the template loader registers the skin
> > alongside with a mapping_type that contains the name of view and the
> > appropriate handler class / the result of passing it together with the
> > required content object to view_builder.
> > So could you add a call like
> > cppcms::views_pool::static_instance::extend_view("skin","view
> >name",&cppcms::views_pool::view_builder<skin::view_class,content::master>);
> > that I could call in the constructor of my plugin?
> > I would then compile the output of cppcms_tmpl_cc excluding the loader
> > to my shared object and could add the required views at runtime to
> > cppcms view_pool.
> > It would be good if there also was a call like reduce_view("skin","view
> > name"); just to remove a view when plugin is unloaded.

Over the last year, I have read that thread dozens of times, and I referred to 
it in my previous email. But I still don't understand the solution; I am even 
confused as to whether there IS a solution or not! :-/

You gave him the same solution (Aug 20th 2010) as you gave me, but something 
still does not add up for me. (sorry to be dense!)

In that email, you wrote:

> > 3. In the plugin itself you can use
> > views_pool::static_instance().render(...)
> > to render the view.

I have been looking at the different overloaded functions render(...):
http://art-blog.no-
ip.info/cppcms_ref_v0_99/classcppcms_1_1application.html#a46054c0f0e55b2638cc73fc8eb07fe4f

and I still fail to see how I can tell each plugin to render their specific 
parts according to their own data and own templates, and to pass the result 
back to the core application which will render the whole page and send it to 
the user.

How would 
 views_pool::static_instance().render(...)
interact with the core template/view/skin?

> > 4. Note: you can't load and unload two plugins simultaneously as it would
> > not be thread safe,

I don't understand what you mean by "loading two plugins simultaneously".

> > We can manually create a header file for the sub .tmpl file (part of the
> > plugin), so that the template at least compiles, but then the core
> > application is not aware of it and doesn't use it.
> 
> I don't really understand what are you trying to say in the paragraph
> above.

I would have sworn that another user asked the very same question fairly 
recently but I have gone through the whole 2011 archives and earlier, and 
couldn't find it.

Basically, what follows is the implementation side of the simple example I 
posted above:

// core content.hpp
struct main : public cppcms::base_content {
        std::string title;
        std::string content;
}

// plugin: plugin_content.hpp
#include "core/content.hpp"
namespace content {
struct example_page : public main {
        std::string my_test;
};
} // namespace content.

As you see, the plugin content are sub-classes of the main content.
In the thread mentioned above,  Julian asked:

>>  it would be nice if cppcms_tmpl_cc could be modified to
>>  generate a header file alongside the cpp, so that I can easily include
>>  it into my plugin

and you denied it (certainly for very good reasons).
Anyhow, I got to manually re-create the header for the core content.cpp, so 
that I could include it in my plugin's template page (as shown above).

Thus the code compiles: one skin shared between core and the various plugins. 
A core content class with sub-content classes within plugins.

However, cppcms_tmpl_cc output .cpp files, at the bottom of which one finds code 
like this:

namespace {
 cppcms::views::generator my_generator;
 struct loader {
  loader() {
   my_generator.name("my_view");
   my_generator.add_view<my_view::page,content::page>("page",true);
   my_generator.add_view<my_view::main,content::main>("main",true);
   my_generator.add_view<my_view::system_error,content::system_error>("system_error",true);
   my_generator.add_view<my_view::page_form,content::page_form>("page_form",true);
   my_generator.add_view<my_view::configure_plugins,content::configure_plugins>("configure_plugins",true);
    cppcms::views::pool::instance().add(my_generator);
 }
 ~loader() {  cppcms::views::pool::instance().remove(my_generator); }
} a_loader;
} // anon 

which obviously registers the view within the pool.

And what I was originally asking, is a method to add a template to an already 
registered skin/view.
I.e. I am looking for a way to tell the core view that a certain function 
(implemented in the base view class) has been replaced by another function 
with the same name in the derived class.

Anyway... this was how I attempted to do it. From what you keep telling us, I 
have the nagging feeling that I use the wrong approach, but, please forgive 
me, I still can't grok the proper approach to take.

Also, I checked the "plugin implementation" that kpeo posted in November but:
1) I can't use it because of licensing issues.
2) It doesn't seem to answer my problem anyway, as it seems to skirt around 
the very issue that's causing me a headache by not implementing what I'm 
after!

> > I didn't fully understand all of your answers, though I tried and read
> > them many times. From what I understood, your answer was two-fold:
> > 
> > 1) On one hand you say that the plugin can recompile the whole skin and
> > load it at run time. Maybe there is a need to properly document this
> > because I didn't fully understand how to do so, and how a plugin would
> > be aware of all
> > 
> > the other sub-templates implemented by the other plugins.
> 
> See the example above. Also plugin should not "recompile" the skin
> in the run time... It can do it you need to implement "recompilation"
> on your own.

The skin is compiled beforehand at the same time as the plugin.
What I'm trying to do at run time is let core know that a certain piece of 
template should be inserted at its proper place within a larger skin.

> > 2) On the other hand, you seemed to say that it was not possible, because
> > it was difficult to do it right, especially considering race conditions
> > in a multi- threaded web server, leading to a crash when one thread is
> > trying to render a sub-template from a plugin that was just disabled in
> > another thread.
> 
> The cppcms::views::pool::render(...) is thread safe
> as it holds shared lock during any rendering process. If you
> would try to unload a skin during this period you would be blocked
> till **all** rendering operations will complete and
> then you'll be able to load/unload the view

Julian asked:

> > Would it be possible to  register a view at runtime? 

You replied:

> > Not sure. It would required making views loading
> > and unloading be thread safe and make the interaction
> > with actual DLL quite complicated.

But that was 18 months ago, so maybe things have changed since then...

> However if you want to load/unload **application** classes
> it is totally different story... I there is no support
> of dynamic **unloading** of CppCMS applications, only templates.

I only have one application, which is core. Core is then supposed to pass one 
relevant data to plugins which are then responsible to output specific blocks 
of content within a large page.
I can already dynamically load/unload plugins at run time, which is the whole 
point ;)

> > So, my request is thus:
> > 
> > a) If what we are trying to implement is fully possible today, can the
> > proper solution be documented?
> 
> Yes... indeed. I think this e-mail content should go to the wiki.
> (after some notes below)

You know me: I always like it when I can help in some small ways, and unlike 
other people here, I have contributed what I could to the wiki. Unfortunately, 
at this time, I am still in the dark about your answer.... :-/

> > b) If not, some projects will not be able to wait until the release of
> > CPPCMS 1.1 (see questions at the top). Could you at least implement in
> > 1.0 some stop- gap measure that would be usable for our purposes, even
> > if not completely ideal?
> 
> Today it should be implementable and not even hard at all.

Well, I am glad to hear that, at least :)

> Can you please provide a sample use case where:
> 
> a) You describe a simple application with several plugins.
> 
>    How do they interact (what calls what, how are they mounted)
>   
> b) For each plugin and the application what templates does it
>    render
> 
> c) what content does they uses.
> 
> d) How dynamic do you want it to be (when loaded, unloaded, etc)
> 

To complete the simple example I gave at the top, here is what I have and what 
I want.

- I have a single cppcms application, which is the core of the application.

- Core defines a polymorphic plugins class which is NOT a CPPCMS related class.

- A couple of key methods in that class are pure virtual methods.

- Each plugin is expected to implement a derived class, obviously implemeting 
the pure virtual methods of the base class, together with any other methods 
the plugin might need.

- I have implemented my own loading/unloading of plugins at runtime, according 
to values saved in a database, as set the administrator (basically, there is a 
plugin administration form, and the administrator chooses which plugins he 
wants enabled, and the core system saves the values and load/unload the 
plugins accordingly).

All of the above work well, although I'm sure you're going to tell me I was 
not supposed to implement it this way!! :-s

- I have a single skin, the base of which is defined by core.

- Core is controlling the general layout of the page.

- Each plugin is supposed to define a variable number of blocks of content to 
insert into the skin, according to templates defined in plugins, and compiled 
at the same time.

Now, let me rephrase my earlier simple example, to make it look more like it 
should in a real case scenario.

The HTML would look this way:

<!DOCTYPE etc... >
<html>
<head>
	title, metas, links here.
</head>
<body>
	<div id="content_of_block_1"> (content from plugin 1 here) </div>
	<div id="content_of_block_2"> (content from plugin 2 here) </div>
	<div id="content_of_block_3"> (content from plugin 3 here) </div>
	<div id="content_of_block_4"> (content from plugin 4 here) </div>
        ....
	<div id="content_of_block_n"> (content from plugin n here) </div>
</body>
</html>

It's obviously schematized, but it should give you the main idea.

The core template would look somehow like this:

<% c++ #include "content.hpp" %>
<% xhtml %>
<% skin my_skin %>
<% view page uses content::page %>
<% template render() %>
<html>
<head>
	title, metas, links here.
</head>
<body>
	<% foreach block in content::blocks %>
                <% item %>
                        <div id="conent_of_block_<%= block.id %>">
                                <%= block.content %>
                        </div>
                <% end %>
	<%end %>
</body>
</html>
<% end template %>
<% end view %>
<% end skin %>

where obviously, each block.content is defined in its own template at the 
plugin level, as stated above:

<% skin my_skin %>
<%view example_page uses content::example_page extends main %>
<% template main_content() %>
        <p>This is the content output by this plugin.</p>
        <%= content | raw %>
<% end template %>
<% end view %>
<% end skin %>

If I can achieve the end result above, then I would have passed a very major 
hurdle.

Sorry for being slow to catch up with you, and for the length of this email.

Thanks, as always,

Augustin.

-- 
Friends: http://www.reuniting.info/
My projects:
http://astralcity.org/ http://3enjeux.overshoot.tv/ http://linux.overshoot.tv/ 
http://overshoot.tv/ http://charityware.info/ http://masquilier.org/
http://openteacher.info/ http://minguo.info/ 
http://www.wechange.org/ http://searching911.info/

.