Re: [Cppcms-users] template plugins

[Artyom B. <art...@ya...>]

> From: augustin <aug...@ov...>

> 
> In reply both to your December 18th email regarding documentation requests, 
> and to yesterday's email regarding the looming code freeze, I'd like to 
> bring 
> up an issue that has been discussed several times on this list.
> 

Ok

> I know that a plugin framework is planned for CPPCMS 1.1. Can you guesstimate 
> when such a release would take place? One year from now? Two years from 
> now...? Maybe more?
> 

I'll explain.

CppCMS project started about 4 years ago and 3 years ago the first stable 
version was released.

However it was soon become clear that in order to move forward major
deep internal changes should be done:

- Redesign concurrency model - which brings COMET support
- Redesign API they way it would be much easier to maintain and
  it would be able to keep backward compatibility
- Redesign 1001 other things like dropping CgiCC and Boost dependency

And that was CppCMS 1.x.x that went to Beta stage as 0.99.1 in June 2010. It required
a lots of work and thanks to this it become what it is today a popular
C++ web development framework.

The long period (year and a half) of beta testing was required for two things:

1. Make sure that the API is clear stable and fits the needs. This allowed
   me to change the API if I felt it was not good, for example:

   - Full rewrite of templates loading API (0.99.11)
     which by the way would make modules generation much
     easier.

   - XSS prevention (0.99.10)
   - Performance improvements in JSON handling (0.99.9)
   - Localization API redesign (0.99.8)
   
   And more...

2. Once stable version is released I would not 
   have a privilege to change API in backward incompatible way.


That is why there was a long beta testing - the testing of the API.
From all other points view the framework was stable, tested actively,
released frequently.

So once 1.0.0 is released there would be immediately created a new
beta version 1.1 that would receive new features and once it is stabilized
it would be released as stable version 1.2 while 1.0.0 would receive
only bug fix updates (1.0.1, 1.0.2 etc). 

So the release frequency would be high and plugin support would
be scheduled according to its priority (which is high BTW)

So 1.1 and 1.2 releases would be **very** different from 0.0.x -> 1.0.0
because that was almost complete refactoring of the code base that
had bring CppCMS to what it is today, while 1.0 -> 1.2 is just
a new features that are introduced to the framework.

So you should not worry about "keeping CppCMS" unmodified for years

> There seems to be a fairly large demand for a plugin framework. Several people 
> have already mentioned it on this list. Fortunately, we can roll out our own 
> without waiting for CPPCMS 1.1. However, I counted at least three different 
> people, maybe four, who brought the same issue over the last 12~18 months, and 
> I would be the fifth person to do so.
> 

Small notes. Today CppCMS provides all the basics for support of plugins.

Templates are already fully pluggable, load shared object and you
get your template in place.

Also there is a new booster::shared_object class that allows to load
dynamic libraries easily.

On the other hand if I spend now lots of time today on plugin framework
and tomorrow on other very important feature I'll never release a stable
version and this is not good for the project. It was in beta stage for
too long time.


> It seems that independently from each other, we all were confronted with the 
> same issue, namely: how does a plugin register a single part of a larger 
> template which is implemented by the core application?
> 

Simple:

Each template shared object has global constructor and destructor.
Once shared object is loaded, constructor is called and it registers
itself in cppcms::views::pool singleton and become visible for everybody.
When it is unloaded it is unregistered via global destructor and can't
be used any more. It is simple and transparent semantics.



> Let's say we have a core application which implements a common template 
> (skin). And we have several plugins, each implementing a functionality, e.g. a 
> block of content with its own specific HTML markup (sub-template). The core of 
> the application does not know in advance what additional sub-templates the 
> plugins will implement, and plugins do not know what other plugins need.
> Basically, what each of us were trying to do is to 'attach' a 
> sub-template to the core main template. And we all hit the problem that you don't offer an 
> API to do so.


If the plugin is linked with the template code it would loads its own
template:

libmy_plugin.so
   my_application.cpp
      // my code 
      {
         ...
         render("my_plugin_skin","some_view",some_content);
      }
   my_view.cpp <- my_view.tmpl
   // this code is auto-generated
     namespace { 
       struct loader { 
         loader() { cppcms::views::pool::instance().add(my_skin_generator()); }
         ~loader() { cppcms::views::pool::instance().remove(my_skin_generator()); }
       } loader_instance;
     }
    
So once you load shared object loader::loader() is executed
and registers a view
and when you unload the shared object loader::~loader() 
unregisters it.


Only thing you should care is to make sure each as its own skin name and
not collide with its other.

> 
> We can manually create a header file for the sub .tmpl file (part of the 
> plugin), so that the template at least compiles, but then the core application 
> is not aware of it and doesn't use it.
> 

I don't really understand what are you trying to say in the paragraph above.

> I didn't fully understand all of your answers, though I tried and read them 
> many times. From what I understood, your answer was two-fold:
> 
> 1) On one hand you say that the plugin can recompile the whole skin and load 
> it at run time. Maybe there is a need to properly document this because I 
> didn't fully understand how to do so, and how a plugin would be aware of all 
> 
> the other sub-templates implemented by the other plugins.

See the example above. Also plugin should not "recompile" the skin
in the run time... It can do it you need to implement "recompilation"
on your own.

You are probably misguided by the "automatic skin" reload support that
can be used for development. Where setting an option "reload" would
check if the shared object had changed and reload it.

> 
> 2) On the other hand, you seemed to say that it was not possible, because it 
> was difficult to do it right, especially considering race conditions in a multi-
> threaded web server, leading to a crash when one thread is trying to render a 
> sub-template from a plugin that was just disabled in another thread.
> 

The cppcms::views::pool::render(...) is thread safe
as it holds shared lock during any rendering process. If you
would try to unload a skin during this period you would be blocked
till **all** rendering operations will complete and
then you'll be able to load/unload the view

However if you want to load/unload **application** classes
it is totally different story... I there is no support
of dynamic **unloading** of CppCMS applications, only templates.

> So, my request is thus:
> 
> a) If what we are trying to implement is fully possible today, can the proper 
> solution be documented?
> 

Yes... indeed. I think this e-mail content should go to the wiki.
(after some notes below)

> b) If not, some projects will not be able to wait until the release of CPPCMS 
> 1.1 (see questions at the top). Could you at least implement in 1.0 some stop-
> gap measure that would be usable for our purposes, even if not completely 
> ideal?
> 

Today it should be implementable and not even hard at all.

> Let me know if you want me to dig up the discussion threads where those topics 
> have already been raised.


Can you please provide a sample use case where:

a) You describe a simple application with several plugins.

   How do they interact (what calls what, how are they mounted)
   
b) For each plugin and the application what templates does it
   render

c) what content does they uses.

d) How dynamic do you want it to be (when loaded, unloaded, etc)


I'll show how and what could be done today and would is
not possible. And you  could write down
it in the wiki :-) till I get more time to implement
plugin framework.

> Thanks,
> 
> Augustin.
> 
> 

Regards,

Artyom