openinteract-help

AW: [Openinteract-help] Bulk SPOPS security checks?

[<And...@Be...>]

Hi Simon,

This is a problem, which we tackled with the following 2 methods:

- using security with skip_security
This one sounds weird, but it works. The idea is, to issue the =
fetch_group
with the skip_security=3D1 parameter and instead join in the security =
table
with the appropriate object_class and object_id params. Here, only one
select is done and only one fetch. Works like it should

- using handler security and a config table
Here, we are only using SPOPS for security to the handler, not the
application data . This is then done within the app itself. To make =
this
easy to configure, we sometimes use a configuration table ( limited =
amount
of entries ), with standard SPOPS security on. Before the select on the
application data is issued, you make a standard fetch_group on the =
config
table and get the information you need to fetch only the data from the
application data tables, which the user may see.

In general, SPOPS is great for small tables, but the more data and the =
more
groups you have, it finds breaks performance in standard mode.

Both methods described above are performant and stable. Method 2 is =
cleaner
and should be preferred, if possible.

Later,

Andreas

-----Urspr=FCngliche Nachricht-----
Von: Chris Winters [mailto:ch...@cw...]=20
Gesendet: Dienstag, 11. November 2003 00:19
An: Vsevolod (Simon) Ilyushchenko
Cc: ope...@li...
Betreff: Re: [Openinteract-help] Bulk SPOPS security checks?


* Vsevolod (Simon) Ilyushchenko (si...@cs...) [031110 17:59]:
> I noticed that SPOPS security is invoked by retrieving the object=20
> first
> and checking its security later. Is there a way to shift the security =

> checks down to the SQL level so that only objects readable by the=20
> current user are selected from the database?

The folks at bertelsmann implemented something like this. Andreas?

Chris

--=20
Chris Winters (ch...@cw...)
Building enterprise-capable snack solutions since 1988


-------------------------------------------------------
This SF.Net email sponsored by: ApacheCon 2003,
16-19 November in Las Vegas. Learn firsthand the latest developments in
Apache, PHP, Perl, XML, Java, MySQL, WebDAV, and more!
http://www.apachecon.com/ =
_______________________________________________
openinteract-help mailing list ope...@li...
https://lists.sourceforge.net/lists/listinfo/openinteract-help

Re: [Openinteract-help] Bulk SPOPS security checks?

[Teemu A. <te...@io...>]

> Both methods described above are performant and stable. Method 2 is
> cleaner and should be preferred, if possible.

Hi Andreas,

do you have any other good advices based on your experience how to make
OpenInteract+SPOPS perform better? How many concurrent users you have run
on a single server without problems?
For example, if I run several websites, should I use a separate apache
process for each one of them or should I run them on the same server with
the stash trick?
I'm building a quite heavy application with OI and I'm a little bit
concerned about the speed when we have something like 100 concurrent
users.
-- 
Sincerely,

Teemu Arina

Ionstream Oy / Dicole
Komeetankuja 4 A
02210 Espoo
FINLAND
Tel: +358-(0)50 - 555 7636
http://www.dicole.fi
http://www.mimerdesk.org

Re: [Openinteract-help] Bulk SPOPS security checks?

[Andreas N. <an...@kl...>]

On Tue, 2003-11-18 at 13:03, Teemu Arina wrote:
> > Both methods described above are performant and stable. Method 2 is
> > cleaner and should be preferred, if possible.
> 
> Hi Andreas,
> 
> do you have any other good advices based on your experience how to make
> OpenInteract+SPOPS perform better? How many concurrent users you have run
> on a single server without problems?
Hi Teemu, 

we use a cluster of two dual athlon servers for the webservers ( which
do loadleveling), with one large mysql for most databases, but also a
Netware cluster for LDAP and fileservices ( the HTML and perl files are
pulled in via NFS ) and a few oracle and ms-sql databases. This serves
up 1000 Intranet users. But this is not a regular internet website type
of usage, but at least 200 users are using this as their main callcenter
application, so the OI applications are rather heavy weight.
> For example, if I run several websites, should I use a separate apache
> process for each one of them or should I run them on the same server with
> the stash trick?
You should separate them, I guess. If this is smallscale stuff, then
keep the spawn server settings etc. low. 

Remember the most important tuning rule here: keep your systems from
swapping ! ( under Linux install sysstat package, have sac runnig and
use sar -r, here you should never see more then a few % of swapping. If
you really want to make it fast, keep the system from swapping more than
1 % constantly - 30/40 % constant swapping is already overkill ! )
> I'm building a quite heavy application with OI and I'm a little bit
> concerned about the speed when we have something like 100 concurrent
> users.
You should not worry about this ! ( If you are not using a 486-SX 25 MHz
;-) )

Since I am curious: what type of app are you thinking about ? 

later,

Andreas

PS: this is my private eMail, the other one is my business eMail..

Re: [Openinteract-help] Bulk SPOPS security checks?

[Teemu <te...@io...>]

Dear Andreas,

> we use a cluster of two dual athlon servers for the webservers ( which
> do loadleveling), with one large mysql for most databases, but also a
> Netware cluster for LDAP and fileservices ( the HTML and perl files are
> pulled in via NFS ) and a few oracle and ms-sql databases. This serves
> up 1000 Intranet users. But this is not a regular internet website type
> of usage, but at least 200 users are using this as their main callcenter
> application, so the OI applications are rather heavy weight.

Ok, good to hear. I was thinking about a three web server cluster connected 
with a 1GB lan together, where server 1 operates as a lightweight reverse 
proxy / static webserver / buffer for dynamic content, server 2 with lots of 
memory serving dynamic content from the OI backend and server 3 as the mysql 
database server with fast IO access. If I need load balancing I could add 
more servers that generate dynamic content. 

I'm going to run several these kind of setups serving each up to 3000 users.

> Remember the most important tuning rule here: keep your systems from
> swapping ! ( under Linux install sysstat package, have sac runnig and
> use sar -r, here you should never see more then a few % of swapping. If
> you really want to make it fast, keep the system from swapping more than
> 1 % constantly - 30/40 % constant swapping is already overkill ! )

Good to know. I was also thinking about using a fast journaling filesystem for 
databases and cache. Also there are performance reasons why to use the 
reverse proxy strategy described above.

> You should not worry about this ! ( If you are not using a 486-SX 25 MHz
> ;-) )

More like 2.6 GHz P4 =) The speed problems I have had so far are about SPOPS 
security checking when fetching lots of data and from several different 
tables. Turning off object security and using handler security instead sure 
speeds things a lot.

> Since I am curious: what type of app are you thinking about ?

What I have built so far is a school (student) management system. Some 
screenshots here (shots in finnish, sorry): http://www.mimerdesk.org/RX/oi/

The next application I'm building at the moment is a collaborative learning 
environment. You could compare such systems to groupware solutions to get an 
idea how heavy the application will be.

These applications will be available under the GPL once we have the first 
public versions ready.

Special technical details are the XHTML interface with complete layout control 
through CSS and modified OI themes. The templates we created for the XHTML 
are very generic and allow object oriented approach to web interface design 
(especially useful with some programmers who don't understand how to create 
good user interfaces =) ). We have built an object oriented "Tool" library 
which in the end outputs data structures to be passed for template toolkit, 
which produces the resulting XHTML pages. The system allows creating web 
pages through an object oriented interface similar to Tk/GTK etc. you use to 
build desktop applications. No HTML or anything involved in creating the 
interface.

The benefit in the above example (see link) is that we didn't have to write 
templates for any of the applications we created. We programmers used the 
code to generate tools, the designers optimized the presentation. Everything 
looks uniform. Once we had the interface framework ready we didn't have to 
control layout at all or touch the templates. It was pretty nice to write a 
new application and when you call $R->template->handler that's it. No 
presentation to worry about.

The disadvantage was that the number of TT templates used on each page was 
relatively big (average was something like 100), so generating the page had a 
slight performance hit despite the TT precompiling abilities. We decided to 
heavily use OI page caching features. Still miss the ability to clear cache 
for a different handler output.

Another thing we are working on is a localization system that allows 
translations to be stored separately. We probably use the standard unix 
gettext and .po files for translations and hook that into OpenInteract.

-- 
Sincerely,

Teemu Arina

Ionstream Oy / Dicole
Komeetankuja 4 A
02210 Espoo
FINLAND
Tel: +358-(0)50 - 555 7636
http://www.dicole.fi
http://www.palaveriaika.com

Re: [Openinteract-help] Bulk SPOPS security checks?

[Andreas N. <an...@kl...>]

> Ok, good to hear. I was thinking about a three web server cluster connected 
> with a 1GB lan together, where server 1 operates as a lightweight reverse 
> proxy / static webserver / buffer for dynamic content, server 2 with lots of 
> memory serving dynamic content from the OI backend and server 3 as the mysql 
> database server with fast IO access. If I need load balancing I could add 
> more servers that generate dynamic content. 
This sounds reasonable. We use a setup, where both front and backend
httpd are on the same host, because we thought the local throughput
might be worth it. Also, since we serve multiple applications on this
site, we wanted to be able to route some apps to special servers for
processing - but this was never needed, except for debugging. We had
some problems on the Oracle side ( now we replace Oracle ;-) )
> 
> I'm going to run several these kind of setups serving each up to 3000 users.
Whoa - this is interesting and like lots of fun ;-)

...
> More like 2.6 GHz P4 =) The speed problems I have had so far are about SPOPS 
> security checking when fetching lots of data and from several different 
> tables. Turning off object security and using handler security instead sure 
> speeds things a lot.
Did you already try? 
> 
> > Since I am curious: what type of app are you thinking about ?
> 
> What I have built so far is a school (student) management system. Some 
> screenshots here (shots in finnish, sorry): http://www.mimerdesk.org/RX/oi/
Although I did not understand a word ( only finish I know is from my
stay in the US with 16 years of age, where a girl from Pori tought me
minarakastanasinua ). We are thinking about a callcenter staffing tool (
only prototypes so far ), and this looks similar, but grown up.
> 
> The next application I'm building at the moment is a collaborative learning 
> environment. You could compare such systems to groupware solutions to get an 
> idea how heavy the application will be.
Sounds really interesting.
> 
> These applications will be available under the GPL once we have the first 
> public versions ready.
even better. We have problems making documentation good enough to
publish some apps - time is to scarce...
> 
> Special technical details are the XHTML interface with complete layout control 
> through CSS and modified OI themes. The templates we created for the XHTML 
> are very generic and allow object oriented approach to web interface design 
> (especially useful with some programmers who don't understand how to create 
> good user interfaces =) ). We have built an object oriented "Tool" library 
> which in the end outputs data structures to be passed for template toolkit, 
> which produces the resulting XHTML pages. The system allows creating web 
> pages through an object oriented interface similar to Tk/GTK etc. you use to 
> build desktop applications. No HTML or anything involved in creating the 
> interface.
I`d like to see the results - sounds like a solid promising approach !
> 
> The benefit in the above example (see link) is that we didn't have to write 
> templates for any of the applications we created. We programmers used the 
> code to generate tools, the designers optimized the presentation. Everything 
> looks uniform. Once we had the interface framework ready we didn't have to 
> control layout at all or touch the templates. It was pretty nice to write a 
> new application and when you call $R->template->handler that's it. No 
> presentation to worry about.
I like code that can be deleted, because some intelligent code does
things behind the scenes!
> 
> The disadvantage was that the number of TT templates used on each page was 
> relatively big (average was something like 100), so generating the page had a 
> slight performance hit despite the TT precompiling abilities. We decided to 
> heavily use OI page caching features. Still miss the ability to clear cache 
> for a different handler output.
If you have enough RAM: the amount of cached TTs is tunable somewhere in
the code. I think it was in OpenInteract::Template
> 
> Another thing we are working on is a localization system that allows 
> translations to be stored separately. We probably use the standard unix 
> gettext and .po files for translations and hook that into OpenInteract.

Just a note: there are ideas in OI2 for this.

Re: [Openinteract-help] Bulk SPOPS security checks?

[Andreas N. <an...@kl...>]

> > Since I am curious: what type of app are you thinking about ?
> 
> What I have built so far is a school (student) management system. Some 
> screenshots here (shots in finnish, sorry): http://www.mimerdesk.org/RX/oi/
> 
> The next application I'm building at the moment is a collaborative learning 
> environment. You could compare such systems to groupware solutions to get an 
> idea how heavy the application will be.
.. are you talking about porting mimerdesk to OI??

Re: [Openinteract-help] Bulk SPOPS security checks?

[Teemu <te...@io...>]

> .. are you talking about porting mimerdesk to OI??

Not only porting, but creating something new on top of it in the same time. It 
will have additional functionality necessary in learning organizations and 
education.

For a long time I have searched for time to get rid of current MimerDesk code 
base (which is scarce, complicated and has its' limits) to replace it with 
something better. OI and additional time (money) for open source development 
came to rescue. Now I also have better possibilities to put my improved OO 
programming kung fu in practice..

-- 
Sincerely,

Teemu Arina

Ionstream Oy / Dicole
Komeetankuja 4 A
02210 Espoo
FINLAND
Tel: +358-(0)50 - 555 7636
http://www.dicole.fi

Re: [Openinteract-help] Bulk SPOPS security checks?

[Andreas N. <an...@kl...>]

On Tue, 2003-11-18 at 23:39, Teemu wrote:
> > .. are you talking about porting mimerdesk to OI??
> 
> Not only porting, but creating something new on top of it in the same time. It 
> will have additional functionality necessary in learning organizations and 
> education.
Today I had more time to look at it and the demo and this is really
great ! I see, you aim at the educational area, but some tools like
projects and calendar definetly have a broader usage. Acutally, you
obviously implemented quite a bit, that we are also thinking about!
Great! Also, I talked to a colleague, who does consultancy for public
schools in Germany. He said, that many larger schools are looking for
open source tools for such tasks and that there are funds for this (
called N21 ). We will check this out...

Apart from this, we have our own fund for our tools and would like to
cooperate, if this is imaginable. Our focus in the last months was to
work on a configurable workflow framework, we call NATS. As soon as we
have a sample app and more documentation, we will release this to the
open source. Looking at mimer desk, this could be hiden behind the Tasks
screen and pull in all types of tasks in the workflow?

What do you think ?
> 
> For a long time I have searched for time to get rid of current MimerDesk code 
> base (which is scarce, complicated and has its' limits) to replace it with 
> something better. OI and additional time (money) for open source development 
> came to rescue. Now I also have better possibilities to put my improved OO 
> programming kung fu in practice..
This is the key solution ! When I stumbled over OI two years ago, I was
in the middle of thinking about a self written framework, because at
that time there was really nothing you could work with - all OO Perl
things needed to be done from scratch.
Looking back, we definetly made the right decision to take OI. I am
really convinced, that there is no other tool, that can make you just as
productive - regardless of dev language.
Our first tries had there problems, of course, one of which was the
misuse of SPOPS. Also, at that time there was no
OpenInteract::CommonHandler, which is really the way to go for most
things. Then we put to much functionality in the handler packages, which
is not a good idea. Today we build separete classes that ( most of the
times ) inherit from SPOPS and put the functionality in them. This on
one side lets you use the functionality in scripts without a GUI, but
also gives you the full power directly within the templates !

later,

Andreas

Re: [Openinteract-help] Bulk SPOPS security checks?

[Chris W. <ch...@cw...>]

Andreas Nolte wrote:
 >...
> is not a good idea. Today we build separete classes that ( most of the
> times ) inherit from SPOPS and put the functionality in them. This on
> one side lets you use the functionality in scripts without a GUI, but
> also gives you the full power directly within the templates !

A slight sidenote here: I think this pattern is one I'm going to 
encourage much more explicitly in OI2. That is, something like:

# in spops.ini
----------------------------------------
[myobject]

# This is the generated class
class       = My::ObjectPersist

# This is what OI2 will return on a call of:
# my $object_class = CTX->lookup_object( 'myobject' );
alias_class = My::Object
...
----------------------------------------

And then you'd create:

----------------------------------------
package My::Object;

use strict;
@My::Object::ISA = qw( My::ObjectPersist );

sub my_custom_functionality {
     my ( $self ) = @_;
     $self->{my_field} = 'foo';
     $self->save;
}
...
----------------------------------------

This is in-place of using the 'code_class' element in the SPOPS 
configuration, which I've always considered very hackish. This 
also fits in with the philosophy that you can just add in 
persistence behavior to an existing object, implementing your 
normal object behavior (plus any additional persistence behavior 
like canned queries) in code and your persistence in configuration.

Chris

-- 
Chris Winters (ch...@cw...)
Building enterprise-capable snack solutions since 1988.