Re: [Exist-development] [Exist-commits] Context attributes....please vote!

[Andrzej J. T. <an...@ch...>]

Adam:

> It breaks then tenement of side-effect free XQuery functions.
> Admittedly we have some of these, but they are extension functions and
> very specialised.

Actually, we have a lot of these:  session, request, sql, ldap, email, cache and more!

So we've really moved away from the religiously functional world, and I see that as a good thing, since real-world
applications typically need to integrated with non-functional real-world systems like servlets, sql database, ldap
directories, email and more.

That is a far cry from "some of these". Obviously there is a requirement for some side-effect generating, non-functional
features in eXist. Quite a few of them in fact. ;-)

> The problem was that I had not really discarded the shackles of
> procedural programming and not yet really *really* got my head around
> functional programming elegance. I now believe that such a facility is
> completely unnecessary once you learn the tricks of XQuery programming
> - it does take time though.

You're getting borderline preachy, Adam! ;-)  I don't want to get into a religious war on the benefits of being purely
functional, since we're far from that already and I have no interest in an academic argument.

I know the tricks and value of functional programming....but sometimes the real world intrudes.

> Perhaps you can convince me that such a functionality is really
> nessecary, but I would need to understand the use case and why that
> cannot be met currently.

It's not strictly necessary...more a performance/convenience issue.

I'm happy to remove the two get/set functions from trunk, if the others vote that this should be the way we go. My code
runs fine by using in-memory fragment creation...it's just slow and a bit convoluted to do that, since I have to create
intermediate xml fragments that aggregate multiple xml results so that they can be passed around.  No big deal there
though, since it does work fine.

I may consider putting the functions into a private extension library though, if it buys me enough
performance/convenience at the moment, given the state of in-memory fragments.

> The request attributes are only designed for setting attributes on the
> underlying HttpServletRequest, they were never added for the purpose
> of storing and reusing data within a single XQuery. Neither should
> they be used for transporting data between XQueries. They are really
> for internal interaction of the eXist pipeline, we should come up with
> a better mechanism in future as this ties us too tightly to the Java
> Http Servlet specification.

I don't buy this argument at all. XQuery is a beautiful tool for writing complete, self-contained web applications that
run in a servlet container. Heck...we even support REST that way. To do that you need session-based persistence for
things like authentication and the like.

Regardless of the pragmatic vs theoretical arguments, which are likely to just start a religious war, the cat is out of
the bag.  These features are heavily used in production systems, and so we need to keep them around.

As for what they were intended for, that's irrelevant. If it's available users will use the features....whether they
have side-effects or not.

> The session attributes are just that, designed for persisting (ideally
> small) amounts of data across a users requests during their session.
> Again this should not be used for inter-xquery communication. Ideally
> sessions should be avoided where possible, this leads to greater
> scalability possibilities - there are plenty of articles about
> developing stateless web applications so I wont get into a discussion
> here.

Should, could, would!  If it's there, it will be used....and "abused" in some people's minds.  That's the reality.

> You should not misuse the request or session objects even if they are available.

If all you have is a hammer...everything looks like a nail! LOL

> I dont see why this would become a requirement then, you should still
> be passing data into your XQuery using a different mechanism not the
> context. Each executing instance of an XQuery typically has its own
> context and in a multi-threaded environment this is no different.

Good point.

> It sounds like you have a lot of complexity and issues here that need
> to be broken down into smaller problems and modularised. Perhaps if
> you could give a small specific example of why you need such context
> functionality then I could better understand your exacting
> requirements?

Time demands preclude me from stripping down many thousands of lines of code to create a demonstration case.
Sorry....that's my reality right now. Too many real-world deadlines.

As for the complexity and such, I've already refactored the code into much smaller, more maintainable, decoupled,
independent units as I had described, using a dispatcher paradigm. This has greatly simplified our code.

Don't forget, Adam, I've been writing code for nearly 4 decades, and contrary to some opinions, I do know what I'm doing
and have a long history of writing extremely modular/maintainable code!  Preaching to the converted ain't gonna win you
many points with me. ;-)

> Actually I think such a feature makes code much dirtyier as you cant
> see the flow of data through the system. You just magically stuff
> something into the context using some sort of label and then later
> extract it (assuming its still there). This is very bad in any
> language. 

Tell that to Google, who use distributed hashmaps all over the place (Map/Reduce is based on the concept). Same with
Hadoop.  memcached.  This is a proven, performant design pattern, that unfortunately does not fit nicely into the purist
functional approach.

I don't believe it is a bad thing, if used appropriately. In such cases it can clarify the data flows and make code simpler.

It's all down to the skill/expertise of the individual, and how "good" their code is. That is independent of functional,
distributed hashmap, procedural, declarative or any other approach used.  Purist approaches rarely succeed in the real
world.  Erlang and Lisp come to mind as good examples. LOL

> When I got rid of all of this functionality from a large
> system that I developed in XQuery, the code actually became much
> simpler, and more readable through the extensive refactoring that I
> performed.

That wouldn't be the case with my code. One day, I'll sit down with you and show you what we've got...when time allows.

> Well if you cant show us a concrete example then we definitely cant help you ;-)

True enough. Not that I don't want to...just that there isn't enough time at the moment.

> Perhaps you could show us something, a small exacting example, and we
> could help you refactor it? If we cant help you refactor it, then it
> shows that your use case is valid and that we do need a feature such
> as this.

How about I do some benchmarks and see if the performance gain over building large aggregate in-memory fragments is
worth the compromise? ;-)

The key point is that I am happy to remove the functionality, if the group as a whole votes that way.

Cast yer votes, maties!

Alternatives:

1) Leave it in and see if the world collapses
2) Remove it completely
3) Move it to it's own extension module.

My vote is for option #3.



-- 
Andrzej Taramina
Chaeron Corporation: Enterprise System Solutions
http://www.chaeron.com