exist-development — eXist Developer's List

Re: [Exist-development] New list member offering services :)

[Wolfgang M. <wol...@ex...>]

Hi Joe,

> Even though I learned XSLT first before XQuery, I find it much easier
> to maintain XQuery than XSLT stylesheets, since that's what I work in
> 100% of the time now.  I converted a TEI-to-XHTML stylesheet from XSLT
> (65 templates) to XQuery (41 functions, including the master
> typeswitch) routine.

This is interesting to hear and confirms Dan's observations with which
I tend to agree (though I enjoy writing XSLT as well, but I probably
have enough training in it).

> Another question that has been raised is the DocBook 4 to 5
> transition.  I'm not familiar with DocBook, but given the benefits of
> DocBook 5 and using XQuery for transformation, how disruptive would it
> be to do *both* at once?  (1) Convert the eXist documentation to
> DocBook 5, and (2) convert the stylesheet from DocBook 4-oriented XSLT
> to DocBook 5-oriented XQuery?

This will not be a major effort. Most of the documentation only uses a
subset of docbook which should be easy to convert (maybe even
automatically, using an XSL?).

> Another factor to consider is that I count 23 XSL files in the
> "stylesheets" directory.  Should all of these be converted, or only
> "db2xhtml.xsl"?  If not all 23 need to be converted, which are the key
> ones?

db2xhtml.xsl is the only important one. It is used by nearly all pages
on the website. The others are either old (left over from Cocoon) or
only used for a single purpose (roadmap.xsl). One or two documents
(index.xml) are still using an old stylesheet (doc2html.xsl) and need
to be migrated to docbook.

Wolfgang

Re: [Exist-development] New list member offering services :)

[Wolfgang M. <wol...@ex...>]

> I dont think that doing basic document to document conversion in
> XQuery makes any sense, XSLT is easier to develop this in and more
> intuitive.

I think this is a misunderstanding. A simple document to document
transformation can sometimes be even more intuitive in XQuery ;-) All
you need to learn is typeswitch plus a few function calls. I'm using
this more and more in my own code (for example, see
http://exist.svn.sourceforge.net/viewvc/exist/trunk/eXist/webapp/biblio/jquery.xql?revision=11607&view=markup).
Done in the right way, the control flow in an XQuery can be easier to
understand and maintain than the template matching mechanism of XSLT.

Don't misunderstand me. I like XSLT. I use a pretty large XSLT to
generate multi-volume, 1000 pages PDFs from a set of TEI documents
stored in eXist. The processing only takes a few minutes, despite the
thousands of small queries it sends to eXist, e.g. to assemble
bibliographies and SVG graphics. However, this also shows the
limitations of XSLT: it becomes quite tricky once you need to
integrate data from many sources. Native XSLT support in eXist would
overcome this limit. It's on the way, but there's still a lot of work
left to be done.

Anyway, my main argument for using more XQuery for the website is that
you want to keep the examples as simple as possible and reduce the
number of standards new users have to learn.

Wolfgang

[Exist-development] demo.exist-db.org down?

[Joe W. <jo...@gm...>]

FYI

http://demo.exist-db.org/exist/

503: Service Temporarily Unavailable

The server is temporarily unable to service your request due to
maintenance downtime or capacity problems. Please try again later.
Apache/2.2.9 (Debian) DAV/2 SVN/1.5.1 mod_gnutls/0.5.1 mod_jk/1.2.26
PHP/5.2.6-1+lenny8 with Suhosin-Patch proxy_html/3.0.0
mod_python/3.3.1 Python/2.5.2 mod_ruby/1.2.6 Ruby/1.8.7(2008-08-11)
mod_ssl/2.2.9 OpenSSL/0.9.8g mod_perl/2.0.4 Perl/v5.10.0 Server at
demo.exist-db.org Port 80

Re: [Exist-development] New list member offering services :)

[Loren C. <lor...@gm...>]

I think that we should be converting over to DocBook 5.  We can keep the current 67 documents where they are until we have the new style for the documentation developed and implemented.

We should be developing new documentation using DocBook 5 and developing an organization of the information and a style guide to create a standardized "look and feel" to the documentation.  Consistency makes things easier to read.  The current documentation would then be converted into the new format manually.  The current documentation can be converted from 4 to 5 and then the contents can be manually entered into the new system.

I like the use of XQuery for the presentation.  If we store the sections as separate files then we can use the Physical Divisions capability of DocBook http://www.docbook.org/tdg5/en/html/ch02.html#ch02-physdiv to assemble them.

We can avoid some of the redundancy in the documents:



On installation of eXist, the sections can be loaded (as an installation option) into the database under some collection like /system/docbook or /system/doc.  Once the system is up and running, then there would be an administrative task to go find updates on the http://demo.exist-db.org server and download the updates.

With this approach, we can also have an option that extracts all of the contents under a section into a book and then run XSL FO (on demand) to create a PDF, eBook, etc.


On Apr 9, 2010, at 01:33 PM, Joe Wicentowski wrote:

> Hi Wolfgang, Tony, Martin, Dan,
> 
> Even though I learned XSLT first before XQuery, I find it much easier
> to maintain XQuery than XSLT stylesheets, since that's what I work in
> 100% of the time now.  I converted a TEI-to-XHTML stylesheet from XSLT
> (65 templates) to XQuery (41 functions, including the master
> typeswitch) routine.  I am now much happier than before, when I'd have
> to "relearn" XSLT any time I wanted to make a change.
> 
> One "valid" reason to use XSLT, of course, would be if you want to use
> an existing stylesheet or framework out of the box.  For example, the
> TEI Consortium maintains an incredible array of XSLT stylesheets, for
> TEI to XHTML, XSL-FO, docx, LaTeX, slideshows, etc.; I imagine DocBook
> does something similar.  But if you really want to really customize
> your output and you are going to substantially rewrite the stylesheet
> anyway, I agree with Dan that there are real benefits to moving to an
> all-XQuery workflow.
> 
> Another question that has been raised is the DocBook 4 to 5
> transition.  I'm not familiar with DocBook, but given the benefits of
> DocBook 5 and using XQuery for transformation, how disruptive would it
> be to do *both* at once?  (1) Convert the eXist documentation to
> DocBook 5, and (2) convert the stylesheet from DocBook 4-oriented XSLT
> to DocBook 5-oriented XQuery?
> 
> Another factor to consider is that I count 23 XSL files in the
> "stylesheets" directory.  Should all of these be converted, or only
> "db2xhtml.xsl"?  If not all 23 need to be converted, which are the key
> ones?
> 
> Joe
> 
> 
> On Fri, Apr 9, 2010 at 2:03 PM, Wolfgang Meier <wol...@ex...> wrote:
>> Hi Dan,
>> 
>> so you suggest to migrate our current stylesheets to XQuery? I don't
>> really mind since the stylesheets will need to be revised anyway, but
>> it should be discussed first. We should probably choose the approach
>> which is easier to cope with for most people.
>> 
>> Personally, I sometimes prefer XSLT for simple, top-down
>> transformation processes, but well, we may not need to force beginners
>> to learn both, XSLT and XQuery at the same time.
>> 
>> Wolfgang
>> 
>> ------------------------------------------------------------------------------
>> Download Intel&#174; Parallel Studio Eval
>> Try the new software tools for yourself. Speed compiling, find bugs
>> proactively, and fine-tune applications for parallel performance.
>> See why Intel Parallel Studio got high marks during beta.
>> http://p.sf.net/sfu/intel-sw-dev
>> _______________________________________________
>> Exist-development mailing list
>> Exi...@li...
>> https://lists.sourceforge.net/lists/listinfo/exist-development
>> 
> 
> ------------------------------------------------------------------------------
> Download Intel&#174; Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development

Re: [Exist-development] New list member offering services :)

[Adam R. <ad...@ex...>]

> so you suggest to migrate our current stylesheets to XQuery? I don't
> really mind since the stylesheets will need to be revised anyway, but
> it should be discussed first. We should probably choose the approach
> which is easier to cope with for most people.

I dont think that doing basic document to document conversion in
XQuery makes any sense, XSLT is easier to develop this in and more
intuitive. If you wanted to start doing some processing and business
logic as well during the conversion then sure use XQuery. But for
simple conversion XSLT is much easier.

> Personally, I sometimes prefer XSLT for simple, top-down
> transformation processes, but well, we may not need to force beginners
> to learn both, XSLT and XQuery at the same time.

Sure. The XSLT user base out there is much larger than the XQuery user
base in my opinion, I think even most novice XQuery users are likely
to have had some exposure to XSLT.

> Wolfgang
>
> ------------------------------------------------------------------------------
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development
>



-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

Re: [Exist-development] New list member offering services :)

[Adam R. <ad...@ex...>]

> I think the second statement is true.  XQuery is perfect for quickly getting
> the titles from a large collection of indexed XML documents.   But the first
> is somewhat debatable.

I would argue that XSLT is much better suited to transformations than
XQuery. I think both XSLT and XQuery have their place and should be
used when appropriate like any tool. They both have their strengths
and weaknesses and together they are an excellent package.

> My experience is that I can teach people how to do
> an XQuery typeswitch tranforms in about an hour or two.  Most people prefer
> the syntax of XQuery over XSLT.

In my experience the opposite is true, it is much easier to teach some
basic XSLT than XQuery, the templating approach is very natural and
non-programmer types feel much safer writting XSLT as an XML
vocabulary than they do XQuery code. I would also argue that IDE's
with the Schema for XSLT make it much easier to see problems in the
code as you are writting in than with XQuery.




-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

Re: [Exist-development] New list member offering services :)

[Joe W. <jo...@gm...>]

Hi Wolfgang, Tony, Martin, Dan,

Even though I learned XSLT first before XQuery, I find it much easier
to maintain XQuery than XSLT stylesheets, since that's what I work in
100% of the time now.  I converted a TEI-to-XHTML stylesheet from XSLT
(65 templates) to XQuery (41 functions, including the master
typeswitch) routine.  I am now much happier than before, when I'd have
to "relearn" XSLT any time I wanted to make a change.

One "valid" reason to use XSLT, of course, would be if you want to use
an existing stylesheet or framework out of the box.  For example, the
TEI Consortium maintains an incredible array of XSLT stylesheets, for
TEI to XHTML, XSL-FO, docx, LaTeX, slideshows, etc.; I imagine DocBook
does something similar.  But if you really want to really customize
your output and you are going to substantially rewrite the stylesheet
anyway, I agree with Dan that there are real benefits to moving to an
all-XQuery workflow.

Another question that has been raised is the DocBook 4 to 5
transition.  I'm not familiar with DocBook, but given the benefits of
DocBook 5 and using XQuery for transformation, how disruptive would it
be to do *both* at once?  (1) Convert the eXist documentation to
DocBook 5, and (2) convert the stylesheet from DocBook 4-oriented XSLT
to DocBook 5-oriented XQuery?

Another factor to consider is that I count 23 XSL files in the
"stylesheets" directory.  Should all of these be converted, or only
"db2xhtml.xsl"?  If not all 23 need to be converted, which are the key
ones?

Joe


On Fri, Apr 9, 2010 at 2:03 PM, Wolfgang Meier <wol...@ex...> wrote:
> Hi Dan,
>
> so you suggest to migrate our current stylesheets to XQuery? I don't
> really mind since the stylesheets will need to be revised anyway, but
> it should be discussed first. We should probably choose the approach
> which is easier to cope with for most people.
>
> Personally, I sometimes prefer XSLT for simple, top-down
> transformation processes, but well, we may not need to force beginners
> to learn both, XSLT and XQuery at the same time.
>
> Wolfgang
>
> ------------------------------------------------------------------------------
> Download Intel&#174; Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development
>

Re: [Exist-development] New list member offering services :)

[Wolfgang M. <wol...@ex...>]

Hi Dan,

so you suggest to migrate our current stylesheets to XQuery? I don't
really mind since the stylesheets will need to be revised anyway, but
it should be discussed first. We should probably choose the approach
which is easier to cope with for most people.

Personally, I sometimes prefer XSLT for simple, top-down
transformation processes, but well, we may not need to force beginners
to learn both, XSLT and XQuery at the same time.

Wolfgang

Re: [Exist-development] New list member offering services :)

[Tony | \Zearin\ <ze...@us...>]

Hey Dan!


On Apr 9, 2010, at 11:57 AM, Dan McCreary wrote:
> Hello Tony,
>
> Sorry for the delay in getting back to you.  We really welcome any
> help documenting the eXist system, especially for beginners.
>
> > Also, I thought that XSLT was by design better suited to transform
> what is essentially the same document from one format to another,
> while XQuery is preferable for grabbing subsets of data—for example, a
> bug database.  
>
> I think the second statement is true.  XQuery is perfect for quickly
> getting the titles from a large collection of indexed XML documents.  
> But the first is somewhat debatable.  My experience is that I can
> teach people how to do an XQuery typeswitch tranforms in about an hour
> or two.  Most people prefer the syntax of XQuery over XSLT.  In my
> opinion the only reason to write in XSLT is if you must do the
> transforms in a browser.  There are no other valid reason any more to
> continue to use XSLT.  I say this after spending three years doing
> mostly XSLT and attempting to train others.  My work teaching XQuery
> is that it is much easier to learn.  Especially by people that have
> SQL or PHP experience.
>
> < … >


Oh, no problem.  Although, it makes me sad to hear that XSLT is out of
favor now.  I began learning XSL using Saxon (with XSLT 2.0) and
invested plenty of time into it.  I also found it a delightfully strange
and beautiful way to transform data.  Sad to see it fade.  :(

There was one statement you made that has me curious:


> There are no other valid reason any more to continue to use XSLT.


What did you mean by “any more”?  Specifically:

    * So it used to be preferred, but no longer?  
    * What caused this change? 
    * Does this include XSLT 2.0?  (I completely skipped XSLT 1.0 and
      learned on 2.0.  But I have heard that 2.0 made life worlds easier.)


…

I’m just curious—I’m *not* trying to debate anything, or trying to get
you to switch, etc!  :)

Just trying to understand the evolution of the tools and the evolution
of how people use those tools.  


—Tony

Re: [Exist-development] New list member offering services :)

[Tony R. <to...@go...>]

Hey Dan!


On Apr 9, 2010, at 11:57 AM, Dan McCreary wrote:
> Hello Tony,
>
> Sorry for the delay in getting back to you.  We really welcome any
> help documenting the eXist system, especially for beginners.
>
> > Also, I thought that XSLT was by design better suited to transform
> what is essentially the same document from one format to another,
> while XQuery is preferable for grabbing subsets of data—for example, a
> bug database.  
>
> I think the second statement is true.  XQuery is perfect for quickly
> getting the titles from a large collection of indexed XML documents.  
> But the first is somewhat debatable.  My experience is that I can
> teach people how to do an XQuery typeswitch tranforms in about an hour
> or two.  Most people prefer the syntax of XQuery over XSLT.  In my
> opinion the only reason to write in XSLT is if you must do the
> transforms in a browser.  There are no other valid reason any more to
> continue to use XSLT.  I say this after spending three years doing
> mostly XSLT and attempting to train others.  My work teaching XQuery
> is that it is much easier to learn.  Especially by people that have
> SQL or PHP experience.
>
> < … >


Oh, no problem.  Although, it makes me sad to hear that XSLT is out of
favor now.  I began learning XSL using Saxon (with XSLT 2.0) and
invested plenty of time into it.  I also found it a delightfully strange
and beautiful way to transform data.  Sad to see it fade.  :(

There was one statement you made that has me curious:


> There are no other valid reason any more to continue to use XSLT.


What did you mean by “any more”?  Specifically:

    * So it used to be preferred, but no longer?  
    * What caused this change? 
    * Does this include XSLT 2.0?  (I completely skipped XSLT 1.0 and
      learned on 2.0.  But I have heard that 2.0 made life worlds easier.)


…

I’m just curious—I’m *not* trying to debate anything, or trying to get
you to switch, etc!  :)

Just trying to understand the evolution of the tools and the evolution
of how people use those tools.  


—Tony

Re: [Exist-development] Binary resources not deleted when a collection is moved into existing collection with the same name.

[José M. F. G. <jm...@us...>]

Hi Thomas,
	well, (lack of) recursive deletion of binary contents it is a different bug. AFAIHS it is related to the recursive implementation of NativeBroker.removeCollection, RenameBinaryLoggable and usage of File.renameTo. In current implementation, when a collection is erased, "binary" child subcollections are erased before parent ones. But as it is not a real erase, but a directory movement to a subdirectory inside fs.journal, when File.renameTo is called in parent collection for binary contents, it fails because the destination path already exists.

	Attached patch contains both the previous fix for move collection bug, and new fix for remove collection bug. Could you test it, please? If it works for you I'm sending the changes to eXist trunk.

	Best Regards,
		José María

On 04/09/10 10:12, Thomas White wrote:
> José,
> Thank you very much for helping me with this important issue.
> Unfortunately I when a collection with binary resources is deleted the
> files in the file system under webapp\WEB-INF\data\fs\db still remain.
> If you run the example file you can replicate the problem on your box.
> I am sending you the zip file with the test case (it is the same as the
> file I uploaded when I submitted the bug).
> This example only imports binary files. I think we need to fix this
> first. Then we can continue with the collection move.
> Regards,
> Thomas
>
>
> ------
>
> Thomas White
>
> Mobile:+44 7711 922 966
> Skype: thomaswhite
> gTalk: thomas.0007
> Linked-In:http://www.linkedin.com/in/thomaswhite0007
> facebook: http://www.facebook.com/thomas.0007
>
>
>
> 2010/4/8 José María Fernández González <jm...@us...
> <mailto:jm...@us...>>
>
>     Hi Thomas,
>             I have taken a look at the bug you have reported, and
>     following the codepaths the reason is that in NativeBroker, in
>     moveBinaryFork, there is no check about the (lack of existence of)
>     destination at binary resources level, so the move operation fails
>     when there is already a binary resource with the same path. I have
>     created a fix (see attached source patch), and I would like you to
>     test it before I commit the changes to eXist trunk.
>
>             Best Regards,
>                     José María
>
>
>     On 03/31/10 15:45, Thomas White wrote:
>
>         I was wandering if anybody can have a look at this issue.
>         It is pretty serious to me because after a second file import
>         eXist goes
>         out of sync between the binary resource listed internally and
>         the files
>         that actually are stored in the file system at the location they are
>         expected. And second, the fact that files stay in the file system
>         after collections deletions is a security issue as well.
>         I really hope somebody will be able to go to the bottom of this
>         problem.
>         Thomas
>
>         ------
>
>         Thomas White
>
>         Mobile:+44 7711 922 966
>         Skype: thomaswhite
>         gTalk: thomas.0007
>         Linked-In:http://www.linkedin.com/in/thomaswhite0007
>         facebook: http://www.facebook.com/thomas.0007
>
>
>
>         On 25 March 2010 17:28, Thomas White <tho...@gm...
>         <mailto:tho...@gm...>
>         <mailto:tho...@gm... <mailto:tho...@gm...>>>
>         wrote:
>
>             EXIST_HOME  is  D:\eXist2
>             JAVA SDK HOME C:\Java\jdk1.6.0_16
>             Has anyone been able to reproduce this bug?
>             Thomas
>
>
>
>
>         ------------------------------------------------------------------------------
>         Download Intel&#174; Parallel Studio Eval
>         Try the new software tools for yourself. Speed compiling, find bugs
>         proactively, and fine-tune applications for parallel performance.
>         See why Intel Parallel Studio got high marks during beta.
>         http://p.sf.net/sfu/intel-sw-dev
>
>
>
>         _______________________________________________
>         Exist-development mailing list
>         Exi...@li...
>         <mailto:Exi...@li...>
>         https://lists.sourceforge.net/lists/listinfo/exist-development
>
>
>     --
>     "La violencia es el último recurso del incompetente"
>             - Salvor Hardin en "La Fundación" de Isaac Asimov
>     "Premature optimization is the root of all evil." - Donald Knuth
>
>     José María Fernández González
>     e-mail: jos...@gm... <mailto:jos...@gm...>
>
>

-- 
"La violencia es el último recurso del incompetente"
	- Salvor Hardin en "La Fundación" de Isaac Asimov
"Premature optimization is the root of all evil." - Donald Knuth

José María Fernández González
e-mail: jos...@gm...

Re: [Exist-development] New list member offering services :)

[Dan M. <dan...@gm...>]

Hello Tony,

Sorry for the delay in getting back to you.  We really welcome any help
documenting the eXist system, especially for beginners.

> Also, I thought that XSLT was by design better suited to transform what is
essentially the same document from one format to another, while XQuery is
preferable for grabbing subsets of data—for example, a bug database.

I think the second statement is true.  XQuery is perfect for quickly getting
the titles from a large collection of indexed XML documents.   But the first
is somewhat debatable.  My experience is that I can teach people how to do
an XQuery typeswitch tranforms in about an hour or two.  Most people prefer
the syntax of XQuery over XSLT.  In my opinion the only reason to write in
XSLT is if you must do the transforms in a browser.  There are no other
valid reason any more to continue to use XSLT.  I say this after spending
three years doing mostly XSLT and attempting to train others.  My work
teaching XQuery is that it is much easier to learn.  Especially by people
that have SQL or PHP experience.

In our case with the eXist documentation project we may have 1000 DocBook
documents in the future and we frequently want to get list of the documents
and perform searches in the database of keywords.  So they will all be
indexed in the system anyway.

We are now working on converting all the XSLT programs to XQuery for our
tools.

I would be happy to send you the source if you are interested, but it will
need to be cleaned up and documented.

Here are the first examples:

http://www.syntactica.com/rest/db/org/syntactica/apps/docbook/views/list-items.xq

I still need to get the source code to format like the eXist documents.

- Dan

On Thu, Apr 8, 2010 at 9:37 AM, Tony Rogers <to...@go...> wrote:

> On Apr 8, 2010, at 9:59 AM, Dan McCreary wrote:
>
> Hello Tony!
>
> Welcome to the team!
>
> I am working on an XQuery script that will convert DocBook 5 to XHTML.
>
> Here is the start of two new beginner's guides:
>
>
> http://www.syntactica.com/rest/db/org/syntactica/apps/docbook/views/list-items.xq
>
> Let me know if you are interested in helping.  The conversion is being done
> with a typeswitch transform<http://en.wikibooks.org/wiki/XQuery/Typeswitch_Tranformations>
> .
>
> - Dan
>
>
>
> Oh, I’m interested in helping anything that deals with outputting XHTML &
> CSS.  :)
>
> I admit I have VERY little experience with XQuery.  I’ve basically just
> played around with it now and again in oXygen.  I am much more experienced
> with XSLT 2.
>
> Also, I thought that XSL was by design better suited to transform what is
> essentially the same document from one format to another, while XQuery is
> preferable for grabbing subsets of data—for example, a bug database.
>
> I’m totally interested in helping, even if you have your heart set on
> XQuery!  I just wanted to ask about the choice to use it over XSL for a
> conversion.  :)
>
> (BTW let me know what I can do.  I may be inexperienced with XQuery but I
> know there is a much shorter distance from reading it to understanding what
> it does compared with XSL.  I think I’ll still be able to contribute
> something or other that might be useful.)
>
> —Tony
>



-- 
Dan McCreary
Semantic Solutions Architect
syntactica.com
952-460-1674
VOIP: 111@69.199.167.229

[Exist-development] fyi github now supports subversion

[James F. <jam...@gm...>]

http://intertwingly.net/blog/2010/04/01/GitHub-support-for-Subversion

J

Re: [Exist-development] SSL for Jetty under trunk

[José M. F. G. <jm...@us...>]

Hi everybody,
	although it is not what you need or you are looking for, remember the changes of r11466 commit, which allows relocating the jetty configuration directory using the system property jetty.home on startup.

	José María

On 03/30/10 08:20, Dmitriy Shabanov wrote:
> On Tue, 2010-03-30 at 08:18 +0200, Dannes Wessels wrote:
>> Hi,
>>
>> On Tue, Mar 30, 2010 at 7:26 AM, Dmitriy Shabanov<sha...@gm...>  wrote:
>>> Well, add new mode "jettySSL" should solve problem.
>>>
>>> If no objections, I can make it this way.
>>
>> as long it is switched off by default (and no certificate is deployed)
>
> default will be jetty mode as it now.
>
> BTW, should I add standaloneSSL?
>
>
>
>
> ------------------------------------------------------------------------------
> Download Intel&#174; Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
>
>
>
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development

-- 
"La violencia es el último recurso del incompetente"
	- Salvor Hardin en "La Fundación" de Isaac Asimov
"Premature optimization is the root of all evil." - Donald Knuth

José María Fernández González
e-mail: jos...@gm...

Re: [Exist-development] Binary resources not deleted when a collection is moved into existing collection with the same name.

[José M. F. G. <jm...@us...>]

Hi Thomas,
	I have taken a look at the bug you have reported, and following the codepaths the reason is that in NativeBroker, in moveBinaryFork, there is no check about the (lack of existence of) destination at binary resources level, so the move operation fails when there is already a binary resource with the same path. I have created a fix (see attached source patch), and I would like you to test it before I commit the changes to eXist trunk.

	Best Regards,
		José María

On 03/31/10 15:45, Thomas White wrote:
> I was wandering if anybody can have a look at this issue.
> It is pretty serious to me because after a second file import eXist goes
> out of sync between the binary resource listed internally and the files
> that actually are stored in the file system at the location they are
> expected. And second, the fact that files stay in the file system
> after collections deletions is a security issue as well.
> I really hope somebody will be able to go to the bottom of this problem.
> Thomas
>
> ------
>
> Thomas White
>
> Mobile:+44 7711 922 966
> Skype: thomaswhite
> gTalk: thomas.0007
> Linked-In:http://www.linkedin.com/in/thomaswhite0007
> facebook: http://www.facebook.com/thomas.0007
>
>
>
> On 25 March 2010 17:28, Thomas White <tho...@gm...
> <mailto:tho...@gm...>> wrote:
>
>     EXIST_HOME  is  D:\eXist2
>     JAVA SDK HOME C:\Java\jdk1.6.0_16
>     Has anyone been able to reproduce this bug?
>     Thomas
>
>
>
>
> ------------------------------------------------------------------------------
> Download Intel&#174; Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
>
>
>
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development

-- 
"La violencia es el último recurso del incompetente"
	- Salvor Hardin en "La Fundación" de Isaac Asimov
"Premature optimization is the root of all evil." - Donald Knuth

José María Fernández González
e-mail: jos...@gm...

Re: [Exist-development] [Exist-commits] SF.net SVN: exist:[11637] trunk/eXist/webapp/credits.xml

[Andrzej J. T. <an...@ch...>]

Adam:

> I do not think that ranking by "number of commits" is any kind of useful metric!

I agree with Adam 100%.  In fact, in some many cases too many commits is a "bad thing" and not to be advertised. Fewer,
integrated, high quality commits are worth a lot more than tons of small, disjoint, poor commits.

-- 
Andrzej Taramina
Chaeron Corporation: Enterprise System Solutions
http://www.chaeron.com

Re: [Exist-development] New list member offering services :)

[Loren C. <lor...@gm...>]

On Apr 8, 2010, at 04:50 AM, Leif-Jöran Olsson wrote:

> Den 2010-04-08 11:08, Wolfgang Meier skrev:
> 
>>> True, perhaps the documentation should be moved under
>>> $EXIST_HOME/webapp/documentation and reorganised into sensible
>>> sub-folders.
>> 
>> I agree.
> 
> Well, I am not that happy with having the documentation under webapp. I
> would like it to be copied there on installation if one wishes instead
> or in same step as it is installed into the database.
> 
> Loren you, as the documentation coordinator, probably already have some
> ideas here?
> 

Yes.  I am working on an idea for the documentation to be loaded into the database from one of eXist's servers.  That way the documentation will have any corrections since the release was created.  We would have copies for each release so that the installer would grab the appropriate documentation.  Installing the documentation into the DB also allows the documentation to be searchable.

Loren

Re: [Exist-development] New list member offering services :)

[Loren C. <lor...@gm...>]

Hello Tony,

We welcome the help.  Would you like me to schedule a conference call?  We can talk about ideas.

On Apr 7, 2010, at 08:00 PM, Tony Rogers wrote:

> Hi eXist devs,
> 
> 
> My name is Tony.  I was going through the eXist website and thought that I would offer my services to improve the documentation.  
> 
> 
> Specifically, I do pretty well with:
> 
>  improving spelling, grammar, punctuation, typography, and phrasing (for clarity)
> (Also, I can only do this in English—sorry!  I love languages in general, but I don’t know any well enough to do any real proofreading tweaks outside of English.)
> working with X/HTML and CSS
> I’ve been informed there’s a new site in the works, but if anyone working on the new site is looking for an extra hand I am totally willing!
> 
> Before I start working, I had a few questions about the documentation.  I’m just going to fire off a list of questions from the top of my head, and feel free to answer just one or two if you want.  (Of course, if you don’t mind answering more, that’d be great, too!)
> 
> 
> And now…the questions:
> 
> I noticed that documents using DocBook are not updated to DocBook5.  How come?  
> Would it break anything if I started converting everything to DocBook5?

I am planning on converting to DocBook5, but am waiting for the new design before tackling the conversion of the db2xhtml.xsl file.  We can also develop a style guide so that the eXist documentation is consistent in the organization and presentation.

> Many files appear to be custom XML vocabularies.  Are these vocabularies documented anywhere?  
> In past experiences, I’ve found documentation pretty easy to track down.  But in eXist, with practically everything being based on XML—and no obvious directories like “doc/“—I am a little apprehensive.  

An idea that I have been toying with would be to have the documentation downloaded into the database during installation.  Th\at way, then documentation is always up to date and it would be searchable.

> Is there a particular part of the documentation I should work on first?
> Any other requests for a newbie like me?  :)
> 
> Thanks a lot for helping make such a fantastic tool for XML junkies!  eXist is a pretty fantastic piece of software! :)
> 
> 
> —Tony | “Zearin” 
> 

Again.  Welcome to the party :)

Loren

Re: [Exist-development] New list member offering services :)

[Leif-Jöran O. <lj...@ex...>]

Den 2010-04-08 11:08, Wolfgang Meier skrev:

>> True, perhaps the documentation should be moved under
>> $EXIST_HOME/webapp/documentation and reorganised into sensible
>> sub-folders.
> 
> I agree.

Well, I am not that happy with having the documentation under webapp. I
would like it to be copied there on installation if one wishes instead
or in same step as it is installed into the database.

Loren you, as the documentation coordinator, probably already have some
ideas here?

Leif-Jöran

Re: [Exist-development] New list member offering services :)

[Wolfgang M. <wol...@ex...>]

>> I’ve been informed there’s a new site in the works, but if anyone working on
>> the new site is looking for an extra hand I am totally willing!
>
> Interesting, can I ask where you heard this?

I already had a chat with Anthony yesterday evening and asked him to
write an introduction to this list. It is no secret that we paid a
designer from existing funds to work through our site. We made several
attempts to do this ourselves in the past, but we never got anywhere
(I'm a lousy designer). Anyway, it doesn't make much sense to invest
time into rewriting the CSS and XSLT before we have the redesigned
templates.

>> I noticed that documents using DocBook are not updated to DocBook5.  How
>> come?
>>
>> Would it break anything if I started converting everything to DocBook5?

The main stylesheet we use is webapp/stylesheets/db2xhtml.xsl. It
would be nice to convert this, though we would also need to switch all
the documentation and the search function to use namespaces. Layout
changes can still be done later. Well, Loren is the documentation
coordinator, so we have to ask if a migration to docbook5 would be
feasible now?

>> Many files appear to be custom XML vocabularies.  Are these vocabularies
>> documented anywhere?

Some files, e.g. the main index.xml, use old stylesheets/vocabularies
and should probably be converted to docbook as well.

> True, perhaps the documentation should be moved under
> $EXIST_HOME/webapp/documentation and reorganised into sensible
> sub-folders.

I agree.

Wolfgang

Re: [Exist-development] [Exist-commits] SF.net SVN: exist:[11637] trunk/eXist/webapp/credits.xml

[Adam R. <ad...@ex...>]

I do not think that ranking by "number of commits" is any kind of useful metric!


On 7 April 2010 20:40,  <ix...@us...> wrote:
> Revision: 11637
>          http://exist.svn.sourceforge.net/exist/?rev=11637&view=rev
> Author:   ixitar
> Date:     2010-04-07 19:40:08 +0000 (Wed, 07 Apr 2010)
>
> Log Message:
> -----------
> [documentation] Adding a link the the ohloh site with the live list of the contributors.
>
> Modified Paths:
> --------------
>    trunk/eXist/webapp/credits.xml
>
> Modified: trunk/eXist/webapp/credits.xml
> ===================================================================
> --- trunk/eXist/webapp/credits.xml      2010-04-07 19:28:31 UTC (rev 11636)
> +++ trunk/eXist/webapp/credits.xml      2010-04-07 19:40:08 UTC (rev 11637)
> @@ -45,7 +45,8 @@
>         <section>
>             <title>Credits</title>
>             <para>The following people have contributed to eXist (by writing code or supplying
> -                patches) or are active committers (in strict alphabetical order):</para>
> +                patches) or are active committers (in strict alphabetical order).  To see a live
> +                list, go to <ulink url="https://www.ohloh.net/p/exist/contributors">Ohloh</ulink>.</para>
>             <informaltable cols="5">
>                 <thead>
>                     <tr bgcolor="#0990EE" valign="top">
>
>
> This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
>
> ------------------------------------------------------------------------------
> Download Intel&#174; Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Exist-commits mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-commits
>



-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

Re: [Exist-development] New list member offering services :)

[Adam R. <ad...@ex...>]

> My name is Tony.  I was going through the eXist website and thought that I
> would offer my services to improve the documentation.

Hi Tony, that sounds excellent :-)

> (Also, I can only do this in English—sorry!  I love languages in general,
> but I don’t know any well enough to do any real proofreading tweaks outside
> of English.)

Documentation is only made available in English at present, so no problem there.

> working with X/HTML and CSS
>
> I’ve been informed there’s a new site in the works, but if anyone working on
> the new site is looking for an extra hand I am totally willing!

Interesting, can I ask where you heard this?


> I noticed that documents using DocBook are not updated to DocBook5.  How
> come?
>
> Would it break anything if I started converting everything to DocBook5?

Not sure to be honest, we use some XSLT to convert the DocBook into a
layout for the site. I guess the XSLT may need to be updated as well,
do you know XSLT?

> Many files appear to be custom XML vocabularies.  Are these vocabularies
> documented anywhere?

Can you give me an example of which files do you refer to?

> In past experiences, I’ve found documentation pretty easy to track down.
>  But in eXist, with practically everything being based on XML—and no obvious
> directories like “doc/“—I am a little apprehensive.

True, perhaps the documentation should be moved under
$EXIST_HOME/webapp/documentation and reorganised into sensible
sub-folders.

> Is there a particular part of the documentation I should work on first?

Nothing springs to my mind, perhaps you have seen something ;-)



-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

[Exist-development] New list member offering services :)

[Tony R. <to...@go...>]

Hi eXist devs,


My name is Tony.  I was going through the eXist website and thought that I would offer my services to improve the documentation.  


Specifically, I do pretty well with:

 improving spelling, grammar, punctuation, typography, and phrasing (for clarity)
(Also, I can only do this in English—sorry!  I love languages in general, but I don’t know any well enough to do any real proofreading tweaks outside of English.)
working with X/HTML and CSS
I’ve been informed there’s a new site in the works, but if anyone working on the new site is looking for an extra hand I am totally willing!

Before I start working, I had a few questions about the documentation.  I’m just going to fire off a list of questions from the top of my head, and feel free to answer just one or two if you want.  (Of course, if you don’t mind answering more, that’d be great, too!)


And now…the questions:

I noticed that documents using DocBook are not updated to DocBook5.  How come?  
Would it break anything if I started converting everything to DocBook5?
Many files appear to be custom XML vocabularies.  Are these vocabularies documented anywhere?  
In past experiences, I’ve found documentation pretty easy to track down.  But in eXist, with practically everything being based on XML—and no obvious directories like “doc/“—I am a little apprehensive.  
Is there a particular part of the documentation I should work on first?
Any other requests for a newbie like me?  :)

Thanks a lot for helping make such a fantastic tool for XML junkies!  eXist is a pretty fantastic piece of software! :)


—Tony | “Zearin” 

Re: [Exist-development] [Exist-open] Lucene Analyzer

[Jacob M. <jac...@gm...>]

I am bringing this thread over to devel list since that seems like the
most appropriate location for this part of the discussion.

I was thinking about it more and I came up with a slight variation on
this last approach primarily to satisfy my desire not to accidentally
trample anything. I was able to track down AnalyzerConfig and that is
where I started my work, good to know I was following that part of the
code correctly then. I gave it a few hours and banged together the
following changes which I am posting for initial review from the
developers for some feedback/suggestions.

So org.exist.indexing.lucene.AnalyzerConfig has the following changes:
Added:

    private final static String PARAM_CHILDREN = "params";
    private final static String PARAM_TYPE_ATTRIBUTE = "type";

    private static List<Object> parseParamChildren(NodeList children){
        List<Object> p = new ArrayList<Object>(children.getLength());
        for(int idx = children.getLength() - 1; idx >= 0; --idx){
            p.add(0, parseParamChild(children.item(idx)));
        }
        return p;
    }

    private static Object parseParamChild(Node child) {
        String type = child.getNodeName();
        String subtype =
child.getAttributes().getNamedItem(PARAM_TYPE_ATTRIBUTE).getNodeValue();

        if(type.equals("class")){
            try {
                return Class.forName(child.getNodeValue());
            } catch (ClassNotFoundException ex) {
                //TODO
            }
        } else if (type.equals("string")){
            return child.getNodeValue();
        } else if (type.equals("int")){
            return Integer.parseInt(child.getNodeValue());
        } else if (type.equals("char")){
            return child.getNodeValue().toCharArray();
        } else if (type.equals("float")){
            return Float.parseFloat(child.getNodeValue());
        } else if (type.equals("double")){
            return Double.parseDouble(child.getNodeValue());
        } else if (type.equals("array")){
            if (subtype.equals("string")){
                return
parseParamChildren(child.getChildNodes()).toArray(new String[0]);
            } else if (subtype.equals("int")){
                return
parseParamChildren(child.getChildNodes()).toArray(new Integer[0]);
            } else if (subtype.equals("float")){
                return
parseParamChildren(child.getChildNodes()).toArray(new Float[0]);
            } else if (subtype.equals("double")){
                return
parseParamChildren(child.getChildNodes()).toArray(new Double[0]);
            } else if (subtype.equals("map")) {
                return (Map[])
parseParamChildren(child.getChildNodes()).toArray();
            } else if (subtype.equals("object")) {
                return parseParamChildren(child.getChildNodes()).toArray();
            }
        } else if (type.equals("map")){
            Map map = new HashMap<String, Object>();

            NodeList children = child.getChildNodes();
            for(int idx = children.getLength() - 1; idx >= 0; --idx){
                Node c = children.item(idx);

map.put(c.getAttributes().getNamedItem("key").getNodeValue(),
parseParamChild(c));
            }

            return map;
        } else {
            try {
                return Class.forName(subtype).getConstructor(new
Class[]{String.class}).newInstance(new
Object[]{child.getNodeValue()});
            } catch (ClassNotFoundException ex) {
                //TODO
            } catch (NoSuchMethodException ex) {
                //TODO
            } catch (SecurityException ex) {
                //TODO
            } catch (InstantiationException ex) {
                //TODO
            } catch (IllegalAccessException ex) {
                //TODO
            } catch (IllegalArgumentException ex) {
                //TODO
            } catch (InvocationTargetException ex) {
                //TODO
            }
        }

        return new Object();
    }
//--------------------------------------------------------------------------------------

I also added/modified the configureAnalyzer method:
    protected static Analyzer configureAnalyzer(Element config) throws
DatabaseConfigurationException {
        String className = config.getAttribute(CLASS_ATTRIBUTE);

        List<Object> params =
parseParamChildren(config.getElementsByTagName(PARAM_CHILDREN).item(0).getChildNodes());

        Class[] signature = new Class[params.size()];
        Object[] values = new Object[params.size()];
        int idx = 0;
        for(Object p : params){
            signature[idx] = p.getClass();
            values[idx] = p;
            ++idx;
        }

        if (className != null && className.length() != 0) {
            try {
                Class<?> clazz = Class.forName(className);
                if (!Analyzer.class.isAssignableFrom(clazz))
                    throw new DatabaseConfigurationException("Lucene
index: analyzer class has to be" +
                            " a subclass of " + Analyzer.class.getName());
                if (signature.length > 0){
                    try {
                        return (Analyzer)
clazz.getConstructor(signature).newInstance(values);
                    } catch (NoSuchMethodException ex) {
                        //TODO
                    } catch (SecurityException ex) {
                        //TODO
                    } catch (IllegalArgumentException ex) {
                        //TODO
                    } catch (InvocationTargetException ex) {
                        //TODO
                    }
                }
                return (Analyzer) clazz.newInstance();
///--- snip ---- rest is unmodified


What this gets me, i believe, is the following:

<analyzer  class="org.apache.lucene.analysis.standard.StandardAnalyzer">
<params>
<string>Something</string>
</params>
</analyzer>

This configuration hunts for a single string accepting constructor and
passes in Something to it. The way I have written the parser code
there is predefined "shorthand" for string, int, float, double, char
and class. You can actually put in any full java type that accepts a
string constructor to accept the value being passed in. <int> is just
shorthand for <object type="java.lang.Integer"> for instance. In
addition I have allowed primitive arrays of the common base types as
well as map. I currently have plans to adjust map to allow specifying
the concrete type to implement just in case as well as a list type. At
present maps are recursive, but the primitive arrays are not I don't
think. IE: I can't construct String[][] with this method that I know
of.

This ties configuration of the analyzer into a namespace under the
analyzer tag so it doesn't intrude on other potential things to be
added (params right now, but easily renamed if desired... maybe init
or configure?) and allows for some pretty simple configuration of even
complex objects. Lets say I have an analyzer that takes 2 ints, a
float, and a String array:

<analyzer  class="analyzers.SuperAnalyzer">
<params>
<int name="min">2</int>
<int name="max">10</int>
<float name="boost">.5</float>
<array name="stoplist" type="string">
<string>an</string>
<string>the</string>
<string>then</string>
<string>test</string>
</array>
</params>
</analyzer>

I added name in there, though it is completely unused/ignored, but
would help if trying to understand roughly what the config entries
were mapping too so it is easier to tune by someone else. Note this
still has the mandatory condition that the order of the nodes in the
XML exactly match the constructor signature, or it will not be found.

If people think this is a good approach I can work on making a few of
those changes, most are fairly straight forward. The code I currently
have does compile (can't remember it's exact version/date) but I have
NOT tested it at all, even just to see if it parses without crashing.
I was more interested in the approach and feedback before I start
fiddling with testing related stuff.

Jacob Myers

Re: [Exist-development] How big can you supersize this puppy?

[José M. F. G. <jm...@us...>]

Hi Adam (and everybody),
	yes, that's my very old e-mail! My professional e-mail is now jmf...@cn... since the research group where I'm working on moved to CNIO (Spanish National Cancer Research Centre) three years ago.

	Obviously, you can also use this one :-)

	My old tests (1~2 years ago) were focused on scalability at single document, at collection and at query levels. A nice improvement since then is the FT indexes implementation used in eXist. It has improved A LOT, because the FTI implementation prior to the Lucene index did not scale up. eXist team has also removed many of the existing internal bottlenecks, most of them at intermediate results processing.

	But as Wolfgang has written, there are lots of possible improvements which are only needed when you are working with huge database instances. I guess intermediate results in a complex query on a huge database can still fire an OutOfMemoryError. For instance, a sequence of a hundred thousand in-memory nodes, which is being generated from database content. Other example, when you have to sort a huge sequence of nodes based on a complex condition (which is hopefully being addressed by latest Wolfgang developments).

	Best wishes,
		José María

On 03/31/10 22:43, Adam Retter wrote:
> Andrezj,
>
> A chap on the mailing list has quite some experience of scaling eXist
> into the hundreds of gigabytes range, perhaps if you email him he
> could share some of his experiences with you as well. José María
> Fernández González. jmfernandez<at>  cnb.uam.es
>
> On 29 March 2010 15:59, Andrzej Jan Taramina<an...@ch...>  wrote:
>> Looking to get some guidance on how big you can scale an eXist database.
>>
>> Right now, our instances are about 15-25K documents where each document is in the 25K-2M range, probably averaging
>> around 150-200K.  This results in a dom.dbx = 3.5G, structure.dbx = 1.8G, collections.dbx = 4.2M and values.dbx = 155M,
>> which is not all that large compared to some relational databases.
>>
>> What if we scale up 10x to nearly quarter of a million documents?  The file sizes still shouldn't be all that big for
>> modern hardware, but will the performance scale linearly or close to it, assuming a powerful enough server (say a
>> dual-cpu, 6-Core machine (12 cores, 24 native threads) with gobs of memory)?
>>
>> OK.....if that works how about two orders of magnitude (100x current size)?  That would give us 2.5M documents, 250GB
>> dom.dbx and a structure.dbx in the 180GB range.  Bit too big or practical to cache the whole structure.dbx in memory,
>> regardless of the size of the memory in the server.
>>
>> At what point do I start looking at alternative storage mechanisms, (RDBMS, Hadoop, memcached, etc.) or co-operating
>> distributed eXist instances?
>>
>> Thanks for any insights from those that have pushed big databases in eXist...
>>
>> --
>> Andrzej Taramina
>> Chaeron Corporation: Enterprise System Solutions
>> http://www.chaeron.com
>>
>> ------------------------------------------------------------------------------
>> Download Intel&#174; Parallel Studio Eval
>> Try the new software tools for yourself. Speed compiling, find bugs
>> proactively, and fine-tune applications for parallel performance.
>> See why Intel Parallel Studio got high marks during beta.
>> http://p.sf.net/sfu/intel-sw-dev
>> _______________________________________________
>> Exist-development mailing list
>> Exi...@li...
>> https://lists.sourceforge.net/lists/listinfo/exist-development
>>
>
>
>

-- 
"La violencia es el último recurso del incompetente"
	- Salvor Hardin en "La Fundación" de Isaac Asimov
"Premature optimization is the root of all evil." - Donald Knuth

José María Fernández González
e-mail: jos...@gm...