exist-open — Open discussion

Re: [Exist-open] implicit validation

[Dannes W. <di...@ex...>]

For explicit validation yes, implicit no

Regards

Dannes 

> On 21 Feb 2023, at 08:38, Eduard Drenth <ed...@fr...> wrote:
> 
> 
> Thanks, looks like I can use .rng in a catalog?
> 
> 
> 
> Eduard Drenth,

Re: [Exist-open] implicit validation

[Eduard D. <ed...@fr...>]

Thanks, looks like I can use .rng in a catalog?




Eduard Drenth, Software Architekt


ed...@fr...


Doelestrjitte 8

8911 DX  Ljouwert

(058) 213 14 14

+31 62 094 34 28

https://www.fryske-akademy.nl/
https://frysker.nl/
https://frisian.eu/
[cid:45ab8689-c29e-49f8-a81e-21989d2ee00c]

gpg: pgp.key-server.io/pks/lookup?search=eduarddrenth

________________________________
From: Joe Wicentowski <jo...@gm...>
Sent: Monday, February 20, 2023 11:00 PM
To: Eduard Drenth <ed...@fr...>
Cc: Exist-open <exi...@li...>
Subject: Re: [Exist-open] implicit validation

Hi Eduard,

eXist rejects attempts to store only non-well-formed documents. Failure to adhere to a schema will not trigger a rejection.

eXide - at the application level - performs some limited schema checking - of .xconf files. See:

- Validation code: https://github.com/eXist-db/eXide/blob/develop/modules/validate-xml.xq
- Catalog: https://github.com/eXist-db/eXide/blob/develop/resources/schema/catalog.xml

Joe

On Mon, Feb 20, 2023 at 10:50 AM Eduard Drenth <ed...@fr...<mailto:ed...@fr...>> wrote:
Dear all,

When validation is configured uploading invalid documents (via eXide) seems possible. Is that correct and by design?

Can someone give an overview of when documents are (not) validated?

Regards,


Eduard Drenth, Software Architekt


ed...@fr...<mailto:ed...@fr...>


Doelestrjitte 8

8911 DX  Ljouwert

(058) 213 14 14

+31 62 094 34 28

https://www.fryske-akademy.nl/
https://frysker.nl/
https://frisian.eu/
[cid:18670d3c07833a855901]

gpg: pgp.key-server.io/pks/lookup?search=eduarddrenth<http://pgp.key-server.io/pks/lookup?search=eduarddrenth>

_______________________________________________
Exist-open mailing list
Exi...@li...<mailto:Exi...@li...>
https://lists.sourceforge.net/lists/listinfo/exist-open

Re: [Exist-open] implicit validation

[Joe W. <jo...@gm...>]

Hi Eduard,

eXist rejects attempts to store only non-well-formed documents. Failure to
adhere to a schema will not trigger a rejection.

eXide - at the application level - performs some limited schema checking -
of .xconf files. See:

- Validation code:
https://github.com/eXist-db/eXide/blob/develop/modules/validate-xml.xq
- Catalog:
https://github.com/eXist-db/eXide/blob/develop/resources/schema/catalog.xml

Joe

On Mon, Feb 20, 2023 at 10:50 AM Eduard Drenth <ed...@fr...>
wrote:

> Dear all,
>
> When validation is configured uploading invalid documents (via eXide)
> seems possible. Is that correct and by design?
>
> Can someone give an overview of when documents are (not) validated?
>
> Regards,
>
> Eduard Drenth, Software Architekt
>
>
> ed...@fr...
>
>
> Doelestrjitte 8
>
> 8911 DX  Ljouwert
> (058) 213 14 14
>
> +31 62 094 34 28
> https://www.fryske-akademy.nl/
> https://frysker.nl/
> https://frisian.eu/
>
> gpg: pgp.key-server.io/pks/lookup?search=eduarddrenth
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>

[Exist-open] implicit validation

[Eduard D. <ed...@fr...>]

Dear all,

When validation is configured uploading invalid documents (via eXide) seems possible. Is that correct and by design?

Can someone give an overview of when documents are (not) validated?

Regards,


Eduard Drenth, Software Architekt


ed...@fr...


Doelestrjitte 8

8911 DX  Ljouwert

(058) 213 14 14

+31 62 094 34 28

https://www.fryske-akademy.nl/
https://frysker.nl/
https://frisian.eu/
[cid:bc5c924b-d605-442b-b3c5-266b8fc6e65a]

gpg: pgp.key-server.io/pks/lookup?search=eduarddrenth

[Exist-open] trigger validation

[Eduard D. <ed...@fr...>]

Dear all,

I want to implement validating a document via a before-update-document trigger.

When validation fails I want the update to fail and I want to provide the caller with feedback.

I wrote this function:


declare function local:validate($uri as xs:string) {
    let $report := validation:jing-report(xs:anyURI($uri), doc("xmldb:exist:///db/apps/corpora/schematron/corpora_linguistics.sch"))
    return if ($report/status='invalid') then
        (
            util:log('warn',$report/message),
            error(QName('http://corpora.fa','invalid'))
        )
    else ()
};

It does the job sort of, but:

  1.  the currently stored document is validated, not the user input
  2.  I found no option to get $report/message back to the user (in eXide)

I could perhaps remember the current document in before-update-document and when validation fails in after-update-document restore the documment, but that seems a bit tricky perhaps.

Does any of you have experience with such a scenario?


Eduard Drenth, Software Architekt


ed...@fr...


Doelestrjitte 8

8911 DX  Ljouwert

(058) 213 14 14

+31 62 094 34 28

https://www.fryske-akademy.nl/
https://frysker.nl/
https://frisian.eu/
[cid:b16880ff-4d10-476b-8d0b-2082bf54e33f]

gpg: pgp.key-server.io/pks/lookup?search=eduarddrenth

Re: [Exist-open] backup zip-files-max ignored?

[Loren C. <lor...@gm...>]

Please see issues:

https://github.com/eXist-db/exist/issues/4010
[feature] ConsistencyCheckTask does not have the property zip-files-max that is in BackupSystemTask · Issue #4010 · eXist-db/exist
github.com


https://github.com/eXist-db/exist/issues/4739

> On Feb 16, 2023, at 5:44 AM, Eduard Drenth <ed...@fr...> wrote:
> 
> Dear all,
> 
> With below config in exist-db:6.1.0 the number of backup files written is unlimited.
> 
>         <job type="system" name="checkAndBackup"
>              class="org.exist.storage.ConsistencyCheckTask"
>              cron-trigger="0 0 22 * * ?">
>             <parameter name="output" value="/backup"/>
>             <parameter name="backup" value="yes"/>
>             <parameter name="incremental" value="no"/>
>             <parameter name="incremental-check" value="no"/>
>             <parameter name="max" value="2"/>
>             <parameter name="zip-files-max" value="7"/>
>         </job>
> 
> Am I doing something wrong or should I file an issue?
> 
> Regards,
> 
> Eduard Drenth, Software Architekt
> 
> ed...@fr... <mailto:ed...@fr...>
> 
> Doelestrjitte 8
> 8911 DX  Ljouwert
> (058) 213 14 14
> +31 62 094 34 28
> https://www.fryske-akademy.nl/
> https://frysker.nl/
> https://frisian.eu/
> <Outlook-ms42x3at.png>
> gpg: pgp.key-server.io/pks/lookup?search=eduarddrenth <http://pgp.key-server.io/pks/lookup?search=eduarddrenth>
> _______________________________________________
> Exist-open mailing list
> Exi...@li... <mailto:Exi...@li...>
> https://lists.sourceforge.net/lists/listinfo/exist-open

[Exist-open] backup zip-files-max ignored?

[Eduard D. <ed...@fr...>]

Dear all,

With below config in exist-db:6.1.0 the number of backup files written is unlimited.

        <job type="system" name="checkAndBackup"
             class="org.exist.storage.ConsistencyCheckTask"
             cron-trigger="0 0 22 * * ?">
            <parameter name="output" value="/backup"/>
            <parameter name="backup" value="yes"/>
            <parameter name="incremental" value="no"/>
            <parameter name="incremental-check" value="no"/>
            <parameter name="max" value="2"/>
            <parameter name="zip-files-max" value="7"/>
        </job>

Am I doing something wrong or should I file an issue?

Regards,


Eduard Drenth, Software Architekt


ed...@fr...


Doelestrjitte 8

8911 DX  Ljouwert

(058) 213 14 14

+31 62 094 34 28

https://www.fryske-akademy.nl/
https://frysker.nl/
https://frisian.eu/
[cid:c10a9592-b4aa-4546-b504-0aeb8a7a40ea]

gpg: pgp.key-server.io/pks/lookup?search=eduarddrenth

[Exist-open] Dashboard doesn't work on Mac Venture with exist 6.2

[Martin M. <mar...@no...>]

I tried to update my version of eXist runnin on Mac Ventura but could not get it to work. The Dashboard does not respond to commands, and there is something with the password operation. I am reinstalling the system, recording the responses as I go along.

The short version is that the Java admin client responds properly to correct passwords and supports user management. The Dashboard rejects bad passwords, but doesn’t support any further operations.  It’s frozen

After deleting the current application as well as the data directory org.exist I retrieve eXist-db6.2.0 dmg and drag it into the Applications folder

I click on the eXist icon and the eXist-d Configuration window appears
I click the Save button and click ‘Yes’ on the question whether to create the specified data directory
The eXist-db launcher shows up at the top menubar. I click on it and open the Dashboard
I click on the login button and enter admin without entering a password
The dashboard  reappears but the exist-db icon and the launcher icon are dead
I open the Java Admin client and enter a password
(This has worked in the past, but it has always struck me as an ambiguous procedure, because there was nothing to confirm the original choice)
The client doesn’ t complain but  also does nothing when I click on the Connect icon
I open the client again and click on Connect without a password.  This time I get taken to the exist Admin client
I go to the User Manage and create an account under my name with a password
I assign myself to various groups including dba
I sign out and  try to sign it again, this time with a wrong password. The connection fails as it should
When I sign in again with the correct password I am taken to exist Admin Client
I now look up the admin user, which shows paswords dots, the same length of my user password
I change that password to my password
Now both the admin account and my account respond to the proper password and fail with deliberately wrong passwords
Now I try to log in via the Dashboard.  I get the same behaviour for both the admin and my password
The dashboard complains if I put in the wrong paswords.
It does not complain about the right passwords, but it doesn’t respond to any further commands. I can’t get to the package to install TEI Publisher or anything else

Re: [Exist-open] Upgrade eXist-db from 2.1 to 6.2.0

[Areki, A. <aa...@ri...>]

As usual, thank you, Joe, for your recommendation. I will take note briefly for each release.


--
Alem T. Areki
Senior Web Developer – Web Services
University of Richmond

From: Joe Wicentowski <jo...@gm...>
Date: Saturday, February 11, 2023 at 11:39 AM
To: "Areki, Alem" <aa...@ri...>
Cc: "exi...@li..." <exi...@li...>
Subject: Re: [Exist-open] Upgrade eXist-db from 2.1 to 6.2.0

External Email: Use caution in opening links, attachments, and buying gift cards.

HI Alem,

The eXist-db documentation has a general guide on upgrading: https://exist-db.org/exist/apps/doc/upgrading.

It links to another article listing incompatibilities introduced with specific versions: https://exist-db.org/exist/apps/doc/incompatibilities.

I'd advise to read the release notes of at least every major version release between the two versions, and to take note of the procedures they outline for upgrading. You can find the release notes going back to eXist 1.2 linked from https://github.com/eXist-db/exist/releases.

Joe

On Mon, Feb 6, 2023 at 9:23 PM Areki, Alem <aa...@ri...<mailto:aa...@ri...>> wrote:
Hi eXist users,

Here at the University of Richmond, we still use the old eXist version - eXist 2.1. Now we need to upgrade to the latest one - eXist 6.2.0; what would be your recommendation to upgrade from 2.1 to 6.2.0?

Thanks

--
Alem T. Areki
Senior Web Developer – Web Services
University of Richmond

_______________________________________________
Exist-open mailing list
Exi...@li...<mailto:Exi...@li...>
https://lists.sourceforge.net/lists/listinfo/exist-open

Re: [Exist-open] Upgrade eXist-db from 2.1 to 6.2.0

[Areki, A. <aa...@ri...>]

Thank you all for the excellent advice. Last week I upgraded to 6.2, and most of our queries are working correctly except for minor changes. I notice some issues with the indexing; I will look into this in detail.
Thank you again for your suggestion and recommendation.
--
Alem T. Areki
Senior Web Developer – Web Services
University of Richmond

From: Alexander Henket <ah...@xs...>
Date: Saturday, February 11, 2023 at 1:57 AM
To: "exi...@li..." <exi...@li...>
Subject: Re: [Exist-open] Upgrade eXist-db from 2.1 to 6.2.0

External Email: Use caution in opening links, attachments, and buying gift cards.

All advice from Pieter. Additionally assess your index strategy. The index definitions from 2.1 will work, or at least ours from 2.2 did, but upgrading from old range to new range or more sophisticated methods are worth inspecting.

Also from our perspective: backup/restore is a good start but that works best for data/users not so much for logic/applications. So if you like us have a mix of data and apps in 2.1 you might want to tweak the backup zip so it only contains data and users. Do the applications using xar install.

Xar install might fail until pre/post-install scripts are updated for xquery 3.1. Exist-db 2 used preliminary syntax for xquery 3.0/1 which exist-db >= 5 no longer supports like Pieter mentioned:

eXist-db Documentation<https://exist-db.org/exist/apps/doc/upgrading>
exist-db.org<https://exist-db.org/exist/apps/doc/upgrading>
[exist_icon_16x16.ico]<https://exist-db.org/exist/apps/doc/upgrading>

Most code issues you’ll have started with exist-db 5. The release notes outline what they are and link to the perl script

Atomic Wiki<http://www.exist-db.org/exist/apps/wiki/blogs/eXist/eXistdb500>
exist-db.org<http://www.exist-db.org/exist/apps/wiki/blogs/eXist/eXistdb500>
[exist_icon_16x16.ico]<http://www.exist-db.org/exist/apps/wiki/blogs/eXist/eXistdb500>

Everything mentioned there still holds true in exist-db 6.

It took us over a year to update the code, rethink index, rethink code security, test and finally migrate. For UI we had Orbeon xforms inside the db that we replace with a Roaster/jwt based api and external Vue based logic. Roaster and jwt are in the public repo and work with >= 5 and up. The orbeon to vue move is now 3 years in the making. They still run side by side. We found the best start in Roaster by hiring eXist people. The rest we could largely manage ourselves.

Verstuurd vanaf mijn iPhone


Op 11 feb. 2023 om 07:14 heeft Pieter Lamers <pie...@be...> het volgende geschreven:

I would not follow this advice. Any new release can be indefinitely delayed, and I suspect that even with the promise of some breaking changes between 6.2 and 7 there will be much more common ground between those than between 2.1 and 6.2.

I would start by installing 6.2 locally and see if I can restore a backup onto it. Then see where the code breaks. There is a perl (!) script out there which can trace some now invalid constructs, like maps with key := val instead of  key : val. There is a lot of code refactoring possible now: eXist 6.2 uses a recent version of Saxon which supports text value templates (and XSLT 3.0 in general). eXist's XQuery has a much broader coverage of the 3.1 standard.

Probably some other community members have a more sophisticated approach, so let's see what the others say.

Best,
Pieter
On 07/02/2023 06:44, Martin Mueller wrote:
You might as well wait until the promised version 7 comes out in the spring.

From: "Areki, Alem" <aa...@ri...><mailto:aa...@ri...>
Date: Monday, February 6, 2023 at 8:23 PM
To: "exi...@li..."<mailto:exi...@li...> <exi...@li...><mailto:exi...@li...>
Subject: [Exist-open] Upgrade eXist-db from 2.1 to 6.2.0

Hi eXist users,

Here at the University of Richmond, we still use the old eXist version - eXist 2.1. Now we need to upgrade to the latest one - eXist 6.2.0; what would be your recommendation to upgrade from 2.1 to 6.2.0?

Thanks

--
Alem T. Areki
Senior Web Developer – Web Services
University of Richmond





_______________________________________________

Exist-open mailing list

Exi...@li...<mailto:Exi...@li...>

https://lists.sourceforge.net/lists/listinfo/exist-open

--

Pieter Lamers

John Benjamins Publishing Company

Postal Address: P.O. Box 36224, 1020 ME AMSTERDAM, The Netherlands

Visiting Address: Klaprozenweg 75G, 1033 NN AMSTERDAM, The Netherlands

Warehouse: Kelvinstraat 11-13, 1446 TK PURMEREND, The Netherlands

tel: +31 20 630 4747

web: www.benjamins.com<http://www.benjamins.com>
_______________________________________________
Exist-open mailing list
Exi...@li...
https://lists.sourceforge.net/lists/listinfo/exist-open

Re: [Exist-open] Journal can only support 32767 log files

[Craig B. <cra...@ma...>]

> On Feb 11, 2023, at 10:59 AM, Nick Sincaglia <nsi...@nu...> wrote:
> 
> I just have 1 file in $EXIST_HOME/data called 0000008002.log. There is nothing in this file when I look inside. 

8002 in hex is 32770 in decimal, so my theory about an embedded version number in the filename that exceeds what a short integer can hold is not necessarily wrong.  Not necessarily right, either, but I don't think it can be ruled out just yet.

> 
> Does anyone know what the Journal is checking and where it is these log files might be?
> 
> Nick
> 
> On 2/11/23 10:44 AM, Craig Berry wrote:
>> Do you by any chance have more than 32,767 transaction log files in your eXist data directory with names like 0000000001.log?  Or do you have one file with a name like 0000032767.log where the highest-allowed version number is embedded in the name and something fails when it tries to increment?  These are wild guesses, but seeing what's there might be the beginning of finding out what's actually going on.
>> 
>> If you have a good backup and/or you can rebuild your data if necessary, then you'll probably want to rename those file(s) out of the way while eXist is shut down and then try to restart.  If it comes up cleanly, then you've likely solved your problem.  I believe there is some chance using this method that you could lose data from whatever transaction was in flight when eXist crashed.  I've never had that happen but almost all my data are read-only so don't even really need transactions.
>> 
>> Note that I've never seen the exact error you describe -- more commonly when it detects an unclean shutdown it tries to reindex everything, which takes forever, so I delete the log file before restarting.
>> 
>> 
>>> On Feb 11, 2023, at 9:28 AM, Nick Sincaglia <nsi...@nu...>
>>>  wrote:
>>> 
>>> I have eXist-db v5.2 instance running and it is failing to launch due to the below error. I am wondering if anyone might have some suggestions on how to resolve this issue.
>>> 
>>> 2023-02-11 15:12:46,631 [db.exist.prepare-quartz-scheduler] INFO  (QuartzSchedulerImpl.java [getQuartzProperties]:134) - Successfully loaded quartz.properties 
>>> 2023-02-11 15:12:47,269 [main] INFO  (IndexManager.java [initIndex]:149) - Registered index org.exist.indexing.ngram.NGramIndex as ngram-index 
>>> 2023-02-11 15:12:47,817 [main] INFO  (IndexManager.java [initIndex]:149) - Registered index org.exist.indexing.lucene.LuceneIndex as lucene-index 
>>> 2023-02-11 15:12:47,819 [main] INFO  (IndexManager.java [initIndex]:149) - Registered index org.exist.indexing.sort.SortIndex as sort-index 
>>> 2023-02-11 15:12:47,927 [main] INFO  (IndexManager.java [initIndex]:149) - Registered index org.exist.indexing.range.RangeIndex as range-index 
>>> 2023-02-11 15:12:47,930 [main] INFO  (IndexManager.java [initIndex]:149) - Registered index org.exist.storage.structural.NativeStructuralIndex as structural-index 
>>> 2023-02-11 15:12:47,931 [main] INFO  (SyncTask.java [configure]:77) - Using DATA_DIR: /mnt/data/exist-5.2.0/data. Minimal disk space required for database to continue operations: 1024mb 
>>> 2023-02-11 15:12:47,935 [main] INFO  (SyncTask.java [configure]:80) - Usable space on partition containing DATA_DIR: /mnt/data/exist-5.2.0/data: 113117mb 
>>> 2023-02-11 15:12:48,138 [main] INFO  (RecoveryManager.java [recover]:118) - Unclean shutdown detected. Scanning journal... 
>>> 2023-02-11 15:12:48,144 [main] INFO  (RecoveryManager.java [recover]:193) - Database is in clean state. Nothing to recover from the journal. 
>>> 2023-02-11 15:12:48,212 [main] ERROR (TransactionManager.java [doBeginTransaction]:197) - Failed to create transaction. Error writing to Journal 
>>> org.exist.storage.journal.JournalException: Journal can only support 32767 log files
>>>     at org.exist.storage.journal.Journal.writeToLog(Journal.java:342) ~[exist-core-5.2.0.jar:5.2.0]
>>>     at org.exist.storage.journal.JournalManager.journal(JournalManager.java:90) ~[exist-core-5.2.0.jar:5.2.0]
>>>     at org.exist.storage.txn.TransactionManager.doBeginTransaction(TransactionManager.java:195) [exist-core-5.2.0.jar:5.2.0]
>>>     at org.exist.storage.txn.TransactionManager.beginTransaction(TransactionManager.java:180) [exist-core-5.2.0.jar:5.2.0]
>>> 
>>> Here is information about my eXist-db system
>>> eXist Version:     5.2.0
>>> eXist Build:     20200123133609
>>> Operating System:     Linux 4.14.203-156.332.amzn2.x86_64 amd64
>>> Java Version:     11.0.17
>>> Default Encoding:     UTF8
>>> Instance ID:     exist
>>> System CPU Load:     0.979516
>>> Process CPU Load:     0.975791
>>> Free Physical Memory:     2608074752
>>> Total Physical Memory:     33244393472
>>>  
>>> -- 
>>> Nick Sincaglia 
>>> President/Founder
>>> NueMeta, LLC
>>> Digital Media & Technology
>>> Phone: +1-630-303-7035
>>> 
>>> 
>>> nsi...@nu...
>>> 
>>>  
>>> 
>>> 
>>> http://www.nuemeta.com
>>> 
>>> 
>>> Skype: nsincaglia
>>> 
>>> _______________________________________________
>>> Exist-open mailing list
>>> 
>>> Exi...@li...
>>> https://lists.sourceforge.net/lists/listinfo/exist-open
>> ________________________________________
>> Craig A. Berry
>> 
>> "... getting out of a sonnet is much more
>>  difficult than getting in."
>>                  Brad Leithauser
>> 
>> 
> 
> -- 
> Nick Sincaglia 
> President/Founder
> NueMeta, LLC
> Digital Media & Technology
> Phone: +1-630-303-7035
> 
> nsi...@nu...
>  
> 
> http://www.nuemeta.com
> 
> Skype: nsincaglia
> 

________________________________________
Craig A. Berry

"... getting out of a sonnet is much more
 difficult than getting in."
                 Brad Leithauser

Re: [Exist-open] Journal can only support 32767 log files

[Craig B. <cra...@ma...>]

> On Feb 11, 2023, at 10:59 AM, Nick Sincaglia <nsi...@nu...> wrote:
> 
> I just have 1 file in $EXIST_HOME/data called 0000008002.log. There is nothing in this file when I look inside. 
> 
> Does anyone know what the Journal is checking and where it is these log files might be?

Looks like you reported the same thing about a year ago:

https://github.com/eXist-db/exist/issues/4189

which confirms my speculation about the embedded version number and hex encoding.

Adam fixed it here:

https://github.com/eXist-db/exist/pull/4193

but you need a post-5.2 release to get it.

> 
> On 2/11/23 10:44 AM, Craig Berry wrote:
>> Do you by any chance have more than 32,767 transaction log files in your eXist data directory with names like 0000000001.log?  Or do you have one file with a name like 0000032767.log where the highest-allowed version number is embedded in the name and something fails when it tries to increment?  These are wild guesses, but seeing what's there might be the beginning of finding out what's actually going on.
>> 
>> If you have a good backup and/or you can rebuild your data if necessary, then you'll probably want to rename those file(s) out of the way while eXist is shut down and then try to restart.  If it comes up cleanly, then you've likely solved your problem.  I believe there is some chance using this method that you could lose data from whatever transaction was in flight when eXist crashed.  I've never had that happen but almost all my data are read-only so don't even really need transactions.
>> 
>> Note that I've never seen the exact error you describe -- more commonly when it detects an unclean shutdown it tries to reindex everything, which takes forever, so I delete the log file before restarting.
>> 
>> 
>>> On Feb 11, 2023, at 9:28 AM, Nick Sincaglia <nsi...@nu...>
>>>  wrote:
>>> 
>>> I have eXist-db v5.2 instance running and it is failing to launch due to the below error. I am wondering if anyone might have some suggestions on how to resolve this issue.
>>> 
>>> 2023-02-11 15:12:46,631 [db.exist.prepare-quartz-scheduler] INFO  (QuartzSchedulerImpl.java [getQuartzProperties]:134) - Successfully loaded quartz.properties 
>>> 2023-02-11 15:12:47,269 [main] INFO  (IndexManager.java [initIndex]:149) - Registered index org.exist.indexing.ngram.NGramIndex as ngram-index 
>>> 2023-02-11 15:12:47,817 [main] INFO  (IndexManager.java [initIndex]:149) - Registered index org.exist.indexing.lucene.LuceneIndex as lucene-index 
>>> 2023-02-11 15:12:47,819 [main] INFO  (IndexManager.java [initIndex]:149) - Registered index org.exist.indexing.sort.SortIndex as sort-index 
>>> 2023-02-11 15:12:47,927 [main] INFO  (IndexManager.java [initIndex]:149) - Registered index org.exist.indexing.range.RangeIndex as range-index 
>>> 2023-02-11 15:12:47,930 [main] INFO  (IndexManager.java [initIndex]:149) - Registered index org.exist.storage.structural.NativeStructuralIndex as structural-index 
>>> 2023-02-11 15:12:47,931 [main] INFO  (SyncTask.java [configure]:77) - Using DATA_DIR: /mnt/data/exist-5.2.0/data. Minimal disk space required for database to continue operations: 1024mb 
>>> 2023-02-11 15:12:47,935 [main] INFO  (SyncTask.java [configure]:80) - Usable space on partition containing DATA_DIR: /mnt/data/exist-5.2.0/data: 113117mb 
>>> 2023-02-11 15:12:48,138 [main] INFO  (RecoveryManager.java [recover]:118) - Unclean shutdown detected. Scanning journal... 
>>> 2023-02-11 15:12:48,144 [main] INFO  (RecoveryManager.java [recover]:193) - Database is in clean state. Nothing to recover from the journal. 
>>> 2023-02-11 15:12:48,212 [main] ERROR (TransactionManager.java [doBeginTransaction]:197) - Failed to create transaction. Error writing to Journal 
>>> org.exist.storage.journal.JournalException: Journal can only support 32767 log files
>>>     at org.exist.storage.journal.Journal.writeToLog(Journal.java:342) ~[exist-core-5.2.0.jar:5.2.0]
>>>     at org.exist.storage.journal.JournalManager.journal(JournalManager.java:90) ~[exist-core-5.2.0.jar:5.2.0]
>>>     at org.exist.storage.txn.TransactionManager.doBeginTransaction(TransactionManager.java:195) [exist-core-5.2.0.jar:5.2.0]
>>>     at org.exist.storage.txn.TransactionManager.beginTransaction(TransactionManager.java:180) [exist-core-5.2.0.jar:5.2.0]
>>> 
>>> Here is information about my eXist-db system
>>> eXist Version:     5.2.0
>>> eXist Build:     20200123133609
>>> Operating System:     Linux 4.14.203-156.332.amzn2.x86_64 amd64
>>> Java Version:     11.0.17
>>> Default Encoding:     UTF8
>>> Instance ID:     exist
>>> System CPU Load:     0.979516
>>> Process CPU Load:     0.975791
>>> Free Physical Memory:     2608074752
>>> Total Physical Memory:     33244393472
>>>  
>>> -- 
>>> Nick Sincaglia 
>>> President/Founder
>>> NueMeta, LLC
>>> Digital Media & Technology
>>> Phone: +1-630-303-7035
>>> 
>>> 
>>> nsi...@nu...
>>> 
>>>  
>>> 
>>> 
>>> http://www.nuemeta.com
>>> 
>>> 
>>> Skype: nsincaglia
> 

________________________________________
Craig A. Berry

"... getting out of a sonnet is much more
 difficult than getting in."
                 Brad Leithauser

Re: [Exist-open] Journal can only support 32767 log files

[Nick S. <nsi...@nu...>]

I just have 1 file in $EXIST_HOME/data called 0000008002.log. There is 
nothing in this file when I look inside.

Does anyone know what the Journal is checking and where it is these log 
files might be?

Nick

On 2/11/23 10:44 AM, Craig Berry wrote:
> Do you by any chance have more than 32,767 transaction log files in your eXist data directory with names like 0000000001.log?  Or do you have one file with a name like 0000032767.log where the highest-allowed version number is embedded in the name and something fails when it tries to increment?  These are wild guesses, but seeing what's there might be the beginning of finding out what's actually going on.
>
> If you have a good backup and/or you can rebuild your data if necessary, then you'll probably want to rename those file(s) out of the way while eXist is shut down and then try to restart.  If it comes up cleanly, then you've likely solved your problem.  I believe there is some chance using this method that you could lose data from whatever transaction was in flight when eXist crashed.  I've never had that happen but almost all my data are read-only so don't even really need transactions.
>
> Note that I've never seen the exact error you describe -- more commonly when it detects an unclean shutdown it tries to reindex everything, which takes forever, so I delete the log file before restarting.
>
>> On Feb 11, 2023, at 9:28 AM, Nick Sincaglia<nsi...@nu...>  wrote:
>>
>> I have eXist-db v5.2 instance running and it is failing to launch due to the below error. I am wondering if anyone might have some suggestions on how to resolve this issue.
>>
>> 2023-02-11 15:12:46,631 [db.exist.prepare-quartz-scheduler] INFO  (QuartzSchedulerImpl.java [getQuartzProperties]:134) - Successfully loaded quartz.properties
>> 2023-02-11 15:12:47,269 [main] INFO  (IndexManager.java [initIndex]:149) - Registered index org.exist.indexing.ngram.NGramIndex as ngram-index
>> 2023-02-11 15:12:47,817 [main] INFO  (IndexManager.java [initIndex]:149) - Registered index org.exist.indexing.lucene.LuceneIndex as lucene-index
>> 2023-02-11 15:12:47,819 [main] INFO  (IndexManager.java [initIndex]:149) - Registered index org.exist.indexing.sort.SortIndex as sort-index
>> 2023-02-11 15:12:47,927 [main] INFO  (IndexManager.java [initIndex]:149) - Registered index org.exist.indexing.range.RangeIndex as range-index
>> 2023-02-11 15:12:47,930 [main] INFO  (IndexManager.java [initIndex]:149) - Registered index org.exist.storage.structural.NativeStructuralIndex as structural-index
>> 2023-02-11 15:12:47,931 [main] INFO  (SyncTask.java [configure]:77) - Using DATA_DIR: /mnt/data/exist-5.2.0/data. Minimal disk space required for database to continue operations: 1024mb
>> 2023-02-11 15:12:47,935 [main] INFO  (SyncTask.java [configure]:80) - Usable space on partition containing DATA_DIR: /mnt/data/exist-5.2.0/data: 113117mb
>> 2023-02-11 15:12:48,138 [main] INFO  (RecoveryManager.java [recover]:118) - Unclean shutdown detected. Scanning journal...
>> 2023-02-11 15:12:48,144 [main] INFO  (RecoveryManager.java [recover]:193) - Database is in clean state. Nothing to recover from the journal.
>> 2023-02-11 15:12:48,212 [main] ERROR (TransactionManager.java [doBeginTransaction]:197) - Failed to create transaction. Error writing to Journal
>> org.exist.storage.journal.JournalException: Journal can only support 32767 log files
>>      at org.exist.storage.journal.Journal.writeToLog(Journal.java:342) ~[exist-core-5.2.0.jar:5.2.0]
>>      at org.exist.storage.journal.JournalManager.journal(JournalManager.java:90) ~[exist-core-5.2.0.jar:5.2.0]
>>      at org.exist.storage.txn.TransactionManager.doBeginTransaction(TransactionManager.java:195) [exist-core-5.2.0.jar:5.2.0]
>>      at org.exist.storage.txn.TransactionManager.beginTransaction(TransactionManager.java:180) [exist-core-5.2.0.jar:5.2.0]
>>
>> Here is information about my eXist-db system
>> eXist Version:     5.2.0
>> eXist Build:     20200123133609
>> Operating System:     Linux 4.14.203-156.332.amzn2.x86_64 amd64
>> Java Version:     11.0.17
>> Default Encoding:     UTF8
>> Instance ID:     exist
>> System CPU Load:     0.979516
>> Process CPU Load:     0.975791
>> Free Physical Memory:     2608074752
>> Total Physical Memory:     33244393472
>>   
>> -- 
>> Nick Sincaglia
>> President/Founder
>> NueMeta, LLC
>> Digital Media & Technology
>> Phone: +1-630-303-7035
>>
>> nsi...@nu...
>>   
>>
>> http://www.nuemeta.com
>>
>> Skype: nsincaglia
>>
>> _______________________________________________
>> Exist-open mailing list
>> Exi...@li...
>> https://lists.sourceforge.net/lists/listinfo/exist-open
> ________________________________________
> Craig A. Berry
>
> "... getting out of a sonnet is much more
>   difficult than getting in."
>                   Brad Leithauser
>

-- 
Nick Sincaglia
President/Founder
NueMeta, LLC
Digital Media & Technology
Phone: +1-630-303-7035
nsi...@nu...  
http://www.nuemeta.com
Skype: nsincaglia

Re: [Exist-open] Journal can only support 32767 log files

[Craig B. <cra...@ma...>]

Do you by any chance have more than 32,767 transaction log files in your eXist data directory with names like 0000000001.log?  Or do you have one file with a name like 0000032767.log where the highest-allowed version number is embedded in the name and something fails when it tries to increment?  These are wild guesses, but seeing what's there might be the beginning of finding out what's actually going on.

If you have a good backup and/or you can rebuild your data if necessary, then you'll probably want to rename those file(s) out of the way while eXist is shut down and then try to restart.  If it comes up cleanly, then you've likely solved your problem.  I believe there is some chance using this method that you could lose data from whatever transaction was in flight when eXist crashed.  I've never had that happen but almost all my data are read-only so don't even really need transactions.

Note that I've never seen the exact error you describe -- more commonly when it detects an unclean shutdown it tries to reindex everything, which takes forever, so I delete the log file before restarting.

> On Feb 11, 2023, at 9:28 AM, Nick Sincaglia <nsi...@nu...> wrote:
> 
> I have eXist-db v5.2 instance running and it is failing to launch due to the below error. I am wondering if anyone might have some suggestions on how to resolve this issue.
> 
> 2023-02-11 15:12:46,631 [db.exist.prepare-quartz-scheduler] INFO  (QuartzSchedulerImpl.java [getQuartzProperties]:134) - Successfully loaded quartz.properties 
> 2023-02-11 15:12:47,269 [main] INFO  (IndexManager.java [initIndex]:149) - Registered index org.exist.indexing.ngram.NGramIndex as ngram-index 
> 2023-02-11 15:12:47,817 [main] INFO  (IndexManager.java [initIndex]:149) - Registered index org.exist.indexing.lucene.LuceneIndex as lucene-index 
> 2023-02-11 15:12:47,819 [main] INFO  (IndexManager.java [initIndex]:149) - Registered index org.exist.indexing.sort.SortIndex as sort-index 
> 2023-02-11 15:12:47,927 [main] INFO  (IndexManager.java [initIndex]:149) - Registered index org.exist.indexing.range.RangeIndex as range-index 
> 2023-02-11 15:12:47,930 [main] INFO  (IndexManager.java [initIndex]:149) - Registered index org.exist.storage.structural.NativeStructuralIndex as structural-index 
> 2023-02-11 15:12:47,931 [main] INFO  (SyncTask.java [configure]:77) - Using DATA_DIR: /mnt/data/exist-5.2.0/data. Minimal disk space required for database to continue operations: 1024mb 
> 2023-02-11 15:12:47,935 [main] INFO  (SyncTask.java [configure]:80) - Usable space on partition containing DATA_DIR: /mnt/data/exist-5.2.0/data: 113117mb 
> 2023-02-11 15:12:48,138 [main] INFO  (RecoveryManager.java [recover]:118) - Unclean shutdown detected. Scanning journal... 
> 2023-02-11 15:12:48,144 [main] INFO  (RecoveryManager.java [recover]:193) - Database is in clean state. Nothing to recover from the journal. 
> 2023-02-11 15:12:48,212 [main] ERROR (TransactionManager.java [doBeginTransaction]:197) - Failed to create transaction. Error writing to Journal 
> org.exist.storage.journal.JournalException: Journal can only support 32767 log files
>     at org.exist.storage.journal.Journal.writeToLog(Journal.java:342) ~[exist-core-5.2.0.jar:5.2.0]
>     at org.exist.storage.journal.JournalManager.journal(JournalManager.java:90) ~[exist-core-5.2.0.jar:5.2.0]
>     at org.exist.storage.txn.TransactionManager.doBeginTransaction(TransactionManager.java:195) [exist-core-5.2.0.jar:5.2.0]
>     at org.exist.storage.txn.TransactionManager.beginTransaction(TransactionManager.java:180) [exist-core-5.2.0.jar:5.2.0]
> 
> Here is information about my eXist-db system
> eXist Version:     5.2.0
> eXist Build:     20200123133609
> Operating System:     Linux 4.14.203-156.332.amzn2.x86_64 amd64
> Java Version:     11.0.17
> Default Encoding:     UTF8
> Instance ID:     exist
> System CPU Load:     0.979516
> Process CPU Load:     0.975791
> Free Physical Memory:     2608074752
> Total Physical Memory:     33244393472
>  
> -- 
> Nick Sincaglia 
> President/Founder
> NueMeta, LLC
> Digital Media & Technology
> Phone: +1-630-303-7035
> 
> nsi...@nu...
>  
> 
> http://www.nuemeta.com
> 
> Skype: nsincaglia
> 
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open

________________________________________
Craig A. Berry

"... getting out of a sonnet is much more
 difficult than getting in."
                 Brad Leithauser

Re: [Exist-open] Upgrade eXist-db from 2.1 to 6.2.0

[Joe W. <jo...@gm...>]

HI Alem,

The eXist-db documentation has a general guide on upgrading:
https://exist-db.org/exist/apps/doc/upgrading.

It links to another article listing incompatibilities introduced with
specific versions: https://exist-db.org/exist/apps/doc/incompatibilities.

I'd advise to read the release notes of at least every major version
release between the two versions, and to take note of the procedures they
outline for upgrading. You can find the release notes going back to eXist
1.2 linked from https://github.com/eXist-db/exist/releases.

Joe

On Mon, Feb 6, 2023 at 9:23 PM Areki, Alem <aa...@ri...> wrote:

> Hi eXist users,
>
>
>
> Here at the University of Richmond, we still use the old eXist version -
> eXist 2.1. Now we need to upgrade to the latest one - eXist 6.2.0; what
> would be your recommendation to upgrade from 2.1 to 6.2.0?
>
>
>
> Thanks
>
>
>
> --
>
> Alem T. Areki
>
> Senior Web Developer – Web Services
>
> University of Richmond
>
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>

[Exist-open] Journal can only support 32767 log files

[Nick S. <nsi...@nu...>]

I have eXist-db v5.2 instance running and it is failing to launch due to 
the below error. I am wondering if anyone might have some suggestions on 
how to resolve this issue.

2023-02-11 15:12:46,631 [db.exist.prepare-quartz-scheduler] INFO 
(QuartzSchedulerImpl.java [getQuartzProperties]:134) - Successfully 
loaded quartz.properties
2023-02-11 15:12:47,269 [main] INFO  (IndexManager.java [initIndex]:149) 
- Registered index org.exist.indexing.ngram.NGramIndex as ngram-index
2023-02-11 15:12:47,817 [main] INFO  (IndexManager.java [initIndex]:149) 
- Registered index org.exist.indexing.lucene.LuceneIndex as lucene-index
2023-02-11 15:12:47,819 [main] INFO  (IndexManager.java [initIndex]:149) 
- Registered index org.exist.indexing.sort.SortIndex as sort-index
2023-02-11 15:12:47,927 [main] INFO  (IndexManager.java [initIndex]:149) 
- Registered index org.exist.indexing.range.RangeIndex as range-index
2023-02-11 15:12:47,930 [main] INFO  (IndexManager.java [initIndex]:149) 
- Registered index org.exist.storage.structural.NativeStructuralIndex as 
structural-index
2023-02-11 15:12:47,931 [main] INFO  (SyncTask.java [configure]:77) - 
Using DATA_DIR: /mnt/data/exist-5.2.0/data. Minimal disk space required 
for database to continue operations: 1024mb
2023-02-11 15:12:47,935 [main] INFO  (SyncTask.java [configure]:80) - 
Usable space on partition containing DATA_DIR: 
/mnt/data/exist-5.2.0/data: 113117mb
2023-02-11 15:12:48,138 [main] INFO  (RecoveryManager.java 
[recover]:118) - Unclean shutdown detected. Scanning journal...
2023-02-11 15:12:48,144 [main] INFO  (RecoveryManager.java 
[recover]:193) - Database is in clean state. Nothing to recover from the 
journal.
2023-02-11 15:12:48,212 [main] ERROR (TransactionManager.java 
[doBeginTransaction]:197) - *Failed to create transaction. Error writing 
to Journal **
**org.exist.storage.journal.JournalException: Journal can only support 
32767 log files*
     at org.exist.storage.journal.Journal.writeToLog(Journal.java:342) 
~[exist-core-5.2.0.jar:5.2.0]
     at 
org.exist.storage.journal.JournalManager.journal(JournalManager.java:90) 
~[exist-core-5.2.0.jar:5.2.0]
     at 
org.exist.storage.txn.TransactionManager.doBeginTransaction(TransactionManager.java:195) 
[exist-core-5.2.0.jar:5.2.0]
     at 
org.exist.storage.txn.TransactionManager.beginTransaction(TransactionManager.java:180) 
[exist-core-5.2.0.jar:5.2.0]

Here is information about my eXist-db system
eXist Version:     5.2.0
eXist Build:     20200123133609
Operating System:     Linux 4.14.203-156.332.amzn2.x86_64 amd64
Java Version:     11.0.17
Default Encoding:     UTF8
Instance ID:     exist
System CPU Load:     0.979516
Process CPU Load:     0.975791
Free Physical Memory:     2608074752
Total Physical Memory:     33244393472

-- 
Nick Sincaglia
President/Founder
NueMeta, LLC
Digital Media & Technology
Phone: +1-630-303-7035
nsi...@nu...  
http://www.nuemeta.com
Skype: nsincaglia

Re: [Exist-open] Upgrade eXist-db from 2.1 to 6.2.0

[Alexander H. <ah...@xs...>]

<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto">All advice from Pieter. Additionally assess your index strategy. The index definitions from 2.1 will work, or at least ours from 2.2 did, but upgrading from old range to new range or more sophisticated methods are worth inspecting.&nbsp;<div><br></div><div>Also from our perspective: backup/restore is a good start but that works best for data/users not so much for logic/applications. So if you like us have a mix of data and apps in 2.1 you might want to tweak the backup zip so it only contains data and users. Do the applications using xar install.&nbsp;</div><div><br></div><div>Xar install might fail until pre/post-install scripts are updated for xquery 3.1. Exist-db 2 used preliminary syntax for xquery 3.0/1 which exist-db &gt;= 5 no longer supports like Pieter mentioned:</div><div><br></div><div><div style="display: block;" class=""><div style="-webkit-user-select: all; -webkit-user-drag: element; display: inline-block;" class="apple-rich-link" draggable="true" role="link" data-url="https://exist-db.org/exist/apps/doc/upgrading"><a style="border-radius:10px;font-family:-apple-system, Helvetica, Arial, sans-serif;display:block;-webkit-user-select:none;width:300px;user-select:none;-webkit-user-modify:read-only;user-modify:read-only;overflow:hidden;text-decoration:none;" class="lp-rich-link" rel="nofollow" href="https://exist-db.org/exist/apps/doc/upgrading" dir="ltr" role="button" draggable="false" width="300"><table style="table-layout:fixed;border-collapse:collapse;width:300px;background-color:#E9E9EB;font-family:-apple-system, Helvetica, Arial, sans-serif;" class="lp-rich-link-emailBaseTable" cellpadding="0" cellspacing="0" border="0" width="300"><tbody><tr><td vertical-align="center"><table bgcolor="#E9E9EB" cellpadding="0" cellspacing="0" width="300" style="font-family:-apple-system, Helvetica, Arial, sans-serif;table-layout:fixed;background-color:rgba(233, 233, 235, 1);" class="lp-rich-link-captionBar"><tbody><tr><td style="padding:8px 0px 8px 0px;" class="lp-rich-link-captionBar-textStackItem"><div style="max-width:100%;margin:0px 16px 0px 16px;overflow:hidden;" class="lp-rich-link-captionBar-textStack"><div style="word-wrap:break-word;font-weight:500;font-size:12px;overflow:hidden;text-overflow:ellipsis;text-align:left;" class="lp-rich-link-captionBar-textStack-topCaption-leading"><a rel="nofollow" href="https://exist-db.org/exist/apps/doc/upgrading" style="text-decoration: none" draggable="false"><font color="#000000" style="color: rgba(0, 0, 0, 1);">eXist-db Documentation</font></a></div><div style="word-wrap:break-word;font-weight:400;font-size:11px;overflow:hidden;text-overflow:ellipsis;text-align:left;" class="lp-rich-link-captionBar-textStack-bottomCaption-leading"><a rel="nofollow" href="https://exist-db.org/exist/apps/doc/upgrading" style="text-decoration: none" draggable="false"><font color="#A2A2A9" style="color: rgba(60, 60, 67, 0.6);">exist-db.org</font></a></div></div></td><td style="padding:6px 12px 6px 0px;" class="lp-rich-link-captionBar-rightIconItem" width="36"><a rel="nofollow" href="https://exist-db.org/exist/apps/doc/upgrading" draggable="false"><img style="pointer-events:none !important;display:inline-block;width:36px;height:36px;border-radius:3px;" width="36" height="36" draggable="false" class="lp-rich-link-captionBar-rightIcon" alt="exist_icon_16x16.ico" src="cid:A2534EA8-74A4-40E9-9F0F-0EDBD2FA28A3"></a></td></tr></tbody></table></td></tr></tbody></table></a></div></div><div><br></div>Most code issues you’ll have started with exist-db 5. The release notes outline what they are and link to the perl script</div><div><br></div><div><div style="display: block;" class=""><div style="-webkit-user-select: all; -webkit-user-drag: element; display: inline-block;" class="apple-rich-link" draggable="true" role="link" data-url="http://www.exist-db.org/exist/apps/wiki/blogs/eXist/eXistdb500"><a style="border-radius:10px;font-family:-apple-system, Helvetica, Arial, sans-serif;display:block;-webkit-user-select:none;width:300px;user-select:none;-webkit-user-modify:read-only;user-modify:read-only;overflow:hidden;text-decoration:none;" class="lp-rich-link" rel="nofollow" href="http://www.exist-db.org/exist/apps/wiki/blogs/eXist/eXistdb500" dir="ltr" role="button" draggable="false" width="300"><table style="table-layout:fixed;border-collapse:collapse;width:300px;background-color:#E9E9EB;font-family:-apple-system, Helvetica, Arial, sans-serif;" class="lp-rich-link-emailBaseTable" cellpadding="0" cellspacing="0" border="0" width="300"><tbody><tr><td vertical-align="center"><table bgcolor="#E9E9EB" cellpadding="0" cellspacing="0" width="300" style="font-family:-apple-system, Helvetica, Arial, sans-serif;table-layout:fixed;background-color:rgba(233, 233, 235, 1);" class="lp-rich-link-captionBar"><tbody><tr><td style="padding:8px 0px 8px 0px;" class="lp-rich-link-captionBar-textStackItem"><div style="max-width:100%;margin:0px 16px 0px 16px;overflow:hidden;" class="lp-rich-link-captionBar-textStack"><div style="word-wrap:break-word;font-weight:500;font-size:12px;overflow:hidden;text-overflow:ellipsis;text-align:left;" class="lp-rich-link-captionBar-textStack-topCaption-leading"><a rel="nofollow" href="http://www.exist-db.org/exist/apps/wiki/blogs/eXist/eXistdb500" style="text-decoration: none" draggable="false"><font color="#000000" style="color: rgba(0, 0, 0, 1);">Atomic Wiki</font></a></div><div style="word-wrap:break-word;font-weight:400;font-size:11px;overflow:hidden;text-overflow:ellipsis;text-align:left;" class="lp-rich-link-captionBar-textStack-bottomCaption-leading"><a rel="nofollow" href="http://www.exist-db.org/exist/apps/wiki/blogs/eXist/eXistdb500" style="text-decoration: none" draggable="false"><font color="#A2A2A9" style="color: rgba(60, 60, 67, 0.6);">exist-db.org</font></a></div></div></td><td style="padding:6px 12px 6px 0px;" class="lp-rich-link-captionBar-rightIconItem" width="36"><a rel="nofollow" href="http://www.exist-db.org/exist/apps/wiki/blogs/eXist/eXistdb500" draggable="false"><img style="pointer-events:none !important;display:inline-block;width:36px;height:36px;border-radius:3px;" width="36" height="36" draggable="false" class="lp-rich-link-captionBar-rightIcon" alt="exist_icon_16x16.ico" src="cid:95B27859-1A3B-407D-811D-3365D651DAD6"></a></td></tr></tbody></table></td></tr></tbody></table></a></div></div><div><br></div>Everything mentioned there still holds true in exist-db 6.</div><div><br></div><div>It took us over a year to update the code, rethink index, rethink code security, test and finally migrate. For UI we had Orbeon xforms inside the db that we replace with a Roaster/jwt based api and external Vue based logic. Roaster and jwt are in the public repo and work with &gt;= 5 and up. The orbeon to vue move is now 3 years in the making. They still run side by side. We found the best start in Roaster by hiring eXist people. The rest we could largely manage ourselves.&nbsp;</div><div><div><br><div dir="ltr">Verstuurd vanaf mijn iPhone</div><div dir="ltr"><br><blockquote type="cite">Op 11 feb. 2023 om 07:14 heeft Pieter Lamers &lt;pie...@be...&gt; het volgende geschreven:<br><br></blockquote></div><blockquote type="cite"><div dir="ltr">
  
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  
  
    <p>I would not follow this advice. Any new release can be
      indefinitely delayed, and I suspect that even with the promise of
      some breaking changes between 6.2 and 7 there will be much more
      common ground between those than between 2.1 and 6.2. <br>
    </p>
    <p>I would start by installing 6.2 locally and see if I can restore
      a backup onto it. Then see where the code breaks. There is a perl
      (!) script out there which can trace some now invalid constructs,
      like maps with key := val instead of&nbsp; key : val. There is a lot of
      code refactoring possible now: eXist 6.2 uses a recent version of
      Saxon which supports text value templates (and XSLT 3.0 in
      general). eXist's XQuery has a much broader coverage of the 3.1
      standard. <br>
    </p>
    <p>Probably some other community members have a more sophisticated
      approach, so let's see what the others say.</p>
    <p>Best,<br>
      Pieter<br>
    </p>
    <div class="moz-cite-prefix">On 07/02/2023 06:44, Martin Mueller
      wrote:<br>
    </div>
    <blockquote type="cite" cite="mid:EB4...@ad...">
      <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
      <meta name="Generator" content="Microsoft Word 15 (filtered
        medium)">
      <style>@font-face { font-family: "Cambria Math"; }
@font-face { font-family: Calibri; }
p.MsoNormal, li.MsoNormal, div.MsoNormal { margin: 0in; font-size: 12pt; font-family: Calibri, sans-serif; }
span.EmailStyle19 { font-family: Calibri, sans-serif; color: windowtext; }
.MsoChpDefault { font-size: 10pt; }
div.WordSection1 { page: WordSection1; }</style>
      <div class="WordSection1">
        <p class="MsoNormal"><span style="font-size:11.0pt">You might as
            well wait until the promised version 7 comes out in the
            spring.
            <o:p></o:p></span></p>
        <p class="MsoNormal"><span style="font-size:11.0pt"><o:p>&nbsp;</o:p></span></p>
        <div style="border:none;border-top:solid #B5C4DF
          1.0pt;padding:3.0pt 0in 0in 0in">
          <p class="MsoNormal"><b><span style="color:black">From: </span></b><span style="color:black">"Areki, Alem"
              <a class="moz-txt-link-rfc2396E" href="mailto:aa...@ri...">&lt;aa...@ri...&gt;</a><br>
              <b>Date: </b>Monday, February 6, 2023 at 8:23 PM<br>
              <b>To: </b><a class="moz-txt-link-rfc2396E" href="mailto:exi...@li...">"exi...@li..."</a>
              <a class="moz-txt-link-rfc2396E" href="mailto:exi...@li...">&lt;exi...@li...&gt;</a><br>
              <b>Subject: </b>[Exist-open] Upgrade eXist-db from 2.1 to
              6.2.0<o:p></o:p></span></p>
        </div>
        <div>
          <p class="MsoNormal"><span style="font-size:11.0pt"><o:p>&nbsp;</o:p></span></p>
        </div>
        <p class="MsoNormal"><span style="font-size:11.0pt">Hi eXist
            users, <o:p></o:p></span></p>
        <p class="MsoNormal"><span style="font-size:11.0pt"><o:p>&nbsp;</o:p></span></p>
        <p class="MsoNormal"><span style="font-size:11.0pt">Here at the
            University of Richmond, we still use the old eXist version -
            eXist 2.1. Now we need to upgrade to the latest one - eXist
            6.2.0; what would be your recommendation to upgrade from 2.1
            to 6.2.0?<o:p></o:p></span></p>
        <p class="MsoNormal"><span style="font-size:11.0pt"><o:p>&nbsp;</o:p></span></p>
        <p class="MsoNormal"><span style="font-size:11.0pt">Thanks<o:p></o:p></span></p>
        <p class="MsoNormal"><span style="font-size:11.0pt"><o:p>&nbsp;</o:p></span></p>
        <p class="MsoNormal"><span style="font-size:10.5pt;color:black">--&nbsp;<o:p></o:p></span></p>
        <p class="MsoNormal"><span style="color:black">Alem T. Areki<o:p></o:p></span></p>
        <p class="MsoNormal"><span style="color:black">Senior Web
            Developer – Web Services<o:p></o:p></span></p>
        <p class="MsoNormal"><span style="color:black">University of
            Richmond<o:p></o:p></span></p>
        <p class="MsoNormal"><o:p>&nbsp;</o:p></p>
      </div>
      <br>
      <fieldset class="moz-mime-attachment-header"></fieldset>
      <br>
      <fieldset class="moz-mime-attachment-header"></fieldset>
      <pre class="moz-quote-pre" wrap="">_______________________________________________
Exist-open mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Exi...@li...">Exi...@li...</a>
<a class="moz-txt-link-freetext" href="https://lists.sourceforge.net/lists/listinfo/exist-open">https://lists.sourceforge.net/lists/listinfo/exist-open</a>
</pre>
    </blockquote>
    <pre class="moz-signature" cols="72">-- 
Pieter Lamers
John Benjamins Publishing Company
Postal Address: P.O. Box 36224, 1020 ME AMSTERDAM, The Netherlands
Visiting Address: Klaprozenweg 75G, 1033 NN AMSTERDAM, The Netherlands
Warehouse: Kelvinstraat 11-13, 1446 TK PURMEREND, The Netherlands
tel: +31 20 630 4747
web: <a class="moz-txt-link-abbreviated" href="http://www.benjamins.com">www.benjamins.com</a></pre>
  

<span>_______________________________________________</span><br><span>Exist-open mailing list</span><br><span>Exi...@li...</span><br><span>https://lists.sourceforge.net/lists/listinfo/exist-open</span><br></div></blockquote></div></div></body></html>

Re: [Exist-open] Upgrade eXist-db from 2.1 to 6.2.0

[Claudius T. <cla...@gm...>]

Hi,

What if you pass from 2.1 to 3.1.0 (
https://github.com/eXist-db/exist/releases?page=5), then export and import
to 4.1.0 (https://github.com/eXist-db/exist/releases?page=4), then export
and import to 5.1.0 (https://github.com/eXist-db/exist/releases?page=2),
and finally export and import to 6.2.0 (
https://github.com/eXist-db/exist/releases?page=1)?

Good luck!

Claudius

On Sat, 11 Feb 2023 at 08:12, Pieter Lamers <pie...@be...>
wrote:

> I would not follow this advice. Any new release can be indefinitely
> delayed, and I suspect that even with the promise of some breaking changes
> between 6.2 and 7 there will be much more common ground between those than
> between 2.1 and 6.2.
>
> I would start by installing 6.2 locally and see if I can restore a backup
> onto it. Then see where the code breaks. There is a perl (!) script out
> there which can trace some now invalid constructs, like maps with key :=
> val instead of  key : val. There is a lot of code refactoring possible now:
> eXist 6.2 uses a recent version of Saxon which supports text value
> templates (and XSLT 3.0 in general). eXist's XQuery has a much broader
> coverage of the 3.1 standard.
>
> Probably some other community members have a more sophisticated approach,
> so let's see what the others say.
>
> Best,
> Pieter
> On 07/02/2023 06:44, Martin Mueller wrote:
>
> You might as well wait until the promised version 7 comes out in the
> spring.
>
>
>
> *From: *"Areki, Alem" <aa...@ri...> <aa...@ri...>
> *Date: *Monday, February 6, 2023 at 8:23 PM
> *To: *"exi...@li..."
> <exi...@li...> <exi...@li...>
> <exi...@li...>
> *Subject: *[Exist-open] Upgrade eXist-db from 2.1 to 6.2.0
>
>
>
> Hi eXist users,
>
>
>
> Here at the University of Richmond, we still use the old eXist version -
> eXist 2.1. Now we need to upgrade to the latest one - eXist 6.2.0; what
> would be your recommendation to upgrade from 2.1 to 6.2.0?
>
>
>
> Thanks
>
>
>
> --
>
> Alem T. Areki
>
> Senior Web Developer – Web Services
>
> University of Richmond
>
>
>
>
> _______________________________________________
> Exist-open mailing lis...@li...https://lists.sourceforge.net/lists/listinfo/exist-open
>
> --
> Pieter Lamers
> John Benjamins Publishing Company
> Postal Address: P.O. Box 36224, 1020 ME AMSTERDAM, The Netherlands
> Visiting Address: Klaprozenweg 75G, 1033 NN AMSTERDAM, The Netherlands
> Warehouse: Kelvinstraat 11-13, 1446 TK PURMEREND, The Netherlands
> tel: +31 20 630 4747
> web: www.benjamins.com
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>


-- 
Cu stimă,
Claudius Teodorescu

Re: [Exist-open] Upgrade eXist-db from 2.1 to 6.2.0

[Pieter L. <pie...@be...>]

I would not follow this advice. Any new release can be indefinitely 
delayed, and I suspect that even with the promise of some breaking 
changes between 6.2 and 7 there will be much more common ground between 
those than between 2.1 and 6.2.

I would start by installing 6.2 locally and see if I can restore a 
backup onto it. Then see where the code breaks. There is a perl (!) 
script out there which can trace some now invalid constructs, like maps 
with key := val instead of  key : val. There is a lot of code 
refactoring possible now: eXist 6.2 uses a recent version of Saxon which 
supports text value templates (and XSLT 3.0 in general). eXist's XQuery 
has a much broader coverage of the 3.1 standard.

Probably some other community members have a more sophisticated 
approach, so let's see what the others say.

Best,
Pieter

On 07/02/2023 06:44, Martin Mueller wrote:
>
> You might as well wait until the promised version 7 comes out in the 
> spring.
>
> *From: *"Areki, Alem" <aa...@ri...>
> *Date: *Monday, February 6, 2023 at 8:23 PM
> *To: *"exi...@li..." 
> <exi...@li...>
> *Subject: *[Exist-open] Upgrade eXist-db from 2.1 to 6.2.0
>
> Hi eXist users,
>
> Here at the University of Richmond, we still use the old eXist version 
> - eXist 2.1. Now we need to upgrade to the latest one - eXist 6.2.0; 
> what would be your recommendation to upgrade from 2.1 to 6.2.0?
>
> Thanks
>
> -- 
>
> Alem T. Areki
>
> Senior Web Developer – Web Services
>
> University of Richmond
>
>
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open

-- 
Pieter Lamers
John Benjamins Publishing Company
Postal Address: P.O. Box 36224, 1020 ME AMSTERDAM, The Netherlands
Visiting Address: Klaprozenweg 75G, 1033 NN AMSTERDAM, The Netherlands
Warehouse: Kelvinstraat 11-13, 1446 TK PURMEREND, The Netherlands
tel: +31 20 630 4747
web:www.benjamins.com

Re: [Exist-open] Upgrade eXist-db from 2.1 to 6.2.0

[Martin M. <mar...@no...>]

You might as well wait until the promised version 7 comes out in the spring.

From: "Areki, Alem" <aa...@ri...>
Date: Monday, February 6, 2023 at 8:23 PM
To: "exi...@li..." <exi...@li...>
Subject: [Exist-open] Upgrade eXist-db from 2.1 to 6.2.0

Hi eXist users,

Here at the University of Richmond, we still use the old eXist version - eXist 2.1. Now we need to upgrade to the latest one - eXist 6.2.0; what would be your recommendation to upgrade from 2.1 to 6.2.0?

Thanks

--
Alem T. Areki
Senior Web Developer – Web Services
University of Richmond

[Exist-open] Upgrade eXist-db from 2.1 to 6.2.0

[Areki, A. <aa...@ri...>]

Hi eXist users,

Here at the University of Richmond, we still use the old eXist version - eXist 2.1. Now we need to upgrade to the latest one - eXist 6.2.0; what would be your recommendation to upgrade from 2.1 to 6.2.0?

Thanks

--
Alem T. Areki
Senior Web Developer – Web Services
University of Richmond

Re: [Exist-open] [Ext] [ANN] eXist-db 6.2.0 released

[Thomason, D. B <dth...@ut...>]

Adam, our deepest sympathies on the loss of your mother. Celebrate her life on her birthday, and every day! Don


Donald B. Thomason, Ph.D.
Dean and Professor 
The University of Tennessee Health Science Center
College of Graduate Health Sciences
Suite 807, 920 Madison Ave
Memphis, TN 38163 
dth...@ut...
t: +1 901.448.7224
https://grad.uthsc.edu <https://grad.uthsc.edu/> 



On 2/4/23, 5:34 PM, "Adam Retter" <ad...@ex... <mailto:ad...@ex...>> wrote:


Dear eXist-db users,


I am very happy to let you know that eXist-db 6.2.0 has just been
released. It is a small incremental improvement over 6.1.0 that fixes
a couple of regressions that were discovered almost immediately after
the 6.1.0 release.


On a personal note, I would like to dedicate this release to my mother
Christine McCrorie who recently passed away. Christine supported my
work on eXist-db, not least of all by helping to proof read and edit
my text for the O'Reilly eXist book. Today would have been our
birthday. Happy Birthday Mum!.


The full release notes and downloads can be found here:
https://github.com/eXist-db/exist/releases/tag/eXist-6.2.0 <https://github.com/eXist-db/exist/releases/tag/eXist-6.2.0>


Kind regards. Adam.


-- 
Adam Retter


eXist Core Developer
{ United Kingdom }
ad...@ex... <mailto:ad...@ex...>




_______________________________________________
Exist-open mailing list
Exi...@li... <mailto:Exi...@li...>
https://lists.sourceforge.net/lists/listinfo/exist-open <https://lists.sourceforge.net/lists/listinfo/exist-open>

[Exist-open] [ANN] eXist-db 6.2.0 released

[Adam R. <ad...@ex...>]

Dear eXist-db users,

I am very happy to let you know that eXist-db 6.2.0 has just been
released. It is a small incremental improvement over 6.1.0 that fixes
a couple of regressions that were discovered almost immediately after
the 6.1.0 release.

On a personal note, I would like to dedicate this release to my mother
Christine McCrorie who recently passed away. Christine supported my
work on eXist-db, not least of all by helping to proof read and edit
my text for the O'Reilly eXist book. Today would have been our
birthday. Happy Birthday Mum!.

The full release notes and downloads can be found here:
https://github.com/eXist-db/exist/releases/tag/eXist-6.2.0

Kind regards. Adam.

-- 
Adam Retter

eXist Core Developer
{ United Kingdom }
ad...@ex...

Re: [Exist-open] a very primitive question

[Alasdair D. <ala...@gm...>]

Hi Martin,

It seems you have got the answers from the forum members.  If you permission, have locate running on the box.  I find it helps a lot in locating where things are located, as the name suggests.

Good luck with eXist and never be shy in asking “obvious” questions, because if you need to ask it is not obvious to you.

Alasdair 

Sent from my iPhone

> On 4 Feb 2023, at 12:55 am, Martin Mueller <mar...@no...> wrote:
> 
> 
> Where on a Linux box is exist usually store, and how do you look for it?  This is a very primitive question by somebody who hasn’t yet mastered the complexities of the “find” command.
>  
> I couldn’t find it in the eXist documentation, but I probably didn’t  look closely enough
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open

Re: [Exist-open] a very primitive question

[Eduard D. <ed...@fr...>]

Hi, I'm not aware of any linux package for exist-db, so you'll find it where you install it or where you have it installed for you. Perhaps I do not understand your question....




Eduard Drenth, Software Architekt


ed...@fr...


Doelestrjitte 8

8911 DX  Ljouwert

(058) 213 14 14

+31 62 094 34 28

https://www.fryske-akademy.nl/
https://frysker.nl/
https://frisian.eu/
[cid:d0e64df6-8dd8-47ca-b044-c08fc55cf051]

gpg: pgp.key-server.io/pks/lookup?search=eduarddrenth

________________________________
From: Martin Mueller <mar...@no...>
Sent: Friday, February 3, 2023 3:29 PM
To: Exist-open <exi...@li...>
Subject: [Exist-open] a very primitive question


Where on a Linux box is exist usually store, and how do you look for it?  This is a very primitive question by somebody who hasn’t yet mastered the complexities of the “find” command.



I couldn’t find it in the eXist documentation, but I probably didn’t  look closely enough