exist-open — Open discussion

Re: [Exist-open] Problem with Fore's undefined attributes in eXide

[Claudius T. <cla...@gm...>]

*single space

On Wed, 24 Sep 2025, 13:42 Claudius Teodorescu, <
cla...@gm...> wrote:

> You can also use
>
> /**/
>
> Instead of empty space.
>
> Claudius
>
> On Wed, 24 Sep 2025, 12:45 Henrik Nielsen, <hen...@gm...> wrote:
>
>> Thank you all. I did include the Javascript. To avoid tags being made
>> self-closing when elements are empty I believe I have found a workaround:
>> putting a single space in the otherwise empty element seems to be accepted
>> without subsequent modification by eXist. With regard to the Fore issue
>> itself, I have made a minimal test case that works as it should, so I am
>> building on that instead of what I had previously written.
>>
>> Thanks again,
>>
>> Henrik Thiil NIelsen
>> _______________________________________________
>> Exist-open mailing list
>> Exi...@li...
>> https://lists.sourceforge.net/lists/listinfo/exist-open
>>
>

Re: [Exist-open] Problem with Fore's undefined attributes in eXide

[Claudius T. <cla...@gm...>]

You can also use

/**/

Instead of empty space.

Claudius

On Wed, 24 Sep 2025, 12:45 Henrik Nielsen, <hen...@gm...> wrote:

> Thank you all. I did include the Javascript. To avoid tags being made
> self-closing when elements are empty I believe I have found a workaround:
> putting a single space in the otherwise empty element seems to be accepted
> without subsequent modification by eXist. With regard to the Fore issue
> itself, I have made a minimal test case that works as it should, so I am
> building on that instead of what I had previously written.
>
> Thanks again,
>
> Henrik Thiil NIelsen
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>

Re: [Exist-open] Problem with Fore's undefined attributes in eXide

[Jean-Paul R. <re...@gm...>]

Dear Henrik,

eXist can handle HTML documents in one of two ways:
1. if we want the HTML parsed as nodes and query-able or usable as
templates, it must be XML compliant in format (important: this has nothing
to do with XHTML namespace)
2. otherwise HTML file must be stored as a binary document...in which case
it is no longer nodes and no longer query-able ad it would have to be
streamed to the browser as such

Option 1 means you would have <fx-fore create-nodes=""></fx-fore> in your
document. You can then output it using the following declarations in your
query prolog (i.e. serialization):

declare namespace output = "http://www.w3.org/2010/xslt-xquery-serialization
";

declare option output:method "html5";
declare option output:media-type "text/html";

Your html will appear in the browser as <fx-fore create-nodes>.


HTH,
JPR

On Tue, Sep 23, 2025 at 5:23 PM Henrik Nielsen <hen...@gm...>
wrote:

> I have just started using Fore with exist.db and have run into the problem
> that eXide will not accept undefined attributes in Fore's custom elements.
> So "<fx-fore create-nodes>" (without quotation marks in my page) is not
> treated as a valid opening tag since the attribute "create-nodes" is
> expected to be given a value. Does exist.db parse HTML as XML, disregarding
> both the .html extension and the valid doctype declaration ("<!DOCTYPE
> html>")? Or have I made some stupid mistake?
>
> The list archive was offline and I didn't find anything relevant at Stack
> Overflow.
>
> Henrik Thiil Nielsen
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>

[Exist-open] Problem with Fore's undefined attributes in eXide

[Henrik N. <hen...@gm...>]

Thank you all. I did include the Javascript. To avoid tags being made
self-closing when elements are empty I believe I have found a workaround:
putting a single space in the otherwise empty element seems to be accepted
without subsequent modification by eXist. With regard to the Fore issue
itself, I have made a minimal test case that works as it should, so I am
building on that instead of what I had previously written.

Thanks again,

Henrik Thiil NIelsen

Re: [Exist-open] Problem with Fore's undefined attributes in eXide

[aik01 <ai...@un...>]

Afaik, eXide edits XHTML in your case (see https://www.w3schools.com/html/html_xhtml.asp).
Therefore attribute minimization is not allowed. Beware of selfclosing tags esp in instances because Fore itself will internally parsing your text as html.
The handling of mimetypes within eXistDB is an other thing.  

Peter

On Sunday, September 21, 2025 22:13 CEST, Henrik Nielsen <hen...@gm...> wrote:

> I have just started using Fore with exist.db and have run into the problem
> that eXide will not accept undefined attributes in Fore's custom elements.
> So "<fx-fore create-nodes>" (without quotation marks in my page) is not
> treated as a valid opening tag since the attribute "create-nodes" is
> expected to be given a value. Does exist.db parse HTML as XML, disregarding
> both the .html extension and the valid doctype declaration ("<!DOCTYPE
> html>")? Or have I made some stupid mistake?
> 
> The list archive was offline and I didn't find anything relevant at Stack
> Overflow
> 
> Henrik Thiil Nielsen

Re: [Exist-open] Problem with Fore's undefined attributes in eXide

[Michael W. <wes...@ja...>]

Hi Henrik,

Are you including the required JavaScript? Based on the documentation on
their GitHub page, the `fx-fore` (and other `fx-` tags) appear to be
processed in the JavaScript on the browser. Any results of these processes
will show up in the browser, not in the eXist database without you
explicitly sending it there.

Hope this helps.

Take care.

2025年9月24日(水) 0:23 Henrik Nielsen <hen...@gm...>:

> I have just started using Fore with exist.db and have run into the problem
> that eXide will not accept undefined attributes in Fore's custom elements.
> So "<fx-fore create-nodes>" (without quotation marks in my page) is not
> treated as a valid opening tag since the attribute "create-nodes" is
> expected to be given a value. Does exist.db parse HTML as XML, disregarding
> both the .html extension and the valid doctype declaration ("<!DOCTYPE
> html>")? Or have I made some stupid mistake?
>
> The list archive was offline and I didn't find anything relevant at Stack
> Overflow.
>
> Henrik Thiil Nielsen
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>


-- 
Michael Westbay
Writer/System Administrator
http://www.japanesebaseball.com/

Re: [Exist-open] Problem with Fore's undefined attributes in eXide

[Julian J. <jul...@ad...>]

I’m quite sure eXist only parses XML (and therefore, by extension, 
XHTML). A question based on that might be whether eXist should react to 
"<!DOCTYPE html>" by not trying to parse it? But this is off the top of 
my hat.

Best,
Julian

Am 21.09.25 um 22:13 schrieb Henrik Nielsen:
> I have just started using Fore with exist.db and have run into the 
> problem that eXide will not accept undefined attributes in Fore's 
> custom elements. So "<fx-fore create-nodes>" (without quotation marks 
> in my page) is not treated as a valid opening tag since the attribute 
> "create-nodes" is expected to be given a value. Does exist.db parse 
> HTML as XML, disregarding both the .html extension and the valid 
> doctype declaration ("<!DOCTYPE html>")? Or have I made some stupid 
> mistake?
>
> The list archive was offline and I didn't find anything relevant at 
> Stack Overflow.
>
> Henrik Thiil Nielsen
>
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open

[Exist-open] Problem with Fore's undefined attributes in eXide

[Henrik N. <hen...@gm...>]

I have just started using Fore with exist.db and have run into the problem
that eXide will not accept undefined attributes in Fore's custom elements.
So "<fx-fore create-nodes>" (without quotation marks in my page) is not
treated as a valid opening tag since the attribute "create-nodes" is
expected to be given a value. Does exist.db parse HTML as XML, disregarding
both the .html extension and the valid doctype declaration ("<!DOCTYPE
html>")? Or have I made some stupid mistake?

The list archive was offline and I didn't find anything relevant at Stack
Overflow.

Henrik Thiil Nielsen

Re: [Exist-open] Listing documents owned by user

[Alberto S. <am...@zb...>]

Hi, Michael

Thanks, I need to do that programmatically.
The collection has more than 100k documents :-)

Will try your proposal, thanks

Alberto Simões

On Wednesday, August 6th, 2025 at 01:19, Michael Westbay <wes...@ja...> wrote:

> Hi Alberto,
>
> The easiest way is to use the "File" → "Manage" menu item in eXide. That gives you a Finder/File Explorer dialog of all of your collections.
>
> Programmatically, sm:get-permissions(uri) (import module namespace sm="http://exist-db.org/xquery/securitymanager";) will get the owner, group, and mode of a given resource.
>
> Hope this helps.
>
> Take care.
>
> 2025年8月6日(水) 1:00 Alberto Simões <am...@zb...>:
>
>> Hi
>>
>> Is there a way to list all documents in a collection owned by a specific user?
>>
>> Thanks
>>
>> Alberto Simões
>>
>> _______________________________________________
>> Exist-open mailing list
>> Exi...@li...
>> https://lists.sourceforge.net/lists/listinfo/exist-open
>
> --
>
> Michael Westbay
> Writer/System Administrator
> http://www.japanesebaseball.com/

Re: [Exist-open] Listing documents owned by user

[Juri L. <ju...@ex...>]

On 04.08.25 16:05, Alberto Simões wrote:
> Is there a way to list all documents in a collection owned by a specific user?
>
If I would have to do that:

Search for all resources owned by a specific user in an exist-db instance.


I would use the command line client xst (see 
https://www.npmjs.com/package/@existdb/xst)

With the necessary environment variables set to connect to the database 
you wish to query you can call:

   xst ls /db --long --recursive

This will list all collections and resources, the connecting user has 
access to.

Piping the output into grep allows then to filter by a user name (here 
eXide)

   xst ls /db --long --recursive | grep eXide

Since you want to limit output to a certain collection

   xst ls /db/my/collection --long --recursive | grep eXide



Hope this helps,
Juri Leino

Re: [Exist-open] Listing documents owned by user

[Michael W. <wes...@ja...>]

Hi Alberto,

The easiest way is to use the "File" → "Manage" menu item in eXide. That
gives you a Finder/File Explorer dialog of all of your collections.

Programmatically, sm:get-permissions(uri) (import module namespace sm="
http://exist-db.org/xquery/securitymanager";)  will get the owner, group,
and mode of a given resource.

Hope this helps.

Take care.

2025年8月6日(水) 1:00 Alberto Simões <am...@zb...>:

> Hi
>
> Is there a way to list all documents in a collection owned by a specific
> user?
>
> Thanks
>
>
>
> Alberto Simões
>
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>


-- 
Michael Westbay
Writer/System Administrator
http://www.japanesebaseball.com/

[Exist-open] Listing documents owned by user

[Alberto S. <am...@zb...>]

Hi

Is there a way to list all documents in a collection owned by a specific user?

Thanks



Alberto Simões

Re: [Exist-open] "Collection /db/system/security/exist/accounts cannot be created."

[Michael W. <wes...@ja...>]

Hi Tony,

Okay, the "Failed to open database" message concerns me.

under /usr/local/src/exist3-data do you have the following files?

-rw-rw-r--.  1 exist exist       1451 Jul 25 08:57 00000000d6.log
-rw-rw-r--.  1 exist exist   13991936 Jul 24 22:40 collections.dbx
-rw-r--r--.  1 exist exist       1515 Jul 24 22:38 counters
-rw-rw-r--.  1 exist exist         16 Jul 25 08:57 dbx_dir.lck
-rw-rw-r--.  1 exist exist 6785642496 Jul 24 22:38 dom.dbx
drwxrwxr-x. 15 exist exist       4096 Sep 27  2020 expathrepo
drwxrwxr-x.  3 exist exist        132 May 20 12:14 export
drwxrwxr-x.  3 exist exist         16 Apr  3  2018 fs
drwxrwxr-x.  2 exist exist          6 Jul 21 20:47 fs.journal
-rw-rw-r--.  1 exist exist        140 Apr 16  2018 jmxservlet.token
-rw-rw-r--.  1 exist exist         16 Jul 25 08:57 journal.lck
drwxrwxr-x.  2 exist exist        103 Jul 23  2022 lucene
-rw-rw-r--.  1 exist exist    4636672 May 22 21:46 ngram.dbx
drwxrwxr-x.  2 exist exist       8192 Jul 25 08:44 range
-rw-rw-r--.  1 exist exist        120 Apr  9  2018 README
-rw-rw-r--.  1 exist exist        257 Apr  9  2018 RECOVERY
-rw-------.  1 exist exist         11 Mar  8 10:20 restxq.registry
drwxrwxr-x.  2 exist exist         42 Jul 23  2022 sanity
-rw-rw-r--.  1 exist exist       8192 Jul 23  2022 sort.dbx
-rw-rw-r--.  1 exist exist 6046371840 Jul 24 22:40 structure.dbx
-rw-rw-r--.  1 exist exist      63342 Jul 21 20:35 symbols.dbx
-rw-rw-r--.  1 exist exist   46583808 Jul 24 22:40 values.dbx

The dom.dbx file should be very large. That's where all of your data is at.
If this file is
broken, then the data isn't recoverable.

The RECOVERY file states:

RECOVERY
========
If the index got corrupted, this is how to recover:

- stop database
- remove all ".log" transaction log files
- remove all ".lck" lock files
- remove all ".dbx" files *EXCEPT dom.dbx, collections.dbx and symbols.dbx*
- start database

I really don't like doing it without a backup. So first, make a tarball
of /usr/local/src/exist3-data
as a backup of the raw data then start the recovery process above.

And once the database is up and running again, run:

cd /usr/local/src/exist3
bin/backup.sh --dir ~/tmp/2025-07-25-backup --backup /db

And this is my script modified for your setup that I run daily via crontab:

#!/bin/sh
unset JAVA_OPTS
export JAVA_HOME=/usr
export EXIST_HOME=/usr/local/src/exist3
export TARGET=/tmp
export DATE=`date +"%Y-%m-%d"`
export PATH=$JAVA_HOME/bin:/usr/local/bin:/usr/bin:/bin
cd $EXIST_HOME
bin/backup.sh --backup /db --dir $TARGET/full-$DATE -u "admin" -p "password"

I thought it was supposed to get the username and password from the
$EXIST_HOME/backup.properties
file, but when I tested it by hand just now without the -u/-p parameters it
failed. Not sure why.

Anyway, daily backups going forward would be a very good idea.

Take care.

2025年7月24日(木) 23:21 Tony Graham <tg...@an...>:

> Michael,
>
> Thanks for following through with this.
>
> On 23/07/2025 09:00, Michael Westbay wrote:
> > Okay. Let's first find EXIST_HOME by finding where the conf.xml file is.
> >
> > find /usr/local -name conf.xml
>
> It found multiple, but the ones of interest are:
>
> /usr/local/src/exist3/installer/conf.xml
> /usr/local/src/exist3/conf.xml
>
> > If /usr/local is EXIST_HOME, then run.sh will be in /usr/local/bin along
> > with all of the other eXist executables.
>
> The only things in '/usr/local/bin' are 'tracd' and 'trac-admin' for use
> with Trac. (Still the best issue tracking/wiki system.)
>
> > This conf.xml file will then specify where the data directory is:
>
> '/usr/local/src/exist3/conf.xml' has:
>
>      <db-connection cacheSize="256M" checkMaxCacheSize="true"
> collectionCache="64M" database="native"
>          files="/usr/local/src/exist3-data" pageSize="4096"
> nodesBuffer="1000" cacheShrinkThreshold="10000"
>          doc-ids="default" minDiskSpace="128M">
>
> > You could also search for run.sh the same way:
> >
> > find /usr/local -name run.sh
>
> As before, I don't have a 'run.sh' (unless you count 'run.sh' for
> Antenna House Formatter).
>
> Running the Java installer as a non-root user has never given me a
> 'run.sh'.
>
> I downloaded the 'run.sh' [1] for that eXist-db version from GitHub,
> edited it to use the same Java alternative as used for the database, ran
> it, and got:
>
> ----
> cd "/usr/local/src/exist3/" ; ./bin/run.sh org.exist.backup.ExportMain
> -x -d /mnt/tmp
> ERROR: Failed to open database:
> org.exist.storage.BrokerPoolServiceException: org.exist.EXistException:
> java.io.IOException: Collection /db/system/security/exist/accounts
> cannot be created.
> ----
>
> If this can be solved, the problem then becomes that we don't know how
> many other collections will have similar problems, so I'm not hopeful.
>
> Regards,
>
>
> Tony Graham.
> --
> Senior Architect
> XML Division
> Antenna House, Inc.
> ----
> Skerries, Ireland
> tg...@an...
>
> [1]
>
> https://raw.githubusercontent.com/eXist-db/exist/refs/tags/eXist-3.0.RC2/bin/run.sh
>
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>


-- 
Michael Westbay
Writer/System Administrator
http://www.japanesebaseball.com/

Re: [Exist-open] "Collection /db/system/security/exist/accounts cannot be created."

[Tony G. <tg...@an...>]

Michael,

Thanks for following through with this.

On 23/07/2025 09:00, Michael Westbay wrote:
> Okay. Let's first find EXIST_HOME by finding where the conf.xml file is.
> 
> find /usr/local -name conf.xml

It found multiple, but the ones of interest are:

/usr/local/src/exist3/installer/conf.xml
/usr/local/src/exist3/conf.xml

> If /usr/local is EXIST_HOME, then run.sh will be in /usr/local/bin along 
> with all of the other eXist executables.

The only things in '/usr/local/bin' are 'tracd' and 'trac-admin' for use
with Trac. (Still the best issue tracking/wiki system.)

> This conf.xml file will then specify where the data directory is:

'/usr/local/src/exist3/conf.xml' has:

     <db-connection cacheSize="256M" checkMaxCacheSize="true" 
collectionCache="64M" database="native"
         files="/usr/local/src/exist3-data" pageSize="4096" 
nodesBuffer="1000" cacheShrinkThreshold="10000"
         doc-ids="default" minDiskSpace="128M">

> You could also search for run.sh the same way:
> 
> find /usr/local -name run.sh

As before, I don't have a 'run.sh' (unless you count 'run.sh' for
Antenna House Formatter).

Running the Java installer as a non-root user has never given me a 'run.sh'.

I downloaded the 'run.sh' [1] for that eXist-db version from GitHub,
edited it to use the same Java alternative as used for the database, ran
it, and got:

----
cd "/usr/local/src/exist3/" ; ./bin/run.sh org.exist.backup.ExportMain 
-x -d /mnt/tmp
ERROR: Failed to open database: 
org.exist.storage.BrokerPoolServiceException: org.exist.EXistException: 
java.io.IOException: Collection /db/system/security/exist/accounts 
cannot be created.
----

If this can be solved, the problem then becomes that we don't know how
many other collections will have similar problems, so I'm not hopeful.

Regards,


Tony Graham.
-- 
Senior Architect
XML Division
Antenna House, Inc.
----
Skerries, Ireland
tg...@an...

[1] 
https://raw.githubusercontent.com/eXist-db/exist/refs/tags/eXist-3.0.RC2/bin/run.sh

Re: [Exist-open] "Collection /db/system/security/exist/accounts cannot be created."

[Michael W. <wes...@ja...>]

Hi Tony,

Okay. Let's first find EXIST_HOME by finding where the conf.xml file is.

find /usr/local -name conf.xml

If /usr/local is EXIST_HOME, then run.sh will be in /usr/local/bin along
with all of the other eXist executables.

This conf.xml file will then specify where the data directory is:

    <db-connection cacheSize="256M" checkMaxCacheSize="true"
collectionCache="64M" database="native"
        files="*webapp/WEB-INF/data*" pageSize="4096" nodesBuffer="1000"
cacheShrinkThreshold="10000"
        doc-ids="default" minDiskSpace="128M">

[Emphasis added]

You could also search for run.sh the same way:

find /usr/local -name run.sh

Once we know where the executables are, let's go from there.

Take care.


> On Linux and installed from the installer jar.  I don't have 'run.sh',



-- 
Michael Westbay
Writer/System Administrator
http://www.japanesebaseball.com/

Re: [Exist-open] "Collection /db/system/security/exist/accounts cannot be created."

[Tony G. <tg...@an...>]

Michael,

On 22/07/2025 12:25, Michael Westbay wrote:
>     Thanks, but still no luck. I don't have a 'run.sh'. [1] 
> 
> 
> That's odd. I have eXist 3.x and the EXIST_HOME/bin directory has the 
> following:
> 
> backup.bat  batch.d     client.sh   eXist_32x32.ico  run.bat  server.bat 
>   shutdown.bat  startup.bat
> backup.sh   client.bat  deprecated  functions.d *run.sh*   server.sh   
> shutdown.sh   startup.sh

On Linux and installed from the installer jar.  I don't have 'run.sh',
nor do I have any '.bat' files.

Similarly with exist6 installed from installer jar.

> These shell scripts make sure that all of the CLASS_PATH stuff gets 
> setup properly for running the utilities like
> `org.exist.backup.ExportMain`.

I don't doubt it, but I don't have it.  The next thing to try will be to
use a different installer to see if that comes with 'run.sh'.

...
>     I got multiple messages of the form:
> 
>     INFO: None of content dirs exist:
>     '/usr/local/src/exist3-data/expathrepo/doc-0.4.7/content' and
>     '/usr/local/src/exist3-data/expathrepo/doc-0.4.7/doc'
> 
> Are you trying to execute from the build directory? Did you not run the 
> installer to place it all in a
> user's directory? Are you running it from a container and pointing to a 
> data directory on the host
> machine?

I have '/usr/local/src/exist3-data/expathrepo/doc-0.4.7/' but not the
subdirectories.  That directory was last modified in 2022, so the absent
directories are not a by-product of my current problem after all.

>     then I got:
> 
>     17:09:17.550 [main] ERROR org.exist.storage.DBBroker - no data found at
>     tid 25; page 60995
>     java.io.IOException: no data found at tid 25; page 60995
>              at
>     org.exist.storage.index.BFile.getAsStream(BFile.java:492) ~[exist.jar:?]
>              at
>     org.exist.storage.index.BFile.getAsStream(BFile.java:462) ~[exist.jar:?]
>              at
>     org.exist.storage.NativeBroker.openCollection(NativeBroker.java:928)
>     [exist.jar:?]
>     ...
>     17:09:17.554 [main] ERROR org.exist.storage.DBBroker - Collection
>     '/db/system/security/exist/accounts' found in subCollections set but is
>     missing from collections.dbx!
>     17:09:17.569 [main] ERROR org.exist.storage.BrokerPools - Unable to
>     initialize database instance 'exist':
>     org.exist.storage.BrokerPoolServiceException: org.exist.EXistException:
>     java.io.IOException: Collection /db/system/security/exist/accounts
>     cannot be created.
> 
> 
> Before doing anything -- like bringing the eXist server back up, you 
> want to make an emergency
> backup. Be sure to make the target a local directory that exists and 
> that you have write permission
> for. I use "~/backup/date-reason" (where "date" is the date that it shut 
> down and "reason" is "locked"
> or "out-of-memory"). The "backup" directory exists in my exist user's 
> home directory.

I don't have an exist user.  The stack trace is from running
'org.exist.backup.ExportMain', not from trying to bring the server back up.

> /usr/local/* is not usually writable by ordinary users.

I am the only user, so both ordinary and extraordinary (and still 'sudo'
to do things as 'root').

Putting things in '/usr/local/src' is a habit that I picked up over
thirty years ago with multiple people working on projects on a SunOS
machine.  Still seems wrong to put projects under my home directory even
when I am the only user.

> Let's start by locating where the EXIST_HOME/bin directory is.

I installed in '/usr/local/src/exist3', with data in '/usr/local/src/
exist3-data'.

Regards,


Tony Graham.
-- 
Senior Architect
XML Division
Antenna House, Inc.
----
Skerries, Ireland
tg...@an...

Re: [Exist-open] "Collection /db/system/security/exist/accounts cannot be created."

[Michael W. <wes...@ja...>]

Hi Tony,

Thanks, but still no luck. I don't have a 'run.sh'. [1]


That's odd. I have eXist 3.x and the EXIST_HOME/bin directory has the
following:

backup.bat  batch.d     client.sh   eXist_32x32.ico  run.bat  server.bat
 shutdown.bat  startup.bat
backup.sh   client.bat  deprecated  functions.d      *run.sh*   server.sh
shutdown.sh   startup.sh

These shell scripts make sure that all of the CLASS_PATH stuff gets setup
properly for running the
utilities like `org.exist.backup.ExportMain`.

If you're on Windows, you would use `run.bat`.


> After I made a
> script that no longer bombed out from missing dependencies, I ran
> 'org.exist.backup.ExportMain'.
>
> I got multiple messages of the form:
>
> INFO: None of content dirs exist:
> '/usr/local/src/exist3-data/expathrepo/doc-0.4.7/content' and
> '/usr/local/src/exist3-data/expathrepo/doc-0.4.7/doc'
>

Are you trying to execute from the build directory? Did you not run the
installer to place it all in a
user's directory? Are you running it from a container and pointing to a
data directory on the host
machine?


> then I got:
>
> 17:09:17.550 [main] ERROR org.exist.storage.DBBroker - no data found at
> tid 25; page 60995
> java.io.IOException: no data found at tid 25; page 60995
>         at org.exist.storage.index.BFile.getAsStream(BFile.java:492)
> ~[exist.jar:?]
>         at org.exist.storage.index.BFile.getAsStream(BFile.java:462)
> ~[exist.jar:?]
>         at
> org.exist.storage.NativeBroker.openCollection(NativeBroker.java:928)
> [exist.jar:?]
> ...
> 17:09:17.554 [main] ERROR org.exist.storage.DBBroker - Collection
> '/db/system/security/exist/accounts' found in subCollections set but is
> missing from collections.dbx!
> 17:09:17.569 [main] ERROR org.exist.storage.BrokerPools - Unable to
> initialize database instance 'exist':
> org.exist.storage.BrokerPoolServiceException: org.exist.EXistException:
> java.io.IOException: Collection /db/system/security/exist/accounts
> cannot be created.


Before doing anything -- like bringing the eXist server back up, you want
to make an emergency
backup. Be sure to make the target a local directory that exists and that
you have write permission
for. I use "~/backup/date-reason" (where "date" is the date that it shut
down and "reason" is "locked"
or "out-of-memory"). The "backup" directory exists in my exist user's home
directory.

/usr/local/* is not usually writable by ordinary users.

Let's start by locating where the EXIST_HOME/bin directory is.

-- 
Michael Westbay
Writer/System Administrator
http://www.japanesebaseball.com/

Re: [Exist-open] "Collection /db/system/security/exist/accounts cannot be created."

[Tony G. <tg...@an...>]

On 20/07/2025 06:25, Michael Westbay wrote:
> At this point, I do a full emergency backup and restore.
> 
>  From the EXIST_HOME directory, run:
> 
> bin/run.sh org.exist.backup.ExportMain -x -d path/to/backup/file

Thanks, but still no luck. I don't have a 'run.sh'. [1]  After I made a
script that no longer bombed out from missing dependencies, I ran
'org.exist.backup.ExportMain'.

I got multiple messages of the form:

INFO: None of content dirs exist: 
'/usr/local/src/exist3-data/expathrepo/doc-0.4.7/content' and 
'/usr/local/src/exist3-data/expathrepo/doc-0.4.7/doc'

then I got:

17:09:17.550 [main] ERROR org.exist.storage.DBBroker - no data found at 
tid 25; page 60995
java.io.IOException: no data found at tid 25; page 60995
	at org.exist.storage.index.BFile.getAsStream(BFile.java:492) ~[exist.jar:?]
	at org.exist.storage.index.BFile.getAsStream(BFile.java:462) ~[exist.jar:?]
	at org.exist.storage.NativeBroker.openCollection(NativeBroker.java:928) 
[exist.jar:?]
...
17:09:17.554 [main] ERROR org.exist.storage.DBBroker - Collection 
'/db/system/security/exist/accounts' found in subCollections set but is 
missing from collections.dbx!
17:09:17.569 [main] ERROR org.exist.storage.BrokerPools - Unable to 
initialize database instance 'exist': 
org.exist.storage.BrokerPoolServiceException: org.exist.EXistException: 
java.io.IOException: Collection /db/system/security/exist/accounts 
cannot be created.

Regards,


Tony Graham.
-- 
Senior Architect
XML Division
Antenna House, Inc.
----
Skerries, Ireland
tg...@an...

[1] The only reference to it that I found was an email from you to this 
list in 2021.

Re: [Exist-open] "Collection /db/system/security/exist/accounts cannot be created."

[Michael W. <wes...@ja...>]

Hi Tony,

Oh, and if you rely on any counters, once the new "data" directory has been
created, be sure to copy the old "counters" file into it. It's a pain to
fix counter IDs after they've been assigned a second time.

Take care.

2025年7月20日(日) 14:25 Michael Westbay <wes...@ja...>:

> Hi Tony,
>
> At this point, I do a full emergency backup and restore.
>
> From the EXIST_HOME directory, run:
>
> bin/run.sh org.exist.backup.ExportMain -x -d path/to/backup/file
>
> The target filename is usually the date and what went wrong, like
> "2025-07-20-out-of-space."
>
> After making the backup, mv the EXIST_HOME/webapp/WEB-INF/data directory
> elsewhere, then start up eXist.
>
> Next, extract the emergency backup and cd to db and run:
>
> ${EXIST_HOME}/bin/backup.sh -r `pwd`/system/security/__contents__.xml -u
> admin -P "new_password"
>
> Then run:
>
> ${EXIST_HOME}/bin/backup.sh -r `pwd`/__contents__.xml -u admin -p
> "new_password"
>
> The first restore will restore the security collection and set the new
> password for admin.
>
> The second restore will restore the rest of your database using the new
> password for admin.
>
> Finally, I have two scripts that I always run after a restore to restore
> the repos that I use and restore permissions to certain collections.
>
>
> xquery version "3.0";
>
> <fix-repo>{(
>   repo:remove("http://exist-db.org/apps/some_app"),
>   repo:install-and-deploy("http://exist-db.org/apps/shared", "1.2.3", "
> http://exist-db.org/exist/apps/public-repo/some_app"),
>   ...
> )}</fix-repo>
>
> Hope this helps.
>
> Take care.
>
>
>
> 2025年7月20日(日) 11:51 Tony Graham <tg...@an...>:
>
>> I have an eXist-3 (yes, eXist-3) that won't respond and won't shutdown
>> (short of killing the process).  Starting it devolves into multiple
>> stack traces with the message in the subject.  Is there any hope for it?
>>
>> I got here after killing eXist after a server backup filled the
>> filesystem and eXist became unresponsive.  (The backup was because a
>> Jenkins job running an Ant task had created some ghost collections that
>> showed only in the Java client and that the 'admin' user couldn't
>> delete, and I was going to tweak then reload the backup.)
>>
>> Regards,
>>
>>
>> Tony Graham.
>> --
>> Senior Architect
>> XML Division
>> Antenna House, Inc.
>> ----
>> Skerries, Ireland
>> tg...@an...
>>
>>
>>
>> _______________________________________________
>> Exist-open mailing list
>> Exi...@li...
>> https://lists.sourceforge.net/lists/listinfo/exist-open
>>
>
>
> --
> Michael Westbay
> Writer/System Administrator
> http://www.japanesebaseball.com/
>


-- 
Michael Westbay
Writer/System Administrator
http://www.japanesebaseball.com/

Re: [Exist-open] "Collection /db/system/security/exist/accounts cannot be created."

[Michael W. <wes...@ja...>]

Hi Tony,

At this point, I do a full emergency backup and restore.

>From the EXIST_HOME directory, run:

bin/run.sh org.exist.backup.ExportMain -x -d path/to/backup/file

The target filename is usually the date and what went wrong, like
"2025-07-20-out-of-space."

After making the backup, mv the EXIST_HOME/webapp/WEB-INF/data directory
elsewhere, then start up eXist.

Next, extract the emergency backup and cd to db and run:

${EXIST_HOME}/bin/backup.sh -r `pwd`/system/security/__contents__.xml -u
admin -P "new_password"

Then run:

${EXIST_HOME}/bin/backup.sh -r `pwd`/__contents__.xml -u admin -p
"new_password"

The first restore will restore the security collection and set the new
password for admin.

The second restore will restore the rest of your database using the new
password for admin.

Finally, I have two scripts that I always run after a restore to restore
the repos that I use and restore permissions to certain collections.


xquery version "3.0";

<fix-repo>{(
  repo:remove("http://exist-db.org/apps/some_app"),
  repo:install-and-deploy("http://exist-db.org/apps/shared", "1.2.3", "
http://exist-db.org/exist/apps/public-repo/some_app"),
  ...
)}</fix-repo>

Hope this helps.

Take care.



2025年7月20日(日) 11:51 Tony Graham <tg...@an...>:

> I have an eXist-3 (yes, eXist-3) that won't respond and won't shutdown
> (short of killing the process).  Starting it devolves into multiple
> stack traces with the message in the subject.  Is there any hope for it?
>
> I got here after killing eXist after a server backup filled the
> filesystem and eXist became unresponsive.  (The backup was because a
> Jenkins job running an Ant task had created some ghost collections that
> showed only in the Java client and that the 'admin' user couldn't
> delete, and I was going to tweak then reload the backup.)
>
> Regards,
>
>
> Tony Graham.
> --
> Senior Architect
> XML Division
> Antenna House, Inc.
> ----
> Skerries, Ireland
> tg...@an...
>
>
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>


-- 
Michael Westbay
Writer/System Administrator
http://www.japanesebaseball.com/

[Exist-open] "Collection /db/system/security/exist/accounts cannot be created."

[Tony G. <tg...@an...>]

I have an eXist-3 (yes, eXist-3) that won't respond and won't shutdown
(short of killing the process).  Starting it devolves into multiple
stack traces with the message in the subject.  Is there any hope for it?

I got here after killing eXist after a server backup filled the
filesystem and eXist became unresponsive.  (The backup was because a
Jenkins job running an Ant task had created some ghost collections that
showed only in the Java client and that the 'admin' user couldn't
delete, and I was going to tweak then reload the backup.)

Regards,


Tony Graham.
-- 
Senior Architect
XML Division
Antenna House, Inc.
----
Skerries, Ireland
tg...@an...

[Exist-open] [ANN] XML Summer School in Oxford: Register Now for Our 25th Year!

[XML S. S. <he...@xm...>]

Hello XML Summer School Community,

We’re delighted to share that this year marks the 25th Anniversary of the XML Summer School! This is a special milestone for us, and we’re planning a few extra touches to make it a memorable one. Whether you’re a long-time friend of the Summer School or have been curious about joining, we’d love for you to be part of it.

The 2025 Summer School will take place at St Edmund Hall (Teddy Hall), Oxford, from Sunday 14th to Friday 19th September. The schedule and registration information can be viewed on our website: https://xmlsummerschool.org <https://xmlsummerschool.org/>
As a little thank you, we’re offering 10% off registration if you sign up before the end of July; just use the promo code JULY at checkout.

The week will include a mix of tutorials, talks, and discussions covering XML, but also related technologies like web standards, data modeling, accessible publishing, semantics, and more. There’s also a full line-up of social activities every evening; past favourites include punting on the river, guided tours of Oxford, pub nights, museum visits, and a formal dinner in one of Oxford’s historic college halls.

If you happen to arrive early, Oxford Open Doors takes place over the weekend (13–14 September), giving you free access to many of the city’s usually hidden historic sites: https://www.oxfordpreservation.org.uk/oxford-open-doors <https://www.oxfordpreservation.org.uk/oxford-open-doors>
Please feel free to share this with colleagues, friends, or anyone who might enjoy a week of tech, history, and summer fun in Oxford.

Hope to see you there,
The XML Summer School Team

Re: [Exist-open] existdb errors on xQueries

[Joe W. <jo...@gm...>]

Hi Seena,

I'm not certain that the errors you shared indicate data corruption. Seeing
the full stack trace of the error in your initial post in this thread would
help.

The primary cause of database corruption is improper shutdown. This is
typically fixed automatically when next starting eXist, since it detects an
unclean shutdown by the presence of lock files and triggers a reindex. If
this built-in process is not successful, you can perform the emergency
recovery technique of creating a backup and restoring from the backup. This
is documented here:

  https://exist-db.org/exist/apps/doc/backup#emergency-export-tool

Proactively performing backups - via a scheduled job, for example - allows
you to expedite this process by restoring from a recent backup, rather than
needing to perform the backup upon a possibly corrupted database.

Best,
Joe

On Thu, Jun 12, 2025 at 5:29 AM Sanil, Seena via Exist-open <
exi...@li...> wrote:

> Thanks Michael W.
>
>
>
> Currently we are reindexing the entire collection.  For future, we would
> like to know if there is a way to avoid data/collection being corrupted ?
> If it gets corrupted, what is the best way to fix it?
>
>
>
> Thanks
>
> Seena
>
>
>
> *From: *Michael Westbay <wes...@ja...>
> *Date: *Tuesday, June 10, 2025 at 7:52 PM
> *To: *Sanil, Seena <ss...@bl...>
> *Cc: *eXist DB ML <exi...@li...>, Mulvaney, Michael <
> MMu...@bl...>, Lubey, Brian <
> bl...@bl...>
> *Subject: *Re: [Exist-open] existdb errors on xQueries
>
> *CAUTION:* *This email originated from outside of the organization. Do
> not click links or open attachments unless you recognize the sender and
> know the content is safe. Please report suspicious emails to:
> phi...@bl... <phi...@bl...>*
>
>
>
> Hi Seena,
>
>
>
> The return type error is an easy one to fix. The function declaration
> expects to always return 1 and only 1 configuration. But it is getting none.
>
>
>
> If returning no configuration is an option, redefine the function as:
>
>
>
> ```xquery
>
> declare function config:get-configuration() as element(configuration)? {
>
>   ...
>
> };
>
> ```
>
>
>
> If it should always return a configuration, then you need to determine why
> it is not finding the configuration. Since you aren't passing a parameter
> into the function, it doesn't appear to be something on a per-user basis --
> so there should always be the global configuration.
>
>
>
> Hope this helps point you in the right direction.
>
>
>
> Take care.
>
>
>
>
>
>
>
>
>
>
>
> 2025年6月10日(火) 22:08 Sanil, Seena via Exist-open <
> exi...@li...>:
>
> Also errors like this “
>
> exerr:*ERROR* The actual return type does not match the sequence type
> declared in the function's signature: config:get-configuration() as
> element(configuration). Expected cardinality: exactly one, got 0. [at line
> 42, column 42, source: /db/apps/eXide/controller.xq]\nIn
> function:\n\tconfig:get-configuration() as element(configuration)”
>
>
>
> *From: *Sanil, Seena <ss...@bl...>
> *Date: *Monday, June 9, 2025 at 7:44 PM
> *To: *eXist DB ML <exi...@li...>
> *Cc: *Lubey, Brian <bl...@bl...>, Mulvaney, Michael <
> MMu...@bl...>
> *Subject: *existdb errors on xQueries
>
> Hello,
>
> We are getting some errors while running xQuery’s  all at once against
> our eXist database with 1.3 million files from different clients like
> browser, oxygen desktop, Postman.
>
>
>
> The errors are “Error while processing /exist/xmlrpc: An unknown error
> occurred: org.apache.xmlrpc.XmlRpcException: I/O error while processing
> request”. . After that, the existdb instance hosted in the the cloud is
> not coming up. Is there a reason for this behavior? How do we avoid this in
> the future?
>
>
>
> The xQuery we used are :
>
> _query=//cite.bna.reference&_howmany=1000 , on a collection
>
>
>
> Thanks
>
> Seena Sanil
>
>
>
>
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>
>
>
> --
>
> Michael Westbay
> Writer/System Administrator
> http://www.japanesebaseball.com/
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>

Re: [Exist-open] existdb errors on xQueries

[Sanil, S. <ss...@bl...>]

Thanks Michael W.

Currently we are reindexing the entire collection.  For future, we would like to know if there is a way to avoid data/collection being corrupted ? If it gets corrupted, what is the best way to fix it?

Thanks
Seena

From: Michael Westbay <wes...@ja...>
Date: Tuesday, June 10, 2025 at 7:52 PM
To: Sanil, Seena <ss...@bl...>
Cc: eXist DB ML <exi...@li...>, Mulvaney, Michael <MMu...@bl...>, Lubey, Brian <bl...@bl...>
Subject: Re: [Exist-open] existdb errors on xQueries
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. Please report suspicious emails to: phi...@bl...<mailto:phi...@bl...>

Hi Seena,

The return type error is an easy one to fix. The function declaration expects to always return 1 and only 1 configuration. But it is getting none.

If returning no configuration is an option, redefine the function as:

```xquery
declare function config:get-configuration() as element(configuration)? {
  ...
};
```

If it should always return a configuration, then you need to determine why it is not finding the configuration. Since you aren't passing a parameter into the function, it doesn't appear to be something on a per-user basis -- so there should always be the global configuration.

Hope this helps point you in the right direction.

Take care.





2025年6月10日(火) 22:08 Sanil, Seena via Exist-open <exi...@li...<mailto:exi...@li...>>:
Also errors like this “
exerr:ERROR The actual return type does not match the sequence type declared in the function's signature: config:get-configuration() as element(configuration). Expected cardinality: exactly one, got 0. [at line 42, column 42, source: /db/apps/eXide/controller.xq]\nIn function:\n\tconfig:get-configuration() as element(configuration)”

From: Sanil, Seena <ss...@bl...<mailto:ss...@bl...>>
Date: Monday, June 9, 2025 at 7:44 PM
To: eXist DB ML <exi...@li...<mailto:exi...@li...>>
Cc: Lubey, Brian <bl...@bl...<mailto:bl...@bl...>>, Mulvaney, Michael <MMu...@bl...<mailto:MMu...@bl...>>
Subject: existdb errors on xQueries
Hello,
We are getting some errors while running xQuery’s  all at once against our eXist database with 1.3 million files from different clients like browser, oxygen desktop, Postman.

The errors are “Error while processing /exist/xmlrpc: An unknown error occurred: org.apache.xmlrpc.XmlRpcException: I/O error while processing request”. . After that, the existdb instance hosted in the the cloud is not coming up. Is there a reason for this behavior? How do we avoid this in the future?

The xQuery we used are :
_query=//cite.bna.reference&_howmany=1000 , on a collection

Thanks
Seena Sanil


_______________________________________________
Exist-open mailing list
Exi...@li...<mailto:Exi...@li...>
https://lists.sourceforge.net/lists/listinfo/exist-open


--
Michael Westbay
Writer/System Administrator
http://www.japanesebaseball.com/

Re: [Exist-open] existdb errors on xQueries

[Michael W. <wes...@ja...>]

Hi Seena,

The return type error is an easy one to fix. The function declaration
expects to always return 1 and only 1 configuration. But it is getting none.

If returning no configuration is an option, redefine the function as:

```xquery
declare function config:get-configuration() as element(configuration)? {
  ...
};
```

If it should always return a configuration, then you need to determine why
it is not finding the configuration. Since you aren't passing a parameter
into the function, it doesn't appear to be something on a per-user basis --
so there should always be the global configuration.

Hope this helps point you in the right direction.

Take care.





2025年6月10日(火) 22:08 Sanil, Seena via Exist-open <
exi...@li...>:

> Also errors like this “
>
> exerr:*ERROR* The actual return type does not match the sequence type
> declared in the function's signature: config:get-configuration() as
> element(configuration). Expected cardinality: exactly one, got 0. [at line
> 42, column 42, source: /db/apps/eXide/controller.xq]\nIn
> function:\n\tconfig:get-configuration() as element(configuration)”
>
>
>
> *From: *Sanil, Seena <ss...@bl...>
> *Date: *Monday, June 9, 2025 at 7:44 PM
> *To: *eXist DB ML <exi...@li...>
> *Cc: *Lubey, Brian <bl...@bl...>, Mulvaney, Michael <
> MMu...@bl...>
> *Subject: *existdb errors on xQueries
>
> Hello,
>
> We are getting some errors while running xQuery’s  all at once against
> our eXist database with 1.3 million files from different clients like
> browser, oxygen desktop, Postman.
>
>
>
> The errors are “Error while processing /exist/xmlrpc: An unknown error
> occurred: org.apache.xmlrpc.XmlRpcException: I/O error while processing
> request”. . After that, the existdb instance hosted in the the cloud is
> not coming up. Is there a reason for this behavior? How do we avoid this in
> the future?
>
>
>
> The xQuery we used are :
>
> _query=//cite.bna.reference&_howmany=1000 , on a collection
>
>
>
> Thanks
>
> Seena Sanil
>
>
>
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>


-- 
Michael Westbay
Writer/System Administrator
http://www.japanesebaseball.com/