flaim-devel — Discussions related to FLAIM library development.

[Flaim-devel] Xflaim Incremental Backup

[Christopher S. <cst...@no...>]

Ok, I finally got the full backup working.  I also got the restore to work for a full backup.

Now I am trying to do an incremental backup of the database, but when I open it with the following line,

db.backupBegin(FlmBackupType.INCREMENTAL_BACKUP, TransactionType.READ_TRANS, 0, null);

I get an error:
xflaim.XFlaimException: Error code from XFLAIM was 0000C026
	at xflaim.Db._backupBegin(Native Method)
	at xflaim.Db.backupBegin(Db.java:1171)
	at de.suse.backends.IncrementalBackupUtil.StartBackup(IncrementalBackupUtil.java:56)
	at de.suse.backends.IncrementalBackupUtil.run(IncrementalBackupUtil.java:29)
	at BackupStuff.main(BackupStuff.java:15)

The error is defined as NE_FLM_ILLEGAL_OP, why would it throw an illegal operation on this line?

Thanks,
-Chris

Re: [Flaim-devel] Flaim Problems

[Daniel S. <dsa...@no...>]

Can you print out the values for pEnv and jBuff?  As I recall, you are getting a segmentation fault, so it looks like perhaps pEnv or jBuff are not properly aligned, or perhaps jBuff is coming back NULL from the previous call to NewByteArray.  Knowing the addresses will help.  It is likely that jBuff is the culprit, as you have already made multiple calls using pEnv, including this line right before the one that fails:
 
 jBuff = pEnv->NewByteArray( (jsize)uiBytesToWrite);
 
Perhaps the NewByteArray method is not returning a properly aligned buffer or it is NULL.  If NULL, it may be that the JVM cannot allocate memory for some reason.  You might also want to print out uiBytesToWrite.
 
Daniel Sanders

>>> Christopher Stolworthy 4/26/2007 3:04 PM >>>
Ok.  Thanks for helping me to understand how that is working.  I had the wrong idea on how everything was working together, this is my first delve into a pure linux enviroment.  I added the trace statements and used the make install.

According to the output of the trace statements the program fails on the line:
pvBuff = pEnv->GetPrimitiveArrayCritical(jBuff, NULL);

in Jbackup.cpp 

Thanks again,
Chris

>>> Daniel Sanders 04/26/07 2:54 PM >>> 
This is not correct.  If the *so files did not contain jbackup.cpp stuff, you would never have been able to get as far as calling into the WriteData method on the JNIBackupClient method in the first place.  The problem, as I stated in a previous email, is that you have not copied your rebuilt *so file to the correct directory.  Klaas is correct in his diagnosis.

Daniel

>>> "Christopher Stolworthy" <cst...@no...> 4/26/2007 2:20 PM >>>
The problem is that the libxflaim*so files do not contain the jbackup.cpp stuff, only the libxflaim.a library contains the jBackup class that I need.
What I have been doing is using a Java application that launches a thread I made to do the backup.  I have been starting it with the argument  -Djava.library.path="/usr/lib" .  When that doesn't work I would move the libraries to their own folder and change the path to that folder.

Thanks,
-Chris

>>> Klaas Freitag <fr...@su...> 04/26/07 2:11 PM >>> 
Am Donnerstag, 26. April 2007 21:51 schrieb Christopher Stolworthy:

Christopher,

> Ok so I found what I *think* the problem is.  The jBackup.cpp gets compiled 
> into libxflaim.a which is in my /usr/lib/ directory.  I added trace statements 
> to this method, and it is not being accessed at all.  
> It seems like Java is grabbing the libraries inside of libxflaim.so,
> libxflaim.so.3.2,libxflaim.so.3 . So I grabbed just libxflaim.a and put it 
> in its own directory and pointed java at it.  
> When I run I get an exception stating there is no xflaim in that directory.  
> As soon as I add the other libraries it will compile, but starts breaking 
> again but doesn't access libxflaim.a.

The libxflaim.a library is a lib for static linking. Java probably does
not use it at all but the libxflaim*so version which are the dynamically
linkable versions.

You have to make sure that 
-  your xflaim build creates *so versions of the libs if you change things.
  I assume thats default for xflaim
-  your java finds the correct *so versions of the lib additionally to the
  jar files. If you use Tomcat, the following setup works: Copy the jar
  files to $CATALINA_HOME/webapps/webappname/WEB- INF/lib
  Still the tomcat needs access to the native libs libxflaim.so etc. You can 
  have these in your working directory of xflaim but than you need to start
  Tomcat (or your java stack at all) with the option 
  - Djava.library.path=/space/dbxml/dbxml/install/lib

  I do that in the tomcat start script using 
  export CATALINA_OPTS='- Xmx612m - Djava.library.path=/space/dbxml/dbxml/install/lib'

> 
> Any ideas?
Hope that helps, have fun,

Klaas
--  
                                               Klaas Freitag
                          Novell -  SUSE R&D -  Internal Tools

_______________________________________________
Flaim-devel mailing list
Fla...@fo...
http://forge.novell.com/mailman/listinfo/flaim-devel

Re: [Flaim-devel] Flaim Problems

[Christopher S. <cst...@no...>]

Ok.  Thanks for helping me to understand how that is working.  I had the wrong idea on how everything was working together, this is my first delve into a pure linux enviroment.  I added the trace statements and used the make install.

According to the output of the trace statements the program fails on the line:
pvBuff = pEnv->GetPrimitiveArrayCritical(jBuff, NULL);

in Jbackup.cpp 

Thanks again,
Chris
 
>>> Daniel Sanders 04/26/07 2:54 PM >>> 
This is not correct.  If the *so files did not contain jbackup.cpp stuff, you would never have been able to get as far as calling into the WriteData method on the JNIBackupClient method in the first place.  The problem, as I stated in a previous email, is that you have not copied your rebuilt *so file to the correct directory.  Klaas is correct in his diagnosis.
 
Daniel

>>> "Christopher Stolworthy" <cst...@no...> 4/26/2007 2:20 PM >>>
The problem is that the libxflaim*so files do not contain the jbackup.cpp stuff, only the libxflaim.a library contains the jBackup class that I need.
What I have been doing is using a Java application that launches a thread I made to do the backup.  I have been starting it with the argument  -Djava.library.path="/usr/lib" .  When that doesn't work I would move the libraries to their own folder and change the path to that folder.

Thanks,
-Chris

>>> Klaas Freitag <fr...@su...> 04/26/07 2:11 PM >>> 
Am Donnerstag, 26. April 2007 21:51 schrieb Christopher Stolworthy:

Christopher,

> Ok so I found what I *think* the problem is.  The jBackup.cpp gets compiled 
> into libxflaim.a which is in my /usr/lib/ directory.  I added trace statements 
> to this method, and it is not being accessed at all.  
> It seems like Java is grabbing the libraries inside of libxflaim.so,
> libxflaim.so.3.2,libxflaim.so.3 . So I grabbed just libxflaim.a and put it 
> in its own directory and pointed java at it.  
> When I run I get an exception stating there is no xflaim in that directory.  
> As soon as I add the other libraries it will compile, but starts breaking 
> again but doesn't access libxflaim.a.

The libxflaim.a library is a lib for static linking. Java probably does
not use it at all but the libxflaim*so version which are the dynamically
linkable versions.

You have to make sure that 
-  your xflaim build creates *so versions of the libs if you change things.
  I assume thats default for xflaim
-  your java finds the correct *so versions of the lib additionally to the
  jar files. If you use Tomcat, the following setup works: Copy the jar
  files to $CATALINA_HOME/webapps/webappname/WEB- INF/lib
  Still the tomcat needs access to the native libs libxflaim.so etc. You can 
  have these in your working directory of xflaim but than you need to start
  Tomcat (or your java stack at all) with the option 
  - Djava.library.path=/space/dbxml/dbxml/install/lib

  I do that in the tomcat start script using 
  export CATALINA_OPTS='- Xmx612m - Djava.library.path=/space/dbxml/dbxml/install/lib'

> 
> Any ideas?
Hope that helps, have fun,

Klaas
--  
                                               Klaas Freitag
                          Novell -  SUSE R&D -  Internal Tools

_______________________________________________
Flaim-devel mailing list
Fla...@fo... 
http://forge.novell.com/mailman/listinfo/flaim-devel

Re: [Flaim-devel] Flaim Problems

[Daniel S. <dsa...@no...>]

This is not correct.  If the *so files did not contain jbackup.cpp stuff, you would never have been able to get as far as calling into the WriteData method on the JNIBackupClient method in the first place.  The problem, as I stated in a previous email, is that you have not copied your rebuilt *so file to the correct directory.  Klaas is correct in his diagnosis.
 
Daniel

>>> "Christopher Stolworthy" <cst...@no...> 4/26/2007 2:20 PM >>>
The problem is that the libxflaim*so files do not contain the jbackup.cpp stuff, only the libxflaim.a library contains the jBackup class that I need.
What I have been doing is using a Java application that launches a thread I made to do the backup.  I have been starting it with the argument  -Djava.library.path="/usr/lib" .  When that doesn't work I would move the libraries to their own folder and change the path to that folder.

Thanks,
-Chris

>>> Klaas Freitag <fr...@su...> 04/26/07 2:11 PM >>> 
Am Donnerstag, 26. April 2007 21:51 schrieb Christopher Stolworthy:

Christopher,

> Ok so I found what I *think* the problem is.  The jBackup.cpp gets compiled 
> into libxflaim.a which is in my /usr/lib/ directory.  I added trace statements 
> to this method, and it is not being accessed at all.  
> It seems like Java is grabbing the libraries inside of libxflaim.so,
> libxflaim.so.3.2,libxflaim.so.3 . So I grabbed just libxflaim.a and put it 
> in its own directory and pointed java at it.  
> When I run I get an exception stating there is no xflaim in that directory.  
> As soon as I add the other libraries it will compile, but starts breaking 
> again but doesn't access libxflaim.a.

The libxflaim.a library is a lib for static linking. Java probably does
not use it at all but the libxflaim*so version which are the dynamically
linkable versions.

You have to make sure that 
-  your xflaim build creates *so versions of the libs if you change things.
  I assume thats default for xflaim
-  your java finds the correct *so versions of the lib additionally to the
  jar files. If you use Tomcat, the following setup works: Copy the jar
  files to $CATALINA_HOME/webapps/webappname/WEB- INF/lib
  Still the tomcat needs access to the native libs libxflaim.so etc. You can 
  have these in your working directory of xflaim but than you need to start
  Tomcat (or your java stack at all) with the option 
  - Djava.library.path=/space/dbxml/dbxml/install/lib

  I do that in the tomcat start script using 
  export CATALINA_OPTS='- Xmx612m - Djava.library.path=/space/dbxml/dbxml/install/lib'

> 
> Any ideas?
Hope that helps, have fun,

Klaas
--  
                                               Klaas Freitag
                          Novell -  SUSE R&D -  Internal Tools

_______________________________________________
Flaim-devel mailing list
Fla...@fo... 
http://forge.novell.com/mailman/listinfo/flaim-devel

Re: [Flaim-devel] Flaim Problems

[Daniel S. <dsa...@no...>]

Your problem is that although you rebuilt the xflaim shared library, you did not properly install it so that it could be accessed by the Java JVM.  Let me clarify some things for you:
 
1. libxflaim.a is a static library.  It is only used for static linking.  It is never used by the Java JVM for anything.  When a Java JVM accesses native code, it always has to do it by dynamically loading a shared library.  Dynamic loading of static libraries is impossible.
 
2. libxflaim.so.3.2 is the shared xflaim library.  It contains all of the xflaim C++ classes and methods, including the classes and functions in jbackup.cpp.  It is dynamically loaded by the Java JVM when the java code makes native calls (JNI).  If the classes and functions in jbackup.cpp were not present in the xflaim shared library, you would never have gotten as far as attempting to call a WriteData method.  So I know they are present in the library - that is definitely not the issue.
 
3. libxflaim.so and libxflaim.so.3 are simply soft links to libxflaim.so.3.2.  It may be that the Java JVM actually loads either libxflaim.so or libxflaim.so.3.  It really doesn't matter, because they both are really just pointers to libxflaim.so.3.2, which is the library that will actually be loaded.
 
4. If you make a change to xflaim C++ code (including jbackup.cpp) and then rebuild, it will build a new libxflaim.so.3.2 shared library.  However, that new library is not automatically installed in the directory where the Java JVM will be able to access it from.  When it is built, it is stored in a "build" directory underneath the xflaim directory.  If you did a "debug" build (make debug), it will be found here:
 
   build/linux-x86-32/debug/lib/shared/libxflaim.so.3.2
 
Otherwise, it will be found here:
 
   build/linux-x86-32/release/lib/shared/libxflaim.so.3.2
 
Neither of these is where the Java JVM will look for it.  It sounds like your JVM is looking for shared libraries in the /usr/lib directory.  That means you need to copy libxflaim.so.3.2 from the build/xxxxx subdirectory to /usr/lib.  If your JVM is not looking in /usr/lib, you need to determine where your JVM looks for shared libraries.  Perhaps you can consult with someone on your team about that.
 
If you do a "make install", or a "make debug install" (for a debug build), the shared library is copied to the /usr/lib directory automatically after it has been built.  However, you probably need to be the root user to execute a "make install" command, as a non-root user will not typically have permission to modify or delete files in the /usr/lib directory.
 
Good luck.
 
Daniel

>>> Christopher Stolworthy 4/26/2007 1:51 PM >>>
Ok so I found what I *think* the problem is.  The jBackup.cpp gets compiled into libxflaim.a which is in my /usr/lib/ directory.  I added trace statements to this method, and it is not being accessed at all.  It seems like Java is grabbing the libraries inside of libxflaim.so,libxflaim.so.3.2,libxflaim.so.3 . So I grabbed just libxflaim.a and put it in its own directory and pointed java at it.  When I run I get an exception stating there is no xflaim in that directory.  As soon as I add the other libraries it will compile, but starts breaking again but doesn't access libxflaim.a.

Any ideas?

Thanks,
-Chris 

>>> Daniel Sanders 04/26/07 1:36 PM >>> 
Put your trace statements inside the WriteData method - it is in jbackup.cpp, JNI_BackupClient::WriteData.  Figure out where, inside the method, it is crashing.

>>> "Christopher Stolworthy" <cst...@no...> 4/26/2007 12:23 PM >>>
I can not get get a debugger to attach to the C libraries.  However I did go in and add my own trace statements to the code to determine where the problem is at.  The program dies on this line of code

if(RC_BAD(pBackerStream->m_pClient->WriteData(pBackerStream->m_pucOutBuf, *(pBackerStream->m_puiOutOffset)))) 

That is in the file flbackup.cpp in the method F_BackerStream::writeThread

I have determined that both m_pucOutBuf and *(pBackerStream->m_puiOutOffset) have values, so it is not a null problem.  I wonder if it may be a bad pointer?  Am i on the right track, or should I look somewhere else?

Thanks,
Chris

>>> Daniel Sanders 04/26/07 9:19 AM >>> 
It looks like the JNIBackupClient::WriteData method is calling something back in the JVM, but it is impossible to tell from the stack what it is.  Is there any way you can set a breakpoint in the JNIBackupClient::WriteData method (see jbackup.cpp) and step through it to see exactly what call it is making into the JVM that it is crashing on. -- In other words, what line number are we on inside the JNIBackupClient::WriteData method when it crashes?

Thanks,

Daniel Sanders

>>> "Christopher Stolworthy" <cst...@no...> 4/20/2007 2:33 PM >>>
I am having an issue with the Flaim Database.  I am trying to start a backup, and it is dying when I call to actually start the backup. The API manages to create a file in the directory for backups, but it is always a zero length file.

This is the error I am getting. 

Stack: [0xaccd7000,0xad4d8000),  sp=0xad4d7304,  free space=8192k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0x1a3ae0]
C  [libxflaim.so.3.2+0x16f9a6]  _ZN15JNIBackupClient9WriteDataEPKvm+0xe6
C  [libxflaim.so.3.2+0x9ed54]  _ZN14F_BackerStream11writeThreadEP9IF_Thread+0x64
C  [libxflaim.so.3.2+0x16be84]  threadStub+0x84
C  [libpthread.so.0+0x534b]

Here is the call in my Java code where it dies.

back.backup(backupPath, "", null, btm);

back is the object I have from opening a database.  Backup path is /home/bck (I hardcoded it in as well and it still didn't work)

btm is a custom object implementing the BackupMonitor interface.

I have also tried using my own backup client and passing that in to have it use that to write the data, but it still broke.

Thanks,
Chris
_______________________________________________
Flaim-devel mailing list
Fla...@fo...
http://forge.novell.com/mailman/listinfo/flaim-devel

_______________________________________________
Flaim-devel mailing list
Fla...@fo...
http://forge.novell.com/mailman/listinfo/flaim-devel

Re: [Flaim-devel] Flaim Problems

[Christopher S. <cst...@no...>]

The problem is that the libxflaim*so files do not contain the jbackup.cpp stuff, only the libxflaim.a library contains the jBackup class that I need.
What I have been doing is using a Java application that launches a thread I made to do the backup.  I have been starting it with the argument  -Djava.library.path="/usr/lib" .  When that doesn't work I would move the libraries to their own folder and change the path to that folder.

Thanks,
-Chris
 
>>> Klaas Freitag <fr...@su...> 04/26/07 2:11 PM >>> 
Am Donnerstag, 26. April 2007 21:51 schrieb Christopher Stolworthy:

Christopher,

> Ok so I found what I *think* the problem is.  The jBackup.cpp gets compiled 
> into libxflaim.a which is in my /usr/lib/ directory.  I added trace statements 
> to this method, and it is not being accessed at all.  
> It seems like Java is grabbing the libraries inside of libxflaim.so,
> libxflaim.so.3.2,libxflaim.so.3 . So I grabbed just libxflaim.a and put it 
> in its own directory and pointed java at it.  
> When I run I get an exception stating there is no xflaim in that directory.  
> As soon as I add the other libraries it will compile, but starts breaking 
> again but doesn't access libxflaim.a.

The libxflaim.a library is a lib for static linking. Java probably does
not use it at all but the libxflaim*so version which are the dynamically
linkable versions.

You have to make sure that 
-  your xflaim build creates *so versions of the libs if you change things.
  I assume thats default for xflaim
-  your java finds the correct *so versions of the lib additionally to the
  jar files. If you use Tomcat, the following setup works: Copy the jar
  files to $CATALINA_HOME/webapps/webappname/WEB- INF/lib
  Still the tomcat needs access to the native libs libxflaim.so etc. You can 
  have these in your working directory of xflaim but than you need to start
  Tomcat (or your java stack at all) with the option 
  - Djava.library.path=/space/dbxml/dbxml/install/lib

  I do that in the tomcat start script using 
  export CATALINA_OPTS='- Xmx612m - Djava.library.path=/space/dbxml/dbxml/install/lib'

> 
> Any ideas?
Hope that helps, have fun,

Klaas
--  
                                               Klaas Freitag
                          Novell -  SUSE R&D -  Internal Tools

Re: [Flaim-devel] Flaim Problems

[Klaas F. <fr...@su...>]

Am Donnerstag, 26. April 2007 21:51 schrieb Christopher Stolworthy:

Christopher,

> Ok so I found what I *think* the problem is.  The jBackup.cpp gets compiled 
> into libxflaim.a which is in my /usr/lib/ directory.  I added trace statements 
> to this method, and it is not being accessed at all.  
> It seems like Java is grabbing the libraries inside of libxflaim.so,
> libxflaim.so.3.2,libxflaim.so.3 . So I grabbed just libxflaim.a and put it 
> in its own directory and pointed java at it.  
> When I run I get an exception stating there is no xflaim in that directory.  
> As soon as I add the other libraries it will compile, but starts breaking 
> again but doesn't access libxflaim.a.

The libxflaim.a library is a lib for static linking. Java probably does
not use it at all but the libxflaim*so version which are the dynamically
linkable versions.

You have to make sure that 
- your xflaim build creates *so versions of the libs if you change things.
  I assume thats default for xflaim
- your java finds the correct *so versions of the lib additionally to the
  jar files. If you use Tomcat, the following setup works: Copy the jar
  files to $CATALINA_HOME/webapps/webappname/WEB-INF/lib
  Still the tomcat needs access to the native libs libxflaim.so etc. You can 
  have these in your working directory of xflaim but than you need to start
  Tomcat (or your java stack at all) with the option 
  -Djava.library.path=/space/dbxml/dbxml/install/lib

  I do that in the tomcat start script using 
  export CATALINA_OPTS='-Xmx612m -Djava.library.path=/space/dbxml/dbxml/install/lib'

> 
> Any ideas?
Hope that helps, have fun,

Klaas
-- 
                                               Klaas Freitag
                          Novell - SUSE R&D - Internal Tools

Re: [Flaim-devel] Flaim Problems

[Christopher S. <cst...@no...>]

Ok so I found what I *think* the problem is.  The jBackup.cpp gets compiled into libxflaim.a which is in my /usr/lib/ directory.  I added trace statements to this method, and it is not being accessed at all.  It seems like Java is grabbing the libraries inside of libxflaim.so,libxflaim.so.3.2,libxflaim.so.3 . So I grabbed just libxflaim.a and put it in its own directory and pointed java at it.  When I run I get an exception stating there is no xflaim in that directory.  As soon as I add the other libraries it will compile, but starts breaking again but doesn't access libxflaim.a.

Any ideas?

Thanks,
-Chris 
 
>>> Daniel Sanders 04/26/07 1:36 PM >>> 
Put your trace statements inside the WriteData method - it is in jbackup.cpp, JNI_BackupClient::WriteData.  Figure out where, inside the method, it is crashing.

>>> "Christopher Stolworthy" <cst...@no...> 4/26/2007 12:23 PM >>>
I can not get get a debugger to attach to the C libraries.  However I did go in and add my own trace statements to the code to determine where the problem is at.  The program dies on this line of code

if(RC_BAD(pBackerStream->m_pClient->WriteData(pBackerStream->m_pucOutBuf, *(pBackerStream->m_puiOutOffset)))) 

That is in the file flbackup.cpp in the method F_BackerStream::writeThread

I have determined that both m_pucOutBuf and *(pBackerStream->m_puiOutOffset) have values, so it is not a null problem.  I wonder if it may be a bad pointer?  Am i on the right track, or should I look somewhere else?

Thanks,
Chris

>>> Daniel Sanders 04/26/07 9:19 AM >>> 
It looks like the JNIBackupClient::WriteData method is calling something back in the JVM, but it is impossible to tell from the stack what it is.  Is there any way you can set a breakpoint in the JNIBackupClient::WriteData method (see jbackup.cpp) and step through it to see exactly what call it is making into the JVM that it is crashing on. -- In other words, what line number are we on inside the JNIBackupClient::WriteData method when it crashes?

Thanks,

Daniel Sanders

>>> "Christopher Stolworthy" <cst...@no...> 4/20/2007 2:33 PM >>>
I am having an issue with the Flaim Database.  I am trying to start a backup, and it is dying when I call to actually start the backup. The API manages to create a file in the directory for backups, but it is always a zero length file.

This is the error I am getting. 

Stack: [0xaccd7000,0xad4d8000),  sp=0xad4d7304,  free space=8192k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0x1a3ae0]
C  [libxflaim.so.3.2+0x16f9a6]  _ZN15JNIBackupClient9WriteDataEPKvm+0xe6
C  [libxflaim.so.3.2+0x9ed54]  _ZN14F_BackerStream11writeThreadEP9IF_Thread+0x64
C  [libxflaim.so.3.2+0x16be84]  threadStub+0x84
C  [libpthread.so.0+0x534b]

Here is the call in my Java code where it dies.

back.backup(backupPath, "", null, btm);

back is the object I have from opening a database.  Backup path is /home/bck (I hardcoded it in as well and it still didn't work)

btm is a custom object implementing the BackupMonitor interface.

I have also tried using my own backup client and passing that in to have it use that to write the data, but it still broke.

Thanks,
Chris
_______________________________________________
Flaim-devel mailing list
Fla...@fo... 
http://forge.novell.com/mailman/listinfo/flaim-devel 

_______________________________________________
Flaim-devel mailing list
Fla...@fo... 
http://forge.novell.com/mailman/listinfo/flaim-devel

Re: [Flaim-devel] Flaim Problems

[Daniel S. <dsa...@no...>]

Put your trace statements inside the WriteData method - it is in jbackup.cpp, JNI_BackupClient::WriteData.  Figure out where, inside the method, it is crashing.

>>> "Christopher Stolworthy" <cst...@no...> 4/26/2007 12:23 PM >>>
I can not get get a debugger to attach to the C libraries.  However I did go in and add my own trace statements to the code to determine where the problem is at.  The program dies on this line of code

if(RC_BAD(pBackerStream->m_pClient->WriteData(pBackerStream->m_pucOutBuf, *(pBackerStream->m_puiOutOffset)))) 

That is in the file flbackup.cpp in the method F_BackerStream::writeThread

I have determined that both m_pucOutBuf and *(pBackerStream->m_puiOutOffset) have values, so it is not a null problem.  I wonder if it may be a bad pointer?  Am i on the right track, or should I look somewhere else?

Thanks,
Chris

>>> Daniel Sanders 04/26/07 9:19 AM >>> 
It looks like the JNIBackupClient::WriteData method is calling something back in the JVM, but it is impossible to tell from the stack what it is.  Is there any way you can set a breakpoint in the JNIBackupClient::WriteData method (see jbackup.cpp) and step through it to see exactly what call it is making into the JVM that it is crashing on. -- In other words, what line number are we on inside the JNIBackupClient::WriteData method when it crashes?

Thanks,

Daniel Sanders

>>> "Christopher Stolworthy" <cst...@no...> 4/20/2007 2:33 PM >>>
I am having an issue with the Flaim Database.  I am trying to start a backup, and it is dying when I call to actually start the backup. The API manages to create a file in the directory for backups, but it is always a zero length file.

This is the error I am getting. 

Stack: [0xaccd7000,0xad4d8000),  sp=0xad4d7304,  free space=8192k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0x1a3ae0]
C  [libxflaim.so.3.2+0x16f9a6]  _ZN15JNIBackupClient9WriteDataEPKvm+0xe6
C  [libxflaim.so.3.2+0x9ed54]  _ZN14F_BackerStream11writeThreadEP9IF_Thread+0x64
C  [libxflaim.so.3.2+0x16be84]  threadStub+0x84
C  [libpthread.so.0+0x534b]

Here is the call in my Java code where it dies.

back.backup(backupPath, "", null, btm);

back is the object I have from opening a database.  Backup path is /home/bck (I hardcoded it in as well and it still didn't work)

btm is a custom object implementing the BackupMonitor interface.

I have also tried using my own backup client and passing that in to have it use that to write the data, but it still broke.

Thanks,
Chris
_______________________________________________
Flaim-devel mailing list
Fla...@fo... 
http://forge.novell.com/mailman/listinfo/flaim-devel 

_______________________________________________
Flaim-devel mailing list
Fla...@fo... 
http://forge.novell.com/mailman/listinfo/flaim-devel

Re: [Flaim-devel] Flaim Problems

[Christopher S. <cst...@no...>]

I can not get get a debugger to attach to the C libraries.  However I did go in and add my own trace statements to the code to determine where the problem is at.  The program dies on this line of code

if(RC_BAD(pBackerStream->m_pClient->WriteData(			pBackerStream->m_pucOutBuf, *(pBackerStream->m_puiOutOffset)))) 

That is in the file flbackup.cpp in the method F_BackerStream::writeThread

I have determined that both m_pucOutBuf and *(pBackerStream->m_puiOutOffset) have values, so it is not a null problem.  I wonder if it may be a bad pointer?  Am i on the right track, or should I look somewhere else?

Thanks,
Chris
 
>>> Daniel Sanders 04/26/07 9:19 AM >>> 
It looks like the JNIBackupClient::WriteData method is calling something back in the JVM, but it is impossible to tell from the stack what it is.  Is there any way you can set a breakpoint in the JNIBackupClient::WriteData method (see jbackup.cpp) and step through it to see exactly what call it is making into the JVM that it is crashing on. -- In other words, what line number are we on inside the JNIBackupClient::WriteData method when it crashes?
 
Thanks,

Daniel Sanders

>>> "Christopher Stolworthy" <cst...@no...> 4/20/2007 2:33 PM >>>
I am having an issue with the Flaim Database.  I am trying to start a backup, and it is dying when I call to actually start the backup. The API manages to create a file in the directory for backups, but it is always a zero length file.

This is the error I am getting. 

Stack: [0xaccd7000,0xad4d8000),  sp=0xad4d7304,  free space=8192k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0x1a3ae0]
C  [libxflaim.so.3.2+0x16f9a6]  _ZN15JNIBackupClient9WriteDataEPKvm+0xe6
C  [libxflaim.so.3.2+0x9ed54]  _ZN14F_BackerStream11writeThreadEP9IF_Thread+0x64
C  [libxflaim.so.3.2+0x16be84]  threadStub+0x84
C  [libpthread.so.0+0x534b]

Here is the call in my Java code where it dies.

back.backup(backupPath, "", null, btm);

back is the object I have from opening a database.  Backup path is /home/bck (I hardcoded it in as well and it still didn't work)

btm is a custom object implementing the BackupMonitor interface.

I have also tried using my own backup client and passing that in to have it use that to write the data, but it still broke.

Thanks,
Chris
_______________________________________________
Flaim-devel mailing list
Fla...@fo... 
http://forge.novell.com/mailman/listinfo/flaim-devel

Re: [Flaim-devel] Flaim Problems

[Daniel S. <dsa...@no...>]

It looks like the JNIBackupClient::WriteData method is calling something back in the JVM, but it is impossible to tell from the stack what it is.  Is there any way you can set a breakpoint in the JNIBackupClient::WriteData method (see jbackup.cpp) and step through it to see exactly what call it is making into the JVM that it is crashing on. -- In other words, what line number are we on inside the JNIBackupClient::WriteData method when it crashes?
 
Thanks,

Daniel Sanders

>>> "Christopher Stolworthy" <cst...@no...> 4/20/2007 2:33 PM >>>
I am having an issue with the Flaim Database.  I am trying to start a backup, and it is dying when I call to actually start the backup. The API manages to create a file in the directory for backups, but it is always a zero length file.

This is the error I am getting. 

Stack: [0xaccd7000,0xad4d8000),  sp=0xad4d7304,  free space=8192k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0x1a3ae0]
C  [libxflaim.so.3.2+0x16f9a6]  _ZN15JNIBackupClient9WriteDataEPKvm+0xe6
C  [libxflaim.so.3.2+0x9ed54]  _ZN14F_BackerStream11writeThreadEP9IF_Thread+0x64
C  [libxflaim.so.3.2+0x16be84]  threadStub+0x84
C  [libpthread.so.0+0x534b]

Here is the call in my Java code where it dies.

back.backup(backupPath, "", null, btm);

back is the object I have from opening a database.  Backup path is /home/bck (I hardcoded it in as well and it still didn't work)

btm is a custom object implementing the BackupMonitor interface.

I have also tried using my own backup client and passing that in to have it use that to write the data, but it still broke.

Thanks,
Chris
_______________________________________________
Flaim-devel mailing list
Fla...@fo... 
http://forge.novell.com/mailman/listinfo/flaim-devel

Re: [Flaim-devel] keeper profiling

[Daniel S. <dsa...@no...>]

Can you profile it with search to see what percentage search is taking versus exportXML?
 
Also, could you accomplish what you need to without exporting the document to a file?  For example, if you are exporting the document to a file only so you can feed it into another module that will then just reparse the document back into DOM, it is wasteful and unnecessary.  Why not just use the XFLAIM methods (which are very DOM-like) to do whatever it is you need to do with the document?  Exporting the document may be an unneeded step, depending on what you are doing with the exported document.  The exportXML method was written almost as an afterthought, without much attention being given to its performance.  It was never anticipated that it would be used extensively.  We expected that people would simply operate on documents directly in the XFLAIM database using the XFLAIM DOM-like APIs that are available - why export documents when you can programmatically access them and work on them directly from the database?  The only reason to export them is to transport them across the wire. -- If exportXML is to be used extensively, it could definitely stand some performance tuning.  There may be some things that could be done to improve exportXML performance - such as buffering.   Problem is finding time to do it.
 
Another possibility would be to store the original imported document as an extra node in the xflaim document.  You would import the document, then create an extra node on the document and put the entire imported document on that node.  This is basically storing the document twice - once in xflaim format, and once in original format - so you don't have to export it.  When you want the document, you just retrieve that extra node and grab its value rather than doing an export.  I don't particularly care for this solution as it wastes space and you have to make sure that the node containing the original document always stays up to date if you happen to modify the xflaim document using the XFLAIM methods.
 
Daniel Sanders

>>> Klaas Freitag <fr...@su...> 4/23/2007 9:54 AM >>>
Hi, find attached an overview of the time spent when I query all features from the xflaim based keeper. It shows that more than 90% of the time is spent in xflaim.Db.exportXML. Overall time is 11.5 sec while 10.5 sec are spent there. Note that this call does not include a query because it fetches all features. Any suggestions and/or comments? Klaas Sorry for crossposting AND attaching a file, but it if fairly small... -- Klaas Freitag Novell - SUSE R&D - Internal Tools 

Re: [Flaim-devel] XFlaim Error Codes

[Daniel S. <dsa...@no...>]

Currently, the only documentation for error codes is in header files: xflaim/src/xflaim.h (xflaim specific) or ftk/src/ftk.h (common toolkit codes).  All of the error codes in xflaim.h are #defines that begin with NE_XFLM_.  All of the error codes in ftk.h are #defines that begin with NE_FLM_.  Some of the xflaim error codes are defined as a toolkit error code.  For example, NE_XFLM_MEM is defined as NE_FLM_MEM.
 
All of the error codes are defined as HEX numbers.
 
To find documentation for a particular error code, look up the hex string (for 0xC209, look for the "C209" string) in either the xflaim.h or ftk.h file.  You will find 0xC209 is defined as NE_FLM_IO_PATH_NOT_FOUND in ftk.h - with a comment of "file not found."  Note that xflaim.h also defines NE_XFLM_IO_PATH_NOT_FOUND as NE_FLM_IO_PATH_NOT_FOUND.  Thus, to find all of the places in code that could return C209, you would want to do a search for either NE_FLM_IO_PATH_NOT_FOUND or NE_XFLM_IO_PATH_NOT_FOUND.  You would want to search both the toolkit code (ftk/src) and the xflaim specific code (xflaim/src).
 
It would be nice to have a place in our documentation where all error codes were documented.  It just takes time - time that we have not yet been able to find.
 
Thanks,
 
Daniel Sanders

>>> "Christopher Stolworthy" <cst...@no...> 4/19/2007 4:00 PM >>>
I was wondering if there is any documentation available for XFlaim error codes.  Also is there any way to search the source code for error codes so I can link them to a description?

I am getting the following error code from XFlaim:
Error code from XFLAIM was 0000C209

Thanks!
-Chris
_______________________________________________
Flaim-devel mailing list
Fla...@fo... 
http://forge.novell.com/mailman/listinfo/flaim-devel

[Flaim-devel] Flaim Problems - Logs

[Christopher S. <cst...@no...>]

I thought I should attach the whole error log produced when the problem occurs.

-Chris

[Flaim-devel] keeper profiling

[Klaas F. <fr...@su...>]

Hi,

find attached an overview of the time spent when I query all
features from the xflaim based keeper. It shows that more than
90% of the time is spent in xflaim.Db.exportXML.
Overall time is 11.5 sec while 10.5 sec are spent there.

Note that this call does not include a query because it fetches
all features.

Any suggestions and/or comments?

Klaas

Sorry for crossposting AND attaching a file, but it if fairly
small...

-- 
                                               Klaas Freitag
                          Novell - SUSE R&D - Internal Tools

[Flaim-devel] Flaim Problems

[Christopher S. <cst...@no...>]

I am having an issue with the Flaim Database.  I am trying to start a backup, and it is dying when I call to actually start the backup. The API manages to create a file in the directory for backups, but it is always a zero length file.

This is the error I am getting. 

Stack: [0xaccd7000,0xad4d8000),  sp=0xad4d7304,  free space=8192k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0x1a3ae0]
C  [libxflaim.so.3.2+0x16f9a6]  _ZN15JNIBackupClient9WriteDataEPKvm+0xe6
C  [libxflaim.so.3.2+0x9ed54]  _ZN14F_BackerStream11writeThreadEP9IF_Thread+0x64
C  [libxflaim.so.3.2+0x16be84]  threadStub+0x84
C  [libpthread.so.0+0x534b]

Here is the call in my Java code where it dies.

back.backup(backupPath, "", null, btm);

back is the object I have from opening a database.  Backup path is /home/bck (I hardcoded it in as well and it still didn't work)

btm is a custom object implementing the BackupMonitor interface.

I have also tried using my own backup client and passing that in to have it use that to write the data, but it still broke.

Thanks,
Chris

Re: [Flaim-devel] XFlaim Error Codes

[Thomas S. <tsc...@su...>]

Christopher Stolworthy wrote:
> I was wondering if there is any documentation available for XFlaim error codes.  Also is there any way to search the source code for error codes so I can link them to a description?
> 
> I am getting the following error code from XFlaim:
> Error code from XFLAIM was 0000C209

Hi, the codes are declared in java/xflaim/RCODE.java
Greetings

-- 
Thomas Schmidt (tschmidt [at] suse.de)
SUSE Linux Products GmbH :: Research & Development :: Internal Tools
"memory management can be a bitch" (linux/kernel/fork.c:11)

[Flaim-devel] XFlaim Error Codes

[Christopher S. <cst...@no...>]

I was wondering if there is any documentation available for XFlaim error codes.  Also is there any way to search the source code for error codes so I can link them to a description?

I am getting the following error code from XFlaim:
Error code from XFLAIM was 0000C209

Thanks!
-Chris

[Flaim-devel] xflaim: query performace question

[Thomas S. <tsc...@su...>]

Hi, I am currently testing queries with this container:
http://w3.suse.de/~tschmidt/xflaim/featuretest50000.tar.bz2
for example this query:
/feature[productcontext/product[productid='SLES-15']]
returns 11230 documents.
The strange thing is, that with a cache size of about 1GB (machine has
8GB RAM) the first request is
_faster_ than the subsequent ones with the same query.

When I do the query with a small cache size (30Mb), the query time is
faster, and constant.

Tests with smaller containers showed another behaviour: When using a
large cache, the first request
took the same time as without specifying a cache size, and the
subsequent ones were very fast.
This behaviour looks more obvious to me, so I am a bit curious.

Greetings

-- 
Thomas Schmidt (tschmidt [at] suse.de)
SUSE Linux Products GmbH :: Research & Development :: Internal Tools
"Wir sind nicht in Vietnam, sondern beim Bowling. Hier gibt es Regeln."
John Goodman in Big Lebowsky

[Flaim-devel] Setting cache limit for xflaim

[Daniel S. <dsa...@no...>]

Thomas,
 
Here is the Java method for setting the cache limit for xflaim.  It is a method on the DbSystem class:
 
 public void setHardMemoryLimit(
  int  iPercent,
  boolean bPercentOfAvail,
  int  iMin,
  int  iMax,
  int  iMinToLeave,
  boolean bPreallocate) throws XFlaimException
 
I recommend you simply set iMax to the value you want, as follows:
 
DbSystem dbSys = new DbSystem;
 
// Set 1 gb cache limit, preallocate the cache.
 
dbSys.setHardMemoryLimit( 0, false, 0, 1000000000, 0, true);
Note that if you don't set a cache limit, xflaim attempts to dynamically calculate an appropriate limit, based on how much physical memory it currently thinks is available.  The cache limit is recalculated every 15 seconds so as to be responsive to fluctuations on the system.  That mode of operation is not always the best.
 
Daniel Sanders
 

Re: [Flaim-devel] xflaim: query on indexed field returns documents multiple times?

[Thomas S. <tsc...@su...>]

Daniel Sanders schrieb:
> There is a method on the Query class called setDupHandling.  Call it as follows:
>  
>  // Assuming that q is a Query object:
>  
>  q.setDupHandling( true);

Hi, thanks for the explanation, I will test if this is a performance 
issue fo our use.
I noticed, that the setDupHandling() call only works when called after 
setupQueryExpr(),
maybe that should be added to the method documentation.
Greetings

> There are subtle differences in how XFLAIM evaluates queries using an index versus how it is done without an index.  XFLAIM remembers the last node that was returned from any call to getFirst, getLast, getNext, or getPrev.  When no index is used, the last node returned becomes the starting point for subsequent calls to getNext and getPrev.  XFLAIM will start from the next or previous node relative to the last node that was returned.  This has the unintended (but useful) side effect of preventing nodes from ever being returned twice.  NOTE: If you make calls to getNext interspersed with calls to getPrev, you will obviously get the same node returned multiple times - so this really only applies if you are traversing in one direction - forward (next) or backward (prev).
>  
> When an index is used, on the other hand, XFLAIM retrieves nodes as they are referenced from the index instead of doing an orderly traversal through documents.  An index will not have multiple entries that point to the same node.  However, that is not sufficient to guarantee that XFLAIM will not return the same node twice.  The nodes pointed to in the index are not necessarily the nodes that are returned from the xpath expression.  In your case, the index points to two distinct <productid> nodes, but both of those nodes are subordinate to the same <feature> node, which is what is ultimately returned.  The mechanism that prevents the same node from being returned twice for non-indexed searches does not work here.  It is not enough to simply remember the last node that was returned and make that the starting point for a subsequent call to getNext or getPrev.  When using an index, the starting point for subsequent calls to getNext and getPrev has to be relative to the last ind
ex entry that was used, not the last node that was returned.  It is conceivable that we might traverse index entries 3, 4, and 5, and return nodes 10, 7, and 9.  So far so good - no duplicate nodes returned.  But then suppose we go to index entry 6 which causes us to return node 10.  Our last node returned was 9, so simply checking that is not sufficient to prevent the duplication.  To prevent duplication, we must keep a list of ALL nodes that have been returned thus far, and then check against that list before returning a node.
>  
> As you might expect, this could be an expensive kind of overhead to have - especially if you have a query that could potentially return hundreds of thousands of nodes.  As we wrestled with what to do, we finally concluded that we would provide a setDupHandling method which gave the application developer control over whether he wanted to incur the cost of checking for duplicates.  For queries that don't return hundreds of thousands of nodes, the cost is minimal - we keep a sorted list in memory, and checking it is very fast. -- In general, I would say that this call should be used.
>  
> Daniel Sanders
>  
>>>> Thomas Schmidt <tsc...@su...> 3/26/2007 10:41 AM >>>
> Hi,
> when I have a document layout like this:
> 
> <feature xmlns:k="http://inttools.suse.de/sxkeeper/schema/keeper" 
> k:id="1" k:revision="1">
>    ...
>    <productcontext>
> ...
>      <product>
>        <productid>SLES-13</productid>
>      </product>
>    </productcontext>
>    <productcontext>
>      <product>
>        <productid>SLES-13</productid>
>      </product>
>    </productcontext>
> </feature>
> 
> and I query for: /feature[productcontext/product[productid='SLES-13']]
> I get all the documents that have a productid of 'SLES-13' anywhere in 
> the document. Ok.
> 
> But after adding an index on the element 'productid', the resultlist 
> contains the documents that match
> the query multiple times. In the example above, the document would be 
> included twice in the resultlist.
> Is there something I can do against this behaviour?
> 
> Greetings
> 


-- 
Thomas Schmidt (tschmidt [at] suse.de)
SUSE Linux Products GmbH :: Research & Development :: Internal Tools
"Computers are useless. They can only give you answers.", Pablo Picasso

Re: [Flaim-devel] xflaim: query for @name attribute

[Thomas S. <tsc...@su...>]

Daniel Sanders schrieb:
> This is a defect, which has now been fixed.  The file src/fxpath.cpp needed to be changed.  The fix will be in Subversion versions 1043 and greater.

Hi Daniel, I can confirm that this is fixed in r1043. Thanks.


> XFLAIM does have an internal "name" attribute defined, but it is in its own namespace: http://www.novell.com/XMLDatabase/Schema.  It should never conflict with a "name" attribute defined by an application.
>  
> The xpath parsing code tries to be nice when it looks up xpath attribute and element names in the dictionary.  If no prefix is defined, it will attempt to lookup the name only, ignoring the namespace.  If there are multiple names that match which are in different namespaces, it will return C507 (NE_XFLM_MULTIPLE_MATCHES) meaning that there are multiple possible definitions which match the specified name, and it does not know which one to use.  However, in this case, the parser should have matched the definition where namespace was empty, even if there is another definition where namespace is not empty.  The dictionary holds two definitions for "name":
>  
> 1. <empty namespace> + "name"
> 2. "http://www.novell.com/XMLDatabase/Schema" + "name"
>  
> We should have been allowing it to match on the first on instead of reporting an ambiguity.
>  
> Daniel Sanders
> 
>>>> Thomas Schmidt <tsc...@su...> 3/26/2007 9:49 AM >>>
> Hi,
> is an attribute named "name" handled in a special way in xflaim?
> I have a document in the db:
> https://forgesvn1.novell.com/svn/sxkeeper/trunk/sxkeeper/xml/example_v1.xml 
> 
> The query: /example[item[@author]]
> retrieves the document.
> The query: /example[item[@name]]
> returns C507 NE_XFLM_MULTIPLE_MATCHES
> 
> Greetings
> 


-- 
Thomas Schmidt (tschmidt [at] suse.de)
SUSE Linux Products GmbH :: Research & Development :: Internal Tools
"Wir sind nicht in Vietnam, sondern beim Bowling. Hier gibt es Regeln."
John Goodman in Big Lebowsky

Re: [Flaim-devel] xflaim: query for @name attribute

[Daniel S. <dsa...@no...>]

This is a defect, which has now been fixed.  The file src/fxpath.cpp needed to be changed.  The fix will be in Subversion versions 1043 and greater.
 
XFLAIM does have an internal "name" attribute defined, but it is in its own namespace: http://www.novell.com/XMLDatabase/Schema.  It should never conflict with a "name" attribute defined by an application.
 
The xpath parsing code tries to be nice when it looks up xpath attribute and element names in the dictionary.  If no prefix is defined, it will attempt to lookup the name only, ignoring the namespace.  If there are multiple names that match which are in different namespaces, it will return C507 (NE_XFLM_MULTIPLE_MATCHES) meaning that there are multiple possible definitions which match the specified name, and it does not know which one to use.  However, in this case, the parser should have matched the definition where namespace was empty, even if there is another definition where namespace is not empty.  The dictionary holds two definitions for "name":
 
1. <empty namespace> + "name"
2. "http://www.novell.com/XMLDatabase/Schema" + "name"
 
We should have been allowing it to match on the first on instead of reporting an ambiguity.
 
Daniel Sanders

>>> Thomas Schmidt <tsc...@su...> 3/26/2007 9:49 AM >>>
Hi,
is an attribute named "name" handled in a special way in xflaim?
I have a document in the db:
https://forgesvn1.novell.com/svn/sxkeeper/trunk/sxkeeper/xml/example_v1.xml 

The query: /example[item[@author]]
retrieves the document.
The query: /example[item[@name]]
returns C507 NE_XFLM_MULTIPLE_MATCHES

Greetings

-- 
Thomas Schmidt (tschmidt [at] suse.de)
SUSE Linux Products GmbH :: Research & Development :: Internal Tools
"Wir sind nicht in Vietnam, sondern beim Bowling. Hier gibt es Regeln."
John Goodman in Big Lebowsky
_______________________________________________
Flaim-devel mailing list
Fla...@fo... 
http://forge.novell.com/mailman/listinfo/flaim-devel

Re: [Flaim-devel] xflaim: query on indexed field returns documents multiple times?

[Daniel S. <dsa...@no...>]

There is a method on the Query class called setDupHandling.  Call it as follows:
 
 // Assuming that q is a Query object:
 
 q.setDupHandling( true);
 
There are subtle differences in how XFLAIM evaluates queries using an index versus how it is done without an index.  XFLAIM remembers the last node that was returned from any call to getFirst, getLast, getNext, or getPrev.  When no index is used, the last node returned becomes the starting point for subsequent calls to getNext and getPrev.  XFLAIM will start from the next or previous node relative to the last node that was returned.  This has the unintended (but useful) side effect of preventing nodes from ever being returned twice.  NOTE: If you make calls to getNext interspersed with calls to getPrev, you will obviously get the same node returned multiple times - so this really only applies if you are traversing in one direction - forward (next) or backward (prev).
 
When an index is used, on the other hand, XFLAIM retrieves nodes as they are referenced from the index instead of doing an orderly traversal through documents.  An index will not have multiple entries that point to the same node.  However, that is not sufficient to guarantee that XFLAIM will not return the same node twice.  The nodes pointed to in the index are not necessarily the nodes that are returned from the xpath expression.  In your case, the index points to two distinct <productid> nodes, but both of those nodes are subordinate to the same <feature> node, which is what is ultimately returned.  The mechanism that prevents the same node from being returned twice for non-indexed searches does not work here.  It is not enough to simply remember the last node that was returned and make that the starting point for a subsequent call to getNext or getPrev.  When using an index, the starting point for subsequent calls to getNext and getPrev has to be relative to the last index entry that was used, not the last node that was returned.  It is conceivable that we might traverse index entries 3, 4, and 5, and return nodes 10, 7, and 9.  So far so good - no duplicate nodes returned.  But then suppose we go to index entry 6 which causes us to return node 10.  Our last node returned was 9, so simply checking that is not sufficient to prevent the duplication.  To prevent duplication, we must keep a list of ALL nodes that have been returned thus far, and then check against that list before returning a node.
 
As you might expect, this could be an expensive kind of overhead to have - especially if you have a query that could potentially return hundreds of thousands of nodes.  As we wrestled with what to do, we finally concluded that we would provide a setDupHandling method which gave the application developer control over whether he wanted to incur the cost of checking for duplicates.  For queries that don't return hundreds of thousands of nodes, the cost is minimal - we keep a sorted list in memory, and checking it is very fast. -- In general, I would say that this call should be used.
 
Daniel Sanders
 
>>> Thomas Schmidt <tsc...@su...> 3/26/2007 10:41 AM >>>
Hi,
when I have a document layout like this:

<feature xmlns:k="http://inttools.suse.de/sxkeeper/schema/keeper" 
k:id="1" k:revision="1">
   ...
   <productcontext>
...
     <product>
       <productid>SLES-13</productid>
     </product>
   </productcontext>
   <productcontext>
     <product>
       <productid>SLES-13</productid>
     </product>
   </productcontext>
</feature>

and I query for: /feature[productcontext/product[productid='SLES-13']]
I get all the documents that have a productid of 'SLES-13' anywhere in 
the document. Ok.

But after adding an index on the element 'productid', the resultlist 
contains the documents that match
the query multiple times. In the example above, the document would be 
included twice in the resultlist.
Is there something I can do against this behaviour?

Greetings

-- 
Thomas Schmidt (tschmidt [at] suse.de)
SUSE Linux Products GmbH :: Research & Development :: Internal Tools
"From the age of uniformity, from the age of solitude,
from the age of Big Brother, from the age of doublethink - greetings!", 1984
_______________________________________________
Flaim-devel mailing list
Fla...@fo... 
http://forge.novell.com/mailman/listinfo/flaim-devel

[Flaim-devel] xflaim: query on indexed field returns documents multiple times?

[Thomas S. <tsc...@su...>]

Hi,
when I have a document layout like this:

<feature xmlns:k="http://inttools.suse.de/sxkeeper/schema/keeper" 
k:id="1" k:revision="1">
   ...
   <productcontext>
	...
     <product>
       <productid>SLES-13</productid>
     </product>
   </productcontext>
   <productcontext>
     <product>
       <productid>SLES-13</productid>
     </product>
   </productcontext>
</feature>

and I query for: /feature[productcontext/product[productid='SLES-13']]
I get all the documents that have a productid of 'SLES-13' anywhere in 
the document. Ok.

But after adding an index on the element 'productid', the resultlist 
contains the documents that match
the query multiple times. In the example above, the document would be 
included twice in the resultlist.
Is there something I can do against this behaviour?

Greetings

-- 
Thomas Schmidt (tschmidt [at] suse.de)
SUSE Linux Products GmbH :: Research & Development :: Internal Tools
"From the age of uniformity, from the age of solitude,
from the age of Big Brother, from the age of doublethink - greetings!", 1984