exist-commits — Receive notifications about CVS commits.

[Exist-commits] eXist-1.0/src/org/exist/util ReentrantReadWriteLock.java,1.5,1.6 MultiReadReentrantLock.java,1.2,1.3 Lock.java,1.4,1.5

[Wolfgang M. M. <wol...@us...>]

Update of /cvsroot/exist/eXist-1.0/src/org/exist/util
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8244/src/org/exist/util

Modified Files:
	ReentrantReadWriteLock.java MultiReadReentrantLock.java 
	Lock.java 
Log Message:
Revised collection locking to fix various concurrency errors. It is now the responsibility of the 
caller to lock/unlock a collection. A collection can be retrieved and locked via the new 
openCollection() method provided by DBBroker. After reading/modifying the collection, it should be 
unlocked by calling Collection.release.

The local XML:DB implementation and the XMLRPC interface have been changed to reflect the new
collection locking rules.

Index: Lock.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/util/Lock.java,v
retrieving revision 1.4
retrieving revision 1.5
diff -C2 -d -r1.4 -r1.5
*** Lock.java	2 Jun 2004 11:34:34 -0000	1.4
--- Lock.java	3 Aug 2004 15:25:57 -0000	1.5
***************
*** 27,33 ****
  	public final static int READ_LOCK = 0;
  	public final static int WRITE_LOCK = 1;
  	
  	/**
! 	 * Attempt to acquire a lock for read.
  	 * 
  	 * @return
--- 27,34 ----
  	public final static int READ_LOCK = 0;
  	public final static int WRITE_LOCK = 1;
+ 	public final static int NO_LOCK = -1;
  	
  	/**
! 	 * Acquire a lock for read.
  	 * 
  	 * @return
***************
*** 37,41 ****
      
      /**
!      * Attempt to acquire a lock for read or write.
       * mode is one of {@link #READ_LOCK} or
       * {@link #WRITE_LOCK}.
--- 38,42 ----
      
      /**
!      * Acquire a lock for read or write.
       * mode is one of {@link #READ_LOCK} or
       * {@link #WRITE_LOCK}.
***************
*** 48,51 ****
--- 49,62 ----
  	
  	/**
+ 	 * Attempt to acquire a lock for read or write. This method
+ 	 * will fail immediately if the lock cannot be acquired.
+ 	 *  
+ 	 * @param mode
+ 	 * @return
+ 	 * @throws LockException
+ 	 */
+ 	public boolean attempt( int mode );
+ 	
+ 	/**
  	 * Release a lock. This method assumes that the
  	 * lock is a read lock.

Index: MultiReadReentrantLock.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/util/MultiReadReentrantLock.java,v
retrieving revision 1.2
retrieving revision 1.3
diff -C2 -d -r1.2 -r1.3
*** MultiReadReentrantLock.java	8 Jun 2004 08:16:14 -0000	1.2
--- MultiReadReentrantLock.java	3 Aug 2004 15:25:57 -0000	1.3
***************
*** 93,96 ****
--- 93,103 ----
      }
  
+     /* (non-Javadoc)
+ 	 * @see org.exist.util.Lock#attempt(int)
+ 	 */
+ 	public boolean attempt(int mode) {
+ 		throw new RuntimeException("Not implemented");
+ 	}
+ 	
      /**
       * Issue a read lock if there is no outstanding write lock or threads

Index: ReentrantReadWriteLock.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/util/ReentrantReadWriteLock.java,v
retrieving revision 1.5
retrieving revision 1.6
diff -C2 -d -r1.5 -r1.6
*** ReentrantReadWriteLock.java	8 Jun 2004 08:16:14 -0000	1.5
--- ReentrantReadWriteLock.java	3 Aug 2004 15:25:57 -0000	1.6
***************
*** 99,102 ****
--- 99,122 ----
  	}
  
+ 	public boolean attempt(int mode) {
+ 		Thread caller = Thread.currentThread();
+ 		synchronized (this) {
+ 			if (caller == owner_) {
+ 				++holds_;
+ 				mode_ = mode;
+ //				System.out.println("thread " + caller.getName() + " acquired lock on " + id_ +
+ //					"; locks held = " + holds_);
+ 				return true;
+ 			} else if (owner_ == null) {
+ 				owner_ = caller;
+ 				holds_ = 1;
+ 				mode_ = mode;
+ //				System.out.println("thread " + caller.getName() + " acquired lock on " + id_);
+ 				return true;
+ 			} else {
+ 				return false;
+ 			}
+ 		}
+ 	}
  	
      /* (non-Javadoc)

[Exist-commits] eXist-1.0/src/org/exist/http/webdav/methods Delete.java,1.3,1.4 Move.java,1.3,1.4 Copy.java,1.1,1.2

[Wolfgang M. M. <wol...@us...>]

Update of /cvsroot/exist/eXist-1.0/src/org/exist/http/webdav/methods
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8244/src/org/exist/http/webdav/methods

Modified Files:
	Delete.java Move.java Copy.java 
Log Message:
Revised collection locking to fix various concurrency errors. It is now the responsibility of the 
caller to lock/unlock a collection. A collection can be retrieved and locked via the new 
openCollection() method provided by DBBroker. After reading/modifying the collection, it should be 
unlocked by calling Collection.release.

The local XML:DB implementation and the XMLRPC interface have been changed to reflect the new
collection locking rules.

Index: Move.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/http/webdav/methods/Move.java,v
retrieving revision 1.3
retrieving revision 1.4
diff -C2 -d -r1.3 -r1.4
*** Move.java	8 Jun 2004 08:16:07 -0000	1.3
--- Move.java	3 Aug 2004 15:25:58 -0000	1.4
***************
*** 107,111 ****
                      return;
                  }
!                 broker.removeCollection(destCollection.getName());
                  replaced = true;
              }
--- 107,111 ----
                      return;
                  }
!                 broker.removeCollection(destCollection);
                  replaced = true;
              }

Index: Copy.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/http/webdav/methods/Copy.java,v
retrieving revision 1.1
retrieving revision 1.2
diff -C2 -d -r1.1 -r1.2
*** Copy.java	22 Jun 2004 10:10:17 -0000	1.1
--- Copy.java	3 Aug 2004 15:25:58 -0000	1.2
***************
*** 108,114 ****
          boolean replaced = false;
          DBBroker broker = null;
          try {
              broker = pool.get(user);
!             Collection destCollection = broker.getCollection(destination);
              if(destCollection == null) {
                  response.sendError(HttpServletResponse.SC_CONFLICT,
--- 108,115 ----
          boolean replaced = false;
          DBBroker broker = null;
+         Collection destCollection = null;
          try {
              broker = pool.get(user);
!             destCollection = broker.getCollection(destination);
              if(destCollection == null) {
                  response.sendError(HttpServletResponse.SC_CONFLICT,
***************
*** 138,141 ****
--- 139,144 ----
              response.sendError(HttpServletResponse.SC_INTERNAL_SERVER_ERROR, e.getMessage());
          } finally {
+         	if(destCollection != null)
+         		destCollection.release();
              pool.release(broker);
          }

Index: Delete.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/http/webdav/methods/Delete.java,v
retrieving revision 1.3
retrieving revision 1.4
diff -C2 -d -r1.3 -r1.4
*** Delete.java	8 Jun 2004 08:16:07 -0000	1.3
--- Delete.java	3 Aug 2004 15:25:58 -0000	1.4
***************
*** 65,69 ****
  			broker = pool.get(user);
  			if(resource == null) {
! 				broker.removeCollection(collection.getName());
  			} else {
  				if(resource.getResourceType() == DocumentImpl.BINARY_FILE)
--- 65,69 ----
  			broker = pool.get(user);
  			if(resource == null) {
! 				broker.removeCollection(collection);
  			} else {
  				if(resource.getResourceType() == DocumentImpl.BINARY_FILE)

[Exist-commits] eXist-1.0/src/org/exist/xmlrpc RpcConnection.java,1.63,1.64

[Wolfgang M. M. <wol...@us...>]

Update of /cvsroot/exist/eXist-1.0/src/org/exist/xmlrpc
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8244/src/org/exist/xmlrpc

Modified Files:
	RpcConnection.java 
Log Message:
Revised collection locking to fix various concurrency errors. It is now the responsibility of the 
caller to lock/unlock a collection. A collection can be retrieved and locked via the new 
openCollection() method provided by DBBroker. After reading/modifying the collection, it should be 
unlocked by calling Collection.release.

The local XML:DB implementation and the XMLRPC interface have been changed to reflect the new
collection locking rules.

Index: RpcConnection.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/xmlrpc/RpcConnection.java,v
retrieving revision 1.63
retrieving revision 1.64
diff -C2 -d -r1.63 -r1.64
*** RpcConnection.java	2 Jul 2004 18:00:35 -0000	1.63
--- RpcConnection.java	3 Aug 2004 15:26:00 -0000	1.64
***************
*** 28,31 ****
--- 28,32 ----
  import java.io.IOException;
  import java.io.InputStream;
+ import java.io.RandomAccessFile;
  import java.io.StringReader;
  import java.io.StringWriter;
***************
*** 49,52 ****
--- 50,54 ----
  import org.apache.log4j.Logger;
[...1059 lines suppressed...]
! 			Collection destination = broker.getCollection(destinationPath);
  			if(destination == null)
  			    throw new EXistException("Destination collection " + destinationPath + " not found");
--- 2054,2058 ----
  			
  			// get destination collection
! 			destination = broker.openCollection(destinationPath, Lock.WRITE_LOCK);
  			if(destination == null)
  			    throw new EXistException("Destination collection " + destinationPath + " not found");
***************
*** 1983,1986 ****
--- 2063,2070 ----
              throw new PermissionDeniedException(e.getMessage());
          } finally {
+         	if(collection != null)
+         		collection.release();
+         	if(destination != null)
+         		destination.release();
  			brokerPool.release(broker);
  		}

[Exist-commits] eXist-1.0/src/org/exist/storage DBBroker.java,1.39,1.40 BrokerPool.java,1.25,1.26 NativeBroker.java,1.91,1.92

[Wolfgang M. M. <wol...@us...>]

Update of /cvsroot/exist/eXist-1.0/src/org/exist/storage
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8244/src/org/exist/storage

Modified Files:
	DBBroker.java BrokerPool.java NativeBroker.java 
Log Message:
Revised collection locking to fix various concurrency errors. It is now the responsibility of the 
caller to lock/unlock a collection. A collection can be retrieved and locked via the new 
openCollection() method provided by DBBroker. After reading/modifying the collection, it should be 
unlocked by calling Collection.release.

The local XML:DB implementation and the XMLRPC interface have been changed to reflect the new
collection locking rules.

Index: NativeBroker.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/storage/NativeBroker.java,v
retrieving revision 1.91
retrieving revision 1.92
diff -C2 -d -r1.91 -r1.92
*** NativeBroker.java	28 Jul 2004 18:54:55 -0000	1.91
--- NativeBroker.java	3 Aug 2004 15:25:58 -0000	1.92
***************
*** 336,360 ****
  	public DocumentSet getAllDocuments(DocumentSet docs) {
  		long start = System.currentTimeMillis();
! 		Collection root = getCollection(ROOT_COLLECTION);
! 		root.allDocs(this, docs, true, false);
!         if (LOG.isDebugEnabled()) {
!             LOG.debug("getAllDocuments(DocumentSet) - end - "
!                 + "loading "
!                 + docs.getLength()
!                 + " documents from "
!                 + docs.getCollectionCount()
!                 + "collections took "
!                 + (System.currentTimeMillis() - start)
!                 + "ms.");
!         }
! 		return docs;
  	}
  
  	public Collection getCollection(String name) {
! 		return getCollection(name, -1);
  	}
  
  	/**
! 	 *  get collection object. If the collection does not exist, null is
  	 *  returned.
  	 *
--- 336,373 ----
  	public DocumentSet getAllDocuments(DocumentSet docs) {
  		long start = System.currentTimeMillis();
! 		Collection root = null;
! 		try {
! 			root = openCollection(ROOT_COLLECTION, Lock.READ_LOCK);
! 			root.allDocs(this, docs, true, false);
! 			if (LOG.isDebugEnabled()) {
! 				LOG.debug("getAllDocuments(DocumentSet) - end - "
! 						+ "loading "
! 						+ docs.getLength()
! 						+ " documents from "
! 						+ docs.getCollectionCount()
! 						+ "collections took "
! 						+ (System.currentTimeMillis() - start)
! 						+ "ms.");
! 			}
! 			return docs;
! 		} finally {
! 			root.release();
! 		}
  	}
  
  	public Collection getCollection(String name) {
! 		return openCollection(name, -1, Lock.NO_LOCK);
! 	}
! 	
! 	public Collection getCollection(String name, long addr) {
! 		return openCollection(name, addr, Lock.NO_LOCK);
! 	}
! 	
! 	public Collection openCollection(String name, int lockMode) {
! 		return openCollection(name, -1, lockMode);
  	}
  
  	/**
! 	 *  Get collection object. If the collection does not exist, null is
  	 *  returned.
  	 *
***************
*** 362,366 ****
  	 *@return       The collection value
  	 */
! 	public Collection getCollection(String name, long addr) {
  		//	final long start = System.currentTimeMillis();
  		name = normalizeCollectionName(name);
--- 375,379 ----
  	 *@return       The collection value
  	 */
! 	public Collection openCollection(String name, long addr, int lockMode) {
  		//	final long start = System.currentTimeMillis();
  		name = normalizeCollectionName(name);
***************
*** 373,425 ****
  		if (name.endsWith("/") && name.length() > 1)
  			name = name.substring(0, name.length() - 1);
! 		Value key = null;
! 		if (addr == -1)
! 			try {
! 				key = new Value(name.getBytes("UTF-8"));
! 			} catch (UnsupportedEncodingException uee) {
! 				key = new Value(name.getBytes());
! 			}
! 		Collection collection = null;
! 		VariableByteInput is = null;
! 		Lock lock = collectionsDb.getLock();
! 		try {
! 			lock.acquire(Lock.READ_LOCK);
! 			collection = collectionsDb.getCollectionCache().get(name);
! 			if (collection != null) {
! 				return collection;
! 			}
! 			collection = new Collection(collectionsDb, name);
! 			try {
! 				if (addr < 0) {
! 					is = collectionsDb.getAsStream(key);
! 				} else {
! 					is = collectionsDb.getAsStream(addr);
  				}
- 			} catch (IOException ioe) {
- 				LOG.warn(ioe.getMessage(), ioe);
  			}
! 			if (is == null)
! 				return null;
! 
! 			try {
! 				collection.read(this, is);
! 			} catch (IOException ioe) {
! 				LOG.warn(ioe);
! 				return null;
  			}
! 			collectionsDb.getCollectionCache().add(collection);
! 		} catch (LockException e) {
! 			LOG.warn("failed to acquire lock on collections.dbx");
! 			return null;
! 		} finally {
! 			lock.release();
  		}
- 		//			LOG.debug(
- 		//				"loading collection "
- 		//					+ name
- 		//					+ " took "
- 		//					+ (System.currentTimeMillis() - start)
- 		//					+ "ms.");
- 		return collection;
  	}
  
--- 386,445 ----
  		if (name.endsWith("/") && name.length() > 1)
  			name = name.substring(0, name.length() - 1);
! 
! 		synchronized(collectionsCache) {
! 			Collection collection = collectionsCache.get(name);
! 			if(collection == null) {
! //				LOG.debug("loading collection " + name);
! 				VariableByteInput is = null;
! 				Lock lock = collectionsDb.getLock();
! 				try {
! 					lock.acquire(Lock.READ_LOCK);
! 					
! 					collection = new Collection(collectionsDb, name);
! 					Value key = null;
! 					if (addr == -1) {
! 						try {
! 							key = new Value(name.getBytes("UTF-8"));
! 						} catch (UnsupportedEncodingException uee) {
! 							key = new Value(name.getBytes());
! 						}
! 					}
! 					try {
! 						if (addr < 0) {
! 							is = collectionsDb.getAsStream(key);
! 						} else {
! 							is = collectionsDb.getAsStream(addr);
! 						}
! 						if (is == null)
! 							return null;
! 						collection.read(this, is);
! 					} catch (IOException ioe) {
! 						LOG.warn(ioe.getMessage(), ioe);
! 					}
! 				} catch (LockException e) {
! 					LOG.warn("failed to acquire lock on collections.dbx");
! 					return null;
! 				} finally {
! 					lock.release();
  				}
  			}
! 			if(lockMode != Lock.NO_LOCK) {
! 				try {
! 					//			LOG.debug("acquiring lock on " + collection.getName());
! 					collection.getLock().acquire(lockMode);
! 					//			Thread.dumpStack();
! 				} catch (LockException e1) {
! 					LOG.warn("Could not acquire lock on collection " + name);
! 				}
  			}
! 			collectionsCache.add(collection);
! 			//			LOG.debug(
! 			//				"loading collection "
! 			//					+ name
! 			//					+ " took "
! 			//					+ (System.currentTimeMillis() - start)
! 			//					+ "ms.");
! 			return collection;
  		}
  	}
  
***************
*** 521,524 ****
--- 541,580 ----
  	}
  
+ 	public Document openDocument(String docPath, int lockMode) throws PermissionDeniedException {
+ 		if (!docPath.startsWith("/"))
+ 			docPath = '/' + docPath;
+ 		if (!docPath.startsWith("/db"))
+ 		    docPath = "/db" + docPath;
+ 
+ 		int pos = docPath.lastIndexOf('/');
+ 		String collName = docPath.substring(0, pos);
+ 		String docName = docPath.substring(pos + 1);
+ 		
+ 		Collection collection = null;
+ 		try {
+ 			collection = openCollection(collName, lockMode);
+ 			if (collection == null) {
+ 				LOG.debug("collection " + collName + " not found!");
+ 				return null;
+ 			}
+ 			if (!collection.getPermissions().validate(user, Permission.READ)) {
+ 				throw new PermissionDeniedException("permission denied to read collection");
+ 			}
+ 			DocumentImpl doc = collection.getDocumentWithLock(this, docName, lockMode);
+ 			if (doc == null) {
+ 				LOG.debug("document " + docPath + " not found!");
+ 				return null;
+ 			}
+ 	//		if (!doc.getPermissions().validate(user, Permission.READ))
+ 	//			throw new PermissionDeniedException("not allowed to read document");
+ 			return doc;
+ 		} catch (LockException e) {
+ 			LOG.warn("Could not acquire lock on document " + docPath, e);
+ 		} finally {
+ 			collection.release();
+ 		}
+ 		return null;
+ 	}
+ 	
  	public DocumentSet getDocumentsByCollection(String collection, DocumentSet docs)
  		throws PermissionDeniedException {
***************
*** 1483,1537 ****
  			name = name.substring(0, name.length() - 1);
  
! 		Collection current = null;
! 		Lock lock = collectionsDb.getLock();
! 		try {
! 			lock.acquire();
! 			StringTokenizer tok = new StringTokenizer(name, "/");
! 			String temp = tok.nextToken();
! 			String path = ROOT_COLLECTION;
! 			Collection sub;
! 			current = getCollection(ROOT_COLLECTION);
! 			if (current == null) {
! 				LOG.debug("creating root collection /db");
! 				current = new Collection(collectionsDb, ROOT_COLLECTION);
! 				current.getPermissions().setPermissions(0777);
! 				current.getPermissions().setOwner(user);
! 				current.getPermissions().setGroup(user.getPrimaryGroup());
! 				current.setId(getNextCollectionId());
! 				current.setCreationTime(System.currentTimeMillis());
! 				saveCollection(current);
! 			}
! 			while (tok.hasMoreTokens()) {
! 				temp = tok.nextToken();
! 				path = path + "/" + temp;
! 				if (current.hasSubcollection(temp)) {
! 					current = getCollection(path);
! 				} else {
! 					if (!current.getPermissions().validate(user, Permission.WRITE)) {
! 						LOG.debug("permission denied to create collection " + path);
! 						throw new PermissionDeniedException("not allowed to write to collection");
! 					}
! 					LOG.debug("creating collection " + path);
! 					sub = new Collection(collectionsDb, path);
! 					sub.getPermissions().setOwner(user);
! 					sub.getPermissions().setGroup(user.getPrimaryGroup());
! 					sub.setId(getNextCollectionId());
! 					sub.setCreationTime(System.currentTimeMillis());
! 					current.addCollection(sub);
  					saveCollection(current);
- 					current = sub;
  				}
  			}
- 		} catch (ReadOnlyException e) {
- 			LOG.debug("database read-only");
- 			return null;
- 		} catch (LockException e) {
- 			LOG.warn("failed to acquire lock on collections store");
- 		} finally {
- 			lock.release();
  		}
- 		//			LOG.debug("getOrCreateCollection took " + 
- 		//				(System.currentTimeMillis() - start) + "ms.");
- 		return current;
  	}
  
--- 1539,1587 ----
  			name = name.substring(0, name.length() - 1);
  
! 		synchronized(collectionsCache) {
! 			try {
! 				StringTokenizer tok = new StringTokenizer(name, "/");
! 				String temp = tok.nextToken();
! 				String path = ROOT_COLLECTION;
! 				Collection sub;
! 				Collection current = getCollection(ROOT_COLLECTION);
! 				if (current == null) {
! 					LOG.debug("creating root collection /db");
! 					current = new Collection(collectionsDb, ROOT_COLLECTION);
! 					current.getPermissions().setPermissions(0777);
! 					current.getPermissions().setOwner(user);
! 					current.getPermissions().setGroup(user.getPrimaryGroup());
! 					current.setId(getNextCollectionId());
! 					current.setCreationTime(System.currentTimeMillis());
  					saveCollection(current);
  				}
+ 				while (tok.hasMoreTokens()) {
+ 					temp = tok.nextToken();
+ 					path = path + "/" + temp;
+ 					if (current.hasSubcollection(temp)) {
+ 						current = getCollection(path);
+ 					} else {
+ 						if (!current.getPermissions().validate(user, Permission.WRITE)) {
+ 							LOG.debug("permission denied to create collection " + path);
+ 							throw new PermissionDeniedException("not allowed to write to collection");
+ 						}
+ 						LOG.debug("creating collection " + path);
+ 						sub = new Collection(collectionsDb, path);
+ 						sub.getPermissions().setOwner(user);
+ 						sub.getPermissions().setGroup(user.getPrimaryGroup());
+ 						sub.setId(getNextCollectionId());
+ 						sub.setCreationTime(System.currentTimeMillis());
+ 						current.addCollection(sub);
+ 						saveCollection(current);
+ 						current = sub;
+ 					}
+ 				}
+ 				//			LOG.debug("getOrCreateCollection took " + 
+ 				//				(System.currentTimeMillis() - start) + "ms.");
+ 				return current;
+ 			} catch(ReadOnlyException e) {
+ 				throw new PermissionDeniedException(DATABASE_IS_READ_ONLY);
  			}
  		}
  	}
  
***************
*** 1691,1819 ****
  	}
  	
! 	public boolean removeCollection(String name) throws PermissionDeniedException {
  	    if (readOnly)
  	        throw new PermissionDeniedException(DATABASE_IS_READ_ONLY);
- 	    if (!name.startsWith(ROOT_COLLECTION))
- 	        name = ROOT_COLLECTION + name;
  	    
- 	    Collection collection = getCollection(name);
- 	    if (collection == null) {
- 	        LOG.debug("collection " + name + " not found!");
- 	        return false;
- 	    }
  	    if (!collection.getPermissions().validate(user, Permission.WRITE))
! 	        throw new PermissionDeniedException("not allowed to remove collection");
  	    
! 	    String childCollection;
! 	    LOG.debug("removing sub-collections");
! 	    for (Iterator i = collection.collectionIterator(); i.hasNext();) {
! 	        childCollection = (String) i.next();
! 	        removeCollection(
! 	                (name.equals("/")
! 	                        ? "/" + childCollection
! 	                                : name + "/" + childCollection));
! 	    }
! 	    Lock lock = collectionsDb.getLock();
! 	    try {
! 	        lock.acquire(Lock.WRITE_LOCK);
! 	        
! 	        // if this is not the root collection remove it completely
! 	        if (name.equals(ROOT_COLLECTION))
! 	            saveCollection(collection);
! 	        else {
! 	            Value key;
! 	            try {
! 	                key = new Value(name.getBytes("UTF-8"));
! 	            } catch (UnsupportedEncodingException uee) {
! 	                key = new Value(name.getBytes());
! 	            }	
! 	            collectionsDb.remove(key);
! 	        }
! 	        if (!name.equals(ROOT_COLLECTION))
! 	            collectionsDb.getCollectionCache().remove(collection);
! 	        Collection parent = collection.getParent(this);
! 	        if (parent != null) {
! 	            parent.removeCollection(name.substring(name.lastIndexOf("/") + 1));
! 	            saveCollection(parent);
! 	        }
! 	    } catch (LockException e) {
! 	        LOG.warn("Failed to acquire lock on collections.dbx");
! 	    } catch (ReadOnlyException e) {
!             throw new PermissionDeniedException(DATABASE_IS_READ_ONLY);
!         } finally {
! 	        lock.release();
! 	    }
! 	    freeCollection(collection.getId());
  	    
! 	    textEngine.dropIndex(collection);
! 	    elementIndex.dropIndex(collection);
  	    
! 	    LOG.debug("removing dom nodes ...");
! 	    for (Iterator i = collection.iterator(this); i.hasNext();) {
! 	        final DocumentImpl doc = (DocumentImpl) i.next();
! 	        LOG.debug("removing document " + doc.getFileName());
! 	        new DOMTransaction(this, domDb, Lock.WRITE_LOCK) {
! 	            public Object start() {
! 	                if(doc.getResourceType() == DocumentImpl.BINARY_FILE) {
! 	                    domDb.remove(doc.getAddress());
! 	                    domDb.removeOverflowValue(((BinaryDocument)doc).getPage());
! 	                } else {
! 	                    NodeImpl node = (NodeImpl)doc.getFirstChild();
! 	                    domDb.removeAll(node.getInternalAddress());
! 	                }
! 	                return null;
! 	            }
! 	        }
! 	        .run();
! 	        new DOMTransaction(this, domDb, Lock.WRITE_LOCK) {
! 	            public Object start() {
! 	                try {
! 	                    Value ref = new NodeRef(doc.getDocId());
! 	                    IndexQuery query =
! 	                        new IndexQuery(IndexQuery.TRUNC_RIGHT, ref);
! 	                    domDb.remove(query, null);
! 	                    domDb.flush();
! 	                } catch (BTreeException e) {
! 	                    LOG.warn("btree error while removing document", e);
! 	                } catch (DBException e) {
! 	                    LOG.warn("db error while removing document", e);
! 	                } catch (IOException e) {
! 	                    LOG.warn("io error while removing document", e);
! 	                } catch (TerminatedException e) {
! 	                    LOG.warn("method terminated", e);
! 	                }
! 	                return null;
! 	            }
! 	        }
! 	        .run();
! 	        freeDocument(doc.getDocId());
  	    }
- 	    return true;
  	}
  
! 	public void removeDocument(String docName, boolean freeDocId) throws PermissionDeniedException {
  		if (readOnly)
  			throw new PermissionDeniedException(DATABASE_IS_READ_ONLY);
  		try {
- 			if (!docName.startsWith("/"))
- 				docName = '/' + docName;
- 			final DocumentImpl doc = (DocumentImpl) getDocument(docName);
- 			if (doc == null) {
-                 if (LOG.isDebugEnabled()) {
-                     LOG.debug("removeDocument(String) - end - document "
-                         + docName
-                         + " not found");
-                 }
- 				return;
- 			}
              if (LOG.isInfoEnabled()) {
                  LOG.info("removeDocument() - "
                      + "removing document "
!                     + doc.getDocId()
                      + " ...");
              }
  
! 			elementIndex.dropIndex(doc);
! 			textEngine.dropIndex(doc);
              if (LOG.isDebugEnabled()) {
                  LOG.debug("removeDocument() - removing dom");
--- 1741,1868 ----
  	}
  	
! 	public boolean removeCollection(Collection collection) throws PermissionDeniedException {
  	    if (readOnly)
  	        throw new PermissionDeniedException(DATABASE_IS_READ_ONLY);
  	    
  	    if (!collection.getPermissions().validate(user, Permission.WRITE))
! 	    	throw new PermissionDeniedException("not allowed to remove collection");
  	    
! 	    synchronized(collectionsCache) {
! 		    String name = collection.getName();
! 		    // remove from parent collection
! 		    Collection parent = openCollection(collection.getParentPath(), Lock.WRITE_LOCK);
! 		    if (parent != null) {
! 		    	try {
! 		    		parent.removeCollection(name.substring(name.lastIndexOf("/") + 1));
! 		    		saveCollection(parent);
! 		    	} catch (LockException e) {
! 		    		LOG.warn("LockException while removing collection " + name);
! 		    	} finally {
! 		    		parent.getLock().release();
! 		    	}
! 		    }
! 		    
! 		    // remove child collections
! 		    String childName;
! 		    Collection childCollection;
! 		    LOG.debug("removing sub-collections");
! 		    for (Iterator i = collection.collectionIterator(); i.hasNext();) {
! 		    	childName = (String) i.next();
! 		    	childCollection = openCollection(name + '/' + childName, Lock.WRITE_LOCK);
! 		    	try {
! 		    		removeCollection(childCollection);
! 		    	} finally {
! 		    		childCollection.getLock().release();
! 		    	}
! 		    }
! 		    
! 		    Lock lock = collectionsDb.getLock();
! 		    try {
! 		    	lock.acquire(Lock.WRITE_LOCK);
! 		    	
! 		    	// if this is not the root collection remove it completely
! 		    	if (name.equals(ROOT_COLLECTION))
! 		    		saveCollection(collection);
! 		    	else {
! 		    		Value key;
! 		    		try {
! 		    			key = new Value(name.getBytes("UTF-8"));
! 		    		} catch (UnsupportedEncodingException uee) {
! 		    			key = new Value(name.getBytes());
! 		    		}	
! 		    		collectionsDb.remove(key);
! 		    	}
! 		    	if (!name.equals(ROOT_COLLECTION))
! 		    		collectionsCache.remove(collection);
! 		    } catch (LockException e) {
! 		    	LOG.warn("Failed to acquire lock on collections.dbx");
! 		    } catch (ReadOnlyException e) {
! 		    	throw new PermissionDeniedException(DATABASE_IS_READ_ONLY);
! 		    } finally {
! 		    	lock.release();
! 		    }
  	    
! 		    freeCollection(collection.getId());
  	    
! 		    textEngine.dropIndex(collection);
! 		    elementIndex.dropIndex(collection);
! 		    
! 		    LOG.debug("removing resources ...");
! 		    for (Iterator i = collection.iterator(this); i.hasNext();) {
! 		    	final DocumentImpl doc = (DocumentImpl) i.next();
! 		    	LOG.debug("removing document " + doc.getFileName());
! 		    	new DOMTransaction(this, domDb, Lock.WRITE_LOCK) {
! 		    		public Object start() {
! 		    			if(doc.getResourceType() == DocumentImpl.BINARY_FILE) {
! 		    				domDb.remove(doc.getAddress());
! 		    				domDb.removeOverflowValue(((BinaryDocument)doc).getPage());
! 		    			} else {
! 		    				NodeImpl node = (NodeImpl)doc.getFirstChild();
! 		    				domDb.removeAll(node.getInternalAddress());
! 		    			}
! 		    			return null;
! 		    		}
! 		    	}
! 		    	.run();
! 		    	new DOMTransaction(this, domDb, Lock.WRITE_LOCK) {
! 		    		public Object start() {
! 		    			try {
! 		    				Value ref = new NodeRef(doc.getDocId());
! 		    				IndexQuery query =
! 		    					new IndexQuery(IndexQuery.TRUNC_RIGHT, ref);
! 		    				domDb.remove(query, null);
! 		    				domDb.flush();
! 		    			} catch (BTreeException e) {
! 		    				LOG.warn("btree error while removing document", e);
! 		    			} catch (DBException e) {
! 		    				LOG.warn("db error while removing document", e);
! 		    			} catch (IOException e) {
! 		    				LOG.warn("io error while removing document", e);
! 		    			} catch (TerminatedException e) {
! 		    				LOG.warn("method terminated", e);
! 		    			}
! 		    			return null;
! 		    		}
! 		    	}
! 		    	.run();
! 		    	freeDocument(doc.getDocId());
! 		    }
! 		    return true;
  	    }
  	}
  
! 	public void removeDocument(final DocumentImpl document, boolean freeDocId) throws PermissionDeniedException {
  		if (readOnly)
  			throw new PermissionDeniedException(DATABASE_IS_READ_ONLY);
  		try {
              if (LOG.isInfoEnabled()) {
                  LOG.info("removeDocument() - "
                      + "removing document "
!                     + document.getDocId()
                      + " ...");
              }
  
! 			elementIndex.dropIndex(document);
! 			textEngine.dropIndex(document);
              if (LOG.isDebugEnabled()) {
                  LOG.debug("removeDocument() - removing dom");
***************
*** 1821,1825 ****
  			new DOMTransaction(this, domDb) {
  				public Object start() {
! 				    NodeImpl node = (NodeImpl)doc.getFirstChild();
  					domDb.removeAll(node.getInternalAddress());
  					return null;
--- 1870,1874 ----
  			new DOMTransaction(this, domDb) {
  				public Object start() {
! 				    NodeImpl node = (NodeImpl)document.getFirstChild();
  					domDb.removeAll(node.getInternalAddress());
  					return null;
***************
*** 1828,1832 ****
  			.run();
  			
! 			NodeRef ref = new NodeRef(doc.getDocId());
  			final IndexQuery idx = new IndexQuery(IndexQuery.TRUNC_RIGHT, ref);
  			new DOMTransaction(this, domDb) {
--- 1877,1881 ----
  			.run();
  			
! 			NodeRef ref = new NodeRef(document.getDocId());
  			final IndexQuery idx = new IndexQuery(IndexQuery.TRUNC_RIGHT, ref);
  			new DOMTransaction(this, domDb) {
***************
*** 1849,1853 ****
  			.run();
  			if(freeDocId)
! 			    freeDocument(doc.getDocId());
  		} catch (ReadOnlyException e) {
              LOG.warn("removeDocument(String) - " + DATABASE_IS_READ_ONLY);
--- 1898,1902 ----
  			.run();
  			if(freeDocId)
! 			    freeDocument(document.getDocId());
  		} catch (ReadOnlyException e) {
              LOG.warn("removeDocument(String) - " + DATABASE_IS_READ_ONLY);
***************
*** 1960,1963 ****
--- 2009,2013 ----
  		if (readOnly)
  			throw new PermissionDeniedException(DATABASE_IS_READ_ONLY);
+ 		collectionsCache.add(collection);
  		Lock lock = null;
  		try {
***************
*** 1985,1989 ****
  				}
  				collection.setAddress(addr);
- 				collectionsDb.getCollectionCache().add(collection);
  				ostream.close();
  			} catch (IOException ioe) {
--- 2035,2038 ----
***************
*** 2080,2083 ****
--- 2129,2141 ----
  	    if(newName.indexOf('/') > -1)
  	        throw new PermissionDeniedException("New collection name is illegal (may not contain a '/')");
+ 	    //	check if another collection with the same name exists at the destination
+ 	    Collection old = openCollection(newName, Lock.WRITE_LOCK);
+ 	    if(old != null) {
+ 	    	try {
+ 	    		removeCollection(old);
+ 	    	} finally {
+ 	    		old.release();
+ 	    	}
+ 	    }
  	    Lock lock = null;
  	    try {
***************
*** 2085,2092 ****
  	        lock.acquire(Lock.WRITE_LOCK);
  	        newName = destination.getName() + '/' + newName;
! 	        // check if another collection with the same name exists at the destination
! 		    Collection old = getCollection(newName);
! 		    if(old != null)
! 		        removeCollection(old.getName());
  		    Collection destCollection = getOrCreateCollection(newName);
  		    for(Iterator i = collection.iterator(this); i.hasNext(); ) {
--- 2143,2147 ----
  	        lock.acquire(Lock.WRITE_LOCK);
  	        newName = destination.getName() + '/' + newName;
! 	        
  		    Collection destCollection = getOrCreateCollection(newName);
  		    for(Iterator i = collection.iterator(this); i.hasNext(); ) {
***************
*** 2126,2176 ****
  	    if(newName.indexOf('/') > -1)
  	        throw new PermissionDeniedException("New collection name is illegal (may not contain a '/')");
- 	    Lock lock = null;
- 	    try {
- 	        lock = collectionsDb.getLock();
- 	        lock.acquire(Lock.WRITE_LOCK);
  	        // check if another collection with the same name exists at the destination
! 		    Collection old = getCollection(destination.getName() + '/' + newName);
! 		    if(old != null)
! 		        removeCollection(old.getName());
! 		    
! 	        String name = collection.getName();
! 	        Collection parent = collection.getParent(this);
! 	        if(parent != null)
! 	            parent.removeCollection(name.substring(name.lastIndexOf("/") + 1));
! 		    
! 	        collectionsDb.getCollectionCache().remove(collection);
! 		    Value key;
! 			try {
! 				key = new Value(name.getBytes("UTF-8"));
! 			} catch (UnsupportedEncodingException uee) {
! 				key = new Value(name.getBytes());
! 			}	
! 			collectionsDb.remove(key);
! 			
! 		    collection.setName(destination.getName() + '/' + newName);
! 		    collection.setCreationTime(System.currentTimeMillis());
! 		    
! 		    destination.addCollection(collection);
! 		    if(parent != null)
! 		        saveCollection(parent);
! 		    if(parent != destination)
! 		        saveCollection(destination);
! 		    saveCollection(collection);
! 		    
! 		    String childName;
  		    Collection child;
  		    for(Iterator i = collection.collectionIterator(); i.hasNext(); ) {
  		        childName = (String)i.next();
! 		        child = getCollection(name + '/' + childName);
  		        if(child == null)
  		            LOG.warn("Child collection " + childName + " not found");
! 		        else
! 		            moveCollection(child, collection, childName);
  		    }
- 	    } catch (ReadOnlyException e) {
-             throw new PermissionDeniedException(DATABASE_IS_READ_ONLY);
-         } finally {
- 	        lock.release();
  	    }
  	}
--- 2181,2246 ----
  	    if(newName.indexOf('/') > -1)
  	        throw new PermissionDeniedException("New collection name is illegal (may not contain a '/')");
  	        // check if another collection with the same name exists at the destination
! 	    Collection old = openCollection(destination.getName() + '/' + newName, Lock.WRITE_LOCK);
! 	    if(old != null) {
! 	    	try {
! 	    		removeCollection(old);
! 	    	} finally {
! 	    		old.release();
! 	    	}
! 	    }
! 	    String name = collection.getName();
! 	    synchronized(collectionsCache) {
! 		    Collection parent = openCollection(collection.getParentPath(), Lock.WRITE_LOCK);
! 	        if(parent != null) {
! 	        	try {
! 	        		parent.removeCollection(name.substring(name.lastIndexOf("/") + 1));
! 	        	} finally {
! 	        		parent.release();
! 	        	}
! 	        }
! 		    Lock lock = null;
! 		    try {
! 		    	lock = collectionsDb.getLock();
! 		    	lock.acquire(Lock.WRITE_LOCK);
! 			    
! 		        collectionsCache.remove(collection);
! 			    Value key;
! 				try {
! 					key = new Value(name.getBytes("UTF-8"));
! 				} catch (UnsupportedEncodingException uee) {
! 					key = new Value(name.getBytes());
! 				}	
! 				collectionsDb.remove(key);
! 				
! 			    collection.setName(destination.getName() + '/' + newName);
! 			    collection.setCreationTime(System.currentTimeMillis());
! 			    
! 			    destination.addCollection(collection);
! 			    if(parent != null)
! 			        saveCollection(parent);
! 			    if(parent != destination)
! 			        saveCollection(destination);
! 			    saveCollection(collection);
! 		    } catch (ReadOnlyException e) {
! 	            throw new PermissionDeniedException(DATABASE_IS_READ_ONLY);
! 	        } finally {
! 		        lock.release();
! 		    }
! 	        String childName;
  		    Collection child;
  		    for(Iterator i = collection.collectionIterator(); i.hasNext(); ) {
  		        childName = (String)i.next();
! 		        child = openCollection(name + '/' + childName, Lock.WRITE_LOCK);
  		        if(child == null)
  		            LOG.warn("Child collection " + childName + " not found");
! 		        else {
! 		            try {
! 		            	moveCollection(child, collection, childName);
! 		            } finally {
! 		            	child.release();
! 		            }
! 		        }
  		    }
  	    }
  	}
***************
*** 2385,2389 ****
  				}
  				doc.setAddress(domDb.add(data));
! //				LOG.debug("Document metadata stored to " + StorageAddress.toString(doc.getAddress()));
  				return null;
  			}
--- 2455,2459 ----
  				}
  				doc.setAddress(domDb.add(data));
! 				LOG.debug("Document metadata stored to " + StorageAddress.toString(doc.getAddress()));
  				return null;
  			}
***************
*** 2393,2404 ****
  
  	public void updateDocument(DocumentImpl doc) throws LockException, PermissionDeniedException {
! 		Lock lock = collectionsDb.getLock();
! 		try {
! 			lock.acquire(Lock.WRITE_LOCK);
! 			storeDocument(doc);
! 			saveCollection(doc.getCollection());
! 		} finally {
! 			lock.release();
! 		}
  	}
  	
--- 2463,2468 ----
  
  	public void updateDocument(DocumentImpl doc) throws LockException, PermissionDeniedException {
! 		storeDocument(doc);
! 		saveCollection(doc.getCollection());
  	}
  	

Index: DBBroker.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/storage/DBBroker.java,v
retrieving revision 1.39
retrieving revision 1.40
diff -C2 -d -r1.39 -r1.40
*** DBBroker.java	28 Jul 2004 18:54:54 -0000	1.39
--- DBBroker.java	3 Aug 2004 15:25:58 -0000	1.40
***************
*** 34,37 ****
--- 34,38 ----
  import org.exist.EXistException;
  import org.exist.collections.Collection;
+ import org.exist.collections.CollectionCache;
  import org.exist.dom.BinaryDocument;
  import org.exist.dom.DocumentImpl;
***************
*** 73,76 ****
--- 74,83 ----
  	public final static int DBM = 3;
  
+ 	//	size of the internal buffer for collection objects
+ 	public static final int COLLECTION_BUFFER_SIZE = 128;
+ 	
+ 	protected static CollectionCache collectionsCache = 
+ 		new CollectionCache(COLLECTION_BUFFER_SIZE);
+ 	
  	protected final static Logger LOG = Logger.getLogger(DBBroker.class);
  	
***************
*** 149,152 ****
--- 156,163 ----
  	}
  
+ 	public CollectionCache getCollectionsCache() {
+ 		return collectionsCache;
+ 	}
+ 	
  	public DBBroker(BrokerPool pool, Configuration config)
  		throws EXistException {
***************
*** 235,238 ****
--- 246,271 ----
  
  	/**
+ 	 * Returns the database collection identified by the specified path.
+ 	 * The storage address is used to locate the collection without
+ 	 * looking up the path in the btree.
+ 	 * 
+ 	 * @return
+ 	 */
+ 	public abstract Collection getCollection(String name, long address);
+ 	
+ 	/**
+ 	 * Open a collection for reading or writing. The collection is identified by its
+ 	 * absolute path, e.g. /db/system. It will be loaded and locked according to the
+ 	 * lockMode argument. 
+ 	 * 
+ 	 * The caller should take care to release the collection lock properly.
+ 	 * 
+ 	 * @param name the collection path
+ 	 * @param lockMode one of the modes specified in class {@link org.exist.util.Lock}
+ 	 * @return collection or null if no collection matches the path
+ 	 */
+ 	public abstract Collection openCollection(String name, int lockMode);
+ 	
+ 	/**
  	 *  Returns the database collection identified by the specified path.
  	 * If the collection does not yet exist, it is created - including all
***************
*** 245,257 ****
  		return null;
  	}
- 	
- 	/**
- 	 * Returns the database collection identified by the specified path.
- 	 * The storage address is used to locate the collection without
- 	 * looking up the path in the btree.
- 	 * 
- 	 * @return
- 	 */
- 	public abstract Collection getCollection(String name, long address);
  
  	public abstract void reloadCollection(Collection collection);
--- 278,281 ----
***************
*** 305,308 ****
--- 329,335 ----
  		throws PermissionDeniedException;
  
+ 	public abstract Document openDocument(String docPath, int lockMode) 
+ 		throws PermissionDeniedException;
+ 	
  	/**
  	 *  Returns a DocumentSet containing all the documents found in the
***************
*** 396,400 ****
  	 * 
  	 */
! 	public abstract boolean removeCollection(String name)
  		throws PermissionDeniedException;
  
--- 423,427 ----
  	 * 
  	 */
! 	public abstract boolean removeCollection(Collection collection)
  		throws PermissionDeniedException;
  
***************
*** 403,412 ****
  	 *
  	 */
! 	public void removeDocument(String docName)
  	throws PermissionDeniedException {
! 	    removeDocument(docName, true);
  	}
  	
! 	public abstract void removeDocument(String docName, boolean freeDocId)
  		throws PermissionDeniedException;
  
--- 430,439 ----
  	 *
  	 */
! 	public void removeDocument(DocumentImpl document)
  	throws PermissionDeniedException {
! 	    removeDocument(document, true);
  	}
  	
! 	public abstract void removeDocument(DocumentImpl document, boolean freeDocId)
  		throws PermissionDeniedException;
  

Index: BrokerPool.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/storage/BrokerPool.java,v
retrieving revision 1.25
retrieving revision 1.26
diff -C2 -d -r1.25 -r1.26
*** BrokerPool.java	28 Jul 2004 18:54:54 -0000	1.25
--- BrokerPool.java	3 Aug 2004 15:25:58 -0000	1.26
***************
*** 250,260 ****
  		}
  		synchronized(this) {
- 			// force the thread to wait until a pending sync has finished
- 			while(syncRequired && threads.size() > 0) {
- 				try {
- 					this.wait();
- 				} catch(InterruptedException e) {
- 				}
- 			}
  			if (pool.isEmpty()) {
  				if (brokers < max)
--- 250,253 ----
***************
*** 272,276 ****
  			threads.put(Thread.currentThread(), broker);
  			broker.incReferenceCount();
! //			this.notifyAll();
  			return broker;
  		}
--- 265,269 ----
  			threads.put(Thread.currentThread(), broker);
  			broker.incReferenceCount();
! 			this.notifyAll();
  			return broker;
  		}

[Exist-commits] eXist-1.0/src/org/exist/soap AdminSoapBindingImpl.java,1.15,1.16

[Wolfgang M. M. <wol...@us...>]

Update of /cvsroot/exist/eXist-1.0/src/org/exist/soap
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8244/src/org/exist/soap

Modified Files:
	AdminSoapBindingImpl.java 
Log Message:
Revised collection locking to fix various concurrency errors. It is now the responsibility of the 
caller to lock/unlock a collection. A collection can be retrieved and locked via the new 
openCollection() method provided by DBBroker. After reading/modifying the collection, it should be 
unlocked by calling Collection.release.

The local XML:DB implementation and the XMLRPC interface have been changed to reflect the new
collection locking rules.

Index: AdminSoapBindingImpl.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/soap/AdminSoapBindingImpl.java,v
retrieving revision 1.15
retrieving revision 1.16
diff -C2 -d -r1.15 -r1.16
*** AdminSoapBindingImpl.java	2 Jul 2004 16:53:55 -0000	1.15
--- AdminSoapBindingImpl.java	3 Aug 2004 15:25:59 -0000	1.16
***************
*** 90,94 ****
  	}
  
! 	public boolean removeCollection(String sessionId, String collection)
  		throws RemoteException {
  		Session session = getSession(sessionId);
--- 90,94 ----
  	}
  
! 	public boolean removeCollection(String sessionId, String name)
  		throws RemoteException {
  		Session session = getSession(sessionId);
***************
*** 96,100 ****
  		try {
  			broker = pool.get(session.getUser());
! 			if (broker.getCollection(collection) == null)
  				return false;
  			return broker.removeCollection(collection);
--- 96,101 ----
  		try {
  			broker = pool.get(session.getUser());
! 			Collection collection = broker.getCollection(name);
! 			if(collection == null)
  				return false;
  			return broker.removeCollection(collection);

[Exist-commits] eXist-1.0/src/org/exist/http RESTServer.java,1.14,1.15

[Wolfgang M. M. <wol...@us...>]

Update of /cvsroot/exist/eXist-1.0/src/org/exist/http
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8244/src/org/exist/http

Modified Files:
	RESTServer.java 
Log Message:
Revised collection locking to fix various concurrency errors. It is now the responsibility of the 
caller to lock/unlock a collection. A collection can be retrieved and locked via the new 
openCollection() method provided by DBBroker. After reading/modifying the collection, it should be 
unlocked by calling Collection.release.

The local XML:DB implementation and the XMLRPC interface have been changed to reflect the new
collection locking rules.

Index: RESTServer.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/http/RESTServer.java,v
retrieving revision 1.14
retrieving revision 1.15
diff -C2 -d -r1.14 -r1.15
*** RESTServer.java	2 Jul 2004 16:53:56 -0000	1.14
--- RESTServer.java	3 Aug 2004 15:26:00 -0000	1.15
***************
*** 395,399 ****
  				// remove the collection
  				LOG.debug("removing collection " + path);
! 				broker.removeCollection(path);
  				response = new Response();
  				response.setDescription("Collection " + path + " removed.");
--- 395,399 ----
  				// remove the collection
  				LOG.debug("removing collection " + path);
! 				broker.removeCollection(collection);
  				response = new Response();
  				response.setDescription("Collection " + path + " removed.");

[Exist-commits] eXist-1.0/src/org/exist/storage/cache LRDCache.java,1.6,1.7

[Wolfgang M. M. <wol...@us...>]

Update of /cvsroot/exist/eXist-1.0/src/org/exist/storage/cache
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8244/src/org/exist/storage/cache

Modified Files:
	LRDCache.java 
Log Message:
Revised collection locking to fix various concurrency errors. It is now the responsibility of the 
caller to lock/unlock a collection. A collection can be retrieved and locked via the new 
openCollection() method provided by DBBroker. After reading/modifying the collection, it should be 
unlocked by calling Collection.release.

The local XML:DB implementation and the XMLRPC interface have been changed to reflect the new
collection locking rules.

Index: LRDCache.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/storage/cache/LRDCache.java,v
retrieving revision 1.6
retrieving revision 1.7
diff -C2 -d -r1.6 -r1.7
*** LRDCache.java	28 Jul 2004 18:54:54 -0000	1.6
--- LRDCache.java	3 Aug 2004 15:25:59 -0000	1.7
***************
*** 41,45 ****
  public class LRDCache extends GClockCache {
  	
! 	private int totalReferences = 0;
  
  	private int nextCleanup;
--- 41,45 ----
  public class LRDCache extends GClockCache {
  	
! 	protected int totalReferences = 0;
  
  	private int nextCleanup;

[Exist-commits] eXist-1.0/src/org/exist/storage/store CollectionStore.java,1.5,1.6 DOMFile.java,1.43,1.44

[Wolfgang M. M. <wol...@us...>]

Update of /cvsroot/exist/eXist-1.0/src/org/exist/storage/store
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8244/src/org/exist/storage/store

Modified Files:
	CollectionStore.java DOMFile.java 
Log Message:
Revised collection locking to fix various concurrency errors. It is now the responsibility of the 
caller to lock/unlock a collection. A collection can be retrieved and locked via the new 
openCollection() method provided by DBBroker. After reading/modifying the collection, it should be 
unlocked by calling Collection.release.

The local XML:DB implementation and the XMLRPC interface have been changed to reflect the new
collection locking rules.

Index: DOMFile.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/storage/store/DOMFile.java,v
retrieving revision 1.43
retrieving revision 1.44
diff -C2 -d -r1.43 -r1.44
*** DOMFile.java	28 Jul 2004 18:54:56 -0000	1.43
--- DOMFile.java	3 Aug 2004 15:25:59 -0000	1.44
***************
*** 1192,1200 ****
      
      public void removeAll(long p) {
! //    	StringBuffer debug = new StringBuffer();
! //    	debug.append("Removed pages: ");
          long pnum = StorageAddress.pageFromPointer(p);
          while(-1 < pnum) {
! //        	debug.append(' ').append(pnum);
              DOMPage page = getCurrentPage(pnum);
              pnum = page.getPageHeader().getNextDataPage();
--- 1192,1200 ----
      
      public void removeAll(long p) {
!     	StringBuffer debug = new StringBuffer();
!     	debug.append("Removed pages: ");
          long pnum = StorageAddress.pageFromPointer(p);
          while(-1 < pnum) {
!         	debug.append(' ').append(pnum);
              DOMPage page = getCurrentPage(pnum);
              pnum = page.getPageHeader().getNextDataPage();
***************
*** 1213,1217 ****
              }
          }
! //        LOG.debug(debug.toString());
      }
      
--- 1213,1217 ----
              }
          }
!         LOG.debug(debug.toString());
      }
      

Index: CollectionStore.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/storage/store/CollectionStore.java,v
retrieving revision 1.5
retrieving revision 1.6
diff -C2 -d -r1.5 -r1.6
*** CollectionStore.java	4 Jun 2004 09:47:25 -0000	1.5
--- CollectionStore.java	3 Aug 2004 15:25:59 -0000	1.6
***************
*** 8,21 ****
  import java.io.File;
  
- import org.exist.collections.CollectionCache;
  import org.exist.storage.BrokerPool;
  
  public class CollectionStore extends BFile {
  
- 	//	size of the internal buffer for collection objects
- 	public static final int COLLECTION_BUFFER_SIZE = 128;
- 	
- 	private CollectionCache collectionsCache = new CollectionCache(COLLECTION_BUFFER_SIZE);
- 
  	/**
  	 * @param file
--- 8,15 ----
***************
*** 26,33 ****
  		super(pool, file, btreeBuffers, dataBuffers);
  	}
- 
- 	public CollectionCache getCollectionCache() {
- 		return collectionsCache;
- 	}
  	
  	
--- 20,23 ----

[Exist-commits] eXist-1.0/src/org/exist/http/webdav WebDAV.java,1.5,1.6

[Wolfgang M. M. <wol...@us...>]

Update of /cvsroot/exist/eXist-1.0/src/org/exist/http/webdav
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8244/src/org/exist/http/webdav

Modified Files:
	WebDAV.java 
Log Message:
Revised collection locking to fix various concurrency errors. It is now the responsibility of the 
caller to lock/unlock a collection. A collection can be retrieved and locked via the new 
openCollection() method provided by DBBroker. After reading/modifying the collection, it should be 
unlocked by calling Collection.release.

The local XML:DB implementation and the XMLRPC interface have been changed to reflect the new
collection locking rules.

Index: WebDAV.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/http/webdav/WebDAV.java,v
retrieving revision 1.5
retrieving revision 1.6
diff -C2 -d -r1.5 -r1.6
*** WebDAV.java	21 Jun 2004 15:27:40 -0000	1.5
--- WebDAV.java	3 Aug 2004 15:26:00 -0000	1.6
***************
*** 43,46 ****
--- 43,47 ----
  import org.exist.storage.DBBroker;
  import org.exist.storage.serializers.EXistOutputKeys;
+ import org.exist.util.Lock;
  
  /**
***************
*** 122,131 ****
  			broker = pool.get(user);
  			
- 			collection = broker.getCollection(path);
- 			if(collection == null) {
- 				resource = (DocumentImpl)broker.getDocument(path);
- 				if(resource != null)
- 					collection = resource.getCollection();
- 			}
  			method = WebDAVMethodFactory.create(request.getMethod(), pool);
  			if(method == null) {
--- 123,126 ----
***************
*** 134,137 ****
--- 129,138 ----
  				return;
  			}
+ 			collection = broker.openCollection(path, Lock.READ_LOCK);
+ 			if(collection == null) {
+ 				resource = (DocumentImpl)broker.openDocument(path, Lock.READ_LOCK);
+ 				if(resource != null)
+ 					collection = resource.getCollection();
+ 			}
  		} catch (EXistException e) {
  			throw new ServletException("An error occurred while retrieving resource: " + e.getMessage(), e);
***************
*** 141,145 ****
  			pool.release(broker);
  		}
! 		method.process(user, request, response, collection, resource);
  	}
  	
--- 142,153 ----
  			pool.release(broker);
  		}
! 		try {
! 			method.process(user, request, response, collection, resource);
! 		} finally {
! 			if(collection != null)
! 				collection.release();
! 			if(resource != null)
! 				resource.getUpdateLock().release();
! 		}
  	}
  	

[Exist-commits] eXist-1.0/src/org/exist Indexer.java,1.13,1.14

[Wolfgang M. M. <wol...@us...>]

Update of /cvsroot/exist/eXist-1.0/src/org/exist
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8244/src/org/exist

Modified Files:
	Indexer.java 
Log Message:
Revised collection locking to fix various concurrency errors. It is now the responsibility of the 
caller to lock/unlock a collection. A collection can be retrieved and locked via the new 
openCollection() method provided by DBBroker. After reading/modifying the collection, it should be 
unlocked by calling Collection.release.

The local XML:DB implementation and the XMLRPC interface have been changed to reflect the new
collection locking rules.

Index: Indexer.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/Indexer.java,v
retrieving revision 1.13
retrieving revision 1.14
diff -C2 -d -r1.13 -r1.14
*** Indexer.java	27 Jul 2004 13:42:48 -0000	1.13
--- Indexer.java	3 Aug 2004 15:26:00 -0000	1.14
***************
*** 57,60 ****
--- 57,61 ----
  import org.xml.sax.SAXNotSupportedException;
  import org.xml.sax.SAXParseException;
+ import org.xml.sax.XMLReader;
  import org.xml.sax.ext.LexicalHandler;
  
***************
*** 80,83 ****
--- 81,85 ----
  	
  	protected DocumentImpl document = null;
+ 	protected XMLReader reader = null;
  	protected boolean insideDTD = false;
  	protected boolean validate = false;
***************
*** 152,155 ****
--- 154,165 ----
  	}
  
+ 	public void setReader(XMLReader reader) {
+ 		this.reader = reader;
+ 	}
+ 	
+ 	public XMLReader getReader() {
+ 		return this.reader;
+ 	}
+ 	
  	/**
  	 * Prepare the indexer for parsing a new document. This will
***************
*** 178,181 ****
--- 188,195 ----
  	}
  
+ 	public DocumentImpl getDocument() {
+ 		return document;
+ 	}
+ 	
  	public void characters(char[] ch, int start, int length) {
  		if (length <= 0)

[Exist-commits] eXist-1.0/src/org/exist/collections Collection.java,1.43,1.44 CollectionCache.java,1.6,1.7

[Wolfgang M. M. <wol...@us...>]

Update of /cvsroot/exist/eXist-1.0/src/org/exist/collections
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8244/src/org/exist/collections

Modified Files:
	Collection.java CollectionCache.java 
Log Message:
Revised collection locking to fix various concurrency errors. It is now the responsibility of the 
caller to lock/unlock a collection. A collection can be retrieved and locked via the new 
openCollection() method provided by DBBroker. After reading/modifying the collection, it should be 
unlocked by calling Collection.release.

The local XML:DB implementation and the XMLRPC interface have been changed to reflect the new
collection locking rules.

Index: CollectionCache.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/collections/CollectionCache.java,v
retrieving revision 1.6
retrieving revision 1.7
diff -C2 -d -r1.6 -r1.7
*** CollectionCache.java	25 May 2004 12:52:01 -0000	1.6
--- CollectionCache.java	3 Aug 2004 15:25:58 -0000	1.7
***************
*** 3,6 ****
--- 3,7 ----
  import org.exist.storage.cache.Cacheable;
  import org.exist.storage.cache.LRDCache;
+ import org.exist.util.Lock;
  import org.exist.util.hashtable.Object2LongHashMap;
  
***************
*** 21,25 ****
  		names = new Object2LongHashMap(blockBuffers);
  	}
! 
  	public void add(Collection collection) {
  		add(collection, 1);
--- 22,26 ----
  		names = new Object2LongHashMap(blockBuffers);
  	}
! 	
  	public void add(Collection collection) {
  		add(collection, 1);
***************
*** 42,50 ****
  	}
  
  	protected Cacheable removeOne(Cacheable item) {
! 		Cacheable old = super.removeOne(item);
! 		if(old != null) {
! 			names.remove(((Collection)old).getName());
  		}
  		return old;
  	}
--- 43,86 ----
  	}
  
+ 	/**
+ 	 * Overwritten to lock collections before they are removed.
+ 	 */
  	protected Cacheable removeOne(Cacheable item) {
! 		Collection old;
! 		Lock lock;
! 		double rd = 0, minRd = -1;
! 		int bucket = -1;
! 		boolean unloadable;
! 		for (int i = 0; i < items.length; i++) {
! 			old = (Collection)items[i];
! 			if (old == null) {
! 				bucket = i;
! 				break;
! 			} else {
! 				lock = old.getLock(); 
! 					// calculate the reference density
! 					rd =
! 						old.getReferenceCount()
! 							/ (double)(totalReferences - old.getTimestamp());
! 					// attempt to acquire a read lock on the collection.
! 					// the collection is not considered for removal if the lock 
! 					// cannot be acquired immediately.
! 					if(lock.attempt(Lock.READ_LOCK)) {
! 						if ((minRd < 0 || rd < minRd) && old.allowUnload()) {
! 							minRd = rd;
! 							bucket = i;
! 						}
! 						lock.release();
! 					}
! 			}
  		}
+ 		old = (Collection)items[bucket];
+ 		if (old != null) {
+ 			map.remove(old.getKey());
+ 			names.remove(old.getName());
+ 			old.sync();
+ 		}
+ 		items[bucket] = item;
+ 		map.put(item.getKey(), item);
  		return old;
  	}

Index: Collection.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/collections/Collection.java,v
retrieving revision 1.43
retrieving revision 1.44
diff -C2 -d -r1.43 -r1.44
*** Collection.java	28 Jul 2004 19:40:42 -0000	1.43
--- Collection.java	3 Aug 2004 15:25:58 -0000	1.44
***************
*** 77,83 ****
  
  /**
!  * This class represents a collection in the database.
   * 
!  * A collection maintains a list of sub-collections and documents.
   *  
   * @author wolf
--- 77,86 ----
  
  /**
!  * This class represents a collection in the database. A collection maintains a list of 
!  * sub-collections and documents, and provides the methods to store/remove resources.
   * 
!  * Collections are shared between {@link org.exist.storage.DBBroker} instances. The caller
!  * is responsible to lock/unlock the collection. Call {@link DBBroker#openCollection(String, int)}  
!  * to get a collection with a read or write lock and {@link #release()} to release the lock.
   *  
   * @author wolf
***************
*** 149,152 ****
--- 152,159 ----
  	}
  	
+ 	public Lock getLock() {
+ 		return lock;
+ 	}
+ 	
  	/**
  	 *  Add a new sub-collection to the collection.
***************
*** 161,164 ****
--- 168,184 ----
  	}
  
+ 	public boolean hasChildCollection(String name) {
+ 		return subcollections.contains(name);
+ 	}
+ 	
+ 	/**
+ 	 * Closes the collection, i.e. releases the lock held by 
+ 	 * the current thread. This is a shortcut for getLock().release().
+ 	 */
+ 	public void release() {
+ //		LOG.debug("releasing lock on " + name);
+ 		lock.release();
+ 	}
+ 	
  	/**
  	 * Update the specified child-collection.
***************
*** 272,278 ****
  			boolean recursive, boolean checkPermissions) {
  		if (permissions.validate(broker.getUser(), Permission.READ)) {
! 			getDocuments(broker, docs, checkPermissions);
! 			if (recursive)
! 				allDocs(broker, docs, checkPermissions);
  		}
  		return docs;
--- 292,301 ----
  			boolean recursive, boolean checkPermissions) {
  		if (permissions.validate(broker.getUser(), Permission.READ)) {
! 			CollectionCache cache = broker.getCollectionsCache();
! 			synchronized (cache) {
! 				getDocuments(broker, docs, checkPermissions);
! 				if (recursive)
! 					allDocs(broker, docs, checkPermissions);
! 			}
  		}
  		return docs;
***************
*** 290,294 ****
  				if(child == null) {
  					LOG.warn("child collection " + childName + " not found. Skipping ...");
! 				// we always check if we have permissions to read the child collection
  				} else if (child.permissions.validate(broker.getUser(), Permission.READ)) {
  					child.getDocuments(broker, docs, checkPermissions);
--- 313,317 ----
  				if(child == null) {
  					LOG.warn("child collection " + childName + " not found. Skipping ...");
! 					// we always check if we have permissions to read the child collection
  				} else if (child.permissions.validate(broker.getUser(), Permission.READ)) {
  					child.getDocuments(broker, docs, checkPermissions);
***************
*** 446,449 ****
--- 469,498 ----
  
  	/**
+ 	 * Retrieve a child resource after putting a read lock on it. With this method,
+ 	 * access to the received document object is safe. 
+ 	 * 
+ 	 * @param broker
+ 	 * @param name
+ 	 * @return
+ 	 * @throws LockException
+ 	 */
+ 	public DocumentImpl getDocumentWithLock(DBBroker broker, String name, int lockMode) 
+ 	throws LockException {
+ 	    try {
+ 	        lock.acquire(Lock.READ_LOCK);
+ 	        if(reloadRequired) {
+ 			    broker.reloadCollection(this);
+ 			    reloadRequired = false;
+ 			}
+ 	        DocumentImpl doc = (DocumentImpl) documents.get(name);
+ 	        Lock updateLock = doc.getUpdateLock();
+ 	        updateLock.acquire(lockMode);
+ 	        return doc;
+         } finally {
+ 	        lock.release();
+ 	    }
+ 	}
+ 	
+ 	/**
  	 * Release any locks held on the document.
  	 * 
***************
*** 497,506 ****
  	 *is the root collection.
  	 */
! 	public Collection getParent(DBBroker broker) {
  		if (name.equals("/db"))
  			return null;
  		String parent = (name.lastIndexOf("/") < 1 ? "/db" : name.substring(0,
  				name.lastIndexOf("/")));
! 		return broker.getCollection(parent);
  	}
  
--- 546,555 ----
  	 *is the root collection.
  	 */
! 	public String getParentPath() {
  		if (name.equals("/db"))
  			return null;
  		String parent = (name.lastIndexOf("/") < 1 ? "/db" : name.substring(0,
  				name.lastIndexOf("/")));
! 		return parent;
  	}
  
***************
*** 663,667 ****
  						doc);
  			}
! 			broker.removeDocument(getName() + '/' + docname);
  			documents.remove(docname);
  			broker.saveCollection(this);
--- 712,716 ----
  						doc);
  			}
! 			broker.removeDocument(doc);
  			documents.remove(docname);
  			broker.saveCollection(this);
***************
*** 715,718 ****
--- 764,893 ----
  	}
  
+ 	public Indexer validate(DBBroker broker, String name, InputSource source) 
+ 	throws EXistException, PermissionDeniedException, TriggerException,
+ 	SAXException, LockException {
+ 		if (broker.isReadOnly())
+ 			throw new PermissionDeniedException("Database is read-only");
+ 		try {
+ 			lock.acquire(Lock.WRITE_LOCK);
+ 			XMLReader currentReader = getReader(broker);
+ 			DocumentImpl oldDoc = (DocumentImpl)documents.get(name);
+ 			DocumentImpl document = new DocumentImpl(broker, name, this);
+ 			// first pass: parse the document to determine tree structure
+ 			return determineTreeStructure(broker, name, document, oldDoc, currentReader, source);
+ 		} finally {
+ 			lock.release();
+ 		}
+ 	}
+ 	
+ 	public void store(DBBroker broker, Indexer indexer, InputSource source, boolean privileged)
+ 	throws EXistException, PermissionDeniedException, TriggerException,
+ 	SAXException, LockException {
+ 		// reset the input source
+ 		try {
+ 			final InputStream is = source.getByteStream();
+ 			if (is != null)
+ 				is.reset();
+ 			else {
+ 				final Reader cs = source.getCharacterStream();
+ 				if (cs != null)
+ 					cs.reset();
+ 			}
+ 		} catch (IOException e) {
+ 			LOG.debug("could not reset input source", e);
+ 		}
+ 		// second pass: store the document
+ 		DocumentImpl document = indexer.getDocument();
+ 		LOG.debug("storing document " + document.getDocId() + "; " + document.getFileName() + " ...");
+ 		try {
+ 			try {
+ 				indexer.getReader().parse(source);
+ 			} catch (IOException e) {
+ 				throw new EXistException(e);
+ 			}
+ 			if(!hasDocument(document.getFileName())) {
+ 				addDocument(broker, document);
+ 				broker.addDocument(this, document);
+ 			} else {
+ 				broker.updateDocument(document);
+ 			}
+ 			broker.closeDocument();
+ 			broker.flush();
+ 			broker.checkTree(document);
+ 			LOG.debug("document stored.");
+ 			// if we are running in privileged mode (e.g. backup/restore)
+ 			// notify the SecurityManager about changes
+ 			if (getName().equals(SecurityManager.SYSTEM) && document.getFileName().equals(SecurityManager.ACL_FILE)
+ 			        && privileged == false) {
+ 			    // inform the security manager that system data has changed
+ 			    LOG.debug("users.xml changed");
+ 			    broker.getBrokerPool().reloadSecurityManager(broker);
+ 			}
+ 		} finally {
+ 			document.getUpdateLock().release(Lock.WRITE_LOCK);
+ 		}
+ 		broker.deleteObservers();
+ 		return;
+ 	}
+ 	
+ 	public Indexer validate(DBBroker broker, String name, String data)
+ 	throws EXistException, PermissionDeniedException, TriggerException,
+ 	SAXException, LockException {
+ 		if (broker.isReadOnly())
+ 			throw new PermissionDeniedException("Database is read-only");
+ 		InputSource source;
+ 		try {
+ 			lock.acquire(Lock.WRITE_LOCK);
+ 			source = new InputSource(new StringReader(data));
+ 			XMLReader currentReader = getReader(broker);
+ 			DocumentImpl oldDoc = (DocumentImpl)documents.get(name);
+ 			DocumentImpl document = new DocumentImpl(broker, name, this);
+ 			// first pass: parse the document to determine tree structure
+ 			return determineTreeStructure(broker, name, document, oldDoc, currentReader, source);
+ 		} finally {
+ 			lock.release();
+ 		}
+ 	}
+ 	
+ 	public void store(DBBroker broker, Indexer indexer, String data, boolean privileged)
+ 	throws EXistException, PermissionDeniedException, TriggerException,
+ 	SAXException, LockException {
+ 		InputSource source = new InputSource(new StringReader(data));
+ 
+ 		// second pass: store the document
+ 		DocumentImpl document = indexer.getDocument();
+ 		LOG.debug("storing document " + document.getDocId() + " ...");
+ 		try {
+ 			try {
+ 				indexer.getReader().parse(source);
+ 			} catch (IOException e) {
+ 				throw new EXistException(e);
+ 			}
+ 
+ 			if(!hasDocument(document.getFileName())) {
+ 				addDocument(broker, document);
+ 				broker.addDocument(this, document);
+ 			} else {
+ 				broker.updateDocument(document);
+ 			}
+ 			broker.closeDocument();
+ 			broker.flush();
+ //			broker.checkTree(document);
+ 			LOG.debug("document stored.");
+ 			// if we are running in privileged mode (e.g. backup/restore)
+ 			// notify the SecurityManager about changes
+ 			if (getName().equals(SecurityManager.SYSTEM) && document.getFileName().equals(SecurityManager.ACL_FILE)
+ 			        && privileged == false) {
+ 			    // inform the security manager that system data has changed
+ 			    LOG.debug("users.xml changed");
+ 			    broker.getBrokerPool().reloadSecurityManager(broker);
+ 			}
+ 		} finally {
+ 			document.getUpdateLock().release(Lock.WRITE_LOCK);
+ 		}
+ 		broker.deleteObservers();
+ 		return;
+ 	}
+ 	
  	public DocumentImpl addDocument(DBBroker broker, String name, String data)
  			throws EXistException, PermissionDeniedException, TriggerException,
***************
*** 734,743 ****
  			lock.acquire(Lock.WRITE_LOCK);
  			source = new InputSource(new StringReader(data));
! 			document = new DocumentImpl(broker, name,	this);
  			reader = getReader(broker);
  			oldDoc = (DocumentImpl) documents.get(name);
  			
  			// first pass: parse the document to determine tree structure
! 			document = determineTreeStructure(broker, name, document, oldDoc, reader, source);
  		} finally {
  			lock.release();
--- 909,919 ----
  			lock.acquire(Lock.WRITE_LOCK);
  			source = new InputSource(new StringReader(data));
! 			document = new DocumentImpl(broker, name, this);
  			reader = getReader(broker);
  			oldDoc = (DocumentImpl) documents.get(name);
  			
  			// first pass: parse the document to determine tree structure
! 			Indexer indexer = determineTreeStructure(broker, name, document, oldDoc, reader, source);
! 			document = indexer.getDocument();
  		} finally {
  			lock.release();
***************
*** 795,799 ****
  	 * @throws TriggerException
  	 */
! 	private DocumentImpl determineTreeStructure(DBBroker broker, String name, DocumentImpl document, DocumentImpl oldDoc, XMLReader reader, InputSource source) throws LockException, EXistException, SAXException, PermissionDeniedException, TriggerException {
  		try {
  			checkPermissions(broker, name, oldDoc);
--- 971,975 ----
  	 * @throws TriggerException
  	 */
! 	private Indexer determineTreeStructure(DBBroker broker, String name, DocumentImpl document, DocumentImpl oldDoc, XMLReader reader, InputSource source) throws LockException, EXistException, SAXException, PermissionDeniedException, TriggerException {
  		try {
  			checkPermissions(broker, name, oldDoc);
***************
*** 825,829 ****
  				else {
  //					broker.checkTree(oldDoc);
! 					broker.removeDocument(getName() + '/' + oldDoc.getFileName(), false);
  				}
  				// we continue to use the old document object and just replace its contents
--- 1001,1005 ----
  				else {
  //					broker.checkTree(oldDoc);
! 					broker.removeDocument(oldDoc, false);
  				}
  				// we continue to use the old document object and just replace its contents
***************
*** 838,841 ****
--- 1014,1018 ----
  			if (trigger != null)
  				trigger.setValidating(false);
+ 			return indexer;
  		} catch(EXistException e) {
  		    if(oldDoc != null) oldDoc.getUpdateLock().release(Lock.WRITE_LOCK);
***************
*** 851,855 ****
  		    throw e;
  		}
- 		return document;
  	}
  
--- 1028,1031 ----
***************
*** 894,897 ****
--- 1070,1074 ----
  		}
  		reader.setErrorHandler(indexer);
+ 		indexer.setReader(reader);
  		//return reader;
  	}
***************
*** 933,937 ****
  	}
  
! 	/** Check Permissions about user and document, and throw exceptions if necessary.
  	 * @param broker
  	 * @param name
--- 1110,1116 ----
  	}
  
! 	/** 
! 	 * Check Permissions about user and document, and throw exceptions if necessary.
! 	 * 
  	 * @param broker
  	 * @param name
***************
*** 943,947 ****
  		if (oldDoc != null) {
  
! //			LOG.debug("Found old doc " + oldDoc.getDocId() + "; identity = " + oldDoc.hashCode());
  			// check if the document is locked by another user
  			User lockUser = oldDoc.getUserLock();
--- 1122,1127 ----
  		if (oldDoc != null) {
  
! 			LOG.debug("Found old doc " + oldDoc.getDocId() + "; identity = " + oldDoc.hashCode() +
! 					"; collection = " + hashCode());
  			// check if the document is locked by another user
  			User lockUser = oldDoc.getUserLock();
***************
*** 990,994 ****
  	
  			// first pass: parse the document to determine tree structure
! 			document = determineTreeStructure(broker, name, document, oldDoc, reader, source);
  		} finally {
  			lock.release();
--- 1170,1175 ----
  	
  			// first pass: parse the document to determine tree structure
! 			Indexer indexer = determineTreeStructure(broker, name, document, oldDoc, reader, source);
! 			document = indexer.getDocument();
  		} finally {
  			lock.release();
***************
*** 1096,1100 ****
  					broker.removeBinaryResource((BinaryDocument) oldDoc);
  				else
! 					broker.removeDocument(getName() + '/' + oldDoc.getFileName(), false);
  				oldDoc.copyOf(document);
  				indexer.setDocumentObject(oldDoc);
--- 1277,1281 ----
  					broker.removeBinaryResource((BinaryDocument) oldDoc);
  				else
! 					broker.removeDocument(oldDoc, false);
  				oldDoc.copyOf(document);
  				indexer.setDocumentObject(oldDoc);
***************
*** 1189,1192 ****
--- 1370,1374 ----
  		blob = new BinaryDocument(broker, name, this);
  		try {
+ 			lock.acquire(Lock.WRITE_LOCK);
  			checkPermissions(broker, name, oldDoc);
  
***************
*** 1198,1202 ****
  				broker.removeBinaryResource((BinaryDocument) oldDoc);
  			else
! 				broker.removeDocument(getName() + '/' + oldDoc.getFileName());
  			}
  
--- 1380,1384 ----
  				broker.removeBinaryResource((BinaryDocument) oldDoc);
  			else
! 				broker.removeDocument(oldDoc);
  			}
  

[Exist-commits] eXist-1.0/src/org/exist/xquery LocationStep.java,1.14,1.15

[Wolfgang M. M. <wol...@us...>]

Update of /cvsroot/exist/eXist-1.0/src/org/exist/xquery
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8244/src/org/exist/xquery

Modified Files:
	LocationStep.java 
Log Message:
Revised collection locking to fix various concurrency errors. It is now the responsibility of the 
caller to lock/unlock a collection. A collection can be retrieved and locked via the new 
openCollection() method provided by DBBroker. After reading/modifying the collection, it should be 
unlocked by calling Collection.release.

The local XML:DB implementation and the XMLRPC interface have been changed to reflect the new
collection locking rules.

Index: LocationStep.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/xquery/LocationStep.java,v
retrieving revision 1.14
retrieving revision 1.15
diff -C2 -d -r1.14 -r1.15
*** LocationStep.java	28 Jul 2004 18:54:56 -0000	1.14
--- LocationStep.java	3 Aug 2004 15:25:59 -0000	1.15
***************
*** 283,287 ****
  //					ElementValue.ELEMENT, docs, test.getName(), selector
  //			);
- 			LOG.debug("Found " + result.getLength() + " for " + pprint());
  			return result;
  		}
--- 283,286 ----

[Exist-commits] eXist-1.0/src/org/exist/client ClientFrame.java,1.23,1.24

[Giulio V. <gva...@us...>]

Update of /cvsroot/exist/eXist-1.0/src/org/exist/client
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv18727/src/org/exist/client

Modified Files:
	ClientFrame.java 
Log Message:
Fix accellerator key

Index: ClientFrame.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/client/ClientFrame.java,v
retrieving revision 1.23
retrieving revision 1.24
diff -C2 -d -r1.23 -r1.24
*** ClientFrame.java	29 Jul 2004 14:57:12 -0000	1.23
--- ClientFrame.java	30 Jul 2004 07:38:14 -0000	1.24
***************
*** 349,353 ****
  		fileMenu.add(item);
  
! 		item = new JMenuItem("Create blank document", KeyEvent.VK_N);
  		item.setAccelerator(KeyStroke.getKeyStroke("control B"));
  		item.addActionListener(new ActionListener() {
--- 349,353 ----
  		fileMenu.add(item);
  
! 		item = new JMenuItem("Create blank document", KeyEvent.VK_B);
  		item.setAccelerator(KeyStroke.getKeyStroke("control B"));
  		item.addActionListener(new ActionListener() {

[Exist-commits] eXist-1.0 .classpath,1.21,1.22

[Wolfgang M. M. <wol...@us...>]

Update of /cvsroot/exist/eXist-1.0
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv32355

Modified Files:
	.classpath 
Log Message:
Updated eclipse configuration with the correct jars.

Index: .classpath
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/.classpath,v
retrieving revision 1.21
retrieving revision 1.22
diff -C2 -d -r1.21 -r1.22
*** .classpath	28 May 2004 10:54:11 -0000	1.21
--- .classpath	29 Jul 2004 15:56:24 -0000	1.22
***************
*** 6,21 ****
  	<classpathentry kind="lib" path="lib/core/antlr.jar"/>
  	<classpathentry kind="lib" path="lib/core/excalibur-cli-1.0.jar"/>
- 	<classpathentry kind="lib" path="lib/core/jakarta-oro-2.0.6.jar"/>
  	<classpathentry kind="lib" path="lib/core/libreadline-java.jar"/>
- 	<classpathentry kind="lib" path="lib/core/log4j.jar"/>
  	<classpathentry kind="lib" path="lib/core/resolver-20030708.jar"/>
  	<classpathentry kind="lib" path="lib/core/xmldb.jar"/>
  	<classpathentry kind="lib" path="lib/core/xmlrpc-1.2.jar"/>
- 	<classpathentry kind="lib" path="lib/optional/avalon-framework-4.1.4.jar"/>
  	<classpathentry kind="lib" path="lib/optional/axis-1.1.jar"/>
  	<classpathentry kind="lib" path="lib/optional/axis-ant.jar"/>
  	<classpathentry kind="lib" path="lib/optional/axis-jaxrpc-1.1.jar"/>
  	<classpathentry kind="lib" path="lib/optional/axis-saaj-1.1.jar"/>
- 	<classpathentry kind="lib" path="lib/optional/cocoon-deprecated.jar"/>
  	<classpathentry kind="lib" path="lib/optional/commons-discovery-0.2.jar"/>
  	<classpathentry kind="lib" path="lib/optional/commons-jxpath-20030909.jar"/>
--- 6,17 ----
***************
*** 23,28 ****
  	<classpathentry kind="lib" path="lib/optional/commons-logging-1.0.3.jar"/>
  	<classpathentry kind="lib" path="lib/optional/excalibur-datasource-1.1.1.jar"/>
- 	<classpathentry kind="lib" path="lib/optional/excalibur-event-api-1.0.4-dev.jar"/>
- 	<classpathentry kind="lib" path="lib/optional/excalibur-event-impl-1.0.4-dev.jar"/>
  	<classpathentry kind="lib" path="lib/optional/excalibur-i18n-1.1.jar"/>
  	<classpathentry kind="lib" path="lib/optional/excalibur-instrument-1.0.jar"/>
--- 19,22 ----
***************
*** 30,44 ****
  	<classpathentry kind="lib" path="lib/optional/excalibur-instrument-manager-interfaces-1.0.jar"/>
  	<classpathentry kind="lib" path="lib/optional/excalibur-io-1.1.jar"/>
- 	<classpathentry kind="lib" path="lib/optional/excalibur-logger-1.0.1.jar"/>
  	<classpathentry kind="lib" path="lib/optional/excalibur-monitor-1.0.2.jar"/>
  	<classpathentry kind="lib" path="lib/optional/excalibur-naming-1.0.jar"/>
  	<classpathentry kind="lib" path="lib/optional/excalibur-pool-1.2.jar"/>
- 	<classpathentry kind="lib" path="lib/optional/excalibur-sourceresolve-1.0.2-dev.jar"/>
  	<classpathentry kind="lib" path="lib/optional/excalibur-util-1.0.jar"/>
- 	<classpathentry kind="lib" path="lib/optional/excalibur-xmlutil-1.0-dev.jar"/>
  	<classpathentry kind="lib" path="lib/optional/jakarta-regexp-1.3.jar"/>
  	<classpathentry kind="lib" path="lib/optional/jisp-2.5.1.jar"/>
- 	<classpathentry kind="lib" path="lib/optional/logkit-1.2.jar"/>
- 	<classpathentry kind="lib" path="lib/optional/util.concurrent-1.3.2.jar"/>
  	<classpathentry kind="lib" path="lib/optional/wsdl4j-1.4.jar"/>
  	<classpathentry kind="lib" path="lib/endorsed/xml-apis.jar"/>
--- 24,33 ----
***************
*** 46,66 ****
  	<classpathentry sourcepath="/jEdit-syntax" kind="lib" path="lib/core/jEdit-syntax.jar"/>
  	<classpathentry kind="lib" path="Jetty-4.1.4/lib/org.mortbay.jetty.jar"/>
- 	<classpathentry kind="lib" path="lib/endorsed/xalan-2.5.2.jar"/>
  	<classpathentry kind="lib" path="tools/lib/ant.jar"/>
  	<classpathentry kind="lib" path="tools/lib/junit.jar"/>
  	<classpathentry kind="lib" path="lib/core/javax.servlet.jar"/>
- 	<classpathentry kind="lib" path="lib/endorsed/xerces-2.6.1.jar"/>
- 	<classpathentry kind="lib" path="lib/optional/cocoon-2.1.4.jar"/>
  	<classpathentry kind="lib" path="lib/optional/cocoon-databases-block.jar"/>
  	<classpathentry kind="lib" path="lib/optional/cocoon-xmldb-block.jar"/>
  	<classpathentry kind="lib" path="lib/optional/commons-collections-3.0.jar"/>
- 	<classpathentry kind="lib" path="lib/optional/commons-httpclient-2.0-rc3.jar"/>
  	<classpathentry kind="lib" path="lib/optional/commons-jexl-1.0-beta-1-20040113.jar"/>
- 	<classpathentry kind="lib" path="lib/optional/excalibur-component-20040122.jar"/>
- 	<classpathentry kind="lib" path="lib/optional/excalibur-store-1.0-dev-20040206.jar"/>
- 	<classpathentry kind="lib" path="lib/optional/jdtcore-2.1.0.jar"/>
  	<classpathentry kind="lib" path="lib/optional/xml-commons-resolver-1.1.jar"/>
  	<classpathentry kind="lib" path="/home/wolf/Java/jakarta-tomcat-5.0.19/common/lib/jmx.jar"/>
  	<classpathentry kind="lib" path="/home/wolf/Java/jakarta-tomcat-5.0.19/common/lib/catalina.jar"/>
  	<classpathentry kind="output" path="build/classes"/>
  </classpath>
--- 35,72 ----
  	<classpathentry sourcepath="/jEdit-syntax" kind="lib" path="lib/core/jEdit-syntax.jar"/>
  	<classpathentry kind="lib" path="Jetty-4.1.4/lib/org.mortbay.jetty.jar"/>
  	<classpathentry kind="lib" path="tools/lib/ant.jar"/>
  	<classpathentry kind="lib" path="tools/lib/junit.jar"/>
  	<classpathentry kind="lib" path="lib/core/javax.servlet.jar"/>
  	<classpathentry kind="lib" path="lib/optional/cocoon-databases-block.jar"/>
  	<classpathentry kind="lib" path="lib/optional/cocoon-xmldb-block.jar"/>
  	<classpathentry kind="lib" path="lib/optional/commons-collections-3.0.jar"/>
  	<classpathentry kind="lib" path="lib/optional/commons-jexl-1.0-beta-1-20040113.jar"/>
  	<classpathentry kind="lib" path="lib/optional/xml-commons-resolver-1.1.jar"/>
  	<classpathentry kind="lib" path="/home/wolf/Java/jakarta-tomcat-5.0.19/common/lib/jmx.jar"/>
  	<classpathentry kind="lib" path="/home/wolf/Java/jakarta-tomcat-5.0.19/common/lib/catalina.jar"/>
+ 	<classpathentry kind="lib" path="lib/core/jakarta-oro-2.0.8.jar"/>
+ 	<classpathentry kind="lib" path="lib/core/log4j-1.2.8.jar"/>
+ 	<classpathentry kind="lib" path="lib/endorsed/jakarta-bcel-20040329.jar"/>
+ 	<classpathentry kind="lib" path="lib/endorsed/jakarta-regexp-1.3.jar"/>
+ 	<classpathentry kind="lib" path="lib/endorsed/xalan-2.6.0.jar"/>
+ 	<classpathentry kind="lib" path="lib/endorsed/xercesImpl-2.6.2.jar"/>
+ 	<classpathentry kind="lib" path="lib/optional/avalon-framework-api-4.1.5.jar"/>
+ 	<classpathentry kind="lib" path="lib/optional/avalon-framework-impl-4.1.5.jar"/>
+ 	<classpathentry kind="lib" path="lib/optional/cocoon-2.1.5.1.jar"/>
+ 	<classpathentry kind="lib" path="lib/optional/cocoon-2.1.5.1-deprecated.jar"/>
+ 	<classpathentry kind="lib" path="lib/optional/cocoon-xsp-block.jar"/>
+ 	<classpathentry kind="lib" path="lib/optional/commons-httpclient-2.0-final.jar"/>
+ 	<classpathentry kind="lib" path="lib/optional/excalibur-component-1.2.jar"/>
+ 	<classpathentry kind="lib" path="lib/optional/excalibur-event-api-1.1.jar"/>
+ 	<classpathentry kind="lib" path="lib/optional/excalibur-event-impl-1.1.jar"/>
+ 	<classpathentry kind="lib" path="lib/optional/excalibur-logger-1.1.jar"/>
+ 	<classpathentry kind="lib" path="lib/optional/excalibur-sourceresolve-1.1.jar"/>
+ 	<classpathentry kind="lib" path="lib/optional/excalibur-store-1.0.jar"/>
+ 	<classpathentry kind="lib" path="lib/optional/excalibur-xmlutil-1.0.jar"/>
+ 	<classpathentry kind="lib" path="lib/optional/jcs-1.0-dev-20040516.jar"/>
+ 	<classpathentry kind="lib" path="lib/optional/jdtcore-2.1.3.jar"/>
+ 	<classpathentry kind="lib" path="lib/optional/logkit-1.2.2.jar"/>
+ 	<classpathentry kind="lib" path="lib/optional/pizza-1.1.jar"/>
+ 	<classpathentry kind="lib" path="lib/optional/util.concurrent-1.3.3.jar"/>
  	<classpathentry kind="output" path="build/classes"/>
  </classpath>

[Exist-commits] eXist-1.0/src/org/exist/client ClientFrame.java,1.22,1.23

[Giulio V. <gva...@us...>]

Update of /cvsroot/exist/eXist-1.0/src/org/exist/client
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv21517/src/org/exist/client

Modified Files:
	ClientFrame.java 
Log Message:
Create blank document

Index: ClientFrame.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/client/ClientFrame.java,v
retrieving revision 1.22
retrieving revision 1.23
diff -C2 -d -r1.22 -r1.23
*** ClientFrame.java	26 Jul 2004 13:50:25 -0000	1.22
--- ClientFrame.java	29 Jul 2004 14:57:12 -0000	1.23
***************
*** 108,111 ****
--- 108,113 ----
  import org.xmldb.api.base.XMLDBException;
  import org.xmldb.api.modules.CollectionManagementService;
+ import org.xmldb.api.modules.XMLResource;
+ import org.xmldb.api.base.ErrorCodes;
  
  public class ClientFrame extends JFrame
***************
*** 347,350 ****
--- 349,379 ----
  		fileMenu.add(item);
  
+ 		item = new JMenuItem("Create blank document", KeyEvent.VK_N);
+ 		item.setAccelerator(KeyStroke.getKeyStroke("control B"));
+ 		item.addActionListener(new ActionListener() {
+ 			public void actionPerformed(ActionEvent e) {
+                 //FIXME: Prevent owerwrite. Security?
+ 				Collection collection = client.current;
+ 				XMLResource result = null;
+ 				String nameres = JOptionPane.showInputDialog(null,
+ 				"Name of the XML resource (extension incluse)");
+ 				if (nameres != null) {
+ 					try {
+ 						result = (XMLResource) collection.createResource(
+ 								nameres, XMLResource.RESOURCE_TYPE);
+ 						result.setContent("<template></template>");
+ 						collection.storeResource(result);
+ 						collection.close();
+ 						client.reloadCollection();
+ 					} catch (XMLDBException ev) {
+ 						showErrorMessage(ev.getMessage(), ev);
+ 					}
+ 
+ 				}
+ 			}
+ 		});
+ 		fileMenu.add(item);
+ 		fileMenu.addSeparator();
+ 
  		item = new JMenuItem("Remove");
  		item.setAccelerator(KeyStroke.getKeyStroke("control D"));
***************
*** 373,376 ****
--- 402,406 ----
  		});
  		fileMenu.add(item);
+ 		fileMenu.addSeparator();
  		
  		item = new JMenuItem("Reindex collection", KeyEvent.VK_R);

[Exist-commits] eXist-1.0/src/org/exist/collections Collection.java,1.42,1.43

[Wolfgang M. M. <wol...@us...>]

Update of /cvsroot/exist/eXist-1.0/src/org/exist/collections
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv13335/src/org/exist/collections

Modified Files:
	Collection.java 
Log Message:
* Fixed locking error: a collection object could be unloaded from the cache by one thread, while another thread had still been writing a document 
being a member of the collection. A moment later, a third thread reloaded the collection and started to write to the same document. However, as 
the collection had been recreated, there were two instances of the same document in memory. The third thread did thus not respect the lock held 
by the first thread and started to write without waiting. As a consequence, various page errors were thrown.

* When updating a document, the old document metadata had not been correctly removed. Usually, this had no consequences as the collection class 
only used the last record it found. In some cases however, this may have led to errors.

Index: Collection.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/collections/Collection.java,v
retrieving revision 1.42
retrieving revision 1.43
diff -C2 -d -r1.42 -r1.43
*** Collection.java	28 Jul 2004 18:54:54 -0000	1.42
--- Collection.java	28 Jul 2004 19:40:42 -0000	1.43
***************
*** 943,947 ****
  		if (oldDoc != null) {
  
! 			LOG.debug("Found old doc " + oldDoc.getDocId() + "; identity = " + oldDoc.hashCode());
  			// check if the document is locked by another user
  			User lockUser = oldDoc.getUserLock();
--- 943,947 ----
  		if (oldDoc != null) {
  
! //			LOG.debug("Found old doc " + oldDoc.getDocId() + "; identity = " + oldDoc.hashCode());
  			// check if the document is locked by another user
  			User lockUser = oldDoc.getUserLock();

[Exist-commits] eXist-1.0/src/org/exist/storage BrokerPool.java,1.24,1.25 DBBroker.java,1.38,1.39 NativeBroker.java,1.90,1.91

[Wolfgang M. M. <wol...@us...>]

Update of /cvsroot/exist/eXist-1.0/src/org/exist/storage
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4115/src/org/exist/storage

Modified Files:
	BrokerPool.java DBBroker.java NativeBroker.java 
Log Message:
* Fixed locking error: a collection object could be unloaded from the cache by one thread, while another thread had still been writing a document 
being a member of the collection. A moment later, a third thread reloaded the collection and started to write to the same document. However, as 
the collection had been recreated, there were two instances of the same document in memory. The third thread did thus not respect the lock held 
by the first thread and started to write without waiting. As a consequence, various page errors were thrown.

* When updating a document, the old document metadata had not been correctly removed. Usually, this had no consequences as the collection class 
only used the last record it found. In some cases however, this may have led to errors.

Index: NativeBroker.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/storage/NativeBroker.java,v
retrieving revision 1.90
retrieving revision 1.91
diff -C2 -d -r1.90 -r1.91
*** NativeBroker.java	23 Jul 2004 14:19:52 -0000	1.90
--- NativeBroker.java	28 Jul 2004 18:54:55 -0000	1.91
***************
*** 1340,1344 ****
  			LOG.debug("Checking document " + doc.getFileName());
  			checkTree(doc);
! 			elementIndex.consistencyCheck(doc);
  		}
  	}
--- 1340,1344 ----
  			LOG.debug("Checking document " + doc.getFileName());
  			checkTree(doc);
! //			elementIndex.consistencyCheck(doc);
  		}
  	}
***************
*** 1346,1381 ****
  	public void checkTree(final DocumentImpl doc) {
  		LOG.debug("Checking DOM tree for document " + doc.getFileName());
! 		new DOMTransaction(this, domDb, Lock.READ_LOCK) {
! 			public Object start() throws ReadOnlyException {
! 				LOG.debug("Pages used: " + domDb.debugPages(doc));
! 				return null;
  			}
! 		}.run();
! 		
! 		NodeList nodes = doc.getChildNodes();
! 		NodeImpl n;
! 		for (int i = 0; i < nodes.getLength(); i++) {
! 		    n = (NodeImpl) nodes.item(i);
! 		    Iterator iterator =
! 		        getNodeIterator(
! 		                new NodeProxy(doc, n.getGID(), n.getInternalAddress()));
! 		    iterator.next();
! 		    checkTree(iterator, n);
! 		}
! 		NodeRef ref = new NodeRef(doc.getDocId());
! 		final IndexQuery idx = new IndexQuery(IndexQuery.TRUNC_RIGHT, ref);
! 		new DOMTransaction(this, domDb) {
! 			public Object start() {
! 				try {
! 					domDb.findKeys(idx);
! 				} catch (BTreeException e) {
! 		            LOG.warn("start() - " + "error while removing doc", e);
! 				} catch (IOException e) {
! 		            LOG.warn("start() - " + "error while removing doc", e);
  				}
- 				return null;
  			}
  		}
- 		.run();
  	}
  	
--- 1346,1383 ----
  	public void checkTree(final DocumentImpl doc) {
  		LOG.debug("Checking DOM tree for document " + doc.getFileName());
! 		if(xupdateConsistencyChecks) {
! 			new DOMTransaction(this, domDb, Lock.READ_LOCK) {
! 				public Object start() throws ReadOnlyException {
! 					LOG.debug("Pages used: " + domDb.debugPages(doc));
! 					return null;
! 				}
! 			}.run();
! 			
! 			NodeList nodes = doc.getChildNodes();
! 			NodeImpl n;
! 			for (int i = 0; i < nodes.getLength(); i++) {
! 			    n = (NodeImpl) nodes.item(i);
! 			    Iterator iterator =
! 			        getNodeIterator(
! 			                new NodeProxy(doc, n.getGID(), n.getInternalAddress()));
! 			    iterator.next();
! 			    checkTree(iterator, n);
  			}
! 			NodeRef ref = new NodeRef(doc.getDocId());
! 			final IndexQuery idx = new IndexQuery(IndexQuery.TRUNC_RIGHT, ref);
! 			new DOMTransaction(this, domDb) {
! 				public Object start() {
! 					try {
! 						domDb.findKeys(idx);
! 					} catch (BTreeException e) {
! 			            LOG.warn("start() - " + "error while removing doc", e);
! 					} catch (IOException e) {
! 			            LOG.warn("start() - " + "error while removing doc", e);
! 					}
! 					return null;
  				}
  			}
+ 			.run();
  		}
  	}
  	
***************
*** 2383,2386 ****
--- 2385,2389 ----
  				}
  				doc.setAddress(domDb.add(data));
+ //				LOG.debug("Document metadata stored to " + StorageAddress.toString(doc.getAddress()));
  				return null;
  			}
***************
*** 2389,2392 ****
--- 2392,2406 ----
  	}
  
+ 	public void updateDocument(DocumentImpl doc) throws LockException, PermissionDeniedException {
+ 		Lock lock = collectionsDb.getLock();
+ 		try {
+ 			lock.acquire(Lock.WRITE_LOCK);
+ 			storeDocument(doc);
+ 			saveCollection(doc.getCollection());
+ 		} finally {
+ 			lock.release();
+ 		}
+ 	}
+ 	
  	public void storeBinaryResource(final BinaryDocument blob, final byte[] data) {
  		new DOMTransaction(this, domDb, Lock.WRITE_LOCK) {
***************
*** 2402,2406 ****
  		.run();
  	}
! 
  	public byte[] getBinaryResourceData(final BinaryDocument blob) {
  		byte[] data = (byte[]) new DOMTransaction(this, domDb, Lock.WRITE_LOCK) {
--- 2416,2420 ----
  		.run();
  	}
! 	
  	public byte[] getBinaryResourceData(final BinaryDocument blob) {
  		byte[] data = (byte[]) new DOMTransaction(this, domDb, Lock.WRITE_LOCK) {

Index: DBBroker.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/storage/DBBroker.java,v
retrieving revision 1.38
retrieving revision 1.39
diff -C2 -d -r1.38 -r1.39
*** DBBroker.java	19 Jul 2004 13:06:24 -0000	1.38
--- DBBroker.java	28 Jul 2004 18:54:54 -0000	1.39
***************
*** 458,462 ****
  	/**
  	 *  Store a document into the database. This method will save the document
! 	 * metadata and add the document to the collection.
  	 *
  	 *@param  doc
--- 458,462 ----
  	/**
  	 *  Store a document into the database. This method will save the document
! 	 * metadata.
  	 *
  	 *@param  doc
***************
*** 464,467 ****
--- 464,470 ----
  	public abstract void storeDocument(DocumentImpl doc);
  
+ 	public abstract void updateDocument(DocumentImpl doc) 
+ 	throws LockException, PermissionDeniedException;
+ 	
  	public abstract void storeBinaryResource(BinaryDocument blob, byte[] data);
  	

Index: BrokerPool.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/storage/BrokerPool.java,v
retrieving revision 1.24
retrieving revision 1.25
diff -C2 -d -r1.24 -r1.25
*** BrokerPool.java	5 Jul 2004 20:02:46 -0000	1.24
--- BrokerPool.java	28 Jul 2004 18:54:54 -0000	1.25
***************
*** 250,253 ****
--- 250,260 ----
  		}
  		synchronized(this) {
+ 			// force the thread to wait until a pending sync has finished
+ 			while(syncRequired && threads.size() > 0) {
+ 				try {
+ 					this.wait();
+ 				} catch(InterruptedException e) {
+ 				}
+ 			}
  			if (pool.isEmpty()) {
  				if (brokers < max)
***************
*** 265,269 ****
  			threads.put(Thread.currentThread(), broker);
  			broker.incReferenceCount();
! 			this.notifyAll();
  			return broker;
  		}
--- 272,276 ----
  			threads.put(Thread.currentThread(), broker);
  			broker.incReferenceCount();
! //			this.notifyAll();
  			return broker;
  		}
***************
*** 341,345 ****
  		    threads.remove(Thread.currentThread());
  			pool.push(broker);
! 			if (syncRequired && pool.size() == brokers) {
  				sync(broker);
  				syncRequired = false;
--- 348,352 ----
  		    threads.remove(Thread.currentThread());
  			pool.push(broker);
! 			if (syncRequired && threads.size() == 0) {
  				sync(broker);
  				syncRequired = false;

[Exist-commits] eXist-1.0/src/org/exist/xmldb LocalCollection.java,1.35,1.36 LocalXMLResource.java,1.36,1.37

[Wolfgang M. M. <wol...@us...>]

Update of /cvsroot/exist/eXist-1.0/src/org/exist/xmldb
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4115/src/org/exist/xmldb

Modified Files:
	LocalCollection.java LocalXMLResource.java 
Log Message:
* Fixed locking error: a collection object could be unloaded from the cache by one thread, while another thread had still been writing a document 
being a member of the collection. A moment later, a third thread reloaded the collection and started to write to the same document. However, as 
the collection had been recreated, there were two instances of the same document in memory. The third thread did thus not respect the lock held 
by the first thread and started to write without waiting. As a consequence, various page errors were thrown.

* When updating a document, the old document metadata had not been correctly removed. Usually, this had no consequences as the collection class 
only used the last record it found. In some cases however, this may have led to errors.

Index: LocalCollection.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/xmldb/LocalCollection.java,v
retrieving revision 1.35
retrieving revision 1.36
diff -C2 -d -r1.35 -r1.36
*** LocalCollection.java	13 Jul 2004 15:13:21 -0000	1.35
--- LocalCollection.java	28 Jul 2004 18:54:53 -0000	1.36
***************
*** 36,39 ****
--- 36,40 ----
  
  import org.apache.log4j.Category;
+ import org.apache.log4j.Logger;
  import org.exist.EXistException;
  import org.exist.collections.Collection;
***************
*** 68,72 ****
  public class LocalCollection extends Observable implements CollectionImpl {
  
! 	private static Category LOG = Category.getInstance(LocalCollection.class.getName());
  
  	protected final static Properties defaultProperties = new Properties();
--- 69,73 ----
  public class LocalCollection extends Observable implements CollectionImpl {
  
! 	private static Logger LOG = Logger.getLogger(LocalCollection.class.getName());
  
  	protected final static Properties defaultProperties = new Properties();

Index: LocalXMLResource.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/xmldb/LocalXMLResource.java,v
retrieving revision 1.36
retrieving revision 1.37
diff -C2 -d -r1.36 -r1.37
*** LocalXMLResource.java	2 Jul 2004 16:53:53 -0000	1.36
--- LocalXMLResource.java	28 Jul 2004 18:54:53 -0000	1.37
***************
*** 229,233 ****
  		// case 3: content is an internal node or a document
  		} else {
- 		    DocumentImpl document = null;
  			try {
  				broker = brokerPool.get(user);
--- 229,232 ----
***************
*** 240,256 ****
  					serializer.toSAX((NodeValue) root);
  				else {
! 					document = getDocument(broker, true);
! 					if (!document.getPermissions().validate(user,
! 							Permission.READ))
! 						throw new XMLDBException(ErrorCodes.PERMISSION_DENIED,
! 								"permission denied to read resource");
! 					String xml;
! 					if (id < 0)
! 						serializer.toSAX(document);
! 					else {
! 						if (proxy == null)
! 							proxy = new NodeProxy(document, id);
! 
! 						serializer.toSAX(proxy);
  					}
  				}
--- 239,260 ----
  					serializer.toSAX((NodeValue) root);
  				else {
! 					DocumentImpl document = getDocument(broker, true);
! 					try {
! 						if (!document.getPermissions().validate(user,
! 								Permission.READ))
! 							throw new XMLDBException(ErrorCodes.PERMISSION_DENIED,
! 							"permission denied to read resource");
! 						String xml;
! 						if (id < 0)
! 							serializer.toSAX(document);
! 						else {
! 							if (proxy == null)
! 								proxy = new NodeProxy(document, id);
! 							
! 							serializer.toSAX(proxy);
! 						}
! 					} finally {
! 					    parent.getCollection().releaseDocument(document);
! 						brokerPool.release(broker);
  					}
  				}
***************
*** 261,267 ****
  				throw new XMLDBException(ErrorCodes.VENDOR_ERROR, e
  						.getMessage(), e);
- 			} finally {
- 			    parent.getCollection().releaseDocument(document);
- 				brokerPool.release(broker);
  			}
  		}
--- 265,268 ----
***************
*** 407,411 ****
  	protected DocumentImpl getDocument(DBBroker broker, boolean lock) throws XMLDBException {
  	    DocumentImpl document = null;
! 	    if(lock)
              try {
                  document = parent.getCollection().getDocumentWithLock(broker, docId);
--- 408,412 ----
  	protected DocumentImpl getDocument(DBBroker broker, boolean lock) throws XMLDBException {
  	    DocumentImpl document = null;
! 	    if(lock) {
              try {
                  document = parent.getCollection().getDocumentWithLock(broker, docId);
***************
*** 414,421 ****
                          "Failed to acquire lock on document " + docId);
              }
!         else
  	        document = parent.getCollection().getDocument(broker, docId);
! 	    if (document == null)
  	        throw new XMLDBException(ErrorCodes.INVALID_RESOURCE);
  	    return document;
  	}
--- 415,424 ----
                          "Failed to acquire lock on document " + docId);
              }
! 	    } else {
  	        document = parent.getCollection().getDocument(broker, docId);
! 	    }
! 	    if (document == null) {
  	        throw new XMLDBException(ErrorCodes.INVALID_RESOURCE);
+ 	    }
  	    return document;
  	}

[Exist-commits] eXist-1.0/src/org/exist/util/serializer DOMStreamer.java,1.4,1.5

[Wolfgang M. M. <wol...@us...>]

Update of /cvsroot/exist/eXist-1.0/src/org/exist/util/serializer
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4115/src/org/exist/util/serializer

Modified Files:
	DOMStreamer.java 
Log Message:
* Fixed locking error: a collection object could be unloaded from the cache by one thread, while another thread had still been writing a document 
being a member of the collection. A moment later, a third thread reloaded the collection and started to write to the same document. However, as 
the collection had been recreated, there were two instances of the same document in memory. The third thread did thus not respect the lock held 
by the first thread and started to write without waiting. As a consequence, various page errors were thrown.

* When updating a document, the old document metadata had not been correctly removed. Usually, this had no consequences as the collection class 
only used the last record it found. In some cases however, this may have led to errors.

Index: DOMStreamer.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/util/serializer/DOMStreamer.java,v
retrieving revision 1.4
retrieving revision 1.5
diff -C2 -d -r1.4 -r1.5
*** DOMStreamer.java	10 May 2004 11:22:51 -0000	1.4
--- DOMStreamer.java	28 Jul 2004 18:54:53 -0000	1.5
***************
*** 219,222 ****
--- 219,223 ----
  				break;
  			default :
+ 				System.out.println("Found node: " + node.getNodeType());
  				break;
  		}

[Exist-commits] eXist-1.0/src/org/exist/storage/store DOMFile.java,1.42,1.43 NodeIterator.java,1.13,1.14

[Wolfgang M. M. <wol...@us...>]

Update of /cvsroot/exist/eXist-1.0/src/org/exist/storage/store
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4115/src/org/exist/storage/store

Modified Files:
	DOMFile.java NodeIterator.java 
Log Message:
* Fixed locking error: a collection object could be unloaded from the cache by one thread, while another thread had still been writing a document 
being a member of the collection. A moment later, a third thread reloaded the collection and started to write to the same document. However, as 
the collection had been recreated, there were two instances of the same document in memory. The third thread did thus not respect the lock held 
by the first thread and started to write without waiting. As a consequence, various page errors were thrown.

* When updating a document, the old document metadata had not been correctly removed. Usually, this had no consequences as the collection class 
only used the last record it found. In some cases however, this may have led to errors.

Index: DOMFile.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/storage/store/DOMFile.java,v
retrieving revision 1.42
retrieving revision 1.43
diff -C2 -d -r1.42 -r1.43
*** DOMFile.java	28 Jul 2004 17:40:05 -0000	1.42
--- DOMFile.java	28 Jul 2004 18:54:56 -0000	1.43
***************
*** 1192,1200 ****
      
      public void removeAll(long p) {
!     	StringBuffer debug = new StringBuffer();
!     	debug.append("Removed pages: ");
          long pnum = StorageAddress.pageFromPointer(p);
          while(-1 < pnum) {
!         	debug.append(' ').append(pnum);
              DOMPage page = getCurrentPage(pnum);
              pnum = page.getPageHeader().getNextDataPage();
--- 1192,1200 ----
      
      public void removeAll(long p) {
! //    	StringBuffer debug = new StringBuffer();
! //    	debug.append("Removed pages: ");
          long pnum = StorageAddress.pageFromPointer(p);
          while(-1 < pnum) {
! //        	debug.append(' ').append(pnum);
              DOMPage page = getCurrentPage(pnum);
              pnum = page.getPageHeader().getNextDataPage();
***************
*** 1213,1217 ****
              }
          }
!         LOG.debug(debug.toString());
      }
      
--- 1213,1217 ----
              }
          }
! //        LOG.debug(debug.toString());
      }
      
***************
*** 1408,1412 ****
  	        	getNodeValue(os, rec, false);
  	        	if (children - attributes > 1)
! 	        		os.write((byte) 0x20);
  	        }
  	        return;
--- 1408,1412 ----
  	        	getNodeValue(os, rec, false);
  	        	if (children - attributes > 1)
! 	        		os.write((byte) ' ');
  	        }
  	        return;

Index: NodeIterator.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/storage/store/NodeIterator.java,v
retrieving revision 1.13
retrieving revision 1.14
diff -C2 -d -r1.13 -r1.14
*** NodeIterator.java	12 Jul 2004 17:17:41 -0000	1.13
--- NodeIterator.java	28 Jul 2004 18:54:56 -0000	1.14
***************
*** 213,216 ****
--- 213,218 ----
  		} else if (-1 < startAddress) {
  			final DOMFile.RecordPos rec = db.findRecord(startAddress);
+ 			if(rec == null)
+ 				throw new IOException("Node not found at specified address.");
  			page = rec.page.getPageNum();
  			offset = rec.offset - 2;

[Exist-commits] eXist-1.0/src/org/exist/collections Collection.java,1.41,1.42

[Wolfgang M. M. <wol...@us...>]

Update of /cvsroot/exist/eXist-1.0/src/org/exist/collections
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4115/src/org/exist/collections

Modified Files:
	Collection.java 
Log Message:
* Fixed locking error: a collection object could be unloaded from the cache by one thread, while another thread had still been writing a document 
being a member of the collection. A moment later, a third thread reloaded the collection and started to write to the same document. However, as 
the collection had been recreated, there were two instances of the same document in memory. The third thread did thus not respect the lock held 
by the first thread and started to write without waiting. As a consequence, various page errors were thrown.

* When updating a document, the old document metadata had not been correctly removed. Usually, this had no consequences as the collection class 
only used the last record it found. In some cases however, this may have led to errors.

Index: Collection.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/collections/Collection.java,v
retrieving revision 1.41
retrieving revision 1.42
diff -C2 -d -r1.41 -r1.42
*** Collection.java	13 Jul 2004 15:09:44 -0000	1.41
--- Collection.java	28 Jul 2004 18:54:54 -0000	1.42
***************
*** 130,134 ****
  	private int refCount = 0;
  	private int timestamp = 0;
! 
  	// the collection store where this collections is stored.
  	private CollectionStore db;
--- 130,134 ----
  	private int refCount = 0;
  	private int timestamp = 0;
! 	
  	// the collection store where this collections is stored.
  	private CollectionStore db;
***************
*** 342,347 ****
  		}
  		return true;
  	}
! 
  	public int compareTo(Object obj) {
  		Collection other = (Collection) obj;
--- 342,361 ----
  		}
  		return true;
+ //		try {
+ //			lock.acquire(Lock.WRITE_LOCK);
+ //			for (Iterator i = documents.values().iterator(); i.hasNext(); ) {
+ //				DocumentImpl doc = (DocumentImpl) i.next();
+ //				if (doc.isLockedForWrite())
+ //					return false;
+ //			}
+ //			return true;
+ //		} catch (LockException e) {
+ //			LOG.warn("Failed to acquire lock on collection: " + getName(), e);
+ //		} finally {
+ //			lock.release();
+ //		}
+ //		return false;
  	}
! 	
  	public int compareTo(Object obj) {
  		Collection other = (Collection) obj;
***************
*** 716,728 ****
  		DocumentImpl document, oldDoc = null;
  		XMLReader reader;
! 
! 		InputSource source = new InputSource(new StringReader(data));
! 		oldDoc = getDocument(broker, name);
! 		document = new DocumentImpl(broker, name,	this);
! 		reader = getReader(broker);
  		
- 		// first pass: parse the document to determine tree structure
- 		document = determineTreeStructure(broker, name, document, oldDoc, reader, source);
- 
  		// reset the input source
  		source = new InputSource(new StringReader(data));
--- 730,747 ----
  		DocumentImpl document, oldDoc = null;
  		XMLReader reader;
! 		InputSource source;
! 		try {
! 			lock.acquire(Lock.WRITE_LOCK);
! 			source = new InputSource(new StringReader(data));
! 			document = new DocumentImpl(broker, name,	this);
! 			reader = getReader(broker);
! 			oldDoc = (DocumentImpl) documents.get(name);
! 			
! 			// first pass: parse the document to determine tree structure
! 			document = determineTreeStructure(broker, name, document, oldDoc, reader, source);
! 		} finally {
! 			lock.release();
! 		}
  		
  		// reset the input source
  		source = new InputSource(new StringReader(data));
***************
*** 736,746 ****
  				throw new EXistException(e);
  			}
! 	
! 			if(oldDoc == null)
! 			    addDocument(broker, document);
! //			broker.checkTree(document);
! 			broker.addDocument(this, document);
  			broker.closeDocument();
  			broker.flush();
  			LOG.debug("document stored.");
  			// if we are running in privileged mode (e.g. backup/restore)
--- 755,768 ----
  				throw new EXistException(e);
  			}
! 
! 			if(oldDoc == null) {
! 				addDocument(broker, document);
! 				broker.addDocument(this, document);
! 			} else {
! 				broker.updateDocument(document);
! 			}
  			broker.closeDocument();
  			broker.flush();
+ //			broker.checkTree(document);
  			LOG.debug("document stored.");
  			// if we are running in privileged mode (e.g. backup/restore)
***************
*** 795,806 ****
  			document.setMaxDepth(document.getMaxDepth() + 1);
  			document.calculateTreeLevelStartPoints();
- 			// new document is valid: remove old document
  			
  			if (oldDoc != null) {
  				LOG.debug("removing old document " + oldDoc.getFileName());
  				if (oldDoc.getResourceType() == DocumentImpl.BINARY_FILE)
  					broker.removeBinaryResource((BinaryDocument) oldDoc);
! 				else
  					broker.removeDocument(getName() + '/' + oldDoc.getFileName(), false);
  				// we continue to use the old document object and just replace its contents
  				oldDoc.copyOf(document);
--- 817,830 ----
  			document.setMaxDepth(document.getMaxDepth() + 1);
  			document.calculateTreeLevelStartPoints();
  			
+ 			// new document is valid: remove old document
  			if (oldDoc != null) {
  				LOG.debug("removing old document " + oldDoc.getFileName());
  				if (oldDoc.getResourceType() == DocumentImpl.BINARY_FILE)
  					broker.removeBinaryResource((BinaryDocument) oldDoc);
! 				else {
! //					broker.checkTree(oldDoc);
  					broker.removeDocument(getName() + '/' + oldDoc.getFileName(), false);
+ 				}
  				// we continue to use the old document object and just replace its contents
  				oldDoc.copyOf(document);
***************
*** 826,831 ****
  		    if(oldDoc != null) oldDoc.getUpdateLock().release(Lock.WRITE_LOCK);
  		    throw e;
- 		} finally {
- 			lock.release();
  		}
  		return document;
--- 850,853 ----
***************
*** 919,925 ****
  	 */
  	private void checkPermissions(DBBroker broker, String name, DocumentImpl oldDoc) throws LockException, PermissionDeniedException {
! 		lock.acquire(Lock.WRITE_LOCK);
! 		if (hasDocument(name) && (oldDoc ) != null) {
  
  			// check if the document is locked by another user
  			User lockUser = oldDoc.getUserLock();
--- 941,947 ----
  	 */
  	private void checkPermissions(DBBroker broker, String name, DocumentImpl oldDoc) throws LockException, PermissionDeniedException {
! 		if (oldDoc != null) {
  
+ 			LOG.debug("Found old doc " + oldDoc.getDocId() + "; identity = " + oldDoc.hashCode());
  			// check if the document is locked by another user
  			User lockUser = oldDoc.getUserLock();
***************
*** 929,934 ****
  			
  			// check if the document is currently being changed by someone else
! 			Lock oldLock = oldDoc.getUpdateLock();
! 			oldLock.acquire(Lock.WRITE_LOCK);
  			
  			// do we have permissions for update?
--- 951,955 ----
  			
  			// check if the document is currently being changed by someone else
! 			oldDoc.getUpdateLock().acquire(Lock.WRITE_LOCK);
  			
  			// do we have permissions for update?
***************
*** 962,972 ****
  		DocumentImpl document = null, oldDoc = null;
  		XMLReader reader;
! 		oldDoc = getDocument(broker, name);
! 		document = new DocumentImpl(broker, name,	this);
! 		reader = getReader(broker);
! 
! 		// first pass: parse the document to determine tree structure
! 		document = determineTreeStructure(broker, name, document, oldDoc, reader, source);
! 
  		// reset the input source
  		try {
--- 983,998 ----
  		DocumentImpl document = null, oldDoc = null;
  		XMLReader reader;
! 		try {
! 			lock.acquire(Lock.WRITE_LOCK);
! 			oldDoc = getDocument(broker, name);
! 			document = new DocumentImpl(broker, name,	this);
! 			reader = getReader(broker);
! 	
! 			// first pass: parse the document to determine tree structure
! 			document = determineTreeStructure(broker, name, document, oldDoc, reader, source);
! 		} finally {
! 			lock.release();
! 		}
! 		
  		// reset the input source
  		try {
***************
*** 991,998 ****
  				throw new EXistException(e);
  			}
! 	
! 			if(oldDoc == null)
! 			    addDocument(broker, document);
! 			broker.addDocument(this, document);
  			broker.closeDocument();
  			broker.flush();
--- 1017,1027 ----
  				throw new EXistException(e);
  			}
! 
! 			if(oldDoc == null) {
! 				addDocument(broker, document);
! 				broker.addDocument(this, document);
! 			} else {
! 				broker.updateDocument(document);
! 			}
  			broker.closeDocument();
  			broker.flush();
***************
*** 1027,1034 ****
  		DocumentImpl document, oldDoc = null;
  		DOMStreamer streamer;
- 		oldDoc = getDocument(broker, name);
- 		document = new DocumentImpl(broker, name,	this);
- 		
  		try {
  			checkPermissions(broker, name, oldDoc);
  			
--- 1056,1064 ----
  		DocumentImpl document, oldDoc = null;
  		DOMStreamer streamer;
  		try {
+ 			lock.acquire(Lock.WRITE_LOCK);
+ 			oldDoc = getDocument(broker, name);
+ 			document = new DocumentImpl(broker, name,	this);
+ 		
  			checkPermissions(broker, name, oldDoc);
  			
***************
*** 1093,1096 ****
--- 1123,1127 ----
  			lock.release();
  		}//ffffffffffffffffffffffffffffffffffffffffffffffff
+ 		
  		try {
  			// second pass: store the document
***************
*** 1099,1105 ****
  			streamer.serialize(node, true);
  	
! 			if(oldDoc == null)
! 			    addDocument(broker, document);
! 			broker.addDocument(this, document);
  			broker.closeDocument();
  			broker.flush();
--- 1130,1139 ----
  			streamer.serialize(node, true);
  	
! 			if(oldDoc == null) {
! 				addDocument(broker, document);
! 				broker.addDocument(this, document);
! 			} else {
! 				broker.updateDocument(document);
! 			}
  			broker.closeDocument();
  			broker.flush();

[Exist-commits] eXist-1.0/src/org/exist/storage/cache LRDCache.java,1.5,1.6

[Wolfgang M. M. <wol...@us...>]

Update of /cvsroot/exist/eXist-1.0/src/org/exist/storage/cache
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4115/src/org/exist/storage/cache

Modified Files:
	LRDCache.java 
Log Message:
* Fixed locking error: a collection object could be unloaded from the cache by one thread, while another thread had still been writing a document 
being a member of the collection. A moment later, a third thread reloaded the collection and started to write to the same document. However, as 
the collection had been recreated, there were two instances of the same document in memory. The third thread did thus not respect the lock held 
by the first thread and started to write without waiting. As a consequence, various page errors were thrown.

* When updating a document, the old document metadata had not been correctly removed. Usually, this had no consequences as the collection class 
only used the last record it found. In some cases however, this may have led to errors.

Index: LRDCache.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/storage/cache/LRDCache.java,v
retrieving revision 1.5
retrieving revision 1.6
diff -C2 -d -r1.5 -r1.6
*** LRDCache.java	4 Jun 2004 09:47:24 -0000	1.5
--- LRDCache.java	28 Jul 2004 18:54:54 -0000	1.6
***************
*** 101,105 ****
  					old.getReferenceCount()
  						/ (double)(totalReferences - old.getTimestamp());
! 				if (minRd < 0 || rd < minRd) {
  					minRd = rd;
  					bucket = i;
--- 101,105 ----
  					old.getReferenceCount()
  						/ (double)(totalReferences - old.getTimestamp());
! 				if ((minRd < 0 || rd < minRd) && old.allowUnload()) {
  					minRd = rd;
  					bucket = i;

[Exist-commits] eXist-1.0/src/org/exist/xquery LocationStep.java,1.13,1.14

[Wolfgang M. M. <wol...@us...>]

Update of /cvsroot/exist/eXist-1.0/src/org/exist/xquery
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4115/src/org/exist/xquery

Modified Files:
	LocationStep.java 
Log Message:
* Fixed locking error: a collection object could be unloaded from the cache by one thread, while another thread had still been writing a document 
being a member of the collection. A moment later, a third thread reloaded the collection and started to write to the same document. However, as 
the collection had been recreated, there were two instances of the same document in memory. The third thread did thus not respect the lock held 
by the first thread and started to write without waiting. As a consequence, various page errors were thrown.

* When updating a document, the old document metadata had not been correctly removed. Usually, this had no consequences as the collection class 
only used the last record it found. In some cases however, this may have led to errors.

Index: LocationStep.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/xquery/LocationStep.java,v
retrieving revision 1.13
retrieving revision 1.14
diff -C2 -d -r1.13 -r1.14
*** LocationStep.java	8 Jun 2004 08:16:09 -0000	1.13
--- LocationStep.java	28 Jul 2004 18:54:56 -0000	1.14
***************
*** 283,286 ****
--- 283,287 ----
  //					ElementValue.ELEMENT, docs, test.getName(), selector
  //			);
+ 			LOG.debug("Found " + result.getLength() + " for " + pprint());
  			return result;
  		}

[Exist-commits] eXist-1.0/src/org/dbxml/core/filer Paged.java,1.24,1.25

[Wolfgang M. M. <wol...@us...>]

Update of /cvsroot/exist/eXist-1.0/src/org/dbxml/core/filer
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4115/src/org/dbxml/core/filer

Modified Files:
	Paged.java 
Log Message:
* Fixed locking error: a collection object could be unloaded from the cache by one thread, while another thread had still been writing a document 
being a member of the collection. A moment later, a third thread reloaded the collection and started to write to the same document. However, as 
the collection had been recreated, there were two instances of the same document in memory. The third thread did thus not respect the lock held 
by the first thread and started to write without waiting. As a consequence, various page errors were thrown.

* When updating a document, the old document metadata had not been correctly removed. Usually, this had no consequences as the collection class 
only used the last record it found. In some cases however, this may have led to errors.

Index: Paged.java
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/dbxml/core/filer/Paged.java,v
retrieving revision 1.24
retrieving revision 1.25
diff -C2 -d -r1.24 -r1.25
*** Paged.java	12 Jul 2004 17:17:43 -0000	1.24
--- Paged.java	28 Jul 2004 18:55:05 -0000	1.25
***************
*** 388,391 ****
--- 388,406 ----
  	}
  
+ 	private boolean isRemovedPage(long pageNum) throws IOException {
+ 		long nextNum = fileHeader.firstFreePage;
+ 		while(nextNum != -1) {
+ 			if(nextNum == pageNum) {
+ 				LOG.error("Page " + pageNum + " has already been removed");
+ 				Thread.dumpStack();
+ 				return true;
+ 			}
+ 			Page next = getPage(nextNum);
+ 			next.read();
+ 			nextNum = next.header.nextPage;
+ 		}
+ 		return false;
+ 	}
+ 	
  	/**
  	 *  setFile sets the file object for this Paged.
***************
*** 417,422 ****
  //			if(getFile().getName().equals("words.dbx"))
  //				LOG.debug("freeing page " + page.getPageNum());
  			// Walk the chain and add it to the unused list
- 			long firstPage = page.pageNum;
  			page.header.setStatus(UNUSED);
  			synchronized (fileHeader) {
--- 432,438 ----
  //			if(getFile().getName().equals("words.dbx"))
  //				LOG.debug("freeing page " + page.getPageNum());
+ 			if(isRemovedPage(page.getPageNum()))
+ 				return;
  			// Walk the chain and add it to the unused list
  			page.header.setStatus(UNUSED);
  			synchronized (fileHeader) {

[Exist-commits] eXist-1.0/src/org/exist/security security.properties,1.4,1.5

[Wolfgang M. M. <wol...@us...>]

Update of /cvsroot/exist/eXist-1.0/src/org/exist/security
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4115/src/org/exist/security

Modified Files:
	security.properties 
Log Message:
* Fixed locking error: a collection object could be unloaded from the cache by one thread, while another thread had still been writing a document 
being a member of the collection. A moment later, a third thread reloaded the collection and started to write to the same document. However, as 
the collection had been recreated, there were two instances of the same document in memory. The third thread did thus not respect the lock held 
by the first thread and started to write without waiting. As a consequence, various page errors were thrown.

* When updating a document, the old document metadata had not been correctly removed. Usually, this had no consequences as the collection class 
only used the last record it found. In some cases however, this may have led to errors.

Index: security.properties
===================================================================
RCS file: /cvsroot/exist/eXist-1.0/src/org/exist/security/security.properties,v
retrieving revision 1.4
retrieving revision 1.5
diff -C2 -d -r1.4 -r1.5
*** security.properties	21 Jun 2004 15:30:23 -0000	1.4
--- security.properties	28 Jul 2004 18:55:05 -0000	1.5
***************
*** 10,12 ****
  # the database will not recognize the old passwords.
  
! passwords.encoding=md5
--- 10,12 ----
  # the database will not recognize the old passwords.
  
! passwords.encoding=simple-md5