We have successufully implemented and tested your patch :=20
- no more duplicated indexation within the same transformation,
- index map is released when document is released.
Thanks for your help.
Enclosed the fixed KeyManager source code.
By the way, can you tell us when the next version is planned to be re=
leased.
-----Message d'origine-----
De: Michael Kay [mailto:michael.h.kay@...]
Date: mercredi 8 janvier 2003 12:41
=C0: FERRER Valerie
Cc: saxon-help@...
Objet: RE: Weak references in saxon 7.2 KeyManager
Thanks for your work on this. A delicate piece of code, not easy to
test.
Here's a fix which I haven't tested:
To method putIndex(), add a fourth argument "Controller controller".
Supply the controller as an argument in the two calls on putIndex().
In method putIndex(), after the line:
indexList =3D new HashMap();
add:
controller.setUserData(doc, "key-index", indexList);
=20
This will create a firm reference to docIndexes held by the Controlle=
r,
which therefore prevents it being garbage collected during the life o=
f a
transformation.
I'd be grateful if you could try this out for me.
Michael Kay
Software AG
home: Michael.H.Kay@...
work: Michael.Kay@...
> -----Original Message-----
> From: FERRER Valerie [mailto:valerie.ferrer@...
> Sent: 08 January 2003 09:06
> To: 'michael.h.kay@...'
> Cc: 'saxon-help@...'
> Subject: RE: Weak references in saxon 7.2 KeyManager
>=20
>=20
> We finally found out the problem causing several indexations=20
> within a single transformation. Using debugger in method=20
> getIndex (class KeyManager), it appeared that after garbage=20
> collection, indexList became null :
>=20
> =09private synchronized Object getIndex(DocumentInfo doc, int
> keyFingerprint)=20
> =09{
> // ref is found in docIndexes WeakHashMap ...
> WeakReference ref =3D (WeakReference)docIndexes.get(doc);
> if (ref=3D=3Dnull) return null;
>=20
> =09 // ... but the referenced object has been garbage=20
> collected : indexList is null !
> HashMap indexList =3D (HashMap)ref.get();
> if (indexList=3D=3Dnull) return null;
> return indexList.get(new Integer(keyFingerprint));
> =09} =20
>=20
> The problem seems to be in putIndex method :
>=20
> =09private synchronized void putIndex(DocumentInfo doc,=20
> int keyFingerprint, Object index)=20
> =09{
> WeakReference indexRef =3D (WeakReference)docIndexes.get(do=
c);
> HashMap indexList;
> if (indexRef=3D=3Dnull || indexRef.get()=3D=3Dnull)=20
> =09 {
> indexList =3D new HashMap();
> docIndexes.put(doc, new WeakReference(indexList));
> }=20
> =09 else=20
> =09 {
> indexList =3D (HashMap)indexRef.get();
> }
> indexList.put(new Integer(keyFingerprint), index);
> =09}
> No strong reference is kept to indexList object. So it is=20
> finalized each time garbage collector is called. We are aware=20
> of the memory leak in 7.1 KeyManager implementation, and that=20
> reference link between indexList content and Document=20
> (NodeInfo referencing
> DocumentInfo) has to be broken to allow Document to be freed;=20
> nevertheless, xsl:key looses its interest if key map is not=20
> persistent along the whole transformation. We could not find=20
> a solution to our problem, do you have any clue ?
>=20
> Thanks.
>=20
> Valerie
>=20
>=20
> -----Message d'origine-----
> De: FERRER Valerie=20
> Date: mardi 7 janvier 2003 15:22
> =C0: 'michael.h.kay@...'
> Cc: 'saxon-help@...'
> Objet: RE: Weak references in saxon 7.2 KeyManager
>=20
>=20
> Thank you for your reply.
>=20
> The source document is a DOMSource wrapped with a JDOM-like=20
> document wrapper (fitted to Infonyte Persistent DOM).
>=20
> The problem we encountered is that indexes are rebuilt=20
> several times within a single transformation. With your=20
> KeyManager explanations and after reading your code more=20
> carefully, it appears that the index hash map shouldn't be=20
> garbage collected while the source document isn't released.=20
> However it is possible that Infonyte component internally=20
> works with several document references for the same input=20
> document. We are investigating...
>=20
> Valerie Ferrer
>=20
>=20
> -----Message d'origine-----
> De: Michael Kay [mailto:michael.h.kay@...]
> Date: mardi 7 janvier 2003 14:44
> =C0: FERRER Valerie
> Objet: RE: Weak references in saxon 7.2 KeyManager
>=20
>=20
> I need to understand better what you are trying to do.
>=20
> How are you supplying the source document to Saxon? Is it as=20
> a DOMSource?
>=20
>=20
> Is the index being rebuilt several times within a single=20
> transformation? Or are you trying to maintain indexes across=20
> multiple transformations?
>=20
> The KeyManager was (as I'm sure you can tell) designed very=20
> carefully to avoid locking things into memory that aren't=20
> needed. It is part of the compiled stylesheet, but it holds=20
> the document indexes which point into source documents. Since=20
> the same compiled stylesheet may be used thousands of times=20
> to process different source documents, it mustn't lock the=20
> source documents into memory, and it must release the indexes=20
> themselves when the source documents are released. On the=20
> other hand, it does allow the index to be reused if multiple=20
> transformations are carried out using the same compiled=20
> stylesheet and the same source document tree.
>=20
> The change in 7.2 was made to fix bug 583939: see=20
> http://sourceforge.net/tracker/index.php?func=3Ddetail&aid=3D58393
> 9&group_id
> =3D29872&atid=3D397617
>=20
> Michael Kay
>=20
> PS: I do prefer it if people raise problems via the=20
> saxon-help list at sourceforge. That way, there is a record=20
> of the correspondence available to everyone.
>=20
>=20
>=20
> > -----Original Message-----
> > From: FERRER Valerie [mailto:valerie.ferrer@...]
> > Sent: 07 January 2003 12:54
> > To: 'michael.h.kay@...'
> > Subject: Weak references in saxon 7.2 KeyManager
> >=20
> >=20
> > Hi,
> >=20
> > We are currently using Saxon 7.x to process huge xml
> > documents (at least 200 Mo), this is possible thanks to=20
> > persistant DOM (supplied by Infonyte). We have encountered=20
> > performance problems with key xpath functions when migrating=20
> > from saxon 7.1 to saxon 7.2.
> >=20
> > After profiling saxon transformation with optimize it, we
> > found out that the indexation was made each time the key=20
> > function is called, this is due to the use of WeakReference=20
> > to keymap implemented since 7.2 release.=20
> > Since our transformation requires large amount of memory, we=20
> > can't afford solving this problem by setting the heap to a=20
> > larger size.
> >=20
> > Thanks for any help, clue or patch.
> >=20
> > Valerie Ferrer
> >=20
> >=20
> >=20
> > This e-mail is intended only for the above addressee. It may
> > contain privileged information. If you are not the addressee=20
> > you must not copy, distribute, disclose or use any of the=20
> > information in it. If you have received it in error please=20
> > delete it and immediately notify the sender. Security Notice:=
=20
> > all e-mail, sent to or from this address, may be accessed by=20
> > someone other than the recipient, for system management and=20
> > security reasons. This access is controlled under Regulation=20
> > of Investigatory Powers Act 2000, Lawful Business Practises.
> >=20
>=20
>=20
> This mail has originated outside your organization,
> either from an external partner or the Global Internet.=20
> Keep this in mind if you answer this message.
>=20
> This e-mail is intended only for the above addressee. It may contai=
n
> privileged information. If you are not the addressee you must=20
> not copy,
> distribute, disclose or use any of the information in it. If you ha=
ve
> received it in error please delete it and immediately notify=20
> the sender.
> Security Notice: all e-mail, sent to or from this address, may be
> accessed by someone other than the recipient, for system=20
> management and
> security reasons. This access is controlled under Regulation of
> Investigatory Powers Act 2000, Lawful Business Practises.
>=20
This mail has originated outside your organization,
either from an external partner or the Global Internet.=20
Keep this in mind if you answer this message.
This e-mail is intended only for the above addressee. It may contain
privileged information. If you are not the addressee you must not cop=
y,
distribute, disclose or use any of the information in it. If you have
received it in error please delete it and immediately notify the send=
er.
Security Notice: all e-mail, sent to or from this address, may be
accessed by someone other than the recipient, for system management a=
nd
security reasons. This access is controlled under Regulation of
Investigatory Powers Act 2000, Lawful Business Practises.
|