Thread: [Xforge-users] New Feature implemented: FilesMonitor
Brought to you by:
agaroffolo,
nemorino
From: Ulrich M. <ul...@de...> - 2003-02-05 12:12:47
|
Hello, there's a new class FilesMonitor, which is like the existing FileMonitor, only that it can monitor several files. If any of those files changes, the MonitorsManager will pick it up, because the maximum modification time of all files is returned. I have updated the XSL component to use the FilesManager. If a parameter "xmlfileid" is configured in the configuration file, then the XSL component will use the new FilesMonitor to monitor the stylesheet and the XML file. If it is not configured, the old FileMonitor will be used to just monitor the stylesheet. The "xmlfileid" parameter needs to be set to an existing XML file (it would make sense to use the XML file that contains the call to the XSL component). The test application has been doing this all along, it set "testfile" to the file being processed, so I updated components.xml to use "testfile" as the value of the "xmlfileid" parameter. Perhaps it would be good to have a central X:Forge mechanism, that would allow me to get the identifier of the current resource being processed, but we have to consider that this resource does not have to be a file. And if it's not a file, it may be hard to invent a unique identifier for it. Thoughts? cheers, Ulrich -- Ulrich Mayring DENIC eG, Systementwicklung |
From: Ulrich M. <ul...@de...> - 2003-02-12 13:26:15
|
Ulrich Mayring wrote: > > Perhaps it would be good to have a central X:Forge mechanism, that would > allow me to get the identifier of the current resource being processed, > but we have to consider that this resource does not have to be a file. > And if it's not a file, it may be hard to invent a unique identifier for > it. Thoughts? I think we should introduce a new member variable to the XForgeProcessor called processedURI with a get and set method. It could be set externally (say in the Test program) and a component could then retrieve it. Then we would have a standard way for a component to find out the URI of the XML document being currently processed (e.g. for monitoring or cacheing purposes). If the component uses this feature, it acquires the responsibility for dealing with the processedURI being null (e.g. when it was not set externally). Agree? cheers, Ulrich -- Ulrich Mayring DENIC eG, Systementwicklung |
From: Ulrich M. <ul...@de...> - 2003-02-12 13:48:37
|
Ulrich Mayring wrote: > I think we should introduce a new member variable to the XForgeProcessor > called processedURI with a get and set method. It could be set > externally (say in the Test program) and a component could then retrieve > it. Whoops, that doesn't work of course, as I can't get a reference to the XForgeProcessor in a component (and probably shouldn't be able to). More thinking to do... :) Ulrich -- Ulrich Mayring DENIC eG, Systementwicklung |
From: Ulrich M. <ul...@de...> - 2003-02-12 16:16:53
|
Ulrich Mayring wrote: [...retrieve id of processed XML document in component...] Ok... I have an idea now. It is a fairly major change to the codebase, but it might save us a lot of work in the future: We define a new class ProcessingOptions (abbreviated PO in the following text), which contains fields and get/set methods for encoding and processedResourceURI (and more, if we need something else later on). The external program (like Test.java) can instantiate such an object and pass it to the XForgeProcessor. The XForgeProcessor passes the PO to the ElementFactory and it passes them to the element created. This element sets it to the component it creates. Then every component and every element would automatically have a PO object (possibly with default values, if not set externally). We could easily extend the PO class later on and would not have to make any major changes to the codebase anymore, because the extended PO class would be backwards compatible. The new fields would be used, where needed, but the old code can remain the same, if it doesn't need the new fields. Here's the plan in detail: 1. Make PO class. Initially with String fields and getters/setters for encoding and URI. Make constructor without arguments. 2. Give XForgeProcessor a private field for a PO object. During configure() method initialize the field to a default PO: Set PO's encoding to encoding from XForgeProcessor configuration. If not present there, then set to XForgeProcessor.DEFAULT_ENCODING. Set PO's URI to "". Add a public setPO(...) method to the XForgeProcessor. 3. Change XForgeProcessor to pass PO instead of encoding to ElementFactory. 4. Change ElementFactory to pass PO to AutomationElement and ProcessElement. Note: what about reflection-mapped Beans? What element do they use? 5. Add a method setPO(...) to AbstractXForgeComponent, which sets the received PO to a private field. 6. Change AutomationElement and ProcessElement to replace call to component.setEncoding() with call to component.setPO(...). Since all components extend AbstractXForgeComponent they should have a setPO(...) method by now. 7. Replace all calls to getEncoding() in components with this.PO.getEncoding(). 8. Change XSL component to get rid of the xmlfileid parameter and use this.PO.getURI() instead. Change XSL component docs to reflect this. 9. Change Test.java to instantiate a PO object and set it to the XForgeProcessor instead of setting the encoding. Now we do some testing. If everything works, we should clean up: 1. Change XForgeComponent interface: delete getEncoding() and setEncoding() and add setPO(). 2. Delete setEncoding(...) and getEncoding() methods and protected encoding field in AbstractXForgeComponent. 3. Delete method setEncoding(...) and protected field encoding from XForgeProcessor. Now we can deal with the rest of the *Element classes: 1. Change ElementFactory to pass a PO object to all Elements, instead of the encoding. 2. Change all *Element classes to receive a PO object instead of the encoding. 3. Replace all calls to getEncoding() in *Element classes with this.PO.getEncoding(). 4. Change ParameterElement to create ParameterValueImpl with a PO instead of the encoding. 5. Change ParameterValueImpl to receive a PO instead of the encoding. Make private field for PO object and set it from constructor. Replace all calls to this.encoding with this.PO.getEncoding(). Now we do some testing. If everything works, we should clean up: 1. Delete getEncoding() from interface Element. 2. Delete getEncoding() and protected field encoding from all *Element classes. 3. Delete private field encoding from ParameterValueImpl. Ok... any comments? :) cheers, Ulrich -- Ulrich Mayring DENIC eG, Systementwicklung |
From: Alberto G. <alb...@ge...> - 2003-02-12 16:13:31
|
----- Original Message ----- From: "Ulrich Mayring" <ul...@de...> To: "xforge-users Mailinglist" <xfo...@li...> Sent: Wednesday, February 12, 2003 2:22 PM Subject: Re: [Xforge-users] New Feature implemented: FilesMonitor > Ulrich Mayring wrote: > > > > Perhaps it would be good to have a central X:Forge mechanism, that would > > allow me to get the identifier of the current resource being processed, > > but we have to consider that this resource does not have to be a file. > > And if it's not a file, it may be hard to invent a unique identifier for > > it. Thoughts? > Hhmmm .... probably i didn' t got the point .... Why do u need to have a unique identifier associated with the current resource? Monitors are already handled by the MonitorsManager which after a request is processed starts a context containing all the monitors needed to "monitor" components used, according to the context class specified by the current request context (see XForgeC2URLContext / XForgeC1URLContext / XForgeTestURLContext). Perhaps i misunderstood .... Ciao, --- Alberto. |
From: Ulrich M. <ul...@de...> - 2003-02-12 16:37:00
|
Alberto Garoffolo wrote: > Hhmmm .... probably i didn' t got the point .... > Why do u need to have a unique identifier associated with the current > resource? > Monitors are already handled by the MonitorsManager which after a request is > processed starts a context containing all the monitors needed to "monitor" > components used, according to the context class specified by the current > request > context > (see XForgeC2URLContext / XForgeC1URLContext / XForgeTestURLContext). > > Perhaps i misunderstood .... In the XSLComponent I need to know an identifier for the current XML resource being processed. Because if that identifier is a file (I test for that), then I have to return a FilesMonitor, which monitors both files (the XML file and the stylesheet) for modifications. For example, run this test: ./run.sh test/XSLT.xml test/result.xml 100000 true Then during the run do: touch test/XSLT.xml touch test/test.xsl You will see in xforge.log (grep xforge.log "isn't changed"|wc -l) that each touch is picked up correctly by the monitoring system. The component is run three times altogether: The first time during the very first invocation. Then after the touch to the XML file, then again after the touch to the stylesheet. If, however, you change in Test.java this line: broker.putObject(threadcontext,"testfile",src); to something like: broker.putObject(threadcontext,"testfile","ThisIsNoFile"); and redo the same test, you will find that the monitoring system only picks up the touch to the stylesheet. This is because the XSL component found that "ThisIsNoFile" is not a file and so cannot return a Monitor for it. cheers, Ulrich -- Ulrich Mayring DENIC eG, Systementwicklung |
From: Alberto G. <alb...@ge...> - 2003-02-13 14:28:43
|
Hi, IMHO cacheing/monitoring implemented in XSLComponent overlaps with the native X:Forge monitoring system. X:Forge monitoring system philosophy is: Components doesn' t have to care about changes, when they are invoked they just produce xml and, if they are monitorizable, return a monitor which checks if the resources used in the *current request* change. If monitored resources for a specific request aren' t changed, associated components will not be called at all. In the new XSLComponent version, there is also the possibilty to monitor the whole xml file used by x:forge. In my opinion this should be done by the generator/producer/"who loads the file" in a cocoon/servlet like system. Transformer cache in XSLComponent, doesn' t consider that a single Component instance can be used to apply different stylesheets: <code> if (hasChanged(mFiles[0])) { try { // Initialize a new Transformer log.debug("Initializing a new transformer."); StreamSource stylesheet = new StreamSource(mFiles[0]); mTransformer = mTransformerFactory.newTransformer(stylesheet); mTransformer.setParameter("lastModified",new Long(mFiles[0].lastModified())); } catch (TransformerConfigurationException tce) { throw new XForgeException("Something wrong with the transformer: " + tce.toString()); } } else { log.debug("Using cached transformer."); } [...] // do the transformation try { mTransformer.transform(src,dest); } catch (TransformerException te) { throw new XForgeException("Problem transforming " + xml + " with " + xsl + ": " + te.toString()); } </code> <bug> <xsltest1> <xsl:apply> <xf:parameter name="xml"> <xsldate> <date:xmlnow/> </xsldate> </xf:parameter> <xf:parameter name="xsl">test/test.xsl</xf:parameter> </xsl:apply> </xsltest1> <xsltest2> <xsl:apply> <xf:parameter name="xml"> <xsldate> <date:xmlnow/> </xsldate> </xf:parameter> <xf:parameter name="xsl">test/test2.xsl</xf:parameter> </xsl:apply> </xsltest2> <xsltest3> <xsl:apply> <xf:parameter name="xml"> <xsldate> <date:xmlnow/> </xsldate> </xf:parameter> <xf:parameter name="xsl">test/test.xsl</xf:parameter> </xsl:apply> </xsltest3> </bug> The third trasformation will apply test2.xsl instead of test.xsl. As u can see to make it work correctly, more code has to be written in the xslcomponent, weighting down it a lot. Consider also that if we follow this philosophy, each component will reinvent the wheel each time needs to cache something. The advantage of the X:Forge monitoring system philosphy is to make component logic easier and cleaner. The disadvantage is to re-run a component also if changed another component used in that request. To avoid this we should watch at the problem in a more general way; for example (1) thinking about cacheing each component sax result. For example: <test> <comp1:dosomething> <xf:param name="param1"> <comp2:dosomething/> </xf:param> </comp1:dosomething> <comp3:dosomething> <xf:param name="param1"> <comp4:dosomething/> </xf:param> </comp1:dosomething> </test> If comp2 changes, we must invalidate all it' s ancestors results, but we can reuse the ouput of comp3. This is just an idea ... But probably u need even something more: You want to cache one of the params passed to the component (in this case the Transfomation object created parsing the xsl) .... This is more difficult ... Hhhhmmm .... Do u really need it ( considered (1) ) ? :) Ciao, --- Alberto. |
From: Ulrich M. <ul...@de...> - 2003-02-13 14:55:12
|
Alberto Garoffolo wrote: > Hi, > > IMHO cacheing/monitoring implemented in XSLComponent > overlaps with the native X:Forge monitoring system. Yes, unfortunately. > X:Forge monitoring system philosophy is: Components > doesn' t have to care about changes, when they are invoked > they just produce xml and, if they are monitorizable, > return a monitor which checks if the resources used in the > *current request* change. If monitored resources for a > specific request aren' t changed, associated components > will not be called at all. Ok. > In the new XSLComponent version, there is also the possibilty > to monitor the whole xml file used by x:forge. In my opinion > this should be done by the generator/producer/"who loads the file" > in a cocoon/servlet like system. > > Transformer cache in XSLComponent, doesn' t consider that a > single Component instance can be used to apply different stylesheets: > > <bug> [...] > </bug> > The third trasformation will apply test2.xsl instead of test.xsl. Hmm... I haven't tried it, but I think you're right that there's a bug. > As u can see to make it work correctly, more code has to be > written in the xslcomponent, weighting down it a lot. Consider > also that if we follow this philosophy, each component will reinvent > the wheel each time needs to cache something. I absolutely agree with you. It's just that I have no better solution :( > The advantage of the X:Forge monitoring system philosphy is > to make component logic easier and cleaner. > > The disadvantage is to re-run a component also if changed > another component used in that request. To avoid this we > should watch at the problem in a more general way; for > example (1) thinking about cacheing each component sax result. > For example: > > <test> > <comp1:dosomething> > <xf:param name="param1"> > <comp2:dosomething/> > </xf:param> > </comp1:dosomething> > > <comp3:dosomething> > <xf:param name="param1"> > <comp4:dosomething/> > </xf:param> > </comp1:dosomething> > </test> > > If comp2 changes, we must invalidate all it' s ancestors results, > but we can reuse the ouput of comp3. > This is just an idea ... > > But probably u need even something more: You want to cache > one of the params passed to the component (in this case the > Transfomation object created parsing the xsl) .... This is more > difficult ... > Hhhhmmm .... Do u really need it ( considered (1) ) ? :) Stylesheet creation is an expensive operation and in many real-world scenarios you must be able to do it. Consider a Cocoon-like system, where you have many users accessing a page at the same time. If you have to recreate the transformer each time, it takes 2 seconds longer to wait for the user on every request. On the other hand the system has to pick up changes to the stylesheet and the XML files being transformed, because in a production scenario you can't constantly restart your application. I'm absolutely in favor of solving this problem outside of the XSL component, but currently have no idea how to change the monitoring system, so that it supports such things. If you want we can do some brainstorming: Maybe we need a FileComponent, which we can pass as parameter to the XSLComponent. The FileComponent could return a FileMonitor in its getMonitor() method, the XSLComponent could return a ComponentMonitor for the passed FileComponent. The ComponentMonitor's getLastModified() method would return this.component.getMonitor().getLastModified(). Ulrich -- Ulrich Mayring DENIC eG, Systementwicklung |
From: Ulrich M. <ul...@de...> - 2003-02-17 16:52:06
|
Ulrich Mayring wrote: > Alberto Garoffolo wrote: > >> The disadvantage is to re-run a component also if changed >> another component used in that request. To avoid this we >> should watch at the problem in a more general way; for >> example (1) thinking about cacheing each component sax result. >> For example: >> >> <test> >> <comp1:dosomething> >> <xf:param name="param1"> >> <comp2:dosomething/> >> </xf:param> >> </comp1:dosomething> >> >> <comp3:dosomething> >> <xf:param name="param1"> >> <comp4:dosomething/> >> </xf:param> >> </comp1:dosomething> >> </test> >> >> If comp2 changes, we must invalidate all it' s ancestors results, >> but we can reuse the ouput of comp3. You cannot always assume a hierarchical structure, look at this example: <mycomponent1:setDatabase> <xf:parameter name="dbname">PRODUCTION_DB</xf:parameter> </mycomponent1:setDatabase> <mycomponent2:getContactFromDB> <xf:parameter name="contactId">12345</xf:parameter> </mycomponent2:getContactFromDB> Suppose someone changes "PRODUCTION_DB" to "DEV_DB". It is quite possible that the contact with ID 12345 is a different person in DEV_DB or may not even exist there. So, we need to monitor arbitrary XML elements (which would also include the <xf:parameter> element). But we cannot, for example, refer up the tree or sideways, that would violate IOC. Therefore, instead of defining monitors locally to components, they must be defined centrally. Basically we need something like the Sitemap in Cocoon. But since I don't like the Cocoon Sitemap concept (it basically brought me to X:Forge) I won't do it ;-) The answer IMHO is to design components correctly. The above example is a braindead design and so is the XSLComponent. A better design would be this: 1) Make a TransformerComponent. It will always know, when it has changed via a simple FileMonitor. 2) Invent a way to pass objects instead of Strings with the <xf:parameter> element. For example: <xsl:apply> <xf:parameter name="xml"> <!-- will be passed as String --> <foo:bar/> </xf:parameter> <xf:parameter name="xsl" type="object"> <!-- will be passed as Object --> <transformer:get> <xf:parameter> test.xsl </xf:parameter> </transformer:get> </xf:parameter> </xsl:apply> 3) Now the XSLComponent can implement Unchangeable, because changes only occur, when either the transformer component or the foo component changes. This solves all problems except where the XML input is at least partly static like here: <xf:parameter name="xml"> <list1> <foo:bar/> </list1> </xf:parameter> Suppose <foo:bar/> is unchangeable, but someone changes "list1" to "list2", so that after stylesheet application he gets a different layout. Right now static (non-namespace prefixed tags) are simply converted into SAX events (parent.startElement(...) etc.). Perhaps we could design a generic component, which is responsible for all tags that have no namespace prefix (i.e. static tags). It could generate the SAX events like now, but have additional features: 1) It could implement Monitorizable and the getMonitor() method would return "this". 2) It could implement Monitor and the lastModified() method would return an internal timestamp. 3) The timestamp would be set on the first invocation and updated to now, whenever the current SAX events are different from the SAX events generated by the last invocation. Now, is this totally stupid or what? :) Ulrich -- Ulrich Mayring DENIC eG, Systementwicklung |
From: Alberto G. <alb...@ge...> - 2003-02-18 10:33:56
|
----- Original Message ----- From: "Ulrich Mayring" <ul...@de...> To: "xforge-users Mailinglist" <xfo...@li...> Sent: Monday, February 17, 2003 6:51 PM Subject: [Xforge-users] Re: Monitoring/Cacheing > Ulrich Mayring wrote: > > > Alberto Garoffolo wrote: > > > >> The disadvantage is to re-run a component also if changed > >> another component used in that request. To avoid this we > >> should watch at the problem in a more general way; for > >> example (1) thinking about cacheing each component sax result. > >> For example: > >> > >> <test> > >> <comp1:dosomething> > >> <xf:param name="param1"> > >> <comp2:dosomething/> > >> </xf:param> > >> </comp1:dosomething> > >> > >> <comp3:dosomething> > >> <xf:param name="param1"> > >> <comp4:dosomething/> > >> </xf:param> > >> </comp1:dosomething> > >> </test> > >> > >> If comp2 changes, we must invalidate all it' s ancestors results, > >> but we can reuse the ouput of comp3. > > You cannot always assume a hierarchical structure, look at this example: > > <mycomponent1:setDatabase> > <xf:parameter name="dbname">PRODUCTION_DB</xf:parameter> > </mycomponent1:setDatabase> > > <mycomponent2:getContactFromDB> > <xf:parameter name="contactId">12345</xf:parameter> > </mycomponent2:getContactFromDB> > > Suppose someone changes "PRODUCTION_DB" to "DEV_DB". It is quite > possible that the contact with ID 12345 is a different person in DEV_DB > or may not even exist there. > (1) > So, we need to monitor arbitrary XML elements (which would also include > the <xf:parameter> element). But we cannot, for example, refer up the > tree or sideways, that would violate IOC. > > Therefore, instead of defining monitors locally to components, they must > be defined centrally. Basically we need something like the Sitemap in > Cocoon. But since I don't like the Cocoon Sitemap concept (it basically > brought me to X:Forge) I won't do it ;-) > > The answer IMHO is to design components correctly. The above example is > a braindead design and so is the XSLComponent. A better design would be > this: > > 1) Make a TransformerComponent. It will always know, when it has changed > via a simple FileMonitor. > > 2) Invent a way to pass objects instead of Strings with the > <xf:parameter> element. For example: > > <xsl:apply> > > <xf:parameter name="xml"> > <!-- will be passed as String --> > <foo:bar/> > </xf:parameter> > > <xf:parameter name="xsl" type="object"> > <!-- will be passed as Object --> > <transformer:get> > <xf:parameter> > test.xsl > </xf:parameter> > </transformer:get> > </xf:parameter> > > </xsl:apply> > I thought the same thing :) I totally agree with you! > 3) Now the XSLComponent can implement Unchangeable, because changes only > occur, when either the transformer component or the foo component > changes. This solves all problems except where the XML input is at least > partly static like here: > > <xf:parameter name="xml"> > <list1> > <foo:bar/> > </list1> > </xf:parameter> > > Suppose <foo:bar/> is unchangeable, but someone changes "list1" to > "list2", so that after stylesheet application he gets a different layout. > Here is, in my opinion, a misunderstanding (also related to (1)): If someone changes the input xml (for example test.xml) , all the trasformation must be re-performed. We have no way to know if and what is changed inside the xml. We can only ask to our components if, with *unchanged input*, something external has changed. [...] Ciao, --- Alberto. |
From: Ulrich M. <ul...@de...> - 2003-02-18 11:05:48
|
Alberto Garoffolo wrote: > > Here is, in my opinion, a misunderstanding (also related to (1)): > If someone changes the input xml (for example test.xml) , > all the trasformation must be re-performed. We have no way to know > if and what is changed inside the xml. We can only ask to our components > if, with *unchanged input*, something external has changed. Yes, I agree with you. But that means we still need a way to find out whether the input XML has changed. Because only if it hasn't changed, we should use the monitoring system. And since we don't know whether the input XML is a file or a stream or something else, we need a very generic mechanism to monitor it. My suggestion with monitoring static tags is meant for exactly that problem. It is a little overkill and I'd prefer something simpler, but currently I have no other idea how to monitor the input XML. cheers, Ulrich -- Ulrich Mayring DENIC eG, Systementwicklung |
From: Ulrich M. <ul...@de...> - 2003-02-24 10:01:49
|
> Ulrich Mayring wrote: > > 1) Make a TransformerComponent. It will always know, when it has changed > via a simple FileMonitor. > > 2) Invent a way to pass objects instead of Strings with the > <xf:parameter> element. > > 3) Now the XSLComponent can implement Unchangeable, because changes only > occur, when the transformer component changes. Suppose we implement this and have the following code: <xsl:apply> <xf:parameter name="xml"><foo>bar</foo></xf:parameter> <xf:parameter name="xsl" type="object"> <transformer:get> <xf:parameter>test1.xsl</xf:parameter> </transformer:get> </xf:parameter> </xsl:apply> ... <xsl:apply> <xf:parameter name="xml"><foo>bar</foo></xf:parameter> <xf:parameter name="xsl" type="object"> <transformer:get> <xf:parameter>test2.xsl</xf:parameter> </transformer:get> </xf:parameter> </xsl:apply> The Transformer component is first called with test1.xsl, then with test2.xsl. It needs to keep an internal Hashtable of all transformers it creates and when it receives a request for a specific transformer it either already has it and returns it or it creates it and saves it internally. However, we have no way of knowing which Transformer component is returned by the component manager, so *all* Transformer components have to have the same internal Hashtable of transformers. This has two disadvantages: 1) Potential for a lot of duplication (every Transformer components needs to save every transformer internally). 2) Unnecessary creation of a new transformer (several Transformer components may already have it, but the one that is returned to us doesn't have it). Any ideas? Ulrich -- Ulrich Mayring DENIC eG, Systementwicklung |