In case an additional element appears in the XML which is not available in CAM
template, it throws error as "Extra Unknown Elements".
Is there a configuration, possibility that this element be ignored? As our
schema files are huge owing to following industry standards, I have 2 choices:
Use the big exhaustive schema files to generate CAM templates
Make CAM somehow ignore the unnecessary element but focus only on the validations which are provided.
Consider this.... A huge XML is provided, CAM should validate only the
required elements and need not bother about other elements whether they are
present or not.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Firstly, this is a great point to make. With JCAM you can invoke the
validation using the lax validator which will not throw the Extra Elements
errors if it finds unknown elements.
This is done by changing the Container definition file NanoConfig.xml. in this
file you will see some commented out line you need to use the
UnOrderedValidatorLax validator. Remember to comment out the line for the
other validator as only one should be used at any one time. Please note that
XML supplied willnot be checked for order information, i.e. Elements may occur
in any order within a given parent.
For CAMV the way round this is to define the parent as having a datatype of
any. If an child element has a rule on it it will be checked, so having
mandatory child elements will still get checked. The extra elements 'may be'
view during the check so there may be a performance hit.
I hope this helps.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Can you please elaborate on "define the parent as having a datatype of any."
as far as I know CAM does not uses / imports data types so am bit confused.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
There is an action within CAM called datatype that allows you to define the
datatype of a field. There is a type called "any" that has this feature of
allowing extra input.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
My suggestion is to still allow a compilation option that could take a three
way option.
Default - if no rule applied then field is Mandatory.
Optional - if no rule applied then field is Optional
Exclude - if no rule applied the field must not exist.
What do you think? The advantage of using the compiler is that it does not
affect the editors handling of things and it does allow a flexible approach to
the problem.
An alternative would be to introduce a Structure level attribute that does the
same thing and this would drive the compiling process. The advantage of this
would be that it could affect documentation as well making it very clear to
the user what will happen.
Thanks
Martin
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
ok I understand but as per your suggestion, all this happens at Design time or
at template level but it certainly cannot be decided at design-time but has to
happen at run time.
Let me explain it:
We are exposing WebSerices which will comply to an Industry Standard Schema indicating that we do not have any control or make any modifications to the schema.
For a particular service, in the schema there are approx. 100 elements which are mostly Optional in nature.
We only need say 5 elements for our implementation and can discard remaining 95.
A particular consumer can choose to send XML with only 5 required elements.
A particular consumer say can choose to send complete XML with 5 required elements populated and remaining empty tags.
Both the cosumers are sending a "Valid" XML as both of the requests comply
with schema.
Now I have two choices:
Create CAM template based upon the complete schema and mark 95 elements as optional (Effort, Performance are issues).
Create a cam template with only 5 elements (Effort, Performance are pluses)
As I understand you suggestions is about defining the cam template with
complete schema and use options at design time but it will not be able to
handle both the scenarios where consumers can choose to send complete XML or
minimal XML.
Did I understand your response correctly?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I believe you can construct a small template that validates the 5 items you
need and by marking the parents of any of the extra nodes as having a datatype
of any the extra nodes will be ignored.
Martin
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Martin, I agree. The hybrid approach here with a partial structure outline
should work. Include the pieces you definitely want - then for the other parts
include just the parent node in the tree - mark it optional and datatype(any)
- which should take care of all the permutations,
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
ok it works, but it works for only the element where it is specified and not
for all the nested elements or the tree underneath. Can you please confirm, if
it is the desired behavior.
If yes, then it it will reduce specifying the Optional at each elemet to each
complex part.
I was expecting to put only one at the root element which apparently is not
the behavior, please confirm
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi, Can I take it that you want to go down trees that are defined in the
template but when you come to an element that is 'extra' you do not want to
traverse its tree. This is the behaviour I had intended to implement and
looking at the code I can see that I need to put in another check before I
look at children. Is this what you want?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I have checked the behaviour and I am confident that it is as you found.
That is: the element with any is allowed to contain extra elements directly
within it. These extra elements can be whole structures. Within the element
with the any only the elements defined in the template are allowed so it you
need them to contain extra elements you need to put an 'any' on their parent
element.
I believe this is the same behaviour as you would find in XML schema using the
xs:any element. i.e. If a sequence contains named fields and an any then the
fields will have to be valid according to their rules and the any will allow
extra elements.
Putting an any in on the few elements that you know may contain extras will be
very short work in the editor if you define them in one go the editor add rule
wizard remembers the last command so a simple point and click is all that is
require.
As this behaviour is what I would have expected I plan to do no changes as a
result of this.
BTW I have no idea if JCAM can cope with this behaviour.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thanks Martin for the reply, here are my responses.
In XML schema, in case data type is specified as xsd:any, there can be anything specified underneath and yes it can be a whole complex part which can constitute of complete tree. This is the way XML schema works and is used extensively by the open standards we are following i.e the STAR standards.
Specifying datatype any at each level will be really cumbersome, though I understand that editor will ease that out till some extent. Maybe you want to look at some sample schemas available from http://www.starstandard.org/SIGXML/XMLSchemas
The schemas are really nested and way big.
However if you have decided not to implement it, I will look at my end what best can be done. May be in light of above two points, you might want to reconsider your decision.
I have also tried the behavior of putting datatype any in JCAM and have two observations a) After an unknown element is found and reported, it does not reports any subsequent extra elements. b) If datatype <any> is specified at say root level, it stops validating the tree underneath. Hence any cannot be used effectively in JCAM.</any>
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi,
1) I agree that any is a useful and widely used if frustrating feature of XSD.
2) it is possible in CAM to use the ability of xpath to select a group of
elements for which the rule will be applied. For example in this case it would
be possible to use an xpath that selected all elements that have children. For
example "//*" selects all elements that are parent.
This allows you to set a rule once and have it apply to many nodes.
3) I think the above covers your need.
4) I am sorry but JCAM was implemented before any was really sorted and I am
not surprised to find it does work. Sorry.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Yes that's a great idea, "//" should work. I will try it today.
But just to let you know that we are getting performance issues with CAMV
component when we are having large XML files. This I believe will be due to
CAMV parsing each and every line of XML.
Here is the scenario:
We are receiving Orders with multiple line items. The line items can be
multiple and say this instance it is around 500 leading to a large XML.
There is only one validation which should be performed i.e. quantity must be
I believe CAMV is parsing each and every node in XML atleast for mandatory
check or it's additional existance.
As a result the performance of CAMV is directly proportionate to the size of
XML being validated and not on the number of checks to be performed. This does
not looks a very good design to me. To me the performance should be directly
proportionate to the number of checks to be performed along with the size of
XML.
If I have 500 line items, with 5 elements each, I believe currently it will
perform 7500 checks = 50053
Vs. if it performs only required checks and no implict checks:
500 checks = 500*1
I still believe it will be a huge saving for processing and should be able to
be achive as a configurable setting.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
In case an additional element appears in the XML which is not available in CAM
template, it throws error as "Extra Unknown Elements".
Is there a configuration, possibility that this element be ignored? As our
schema files are huge owing to following industry standards, I have 2 choices:
Consider this.... A huge XML is provided, CAM should validate only the
required elements and need not bother about other elements whether they are
present or not.
Firstly, this is a great point to make. With JCAM you can invoke the
validation using the lax validator which will not throw the Extra Elements
errors if it finds unknown elements.
This is done by changing the Container definition file NanoConfig.xml. in this
file you will see some commented out line you need to use the
UnOrderedValidatorLax validator. Remember to comment out the line for the
other validator as only one should be used at any one time. Please note that
XML supplied willnot be checked for order information, i.e. Elements may occur
in any order within a given parent.
For CAMV the way round this is to define the parent as having a datatype of
any. If an child element has a rule on it it will be checked, so having
mandatory child elements will still get checked. The extra elements 'may be'
view during the check so there may be a performance hit.
I hope this helps.
Can you please elaborate on "define the parent as having a datatype of any."
as far as I know CAM does not uses / imports data types so am bit confused.
There is an action within CAM called datatype that allows you to define the
datatype of a field. There is a type called "any" that has this feature of
allowing extra input.
ok I will try using that, but in conjunction with the mandatory elements
thread, here is my suggestion:
CAM should validate only the provided rules (checks or constraints) and should
not evaluate others elements unless specified.
This is about configuring the implicit behavior on validating for missing
elements or validating for mandatory elements.
My suggestion is to still allow a compilation option that could take a three
way option.
Default - if no rule applied then field is Mandatory.
Optional - if no rule applied then field is Optional
Exclude - if no rule applied the field must not exist.
What do you think? The advantage of using the compiler is that it does not
affect the editors handling of things and it does allow a flexible approach to
the problem.
An alternative would be to introduce a Structure level attribute that does the
same thing and this would drive the compiling process. The advantage of this
would be that it could affect documentation as well making it very clear to
the user what will happen.
Thanks
Martin
ok I understand but as per your suggestion, all this happens at Design time or
at template level but it certainly cannot be decided at design-time but has to
happen at run time.
Let me explain it:
We are exposing WebSerices which will comply to an Industry Standard Schema indicating that we do not have any control or make any modifications to the schema.
For a particular service, in the schema there are approx. 100 elements which are mostly Optional in nature.
We only need say 5 elements for our implementation and can discard remaining 95.
A particular consumer can choose to send XML with only 5 required elements.
A particular consumer say can choose to send complete XML with 5 required elements populated and remaining empty tags.
Both the cosumers are sending a "Valid" XML as both of the requests comply
with schema.
Now I have two choices:
As I understand you suggestions is about defining the cam template with
complete schema and use options at design time but it will not be able to
handle both the scenarios where consumers can choose to send complete XML or
minimal XML.
Did I understand your response correctly?
I believe you can construct a small template that validates the 5 items you
need and by marking the parents of any of the extra nodes as having a datatype
of any the extra nodes will be ignored.
Martin
Martin, I agree. The hybrid approach here with a partial structure outline
should work. Include the pieces you definitely want - then for the other parts
include just the parent node in the tree - mark it optional and datatype(any)
- which should take care of all the permutations,
Sure, Let me try and I will revert back, Thanks
ok it works, but it works for only the element where it is specified and not
for all the nested elements or the tree underneath. Can you please confirm, if
it is the desired behavior.
If yes, then it it will reduce specifying the Optional at each elemet to each
complex part.
I was expecting to put only one at the root element which apparently is not
the behavior, please confirm
Hi, Can I take it that you want to go down trees that are defined in the
template but when you come to an element that is 'extra' you do not want to
traverse its tree. This is the behaviour I had intended to implement and
looking at the code I can see that I need to put in another check before I
look at children. Is this what you want?
Ignore my last reply!
I have checked the behaviour and I am confident that it is as you found.
That is: the element with any is allowed to contain extra elements directly
within it. These extra elements can be whole structures. Within the element
with the any only the elements defined in the template are allowed so it you
need them to contain extra elements you need to put an 'any' on their parent
element.
I believe this is the same behaviour as you would find in XML schema using the
xs:any element. i.e. If a sequence contains named fields and an any then the
fields will have to be valid according to their rules and the any will allow
extra elements.
Putting an any in on the few elements that you know may contain extras will be
very short work in the editor if you define them in one go the editor add rule
wizard remembers the last command so a simple point and click is all that is
require.
As this behaviour is what I would have expected I plan to do no changes as a
result of this.
BTW I have no idea if JCAM can cope with this behaviour.
Thanks Martin for the reply, here are my responses.
In XML schema, in case data type is specified as xsd:any, there can be anything specified underneath and yes it can be a whole complex part which can constitute of complete tree. This is the way XML schema works and is used extensively by the open standards we are following i.e the STAR standards.
Specifying datatype any at each level will be really cumbersome, though I understand that editor will ease that out till some extent. Maybe you want to look at some sample schemas available from http://www.starstandard.org/SIGXML/XMLSchemas
The schemas are really nested and way big.
However if you have decided not to implement it, I will look at my end what best can be done. May be in light of above two points, you might want to reconsider your decision.
I have also tried the behavior of putting datatype any in JCAM and have two observations a) After an unknown element is found and reported, it does not reports any subsequent extra elements. b) If datatype <any> is specified at say root level, it stops validating the tree underneath. Hence any cannot be used effectively in JCAM.</any>
Hi,
1) I agree that any is a useful and widely used if frustrating feature of XSD.
2) it is possible in CAM to use the ability of xpath to select a group of
elements for which the rule will be applied. For example in this case it would
be possible to use an xpath that selected all elements that have children. For
example "//*" selects all elements that are parent.
This allows you to set a rule once and have it apply to many nodes.
3) I think the above covers your need.
4) I am sorry but JCAM was implemented before any was really sorted and I am
not surprised to find it does work. Sorry.
Yes that's a great idea, "//" should work. I will try it today.
But just to let you know that we are getting performance issues with CAMV
component when we are having large XML files. This I believe will be due to
CAMV parsing each and every line of XML.
Here is the scenario:
We are receiving Orders with multiple line items. The line items can be
multiple and say this instance it is around 500 leading to a large XML.
There is only one validation which should be performed i.e. quantity must be
I believe CAMV is parsing each and every node in XML atleast for mandatory
check or it's additional existance.
As a result the performance of CAMV is directly proportionate to the size of
XML being validated and not on the number of checks to be performed. This does
not looks a very good design to me. To me the performance should be directly
proportionate to the number of checks to be performed along with the size of
XML.
If I have 500 line items, with 5 elements each, I believe currently it will
perform 7500 checks = 50053
Vs. if it performs only required checks and no implict checks:
500 checks = 500*1
I still believe it will be a huge saving for processing and should be able to
be achive as a configurable setting.
Hi again,
Can you via my normal email send me a copy of your full template as you have
it and also a real xml file that you want to validate.
Thanks
Martin
ok sorry for the delay. Sent the sample and CAM file to
martin.me.roberts@bt.com address. Let me know if you get it. Also is this the
right address?
Sorry I no longer work for BT so my email is martin.me.roberts at
googlemail.com