How To Do Fine-Grained Namespace Control

Help
dzerugral
2013-03-08
2013-05-22
  • dzerugral
    dzerugral
    2013-03-08

    I have read many posts and have read documentation concerning generating python modules from schemas that reference each other.

    Currently, I have many schemas that all have the same complex type "Setting". So, my code involves instantiating the "Setting" type unique to each schema (but to the human it is the same type just different namespace!).

    My work around is getting sloppy and I would like to have a central xml schema (setting.xsd) that only contains the "Setting" type. Then, I dont have to track down the particular schema who I am generating a Setting for - they should all reference my setting.xsd schema and the "Setting" type is completely encapsulated such that whoever it is a setting for doesn't matter.

    But, how to do this? Is it possible for someone to provide a sample/example? Another complication is that different projects will be referencing this setting.xsd schema so the relative file paths may not be a constant. Does that matter?

    In summary, I want my setting.xsd containing the complex type "Setting" to be referenced by any other xml schema located remotely away from setting.xsd.

    Just a simple example please! :)

     
  • Peter A. Bigot
    Peter A. Bigot
    2013-03-08

    Probably the best simple existing example is in the PyXB examples/manual directory; it corresponds to what's in the documentation. Look at demo3c.sh which builds a shared schema for an address complex type, and demo4.sh which references it from another schema. Start at http://pyxb.sourceforge.net/userref_pyxbgen.html#multi-document-schema ; the material on fine-grained namespace control is beyond what you should need.

    The relative file paths in the schema don't matter; once the binding is generated the namespace is all that matters.

     
  • dzerugral
    dzerugral
    2013-03-11

    I understand the examples, but they dont use multiple imports. For instance, I want to import a schema that has the "setting" type, instantiate a new Setting() and then append it to an element from another imported schema binding (for instance import order_schema).

    import set_schema, order_schema

    set = set_schema.Setting()
    order = order_schema.Order()
    order.setting = set

    So, although setting is a type that order_schema includes, I want to instantiate it from the set_schema binding (I dont want to instantiate setting from the order_schema binding).

    When I attempt the above code, pyxb correctly binds order_schema such that it includes the setting type from set_schema but it cannot import both bindings simultaneously since they have the same namespace and both contain definitions of the type Setting.

    I've tried using different namespaces, but I've never been able to assign an element with a type from a different binding.

     
  • Peter A. Bigot
    Peter A. Bigot
    2013-03-12

    In any program you are permitted to import only one PyXB module corresponding to any given namespace. That module may be generated using multiple schema, as demonstrated by the fine-grained namespace control. There could be another module that has the same namespace and different content, but you aren't allowed to import both into the same program.

    An element defined in one namespace cannot be substituted for an element in another namespace even if they are structurally equivalent and have the same local name (ignoring the namespace). From the perspective of XML schema, the namespace is part of the name. Thus, if there were a Setting declaration in both set_schema and order_schema, they would not be the same, and you cannot legally substitute the one from set_schema for the one in order_schema.

    If you want to share the schema text that defines a complexType or element Setting between different schema that describe different namespaces, without it being compatible between the namespaces, use xs:include instead of xs:import. The two definitions will not be compatible. I don't think this is what you want to do.

    If you want to share a single Setting binding and element/type between different namespaces, use xs:import. The bindings for the namespace holding Setting are distinct from the bindings for the namespaces that reference it. You should generate the bindings for both namespaces in the same pyxbgen invocation, or you should use the archive facility so that the dependent bindings can reference the previously translated bindings for the namespace that is being included. In your example of set_schema versus order_schema, the import would mean that the definition of Setting in order_schema is the one from set_schema: there is no Setting in order_schema. The order_schema binding in that case would import the set_schema to get access to the correct binding.

    In short, I'm not entirely clear on what you're trying to do, but I don't think you are permitted to do it within the bounds of XML schema unless you use distinct namespaces.

     
  • dzerugral
    dzerugral
    2013-03-12

    Yes, perhaps this isn't possible. Here is one last explanation so please confirm if it makes sense (thanks a lot for your help too!):

    Here's a simple dummy example of what I want to work (shown before):

    import set_schema, order_schema
    set = set_schema.Setting()
    order = order_schema.Order()
    order.setting = set
    

    Also, I did exactly this (as you say):

    The order_schema binding in that case would import the set_schema to get access to the >correct binding.

    So, my current work around is:

    do not import set_schema. Instead, import only order_schema. So, lets say we have order2_schema (a different schema that also contains type Setting).

    order_schema and order2_schema have different namespaces so that they may be imported without conflict. Then, I want to give one of them a new Setting. I have a dummy helper piece of code that instantiates a Setting for whichever order it may be:

    def make_me_a_setting_please(schema):
        set = schema.Setting()
        set.default_thing = 'the default'
        order = schema.Order()
        order.setting = set
    

    so, maybe now it makes sense? Rather than having to specifically reference the schema in which the setting comes from, I'd like to encapsulate the Setting type and freely instantiate it for any order (order_schema, order2_schema, etc.). For instance:

    def make_anyone_a_setting_please():
        set = set_schema.Setting()
        set.default_thing = 'the default'
        return set
    
     
  • Peter A. Bigot
    Peter A. Bigot
    2013-03-13

    I'm still not understanding what you're doing.

    I think you have three namespaces: one (nsA) that holds Setting, and two (nsB and nsC) that each import (not include) nsA and define distinct Order elements that each reference nsA:Setting. In that case, your dummy example should work fine. If it doesn't, there's something else going wrong.

    If instead you have a Setting declaration in a schema file without a namespace and that is included (not imported) into order_schema (nsB) and order2_schema (nsC) each of which reference the included Setting, then you have nsB:Setting and nsC:Setting which are different and you will have to create the instance using the appropriate binding.

    If you want to pursue this further, please create minimal schema showing the two Order declarations and the Setting declaration, along with a demonstration program that does not work, and attach all that to a posting on this topic that shows how you're generating the bindings, what you see with the demonstration program, and what you would like to see instead. Then maybe I can help.

     
  • dzerugral
    dzerugral
    2013-03-13

    Yes, sorry this is getting confusing.

    I attached the dummy example. Also, here is what I have tried:

    importing the set_schema.xsd (allows different namespaces)
    Including the set_schema.xsd (requires same namespace)

    If they have the same namespace then, from what I understand, I cannot import them into a python module simultaneously since they both contain a reference to type "Setting"

    If they have different namespaces, and if I import set_schema, then Pyxb produces an additional module "_pref1.py". That module also contains a reference to Setting and so I get the following error:

    pyxb.exceptions_.NamespaceUniquenessError: http://www.example.org/set: name Setting used for multiple values in typeBinding

    I attached all relevant files that produce this issue. "test.py" is the main file to run. Thank you very much for your help!

     
    Attachments
  • Peter A. Bigot
    Peter A. Bigot
    2013-03-13

    OK, there's lots of red herrings here.

    First, you didn't mention you're using PyXB 1.1.4. The current 1.1 release is 1.1.5, but you'd be best off moving to 1.2.1. That probably doesn't matter; I checked the results below on 1.1.5 and the current development head (based on 1.2.1).

    Second, everything you're doing is in the same namespace, so you should be creating only one binding file. Since order_schema.xsd includes set_schema.xsd, you shouldn't generate bindings from set_schema.xsd. All you need to do is:

    pyxbgen -m order_schema -u order_schema.xsd
    

    PyXB will detect the include, read set_schema.xsd, and add its declarations to the namespace for order_schema.

    You also are declaring multiple namespace identifiers for the same namespace. Somehow the way you're generating the bindings confuses pyxb into generating a _pref1 module that shouldn't exist. If you use the single command above, that module does not exist, but you should remove xmlns:pref and use xmlns:tns.

    Finally, you missed the warning:

    WARNING:pyxb.binding.generate:Element use {http://www.example.org/set}Setting.{http://www.example.org/set}value renamed to value_
    

    You can have an element or attribute named "value", but PyXB will rename it because it reserves the name "value" to extract the value of a simple type. When I run test.py, I get an exception:

    llc[23]$ python test.py
    Traceback (most recent call last):
      File "test.py", line 10, in <module>
        set.value = 'default'
      File "/opt/pyxb-1.1/pyxb/binding/basis.py", line 55, in __setattr__
        raise pyxb.BindingError('Attempt to set reserved name %s in instance of %s' % (name, type(self)))
    pyxb.exceptions_.BindingError: Attempt to set reserved name value in instance of <class 'order_schema.Setting'>
    

    When I correct these problems I can do:

    pyxbgen -m order_schema -u order_schema.xsd
    python test.py
    

    and get:

    <?xml version="1.0" ?><ns1:Order_Root xmlns:ns1="http://www.example.org/set"><ns1:setting name="dummy" ordinal="0"><ns1:value>default</ns1:value></ns1:setting></ns1:Order_Root>
    

    in a directory that contains only the files in the attached zip file. I removed all the generated files, and edited test.py to fix the problems above.

     
    Attachments
  • Peter A. Bigot
    Peter A. Bigot
    2013-03-13

    BTW: If you want to re-use set_schema.xsd by including it in multiple schemas from different namespaces, you'll have to use the chameleon schema pattern, which involves using an explicit namespace like xmlns:xs for the XML schema namespace, removing the targetnamespace attribute from set_schema.xsd, and making the target namespace the default namespace in the schema that include it. See http://www.xfront.com/ZeroOneOrManyNamespaces.html#design.

     
  • dzerugral
    dzerugral
    2013-03-13

    Unfortunately I think there is still a miscommunication. Thank you for your help, if you don't mind lets try this once more.

    I have two responses (a short one and a long one). They are listed below.

    In Short:

    Lets pretend the silly "value" and version red herrings didnt exist. Is it possible to make my "test.py" module work without changing a single line of its code (only change schemas and/or pyxb command line)?

    Full Explanation:

    As in the code I attached above, I want set_schema and order_schema in separate bindings. I completely understand why your code works (and this is already what I have been doing for sometime now).

    Think about this scenario: You have 20 types of orders - each with unique schemas for their specific application (for instance an order for restaurants, zoos, theme parks, etc.). They all have 1 thing in common: the complex type Setting.

    Now, lets say we have a base class that handles creating these Setting types. It would be very convenient if the base class creates a Setting without knowledge of the specific order schema it is meant for. In essence, I am trying to simplify my code so that each individual order_schema doesnt need its own unique piece of code just to instantiate this one object (Setting).

    A 2nd approach would be to allow this base class to store a copy of each schema. Then, depending on the order, it would use that particular schema to instantiate the setting... Though this is less code it is seemingly more sloppy and harder to read!

    Thus, in the code I gave you, it is essential that the Setting type be instantiated from the "set_schema".

    Is this possible? Does my crazy idea make sense?! I appreciate your help.

     
  • Peter A. Bigot
    Peter A. Bigot
    2013-03-13

    Of course it's possible. The shared type has to be in a different namespace, and so you have to use xs:import, as I tried to explain in previous postings.

    So here's set_schema.xsd:

    <?xml version="1.0" encoding="UTF-8"?>
    <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:tns="urn:set" targetNamespace="urn:set" elementFormDefault="qualified">
      <xs:attributeGroup name="ID_group">
        <xs:attribute name="name" type="xs:string" use="required"/>
        <xs:attribute name="ordinal" type="xs:int" use="required"/>
      </xs:attributeGroup>
      <xs:complexType name="Setting">
        <xs:sequence>
          <xs:element name="value" type="xs:string" maxOccurs="1" minOccurs="1"/>
        </xs:sequence>
        <xs:attributeGroup ref="tns:ID_group"/>
      </xs:complexType>
    </xs:schema>
    

    Here's order_schema.xsd:

    <?xml version="1.0" encoding="UTF-8"?>
    <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:tns="urn:o1" xmlns:s="urn:set" targetNamespace="urn:o1" elementFormDefault="qualified">
      <xs:import namespace="urn:set"/>
      <xs:element name="order_root" type="tns:Order_Root"/>
      <xs:complexType name="Order_Root">
        <xs:sequence>
          <xs:element name="setting" type="s:Setting" maxOccurs="1" minOccurs="1"/>
        </xs:sequence>
      </xs:complexType>
    </xs:schema>
    

    Here's test.py (only change from your example is value rewritten as value_;
    this one you can't avoid):

    import set_schema
    import order_schema
    
    # create a general setting (for any schema that needs a setting)
    set = set_schema.Setting()
    set.name = 'dummy'
    set.ordinal = 0
    set.value_ = 'default'
    
    # create an order.
    order_root = order_schema.Order_Root()
    # This order also wants a setting so assign it the one we just made
    order_root.setting = set
    
    # if the order can take the setting made from set_schema - Im happy!
    print order_root.toxml()
    

    Here's the commands:

    pyxbgen \
       -m set_schema -u set_schema.xsd \
       --archive-to-file set_schema.wxs
    pyxbgen \
       --archive-path=.:+ \
       -m order_schema -u order_schema.xsd
    python test.py
    

    Here's the output they give:

    WARNING:pyxb.binding.generate:Element use {urn:set}Setting.{urn:set}value renamed to value_
    Python for urn:set requires 1 modules
    Python for urn:o1 requires 1 modules
    <?xml version="1.0" ?><ns1:Order_Root xmlns:ns1="urn:o1" xmlns:ns2="urn:set"><ns1:setting name="dummy" ordinal="0"><ns2:value>default</ns2:value></ns1:setting></ns1:Order_Root>
    

    All exactly as in http://pyxb.sourceforge.net/userref_pyxbgen.html#sharing-namespace-bindings, which is part of the page I pointed you to in my initial response.

     
    Attachments
  • dzerugral
    dzerugral
    2013-03-15

    It worked perfectly thank you!

    Your demos are quite concise and make sense to me now. But, here is why I got stumped by them initially (though perhaps the demos in my version are out-dated):

    • demo4.py is the first demo where you import po3 and address bindings like in the shared namespace online doc.
      • But, in this example you don't use the po3 binding (so I didn't think it was relevant).
    • In demo4c.py however you use po4 and address bindings together (much like I was trying to do).
      • But, since the online tutorial for namespace sharing didn't involve po4 I
        disregarded this demo at the time.

    So, although that webpage had the answer I was looking for, things didn't quite line up and as a newbie it was sort of confusing.

    Also, one last thing. Can we change the title of this post to: "How to share namespace bindings"? I couldn't figure out how to change it and the current title is quite misleading!

    Thanks a ton! The quick responses on this forum are extremely helpful!

     
    Last edit: dzerugral 2013-03-15
  • Peter A. Bigot
    Peter A. Bigot
    2013-03-15

    You're welcome; glad it's straightened out.

    I don't see how to change the thread title either, so we'll have to live with it.

     
    • dzerugral
      dzerugral
      2013-05-22

      So, after several months of using this solution I started to wonder if I am using pyxb to its fullest potential.

      When I run your example code attached above, no matter what order_schema.py always contains the following line:

      1
      2
      #!/usr/bin/python
      import set_schema
      

      But, if set_schema.py is in a different directory than order_schema.py I have to go manually change that import statement (to something like import diffpath.set_schema as set_schema)

      So is there an automated way of generating the correct import statement? If not, its not a huge issue but it would be nice if everything worked "out of the box" so to speak (for when I'm rapidly making changes and not wanting to wait for such manual processes).

       
  • Peter A. Bigot
    Peter A. Bigot
    2013-05-22

    I believe that, when order_schema is generated, the import directive for set_schema that is generated is the path that was used when set_schema was generated. You could put set_schema into a package by using (for example) --module-prefix=diffpath when generating it. See: http://pyxb.sourceforge.net/userref_pyxbgen.html#generating-related-namespaces