Generating XML from given schema, ComplexType problems.

[Ema Modzel]

Hello,

I've been checking out PyXB (1.2.4) for a project I'm working on
involving the payment's schemas (orignal_xsd).
I have a feeling that PyXB will be a huge help, but I'm having hard time understanding of how to work with ComplexTypes within the PyXB construct.

For the purpose of this topic - and my tests, i've created simplified schema (on the base of mentioned above) as weall as created sample github project to show what's goin on.

sample_schema: https://github.com/ryku123/xsd_cutted_pyxb/blob/master/xsd/pain_cutted.xsd
sample_project: https://github.com/ryku123/xsd_cutted_pyxb
xsd_overview_image

My main objective is to:

1. Map Schema to Python objects (easy, pyxbgen does the job perfectly)
2. Set/Modify values of different tag's (having in mind schema restrictions) and apply some custom logic
3. Generate XML file(s) with all the content that has been made before

Then repeat this proces over and over to feed new data into tags.

First step i am trying to accomplish is to fulfill given schema with some simple values (that meet schema restrictions).

Q1: how to do it properlly? I know that there will be a need of use this magic BIND() method, but after many tries i find myself too nob to get this working.
Q2: Regarding generated module from pyxbgen: there is always a path (raw string) given to a location of base XSD file (_XSDLocation)
Is this possible to make it somehow unique, so i could use this generated module on different machines?
My temporary workaround is that i substituted this string with a variable that i adjust according to machine/OS i am working on

Any hepl much appreciated,

Ryku

---

I'll place a bit of code here in case github issues:

from __future__ import print_function
from pyxb import BIND
from xsd import pain_cutted as pain
from datetime import datetime
import xml.dom.minidom
import pyxb.utils.domutils

grpHdr = pain.GroupHeader32_CH()
grpHdr.MsgId = "MSG-01"
grpHdr.NbOfTxs = "123456789012345"
grpHdr.CreDtTm = datetime.strptime("2010-02-15T07:30:00", '%Y-%m-%dT%H:%M:%S')
grpHdr.CtrlSum = float(15850.00)  # optional

# create complex InitgPty
initgpty = pain.PartyIdentification32_CH_NameAndId()
# create complex CtctDtls
cntdts = pain.ContactDetails2_CH()
cntdts.Nm = "Contact Name"   # set simple
cntdts.Othr = "Other Value"  # set simple

initgpty.Nm = "Some name for tag Nm"  # add simple
initgpty.CtctDtls = cntdts            # add complex CtctDtls

grpHdr.InitgPty = initgpty            # add complex InitgPty

# print(grpHdr.toxml("utf-8", element_name='GrpHdr'))  # works! generates namespace as "ns1" a bit ugly..

# create complex PmtInf
pmtinf = pain.PaymentInstructionInformation3_CH()
pmtinf.PmtInfId="PayID"
pmtinf.PmtMtd="PayMethod"
pmtinf.ReqdExctnDt = datetime.strptime("2016-03-23", '%Y-%m-%d')
pmtinf.Dbtr="PayDbtr"
pmtinf.DbtrAcct="PayAccount"
pmtinf.DbtrAgt="PayDbtrAgent"
pmtinf.CdtTrfTxInf="PayCdtTrfTxInfo"

# create complex CstmrCdtTrfInitn
cstmrCdT = pain.CustomerCreditTransferInitiationV03_CH()

# here comes DIFFICULT PART

# cstmrCdT.append(BIND(grpHdr))
# cstmrCdT.append(BIND(pmtinf))
# cstmrCdT.GrpHdr = grpHdr    # add complex GrpHdr
# cstmrCdT.PmtInf = pmtinf    # add complex PmtInf
#
# print(cstmrCdT.toxml("utf-8", element_name='CstmrCdtTrfInitn'))  # UnrecognizedContentError: Invalid content

# here comes more DIFFICULT PART

# create complex Document (main tag for output XML)
doc = pain.Document_()
# doc.CstmrCdtTrfInitn = cstmrCdT  # add GrpHdr and PmtInf
doc.append(cstmrCdT)

# wish to do
# doc.toxml('utf-8')

# fancy printing

# dom = xml.dom.minidom.parseString(doc.toxml("utf-8", element_name='GroupHeader32-CH'))
# dom = xml.dom.minidom.parseString(doc.toxml("utf-8", element_name='CstmrCdtTrfInitn'))
# print(dom.toprettyxml())

[Peter A. Bigot]

For Q1 you usually only need to use BIND in cases where there's an inner element that doesn't have a named type. The schema you're using is normalized, so that isn't necessary, but you can use it.

After doing:

pyxbgen -u xsd/pain.xsd -m pain

I used this program, taken mostly from your code:

import pyxb
import pain
from datetime import datetime

grpHdr = pain.GroupHeader32_CH()
grpHdr.MsgId = "MSG-01"
grpHdr.NbOfTxs = "123456789012345"
grpHdr.CreDtTm = datetime.strptime("2010-02-15T07:30:00", '%Y-%m-%dT%H:%M:%S')
grpHdr.CtrlSum = float(15850.00)  # optional

pmtinf = pain.PaymentInstructionInformation3_CH()
pmtinf.PmtInfId="PayID"
pmtinf.PmtMtd="PayMethod"
pmtinf.ReqdExctnDt = datetime.strptime("2016-03-23", '%Y-%m-%d')
pmtinf.Dbtr="PayDbtr"
pmtinf.DbtrAcct="PayAccount"
pmtinf.DbtrAgt="PayDbtrAgent"
pmtinf.CdtTrfTxInf="PayCdtTrfTxInfo"

cstmrCdT = pain.CustomerCreditTransferInitiationV03_CH()
cstmrCdT.GrpHdr = grpHdr
# PmtInf is a multi-valued element so it's a list: append the new value
cstmrCdT.PmtInf.append(pmtinf)

doc = pain.Document()
doc.CstmrCdtTrfInitn = cstmrCdT

# Missing piece:
#grpHdr.InitgPty = pain.PartyIdentification32_CH_NameAndId('name',pain.Party6Choice_CH(pain.OrganisationIdentification4_CH('org')))
# or
#grpHdr.InitgPty = pyxb.BIND('name', pyxb.BIND());
#grpHdr.InitgPty.Id = pyxb.BIND();
#grpHdr.InitgPty.Id.OrgId = 'org';

try:
    doc.validateBinding()
    print doc.toDOM().toprettyxml()
except pyxb.ValidationError as e:
    print(e.details())

This produces:

The containing element {http://www.six-interbank-clearing.com/de/pain.001.001.03.ch.02.xsd}GrpHdr is defined at pain_cutted.xsd[11:3].
The containing element type {http://www.six-interbank-clearing.com/de/pain.001.001.03.ch.02.xsd}GroupHeader32-CH is defined at pain_cutted.xsd[36:1]
The {http://www.six-interbank-clearing.com/de/pain.001.001.03.ch.02.xsd}GroupHeader32-CH automaton is not in an accepting state.
The last accepted content was {http://www.six-interbank-clearing.com/de/pain.001.001.03.ch.02.xsd}CtrlSum
The following element and wildcard content would be accepted:
    An element {http://www.six-interbank-clearing.com/de/pain.001.001.03.ch.02.xsd}InitgPty per pain_cutted.xsd[42:3]
No content remains unconsumed

This relates to Q2: _XSDLocation is informative only; nothing in PyXB relies on its value. It's there only to assist with diagnostics, to tell somebody where the schema type definition can be found when the content supplied for it is incorrect. The error above tells you what's missing, but you have to look at the schema to understand what to provide.

By uncommenting one of the two alternative sequences after Missing Piece you will get a valid document. The first makes what you're constructing clear; if those types weren't named at the schema top level you'd have to use the second, where PyXB would guess what you mean by looking at what the schema allows for the member elements.

BTW: In that document you'll also see that the assignment to pmtinf.CdtTrfTxInf is incorrect because the value of that contained element is a list, just as is cstmrCdT.PmtInf (for which I corrected the assignment copied from your code).

The following shows another way to construct the same value, one with positional arguments (placed into the right element based on the allowed content) and one with keywords (helpful where you need to skip over elements that would accept the value intended for a subsequent element):

g = pain.GroupHeader32_CH('MSG-01',
                          datetime.strptime("2010-02-15T07:30:00", '%Y-%m-%dT%H:%M:%S'),
                          '123456789012345',
                          15850.0,
                          pain.PartyIdentification32_CH_NameAndId('name',
                                                                  pain.Party6Choice_CH(pain.OrganisationIdentification4_CH('org'))))
p = pain.PaymentInstructionInformation3_CH(PmtInfId='PayID',
                                           PmtMtd='PayMethod',
                                           ReqdExctnDt=datetime.strptime("2016-03-23", '%Y-%m-%d'),
                                           Dbtr='PayDbtr',
                                           DbtrAcct='Payaccount',
                                           DbtrAgt='PayDbtrAgent',
                                           CdtTrfTxInf = ['PayCdtTrfTxInfo'])

doc = pain.Document(pain.CustomerCreditTransferInitiationV03_CH(g, p));

[Ema Modzel]

[Ema Modzel]

update: I've managed to get rid of ns1 thign adding:

pyxb.utils.domutils.BindingDOMSupport.SetDefaultNamespace("http://www.six-interbank-clearing.com/de/pain.001.001.03.ch.02.xsd")

Now what i am trying to do is to only extend the content of (root) <Document> tag with this two elements:

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.six-interbank-clearing.com/de/pain.001.001.03.ch.02.xsd pain.001.001.03.ch.02.xsd"

to receive:

<?xml version="1.0" encoding="utf-8"?>
<Document 
xmlns="http://www.six-interbank-clearing.com/de/pain.001.001.03.ch.02.xsd" 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
xsi:schemaLocation="http://www.six-interbank-clearing.com/de/pain.001.001.03.ch.02.xsd  pain.001.001.03.ch.02.xsd">
    <CstmrCdtTrfInitn>
        <GrpHdr>
            <MsgId>MSG-01</MsgId>
            <CreDtTm>2010-02-15T07:30:00</CreDtTm>
            <NbOfTxs>123456789012345</NbOfTxs>

Thnx!

[Peter A. Bigot]

The simplest approach would be to add the additional attributes to the DOM object directly using the standard Python xml.dom API before converting it to XML text.

[Ema Modzel]

Hello Peter,

I am still fighting with this namespace issue, but thanks to Your advice i think i am on a good way.
Meanwhile i've been searching through this forum and I see that this is quite common issue.
Mostly it's recommended to use this xml.dom API or to look through source code tests/examples (however i have not found solution yet).

Anyway i have an idea why this causes issues especially for novice users using (as most) PyCharm.
The thing is that PyCharm does not suggest methods on object created using *.toDom() method.

Here is short example:

...
import pyxb.utils.domutils
import xml.dom.minidom

doc = pain.Document()

xb_obj = doc.toDOM()  # xb_obj. <- does not suggest methods
domobj = xml.dom.minidom.Document() # domobj. <- suggest methods

print "xb_obj   ", xb_obj.__class__, type(xb_obj)  # these are
print "domobj", domobj.__class__, type(domobj)     # the same 

I dont know if its PyCharm 'feature' or what but this may lead to confusions.

Thanks.

[Peter A. Bigot]

Interesting; I've never heard of PyCharm and have no idea how it guesses what methods might be available.