Re: [Generateds-discuss] How to create/ generate new xml from generated API

[Dave K. <dku...@da...>]

Daniel,

What?  You don't seem like a Python newbie to me.

About the error -- I believe that exception message is telling us
that it wants a different type for the value of
`testResults01.Personnel.set_SystemOperator`.

Short version -- see attached file `test01.py`.

Long attempted explanation -- see the following:

In the XML schema (`TestResults.xsd`), search for "SystemOperator".
You will find the following:

     <xs:element name="SystemOperator" type="Person">
         <xs:annotation>
             <xs:documentation>Identification information for the  
operator of the ATE.</xs:documentation>
         </xs:annotation>
     </xs:element>

The `type="Person"` tells us it want an instance of class `Person`.
So, look at the definition of class `Person` in the module that you
generated with `generateDS.py` (`atdsApi.py`, I believe).

We have to construct a instance of class Person.  And, since class
`Person` contains an `Address`, we can construct an instance of
class `Address`.

I've attached a file (`test01.py`) with a modified version of your code.  And,
here is a snippet

     mailingaddress = lib.MailingAddress(
         Address1="123 Pretty Lane",
         City="Bigcity",
         State="Arizona",
         Country="USA",
         PostalCode="98765",
     )
     sysOp = lib.Person(
         name="Dan",
         Address=mailingaddress,
         email="da...@so...",
         phone="123.456.7890",
         # etc
         )
     testResults01.Personnel.set_SystemOperator(sysOp)
     print (testResults01.Personnel.get_SystemOperator())
     testResults01.export(sys.stdout, 0)

You can get clues about this if you look in the `build`,
`_buildAttributes`, and `_buildChildren` methods in classes
`PersonnelType`, `Person`, `Operator`, etc.  Notice that, for the
`SystemOperator` member, it creates and instance of class `Person`.
And, since class `Person` is a subclass of class `Operator`, you may
want to look at the implementation of class `Operator` in
`atdsApi.py`.

It's a bit confusing.  It is to me at least.  But, then, I suppose
XML and XML schema is supposed to be.  Sigh.

Hope this helps.

Dave



Quoting "Dewitte, Daniel W (US)" <dan...@ba...>:

> Hi Dave,
>
> Thank you for the pointers, yes this worked great for me.  I still  
> get a few errors when creating the API but I will worry about those  
> when I get to the external validator I have to use to ensure that  
> everything is working.
>
> I also wanted to thank you, I am new to Python with this being my  
> first major project.  I have played around with an RPi but nothing  
> major like up to this point.  So some of the syntax and ways that  
> Python uses is new to me and I am trying to learn.
>
> I do have some more questions at the moment and then I think I will  
> be more then off and running on this next phase of this project.
> 1)  You have been able to show me on how to create the initial  
> instance of the Root Element, is that correct?  As long as the line  
> "testResults01 = lib.TestResults()" calls the root element class.   
> Is that correct?
>
> 2)  In the def __init__ of that element, I see that the input  
> arguments are the attributes and child elements of that root  
> element.  See attached image.
> The first two in the init at the elements attributes, the rest are  
> child elements.  I can set the attributes by just replacing "None"  
> with their values (hardcoded or generated).  How to I generate the  
> child elements (i.e. "Personnel") so that I can start setting their  
> attributes and parameters?
>
> I have tried the following code:
> import sys
> import atdsApi as lib
>
> testResults01 = lib.TestResults(uuid = "1", securityClassification =  
> "Unclassified",
>                                 Personnel = lib.PersonnelType())
> print (testResults01.get_uuid())
> print (testResults01.get_securityClassification())
> testResults01.export(sys.stdout, 0)
>
> I get this result:
> 1
> Unclassified
> <c:TestResults  xmlns:None="http://www.ieee.org/ATML/2006/Common"   
> xmlns:c="http://www.ieee.org/ATML/2006/Common"  uuid="1"  
> securityClassification="Unclassified">
>     <c:Personnel/>
> </c:TestResults>
>
> When I try to then go and set the attributes of Personnel which is  
> controlled by PersonnelType() I get an error.
> Code:
> import sys
> import atdsApi as lib
>
> testResults01 = lib.TestResults(uuid = "1", securityClassification =  
> "Unclassified",
>                                 Personnel = lib.PersonnelType())
> print (testResults01.get_uuid())
> print (testResults01.get_securityClassification())
> testResults01.Personnel.set_SystemOperator("Dan")
> print (testResults01.Personnel.get_SystemOperator())
> testResults01.export(sys.stdout, 0)
>
> Error:
> 1
> Unclassified
> Dan
> <c:TestResults  xmlns:None="http://www.ieee.org/ATML/2006/Common"   
> xmlns:c="http://www.ieee.org/ATML/2006/Common"  uuid="1"  
> securityClassification="Unclassified">
>     <c:Personnel>
> Traceback (most recent call last):
>
>   File "H:\C4ISR S2 NPI\Projects\Voice of the Process\ATDS TDR  
> Importer - Python\ATDS File Creator\generateDS_xmlCreate.py", line  
> 16, in <module>
>     testResults01.export(sys.stdout, 0)
>
>   File "H:\C4ISR S2 NPI/Projects/Voice of the Process/ATDS TDR  
> Importer - Python/ATDS File Creator\atdsApi.py", line 1314, in export
>     self._exportChildren(outfile, level + 1, namespaceprefix_,  
> namespacedef_, name_='TestResults', pretty_print=pretty_print)
>
>   File "H:\C4ISR S2 NPI/Projects/Voice of the Process/ATDS TDR  
> Importer - Python/ATDS File Creator\atdsApi.py", line 1342, in  
> _exportChildren
>     self.Personnel.export(outfile, level, namespaceprefix_,  
> namespacedef_='', name_='Personnel', pretty_print=pretty_print)
>
>   File "H:\C4ISR S2 NPI/Projects/Voice of the Process/ATDS TDR  
> Importer - Python/ATDS File Creator\atdsApi.py", line 10284, in export
>     self._exportChildren(outfile, level + 1, namespaceprefix_,  
> namespacedef_, name_='PersonnelType', pretty_print=pretty_print)
>
>   File "H:\C4ISR S2 NPI/Projects/Voice of the Process/ATDS TDR  
> Importer - Python/ATDS File Creator\atdsApi.py", line 10304, in  
> _exportChildren
>     self.SystemOperator.export(outfile, level, namespaceprefix_,  
> namespacedef_='', name_='SystemOperator', pretty_print=pretty_print)
>
> AttributeError: 'str' object has no attribute 'export'
>
> By reading the error it looks like it is actually on the export  
> function and not the set function as it is setting the System  
> Operator.
> Any ideas?
>
> I am sure to have more questions soon.
> Thank you again,
>
> Daniel DeWitte
>
> T:  +1-603-885-2602
> M: +1-603-759-4476
> E: dan...@ba...
>
> -----Original Message-----
> From: Dave Kuhlman <dku...@da...>
> Sent: Thursday, June 2, 2022 3:56 PM
> To: Dewitte, Daniel W (US) <dan...@ba...>
> Cc: Gen...@li...
> Subject: Re: [Generateds-discuss] How to create/ generate new xml  
> from generated API
>
> External Email Alert
>
> This email has been sent from an account outside of the BAE Systems network.
>
> Please treat the email with caution, especially if you are requested  
> to click on a link, decrypt/open an attachment, or enable macros.   
> For further information on how to spot phishing, access  
> “Cybersecurity OneSpace Page” and report phishing by clicking the  
> button “Report Phishing” on the Outlook toolbar.
>
>
> Daniel,
>
> Thanks for providing that info, command line, generated module, etc.
>
> When you run `generateDS.py`, the "-o" option means generate a main  
> module, and the "-s" option says generate a subclass module.  The  
> "--super=xxxx" tells `generateDS.py` to generate an `import`  
> statement in the subclass module that imports module `atdsApi.py`.
> Since you have given the same name for "-o" and "-s" that module  
> imports itself.  And, maybe, the main module is over-written by the  
> subclass module.
>
> I don't know what your needs are, but it's likely that you do not  
> want to use either "-s" or "--super" command line options.  If you  
> do not need the subclass module, do not generate it.
>
> My example command line does not make this very clear because,  
> although it does use different names for "-o" and "-s", there is  
> only one letter difference.  Sorry for the confusion.
>
> Try running the following instead:
>
>      $ python generateDS.py -f -o atdsApi.py --member-specs=dict  
> --export="write etree validate" TestResults.xsd
>
> Then the following Python code should work:
>
>      import sys
>      import atdsApi as lib
>      testResults01 = lib.TestResult()
>      testResults01.export(sys.stdout, 0)
>
> Notice that I added "sys." in front of "stdout".
>
> Hope this helps.  Let me know if/when you have more questions.
>
> Dave
>
>
> Quoting "Dewitte, Daniel W (US)" <dan...@ba...>:
>
>> Hi Dave,
>>
>> Thank you for your quick response, so far the first part of what you
>> showed me is working, I do get some warnings though.  It looks like
>> because the TestResults.xsd names multiple namespace prefix's it is
>> not carrying the "c:" prefix.
>>
>> 1)  H:\Python Training\GenerateDS_Docs>python generateDS.py -f -o
>> atdsApi.py -s atdsApi.py --super=atdsApi --member-specs=dict
>> --export="write etree validate" TestResults.xsd
>> *** No suitable ns prefix for element "<XschemaElement name:
>> "Outcome" type: "Outcome">".
>> WARNING - H:\Python Training\GenerateDS_Docs\generateDS.py - _lookupNS
>> - Unknown namespace prefix "c" used.
>> WARNING - H:\Python Training\GenerateDS_Docs\generateDS.py - _lookupNS
>> - Unknown namespace prefix "c" used.
>> WARNING - H:\Python Training\GenerateDS_Docs\generateDS.py - _lookupNS
>> - Unknown namespace prefix "c" used.
>>
>> 2)  I created a new .py file and have the following:
>> # -*- coding: utf-8 -*-
>> """
>> Created on Wed May 25 11:02:45 2022
>>
>> @author: daniel.dewitte
>> """
>> import sys
>> import atdsApi as lib
>>
>> testResults01 = lib.TestResult()
>> testResults01.export(stdout, 0)
>>
>> When I try to run this .py I get the following errors:
>> runfile('H:/C4ISR S2 NPI/Projects/Voice of the Process/ATDS TDR
>> Importer - Python/ATDS File Creator/generateDS_xmlCreate.py',
>> wdir='H:/C4ISR S2 NPI/Projects/Voice of the Process/ATDS TDR Importer
>> - Python/ATDS File Creator') Traceback (most recent call last):
>>
>>   File "H:\C4ISR S2 NPI\Projects\Voice of the Process\ATDS TDR
>> Importer - Python\ATDS File Creator\generateDS_xmlCreate.py", line 8,
>> in <module>
>>     import atdsApi as lib
>>
>>   File "H:\C4ISR S2 NPI/Projects/Voice of the Process/ATDS TDR
>> Importer - Python/ATDS File Creator\atdsApi.py", line 68, in <module>
>>     class TestResultsSub(supermod.TestResults):
>>
>> AttributeError: partially initialized module 'atdsApi' has no
>> attribute 'TestResults' (most likely due to a circular import)
>>
>> Attached the api.py for reference.
>>
>> Thanks,
>> Daniel DeWitte
>>
>> -----Original Message-----
>> From: Dave Kuhlman <dku...@da...>
>> Sent: Wednesday, June 1, 2022 7:08 PM
>> To: Dewitte, Daniel W (US) <dan...@ba...>
>> Cc: Gen...@li...
>> Subject: Re: [Generateds-discuss] How to create/ generate new xml from
>> generated API
>>
>> External Email Alert
>>
>> This email has been sent from an account outside of the BAE Systems network.
>>
>> Please treat the email with caution, especially if you are requested
>> to click on a link, decrypt/open an attachment, or enable macros.
>> For further information on how to spot phishing, access “Cybersecurity
>> OneSpace Page” and report phishing by clicking the button “Report
>> Phishing” on the Outlook toolbar.
>>
>>
>> Daniel,
>>
>> Thanks for taking the time to report this.
>>
>> Here is what I've tried so far:
>>
>> 1. I copied TestResults.xml and Common.xsd to a directory.
>>
>> 2. I ran the following command:
>>
>>         ./generateDS.py -f -o tmp01sup.py -s tmp01sub.py --super
>> tmp01sup --member-specs=dict '--export=write etree validate'
>> TestResults.xsd
>>
>>     That generated a module `tmp01sup.py`.
>>
>> 3. So, I started IPython (the plain Python interactive prompt would
>>     also work) and did this:
>>
>>         [ins] In [1]: import tmp01sup as lib
>>
>>     Which also worked.
>>
>> 4. Then I created an instance of a class in the generated module:
>>
>>         [ins] In [5]: testresults01 = lib.TestResult()
>>         [ins] In [6]: testresults01.export(sys.stdout, 0)
>>         <c:TestResult
>> xmlns:None="http://www.ieee.org/ATML/2006/Common"
>> xmlns:c="http://www.ieee.org/ATML/2006/Common" />
>>
>>     Which also seemed to work.
>>
>> So, now I've got to read your message a bit more carefully so that I
>> can figure out what is wrong for your case and how to fix it.
>>
>> I do not have the file `collect_schema_locations.py`, which you show
>> in your message as generating an exception.  So, I can't test that.
>> Did you generate that file with `generateDS.py`?
>>
>> However the exception "TypeError: empty namespace prefix is not
>> supported in XPath" seems to occur when you have a key in the nsmap
>> that is `None`.  Try removing it.  Then do the `xpath` search:
>>
>>      [ins] In [23]: nsmap = root.nsmap
>>      [ins] In [24]: nsmap
>>      Out[24]:
>>      {'xs': 'http://www.w3.org/2001/XMLSchema',
>>       'c': 'http://www.ieee.org/ATML/2006/Common',
>>       None: 'http://www.ieee.org/ATML/2007/TestResults'}
>>      [ins] In [25]: nsmap.pop(None)
>>      Out[25]: 'http://www.ieee.org/ATML/2007/TestResults'
>>      [ins] In [26]: nsmap
>>      Out[26]:
>>      {'xs': 'http://www.w3.org/2001/XMLSchema',
>>       'c': 'http://www.ieee.org/ATML/2006/Common'}
>>      [ins] In [27]: root.xpath('xs:import', namespaces=nsmap)
>>      Out[27]: [<Element {http://www.w3.org/2001/XMLSchema}import at
>> 0x7faf573acac0>]
>>
>> Does that help with your work?
>>
>> By the way, the "--one-file-per-xsd" capability is something that I
>> do not understand, myself.  That code was contributed by someone else.
>>
>> Dave
>>
>>
>> Quoting "Dewitte, Daniel W (US) via Generateds-discuss"
>> <gen...@li...>:
>>
>>> Here are the actual files this time :)
>>>
>>> Daniel DeWitte
>>>
>>> From: Dewitte, Daniel W (US)
>>> Sent: Wednesday, June 1, 2022 1:06 PM
>>> To: 'Gen...@li...'
>>> <Gen...@li...>
>>> Subject: How to create/ generate new xml from generated API
>>>
>>> Hi Dave,
>>>
>>> This is one of the better tools that I have found and I think the only
>>> one that can work the way I need it too, I just need some help with
>>> getting to the finish line.
>>>
>>> My goal is generate a new .xml using a known schema .xsd with Python
>>> as the base language (required).  I have 13 different reports that all
>>> have a different format that are generated on a constant basis that I
>>> need to import into a known .xsd format for upload to a database.
>>>
>>> I believe I either want to use the build methods, or the "one-per"
>>> options, but I can't get either of them to work.  I asked about the
>>> build method problem on stackover flow here:  Stackoverflow - Creating
>>> a new XML from an XSD using GenerateDS and
>>> Python<https://stackoverflow.com/questions/72397436/creating-a-new-xml
>>> -from-an-xsd-using-generateds-and-python>
>>>
>>> But I am not sure that the build method is what I want to use.
>>>
>>> When I try the "one-per" I get the following errors:
>>> Command line method, I believe this error is due to that the .xsd is
>>> in a UTF-8 encoding.  How do I get this to work with UTF-8 encoding?
>>>> (base) H:\Python Training\GenerateDS_Docs>python generateds.py
>>>> --one-file-per-xsd --output-directory="OnePer"
>>>> --module-suffix="One" TestResults.xsd Traceback (most recent call
>>>> last):
>>>> File "H:\Python Training\GenerateDS_Docs\generateds.py", line 9906,
>>>> in <module>
>>>     main()
>>>> File "H:\Python Training\GenerateDS_Docs\generateds.py", line 9872,
>>>> in main
>>>     parseAndGenerate(
>>>> File "H:\Python Training\GenerateDS_Docs\generateds.py", line 9318,
>>>> in parseAndGenerate
>>>     parser.parse(rootFile)
>>>> File "C:\EngTools\Anaconda3\lib\xml\sax\expatreader.py", line 111, in
>>>> parse
>>>     xmlreader.IncrementalParser.parse(self, source)
>>>> File "C:\EngTools\Anaconda3\lib\xml\sax\xmlreader.py", line 123, in
>>>> parse
>>>     buffer = file.read(self._bufsize)
>>>> File "C:\EngTools\Anaconda3\lib\encodings\cp1252.py", line 23, in
>>>> decode
>>>     return codecs.charmap_decode(input,self.errors,decoding_table)[0]
>>>> UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in
>>>> position 4013: character maps to <undefined>
>>>
>>> Approach 2 - Extraction and Generation Utilities:
>>>> (base) h:\Python Training\GenerateDS_Docs\utils>python
>>>> collect_schema_locations.py --force --verbose TestResults.xsd
>>>> testJson.txt Traceback (most recent call last):
>>>> File "h:\Python
>>>> Training\GenerateDS_Docs\utils\collect_schema_locations.py", line
>>>> 157, in <module>
>>>     main()
>>>> File "h:\Python
>>>> Training\GenerateDS_Docs\utils\collect_schema_locations.py", line
>>>> 148, in main
>>>     extract_and_generate(infile, outfile, extract_locations_fn,
>>> options)
>>>> File "h:\Python
>>>> Training\GenerateDS_Docs\utils\collect_schema_locations.py", line 86,
>>>> in extract_and_generate
>>>     locations = extract_locations(infile, options)
>>>> File "h:\Python
>>>> Training\GenerateDS_Docs\utils\collect_schema_locations.py", line 50,
>>>> in extract_locations
>>>     elements = root.xpath('xs:import', namespaces=root.nsmap)
>>>> File "src/lxml/etree.pyx", line 1579, in lxml.etree._Element.xpath
>>>> File "src/lxml/xpath.pxi", line 259, in
>>>> lxml.etree.XPathElementEvaluator.__init__
>>>> File "src/lxml/xpath.pxi", line 131, in
>>>> lxml.etree._XPathEvaluatorBase.__init__
>>>> File "src/lxml/xpath.pxi", line 55, in
>>>> lxml.etree._XPathContext.__init__ File "src/lxml/extensions.pxi",
>>>> line 81, in lxml.etree._BaseContext.__init__
>>>> TypeError: empty namespace prefix is not supported in XPath
>>>
>>> Attached are the .xsd that I am using.  TestResults.xsd references
>>> Common.xsd.
>>>
>>> Any help on getting started with my overall goal will be much
>>> appreciated.  I am looking to create a new XML from a known .xsd and
>>> add in the elements and child elements and attributes as needed.
>>>
>>> Thank you for the help,
>>> Daniel DeWitte
>>
>>
>>
>> --
>> Dave Kuhlman
>> http://www.davekuhlman.org
>
>
>
> --
> Dave Kuhlman
> http://www.davekuhlman.org



-- 
Dave Kuhlman
http://www.davekuhlman.org