Re: [Generateds-discuss] How to create/ generate new xml from generated API
Brought to you by:
dkuhlman
Menu
▾
▴
Re: [Generateds-discuss] How to create/ generate new xml from generated API
|
From: Dewitte, D. W (US) <dan...@ba...> - 2022-06-03 20:39:25
|
Hi Dave,
Thank you for the pointers, yes this worked great for me. I still get a few errors when creating the API but I will worry about those when I get to the external validator I have to use to ensure that everything is working.
I also wanted to thank you, I am new to Python with this being my first major project. I have played around with an RPi but nothing major like up to this point. So some of the syntax and ways that Python uses is new to me and I am trying to learn.
I do have some more questions at the moment and then I think I will be more then off and running on this next phase of this project.
1) You have been able to show me on how to create the initial instance of the Root Element, is that correct? As long as the line "testResults01 = lib.TestResults()" calls the root element class. Is that correct?
2) In the def __init__ of that element, I see that the input arguments are the attributes and child elements of that root element. See attached image.
The first two in the init at the elements attributes, the rest are child elements. I can set the attributes by just replacing "None" with their values (hardcoded or generated). How to I generate the child elements (i.e. "Personnel") so that I can start setting their attributes and parameters?
I have tried the following code:
import sys
import atdsApi as lib
testResults01 = lib.TestResults(uuid = "1", securityClassification = "Unclassified",
Personnel = lib.PersonnelType())
print (testResults01.get_uuid())
print (testResults01.get_securityClassification())
testResults01.export(sys.stdout, 0)
I get this result:
1
Unclassified
<c:TestResults xmlns:None="http://www.ieee.org/ATML/2006/Common" xmlns:c="http://www.ieee.org/ATML/2006/Common" uuid="1" securityClassification="Unclassified">
<c:Personnel/>
</c:TestResults>
When I try to then go and set the attributes of Personnel which is controlled by PersonnelType() I get an error.
Code:
import sys
import atdsApi as lib
testResults01 = lib.TestResults(uuid = "1", securityClassification = "Unclassified",
Personnel = lib.PersonnelType())
print (testResults01.get_uuid())
print (testResults01.get_securityClassification())
testResults01.Personnel.set_SystemOperator("Dan")
print (testResults01.Personnel.get_SystemOperator())
testResults01.export(sys.stdout, 0)
Error:
1
Unclassified
Dan
<c:TestResults xmlns:None="http://www.ieee.org/ATML/2006/Common" xmlns:c="http://www.ieee.org/ATML/2006/Common" uuid="1" securityClassification="Unclassified">
<c:Personnel>
Traceback (most recent call last):
File "H:\C4ISR S2 NPI\Projects\Voice of the Process\ATDS TDR Importer - Python\ATDS File Creator\generateDS_xmlCreate.py", line 16, in <module>
testResults01.export(sys.stdout, 0)
File "H:\C4ISR S2 NPI/Projects/Voice of the Process/ATDS TDR Importer - Python/ATDS File Creator\atdsApi.py", line 1314, in export
self._exportChildren(outfile, level + 1, namespaceprefix_, namespacedef_, name_='TestResults', pretty_print=pretty_print)
File "H:\C4ISR S2 NPI/Projects/Voice of the Process/ATDS TDR Importer - Python/ATDS File Creator\atdsApi.py", line 1342, in _exportChildren
self.Personnel.export(outfile, level, namespaceprefix_, namespacedef_='', name_='Personnel', pretty_print=pretty_print)
File "H:\C4ISR S2 NPI/Projects/Voice of the Process/ATDS TDR Importer - Python/ATDS File Creator\atdsApi.py", line 10284, in export
self._exportChildren(outfile, level + 1, namespaceprefix_, namespacedef_, name_='PersonnelType', pretty_print=pretty_print)
File "H:\C4ISR S2 NPI/Projects/Voice of the Process/ATDS TDR Importer - Python/ATDS File Creator\atdsApi.py", line 10304, in _exportChildren
self.SystemOperator.export(outfile, level, namespaceprefix_, namespacedef_='', name_='SystemOperator', pretty_print=pretty_print)
AttributeError: 'str' object has no attribute 'export'
By reading the error it looks like it is actually on the export function and not the set function as it is setting the System Operator.
Any ideas?
I am sure to have more questions soon.
Thank you again,
Daniel DeWitte
T: +1-603-885-2602
M: +1-603-759-4476
E: dan...@ba...
-----Original Message-----
From: Dave Kuhlman <dku...@da...>
Sent: Thursday, June 2, 2022 3:56 PM
To: Dewitte, Daniel W (US) <dan...@ba...>
Cc: Gen...@li...
Subject: Re: [Generateds-discuss] How to create/ generate new xml from generated API
External Email Alert
This email has been sent from an account outside of the BAE Systems network.
Please treat the email with caution, especially if you are requested to click on a link, decrypt/open an attachment, or enable macros. For further information on how to spot phishing, access “Cybersecurity OneSpace Page” and report phishing by clicking the button “Report Phishing” on the Outlook toolbar.
Daniel,
Thanks for providing that info, command line, generated module, etc.
When you run `generateDS.py`, the "-o" option means generate a main module, and the "-s" option says generate a subclass module. The "--super=xxxx" tells `generateDS.py` to generate an `import` statement in the subclass module that imports module `atdsApi.py`.
Since you have given the same name for "-o" and "-s" that module imports itself. And, maybe, the main module is over-written by the subclass module.
I don't know what your needs are, but it's likely that you do not want to use either "-s" or "--super" command line options. If you do not need the subclass module, do not generate it.
My example command line does not make this very clear because, although it does use different names for "-o" and "-s", there is only one letter difference. Sorry for the confusion.
Try running the following instead:
$ python generateDS.py -f -o atdsApi.py --member-specs=dict --export="write etree validate" TestResults.xsd
Then the following Python code should work:
import sys
import atdsApi as lib
testResults01 = lib.TestResult()
testResults01.export(sys.stdout, 0)
Notice that I added "sys." in front of "stdout".
Hope this helps. Let me know if/when you have more questions.
Dave
Quoting "Dewitte, Daniel W (US)" <dan...@ba...>:
> Hi Dave,
>
> Thank you for your quick response, so far the first part of what you
> showed me is working, I do get some warnings though. It looks like
> because the TestResults.xsd names multiple namespace prefix's it is
> not carrying the "c:" prefix.
>
> 1) H:\Python Training\GenerateDS_Docs>python generateDS.py -f -o
> atdsApi.py -s atdsApi.py --super=atdsApi --member-specs=dict
> --export="write etree validate" TestResults.xsd
> *** No suitable ns prefix for element "<XschemaElement name:
> "Outcome" type: "Outcome">".
> WARNING - H:\Python Training\GenerateDS_Docs\generateDS.py - _lookupNS
> - Unknown namespace prefix "c" used.
> WARNING - H:\Python Training\GenerateDS_Docs\generateDS.py - _lookupNS
> - Unknown namespace prefix "c" used.
> WARNING - H:\Python Training\GenerateDS_Docs\generateDS.py - _lookupNS
> - Unknown namespace prefix "c" used.
>
> 2) I created a new .py file and have the following:
> # -*- coding: utf-8 -*-
> """
> Created on Wed May 25 11:02:45 2022
>
> @author: daniel.dewitte
> """
> import sys
> import atdsApi as lib
>
> testResults01 = lib.TestResult()
> testResults01.export(stdout, 0)
>
> When I try to run this .py I get the following errors:
> runfile('H:/C4ISR S2 NPI/Projects/Voice of the Process/ATDS TDR
> Importer - Python/ATDS File Creator/generateDS_xmlCreate.py',
> wdir='H:/C4ISR S2 NPI/Projects/Voice of the Process/ATDS TDR Importer
> - Python/ATDS File Creator') Traceback (most recent call last):
>
> File "H:\C4ISR S2 NPI\Projects\Voice of the Process\ATDS TDR
> Importer - Python\ATDS File Creator\generateDS_xmlCreate.py", line 8,
> in <module>
> import atdsApi as lib
>
> File "H:\C4ISR S2 NPI/Projects/Voice of the Process/ATDS TDR
> Importer - Python/ATDS File Creator\atdsApi.py", line 68, in <module>
> class TestResultsSub(supermod.TestResults):
>
> AttributeError: partially initialized module 'atdsApi' has no
> attribute 'TestResults' (most likely due to a circular import)
>
> Attached the api.py for reference.
>
> Thanks,
> Daniel DeWitte
>
> -----Original Message-----
> From: Dave Kuhlman <dku...@da...>
> Sent: Wednesday, June 1, 2022 7:08 PM
> To: Dewitte, Daniel W (US) <dan...@ba...>
> Cc: Gen...@li...
> Subject: Re: [Generateds-discuss] How to create/ generate new xml from
> generated API
>
> External Email Alert
>
> This email has been sent from an account outside of the BAE Systems network.
>
> Please treat the email with caution, especially if you are requested
> to click on a link, decrypt/open an attachment, or enable macros.
> For further information on how to spot phishing, access “Cybersecurity
> OneSpace Page” and report phishing by clicking the button “Report
> Phishing” on the Outlook toolbar.
>
>
> Daniel,
>
> Thanks for taking the time to report this.
>
> Here is what I've tried so far:
>
> 1. I copied TestResults.xml and Common.xsd to a directory.
>
> 2. I ran the following command:
>
> ./generateDS.py -f -o tmp01sup.py -s tmp01sub.py --super
> tmp01sup --member-specs=dict '--export=write etree validate'
> TestResults.xsd
>
> That generated a module `tmp01sup.py`.
>
> 3. So, I started IPython (the plain Python interactive prompt would
> also work) and did this:
>
> [ins] In [1]: import tmp01sup as lib
>
> Which also worked.
>
> 4. Then I created an instance of a class in the generated module:
>
> [ins] In [5]: testresults01 = lib.TestResult()
> [ins] In [6]: testresults01.export(sys.stdout, 0)
> <c:TestResult
> xmlns:None="http://www.ieee.org/ATML/2006/Common"
> xmlns:c="http://www.ieee.org/ATML/2006/Common" />
>
> Which also seemed to work.
>
> So, now I've got to read your message a bit more carefully so that I
> can figure out what is wrong for your case and how to fix it.
>
> I do not have the file `collect_schema_locations.py`, which you show
> in your message as generating an exception. So, I can't test that.
> Did you generate that file with `generateDS.py`?
>
> However the exception "TypeError: empty namespace prefix is not
> supported in XPath" seems to occur when you have a key in the nsmap
> that is `None`. Try removing it. Then do the `xpath` search:
>
> [ins] In [23]: nsmap = root.nsmap
> [ins] In [24]: nsmap
> Out[24]:
> {'xs': 'http://www.w3.org/2001/XMLSchema',
> 'c': 'http://www.ieee.org/ATML/2006/Common',
> None: 'http://www.ieee.org/ATML/2007/TestResults'}
> [ins] In [25]: nsmap.pop(None)
> Out[25]: 'http://www.ieee.org/ATML/2007/TestResults'
> [ins] In [26]: nsmap
> Out[26]:
> {'xs': 'http://www.w3.org/2001/XMLSchema',
> 'c': 'http://www.ieee.org/ATML/2006/Common'}
> [ins] In [27]: root.xpath('xs:import', namespaces=nsmap)
> Out[27]: [<Element {http://www.w3.org/2001/XMLSchema}import at
> 0x7faf573acac0>]
>
> Does that help with your work?
>
> By the way, the "--one-file-per-xsd" capability is something that I
> do not understand, myself. That code was contributed by someone else.
>
> Dave
>
>
> Quoting "Dewitte, Daniel W (US) via Generateds-discuss"
> <gen...@li...>:
>
>> Here are the actual files this time :)
>>
>> Daniel DeWitte
>>
>> From: Dewitte, Daniel W (US)
>> Sent: Wednesday, June 1, 2022 1:06 PM
>> To: 'Gen...@li...'
>> <Gen...@li...>
>> Subject: How to create/ generate new xml from generated API
>>
>> Hi Dave,
>>
>> This is one of the better tools that I have found and I think the only
>> one that can work the way I need it too, I just need some help with
>> getting to the finish line.
>>
>> My goal is generate a new .xml using a known schema .xsd with Python
>> as the base language (required). I have 13 different reports that all
>> have a different format that are generated on a constant basis that I
>> need to import into a known .xsd format for upload to a database.
>>
>> I believe I either want to use the build methods, or the "one-per"
>> options, but I can't get either of them to work. I asked about the
>> build method problem on stackover flow here: Stackoverflow - Creating
>> a new XML from an XSD using GenerateDS and
>> Python<https://stackoverflow.com/questions/72397436/creating-a-new-xml
>> -from-an-xsd-using-generateds-and-python>
>>
>> But I am not sure that the build method is what I want to use.
>>
>> When I try the "one-per" I get the following errors:
>> Command line method, I believe this error is due to that the .xsd is
>> in a UTF-8 encoding. How do I get this to work with UTF-8 encoding?
>>> (base) H:\Python Training\GenerateDS_Docs>python generateds.py
>>> --one-file-per-xsd --output-directory="OnePer"
>>> --module-suffix="One" TestResults.xsd Traceback (most recent call
>>> last):
>>> File "H:\Python Training\GenerateDS_Docs\generateds.py", line 9906,
>>> in <module>
>> main()
>>> File "H:\Python Training\GenerateDS_Docs\generateds.py", line 9872,
>>> in main
>> parseAndGenerate(
>>> File "H:\Python Training\GenerateDS_Docs\generateds.py", line 9318,
>>> in parseAndGenerate
>> parser.parse(rootFile)
>>> File "C:\EngTools\Anaconda3\lib\xml\sax\expatreader.py", line 111, in
>>> parse
>> xmlreader.IncrementalParser.parse(self, source)
>>> File "C:\EngTools\Anaconda3\lib\xml\sax\xmlreader.py", line 123, in
>>> parse
>> buffer = file.read(self._bufsize)
>>> File "C:\EngTools\Anaconda3\lib\encodings\cp1252.py", line 23, in
>>> decode
>> return codecs.charmap_decode(input,self.errors,decoding_table)[0]
>>> UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in
>>> position 4013: character maps to <undefined>
>>
>> Approach 2 - Extraction and Generation Utilities:
>>> (base) h:\Python Training\GenerateDS_Docs\utils>python
>>> collect_schema_locations.py --force --verbose TestResults.xsd
>>> testJson.txt Traceback (most recent call last):
>>> File "h:\Python
>>> Training\GenerateDS_Docs\utils\collect_schema_locations.py", line
>>> 157, in <module>
>> main()
>>> File "h:\Python
>>> Training\GenerateDS_Docs\utils\collect_schema_locations.py", line
>>> 148, in main
>> extract_and_generate(infile, outfile, extract_locations_fn,
>> options)
>>> File "h:\Python
>>> Training\GenerateDS_Docs\utils\collect_schema_locations.py", line 86,
>>> in extract_and_generate
>> locations = extract_locations(infile, options)
>>> File "h:\Python
>>> Training\GenerateDS_Docs\utils\collect_schema_locations.py", line 50,
>>> in extract_locations
>> elements = root.xpath('xs:import', namespaces=root.nsmap)
>>> File "src/lxml/etree.pyx", line 1579, in lxml.etree._Element.xpath
>>> File "src/lxml/xpath.pxi", line 259, in
>>> lxml.etree.XPathElementEvaluator.__init__
>>> File "src/lxml/xpath.pxi", line 131, in
>>> lxml.etree._XPathEvaluatorBase.__init__
>>> File "src/lxml/xpath.pxi", line 55, in
>>> lxml.etree._XPathContext.__init__ File "src/lxml/extensions.pxi",
>>> line 81, in lxml.etree._BaseContext.__init__
>>> TypeError: empty namespace prefix is not supported in XPath
>>
>> Attached are the .xsd that I am using. TestResults.xsd references
>> Common.xsd.
>>
>> Any help on getting started with my overall goal will be much
>> appreciated. I am looking to create a new XML from a known .xsd and
>> add in the elements and child elements and attributes as needed.
>>
>> Thank you for the help,
>> Daniel DeWitte
>
>
>
> --
> Dave Kuhlman
> http://www.davekuhlman.org
--
Dave Kuhlman
http://www.davekuhlman.org
|
