Menu

Python.NET | Trouble with importing System.dll

2025-01-16
2025-03-03
  • Andreas Gruber

    Andreas Gruber - 2025-01-16

    Hello to the DWSIM development team.
    First of all a massive thank you for developing such a powerful tool !

    Currently I would like to run a custom script to model the behaviour of a membrane reactor.
    The results of this script are returned in a numpy.array. The script runs fine. The only error / Problem is the following:
    To convert the results into the System.Array needed in DWSIM, I used the command "import System" with clr.AddReference("System")
    as stated in the examples.

    However I get the following error message:

    System.Exception: Reactor TEST: module 'System' has no attribute 'Array'
    line 22, in transOutput
    x_s_trans = System.ArraySystem.Double
    File "<string>", line 152, in <module>
    at Python.Runtime.PythonException.ThrowLastAsClrException()
    at Python.Runtime.PythonException.ThrowIfIsNull(NewReference& ob)
    at Python.Runtime.PythonEngine.RunString(String code, BorrowedReference globals, BorrowedReference locals, RunFlagType flag)
    at Python.Runtime.PythonEngine.Exec(String code, PyDict globals, PyObject locals)
    at DWSIM.UnitOperations.UnitOperations.CustomUO.Calculate(Object args) in C:\Users\danie\source\repos\DanWBR\dwsim\DWSIM.UnitOperations\UnitOperations\PythonScriptUO.vb:line 358 ---> System.Exception: module 'System' has no attribute 'Array'</module></string>

    x_s_trans = System.Array[System.Double](x_s_l)
    

    File "<string>", line 152, in <module>
    at Python.Runtime.PythonException.ThrowLastAsClrException()
    at Python.Runtime.PythonException.ThrowIfIsNull(NewReference& ob)
    at Python.Runtime.PythonEngine.RunString(String code, BorrowedReference globals, BorrowedReference locals, RunFlagType flag)
    at Python.Runtime.PythonEngine.Exec(String code, PyDict globals, PyObject locals)
    at DWSIM.UnitOperations.UnitOperations.CustomUO.Calculate(Object args) in C:\Users\danie\source\repos\DanWBR\dwsim\DWSIM.UnitOperations\UnitOperations\PythonScriptUO.vb:line 358 ---> Python.Runtime.PythonException: module 'System' has no attribute 'Array'
    at Python.Runtime.PythonException.ThrowLastAsClrException()
    at Python.Runtime.PythonException.ThrowIfIsNull(NewReference& ob)
    at Python.Runtime.PythonEngine.RunString(String code, BorrowedReference globals, BorrowedReference locals, RunFlagType flag)
    at Python.Runtime.PythonEngine.Exec(String code, PyDict globals, PyObject locals)
    at DWSIM.UnitOperations.UnitOperations.CustomUO.Calculate(Object args) in C:\Users\danie\source\repos\DanWBR\dwsim\DWSIM.UnitOperations\UnitOperations\PythonScriptUO.vb:line 358
    --- End of inner exception stack trace ---
    at DWSIM.UnitOperations.UnitOperations.CustomUO.Calculate(Object args) in C:\Users\danie\source\repos\DanWBR\dwsim\DWSIM.UnitOperations\UnitOperations\PythonScriptUO.vb:line 380
    at DWSIM.SharedClasses.UnitOperations.BaseClass.Solve() in C:\Users\danie\source\repos\DanWBR\dwsim\DWSIM.SharedClasses\BaseClass\SimulationObjectBaseClasses.vb:line 475
    at DWSIM.FlowsheetSolver.FlowsheetSolver.CalculateObjectAsync(Object fobj, CalculationArgs objArgs, CancellationToken ct) in C:\Users\danie\source\repos\DanWBR\dwsim\DWSIM.FlowsheetSolver\FlowsheetSolver.vb:line 316
    at DWSIM.FlowsheetSolver.FlowsheetSolver.ProcessQueueInternalAsync(Object fobj, CancellationToken ct) in C:\Users\danie\source\repos\DanWBR\dwsim\DWSIM.FlowsheetSolver\FlowsheetSolver.vb:line 662
    --- End of inner exception stack trace ---</module></string>

    The path of the System.dll seems correct ( C:\Windows\Microsoft.NET\Framework\v4.0.30319\System.dll ).
    To me it looks like this .dll file can not be interpreted correctly.

    Do you have any tips on how to solve this ?

    Versions: Python 3.9
    Pythonnet 3.0.5
    DWSIM 8.8.3
    Windows 11 (64 Bit)

    Many Thanks in advance !!

     
    • enjenir

      enjenir - 2025-02-13

      If you just want to update a MaterialStream or something, look towards the last posts of this discussion. System.Array is currently unobtainable.

       
  • Andreas Gruber

    Andreas Gruber - 2025-01-16

    Hi Daniel

    thanks for the reply. I checked this issue but for me it is not the same.

    In the this issue: https://github.com/pythonnet/pythonnet/issues/388 it looks like the module System could not be found.

    However in my case the module was found. I think that it just could not be read correctly.

    The Problem with nameclashes in pythonnet and clr was resolved in Versions 2.5.2 where clr was included in Python.NET I believe.

     
  • enjenir

    enjenir - 2025-02-11

    Hi Daniel and Andreas,
    I have a similar problem. Can you try to run "from System import Array" instead and see if it also yields "unknown location"?
    You see, "import System" may work. However, "from System import Array" will not.

    This was all tested on Win11 with DWSIM 8.8.3 and WinPython 3.8.9.

    Funny enough "from System.Collections import ArrayList()" will also work:

    import clr
    clr.AddReference("System")
    import System
    from System.Collections import ArrayList
    
    Flowsheet.WriteMessage(str(ArrayList))
    

    Will yield "Message <class 'system.collections.arraylist'="">", and replacing "ArrayList" with System will yield "Message <module 'system'="" (namespace)="">".
    If we chance the Flowsheet.WriteMessage to "str(dir(System))" we get: </module></class>

    Message ['Collections', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__']
    

    Checking path it goes to "C:\Windows\System". This seems (very) wrong?!. In interpreter, it will return an empty list, and System.file will be
    "C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System\v4.0_4.0.0.0__b77a5c561934e089\System.dll" instead.

    C:\WPy64-3890\scripts>dotnet --list-runtimes                                                                            Microsoft.NETCore.App 6.0.29 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]                                     Microsoft.NETCore.App 8.0.12 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]                                     Microsoft.WindowsDesktop.App 6.0.29 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]                       Microsoft.WindowsDesktop.App 8.0.12 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
    

    DWSIM requires ".NET Runtime 4.6.2 or newer", so I downloaded the installer from here:
    https://dotnet.microsoft.com/en-us/download/dotnet-framework/thank-you/net462-offline-installer
    And tried to install, but it gave ".NET Framework 4.6.2 or a later update is already installed on this computer."...

    For context: I installed DWSIM 8.8.3, WinPython 3.8.9. Set the Python.NET interpreter to
    "C:\WPy64-3890\python-3.8.9.amd64", then restarted DWSIM to ensure it should be the right intepreter.
    Added a Custom Python Unit to the flowsheet, set it to Python.NET, adding in and out material stream so it can solve (and thus run the python script).

    Out of curiosity I added the following simple snippet:

    import clr
    Flowsheet.WriteMessage(clr.__version__)
    

    This will respond with "3.0.0dev" on a fresh install.

    If I go into the WinPython Interpreter, and run

    import clr
    print(clr.__version__)
    

    I get 2.5.1 and can do "from System import Array". However, I cannot do "from pythonnet import load".

    But if I go into WinPython Command line and run "pip uninstall pythonnet" and "pip install pythonnet", it will install 3.0.5.

    Using

    from pythonnet import load
    load("netfx")
    import clr
    clr.AddReference("System")
    from System import Array
    

    With the error code

    This error was raised during the calculation of a Unit Operation or Material Stream.
    
    System.Exception: cannot import name 'Array' from 'System' (unknown location)
      File "<string>", line 5, in <module>
       at Python.Runtime.PythonException.ThrowLastAsClrException()
       at Python.Runtime.PythonException.ThrowIfIsNull(NewReference& ob)
       at Python.Runtime.PythonEngine.RunString(String code, BorrowedReference globals, BorrowedReference locals, RunFlagType flag)
       at Python.Runtime.PythonEngine.Exec(String code, PyDict globals, PyObject locals)
       at DWSIM.UnitOperations.UnitOperations.CustomUO.Calculate(Object args) in C:\Users\danie\source\repos\DanWBR\dwsim\DWSIM.UnitOperations\UnitOperations\PythonScriptUO.vb:line 358 ---> Python.Runtime.PythonException: cannot import name 'Array' from 'System' (unknown location)
       at Python.Runtime.PythonException.ThrowLastAsClrException()
       at Python.Runtime.PythonException.ThrowIfIsNull(NewReference& ob)
       at Python.Runtime.PythonEngine.RunString(String code, BorrowedReference globals, BorrowedReference locals, RunFlagType flag)
       at Python.Runtime.PythonEngine.Exec(String code, PyDict globals, PyObject locals)
       at DWSIM.UnitOperations.UnitOperations.CustomUO.Calculate(Object args) in C:\Users\danie\source\repos\DanWBR\dwsim\DWSIM.UnitOperations\UnitOperations\PythonScriptUO.vb:line 358
       --- End of inner exception stack trace ---
       at DWSIM.UnitOperations.UnitOperations.CustomUO.Calculate(Object args) in C:\Users\danie\source\repos\DanWBR\dwsim\DWSIM.UnitOperations\UnitOperations\PythonScriptUO.vb:line 380
       at DWSIM.SharedClasses.UnitOperations.BaseClass.Solve() in C:\Users\danie\source\repos\DanWBR\dwsim\DWSIM.SharedClasses\BaseClass\SimulationObjectBaseClasses.vb:line 475
       at DWSIM.FlowsheetSolver.FlowsheetSolver.CalculateObjectAsync(Object fobj, CalculationArgs objArgs, CancellationToken ct) in C:\Users\danie\source\repos\DanWBR\dwsim\DWSIM.FlowsheetSolver\FlowsheetSolver.vb:line 316
       at DWSIM.FlowsheetSolver.FlowsheetSolver.ProcessQueueInternalAsync(Object fobj, CancellationToken ct) in C:\Users\danie\source\repos\DanWBR\dwsim\DWSIM.FlowsheetSolver\FlowsheetSolver.vb:line 662
    

    Furthermore, according to this https://github.com/DanWBR/dwsim/issues/36:

    Inside DWSIM's installation directory you will find a "pythonnet" folder which contains the DLL for python 3.6, 3.7 and 3.8. Just copy the desired one back to the main directory and it should work.

    So I have checked it. "C:\Users\User\AppData\Local\DWSIM" - searching this directory I can find no "pythonnet" folder, after a fresh install, deleting everything related to DWSIM.

    The really confusing part:
    If I

    import sys
    Flowsheet.WriteMessage(sys.version)
    

    it gives

        1   11/02/2025 20.35.49 Message 3.8.9 (tags/v3.8.9:a743f81, Apr  2 2021, 11:10:41) [MSC v.1928 64 bit (AMD64)]  + Info
    

    So I know it's running the WinPython interpreter. It's just not loading clr correctly at all?

    import os
    os.environ["DOTNET_ROOT"] = r"C:\Program Files\dotnet"  # Set .NET Core root path
    from pythonnet import load
    load("netfx")  # Load the .NET Core runtime explicitly
    
    import clr
    clr.AddReference(r"C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System\v4.0_4.0.0.0__b77a5c561934e089\System.dll")
    from System import Array
    

    Will not change the outcome.

    To summarize:
    1. import System should result in System.__file__being something like C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System\v4.0_4.0.0.0__b77a5c561934e089\System.dll - currently it returns "None" in DWSIM.
    2. DWSIM loads the Python.Runtime.dll which is in the installation directory of DWSIM. Unless this file is replaced it will always revert to 3.0.0dev even though another version is available in WinPythons Python.NET distribution
    3. If one does replace the Python.Runtime.dll, from pythonnet import load will not work.
    4. Changing clr.AddReference("System") to clr.AddReference("C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System\v4.0_4.0.0.0__b77a5c561934e089\System.dll"), which is the one that is loaded when loading System from intepreter, results in System.Exception: Unable to cast object of type 'Python.Runtime.CLRObject' to type 'Python.Runtime.ManagedType'., using r"C:..." the System file still remains None.
    5. This indicates that in DWSIM, the System.dll is never actually loaded. Which makes loading Array impossible, since System.Array doesn't even exist (as far as Python knows).
    6. The question now becomes: When we import System, what does DWSIM actually load? And why does System.path become C:\Windows\System?!

    @danwbr After diving into it I'm becoming more and more convinced that there may be some kind of bug that occurs on some machines when it comes to Python.NET and DWSIM.
    Although I still have no idea why exactly it doesn't properly load the System.dll?

     
  • enjenir

    enjenir - 2025-02-12

    Pretty sure it's something with how DWSIM manages clr. import clr always responds with version 3.0.0dev, despite it technically should be loaded dynamically from WinPython environment?

    "ERROR: Ignored the following yanked versions: 3.0.0rc1, 3.0.0" (when trying to pip install a non-existing version).

    From this I gather it is still trying to load clr 3.0.0dev (which may be "yanked", i.e. not really working), even though another version of pythonnet like 2.5.2 (with clr 2.5.1) is installed, worse in pythonnet == 2.5.2 in interpreter it cannot load clr_loader.

    In pythonnet3.0.5 it utilizes clr_loader, clr becomes legacy (https://pythonnet.github.io/clr-loader/).

    Also

    Python.NET uses the PYTHONPATH (sys.path) to look for assemblies to load, in addition to the usual application base and the GAC (if applicable). To ensure that you can import an assembly, put the directory containing the assembly in sys.path.

    Pretty sure it's something with DWSIM loading wrong clr in Python.NET although still not sure why.

     
  • Daniel Medeiros

    Daniel Medeiros - 2025-02-12

    it may be trying to load System.dll from another .NET version like 6, 7, or 8. You can try to force loading System.dll from a .NET Framework version

    https://pythonnet.github.io/clr-loader/reference.html#clr_loader.get_netfx

     
  • enjenir

    enjenir - 2025-02-12

    @danwbr Thanks for the quick reply. Ensuring that I have .NET runtime 4.X installed (Windows Powershell, should be present anyways from start in all versions of Windows above Win8):

    Get-ItemProperty -Path 'HKLM:\SOFTWARE\Microsoft\NET Framework Setup\NDP\v4\Full' -Name Version
    Version      : 4.8.09032
    

    Importing clr_loader works, as does pythonnet. CLR version is now 3.0.5 as expected.

    import os
    import clr_loader
    import pythonnet
    
    runtime = clr_loader.get_netfx()
    pythonnet.load(runtime)
    Flowsheet.WriteMessage(repr(pythonnet.get_runtime_info()))
    
    import clr 
    Flowsheet.WriteMessage(str(clr.__version__))
    clr.AddReference(r'C:\Windows\Microsoft.NET\Framework64\v4.0.30319\System.dll')
    from System import Array
    

    Still returns System.Exception: CUSTOM-1: cannot import name 'Array' from 'System' (unknown location) - if I give it full path to System.dll or if I run it with clr.AddReference("System")does not matter.

    Sadly, clr_loader has an open issue meaning that version will be always displayed as "undefined".

        0   12/02/2025 20.15.24 Message RuntimeInfo(kind='.NET Framework', version='<undefined>', initialized=True, shutdown=False) + Info
    

    However, we have a System.Runtime.InteropService which we can abuse:

    import System.Runtime.InteropServices
    runtime_version = System.Runtime.InteropServices.RuntimeInformation.FrameworkDescription
    Flowsheet.WriteMessage(f".NET Runtime Version: {runtime_version}")
    

    Which results in .NET Runtime Version: .NET Framework 4.8.9290.0, so we know it's loading the correct runtime?

    Interestingly enough, I can import System.Reflection, as specified here.

    Now since we have System.Reflection we can abuse it abit:

    import System.Reflection
    
    system_assembly = System.Reflection.Assembly.Load("System")
    Flowsheet.WriteMessage(f"System.dll loaded from: {system_assembly.Location}")
    

    And it will return from where it loads System.dll:

    System.dll loaded from: C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System\v4.0_4.0.0.0__b77a5c561934e089\System.dll
    

    And thus

     ['Reflection', 'Runtime', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__']
    

    when calling Flowsheet.WriteMessage(repr(dir(System)))

    I don't know what to make of this, since I don't know which System.dll exactly DWISM is supposed to load? Anyways, the from System import Arraystatement still results in an error.

     

    Last edit: enjenir 2025-02-12
  • Daniel Medeiros

    Daniel Medeiros - 2025-02-13

    I filled a bug in pythonnet repo, I've also wasn't able to load Array from System. Tried different versions but none of them worked. https://github.com/pythonnet/pythonnet/issues/2549

     
    👍
    2
  • enjenir

    enjenir - 2025-02-13

    Let's wait and see. Hopefully they give us some information on how to get it working...Custom PythonUO without being able to place the results of ones calculations into a MaterialStream kind of defeats the purpose of having a custom PythonUO in the first place :(

    (Technically ArrayList is still accessible, funnily enough, but that just sounds like a workaround that is more trouble than it's worth)

    Checking their replies...

    That version could be anything from 4.6 on. We only support 4.7.2 and up as these are the only properly .NET Standard-compatible versions of .NET Framework.

    Checking with Get-ItemProperty -Path 'HKLM:\SOFTWARE\Microsoft\NET Framework Setup\NDP\v4\Full' -Name Version I am fairly certain I have version 4.8.09032. I am also somewhat of the opinion that Windows 11 does not ship with a version lower than 4.8:

    .NET Framework 4.8 is included with Windows 11, and runs any .NET Framework 4.x app.

    Checking Settings->System->More Windows Features looks like this.

    I wonder - could ASP.NET 4.8 missing have an impact? I'm not sure if it should.

     

    Last edit: enjenir 2025-02-13
    • Daniel Medeiros

      Daniel Medeiros - 2025-02-13

      he's not willing to help us, I'll need to fix it by myself somehow.

       
  • Daniel Medeiros

    Daniel Medeiros - 2025-02-13

    SetOverallComposition() can be replaced by SetOverallMolarComposition() and SetOverallMassComposition(), which will accept a python array as argument:

    s1.SetOverallMolarComposition({0.5, 0.5})
    
     
    👍
    2
  • enjenir

    enjenir - 2025-02-13

    @danwbr Thank you anyways, your effort is appreciated.

    We only support 4.7.2 and up as these are the only properly .NET Standard-compatible versions of .NET Framework.

    I've checked the Microsoft release page for .NET. Apparently they did not add .NET Standard 2.0 until 4.7.1

    Support for .NET Standard 2.0
    .NET Standard defines a set of APIs that must be available on each .NET implementation that supports that version of the standard. .NET Framework 4.7.1 fully supports .NET Standard 2.0 and adds about 200 APIs that are defined in .NET Standard 2.0 and are missing from .NET Framework 4.6.1, 4.6.2, and 4.7. (Note that these versions of .NET Framework support .NET Standard 2.0 only if additional .NET Standard support files are also deployed on the target system.)

    I'm not sure if I understand correctly what they're trying to say but isn't it somewhat like "Yeah you can compile for netstandard2.0 for target 4.6.2, buuuuut on the target system you need the additional .NET Standard files as well". Still - if .NET framework is installed, one would assume so is netstandard2.0 nowadays - unless netstandard2.0 for 4.6.2 needs specific files for 4.6.2 (idk, it's been a long day...hope it makes sense). Thus, if DWSIM is compiled targeting 4.6.2, on a machine that has said additional netstandard2.0 files, there should be no problem. However, once you move from one machine to another that doesn't have them, these files would be missing?

    SetOverallComposition() can be replaced by SetOverallMolarComposition() and SetOverallMassComposition(), which will accept a python array as argument:

    Thank you so very much.
    Edit: Couldn't get SetOverallComposition() to work, but SetOverallCompoundMolarFlow seems to work - I just have to iterate over my dict of "Compound":float to set the entire material stream :)

    For anybody else suffering the same issue:
    Consider that you've specified compounds Oxygen, Carbon Dioxide, Carbon Monoxide, in that order (in effect it will appear in this order if you check "Compound Amounts" in the material stream).

    # get material stream
    s1 = Flowsheet.GetMaterialStream("stream") # If your stream is named "stream"
    # SetOverallCompoundMolarFlow(str compoundName, double flow)
    # SetOverallCompoundMolarFlow(int32 compoundIndex, double flow)
    EX_DICT = {"O2":0.34, "CO2": 0.43, "CO": 0.03}
    s1.SetOverallCompoundMolarFlow(0, EX_DICT["O2"]) # Sets O2 to 0.34 mol/s
    s1.SetOverallCompoundMolarFlow(1, EX_DICT["CO2"]) # Sets CO2 to 0.43 mol/s
    s1.SetOverallCompoundMolarFlow(2, EX_DICT["CO"]) # Sets CO to 0.03 mol/s
    
    # This will work too, if your string for O2 is "Oyxgen"
    s1.SetOverallCompoundMolarFlow("Oxygen", EX_DICT["O2"])
    
     

    Last edit: enjenir 2025-02-13
  • Andreas Gruber

    Andreas Gruber - 2025-03-03

    Hi all

    First of all thanks for the efforts.

    What I found worked well is the following work around:

    for i in range(len(Composition_sweep_outlet)):
        S_o.SetOverallCompoundMolarFlow(S_i.GetCompoundNames()[i], n_s * Composition_sweep_outlet[i]) 
    
    S_o.NormalizeOverallMoleComposition()
    

    That way the data type "System.array" is not needed. Rather the values are extracted and passed individually as "double". I found that no real performance drawbacks were noticeable (I am only considering 8 compounds).

    This of course does not really solve the issue but rather can be used as a work around.

     
    👍
    1
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.