labpython-users

Re: [LabPython-Users] Updates to the project

[Rolf K. <rol...@gm...>]

Hi Daniele

I think you got all relevant people who have ever provided something to
this project. Basically the C code is all from me, Jim Kring helped with
some of the LabVIEW VIs and testing and the website.

I don't have the interest and the time to dig into LabPython at this time
so if you want to go with your own, then you are free to do so. Just notice
that LabPython was and still is LGPL licensed so you can't relicense it
under a different license at this point.

The dynamic loading of both the Python core and the lvpython part from
within pytscript is actually not just a random complication. The Python
kernel was loaded dynamically since the Python developers choose to release
every new Python version with a different shared library and that would
have made a new release of LabPython necessary for every new Python
release. This worked fine until Python 2.7 which introduced certain changes
in preparation of 3.0 that broke the ABI of the Python shared library.
Removing that will certainly make maintenance of LabPython to work with
more than one single Python version more complicated than easier.

The separation of pytscript and lvpython was done for a different reason.
pytscript in order to work has to be in a very specific location inside
LabVIEW. Anywhere else and the script interface will not recognize the
shared library as scripting server. But that location is not in any of the
paths that LabVIEW will search for shared libraries when used in a Call
Library Node. This makes the whole setup for the LabVIEW VI interface very
brittle and easy to break as the VIs can and sometimes will loose the link
to the location of that shared library when installed into a non standard
LabVIEW system. And when building applications these locations are
different again. By using dynamic linking it is fairly easy to add
additional search paths to the loader rather than havi g to changge a
system global PATH environment variable or similar.

>From the two interfaces pytscript and the LabVIEW VIs I consider the
LabVIEW VI interface the much more useful one, as the scripting interface
can not work with runtime changable scripts. The text of a script is
compiled into the VI at compile time and cannot be changed at runtime (well
it can in the development environment with VI scripting, provided the VI in
question is not part of a running VI hierarchy, but that is a Gorilla
approach to runtime changable scripts and will not work in an executable at
all). Rather than removing the dynamic interface between pytscript and
lvpython I would simply remove the enitre pytscript. That makes a lot more
sense.

Stability problems of LabPython are most likely due to the fact that Python
isn't multitreading safe while LabVIEW will execute Call Library Nodes not
set to run in the UI thread in any of the multiple threads that every
executable system has. Changing the Call Library Nodes to run in the UI
thread is a terrible limitation that will block LabVIEW UI changes whenever
you execute a Python VI that takes more than a few microseconds to do its
stuff and eliminate any possibility to do load sharing with multiple python
sessions,  but adding thread synchronization to each lvpython function to
make sure a particular python context is only invoked from the same thread
is going to be a project where the dynamic loading of shared library calls
looks like peanouts in comparison. It's not something I feel inclined to
work on at the moment. Basically calling thread unsafe code from a
multithreading environment either requires mutex protection of each
resource or thread synchronisation or somethimes even both, which would be
a very good way to create self locking code very easily without VERY
tedious discipline in the C code and possibily even in the LabVIEW code.

Sincerely

Rolf Kalbermatter

Re: [LabPython-Users] Updates to the project

[Rolf K. <rol...@gm...>]

By the way the latest version of LabPython is not on the CVS repository of
sourceforge but rather in the SVN repository.

Sincerely

Rolf Kalbermatter

Re: [LabPython-Users] Updates to the project

[Daniele N. <da...@gr...>]

On 04/03/16 12:38, Rolf Kalbermatter wrote:
> By the way the latest version of LabPython is not on the CVS repository
> of sourceforge but rather in the SVN repository.

Which SVN repository?

Cheers,
Daniele

Re: [LabPython-Users] Updates to the project

[Daniele N. <da...@gr...>]

Hello Rolf,

the first thing I would like to do is to update the LabPython presence
on SourceForge to reflect the current status of the project:

- the code has moved to the OpenG SVN repository,
- python versions up to 2.7 are supported,
- provide a location where to download the latest binaries,
- there is active interest in the project.

Would you like to give me access to the SorceForce project to do that?

Later, I would like to move the code repository to GitLab. What do you
think about this?

On 05/03/16 05:47, Rolf Kalbermatter wrote:
> Not sure which email address Jim would prefer. I'm sure you know about
> his company but he may prefer a different email address mentioned than
> his company address. For me that is certainly true both for legal
> reasons as this was something private rather than from my employer and
> also the fact that work email tends to change sometimes.

While I have been exposed to LabVIEW for quite a long time now, I cannot
consider myself an expert nor I'm much involved with the community. All
the company names I encountered researching contacts for the LabPython
authors were new to me.

> The LabPython code got moved at some point into a subproject of the
> OpenG Toolkit and as such the latest version is in the SVN repository of
> that project. I did recently make minor changes to it to fix the runtime
> errors when loading with Pyton 2.7.

I found the code repository. It seems that it has been imported without
preserving the CVS history. I'll see how difficult it is to stitch
together the two.

> I'm aware that Python has no guarantee about ABI compatibility between
> versions. However reality was that it seemed to work pretty good from
> version 2.2 up to 2.6 and with that minor change I committed recently
> even allows loading with 2.7.

The size of the structures changed, and I believe also the position of
some fields, and some of the API is defined in terms of macros. I'm
surprised this can work at all.

> Another issie is that the Python project
> maintainers never seemed to be very supportive about embedding Python
> into another program and concentrated mainly on extensibility. Any
> inquiry from me about that was at that time either ignored or outrightly
> refused.

I'm surprised. Embedding of the python interpreter is definitely a
supported and documented use-case.

> If you look at the code you will notice that I make use of the
> interpreter lock, but somehow that seems not enough to make Python run
> stable from within a multithreading environment. It could be a
> shortcoming in the interpreter lock handling in the LabPython code,
> maybe caused by internal changes in Python in later version  after 2.3,
> or it could be a fundamental problem, for instance the use of thread
> local storage within Python.

I haven't understood all the details of the implementation, but from a
coarse analysis of the codequick look at the code, there are a few
constructs that look suspicious. Test cases that cause troubles in
LabPython would be definitely useful.

Cheers,
Daniele

Re: [LabPython-Users] Updates to the project

[Daniele N. <da...@gr...>]

Hello Rolf,

thank you for the prompt reply and the useful information.

On 04/03/16 10:50, Rolf Kalbermatter wrote:
> I think you got all relevant people who have ever provided something
> to this project. Basically the C code is all from me, Jim Kring
> helped with some of the LabVIEW VIs and testing and the website.

Great! This has been easy :) However, I would like to know if the email
addresses I found are the "canonical" ones for the persons involved,
such that I can use them in the repository conversion.

> I don't have the interest and the time to dig into LabPython at this
> time so if you want to go with your own, then you are free to do so.
> Just notice that LabPython was and still is LGPL licensed so you can't
> relicense it under a different license at this point.

Re-licensing is definitely not the intent. However, I would prefer to do
not fork the project, but I would like to become a co-maintainer, or
take over maintenance, if as I understand from the above there is
actually no maintainer.

> The dynamic loading of both the Python core and the lvpython part from
> within pytscript is actually not just a random complication. The Python
> kernel was loaded dynamically since the Python developers choose to
> release every new Python version with a different shared library and
> that would have made a new release of LabPython necessary for every new
> Python release. This worked fine until Python 2.7 which introduced
> certain changes in preparation of 3.0 that broke the ABI of the Python
> shared library. Removing that will certainly make maintenance of
> LabPython to work with more than one single Python version more
> complicated than easier.

I may stand corrected on this, but I believe that major python versions
are not guaranteed to be ABI compatible, and thus I don't really see how
dynamically loading the library allows to do not have to recompile
labpython for different python major releases. It may be that doing this
was working, but by accident rather by than design. I believe Python
does not give any guarantee that this works, as has been discovered with
Python 2.7 release.

That said, a new Python major version is released about every 1.5 years.
I don't see recompiling labpython every so often as a major maintenance
burden.

> The separation of pytscript and lvpython was done for a different
> reason. pytscript in order to work has to be in a very specific location
> inside LabVIEW. Anywhere else and the script interface will not
> recognize the shared library as scripting server. But that location is
> not in any of the paths that LabVIEW will search for shared libraries
> when used in a Call Library Node. This makes the whole setup for the
> LabVIEW VI interface very brittle and easy to break as the VIs can and
> sometimes will loose the link to the location of that shared library
> when installed into a non standard LabVIEW system. And when building
> applications these locations are different again. By using dynamic
> linking it is fairly easy to add additional search paths to the loader
> rather than havi g to changge a system global PATH environment variable
> or similar.

I see. I don't know enough about Windows shared library loading system
to comment in any way. However the very small size of lvpython.dll makes
me wonder if it would not be easier to simply ship two independent
shared libraries. Would you like to comment on that possibility?

> From the two interfaces pytscript and the LabVIEW VIs I consider the
> LabVIEW VI interface the much more useful one, as the scripting
> interface can not work with runtime changable scripts. The text of a
> script is compiled into the VI at compile time and cannot be changed at
> runtime (well it can in the development environment with VI scripting,
> provided the VI in question is not part of a running VI hierarchy, but
> that is a Gorilla approach to runtime changable scripts and will not
> work in an executable at all). Rather than removing the dynamic
> interface between pytscript and lvpython I would simply remove the
> enitre pytscript. That makes a lot more sense.

You definitely have a point. However, I believe that for many
applications the functionality offered by pytscript is good enough and
easier to use than the lvpython one.

> Stability problems of LabPython are most likely due to the fact that
> Python isn't multitreading safe

I may not be understanding what you mean, but I think that this is not
the case: Python as the Global Interpreter Lock that serializes access
to the interpreter state from multiple threads. It is through, however,
that the global interpreter lock must be explicitly acquired and
released in threads not initiated by Python itself.

> while LabVIEW will execute Call Library
> Nodes not set to run in the UI thread in any of the multiple threads
> that every executable system has. Changing the Call Library Nodes to run
> in the UI thread is a terrible limitation that will block LabVIEW UI
> changes whenever you execute a Python VI that takes more than a few
> microseconds to do its stuff and eliminate any possibility to do load
> sharing with multiple python sessions,  but adding thread
> synchronization to each lvpython function to make sure a particular
> python context is only invoked from the same thread is going to be a
> project where the dynamic loading of shared library calls looks like
> peanouts in comparison. It's not something I feel inclined to work on at
> the moment. Basically calling thread unsafe code from a multithreading
> environment either requires mutex protection of each resource or thread
> synchronisation or somethimes even both, which would be a very good way
> to create self locking code very easily without VERY tedious discipline
> in the C code and possibily even in the LabVIEW code.

I was under the impression that the Global Interpreter Lock does just
what you describe above. However, I have much more experience extending
Python than embedding Python, thus I may be wrong. I'm happy to
investigate this more.

Thank you for your comments!

Cheers,
Daniele