#507 measurement group with external tango channels fails

[Jan Kotanski]

Hi,

when I try to use Sardana 2.1.0 with Taurus 4.0.1 running ascan from macrogui I get the following error:

Error details:
Traceback (most recent call last):
  File "/home/jkotan/testsar/lib/python2.7/site-packages/sardana/macroserver/msmacromanager.py", line 1192, in __runXMLMacro
    macro_obj, _ = self._prepareXMLMacro(xml)
  File "/home/jkotan/testsar/lib/python2.7/site-packages/sardana/macroserver/msmacromanager.py", line 936, in _prepareXMLMacro
    prepare_result = self._prepareMacroObj(macro_obj, macro_params)
  File "/home/jkotan/testsar/lib/python2.7/site-packages/sardana/macroserver/msmacromanager.py", line 959, in _prepareMacroObj
    prepare_opts=prepare_opts)
  File "/home/jkotan/testsar/lib/python2.7/site-packages/sardana/macroserver/msmacromanager.py", line 756, in prepareMacroObj
    return macro.prepare(*par_list, **prepare_opts)
  File "/usr/lib/python2.7/dist-packages/sardana/macroserver/macros/scan.py", line 338, in prepare
    self._prepare([motor], [start_pos], [final_pos], nr_interv, integ_time, **opts)
  File "/usr/lib/python2.7/dist-packages/sardana/macroserver/macros/scan.py", line 125, in _prepare
    self._gScan = SScan(self, self._stepGenerator, moveables, env, constrains, extrainfodesc)
  File "/home/jkotan/testsar/lib/python2.7/site-packages/sardana/macroserver/scan/gscan.py", line 326, in __init__
    self._setupEnvironment(env)
  File "/home/jkotan/testsar/lib/python2.7/site-packages/sardana/macroserver/scan/gscan.py", line 574, in _setupEnvironment
    channels_info = self.measurement_group.getChannelsEnabledInfo()
  File "/home/jkotan/testsar/lib/python2.7/site-packages/sardana/taurus/core/tango/sardana/pool.py", line 1493, in getChannelsEnabledInfo
    return self.getConfiguration().getChannelsInfoList(only_enabled=True)
  File "/home/jkotan/testsar/lib/python2.7/site-packages/sardana/taurus/core/tango/sardana/pool.py", line 1290, in getChannelsInfoList
    channels_info = self.getChannelsInfo(only_enabled=only_enabled)
  File "/home/jkotan/testsar/lib/python2.7/site-packages/sardana/taurus/core/tango/sardana/pool.py", line 1276, in getChannelsInfo
    for ch_name, (_, _, ch_info) in ret.items():
ValueError: too many values to unpack
Job ended (stopped=False, aborted=False)

while when I try to run spock I get

jkotan@haso228k:~/sources/sardana$ spock
MainThread     INFO     2016-09-14 08:27:03,965 TaurusRootLogger: Using "PyQt4" for Qt
Spock needs itango version >= 0.1.4. No itango installation found

Do I miss something?

Bests,
Jan

[Zbigniew Reszela]

Spock requires itango >= 0.1.4 from version 2.1.0 on.
You can obtain it from PyPI.

We will get back to you with the first problem soon.

[Zbigniew Reszela]

Jan, could you please post here the steps to reproduce the first problem?
We have just tried it here, and were unable to trigger the error.
Please be specific with the measurement group since from the traceback it seems to be related to it.

[Jan Kotanski]

Hi Zibi,

Thanks. Indeed, installing itango solves the second problem.

And the first problem is related to measurement group configuration, i.e. with tango channels. If I remove it everything is OK. Bellow I attach the configuration read in jive.

Does it mean that configuration is not backward compatible or it is just a bug? What should be a proper JSON substring for tango channels?

Bests,
Jan

Read:   {"controllers":{"haso228k:10000/controller/dgg2ctrl/dgg2_exp_01":{"units":{"0":{"channels":{"haso228k:10000/expchan/dgg2_exp_01/1":{"ndim":0,"plot_type":0,"full_name":"haso228k:10000/expchan/dgg2_exp_01/1","data_type":"float64","nexus_path":"","shape":[],"conditioning":"","plot_axes":[],"_unit_id":"0","normalization":0,"index":0,"name":"exp_t01","instrument":null,"enabled":true,"label":"exp_t01","source":"haso228k:10000/expchan/dgg2_exp_01/1/Value","_controller_name":"haso228k:10000/controller/dgg2ctrl/dgg2_exp_01","data_units":"No unit","output":true}},"trigger_type":0,"id":0,"timer":"haso228k:10000/expchan/dgg2_exp_01/1","monitor":"haso228k:10000/expchan/dgg2_exp_01/1"}}},"__tango__":{"units":{"0":{"channels":{"haso228k:10000/p09/vfc/exp.04/Value":{"ndim":null,"plot_type":1,"full_name":"tango://haso228k:10000/p09/vfc/exp.04/Value","data_type":"float64","nexus_path":"","shape":[],"conditioning":"","plot_axes":["<mov>"],"_unit_id":"0","normalization":0,"index":1,"name":"insertion_taper","instrument":null,"enabled":true,"label":"p09/vfc/exp.04/Value","source":"tango://haso228k:10000/p09/vfc/exp.04/Value","_controller_name":"__tango__","data_units":"No unit","output":true}},"id":0}}}},"description":"Measurement Group","monitor":"haso228k:10000/expchan/dgg2_exp_01/1","timer":"haso228k:10000/expchan/dgg2_exp_01/1","label":"mg_test02"}
Set:    {"controllers": {"haso228k:10000/controller/dgg2ctrl/dgg2_exp_01": {"units": {"0": {"channels": {"haso228k:10000/expchan/dgg2_exp_01/1": {"ndim": 0, "index": 0, "name": "exp_t01", "data_type": "float64", "plot_type": 0, "enabled": true, "label": "exp_t01", "instrument": null, "shape": [], "_controller_name": "haso228k:10000/controller/dgg2ctrl/dgg2_exp_01", "conditioning": "", "full_name": "haso228k:10000/expchan/dgg2_exp_01/1", "_unit_id": "0", "normalization": 0, "output": true, "plot_axes": [], "nexus_path": "", "data_units": "No unit", "source": "haso228k:10000/expchan/dgg2_exp_01/1/Value"}}, "trigger_type": 0, "id": 0, "timer": "haso228k:10000/expchan/dgg2_exp_01/1", "monitor": "haso228k:10000/expchan/dgg2_exp_01/1"}}}, "__tango__": {"units": {"0": {"channels": {"tango://haso228k:10000/p09/vfc/exp.04/Value": {"ndim": 0, "index": 1, "name": "insertion_taper", "data_type": "float64", "plot_type": 1, "enabled": true, "label": "p09/vfc/exp.04/Value", "instrument": null, "shape": [], "_controller_name": "__tango__", "conditioning": "", "full_name": "tango://haso228k:10000/p09/vfc/exp.04/Value", "_unit_id": "0", "normalization": 0, "output": true, "plot_axes": ["<mov>"], "nexus_path": "", "data_units": "No unit", "source": "haso228k:10000/p09/vfc/exp.04/Value"}}, "trigger_type": 0, "id": 0, "timer": "haso228k:10000/expchan/dgg2_exp_01/1", "monitor": "haso228k:10000/expchan/dgg2_exp_01/1"}}}}, "monitor": "haso228k:10000/expchan/dgg2_exp_01/1", "description": "Measurement Group", "timer": "haso228k:10000/expchan/dgg2_exp_01/1", "label": "mg_test02"}

[Zbigniew Reszela]

Does it mean that configuration is not backward compatible or it is just a bug? What should be a proper JSON substring for tango channels?

No, there were no backwards incompatibilities introduced in this release. So it must be a bug. Let us check it carefully and we will reply to you soon.

[Zbigniew Reszela]

The patch was sent to the develop mainling list. Jan could you please test if it fix the problem that you had.
If you confirm we will make a hotfix release that includes [#507] and [#508].

[Jan Kotanski]

Hi Zibi,

Thanks. Yes, it fixes th problem.

Cheers,
Jan

[Jan Kotanski]

Hi,

it seems that itango interferes with PyTango 8, i.e. it creates itango which already exists in pytango.

Does it me mean that the current sardana i.e. 2.1 can be used only with PyTango 9 ?

Best regards,
Jan

[Zbigniew Reszela]

The idea was to keep Sardana compatible with PyTango < 9.
For the Sardana needs itango introduced a backwards compatibility layer with the PyTango <9 (it imports stuff from PyTango.ipython when needed) - see here.

What kind of problems do you have?

[Jan Kotanski]

Hi Zibi,

The problem is the when you install itango-0.1.5 via setup.py it creates the itango script in bin directory. A similar itango script in the same location is provided by pytango 8 (python-pytango-8.1.7 on debian). So if we want to install both one overwrites another.

At DESY we cannot create packages without modifiying setup.py of itango, i.e. removing the script creation part.
Otherwise debian packages would be not compatible.

I know that it is more a bug in itango , i.e. cannot work with pytango < 8 but since sardana start to use itango it is also the problem of sardana.

Cheers,
Jan

[Zbigniew Reszela]

The same issue was raised by Frederic in this PR, but we finally ignored it. But of course we could rethink it in order to improve the current behavior (actually we should invite the PyTango people to this discussion...).

Personally I see the problem a little bit tricky cause it stays on the border of different "packaging" systems.

I have done two different types of installations:

Test 1

1. Install PyTango 8 with the official debian package - it installs itango script in /usr/bin.
2. Install itango with pip or python setup.py install - it installs itango script in /usr/local/bin.

Due to the default path precedence the newer script will be used.

Test 2

1. Install PyTango 8 with pip (pip install --egg) or python setup.py install - it installs itango script in /usr/bin/local.
2. Install itango with pip or python setup.py install - it also installs /usr/local/bin.

There is no problem since pip does not protect against overriding files and the second package installs the newer version of the script. I think that installing in the reversed order will be also ok since the itango package has PyTango as one of the install_requires hence the old script will get overridden. Anyone knows if the itango > 0.1.4 will work with the old script?

Maybe for your internal packaging you could have an option of not including the script in the itango package?

[Zbigniew Reszela]

I see that this issue already moved to itango...

[Jan Kotanski]

Hi Zibi,

ad. Test 1: for debian setup.py installs python scripts into /usr/bin as well.

ad. Test 2: But if you install in reverse order and both packages are already installed (i.e. you reinstall packages) the the last will left. If you do it automatically you may event not notice which version is on the top.

For internal packaging you can do the tricks by using some options or modify sources but for me it would be better to do it in the sources. If not It should be explicitly explain installation procedure of sardana 2.1 for pytango 8 e.g. in README.rst, i.e. that itango overrides pytango 8 and therefore one has to install itango without scripts. The current situation is really misleading.

Bests,
Jan

[Zbigniew Reszela]

Regarding the Test1. I have performed the test on debian8 docker container and it installs in /usr/local/bin. I attach the logs. Maybe the destination depends on some other software or configuration?

Regarding the Test2. The part of installing in the reversed order was just my assumption. I really don't know how would it behave in this particular case - I don't know pip enough.

And finally regarding the docs, I totally agree with you. If finally the itango will stay as it is (and from the above mentioned issue I see some inclination into it) we should add a note about the risk of overriding scripts when installing itango dependency. We don't have README.rst file but I think that puting it here is also ok. What do you think? There is already an issue about updating this docs [#426]. I hopefully will have enough time to do it before we release the hotfix 2.1.1...

[Jan Kotanski]

Hi Zibi,

I've talked about it with Vincent at

https://github.com/tango-cs/itango/issues/8

One solution is to slit itango into seperate branches (one for itango for PyTango 8) and (another for itango for PyTango 9) with a different version numbers.

Do you see in this solutions any problems from the point of view of sardana?

Bests,
Jan

[Zbigniew Reszela]

No, I think it is ok for us. More details in this post

[Zbigniew Reszela]

Integrated with hotfix 2.1.1 release.