[tauruslib-devel] Taurus Continuous Integration and Documentation (was Re: Taurus-CI)

[Carlos P. <cpa...@ce...>]

Hi,

Note: this is a reply to a private email from Vincent Michel, but it touches a 
topic that I wanted to open for public discussion, so I CC'ed the taurus-devel 
list. Vincent: I hope you do not mind it...

On Tuesday, October 25, 2016 10:29:05 AM CEST Vincent Michel wrote:
> Last week, the solaris guys told me you were planning to move the taurus
> documentation from readthedocs to github-pages. I'm a bit surprised
> since I've been very satisfied with readthedocs so far.

The reason we were dissatisfied with readthedocs is mainly that we have to 
maintain mocks for many of our dependencies: when we started, there was no 
conda support, so we were forced to develop mocks for many of our dependencies 
(PyQt, Tango, ...). 

For this we developed the <taurus>/doc/buildmock.py script. And since we were 
at it, we also included several other modules (even if they were pip-
installable) in order to save build time (e.g. numpy,...). The buildmock.py 
basically creates a zip with mock modules that, when inserted into the 
pythonpath, allows us to build the docs.

This solution "sort of works"  but it is a PITA when one wants to update or 
change a dependency because usually some manual tweaking (and therefore 
testing) is required.

Now, with the Conda support (and with the work you did in getting the stack 
for PyTango in Conda), I guess we could get rid of most of the mocks, but 
probably not all of them (I haven't checked, but I do not think that e.g. 
epics and pyepics will have cunda packages).

I am also concerned in that, when we finally manage to provide proper plugin 
support and third parties start providing plugins for Taurus (e.g. a new 
scheme for some exotic control system), we will need to support more 
dependencies if we want to document the new additions. And  the odds of them 
being available as Conda packages is less than them being installable in a 
Docker container.

Furthermore, we *already* have docker containers with all the dependencies 
installed [1,2], since we use them for Travis-CI (more about this later). 
Also, the work of having the container updated with new dependencies will need 
to be done anyway because we will want to run automated tests on the new 
features as well.

And, since we *plan* to use Travis for Continuous Deployment of packages, it 
seems natural to make the docs just another artifact from the Travis builds 
and have them delivered to e.g. github pages or pythonhosted, or wherever.

Building the docs in Travis with docker also has the advantage of considering 
the success in building the docs as one more test during the CI.

Finally, one extra advantage that I see from the Docker approach is that 
debugging problems in the doc build becomes simpler, since you can run locally 
exactly the same container that Travis is using.

Note that the ideal solution for us would be that ReadTheDocs supported 
dockers..., but we approached the guy behind RTD and he said that this would 
not be the case in a near future (he said they may use docker containers 
internally to provide some customised machines, but are not considering for 
now giving users the possibility of running their own containers)

Note: If I get some possitive feedback for this option, I could write a TEP...


> Also, I've been working on the pytango CI for the last few months and
> since our projects are quite similar, I thought it would be good to
> compare our CI solutions. Taurus seems to embrace docker while pytango
> is using conda environments, so I'd like to hear about your view on that
> matter ;)

Regarding the use of Docker in Travis instead of using conda directly on top 
of the native Ubuntu  VM provided by Travis, here are some reasons:

- For us, the main reason for going "the docker way" is that it enables us to 
introduce CI tests for configurations that closely match those used by 
different institutions: e.g. if MaxIV was interested in having every commit 
tested on  a system like the ones used in production, all that would be 
required would be for MaxIV to provide the link to a suitable  container in 
dockerhub. This way we could create a large set of test environments while 
keeping the maintenance work distributed (each interested party provides and 
updates its own container(s)).

- Another practical advantage: we can use the same containers outside of 
Travis for our own work: e.g. for debugging, for demos, for running local 
tests in a well-controlled environment, and even for packaging or for running 
internal CI servers with jenkins. The debugging aspect is specially 
interesting: if a user reports a difficult-to-reproduce bug, you can ask him/
her to reproduce it on one of the "official" containers. 

- One aspect that may also be interesting for PyTango (and certainly for 
Taurus): with docker, you can instantiate more than one container in a single 
run and do tests where network communication plays a role. We haven't got 
there yet, but it seems important.

- Also, I think I read that Travis wanted to encourage docker-based  builds, 
even by giving you some more resources if you used Docker... but I cannot find 
a link for that, so maybe I dreamt it...

I hope I managed to cover your questions. Now I would be very interested in 
learning about your reasons, choices and concerns as well. I am sure there is 
a lot of experience than we can share.

Cheers, 

Carlos

[1] https://hub.docker.com/r/cpascual/taurus-test/
[2] https://hub.docker.com/r/reszelaz/sardana-test/

-- 
+----------------------------------------------------+
 Carlos Pascual Izarra
 Scientific Software Coordinator
 Computing Division
 ALBA Synchrotron  [http://www.albasynchrotron.es]
 Carrer de la Llum 2-26
 E-08290 Cerdanyola del Valles (Barcelona), Spain
 E-mail: cpa...@ce...
 Phone: +34 93 592 4428
+----------------------------------------------------+