[Roundup-devel] Continuous Builds [was Code clean-ups & questions]

[John K. <jo...@je...>]

On 20/05/14 01:46, anatoly techtonik wrote:
>>> I'd actually add drone.io integration with status of continuous build,
>>> uploaded code coverage.. but I am unlikely to switch to this task
>>> anytime soon.
>>>
>>> Uploaded coverage would simplify reviews.
>>
>> I am happy to have a look at setting up the test builds with drone.io - I
>> may need to be added to the 'roundup' bitbucket team though. I'm not sure
>> how to handle the coverage stuff though, I assume the reports would need to
>> be uploaded to somewhere else?
>
> Added you to `roundup` team. Note that this is still a mirror.
> I think drones can upload stuff to deploy servers, so it should
> be possible to upload reports even to
> http://roundup.sourceforge.net/

I had a look at setting up continuous builds with drone.io but ran into 
a couple of issues.

The main issue is that they have a hard limit of 15 minutes for "builds" 
and enabling the PostgreSQL tests exceeds this limit (even without the 
MySQL tests). Enabling the MySQL tests without the PostgreSQL tests take 
~10 minutes to run. So it is looking like drone.io isn't really an 
option unless we find some way to dramatically reduce the time it takes 
to run the tests. It also seems to rule out being able to use tox to run 
the test under multiple python versions (which would be handy if a 
python 3 migration is ever to be considered)

For the time being I still have the drone.io instance set up to run the 
default and MySQL tests under python2.6:

   https://drone.io/bitbucket.org/roundup/roundup


A second (minor) issue that arose is that the 'run_tests.py' script 
doesn't seem to return an error code when a test fails, so drone.io 
would return a build success even if a test failed. This could be an 
issue with drone.io or more likely an issue with the 'run_tests.py' 
script. This is something that will need to be looked into further.


I've had a quick look at some other platforms but they also seem to have 
issues:

codeship.io
   - doesn't provide root access, so it doesn't seem that you can 
install extra versions of python to test against
   - committing a change to the repository seems to be the only way to 
trigger a build, which makes it a bit awkward to debug

shippable.com
  - requires a shippable.yaml file to define the build environment, so 
it would require a bit more work to create a correct one of those.


If anyone has any other suggestions I'm happy to have another look if 
some free time comes along.

SeeYa,
John.