Menu
▾
▴
Re: [scipion-users] Spider MDA Workflow not working again
|
From: Hutchison, M. (NIH/N. [C] <hut...@ma...> - 2018-04-25 17:02:51
|
Have you downloaded the appropriate test data set? The test data sets aren't included with the github distribution and are downloaded when you try to run the test (or manually separately from running the test). If your user doesn't have write to scipion/data/tests you'll see this error in the stdout of the import stage as a result of failure to download, which leads to a failure to import the raw data for testing. You should also see a more descriptive error in your terminal that tells you that you have a permissions error (see below for the equivalent when run in a similar scenario on our system). Can you run `/opt/scipion/scipion testdata --list` and confirm that the dataset shows up as a local dataset?
If it doesn't, try running `/opt/scipion/scipion testdata --download hemoglobin_mda` (which should be the correct dataset for this test) as a user that can write to scipion/data/tests. Then rerun the relevant test.
To answer your last point, the test project and all processing data will be created in $SCIPION_USER_DATA from the config file (by default ~/ScipionUserData), but the raw test dataset writes to the scipion installation directory.
My terminal output after replicating this type of failure (the critical issue is the OSError permissions failure):
>>>>> python scripts/run_tests.py em.workflows.test_workflow_spiderMDA
Running tests....
('Creating project at: ', '/mnt/nas/Scipion/projects/TestSpiderConvert/project.sqlite')
>>>> python /usr/local/scipion/scipion testdata --download hemoglobin_mda
/usr/local/scipion/config/scipion.conf /usr/local/scipion/config/scipion.conf
Scipion v1.2 (2018-04-02) Caligula
>>>>> python scripts/sync_data.py --download hemoglobin_mda
Selected datasets: hemoglobin_mda
Dataset hemoglobin_mda not in local machine. Downloading...
Traceback (most recent call last):
File "scripts/sync_data.py", line 440, in <module>
main()
File "scripts/sync_data.py", line 128, in main
download(dataset, url=args.url, verbose=args.verbose)
File "scripts/sync_data.py", line 257, in download
os.makedirs(datasetFolder)
File "/usr/local/scipion/software/lib/python2.7/os.py", line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/usr/local/scipion/data/tests/hemoglobin_mda'
** Running command: 'python /usr/local/scipion/scipion runprotocol pw_protocol_run.py "/mnt/nas/Scipion/projects/TestSpiderConvert" "Runs/000002_ProtImportParticles/logs/run.db" 2'
/usr/local/scipion/config/scipion.conf /usr/local/scipion/config/scipion.conf
Scipion v1.2 (2018-04-02) Caligula
>>>>> python /usr/local/scipion/pyworkflow/apps/pw_protocol_run.py "/mnt/nas/Scipion/projects/TestSpiderConvert" "Runs/000002_ProtImportParticles/logs/run.db" "2"
>>> ERROR running protocol scipion - import particles
[ FAILED ] TestSpiderConvert.test_convert
Traceback (most recent call last):
File "/usr/local/scipion/software/lib/python2.7/unittest/case.py", line 329, in run
testMethod()
File "/usr/local/scipion/pyworkflow/tests/em/workflows/test_workflow_spiderMDA.py", line 52, in test_convert
self.launchProtocol(protImport)
File "/usr/local/scipion/pyworkflow/tests/tests.py", line 106, in launchProtocol
raise Exception("ERROR: Protocol not finished")
Exception: ERROR: Protocol not finished
('Creating project at: ', '/mnt/nas/Scipion/projects/TestSpiderWorkflow/project.sqlite')
>>>> python /usr/local/scipion/scipion testdata --download hemoglobin_mda
/usr/local/scipion/config/scipion.conf /usr/local/scipion/config/scipion.conf
Scipion v1.2 (2018-04-02) Caligula
>>>>> python scripts/sync_data.py --download hemoglobin_mda
Selected datasets: hemoglobin_mda
Dataset hemoglobin_mda not in local machine. Downloading...
Traceback (most recent call last):
File "scripts/sync_data.py", line 440, in <module>
main()
File "scripts/sync_data.py", line 128, in main
download(dataset, url=args.url, verbose=args.verbose)
File "scripts/sync_data.py", line 257, in download
os.makedirs(datasetFolder)
File "/usr/local/scipion/software/lib/python2.7/os.py", line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/usr/local/scipion/data/tests/hemoglobin_mda'
** Running command: 'python /usr/local/scipion/scipion runprotocol pw_protocol_run.py "/mnt/nas/Scipion/projects/TestSpiderWorkflow" "Runs/000002_ProtImportParticles/logs/run.db" 2'
/usr/local/scipion/config/scipion.conf /usr/local/scipion/config/scipion.conf
Scipion v1.2 (2018-04-02) Caligula
>>>>> python /usr/local/scipion/pyworkflow/apps/pw_protocol_run.py "/mnt/nas/Scipion/projects/TestSpiderWorkflow" "Runs/000002_ProtImportParticles/logs/run.db" "2"
>>> ERROR running protocol scipion - import particles
[ FAILED ] TestSpiderWorkflow.test_mdaWorkflow
Traceback (most recent call last):
File "/usr/local/scipion/software/lib/python2.7/unittest/case.py", line 329, in run
testMethod()
File "/usr/local/scipion/pyworkflow/tests/em/workflows/test_workflow_spiderMDA.py", line 96, in test_mdaWorkflow
self.launchProtocol(protImport)
File "/usr/local/scipion/pyworkflow/tests/tests.py", line 106, in launchProtocol
raise Exception("ERROR: Protocol not finished")
Exception: ERROR: Protocol not finished
[==========] run 2 tests (2.991 secs)
[ FAILED ] 2 tests
[ PASSED ] 0 tests
On 4/25/18, 12:38 PM, "Patrick Goetz" <pg...@ma...> wrote:
Hi -
Sorry for continuously spamming the list, but the Spider MDA Workflow
test isn't working again, I just want to make sure I haven't introduced
structural installation problems.
First, I re-installed Scipion in /opt so that the users can access it,
and made sure to install the spider package. Now running scipion as an
ordinary user:
/opt/scipion/scipion tests em.workflows.test_workflow_spiderMDA
I get error messages and "2 Tests failed" again. Snooping around in the
locally created ScipionUserData folder, I found this note in
~/ScipionUserData/projects/TestSpiderWorkflow/Runs/000002_ProtImportParticles/logs/run.stdout:
-------
Traceback (most recent call last):
File "/opt/scipion/pyworkflow/protocol/protocol.py", line 186, in run
self._run()
File "/opt/scipion/pyworkflow/protocol/protocol.py", line 1104, in _run
'Protocol.run: Validation errors:\n' + '\n'.join(errors))
Exception: Protocol.run: Validation errors:
There are no files matching the pattern
/opt/scipion/data/tests/hemoglobin_mda/particles/*.spi
-------
So apparently the test is trying to install data to /opt/scipion/data
rather than ~/ScipionUserData ?
Am I missing a configuration step that would instruct the system to
process all data in the user's ScipionUserData folder?
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
scipion-users mailing list
sci...@li...
https://lists.sourceforge.net/lists/listinfo/scipion-users
|
