[scipion-users] CTFFind4 failing on large datasets

[Anastasina, M. <mar...@he...>]

Dear Scipion users,

I’m get an unexpected error when running CTFFind4 in Scipion 3.0.8 on a large (50K) set of micrographs. The protocol fails after processing about 70 % of micrographs with the error: “sqlite3.OperationalError: database is locked”.

Sometimes re-launching in continuation mode helps to process few more micrographs, but the protocol will eventually fail again with the same error. Breaking the data set in smaller chunks (about 10K micrographs each) helps and the protocol ran in the same conditions succeeds for all the micrographs. 

I wonder does this indicate that CTFFind4 has problems handling large datasets and should be used on smaller datasets? Or is there possibly another problem that I am not noticing? I’m not attaching run.stdout as it is pretty large (31MB), if need I can upload it to eg grdive.

Thank you,
Maria