MOCAT - 2017-07-27

There are a couple of known bugs in some of the scripts. I advise to look
through the various bug reports on this site. I also suggest you to in each
command individually and look at the output log files under
logs/---/samples.

Unfortunately I don't currently have the bandwidth to look into these
issues further.

On Thu, Jul 27, 2017 at 11:56 Alexandre B. de Menezes ademenez@users.sf.net
wrote:

[tickets:#65] Pipeline installation error

Status: New
Created: Thu Jul 27, 2017 03:56 PM UTC by Alexandre B. de Menezes
Last Updated: Thu Jul 27, 2017 03:56 PM UTC
Owner: nobody

Hi,

I am trying use MOCAT, however when I run the scripts to analyse the demo
datasets in the article_datasets folder I fail to get a result. The usearch
and metageneark executables were added to the ext folder. Would you have an
idea about what is the problem with my installation? I had problems with
the 87_species and the simulated_metagenome_100strains datasets. I copied
the log from my test of the 87_species dataset below:

alexandre@FourStrings:~/MOCAT/article_datasets/simulated_metagenome_87$
sh GENERATE_MOTU_AND_TAX_PROFILES.sh

===============================================================================
MOCAT - Metagenomics Analysis Toolkit
v2.0.1
by Jens Roat Kultima, Luis Pedro Coelho, Shinichi Sunagawa @ Bork Group,
EMBL

===============================================================================
EXECUTE COMMAND : -sf sample -rtf
CURRENT USER : alexandre
CURRENT HOST : FourStrings
WORKING DIRECTORY : ~/MOCAT/article_datasets/simulated_metagenome_87
CONFIG FILE :
~/MOCAT/article_datasets/simulated_metagenome_87/MOCAT.cfg
NUMBER OF SAMPLES : 1
SAMPLE NAMES : 87_species
SAMPLE FILE : sample

===============================================================================
MOCAT_LSF_max_concurrent :
MOCAT_LSF_memory_limit :
MOCAT_LSF_qsub_add_param :
MOCAT_LSF_queue :
MOCAT_PBS_qsub_add_param :
MOCAT_SGE_parallell_env : smp
MOCAT_SGE_qsub_add_param :
MOCAT_data_dir : ~/MOCAT//data
MOCAT_data_type : fastx
MOCAT_default_reads : reads.processed
MOCAT_dir : ~/MOCAT/
MOCAT_mapping_mode : allbest
MOCAT_paired_end : yes
MOCAT_pre_execute :
MOCAT_prompt_before_run : no
MOCAT_qsub_system : none
MOCAT_record_script_versions : no
MOCAT_rownames_dir : ~/MOCAT//data
MOCAT_umask : 0002
MOCAT_zip_level : 9
MOCAT_zip_program : pigz

===============================================================================
readtrimfilter_length_cutoff : 45
readtrimfilter_qual_cutoff : 20
readtrimfilter_trim_5_prime : yes
readtrimfilter_use_precalc_5prime_trimming : no
readtrimfilter_use_sanger_scale : auto

===============================================================================
Thu Jul 27 13:27:25 2017: PERFORMING READ TRIM AND FILTER...
Thu Jul 27 13:27:25 2017: Creating readtrimfilter jobs... OK!
Thu Jul 27 13:27:25 2017: Temp directory is
~/MOCAT/article_datasets/simulated_metagenome_87/SAMPLE/temp
Thu Jul 27 13:27:25 2017: Job Identifier is sample_2017Jul27_132725
Thu Jul 27 13:27:25 2017: Executing readtrimfilter jobs...
Thu Jul 27 13:33:23 2017: Execution completed successfully.
Thu Jul 27 13:33:23 2017: Checking files... OK!
Thu Jul 27 13:33:23 2017: COMPLETED READ TRIM AND FILTER.
Files stored in
~/MOCAT/article_datasets/simulated_metagenome_87/<SAMPLE>/reads.processed.fastx/[pair|single].fq.gz

===============================================================================
COMPLETED ALL TASKS SUCCESSFULLY!

===============================================================================

===============================================================================
MOCAT - Metagenomics Analysis Toolkit
v2.0.1
by Jens Roat Kultima, Luis Pedro Coelho, Shinichi Sunagawa @ Bork Group,
EMBL

===============================================================================
EXECUTE COMMAND : -sf sample -s mOTU.v1.padded -r reads.processed
-identity 97 -extracted_files
CURRENT USER : alexandre
CURRENT HOST : FourStrings
WORKING DIRECTORY : ~/MOCAT/article_datasets/simulated_metagenome_87
CONFIG FILE :
~/MOCAT/article_datasets/simulated_metagenome_87/MOCAT.cfg
NUMBER OF SAMPLES : 1
SAMPLE NAMES : 87_species
SAMPLE FILE : sample

===============================================================================
MOCAT_LSF_max_concurrent :
MOCAT_LSF_memory_limit :
MOCAT_LSF_qsub_add_param :
MOCAT_LSF_queue :
MOCAT_PBS_qsub_add_param :
MOCAT_SGE_parallell_env : smp
MOCAT_SGE_qsub_add_param :
MOCAT_data_dir : ~/MOCAT//data
MOCAT_data_type : fastx
MOCAT_default_reads : reads.processed
MOCAT_dir : ~/MOCAT/
MOCAT_mapping_mode : allbest
MOCAT_paired_end : yes
MOCAT_pre_execute :
MOCAT_prompt_before_run : no
MOCAT_qsub_system : none
MOCAT_record_script_versions : no
MOCAT_rownames_dir : ~/MOCAT//data
MOCAT_umask : 0002
MOCAT_zip_level : 9
MOCAT_zip_program : pigz

===============================================================================
screen_fasta_file_additional_usearch_cmd :
screen_fasta_file_blast_e_value : 0.00001
screen_fasta_file_blast_read_min_length : 10
screen_fasta_file_usearch_version : 5
screen_fasta_file_usearch_version_5_exe : usearch
screen_fasta_file_usearch_version_6_exe : usearch6.0.307_i86linux32
screen_length_cutoff : 45
screen_percent_cutoff : 97
screen_save_format : soap
screen_soap_cmd : -M 4
screen_soap_max_mm : 10
screen_soap_seed_length : 30

===============================================================================
Thu Jul 27 13:33:23 2017: PERFORMING SCREEN USING DATABASE...
Thu Jul 27 13:33:23 2017: Checking files... OK!
Thu Jul 27 13:33:23 2017: Creating screen jobs...
Thu Jul 27 13:33:23 2017: DATABASE status: Found mOTU.v1.padded.index in
~/MOCAT//data = ALL OK!
Thu Jul 27 13:33:23 2017: Continuing creating screen jobs...
Thu Jul 27 13:33:23 2017: Created jobs!
Thu Jul 27 13:33:23 2017: Temp directory is
~/MOCAT/article_datasets/simulated_metagenome_87/SAMPLE/temp
Thu Jul 27 13:33:23 2017: Job Identifier is sample_2017Jul27_133323
Thu Jul 27 13:33:23 2017: Executing screen jobs...
Thu Jul 27 13:44:41 2017: Execution completed successfully.
Thu Jul 27 13:44:41 2017: Checking files... OK!
Thu Jul 27 13:44:41 2017: COMPLETED SCREEN USING DATABASE.
Files stored in
~/MOCAT/article_datasets/simulated_metagenome_87/<SAMPLE>/reads.extracted.mOTU.v1.padded.fastx/[pair|single]*.fq.gz
Files stored in
~/MOCAT/article_datasets/simulated_metagenome_87/<SAMPLE>/reads.mapped.mOTU.v1.padded.fastx/

===============================================================================
COMPLETED ALL TASKS SUCCESSFULLY!

===============================================================================

===============================================================================
MOCAT - Metagenomics Analysis Toolkit
v2.0.1
by Jens Roat Kultima, Luis Pedro Coelho, Shinichi Sunagawa @ Bork Group,
EMBL

===============================================================================
EXECUTE COMMAND : -sf sample -f mOTU.v1.padded -r reads.processed
-identity 97
CURRENT USER : alexandre
CURRENT HOST : FourStrings
WORKING DIRECTORY : ~/MOCAT/article_datasets/simulated_metagenome_87
CONFIG FILE :
~/MOCAT/article_datasets/simulated_metagenome_87/MOCAT.cfg
NUMBER OF SAMPLES : 1
SAMPLE NAMES : 87_species
SAMPLE FILE : sample

===============================================================================
MOCAT_LSF_max_concurrent :
MOCAT_LSF_memory_limit :
MOCAT_LSF_qsub_add_param :
MOCAT_LSF_queue :
MOCAT_PBS_qsub_add_param :
MOCAT_SGE_parallell_env : smp
MOCAT_SGE_qsub_add_param :
MOCAT_data_dir : ~/MOCAT//data
MOCAT_data_type : fastx
MOCAT_default_reads : reads.processed
MOCAT_dir : ~/MOCAT/
MOCAT_mapping_mode : allbest
MOCAT_paired_end : yes
MOCAT_pre_execute :
MOCAT_prompt_before_run : no
MOCAT_qsub_system : none
MOCAT_record_script_versions : no
MOCAT_rownames_dir : ~/MOCAT//data
MOCAT_umask : 0002
MOCAT_zip_level : 9
MOCAT_zip_program : pigz

===============================================================================
filter_length_cutoff : 45
filter_make_unique_sorted : no
filter_paired_end_filtering : yes
filter_percent_cutoff : 97
filter_psort_buffer : 2G
filter_remove_mapping_files : yes
filter_samtools_memory : 50000000000

===============================================================================
Thu Jul 27 13:44:42 2017: PERFORMING FILTERING...
Thu Jul 27 13:44:42 2017: Creating filter jobs... OK!
Thu Jul 27 13:44:42 2017: Job Identifier is sample_2017Jul27_134442
Thu Jul 27 13:44:42 2017: Executing filter jobs...
Thu Jul 27 13:46:36 2017: Execution completed successfully.
Thu Jul 27 13:46:36 2017: COMPLETED FILTERING.

===============================================================================
COMPLETED ALL TASKS SUCCESSFULLY!

===============================================================================

===============================================================================
MOCAT - Metagenomics Analysis Toolkit
v2.0.1
by Jens Roat Kultima, Luis Pedro Coelho, Shinichi Sunagawa @ Bork Group,
EMBL

===============================================================================
EXECUTE COMMAND : -sf sample -p mOTU.v1.padded -r reads.processed
-identity 97 -mode mOTU -o RESULTS
CURRENT USER : alexandre
CURRENT HOST : FourStrings
WORKING DIRECTORY : ~/MOCAT/article_datasets/simulated_metagenome_87
CONFIG FILE :
~/MOCAT/article_datasets/simulated_metagenome_87/MOCAT.cfg
NUMBER OF SAMPLES : 1
SAMPLE NAMES : 87_species
SAMPLE FILE : sample

===============================================================================
MOCAT_LSF_max_concurrent :
MOCAT_LSF_memory_limit :
MOCAT_LSF_qsub_add_param :
MOCAT_LSF_queue :
MOCAT_PBS_qsub_add_param :
MOCAT_SGE_parallell_env : smp
MOCAT_SGE_qsub_add_param :
MOCAT_data_dir : ~/MOCAT//data
MOCAT_data_type : fastx
MOCAT_default_reads : reads.processed
MOCAT_dir : ~/MOCAT/
MOCAT_mapping_mode : allbest
MOCAT_paired_end : yes
MOCAT_pre_execute :
MOCAT_prompt_before_run : no
MOCAT_qsub_system : none
MOCAT_record_script_versions : no
MOCAT_rownames_dir : ~/MOCAT//data
MOCAT_umask : 0002
MOCAT_zip_level : 9
MOCAT_zip_program : pigz

===============================================================================
profiling_PRINT_ALL_INSERT : no
profiling_PRINT_COUNT_TYPE :
profiling_PRINT_FUNCTIONAL_LEVELS :
profiling_PRINT_NCBI_LEVELS :
profiling_PRINT_NORM_TYPES :
profiling_PRINT_ONLY_THESE_FUNCTIONAL_LEVELS :
profiling_PRINT_ONLY_THESE_NCBI_LEVELS :
profiling_paired_end_filtering : yes

===============================================================================
Thu Jul 27 13:46:36 2017: PERFORMING PROFILING...
Thu Jul 27 13:46:36 2017: Creating profiling jobs...
Thu Jul 27 13:46:36 2017: Checking mode... OK!
Thu Jul 27 13:46:36 2017: Job Identifier is sample_2017Jul27_134636
Thu Jul 27 13:46:36 2017: Executing profiling jobs...
ERROR & EXIT: Execution failed.
Log files from this run are stored in the folder
~/MOCAT/article_datasets/simulated_metagenome_87/logs/profiling at
~/MOCAT/src/MOCATCore.pm line 842.

===============================================================================
MOCAT - Metagenomics Analysis Toolkit
v2.0.1
by Jens Roat Kultima, Luis Pedro Coelho, Shinichi Sunagawa @ Bork Group,
EMBL

===============================================================================
EXECUTE COMMAND : -sf sample -s RefMG.v1.padded -r mOTU.v1.padded -e
-identity 97
CURRENT USER : alexandre
CURRENT HOST : FourStrings
WORKING DIRECTORY : ~/MOCAT/article_datasets/simulated_metagenome_87
CONFIG FILE :
~/MOCAT/article_datasets/simulated_metagenome_87/MOCAT.cfg
NUMBER OF SAMPLES : 1
SAMPLE NAMES : 87_species
SAMPLE FILE : sample

===============================================================================
MOCAT_LSF_max_concurrent :
MOCAT_LSF_memory_limit :
MOCAT_LSF_qsub_add_param :
MOCAT_LSF_queue :
MOCAT_PBS_qsub_add_param :
MOCAT_SGE_parallell_env : smp
MOCAT_SGE_qsub_add_param :
MOCAT_data_dir : ~/MOCAT//data
MOCAT_data_type : fastx
MOCAT_default_reads : reads.processed
MOCAT_dir : ~/MOCAT/
MOCAT_mapping_mode : allbest
MOCAT_paired_end : yes
MOCAT_pre_execute :
MOCAT_prompt_before_run : no
MOCAT_qsub_system : none
MOCAT_record_script_versions : no
MOCAT_rownames_dir : ~/MOCAT//data
MOCAT_umask : 0002
MOCAT_zip_level : 9
MOCAT_zip_program : pigz

===============================================================================
screen_fasta_file_additional_usearch_cmd :
screen_fasta_file_blast_e_value : 0.00001
screen_fasta_file_blast_read_min_length : 10
screen_fasta_file_usearch_version : 5
screen_fasta_file_usearch_version_5_exe : usearch
screen_fasta_file_usearch_version_6_exe : usearch6.0.307_i86linux32
screen_length_cutoff : 45
screen_percent_cutoff : 97
screen_save_format : soap
screen_soap_cmd : -M 4
screen_soap_max_mm : 10
screen_soap_seed_length : 30

===============================================================================
Thu Jul 27 13:46:37 2017: PERFORMING SCREEN USING DATABASE...
Thu Jul 27 13:46:37 2017: Checking files... OK!
Thu Jul 27 13:46:37 2017: Creating screen jobs...
Thu Jul 27 13:46:37 2017: DATABASE status: Found RefMG.v1.padded.index* in
~/MOCAT//data = ALL OK!
Thu Jul 27 13:46:37 2017: Continuing creating screen jobs...
Thu Jul 27 13:46:37 2017: Created jobs!
Thu Jul 27 13:46:37 2017: Temp directory is
~/MOCAT/article_datasets/simulated_metagenome_87/SAMPLE/temp
Thu Jul 27 13:46:37 2017: Job Identifier is sample_2017Jul27_134637
Thu Jul 27 13:46:37 2017: Executing screen jobs...
Thu Jul 27 13:47:13 2017: Execution completed successfully.
Thu Jul 27 13:47:13 2017: Checking files... OK!
Thu Jul 27 13:47:13 2017: COMPLETED SCREEN USING DATABASE.
Files stored in
~/MOCAT/article_datasets/simulated_metagenome_87/<SAMPLE>/reads.mapped.RefMG.v1.padded.fastx/

===============================================================================
COMPLETED ALL TASKS SUCCESSFULLY!

===============================================================================

===============================================================================
MOCAT - Metagenomics Analysis Toolkit
v2.0.1
by Jens Roat Kultima, Luis Pedro Coelho, Shinichi Sunagawa @ Bork Group,
EMBL

===============================================================================
EXECUTE COMMAND : -sf sample -f RefMG.v1.padded -r mOTU.v1.padded -e
-identity 97
CURRENT USER : alexandre
CURRENT HOST : FourStrings
WORKING DIRECTORY : ~/MOCAT/article_datasets/simulated_metagenome_87
CONFIG FILE :
~/MOCAT/article_datasets/simulated_metagenome_87/MOCAT.cfg
NUMBER OF SAMPLES : 1
SAMPLE NAMES : 87_species
SAMPLE FILE : sample

===============================================================================
MOCAT_LSF_max_concurrent :
MOCAT_LSF_memory_limit :
MOCAT_LSF_qsub_add_param :
MOCAT_LSF_queue :
MOCAT_PBS_qsub_add_param :
MOCAT_SGE_parallell_env : smp
MOCAT_SGE_qsub_add_param :
MOCAT_data_dir : ~/MOCAT//data
MOCAT_data_type : fastx
MOCAT_default_reads : reads.processed
MOCAT_dir : ~/MOCAT/
MOCAT_mapping_mode : allbest
MOCAT_paired_end : yes
MOCAT_pre_execute :
MOCAT_prompt_before_run : no
MOCAT_qsub_system : none
MOCAT_record_script_versions : no
MOCAT_rownames_dir : ~/MOCAT//data
MOCAT_umask : 0002
MOCAT_zip_level : 9
MOCAT_zip_program : pigz

===============================================================================
filter_length_cutoff : 45
filter_make_unique_sorted : no
filter_paired_end_filtering : yes
filter_percent_cutoff : 97
filter_psort_buffer : 2G
filter_remove_mapping_files : yes
filter_samtools_memory : 50000000000

===============================================================================
Thu Jul 27 13:47:13 2017: PERFORMING FILTERING...
Thu Jul 27 13:47:13 2017: Creating filter jobs... OK!
Thu Jul 27 13:47:13 2017: Job Identifier is sample_2017Jul27_134713
Thu Jul 27 13:47:13 2017: Executing filter jobs...
Thu Jul 27 13:47:20 2017: Execution completed successfully.
Thu Jul 27 13:47:20 2017: COMPLETED FILTERING.

===============================================================================
COMPLETED ALL TASKS SUCCESSFULLY!

===============================================================================

===============================================================================
MOCAT - Metagenomics Analysis Toolkit
v2.0.1
by Jens Roat Kultima, Luis Pedro Coelho, Shinichi Sunagawa @ Bork Group,
EMBL

===============================================================================
EXECUTE COMMAND : -sf sample -p RefMG.v1.padded -r mOTU.v1.padded -e
-identity 97 -mode RefMG -previous_db_calc_tax_stats_file -o RESULTS
CURRENT USER : alexandre
CURRENT HOST : FourStrings
WORKING DIRECTORY : ~/MOCAT/article_datasets/simulated_metagenome_87
CONFIG FILE :
~/MOCAT/article_datasets/simulated_metagenome_87/MOCAT.cfg
NUMBER OF SAMPLES : 1
SAMPLE NAMES : 87_species
SAMPLE FILE : sample

===============================================================================
MOCAT_LSF_max_concurrent :
MOCAT_LSF_memory_limit :
MOCAT_LSF_qsub_add_param :
MOCAT_LSF_queue :
MOCAT_PBS_qsub_add_param :
MOCAT_SGE_parallell_env : smp
MOCAT_SGE_qsub_add_param :
MOCAT_data_dir : ~/MOCAT//data
MOCAT_data_type : fastx
MOCAT_default_reads : reads.processed
MOCAT_dir : ~/MOCAT/
MOCAT_mapping_mode : allbest
MOCAT_paired_end : yes
MOCAT_pre_execute :
MOCAT_prompt_before_run : no
MOCAT_qsub_system : none
MOCAT_record_script_versions : no
MOCAT_rownames_dir : ~/MOCAT//data
MOCAT_umask : 0002
MOCAT_zip_level : 9
MOCAT_zip_program : pigz

===============================================================================
profiling_PRINT_ALL_INSERT : no
profiling_PRINT_COUNT_TYPE :
profiling_PRINT_FUNCTIONAL_LEVELS :
profiling_PRINT_NCBI_LEVELS :
profiling_PRINT_NORM_TYPES :
profiling_PRINT_ONLY_THESE_FUNCTIONAL_LEVELS :
profiling_PRINT_ONLY_THESE_NCBI_LEVELS :
profiling_paired_end_filtering : yes

===============================================================================
Thu Jul 27 13:47:21 2017: PERFORMING PROFILING...
Thu Jul 27 13:47:21 2017: Creating profiling jobs...
Thu Jul 27 13:47:21 2017: Checking mode... OK!
ERROR & EXIT: Missing input stats file
~/MOCAT/article_datasets/simulated_metagenome_87/87_species/stats/87_species.extracted.mOTU.v1.padded.after.PE.filter.and.within.padded.region.allbest.fastx.stats
at ~/MOCAT/src/MOCATProfiling.pm line 325.

The generated mOTU and taxonomic profiles should be available in:
RESULTS

Sent from sourceforge.net because you indicated interest in <
https://sourceforge.net/p/mocat/tickets/65/>

To unsubscribe from further messages, please visit <
https://sourceforge.net/auth/subscriptions/>

--
Sent from my iPhone - apologies for any iCorrect mistakes.

 

Related

Tickets: #65