#2 mkdir: org.apache.hadoop.security.AccessControlException: Permission denied: user=myusername, access=WRITE, inode="":hadoop:supergroup:rwxr-xr-x Error could not create input directory /distmap_311014_input folder on hdfs file system

[Anonymous]

Originally created by: Chefa... (code.google.com)@gmail.com

What steps will reproduce the problem?

The manual.

1. Build bwa-0.5.8c
2. Downloading distmap and picard
3. Running an example distmap like in manual's 5.1:

:~$ DistMap_v1.0/distmap --reference-fast data/distmap/reference.fasta --input "data/distmap/read_PE_1.fq,data/distmap/read_PE_2.fq" --mapper bwa --mapper-path ~/bwa-0.5.8c/bwa --picard-mergesamfiles-jar ~/picard_installation/MergeSamFiles.jar --picard-sortsam-jar ~/picard_installation/SortSam.jar --output data/distmap_311014 --hadoop-home /usr/local/hadoop103/

What is the expected output? What do you see instead?

Hadoop permission error:

perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
    LANGUAGE = (unset),
    LC_ALL = (unset),
    LC_PAPER = "el_GR.UTF-8",
    LC_ADDRESS = "el_GR.UTF-8",
    LC_MONETARY = "el_GR.UTF-8",
    LC_NUMERIC = "el_GR.UTF-8",
    LC_TELEPHONE = "el_GR.UTF-8",
    LC_IDENTIFICATION = "el_GR.UTF-8",
    LC_MEASUREMENT = "el_GR.UTF-8",
    LC_TIME = "el_GR.UTF-8",
    LC_NAME = "el_GR.UTF-8",
    LANG = "en_US.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
=======================================================================
Step1: Indexing the genome and uploading into HDFS
=======================================================================

Started at:  31/10/2014 14:38:11 Friday
[bwa_index] Pack FASTA... 1.62 sec
[bwa_index] Reverse the packed sequence... 0.55 sec
[bwa_index] Construct BWT for the packed sequence...
[bwa_index] 64.38 seconds elapse.
[bwa_index] Construct BWT for the reverse packed sequence...
[bwa_index] 64.18 seconds elapse.
[bwa_index] Update BWT... 0.54 sec
[bwa_index] Update reverse BWT... 0.55 sec
[bwa_index] Construct SA from BWT and Occ... 22.02 sec
[bwa_index] Construct SA from reverse BWT and Occ... 21.97 sec
ref/
ref/reference.fa.rpac
ref/reference.fa.amb
ref/reference.fa.sa
ref/reference.fa.rbwt
ref/reference.fa
ref/reference.fa.ann
ref/reference.fa.pac
ref/reference.fa.bwt
ref/reference.fa.rsa
Finished at: 31/10/2014 14:41:50 Friday
Duration: 01:03:40

=======================================================================
Step2: Converting FASTQ into tab seperated file
         and creating archieve to upload into HDFS system
=======================================================================

Started at:  31/10/2014 14:41:50 Friday
Writng fastq files into tab text file
file1: 1000000 Paired-end reads
Copying executables
Creating Archive
bin/
bin/MergeSamFiles.jar
bin/bwa
bin/SortSam.jar
Finished at: 31/10/2014 14:42:10 Friday
Duration: 01:00:21

===============================================================
    Step3: Data loading in HDFS file system
===============================================================

Started at:  31/10/2014 14:42:10 Friday
test: File does not exist: /distmap_311014_input
mkdir: org.apache.hadoop.security.AccessControlException: Permission denied: user=myusername, access=WRITE, inode="":hadoop:supergroup:rwxr-xr-x
Error could not create input directory /distmap_311014_input folder on hdfs file system

What version of the product are you using? On what operating system?

:~$ uname -a
Linux clu26 2.6.28-11-server #42-Ubuntu SMP Fri Apr 17 02:45:36 UTC 2009 x86_64 GNU/Linux

hadoop-1.0.3
bwa-0.5.8c
Distmap v1.0
picard-1.96

Please provide any additional information below.

[Anonymous]

Originally posted by: Chefa... (code.google.com)@gmail.com

FIX: Remove trailing slash from DataUpload.pm:41,111,150 and HaddopMapping.pm:138,139
Thus:
my $input =  "/$args_dict->{'random_id'}"."_input";
must become
my $input =  "$args_dict->{'random_id'}"."_input";
both in lines 41,111

and

#my $input =  "/$args_dict->{'random_id'}"."_input";
must become
#my $input =  "$args_dict->{'random_id'}"."_input";
in line 150.

Similarly in HadoopMapping.pm lines 138,139:
my $input =  "/$args_dict->{'random_id'}"."_input";
my $output = "/$args_dict->{'random_id'}"."_output";

must become:
my $input =  "$args_dict->{'random_id'}"."_input";
my $output = "$args_dict->{'random_id'}"."_output";
   

[Anonymous]

Originally posted by: Chefa... (code.google.com)@gmail.com

There are more. To SUM UP:

file:lines

-- DataUpload.pm: 41,111,150
-- HadoopMapping.pm: 138,139
-- DataCleanUp.pm: 55,56
-- DataDownload.pm: 65,98