You can subscribe to this list here.
| 2008 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
(7) |
Oct
(12) |
Nov
(9) |
Dec
(13) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2009 |
Jan
(9) |
Feb
(22) |
Mar
(20) |
Apr
(10) |
May
(9) |
Jun
(16) |
Jul
(8) |
Aug
(3) |
Sep
(8) |
Oct
(6) |
Nov
|
Dec
(1) |
| 2010 |
Jan
(7) |
Feb
(9) |
Mar
(20) |
Apr
(4) |
May
(8) |
Jun
(12) |
Jul
(3) |
Aug
(4) |
Sep
(9) |
Oct
(10) |
Nov
(13) |
Dec
|
| 2011 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
(3) |
Oct
(7) |
Nov
(3) |
Dec
(4) |
| 2012 |
Jan
(1) |
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: <fa...@us...> - 2012-01-03 14:41:20
|
Revision: 2399
http://tab2mage.svn.sourceforge.net/tab2mage/?rev=2399&view=rev
Author: farne
Date: 2012-01-03 14:41:14 +0000 (Tue, 03 Jan 2012)
Log Message:
-----------
tab2mage submissions no longer supported by ArrayExpress. submission pages updated.
Modified Paths:
--------------
branches/ae-dev/Tab2MAGE/automation/tab2mage.cgi
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/templates/intro.html
Modified: branches/ae-dev/Tab2MAGE/automation/tab2mage.cgi
===================================================================
--- branches/ae-dev/Tab2MAGE/automation/tab2mage.cgi 2011-12-15 17:12:15 UTC (rev 2398)
+++ branches/ae-dev/Tab2MAGE/automation/tab2mage.cgi 2012-01-03 14:41:14 UTC (rev 2399)
@@ -26,22 +26,34 @@
alt => 'ArrayExpress',
},
{
- image => '/microarray/T2M_logo_small.png',
- destination => 'http://tab2mage.sourceforge.net/',
- alt => 'Tab2MAGE',
+ image => '/microarray/MAGETAB_logo_small.png',
+ destination => 'http://tab2mage.sourceforge.net/docs/magetab_subs.html',
+ alt => 'MAGE-TAB',
width => 100,
},
],
sidebar_links => [
{
- text => 'Introduction',
- destination => 'http://www.ebi.ac.uk/miamexpress/help/tab2mage_help.html',
+ text => 'Submitting MAGE-TAB to ArrayExpress',
+ destination => 'http://tab2mage.sourceforge.net/docs/magetab_subs.html',
},
{
- text => 'Creating a spreadsheet',
- destination => 'http://tab2mage.sourceforge.net/docs/spreadsheet.html',
+ text => 'MAGE-TAB overview',
+ destination => 'http://tab2mage.sourceforge.net/docs/magetab_docs.html',
},
{
+ text => 'IDF detailed notes',
+ destination => 'http://tab2mage.sourceforge.net/docs/idf.html',
+ },
+ {
+ text => 'SDRF detailed notes',
+ destination => 'http://tab2mage.sourceforge.net/docs/sdrf.html',
+ },
+ {
+ text => 'MAGE-TAB specification',
+ destination => 'http://www.mged.org/mage-tab/',
+ },
+ {
text => 'Microarray data submissions',
destination => 'http://www.ebi.ac.uk/microarray/submissions.html',
},
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/templates/intro.html
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/templates/intro.html 2011-12-15 17:12:15 UTC (rev 2398)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/templates/intro.html 2012-01-03 14:41:14 UTC (rev 2399)
@@ -2,70 +2,32 @@
<h1><!-- TMPL_VAR NAME=TITLE --></h1>
<table class="contenttable_lmenu">
- <tr>
- <td>
-<b>The Tab2MAGE format will be phased out for submissions by the end of 2011.<br> The MAGE-TAB format should be used instead.<b>
- </td>
- </tr>
+<tr>
+<th>Tab2MAGE submissions no longer supported</th>
+</tr>
+<tr>
+<td class="leftsubheading"><br/>As of January 2012 ArrayExpress no longer accepts submissions in Tab2MAGE format. <br/><br/>
+
+Please submit your experiment description in MAGE-TAB format using the MAGE-TAB submission page: <br/> <a href="http://www.ebi.ac.uk/cgi-bin/microarray/magetab.cgi">ArrayExpress MAGE-TAB submission page</a>
+<br/><br/>
+Existing Tab2MAGE submitters: Your Tab2MAGE submission page login details will work on the MAGE-TAB submission page. Contact us for advice at <a href="mailto:mia...@eb...">mia...@eb...</a> if you already have a Tab2MAGE spreadsheet and need help converting it to MAGE-TAB format,
+or if you need to modify an experiment previously submitted using Tab2MAGE.
+<br/><br/>
+Please see <a href="http://tab2mage.sourceforge.net/docs/magetab_vs_tab2mage.html">this comparison page</a> for more information about why we are now using MAGE-TAB instead of Tab2MAGE.
+</td>
+</tr>
+<tr></tr>
</table>
- <p class="text">Welcome to the <!-- TMPL_VAR NAME=EXPT_TYPE -->
- submissions system. This web page will allow you to submit your
- data files with experimental annotation entered in a
- spreadsheet. If you wish, you may have a template spreadsheet
- automatically generated for you. Alternatively, if you are an
- experienced user of Tab2MAGE, you may create your own spreadsheet
- using the <a
- href="http://tab2mage.sourceforge.net/docs/index.html">Tab2MAGE
- documentation</a>.</p>
- <p class="text">Note that if you wish to submit your data using
- the more flexible <a
- href="http://tab2mage.sourceforge.net/docs/magetab_docs.html">MAGE-TAB
- format</a>, you should use our <a
- href="http://www.ebi.ac.uk/cgi-bin/microarray/magetab.cgi">MAGE-TAB
- submissions web page</a>. MAGE-TAB is particularly recommended
- in the following circumstances:</p>
- <ul>
-
- <li>Submission of high-throughput sequencing data (e.g., Solexa,
- 454).</li>
-
- <li>Submission of other non array-based data (e.g.,
- metabolomics, proteomics).</li>
-
- <li>Cases in which you wish to precisely specify annotation
- terms and their source <br/> (e.g., disease state terms, and their
- accession numbers, from a disease ontology).</li>
-
- </ul>
-
- <p class="text">Please see <a
- href="http://tab2mage.sourceforge.net/docs/magetab_vs_tab2mage.html">this
- comparison page</a> for more detail. ArrayExpress will continue to
- support both Tab2MAGE and MAGE-TAB submissions; if, however, you
- wish to convert your Tab2MAGE spreadsheet into a MAGE-TAB
- document, a <a
- href="http://tab2mage.sourceforge.net/index.html#tabconverter">Tab2MAGE
- to MAGE-TAB converter tool</a> has been created and made <a
- href="http://sourceforge.net/project/showfiles.php?group_id=120325&package_id=226041">available
- for download here</a>.</p>
-
- <p class="text">Login to start a submission and generate a
- template. Please use the "New submitter" link on this page if you
- need to create a new account.</p>
-
- <p class="text">Full help documentation is available using the links to
- the side of this page.</p>
-
<form method="post" action="<!-- TMPL_VAR NAME=CGI_BASE -->" enctype="multipart/form-data">
<div class="navbar">
- <input type="submit"
+ <input type="hidden"
name="<!-- TMPL_VAR NAME=RM -->"
value="<!-- TMPL_VAR NAME=LOGIN -->"/>
- <input type="submit"
+ <input type="hidden"
name="<!-- TMPL_VAR NAME=RM -->"
value="<!-- TMPL_VAR NAME=SIGNUP -->"/>
</div>
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <fa...@us...> - 2011-12-15 14:16:46
|
Revision: 2397
http://tab2mage.svn.sourceforge.net/tab2mage/?rev=2397&view=rev
Author: farne
Date: 2011-12-15 14:16:36 +0000 (Thu, 15 Dec 2011)
Log Message:
-----------
added atlas_fail_score to experiments table and updated subs tracking pages to show this score (previous commit was interrupted)
Modified Paths:
--------------
branches/ae-dev/Tab2MAGE/automation/admin/app/views/miamexps/list.rhtml
Modified: branches/ae-dev/Tab2MAGE/automation/admin/app/views/miamexps/list.rhtml
===================================================================
--- branches/ae-dev/Tab2MAGE/automation/admin/app/views/miamexps/list.rhtml 2011-12-15 11:34:56 UTC (rev 2396)
+++ branches/ae-dev/Tab2MAGE/automation/admin/app/views/miamexps/list.rhtml 2011-12-15 14:16:36 UTC (rev 2397)
@@ -21,7 +21,7 @@
<%= content_tag("th", "Checker score") %>
<%= content_tag("th", "No. of Hybs") %>
<%= content_tag("th", "MIAME score") %>
- <%= content_tag("th", "DW status") %>
+ <%= content_tag("th", "Atlas status") %>
<%= content_tag("th", "Status") %>
<%= content_tag("th", "Date last processed") %>
<%= content_tag("th", "Curator") %>
@@ -46,7 +46,7 @@
<%= content_tag("td", e.checker_score) %>
<%= content_tag("td", e.num_hybridizations) %>
<%= content_tag("td", e.miame_score_html) %>
- <%= content_tag("td", e.dw_status_html) %>
+ <%= content_tag("td", e.atlas_status_html) %>
<%= content_tag("td", e.status) %>
<%= content_tag("td", e.date_last_processed ?
e.date_last_processed.strftime('%Y-%m-%d') :
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <fa...@us...> - 2011-12-15 11:35:02
|
Revision: 2396
http://tab2mage.svn.sourceforge.net/tab2mage/?rev=2396&view=rev
Author: farne
Date: 2011-12-15 11:34:56 +0000 (Thu, 15 Dec 2011)
Log Message:
-----------
added atlas_fail_score to experiments table and updated subs tracking pages to show this score
Modified Paths:
--------------
branches/ae-dev/Tab2MAGE/automation/admin/app/controllers/experiments_controller.rb
branches/ae-dev/Tab2MAGE/automation/admin/app/models/experiment.rb
branches/ae-dev/Tab2MAGE/automation/admin/app/views/experiments/today_list.rhtml
branches/ae-dev/Tab2MAGE/automation/admin/app/views/tab2mages/list.rhtml
branches/ae-dev/Tab2MAGE/automation/admin/db/schema.rb
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/DB/Experiment.pm
Added Paths:
-----------
branches/ae-dev/Tab2MAGE/automation/admin/db/migrate/010_add_atlas_fail_score.rb
Modified: branches/ae-dev/Tab2MAGE/automation/admin/app/controllers/experiments_controller.rb
===================================================================
--- branches/ae-dev/Tab2MAGE/automation/admin/app/controllers/experiments_controller.rb 2011-12-15 11:24:59 UTC (rev 2395)
+++ branches/ae-dev/Tab2MAGE/automation/admin/app/controllers/experiments_controller.rb 2011-12-15 11:34:56 UTC (rev 2396)
@@ -121,7 +121,7 @@
subdir = $1
end
- dir_path = "/nfs/ma/ma-exp/EXPERIMENTS/"+subdir+"/"+acc+"/QM/QMreport/"
+ dir_path = "/net/isilon5/ma/home/arrayexpress/ae2_production/data/EXPERIMENT/"+subdir+"/"+acc+"/QM/QMreport/"
if params[:file]
show_pdf(:dir => dir_path, :file => params[:file], :acc => acc)
return
Modified: branches/ae-dev/Tab2MAGE/automation/admin/app/models/experiment.rb
===================================================================
--- branches/ae-dev/Tab2MAGE/automation/admin/app/models/experiment.rb 2011-12-15 11:24:59 UTC (rev 2395)
+++ branches/ae-dev/Tab2MAGE/automation/admin/app/models/experiment.rb 2011-12-15 11:34:56 UTC (rev 2396)
@@ -80,4 +80,11 @@
'<font color="red">no</font>'
return status
end
+
+ def atlas_status_html
+ status = self.in_data_warehouse? ? '<font color="blue">Loaded</font>' :
+ self.atlas_fail_score.nil? ? '' :
+ '<font color="red">'+self.atlas_fail_score+'</font>'
+ return status
+ end
end
Modified: branches/ae-dev/Tab2MAGE/automation/admin/app/views/experiments/today_list.rhtml
===================================================================
--- branches/ae-dev/Tab2MAGE/automation/admin/app/views/experiments/today_list.rhtml 2011-12-15 11:24:59 UTC (rev 2395)
+++ branches/ae-dev/Tab2MAGE/automation/admin/app/views/experiments/today_list.rhtml 2011-12-15 11:34:56 UTC (rev 2396)
@@ -12,7 +12,7 @@
<%= content_tag("th", "Name") %>
<%= content_tag("th", "Checker score") %>
<%= content_tag("th", "MIAME score") %>
- <%= content_tag("th", "DW status") %>
+ <%= content_tag("th", "Atlas status") %>
<%= content_tag("th", "Status") %>
<%= content_tag("th", "Date last processed") %>
<%= content_tag("th", "Curator") %>
@@ -37,7 +37,7 @@
<%= content_tag("td", e.name) %>
<%= content_tag("td", e.checker_score) %>
<%= content_tag("td", e.miame_score_html) %>
- <%= content_tag("td", e.dw_status_html) %>
+ <%= content_tag("td", e.atlas_status_html) %>
<%= content_tag("td", e.status) %>
<%= content_tag("td", e.date_last_processed ?
e.date_last_processed.localtime.strftime('%Y-%m-%d %H:%M:%S') :
Modified: branches/ae-dev/Tab2MAGE/automation/admin/app/views/tab2mages/list.rhtml
===================================================================
--- branches/ae-dev/Tab2MAGE/automation/admin/app/views/tab2mages/list.rhtml 2011-12-15 11:24:59 UTC (rev 2395)
+++ branches/ae-dev/Tab2MAGE/automation/admin/app/views/tab2mages/list.rhtml 2011-12-15 11:34:56 UTC (rev 2396)
@@ -20,7 +20,7 @@
<%= content_tag("th", "Checker score") %>
<%= content_tag("th", "No. of Hybs") %>
<%= content_tag("th", "MIAME score") %>
- <%= content_tag("th", "DW status") %>
+ <%= content_tag("th", "Atlas status") %>
<%= content_tag("th", "In curation") %>
<%= content_tag("th", "Status") %>
<%= content_tag("th", "Software") %>
@@ -48,7 +48,7 @@
<%= content_tag("td", e.checker_score) %>
<%= content_tag("td", e.num_hybridizations) %>
<%= content_tag("td", e.miame_score_html) %>
- <%= content_tag("td", e.dw_status_html) %>
+ <%= content_tag("td", e.atlas_status_html) %>
<%= content_tag("td", e.in_curation.nil? ? '' :
e.in_curation? ?
'<font color="green">YES</font>' :
Added: branches/ae-dev/Tab2MAGE/automation/admin/db/migrate/010_add_atlas_fail_score.rb
===================================================================
--- branches/ae-dev/Tab2MAGE/automation/admin/db/migrate/010_add_atlas_fail_score.rb (rev 0)
+++ branches/ae-dev/Tab2MAGE/automation/admin/db/migrate/010_add_atlas_fail_score.rb 2011-12-15 11:34:56 UTC (rev 2396)
@@ -0,0 +1,11 @@
+class AddAtlasFailScore < ActiveRecord::Migration
+ def self.up
+ add_column :experiments, :atlas_fail_score, :string
+ remove_column :experiments, :ae_data_warehouse_score
+ end
+
+ def self.down
+ remove_column :experiments, :atlas_fail_score
+ add_column :experiments, :ae_data_warehouse_score, :int
+ end
+end
Property changes on: branches/ae-dev/Tab2MAGE/automation/admin/db/migrate/010_add_atlas_fail_score.rb
___________________________________________________________________
Added: svn:eol-style
+ native
Modified: branches/ae-dev/Tab2MAGE/automation/admin/db/schema.rb
===================================================================
--- branches/ae-dev/Tab2MAGE/automation/admin/db/schema.rb 2011-12-15 11:24:59 UTC (rev 2395)
+++ branches/ae-dev/Tab2MAGE/automation/admin/db/schema.rb 2011-12-15 11:34:56 UTC (rev 2396)
@@ -9,7 +9,7 @@
#
# It's strongly recommended to check this file into your version control system.
-ActiveRecord::Schema.define(:version => 9) do
+ActiveRecord::Schema.define(:version => 10) do
create_table "array_designs", :force => true do |t|
t.integer "miamexpress_subid"
@@ -94,8 +94,6 @@
t.string "user"
end
- add_index "daemon_instances", ["pipeline_id"], :name => "FK_daemon_inst_to_pipeline"
-
create_table "data_files", :force => true do |t|
t.integer "experiment_id", :null => false
t.string "name"
@@ -106,7 +104,7 @@
add_index "data_files", ["experiment_id"], :name => "experiment_id"
create_table "data_formats", :force => true do |t|
- t.string "name", :limit => 50, :default => "", :null => false
+ t.string "name", :limit => 50, :null => false
end
add_index "data_formats", ["name"], :name => "name", :unique => true
@@ -130,7 +128,7 @@
create_table "events", :force => true do |t|
t.integer "array_design_id"
t.integer "experiment_id"
- t.string "event_type", :limit => 50, :default => "", :null => false
+ t.string "event_type", :limit => 50, :null => false
t.integer "was_successful"
t.string "source_db", :limit => 30
t.string "target_db", :limit => 30
@@ -141,7 +139,7 @@
t.string "log_file", :limit => 511
t.integer "jobregister_dbid", :limit => 20
t.text "comment"
- t.integer "is_deleted", :null => false
+ t.integer "is_deleted", :null => false
end
add_index "events", ["array_design_id"], :name => "array_design_id"
@@ -155,31 +153,28 @@
t.datetime "date_calculated"
end
- add_index "experiment_quality_metrics", ["quality_metric_id"], :name => "FK_exp_qm_to_qm"
- add_index "experiment_quality_metrics", ["experiment_id"], :name => "FK_exp_qm_to_exp"
-
create_table "experiments", :force => true do |t|
t.string "accession"
t.string "name"
t.integer "user_id"
t.integer "checker_score"
- t.string "software", :limit => 100
- t.string "status", :limit => 50
+ t.string "software", :limit => 100
+ t.string "status", :limit => 50
t.integer "data_warehouse_ready"
t.datetime "date_last_edited"
t.datetime "date_submitted"
t.datetime "date_last_processed"
t.integer "in_curation"
- t.string "curator", :limit => 30
+ t.string "curator", :limit => 30
t.text "comment"
- t.string "experiment_type", :limit => 30
- t.string "miamexpress_login", :limit => 30
+ t.string "experiment_type", :limit => 30
+ t.string "miamexpress_login", :limit => 30
t.integer "miamexpress_subid"
t.integer "is_affymetrix"
- t.integer "is_uhts", :limit => 4
- t.integer "use_native_datafiles", :limit => 4
+ t.integer "is_uhts", :limit => 4
+ t.integer "use_native_datafiles", :limit => 4
t.integer "is_mx_batchloader"
- t.integer "is_deleted", :null => false
+ t.integer "is_deleted", :null => false
t.integer "miame_score"
t.integer "in_data_warehouse"
t.integer "num_submissions"
@@ -189,15 +184,15 @@
t.integer "num_hybridizations"
t.integer "has_raw_data"
t.integer "has_processed_data"
- t.integer "has_gds", :limit => 4
+ t.integer "has_gds", :limit => 4
t.datetime "release_date"
t.integer "is_released"
t.integer "ae_miame_score"
- t.integer "ae_data_warehouse_score"
t.string "migration_status"
t.text "migration_comment"
t.string "migration_source"
t.string "file_to_load"
+ t.string "atlas_fail_score"
end
add_index "experiments", ["user_id"], :name => "user_id"
@@ -227,17 +222,17 @@
add_index "experiments_quantitation_types", ["experiment_id"], :name => "experiment_id"
create_table "factors", :force => true do |t|
- t.string "name", :limit => 128, :default => "", :null => false
- t.integer "is_deleted", :null => false
+ t.string "name", :limit => 128, :null => false
+ t.integer "is_deleted", :null => false
end
create_table "loaded_data", :force => true do |t|
- t.string "identifier", :default => "", :null => false
- t.string "md5_hash", :limit => 35, :default => "", :null => false
- t.integer "data_format_id", :null => false
- t.integer "is_deleted", :null => false
+ t.string "identifier", :null => false
+ t.string "md5_hash", :limit => 35, :null => false
+ t.integer "data_format_id", :null => false
+ t.integer "is_deleted", :null => false
t.integer "platform_id"
- t.integer "needs_metrics_calculation", :null => false
+ t.integer "needs_metrics_calculation", :null => false
t.datetime "date_hashed"
end
@@ -292,9 +287,9 @@
add_index "organisms", ["taxon_id"], :name => "taxon_id"
create_table "permissions", :force => true do |t|
- t.string "name", :limit => 40, :default => "", :null => false
+ t.string "name", :limit => 40, :null => false
t.string "info", :limit => 80
- t.integer "is_deleted", :null => false
+ t.integer "is_deleted", :null => false
end
create_table "permissions_roles", :id => false, :force => true do |t|
@@ -306,7 +301,7 @@
add_index "permissions_roles", ["permission_id"], :name => "permission_id"
create_table "pipelines", :force => true do |t|
- t.string "submission_type", :default => "", :null => false
+ t.string "submission_type", :null => false
t.integer "instances_to_start", :default => 1
t.string "checker_daemon"
t.string "exporter_daemon"
@@ -319,7 +314,7 @@
end
create_table "platforms", :force => true do |t|
- t.string "name", :limit => 50, :default => "", :null => false
+ t.string "name", :limit => 50, :null => false
end
add_index "platforms", ["name"], :name => "name", :unique => true
@@ -344,14 +339,14 @@
add_index "quality_metrics", ["name"], :name => "type", :unique => true
create_table "quantitation_types", :force => true do |t|
- t.string "name", :limit => 128, :default => "", :null => false
- t.integer "is_deleted", :null => false
+ t.string "name", :limit => 128, :null => false
+ t.integer "is_deleted", :null => false
end
create_table "roles", :force => true do |t|
- t.string "name", :limit => 40, :default => "", :null => false
+ t.string "name", :limit => 40, :null => false
t.string "info", :limit => 80
- t.integer "is_deleted", :null => false
+ t.integer "is_deleted", :null => false
end
create_table "roles_users", :id => false, :force => true do |t|
@@ -362,30 +357,33 @@
add_index "roles_users", ["user_id"], :name => "user_id"
add_index "roles_users", ["role_id"], :name => "role_id"
- create_table "sample_groups", :force => true do |t|
- t.string "accession"
- t.string "user_accession"
- t.string "submission_accession"
- t.string "project_name"
- t.string "source_repository"
- t.string "linking_repositories"
- t.datetime "date_assigned"
- t.datetime "date_last_processed"
- t.text "comment"
- t.integer "is_deleted"
+ create_table "sample_assay", :primary_key => "accession", :force => true do |t|
+ t.string "user_accession", :null => false
+ t.string "submission_accession", :null => false
+ t.datetime "date_assigned", :null => false
+ t.integer "is_deleted", :default => 0, :null => false
end
- create_table "samples", :force => true do |t|
- t.string "accession"
- t.string "user_accession"
- t.string "submission_accession"
- t.datetime "date_assigned"
- t.datetime "date_last_processed"
- t.text "comment"
- t.integer "is_deleted"
- t.string "source_repository"
+ add_index "sample_assay", ["submission_accession", "user_accession"], :name => "submission_user", :unique => true
+
+ create_table "sample_groups", :primary_key => "accession", :force => true do |t|
+ t.string "user_accession", :null => false
+ t.string "submission_accession", :null => false
+ t.datetime "date_assigned", :null => false
+ t.integer "is_deleted", :default => 0, :null => false
end
+ add_index "sample_groups", ["user_accession", "submission_accession"], :name => "submission_user", :unique => true
+
+ create_table "sample_reference", :primary_key => "accession", :force => true do |t|
+ t.string "user_accession", :null => false
+ t.string "submission_accession", :null => false
+ t.datetime "date_assigned", :null => false
+ t.integer "is_deleted", :default => 0, :null => false
+ end
+
+ add_index "sample_reference", ["submission_accession", "user_accession"], :name => "submission_user", :unique => true
+
create_table "spreadsheets", :force => true do |t|
t.integer "experiment_id", :null => false
t.string "name"
@@ -402,14 +400,14 @@
end
create_table "users", :force => true do |t|
- t.string "login", :limit => 40, :default => "", :null => false
+ t.string "login", :limit => 40, :null => false
t.string "name", :limit => 40
- t.string "password", :limit => 40, :default => "", :null => false
+ t.string "password", :limit => 40, :null => false
t.string "email", :limit => 100
t.datetime "modified_at"
t.datetime "created_at"
t.datetime "access"
- t.integer "is_deleted", :null => false
+ t.integer "is_deleted", :null => false
end
add_index "users", ["login"], :name => "login", :unique => true
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/DB/Experiment.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/DB/Experiment.pm 2011-12-15 11:24:59 UTC (rev 2395)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/DB/Experiment.pm 2011-12-15 11:34:56 UTC (rev 2396)
@@ -67,7 +67,7 @@
has_raw_data
has_processed_data
ae_miame_score
- ae_data_warehouse_score
+ atlas_fail_score
comment
migration_status
migration_comment
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <fa...@us...> - 2011-12-15 11:25:10
|
Revision: 2395
http://tab2mage.svn.sourceforge.net/tab2mage/?rev=2395&view=rev
Author: farne
Date: 2011-12-15 11:24:59 +0000 (Thu, 15 Dec 2011)
Log Message:
-----------
array mageml export switched off
Modified Paths:
--------------
branches/ae-dev/Tab2MAGE/automation/mx_reset_array.pl
Modified: branches/ae-dev/Tab2MAGE/automation/mx_reset_array.pl
===================================================================
--- branches/ae-dev/Tab2MAGE/automation/mx_reset_array.pl 2011-12-15 11:16:59 UTC (rev 2394)
+++ branches/ae-dev/Tab2MAGE/automation/mx_reset_array.pl 2011-12-15 11:24:59 UTC (rev 2395)
@@ -41,7 +41,7 @@
# Global variables to switch magetab and mageml export on and off
my $EXPORT_MAGETAB = 1;
-my $EXPORT_MAGEML = 1;
+my $EXPORT_MAGEML = 0;
# FIXME: This script is getting long. Methods should be factored out
# into an array exporter class.
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <fa...@us...> - 2011-12-15 11:17:10
|
Revision: 2394
http://tab2mage.svn.sourceforge.net/tab2mage/?rev=2394&view=rev
Author: farne
Date: 2011-12-15 11:16:59 +0000 (Thu, 15 Dec 2011)
Log Message:
-----------
fixed mistake in ADF heading regex
Modified Paths:
--------------
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/Config.pm
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/Config.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/Config.pm 2011-12-14 16:48:08 UTC (rev 2393)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/Config.pm 2011-12-15 11:16:59 UTC (rev 2394)
@@ -219,9 +219,9 @@
# regex for identifying different ADF formats
MX_ADF_REGEX =>
- qr{[\t^](Meta\s*Column|Reporter\s*Identifier|Composite\s*Sequence\s*Identifier)\s*[\t\n]}ixms,
+ qr{(\t|^)(Meta\s*Column|Reporter\s*Identifier|Composite\s*Sequence\s*Identifier)\s*(\t|\n)}ixms,
MAGETAB_ADF_REGEX =>
- qr{[\t^](Block\s*Column|Block\s*Row|Reporter\s*Name|Composite\s*Element\s*Name)\s*[\t\n]}ixms,
+ qr{(\t|^)(Block\s*Column|Block\s*Row|Reporter\s*Name|Composite\s*Element\s*Name)\s*(\t|\n)}ixms,
# List of characters which cannot be used in data file names
FILENAME_FORBIDDEN_CHARS => qr{ [ \: \\ \" ]+ }ixms,
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <fa...@us...> - 2011-11-29 13:40:28
|
Revision: 2390
http://tab2mage.svn.sourceforge.net/tab2mage/?rev=2390&view=rev
Author: farne
Date: 2011-11-29 13:40:17 +0000 (Tue, 29 Nov 2011)
Log Message:
-----------
regex for finding MX or MAGETAB ADF column headings now more strict about heading boundaries to avoid it matching terms in ADF header
Modified Paths:
--------------
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/Config.pm
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/Config.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/Config.pm 2011-11-29 12:52:06 UTC (rev 2389)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/Config.pm 2011-11-29 13:40:17 UTC (rev 2390)
@@ -219,9 +219,9 @@
# regex for identifying different ADF formats
MX_ADF_REGEX =>
- qr{\b(Meta\s*Column|Reporter\s*Identifier|Composite\s*Sequence\s*Identifier)\b}ixms,
+ qr{[\t^](Meta\s*Column|Reporter\s*Identifier|Composite\s*Sequence\s*Identifier)\s*[\t\n]}ixms,
MAGETAB_ADF_REGEX =>
- qr{^(Block\s*Column|Block\s*Row|Reporter\s*Name|Composite\s*Element\s*Name)\b}ixms,
+ qr{[\t^](Block\s*Column|Block\s*Row|Reporter\s*Name|Composite\s*Element\s*Name)\s*[\t\n]}ixms,
# List of characters which cannot be used in data file names
FILENAME_FORBIDDEN_CHARS => qr{ [ \: \\ \" ]+ }ixms,
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <fa...@us...> - 2011-11-29 12:52:13
|
Revision: 2389
http://tab2mage.svn.sourceforge.net/tab2mage/?rev=2389&view=rev
Author: farne
Date: 2011-11-29 12:52:06 +0000 (Tue, 29 Nov 2011)
Log Message:
-----------
CompositeSequence Type columns now dropped from MX ADF during export
Modified Paths:
--------------
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/ADFParser/ADFConvert.pm
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/ADFParser/ADFConvert.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/ADFParser/ADFConvert.pm 2011-11-28 10:30:33 UTC (rev 2388)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/ADFParser/ADFConvert.pm 2011-11-29 12:52:06 UTC (rev 2389)
@@ -578,6 +578,8 @@
"compositesequencedatabaseentry" => "Composite Element Database Entry",
"compositesequencecomment" => "Composite Element Comment",
"compositebiosequencecomment" => "Composite Element Comment",
+ "compositesequencebiosequencetype" => "DROP",
+ "compositesequencebiosequencepolymertype" => "DROP",
);
return %adf_tag;
}
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <fa...@us...> - 2011-11-28 10:30:42
|
Revision: 2388
http://tab2mage.svn.sourceforge.net/tab2mage/?rev=2388&view=rev
Author: farne
Date: 2011-11-28 10:30:33 +0000 (Mon, 28 Nov 2011)
Log Message:
-----------
bug fix. daemon was returning after export failure instead of continuing to next submission.
Modified Paths:
--------------
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/Daemon/Exporter.pm
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/Daemon/Exporter.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/Daemon/Exporter.pm 2011-10-28 14:08:33 UTC (rev 2387)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/Daemon/Exporter.pm 2011-11-28 10:30:33 UTC (rev 2388)
@@ -212,7 +212,7 @@
);
# do not continue to mageml export if ae2 export has failed
$submission->update();
- return;
+ next SUBMISSION;
}
# Add an event to record this run.
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <fa...@us...> - 2011-10-28 14:08:39
|
Revision: 2387
http://tab2mage.svn.sourceforge.net/tab2mage/?rev=2387&view=rev
Author: farne
Date: 2011-10-28 14:08:33 +0000 (Fri, 28 Oct 2011)
Log Message:
-----------
added new experiment type
Modified Paths:
--------------
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/Common.pm
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/Common.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/Common.pm 2011-10-28 14:06:07 UTC (rev 2386)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/Common.pm 2011-10-28 14:08:33 UTC (rev 2387)
@@ -1059,6 +1059,7 @@
"transcription profiling by high throughput sequencing",
"transcription profiling by tiling array",
"translation profiling",
+ "RNA-seq of non coding RNA",
);
return @expt_types;
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <fa...@us...> - 2011-10-28 14:06:18
|
Revision: 2386
http://tab2mage.svn.sourceforge.net/tab2mage/?rev=2386&view=rev
Author: farne
Date: 2011-10-28 14:06:07 +0000 (Fri, 28 Oct 2011)
Log Message:
-----------
Natalja's changes to support import of data from Sanger
Modified Paths:
--------------
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/SRAtoMAGETAB.pm
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/SRAtoMAGETAB.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/SRAtoMAGETAB.pm 2011-10-28 13:47:57 UTC (rev 2385)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/SRAtoMAGETAB.pm 2011-10-28 14:06:07 UTC (rev 2386)
@@ -198,37 +198,52 @@
$self->get_investigation->set_title($title);
# Set experiment type
- my $study_type = $xml->{STUDY}->{DESCRIPTOR}->{STUDY_TYPE}->{existing_study_type};
- my $efo_type = $EFO_STUDY_TYPE_FOR{$study_type};
- if ($efo_type){
- $study_type = $efo_type;
- }
- else{
- if ($study_type eq "Other"){
- $study_type = $xml->{STUDY}->{DESCRIPTOR}->{STUDY_TYPE}->{new_study_type}
- }
- $self->log("Using study type not mapped to EFO - $study_type", "warn");
- }
- #Added by Natalja
- if (length($study_type)==0){
- $study_type = "transcription profiling by high throughput sequencing";
- }
- $self->_add_expt_comment("AEExperimentType", $study_type);
-
- # get additional study types from STUDY_ATTRIBUTE
- # FIXME: TAG for these has not yet been agreed
+ #my $study_type = $xml->{STUDY}->{DESCRIPTOR}->{STUDY_TYPE}->{existing_study_type};
+
+ my $value = $xml->{STUDY}->{DESCRIPTOR}->{STUDY_TYPE}->{existing_study_type};
+
my @design_types;
- foreach my $att (grep { $_->{TAG} eq $STUDY_TYPE_TAG}
- @{ $xml->{STUDY}->{STUDY_ATTRIBUTES}->{STUDY_ATTRIBUTE} || [] }){
- my $value = $att->{VALUE};
+ #foreach my $att (grep { $_->{TAG} eq $STUDY_TYPE_TAG}
+ # @{ $xml->{STUDY}->{STUDY_ATTRIBUTES}->{STUDY_ATTRIBUTE} || [] }){
+ # my $value = $att->{VALUE};
my $oe = Bio::MAGETAB::ControlledTerm->new({
category => "ExperimentDesignType",
value => $value,
});
push @design_types, $oe;
- }
+ #}
$self->get_investigation->set_designTypes(\@design_types);
+
+ #my $efo_type = $EFO_STUDY_TYPE_FOR{$study_type};
+ #if ($efo_type){
+ # $study_type = $efo_type;
+ #}
+ #else{
+ # if ($study_type eq "Other"){
+ # $study_type = $xml->{STUDY}->{DESCRIPTOR}->{STUDY_TYPE}->{new_study_type}
+ # }
+ # $self->log("Using study type not mapped to EFO - $study_type", "warn");
+ #}
+ #Added by Natalja
+ #if (length($study_type)==0){
+ # $study_type = "transcription profiling by high throughput sequencing";
+ #}
+ #$self->_add_expt_comment("AEExperimentType", $study_type);
+ # get additional study types from STUDY_ATTRIBUTE
+ # FIXME: TAG for these has not yet been agreed
+ #my @design_types;
+ #foreach my $att (grep { $_->{TAG} eq $STUDY_TYPE_TAG}
+ # @{ $xml->{STUDY}->{STUDY_ATTRIBUTES}->{STUDY_ATTRIBUTE} || [] }){
+ # my $value = $att->{VALUE};
+ # my $oe = Bio::MAGETAB::ControlledTerm->new({
+ # category => "ExperimentDesignType",
+ # value => $value,
+ # });
+ # push @design_types, $oe;
+ #}
+ #$self->get_investigation->set_designTypes(\@design_types);
+
# Set description text
my $abstract = $xml->{STUDY}->{DESCRIPTOR}->{STUDY_ABSTRACT};
my $description = $xml->{STUDY}->{DESCRIPTOR}->{STUDY_DESCRIPTION};
@@ -271,6 +286,10 @@
$self->get_investigation->set_publicReleaseDate(substr($date,0,10));
my $e_mail = "";
+
+ if (!($lab)){
+ $lab = "Wellcome Trust Sanger Institute";
+ }
if (trim($lab)=="Wellcome Trust Sanger Institute"){
$e_mail = "datahose\@sanger.ac.uk";
@@ -294,11 +313,46 @@
}
sub parse_sample_xml{
+ my %design_type_mapping = (
+ 'ArrayExpress-Genotype' => 'genotype design',
+ 'ArrayExpress-Phenotype' => 'observational design',
+ 'ArrayExpress-DevelopmentalStage' => 'development or differentiation design',
+ 'ArrayExpress-CellType' => 'cell type comparison design',
+ 'ArrayExpress-DiseaseState' => 'disease state design',
+ 'ArrayExpress-Compound' => 'compound treatment design',
+ 'ArrayExpress-Immunoprecipitate' => 'binding site identification design',
+ 'ArrayExpress-GrowthCondition' => 'growth condition design',
+ 'ArrayExpress-RNAi' => 'cellular modification design',
+ 'ArrayExpress-OrganismPart' => 'organism part comparison design',
+ 'ArrayExpress-Species' => 'species design',
+ 'ArrayExpress-TimePoint' => 'time series design',
+ );
+
+ my %factor_type_mapping = (
+ 'ArrayExpress-Genotype' => 'genotype',
+ 'ArrayExpress-Phenotype' => 'phenotype',
+ 'ArrayExpress-DevelopmentalStage' => 'developmental_stage',
+ 'ArrayExpress-CellType' => 'cell_type',
+ 'ArrayExpress-DiseaseState' => 'disease_state',
+ 'ArrayExpress-Compound' => 'compound',
+ 'ArrayExpress-Immunoprecipitate' => 'immunoprecipitate',
+ 'ArrayExpress-GrowthCondition' => 'growth_condition',
+ 'ArrayExpress-RNAi' => 'rnai',
+ 'ArrayExpress-OrganismPart' => 'organism_part',
+ 'ArrayExpress-Species' => 'organism',
+ 'ArrayExpress-TimePoint' => 'time',
+ );
+
+
my ($self, $sample_xml) = @_;
my $xml = XMLin($sample_xml, ForceArray => \@force_array);
+ my @design_types;
+
+ my %fv_valuecount;
+
foreach my $sample(@{ $xml->{SAMPLE} || [] }){
my $acc = $sample->{accession};
$self->log("Found sample accession: $acc");
@@ -306,6 +360,10 @@
name => $acc
});
$self->_add_comment("ENA_SAMPLE", $acc, $source);
+
+ my $alias = $sample->{alias};
+ $self->_add_comment("ALIAS", $alias, $source);
+
my $descr = $sample->{DESCRIPTION};
$source->set_description($descr) if ($descr and ref($descr) eq "SCALAR");
@@ -317,31 +375,70 @@
$self->_add_comment("INDIVIDUAL_NAME", $sample->{SAMPLE_NAME}->{INDIVIDUAL_NAME}, $source)
if $sample->{SAMPLE_NAME}->{INDIVIDUAL_NAME};
- my @chars;
- my $taxon_id = $sample->{SAMPLE_NAME}->{TAXON_ID};
- my $species = $sample->{SAMPLE_NAME}->{SCIENTIFIC_NAME};
- if ($species or $taxon_id){
- my $species_attr = $self->create_species_attribute($species, $taxon_id);
- push @chars, $species_attr;
- }
- else{
- $self->log("no species found for sample $acc","warn");
- }
-
- foreach my $attr ( @{ $sample->{SAMPLE_ATTRIBUTES}->{SAMPLE_ATTRIBUTE} || [] }){
+ my @chars;
+ my $organism = 0;
+ foreach my $attr ( @{ $sample->{SAMPLE_ATTRIBUTES}->{SAMPLE_ATTRIBUTE} || [] }){
my $category = $attr->{TAG};
next if $category =~ /^ENA-.*/;
my $value = $attr->{VALUE};
- next if $value !=~ /[A-Z][a-z][a-z]*[0-9][0-9]/;
- my $char = Bio::MAGETAB::ControlledTerm->new({
- category => $category,
- value => $attr->{VALUE},
- });
- push @chars, $char;
+ next if (!($value =~ /^(\w+\s*)+$/));
+ next if lc($value) =~ /^not applicable$/;
+ if (exists($design_type_mapping{$category})){
+ $fv_valuecount{$category}{$value}++;
+ $category = $factor_type_mapping{$category};
+ }
+ $self->log("Category: $category Value: $value");
+ if ((($category eq "phenotype") or ($category eq "genotype")) and $value eq "wt"){
+ $value = "wild_type";
+ }
+ if ($category eq "organism"){
+ $organism = 1;
+ my $taxon_id = $sample->{SAMPLE_NAME}->{TAXON_ID};
+ if ($value or $taxon_id){
+ my $species_attr = $self->create_species_attribute($value, $taxon_id);
+ push @chars, $species_attr;
+ }
+ }
+ else{
+ my $char = Bio::MAGETAB::ControlledTerm->new({
+ category => $category,
+ value => $value,
+ });
+ push @chars, $char;
+ }
+
+
}
$source->set_characteristics(\@chars);
push @{ $self->get_all_nodes }, $source;
+
+ if ($organism eq 0){
+ my $taxon_id = $sample->{SAMPLE_NAME}->{TAXON_ID};
+ my $species = $sample->{SAMPLE_NAME}->{SCIENTIFIC_NAME};
+ if ($species or $taxon_id){
+ my $species_attr = $self->create_species_attribute($species, $taxon_id);
+ push @chars, $species_attr;
+ }
+ else{
+ $self->log("no species found for sample $acc","warn");
+ }
+ }
}
+
+ #add study designs
+ while ( my ( $cat, $valhash ) = each %fv_valuecount ) {
+ my %hash = %{$valhash};
+
+ if ( scalar(grep { defined $_ } values %{ $valhash }) >1 or ($cat =~ /Immunoprecipitate/i)) {
+ my $oe = Bio::MAGETAB::ControlledTerm->new({
+ category => "ExperimentDesignType",
+ value => $design_type_mapping{$cat},
+ });
+ push @design_types, $oe;
+ }
+ }
+
+ $self->get_investigation->set_designTypes(\@design_types);
}
@@ -360,6 +457,8 @@
value => "high_throughput_sequencing",
});
+ my %study_type = "";
+ my $paired = 0;
foreach my $expt (@{ $xml->{EXPERIMENT} || [] }){
my $acc = $expt->{accession};
@@ -387,6 +486,20 @@
});
# Add library descriptor paramters as Extract Comments
+ foreach my $type qw(LIBRARY_STRATEGY){
+ my $value = $expt->{DESIGN}->{LIBRARY_DESCRIPTOR}->{$type};
+ $value = trim($value);
+ if ($value =~ m/RNA-Seq/){
+ $value = "transcription profiling by high throughput sequencing";
+ }
+ if (!exists($study_type{$value})){
+ $study_type{$value} = $value;
+ }
+
+ }
+
+
+
foreach my $type qw(LIBRARY_STRATEGY LIBRARY_SOURCE LIBRARY_SELECTION){
my $value = $expt->{DESIGN}->{LIBRARY_DESCRIPTOR}->{$type};
$self->_add_comment($type, $value, $extract);
@@ -398,6 +511,7 @@
$self->_add_comment($type, $value, $extract);
}
if ($expt->{DESIGN}->{LIBRARY_DESCRIPTOR}->{LIBRARY_LAYOUT}->{PAIRED}){
+ $paired = 1;
$value = "PAIRED";
$self->_add_comment($type, $value, $extract);
my $type_paired = "ORIENTATION";
@@ -415,6 +529,22 @@
$value = $expt->{DESIGN}->{LIBRARY_DESCRIPTOR}->{LIBRARY_LAYOUT}->{PAIRED}->{$type_paired};
$self->_add_comment($type_paired, $value, $extract);
}
+
+ #READ_SPEC for PAIRED READS
+ foreach my $read (@{ $expt->{"DESIGN"}->{"SPOT_DESCRIPTOR"}->{"SPOT_DECODE_SPEC"}->{"READ_SPEC"} || [] }){
+ my $type_paired = "READ_TYPE";
+ if ($read->{$type_paired}){
+ my $value = $read->{$type_paired};
+ if ($value eq "BarCode"){
+ $type_paired = "EXPECTED_BASECALL_TABLE";
+ if ($read->{$type_paired}->{BASECALL}->{content}){
+ $value = $read->{$type_paired}->{BASECALL}->{content};
+ $self->_add_comment("BARCODE", $value, $extract);
+ }
+ }
+ }
+ }
+
}
}
@@ -523,6 +653,7 @@
my $fastq_path = trim($entries[16]);
my $run_acc = trim($entries[3]);
$all_study_runs{$run_acc} = 1;
+
my $data = Bio::MAGETAB::DataAcquisition->new({
name => $run_acc,
@@ -560,9 +691,91 @@
# push @{ $self->get_all_nodes }, $data;
# }
#}
+
+ foreach my $run_acc (@{ $links->{"ENA-RUN"} || [] }){
+ # Add scan
+ if (!(exists($all_study_runs{$run_acc}))){
+ $all_study_runs{$run_acc} = 1;
+ if ($paired==1){
+ my $fastq_path = substr($run_acc,0,6);
+ my $scan_name = $run_acc."_1";
+ my $data_1 = Bio::MAGETAB::DataAcquisition->new({
+ name => $scan_name,
+ });
+
+
+ $self->_add_comment("FASTQ_URI", "ftp://ftp.sra.ebi.ac.uk/vol1/fastq/$fastq_path/$run_acc/$scan_name.fastq.gz", $data_1);
+ $self->_add_comment("ENA_RUN", $run_acc, $data_1);
+ my $data_edge = Bio::MAGETAB::Edge->new({
+ inputNode => $assay,
+ outputNode => $data_1,
+ });
+
+ $scan_name = $run_acc."_2";
+ my $data_2 = Bio::MAGETAB::DataAcquisition->new({
+ name => $scan_name,
+ });
+
+ $self->_add_comment("FASTQ_URI", "ftp://ftp.sra.ebi.ac.uk/vol1/fastq/$fastq_path/$run_acc/$scan_name.fastq.gz", $data_2);
+ $self->_add_comment("ENA_RUN", $run_acc, $data_2);
+ my $data_edge = Bio::MAGETAB::Edge->new({
+ inputNode => $assay,
+ outputNode => $data_2,
+ });
+
+
+ #READ_SPEC for PAIRED READS
+ foreach my $read (@{ $expt->{"DESIGN"}->{"SPOT_DESCRIPTOR"}->{"SPOT_DECODE_SPEC"}->{"READ_SPEC"} || [] }){
+
+ my $type_paired = "READ_TYPE";
+ if ($read->{$type_paired}){
+ my $value = $read->{$type_paired};
+ if ($value eq "Forward") {
+ $self->_add_comment($type_paired, $value, $data_1);
+ $type_paired = "BASE_COORD";
+ if ($read->{$type_paired}){
+ $value = $read->{$type_paired};
+ $self->_add_comment($type_paired, $value, $data_1);
+ }
+ }
+ if ($value eq "Reverse"){
+ $self->_add_comment($type_paired, $value, $data_2);
+ $type_paired = "BASE_COORD";
+ if ($read->{$type_paired}){
+ $value = $read->{$type_paired};
+ $self->_add_comment($type_paired, $value, $data_2);
+ }
+ }
+ }
+ }
+ push @{ $self->get_all_nodes }, $data_1;
+ push @{ $self->get_all_nodes }, $data_2;
+ }
+ else {
+ my $data = Bio::MAGETAB::DataAcquisition->new({
+ name => $run_acc,
+ });
+
+ my $fastq_path = substr($run_acc,1,6);
+ $self->_add_comment("FASTQ_URI", "ftp://ftp.sra.ebi.ac.uk/vol1/fastq/$fastq_path/$run_acc/$run_acc.fastq.gz", $data);
+ $self->_add_comment("ENA_RUN", $run_acc, $data);
+ my $data_edge = Bio::MAGETAB::Edge->new({
+ inputNode => $assay,
+ outputNode => $data,
+ });
+ push @{ $self->get_all_nodes }, $data;
+ }
+ }
+ }
+
}
+
+ while( my ($k, $v) = each(%study_type)) {
+ $self->_add_expt_comment("AEExperimentType", $k);
+ }
+
$self->log("Adding run list to IDF");
my $runs = get_range_from_list("ERR", keys %all_study_runs);
$self->_add_expt_comment("SequenceDataURI", "http://www.ebi.ac.uk/ena/data/view/$runs");
@@ -740,7 +953,7 @@
# If a category has more than 1 value make it a factor
foreach my $type (keys %types_to_values){
- #if ( (keys %{ $types_to_values{$type} }) > 1){
+ if ( ((keys %{ $types_to_values{$type} }) > 1) or ($type =~ /Immunoprecipitate/i)){
$self->log("Adding $type factor values");
@@ -776,7 +989,7 @@
$row->set_factorValues( [ @{ $existing || [] }, $fv ] );
}
}
- #}
+ }
}
return;
}
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <fa...@us...> - 2011-10-28 13:48:08
|
Revision: 2385
http://tab2mage.svn.sourceforge.net/tab2mage/?rev=2385&view=rev
Author: farne
Date: 2011-10-28 13:47:57 +0000 (Fri, 28 Oct 2011)
Log Message:
-----------
create 1 run for each pair of csfasta/qual files
Modified Paths:
--------------
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/RunSet.pm
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/RunSet.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/RunSet.pm 2011-10-25 15:48:23 UTC (rev 2384)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/RunSet.pm 2011-10-28 13:47:57 UTC (rev 2385)
@@ -32,6 +32,11 @@
sub create_runs{
my ($self) = @_;
foreach my $file (@{ $self->get_datafiles || []}){
+
+ # Do not create runs for qual files, istead add them
+ # to run with corresponding csfasta file
+ next if ($self->get_type($file->get_name) eq "qual");
+
$self->add_run($file);
}
}
@@ -70,17 +75,48 @@
$xml->emptyTag(
"FILE",
"filename"=>$file_name,
- filetype=>$self->get_type($file_name)
+ "filetype"=>$self->get_type($file_name)
);
+
+ # For csfasta files find corresponding qual file and add
+ if($self->get_type($file_name) eq "csfasta"){
+ my $qual_file = $self->find_file("qual", $assay);
+ if ($qual_file){
+ $xml->emptyTag(
+ "FILE",
+ "filename" => $qual_file,
+ "filetype" => "qual",
+ );
+ }
+ }
+
$xml->endTag;
$xml->endTag;
$xml->endTag;
}
+sub find_file{
+
+ my ($self, $type, $assay) = @_;
+
+ my $assay_name = $assay->getName;
+ print "Getting $type file for assay $assay_name\n";
+ my @mbas = $self->get_mbas_from_pba_name($assay_name);
+ foreach my $mba (@mbas){
+ foreach my $file (@{ $mba->getMeasuredBioAssayData || []}){
+ if ($self->get_type($file->getName) eq $type){
+ return $file->getName;
+ }
+ }
+ }
+
+ return;
+}
+
sub get_type{
my ($self, $name) = @_;
- my @accepted_types = qw(srf sff fastq bam);
+ my @accepted_types = qw(srf sff fastq bam csfasta qual);
# Strip off .gz before attempting to get type
$name =~ s/\.gz$//g;
@@ -88,6 +124,8 @@
$name =~ /.*\.([^\.]*)$/g;
my $type = $1;
+ $type = lc($type);
+
$type = "fastq" if $type eq "fq";
unless (grep $type, @accepted_types){
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <fa...@us...> - 2011-10-25 15:48:30
|
Revision: 2384
http://tab2mage.svn.sourceforge.net/tab2mage/?rev=2384&view=rev
Author: farne
Date: 2011-10-25 15:48:23 +0000 (Tue, 25 Oct 2011)
Log Message:
-----------
improved SPOT_DESCRIPTOR generation. added -x flag to skip data file MD5 calculation.
Modified Paths:
--------------
branches/ae-dev/Tab2MAGE/bin/mtab_to_sra_xml.pl
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/ExperimentSet.pm
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/Submission.pm
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/XML.pm
Modified: branches/ae-dev/Tab2MAGE/bin/mtab_to_sra_xml.pl
===================================================================
--- branches/ae-dev/Tab2MAGE/bin/mtab_to_sra_xml.pl 2011-10-25 15:10:01 UTC (rev 2383)
+++ branches/ae-dev/Tab2MAGE/bin/mtab_to_sra_xml.pl 2011-10-25 15:48:23 UTC (rev 2384)
@@ -47,10 +47,9 @@
"c|clobber" => \$args{clobber},
"s|schema=s" => \$args{schema_dir},
"a|abbreviations=s" => \$args{abbreviations},
+ "x|skip_data" => \$args{skip_data},
);
-
-# FIX: possibly add args to specify platform and single or paired when running script
-# in case this info is not in spreadsheet
+
if ($want_help) {
pod2usage(
-exitval => 255,
@@ -132,7 +131,9 @@
%common_args,
output_file => $acc."_submission.xml",
clobber => $magetab->get_clobber,
+ skip_data => $args->{skip_data},
});
+
print "done\n";
if ($submission){
@@ -237,6 +238,11 @@
documents are not compliant with the MAGE-TAB format specification;
this format is used by ArrayExpress to simplify data submissions.
+=item B<-x>
+
+Skip all data file checking (including presence/absence check).
+File MD5 sums will be missing from submission xml.
+
=item B<-c>
Overwrite preexisting files ("clobber" option).
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/ExperimentSet.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/ExperimentSet.pm 2011-10-25 15:10:01 UTC (rev 2383)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/ExperimentSet.pm 2011-10-25 15:48:23 UTC (rev 2384)
@@ -14,204 +14,6 @@
use base qw(ArrayExpress::SRA_XML::XML);
-#################################################
-# Spot descriptors hard coded based on platform #
-#################################################
-
-my ($illumina_single, $illumina_paired,
- $ls454_single, $ls454_paired,
- $solid_single, $solid_paired ) = ({}, {}, {}, {}, {}, {});
-
-# Example spot descriptor for Illumina Genome Analyzer (Solexa) & SOLiD Single read
-$illumina_single->{spot_descriptor} = <<END;
-
-<SPOT_DECODE_SPEC>
- <SPOT_LENGTH> </SPOT_LENGTH>
- <READ_SPEC>
- <READ_INDEX>0</READ_INDEX>
- <READ_CLASS>Application Read</READ_CLASS>
- <READ_TYPE>Forward</READ_TYPE>
- <BASE_COORD>1</BASE_COORD>
- </READ_SPEC>
-</SPOT_DECODE_SPEC>
-END
-
-$illumina_single->{processing} =<<END;
-
- <BASE_CALLS>
- <SEQUENCE_SPACE>Base Space</SEQUENCE_SPACE>
- <BASE_CALLER>Solexa primary analysis</BASE_CALLER>
- </BASE_CALLS>
- <QUALITY_SCORES qtype="other">
- <QUALITY_SCORER>Solexa primary analysis</QUALITY_SCORER>
- <NUMBER_OF_LEVELS>80</NUMBER_OF_LEVELS>
- <MULTIPLIER>1</MULTIPLIER>
- </QUALITY_SCORES>
-END
-
-# Example spot descriptor for Illumina Genome Analyzer (Solexa) & SOLiD paired read - read length 36
-$illumina_paired->{spot_descriptor} =<<END;
-
-<SPOT_DECODE_SPEC>
- <SPOT_LENGTH> </SPOT_LENGTH>
- <READ_SPEC>
- <READ_INDEX>0</READ_INDEX>
- <READ_CLASS>Application Read</READ_CLASS>
- <READ_TYPE>Forward</READ_TYPE>
- <BASE_COORD>1</BASE_COORD>
- </READ_SPEC>
- <READ_SPEC>
- <READ_INDEX>1</READ_INDEX>
- <READ_CLASS>Application Read</READ_CLASS>
- <READ_TYPE>Reverse</READ_TYPE>
- <BASE_COORD>TODO:READ_INDEX_1_BASE_COORD</BASE_COORD>
- </READ_SPEC>
-</SPOT_DECODE_SPEC>
-END
-
-$illumina_paired->{processing} =<<END;
-
- <BASE_CALLS>
- <SEQUENCE_SPACE>Base Space</SEQUENCE_SPACE>
- <BASE_CALLER>Solexa primary analysis</BASE_CALLER>
- </BASE_CALLS>
- <QUALITY_SCORES qtype="other">
- <QUALITY_SCORER>Solexa primary analysis</QUALITY_SCORER>
- <NUMBER_OF_LEVELS>80</NUMBER_OF_LEVELS>
- <MULTIPLIER>1</MULTIPLIER>
- </QUALITY_SCORES>
-END
-
-# Example spot descriptor for 454 Single read
-$ls454_single->{spot_descriptor} =<<END;
-
-<SPOT_DECODE_SPEC>
-<READ_SPEC>
- <READ_INDEX>0</READ_INDEX>
- <READ_CLASS>Technical Read</READ_CLASS>
- <READ_TYPE>Adapter</READ_TYPE>
- <BASE_COORD>1</BASE_COORD>
- </READ_SPEC>
- <READ_SPEC>
- <READ_INDEX>1</READ_INDEX>
- <READ_CLASS>Application Read</READ_CLASS>
- <READ_TYPE>Forward</READ_TYPE>
- <BASE_COORD>5</BASE_COORD>
- </READ_SPEC>
-</SPOT_DECODE_SPEC>
-END
-
-$ls454_single->{processing} =<<END;
-
- <BASE_CALLS>
- <SEQUENCE_SPACE>Base Space</SEQUENCE_SPACE>
- <BASE_CALLER>454 Basecaller</BASE_CALLER>
- </BASE_CALLS>
- <QUALITY_SCORES qtype="phred">
- <QUALITY_SCORER>454 Basecaller</QUALITY_SCORER>
- <NUMBER_OF_LEVELS>64</NUMBER_OF_LEVELS>
- <MULTIPLIER>1</MULTIPLIER>
- </QUALITY_SCORES>
-END
-
-# Example spot descriptor for 454 Paired read
-$ls454_paired->{spot_descriptor} =<<END;
-
-<SPOT_DECODE_SPEC>
-<ADAPTER_SPEC>TCAG</ADAPTER_SPEC>
-<READ_SPEC>
- <READ_INDEX>0</READ_INDEX>
- <READ_CLASS>Technical Read</READ_CLASS>
- <READ_TYPE>Adapter</READ_TYPE>
- <BASE_COORD>1</BASE_COORD>
-</READ_SPEC>
-<READ_SPEC>
- <READ_INDEX>1</READ_INDEX>
- <READ_CLASS>Application Read</READ_CLASS>
- <READ_TYPE>Forward</READ_TYPE>
- <BASE_COORD>5</BASE_COORD>
-</READ_SPEC>
-<READ_SPEC>
- <READ_INDEX>2</READ_INDEX>
- <READ_CLASS>Technical Read</READ_CLASS>
- <READ_TYPE>Linker</READ_TYPE>
- <EXPECTED_BASECALL>GTTGGAACCGAAAGGGTTTGAATTCAAACCCTTTCGGTTCCAAC</EXPECTED_BASECALL>
-</READ_SPEC>
-<READ_SPEC>
- <READ_INDEX>3</READ_INDEX>
- <READ_CLASS>Application Read</READ_CLASS>
- <READ_TYPE>Reverse</READ_TYPE>
- <EXPECTED_BASECALL>Following Linker</EXPECTED_BASECALL>
-</READ_SPEC>
-</SPOT_DECODE_SPEC>
-END
-
-$ls454_paired->{processing} =<<END;
-
- <BASE_CALLS>
- <SEQUENCE_SPACE>Base Space</SEQUENCE_SPACE>
- <BASE_CALLER>454 Basecaller</BASE_CALLER>
- </BASE_CALLS>
- <QUALITY_SCORES qtype="phred">
- <QUALITY_SCORER>454 Basecaller</QUALITY_SCORER>
- <NUMBER_OF_LEVELS>64</NUMBER_OF_LEVELS>
- <MULTIPLIER>1</MULTIPLIER>
- </QUALITY_SCORES>
-END
-
-$solid_single->{spot_descriptor} =<<END;
-
- <SPOT_DECODE_SPEC>
- <SPOT_LENGTH> </SPOT_LENGTH>
- <READ_SPEC>
- <READ_INDEX>0</READ_INDEX>
- <READ_CLASS>Application Read</READ_CLASS>
- <READ_TYPE>Forward</READ_TYPE>
- <BASE_COORD>1</BASE_COORD>
- </READ_SPEC>
- </SPOT_DECODE_SPEC>
-END
-
-$solid_single->{processing} =<<END;
-
- <BASE_CALLS>
- <SEQUENCE_SPACE>Color Space</SEQUENCE_SPACE>
- <BASE_CALLER>TODO: corona-VERSION OR SOLiD primary analysis</BASE_CALLER>
- </BASE_CALLS>
- <QUALITY_SCORES qtype="other">
- <QUALITY_SCORER>TODO: corona-VERSION OR SOLiD primary analysis</QUALITY_SCORER>
- <NUMBER_OF_LEVELS>100</NUMBER_OF_LEVELS>
- <MULTIPLIER>1</MULTIPLIER>
- </QUALITY_SCORES>
-END
-
-$solid_paired->{spot_descriptor} =<<END;
-
- <SPOT_DECODE_SPEC>
- <SPOT_LENGTH> </SPOT_LENGTH>
- <ADAPTER_SPEC>F3/R3</ADAPTER_SPEC>
- <READ_SPEC>
- <READ_INDEX>0</READ_INDEX>
- <READ_CLASS>Application Read</READ_CLASS>
- <READ_TYPE>Forward</READ_TYPE>
- <BASE_COORD>1</BASE_COORD>
- </READ_SPEC>
- </SPOT_DECODE_SPEC>
-END
-
-$solid_paired->{processing} =<<END;
-
- <BASE_CALLS>
- <SEQUENCE_SPACE>Color Space</SEQUENCE_SPACE>
- <BASE_CALLER>TODO: corona-VERSION OR SOLiD primary analysis</BASE_CALLER>
- </BASE_CALLS>
- <QUALITY_SCORES qtype="other">
- <QUALITY_SCORER>TODO: corona-VERSION OR SOLiD primary analysis</QUALITY_SCORER>
- <NUMBER_OF_LEVELS>100</NUMBER_OF_LEVELS>
- <MULTIPLIER>1</MULTIPLIER>
- </QUALITY_SCORES>
-END
-
sub START{
my ( $self, $id, $args ) = @_;
@@ -328,7 +130,8 @@
print "Getting comments from raw data file ". $mba->getName;
print "\n";
}
- $self->add_spot_descriptor($platform,$layout,\@extracts, $mba);
+
+ $self->add_spot_descriptor($platform,$layout,\@extracts,$assay_name);
$xml->endTag;
@@ -357,107 +160,295 @@
$xml->endTag;
}
-sub get_default_xml{
- my ($self, $platform, $layout) = @_;
-
- my $default = {};
+sub add_spot_descriptor{
+ my ($self, $platform, $layout, $extracts_ref, $assay_name) = @_;
+
+ my @extracts = @$extracts_ref;
+ # If there are multiple extracts per assay then barcodes
+ # must be used to identify sequence for each extract
+ my $is_barcode = $#extracts;
- if ($platform =~ /Illumina/i){
- if ($layout =~ /PAIRED/i){
- $default = $illumina_paired;
- }
- else{
- $default = $illumina_single;
- }
+ my @mbas = $self->get_mbas_from_pba_name($assay_name);
+ my @files;
+ my @mba_comments;
+
+ foreach my $mba (@mbas){
+ push @files, @{ $mba->getMeasuredBioAssayData || []};
+ push @mba_comments, @{ $mba->getPropertySets() || [] };
}
- elsif ($platform =~ /454/){
- if ($layout =~ /PAIRED/i){
- $default = $ls454_paired;
- }
- else{
- $default = $ls454_single;
- }
+
+ print "Data for $assay_name:\n";
+ my %file_types;
+ foreach my $file (@files){
+ my $name = $file->getName;
+ print "File: ".$name."\n";
+
+ # remove archive extensions
+ $name=~s/\.gz//ig;
+ $name=~s/\.zip//ig;
+ $name=~s/\.bz2//ig;
+ $name=~s/\.tar//ig;
+
+ $name=~ /\.(\w*)$/g;
+ my $type = $1;
+ $type = lc($type);
+ $type = "fastq" if $type eq "fq";
+ $file_types{$type}++;
+
}
- elsif ($platform =~ /solid/){
- if ($layout =~ /PAIRED/i){
- $default = $solid_paired;
- }
- else{
- $default = $solid_single;
- }
+
+ while ( my ($key, $value) = each(%file_types) ){
+ print "File type $key: $value found\n";
}
- return $default;
-}
-
-sub add_spot_descriptor{
- my ($self, $platform, $layout, $extracts_ref, $mba) = @_;
- my $xml = $self->get_xml_writer;
+ # No spot descriptor needed for:
+ # BAM files
+ # SFF files if single and no barcode
+ # fastq files if single and no barcode or tech read
+ # 1 pair SOLiD csfasta/qual and single and not barcode
+ # 2 pairs SODiD csfasta/qual and paired and not barcode
- my $default = $self->get_default_xml($platform, $layout);
- my $descriptor = $default->{spot_descriptor};
-
- # Get comments from associated raw data file
- my @comments = @{ $mba->getPropertySets() || [] };
- foreach my $comment (@comments){
- my $type = $comment->getName;
- my $value = $comment->getValue;
-
- # If comment type is present in descriptor then replace
- # TODO:type text with value from comment
- if ($type and $descriptor =~ m/TODO:$type/){
- print "Inserting $type value $value\n";
- $descriptor =~ s/TODO:$type/$value/;
- }
+ # tested
+ if ($file_types{"bam"}
+ and !grep { $_ ne "bam" } keys %file_types){
+
+ print "No spot descriptor needed for bam files\n";
+ return;
}
- # Create barcode read spec if applicable
- my $barcode_read_spec;
- my @extracts = @$extracts_ref;
- if($#extracts){
- print "Creating barcode READ_SPEC\n";
- my $spec_writer = XML::Writer->new(OUTPUT=>\$barcode_read_spec, DATA_MODE=>1, DATA_INDENT=>4, UNSAFE=>1);
- $spec_writer->startTag("READ_SPEC");
- $spec_writer->dataElement("READ_INDEX","1");
- $spec_writer->dataElement("READ_LABEL","barcode_tag");
- $spec_writer->dataElement("READ_CLASS","Technical Read");
- $spec_writer->dataElement("READ_TYPE","BarCode");
- $spec_writer->startTag("EXPECTED_BASECALL_TABLE");
- foreach my $extract (@extracts){
- my $tag = $self->get_nvt($extract, "BARCODE");
- $spec_writer->startTag("BASECALL", read_group_tag=>$tag,
- min_match=>length($tag),
- max_mismatch=>"0",
- match_edge=>"full");
- $spec_writer->characters($tag);
- $spec_writer->endTag();
- }
- $spec_writer->endTag();
- $spec_writer->endTag;
+ # tested
+ if ($file_types{"sff"}
+ and !grep { $_ ne "sff" } keys %file_types
+ and $layout =~ /SINGLE/i
+ and not $is_barcode){
+
+ print "No spot decriptor needed for single read SFF files with no barcodes\n";
+ return;
}
- if($descriptor){
- # insert barcode specific stuff
- if ($barcode_read_spec){
- print "Adding barcode READ_SPEC to spot descriptor\n";
- $descriptor =~ s/<\/SPOT_DECODE_SPEC>/\n$barcode_read_spec\n<\/SPOT_DECODE_SPEC>/g;
- }
- }
- else{
- print "WARNING: no spot descriptor defined for platform $platform\n";
- $descriptor = "<!-- No spot descriptor defined for platorm $platform. This must be completed manually -->\n";
+ # tested
+ if ($file_types{"fastq"}
+ and !grep { $_ ne "fastq" } keys %file_types
+ and $layout =~ /SINGLE/i
+ and not $is_barcode
+ and not grep{ $_->getValue eq "Technical Read" } @mba_comments){
+
+ print "No spot decriptor needed for single read fastq files with no barcodes and no technical reads\n";
+ return;
+ }
+
+ # tested
+ if ($file_types{"csfasta"} == 1
+ and $file_types{"qual"} == 1
+ and @files == 2
+ and $layout =~ /SINGLE/i
+ and not $is_barcode){
- if ($barcode_read_spec){
- # insert barcode specific stuff anyway
- $descriptor.="\n$barcode_read_spec\n";
- }
+ print "No spot descriptor needed for 1 pair of single read csfasta/qual files with no barcodes\n";
+ return;
}
- $xml->startTag("SPOT_DESCRIPTOR");
- $xml->raw($descriptor);
- $xml->endTag;
+ # tested
+ if ($file_types{"csfasta"} == 2
+ and $file_types{"qual"} == 2
+ and @files == 4
+ and $layout =~ /PAIRED/i
+ and not $is_barcode){
+
+ print "No spot descriptor needed for 2 pairs of paired read csfasta/qual files with no barcodes\n";
+ return;
+ }
+
+ # Minimal spot descriptor needed for:
+ # fastq files if paired with no barcodes or other tech reads
+ # SFF files if paired with no barcodes
+
+ # tested
+ if ($file_types{"fastq"} == 2
+ and @files == 2
+ and $layout =~ /PAIRED/i
+ and not $is_barcode
+ and not grep{ $_->getValue eq "Technical Read" } @mba_comments){
+ # FIXME: create example to test this
+ print "Creating minimal spot descriptor for paired read fastq files with no barcodes or technical reads\n";
+ my $spec_0 = { "READ_INDEX" => 0, "READ_LABEL" => "F", "READ_CLASS" => "Application Read", "READ_TYPE" => "Forward", "RELATIVE_ORDER" => ""};
+ my $spec_1 = { "READ_INDEX" => 1, "READ_LABEL" => "R", "READ_CLASS" => "Application Read", "READ_TYPE" => "Reverse", "RELATIVE_ORDER" => ""};
+
+ $self->add_spot_descriptor_from_spec_list($spec_0,$spec_1);
+ return;
+ }
+
+ # tested
+ if ($file_types{"sff"}
+ and not grep {$_ ne "sff"} keys %file_types
+ and $layout =~ /PAIRED/i
+ and not $is_barcode){
+
+ print "Creating minimal spot descriptor for paired read SFF files with no barcodes\n";
+ my $linker_basecall = $self->get_required_comment($assay_name, "LINKER_BASECALL", @mba_comments);
+ my $basecall_table = [{ "BASECALL" => $linker_basecall }];
+ my $spec_0 = { "READ_INDEX" => 0, "READ_LABEL" => "Linker Read", "READ_CLASS" => "Technical Read", "READ_TYPE" => "Linker",
+ "EXPECTED_BASECALL_TABLE" => $basecall_table };
+
+ $self->add_spot_descriptor_from_spec_list($spec_0);
+ return;
+ }
+
+ # Create all the other types of spot descriptor
+
+ # tested
+ my $barcode_spec;
+ if ($is_barcode){
+ print "Creating barcode READ_SPEC\n";
+ my @bc_table;
+ my $index = $self->get_required_comment($assay_name, "BARCODE_READ_INDEX", @mba_comments);
+
+ foreach my $extract (@extracts){
+ my $tag = $self->get_nvt($extract, "BARCODE");
+ my $basecall = { "BASECALL" => $tag, "read_group_tag" => $tag };
+ push @bc_table, $basecall;
+ }
+
+ $barcode_spec = { "READ_INDEX" => $index, "READ_LABEL" =>"Barcode Read", "READ_CLASS" => "Technical Read", "READ_TYPE" => "BarCode",
+ "EXPECTED_BASECALL_TABLE" => \@bc_table };
+ }
+
+ my $forward_0_spec;
+ $forward_0_spec = { "READ_INDEX" => 0, "READ_CLASS" => "Application Read", "READ_TYPE" => "Forward", "BASE_COORD" => 1 };
+
+ # Illumina and SOLiD single are the same - tested
+ if ($layout =~ /SINGLE/i
+ and $platform =~ /(Illumina|SOLiD)/i){
+ $forward_0_spec->{"SPOT_LENGTH"} = $self->get_required_comment($assay_name, "SPOT_LENGTH", @mba_comments);
+ $self->add_spot_descriptor_from_spec_list($forward_0_spec, $barcode_spec);
+ return;
+ }
+
+ # Illumina paired - tested
+ if ($layout =~ /PAIRED/i
+ and $platform =~ /Illumina/i){
+ $forward_0_spec->{"SPOT_LENGTH"} = $self->get_required_comment($assay_name, "SPOT_LENGTH", @mba_comments);
+ my $coord = $self->get_required_comment($assay_name, "READ_INDEX_1_BASE_COORD",@mba_comments);
+ my $spec_1 = { "READ_INDEX" => 1, "READ_CLASS" => "Application Read", "READ_TYPE" => "Reverse", "BASE_COORD" => $coord };
+ $self->add_spot_descriptor_from_spec_list($forward_0_spec, $spec_1, $barcode_spec);
+ return;
+ }
+
+ # SOLiD paired - tested
+ if ($layout =~ /PAIRED/i
+ and $platform =~ /SOLiD/i){
+ $forward_0_spec->{"SPOT_LENGTH"} = $self->get_required_comment($assay_name, "SPOT_LENGTH", @mba_comments);
+ my $coord = $self->get_required_comment($assay_name, "READ_INDEX_1_BASE_COORD",@mba_comments);
+ my $spec_1 = { "READ_INDEX" => 1, "READ_CLASS" => "Application Read", "READ_TYPE" => "Forward", "BASE_COORD" => $coord };
+ $self->add_spot_descriptor_from_spec_list($forward_0_spec, $spec_1, $barcode_spec);
+ return;
+ }
+
+ # 454 single - tested
+ if( $layout =~ /SINGLE/i
+ and $platform =~ /454/){
+ my $spec_0 = { "READ_INDEX" => 0, "READ_CLASS" => "Technical Read", "READ_TYPE" => "Adapter", "BASE_COORD" => 1 };
+ my $spec_1 = { "READ_INDEX" => 1, "READ_CLASS" => "Application Read", "READ_TYPE" => "Forward", "BASE_COORD" => 5 };
+ $self->add_spot_descriptor_from_spec_list($spec_0, $spec_1, $barcode_spec);
+ return;
+ }
+
+ # 454 paired - tested
+ if( $layout =~ /PAIRED/i
+ and $platform =~ /454/){
+ my $linker = $self->get_required_comment($assay_name, "LINKER_BASECALL", @mba_comments);
+ my $spec_0 = { "READ_INDEX" => 0, "READ_CLASS" => "Technical Read", "READ_TYPE" => "Adapter", "BASE_COORD" => 1 };
+ my $spec_1 = { "READ_INDEX" => 1, "READ_CLASS" => "Application Read", "READ_TYPE" => "Forward", "BASE_COORD" => 5 };
+ my $spec_2 = { "READ_INDEX" => 2, "READ_CLASS" => "Technical Read", "READ_TYPE" => "Linker", "EXPECTED_BASECALL" => $linker };
+ my $spec_3 = { "READ_INDEX" => 3, "READ_CLASS" => "Application Read", "READ_TYPE" => "Reverse", "RELATIVE_ORDER" => {"follows_read_index" => 2} };
+ $self->add_spot_descriptor_from_spec_list($spec_0, $spec_1, $spec_2, $spec_3, $barcode_spec);
+ return;
+ }
+
+ # No standard spot descriptor for this technology
+ print "ERROR: no standard spot descriptor available for $platform assay $assay_name. Add it yourself".
+ " (or write more code to fetch it from MAGETAB comments in ExperimentSet.pm add_spot_descriptor subroutine)\n";
+ return;
+
}
+sub get_required_comment{
+
+ my ($self, $item_name, $comment_name, @comments) = @_;
+
+ # FIXME: assumes we never have multiple comments of same type
+ my ($comment) = grep { $_->getName eq $comment_name } @comments;
+
+ if ($comment){
+ return $comment->getValue;
+ }
+ else{
+ print "ERROR - REQUIRED COMMENT MISSING: no $comment_name comment found for $item_name\n";
+ return undef;
+ }
+
+}
+sub add_spot_descriptor_from_spec_list{
+
+ my ($self, @spec_list) = @_;
+
+ my $xml = $self->get_xml_writer;
+
+ $xml->startTag("SPOT_DESCRIPTOR");
+ $xml->startTag("SPOT_DECODE_SPEC");
+
+ my $spot_length = $spec_list[0]->{"SPOT_LENGTH"};
+ if ($spot_length){
+ $xml->dataElement("SPOT_LENGTH", $spot_length);
+ }
+
+ my @possible_elements = qw(
+ READ_INDEX
+ READ_LABEL
+ READ_CLASS
+ READ_TYPE
+ RELATIVE_ORDER
+ BASE_COORD
+ CYCLE_COORD
+ EXPECTED_BASECALL
+ );
+
+ foreach my $spec (@spec_list){
+ next unless $spec;
+ $xml->startTag("READ_SPEC");
+ foreach my $element (@possible_elements){
+ if (exists $spec->{$element}){
+ my $value = $spec->{$element};
+ if (ref($value) eq "HASH"){
+ $xml->dataElement($element, "", %$value);
+ }
+ else{
+ $xml->dataElement($element, $value);
+ }
+ }
+ }
+ my $ebt = "EXPECTED_BASECALL_TABLE";
+ if (exists $spec->{$ebt}){
+ $xml->startTag($ebt);
+ my $basecall_table = $spec->{$ebt};
+ foreach my $basecall (@$basecall_table){
+ my $content = $basecall->{'BASECALL'};
+ delete $basecall->{'BASECALL'};
+ $xml->startTag("BASECALL", %$basecall);
+ $xml->characters($content);
+ $xml->endTag;
+ }
+ $xml->endTag;
+ }
+ $xml->endTag;
+ }
+
+ $xml->endTag;
+ $xml->endTag;
+
+ return;
+}
+
sub add_processing{
my ($self, $platform, $layout) = @_;
my $xml = $self->get_xml_writer;
@@ -465,18 +456,7 @@
# PROCESSING deprecated in v1.2 (November 2010)
# Empty element still needed
$xml->emptyTag("PROCESSING");
-
- #my $default = $self->get_default_xml($platform,$layout);
- #my $processing = $default->{processing};
-
- #unless ($processing){
- # print "WARNING: no processing xml defined for platform $platform\n";
- # $processing = "<!-- No processing xml defined for platorm $platform. This must be completed manually -->\n"
- #}
-
- #$xml->startTag("PROCESSING");
- #$xml->raw($processing);
- #$xml->endTag;
+
}
sub get_assay_hardware{
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/Submission.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/Submission.pm 2011-10-25 15:10:01 UTC (rev 2383)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/Submission.pm 2011-10-25 15:48:23 UTC (rev 2384)
@@ -118,7 +118,7 @@
foreach my $file (@files){
push @missing_files, $file->get_path unless -r $file->get_path;
}
- if (@missing_files){
+ if (@missing_files and !$self->get_skip_data){
die ("ERROR: the following data files were not found or unreadable:\n"
.join "\n", @missing_files);
}
@@ -141,15 +141,17 @@
# Check file exists before trying to get md5.
my $md5;
- if (-r $path){
- $md5 = $file->get_md5_digest;
+ unless($self->get_skip_data){
+ if (-r $path){
+ $md5 = $file->get_md5_digest;
+ }
+ else{
+ die "ERROR: File $name not found or unreadable\n";
+ }
+
+ print "File: $name\nMD5: $md5\n";
}
- else{
- die "ERROR: File $name not found or unreadable\n";
- }
- print "File: $name\nMD5: $md5\n";
-
$self->get_xml_writer->emptyTag('FILE',
filename=>$name,
checksum_method=>'MD5',
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/XML.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/XML.pm 2011-10-25 15:10:01 UTC (rev 2383)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/XML.pm 2011-10-25 15:48:23 UTC (rev 2384)
@@ -24,6 +24,7 @@
my %schema_type : ATTR( :name<schema_type>, :default<undef>);
my %scema_dir : ATTR( :name<schema_dir>, :default<"."> );
my %clobber : ATTR( :name<clobber>, :default<0> );
+my %skip_data : ATTR( :name<skip_data>, :default<0> );
my %name_file : ATTR( :name<name_file>, :default<undef> );
my %names_to_abbrvs : ATTR( :default<undef> );
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <fa...@us...> - 2011-10-25 15:10:07
|
Revision: 2383
http://tab2mage.svn.sourceforge.net/tab2mage/?rev=2383&view=rev
Author: farne
Date: 2011-10-25 15:10:01 +0000 (Tue, 25 Oct 2011)
Log Message:
-----------
bug fix - nested OE values of 0 were not being found
Modified Paths:
--------------
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/MAGETAB/Checker.pm
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/MAGETAB/Checker.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/MAGETAB/Checker.pm 2011-10-25 15:05:10 UTC (rev 2382)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/MAGETAB/Checker.pm 2011-10-25 15:10:01 UTC (rev 2383)
@@ -2171,7 +2171,10 @@
value => $value,
);
}
-
+ else{
+ croak("Error: Could not flatten nested ontology entry for $category");
+ }
+
return $flattened;
}
@@ -2187,9 +2190,10 @@
&& $oe->getCategory() ne $oe->getValue() );
foreach my $child ( @{ $oe->getAssociations() || [] } ) {
- if ( my $value = $self->find_nested_oe_value( $child ) ) {
- return $value;
- }
+ my $value = $self->find_nested_oe_value( $child );
+ if ( defined $value ) {
+ return $value;
+ }
}
return;
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <fa...@us...> - 2011-10-25 15:05:21
|
Revision: 2382
http://tab2mage.svn.sourceforge.net/tab2mage/?rev=2382&view=rev
Author: farne
Date: 2011-10-25 15:05:10 +0000 (Tue, 25 Oct 2011)
Log Message:
-----------
Amy's additional checks for factor values that do not vary
Modified Paths:
--------------
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/ExperimentChecker.pm
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/MAGETAB/Checker.pm
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/ExperimentChecker.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/ExperimentChecker.pm 2011-10-17 13:38:24 UTC (rev 2381)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/ExperimentChecker.pm 2011-10-25 15:05:10 UTC (rev 2382)
@@ -2726,6 +2726,106 @@
return;
}
+
+sub compare_bmcs_to_fvs : RESTRICTED {
+
+ my ( $self, $fvhashlist, $bmchashlist ) = @_;
+
+ # Takes lists of hashrefs, each hashref containing a single
+ # category => value pair, compares them and logs the results if
+ # there's a factor value which is invariable and hence should
+ # have been declared as a biomaterial characteristic.
+
+ # Factor types "COMPOUND" and "IRRADIATE" are associated with
+ # type "DOSE" and "RADIATION_UNIT" respectively, and have to be
+ # considered in pairs (i.e. COMPOUND + DOSE, or IRRADIATE +
+ # RADIATION_UNIT)
+
+ my %listed_as_bmc;
+
+ foreach my $pair ( @{ $bmchashlist } ) {
+ while ( my ( $cat, $val ) = each %{ $pair } ) {
+ $listed_as_bmc{$cat} = 1;
+ }
+ }
+
+ my %fv_valuecount;
+ foreach my $pair ( @{ $fvhashlist } ) {
+ while ( my ( $cat, $val ) = each %{ $pair } ) {
+ $fv_valuecount{$cat}{$val}++;
+ }
+ }
+
+ my ($compound_count, $dose_count, $irradiate_count, $radiation_unit_count) = 0;
+
+ while ( my ( $cat, $valhash ) = each %fv_valuecount ) {
+ my %hash = %{$valhash};
+ my @keys = keys%hash;
+
+ if ($cat =~ /compound/i) {
+ $compound_count = values %{ $valhash };
+ } elsif ($cat =~ /dose/i) {
+ $dose_count = values %{ $valhash };
+ } elsif ($cat =~ /irradiate/i) {
+ $irradiate_count = values % { $valhash };
+ } elsif ($cat =~ /radiation_unit/i) {
+ $radiation_unit_count = values %{ $valhash };
+ } elsif ( scalar(grep { defined $_ } values %{ $valhash }) == 1 ) {
+
+ if ($cat =~ /immunoprecipitate/i) {
+ my $warning = sprintf
+ (qq{Warning: The value of factor "$cat" is invariable but may be tolerated if this is a ChIP experiment.\n});
+ $self->logprint(
+ 'error',
+ $warning,
+ );
+ next;
+ }
+
+ my $error = sprintf(
+ qq{Error: The value of factor "$cat" is invariable and should not be listed as a FactorValue.\n},
+ );
+ $self->logprint(
+ 'error', $error,
+ );
+ $self->add_error( $CONFIG->get_ERROR_PARSEBAD() );
+ if ($self->isa("ArrayExpress::MAGETAB::Checker")){
+ $self->add_validation_error($error);
+ }
+ }
+ }
+
+ if (defined($compound_count) && $compound_count == 1 && $dose_count == 1) {
+ my $error = sprintf(
+ qq{Error: The values of factors "COMPOUND" and "DOSE" are both invariable.\n},
+ );
+ $self->logprint (
+ 'error', $error
+ );
+ $self->add_error( $CONFIG->get_ERROR_PARSEBAD() );
+ if ($self->isa("ArrayExpress::MAGETAB::Checker")){
+ $self->add_validation_error($error);
+ }
+ }
+
+ if (defined($irradiate_count) && $irradiate_count == 1 && $radiation_unit_count == 1) {
+ my $error = sprintf (
+ qq{Error: The values of factors "IRRADIATE" and "RADIATION_UNIT" are both invariable.\n},
+ );
+ $self->logprint (
+ 'error', $error
+ );
+ $self->add_error( $CONFIG->get_ERROR_PARSEBAD() );
+ if ($self->isa("ArrayExpress::MAGETAB::Checker")){
+ $self->add_validation_error($error);
+ }
+ }
+
+ return;
+}
+
+
+
sub check_files_are_used {
my ($self, $files_used) = @_;
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/MAGETAB/Checker.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/MAGETAB/Checker.pm 2011-10-17 13:38:24 UTC (rev 2381)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/MAGETAB/Checker.pm 2011-10-25 15:05:10 UTC (rev 2382)
@@ -2126,9 +2126,26 @@
push @bmc_catvals, { $category => $value };
}
}
- $self->compare_fvs_to_bmcs(\@fv_catvals, \@bmc_catvals);
- }
+ # sometimes there are no characteristics defined for source,
+ # sample, extract or labeledextract. If the hashlist is
+ # empty, there is nothing to compare to factor values.
+
+ if ( scalar(@bmc_catvals) ) {
+ $self->compare_fvs_to_bmcs(\@fv_catvals, \@bmc_catvals);
+ if (scalar @{ $bags->{'pba'}->() } > 1) {
+ $self->compare_bmcs_to_fvs(\@fv_catvals, \@bmc_catvals);
+ } else {
+ my $warning = sprintf
+ (qq{Warning: Not checking factor values as there is only one hybridisation in this experiment.\n});
+ $self->logprint(
+ 'error',
+ $warning,
+ );
+ }
+ }
+ }
+
return;
}
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <fa...@us...> - 2011-10-17 13:38:35
|
Revision: 2381
http://tab2mage.svn.sourceforge.net/tab2mage/?rev=2381&view=rev
Author: farne
Date: 2011-10-17 13:38:24 +0000 (Mon, 17 Oct 2011)
Log Message:
-----------
spot descriptor now uses READ_INDEX_1_BASE_COORD comment from magetab file. spot descriptor construction still needs a lot of improvement.
Modified Paths:
--------------
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/ExperimentSet.pm
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/XML.pm
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/ExperimentSet.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/ExperimentSet.pm 2011-09-29 16:09:00 UTC (rev 2380)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/ExperimentSet.pm 2011-10-17 13:38:24 UTC (rev 2381)
@@ -64,7 +64,7 @@
<READ_INDEX>1</READ_INDEX>
<READ_CLASS>Application Read</READ_CLASS>
<READ_TYPE>Reverse</READ_TYPE>
- <BASE_COORD>37</BASE_COORD>
+ <BASE_COORD>TODO:READ_INDEX_1_BASE_COORD</BASE_COORD>
</READ_SPEC>
</SPOT_DECODE_SPEC>
END
@@ -318,7 +318,17 @@
$xml->endTag;
# Insert default spot descriptor xml
- $self->add_spot_descriptor($platform,$layout,\@extracts);
+ # We need to pass the raw data object so we can get
+ # annotations such as READ_INDEX_1_BASE_COORD from it
+ # to replace some default values
+ my @mbas = $self->get_mbas_from_pba_name($assay_name);
+ my $mba = $mbas[0];
+ if($#mbas){
+ print "Warning: more than one raw data file found for assay $assay_name. ";
+ print "Getting comments from raw data file ". $mba->getName;
+ print "\n";
+ }
+ $self->add_spot_descriptor($platform,$layout,\@extracts, $mba);
$xml->endTag;
@@ -381,11 +391,25 @@
}
sub add_spot_descriptor{
- my ($self, $platform, $layout, $extracts_ref) = @_;
+ my ($self, $platform, $layout, $extracts_ref, $mba) = @_;
my $xml = $self->get_xml_writer;
my $default = $self->get_default_xml($platform, $layout);
my $descriptor = $default->{spot_descriptor};
+
+ # Get comments from associated raw data file
+ my @comments = @{ $mba->getPropertySets() || [] };
+ foreach my $comment (@comments){
+ my $type = $comment->getName;
+ my $value = $comment->getValue;
+
+ # If comment type is present in descriptor then replace
+ # TODO:type text with value from comment
+ if ($type and $descriptor =~ m/TODO:$type/){
+ print "Inserting $type value $value\n";
+ $descriptor =~ s/TODO:$type/$value/;
+ }
+ }
# Create barcode read spec if applicable
my $barcode_read_spec;
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/XML.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/XML.pm 2011-09-29 16:09:00 UTC (rev 2380)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/XML.pm 2011-10-17 13:38:24 UTC (rev 2381)
@@ -511,6 +511,26 @@
return undef;
}
+sub get_mbas_from_pba_name{
+
+ my ($self, $assay_name) = @_;
+
+ my @mba_list;
+ my $all_assays = $self->get_bioassays;
+
+ foreach my $mba(@$all_assays){
+ # We only want MBAs
+ next unless $mba->isa("Bio::MAGE::BioAssay::MeasuredBioAssay");
+ my $fext = $mba->getFeatureExtraction;
+ my $source = $fext->getPhysicalBioAssaySource;
+ if ($source->getName eq $assay_name){
+ push @mba_list, $mba;
+ }
+ }
+
+ return @mba_list;
+}
+
sub DEMOLISH{
my ( $self, $id ) = @_;
my $xml = $self->get_xml_writer;
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <fa...@us...> - 2011-09-29 16:09:10
|
Revision: 2380
http://tab2mage.svn.sourceforge.net/tab2mage/?rev=2380&view=rev
Author: farne
Date: 2011-09-29 16:09:00 +0000 (Thu, 29 Sep 2011)
Log Message:
-----------
some more updates from 2011 (mainly due to AE database redesign)
Modified Paths:
--------------
branches/ae-dev/Tab2MAGE/bin/validate_magetab.pl
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/ADFParser/ADFConvert.pm
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/ADFParser.pm
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/AE2Exporter.pm
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/Spreadsheet.pm
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/templates/intro.html
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/Common.pm
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/Database.pm
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/MAGETAB/Checker.pm
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/MAGETAB/SDRF.pm
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/MXExport/Protocol.pm
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/ExperimentSet.pm
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/SRAtoMAGETAB.pm
Modified: branches/ae-dev/Tab2MAGE/bin/validate_magetab.pl
===================================================================
--- branches/ae-dev/Tab2MAGE/bin/validate_magetab.pl 2011-09-29 15:12:26 UTC (rev 2379)
+++ branches/ae-dev/Tab2MAGE/bin/validate_magetab.pl 2011-09-29 16:09:00 UTC (rev 2380)
@@ -230,6 +230,7 @@
source_directory => $file_dir,
clobber => 1,
in_relaxed_mode => 0,
+ log_to_current_dir => $file_dir,
log_type => "validate",
);
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/ADFParser/ADFConvert.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/ADFParser/ADFConvert.pm 2011-09-29 15:12:26 UTC (rev 2379)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/ADFParser/ADFConvert.pm 2011-09-29 16:09:00 UTC (rev 2380)
@@ -74,6 +74,8 @@
"Comment[Description]",
"Comment[SubmittedName]",
"Comment[Organism]",
+ "Comment[ArrayExpressReleaseDate]",
+ "Comment[ArrayExpressSubmissionDate]",
"Printing Protocol",
"Technology Type",
"Technology Type Term Accession Number",
@@ -125,14 +127,15 @@
}
my $header = {};
-
- $header->{"Array Design Name"} = $info->get_name;
+ $header->{"Array Design Name"} = $info->get_name;
$header->{"Version"} = $info->get_version;
$header->{"Provider"} = $info->get_provider_name." (".$info->get_provider_email.")";
$header->{"Comment[ArrayExpressAccession]"} = $info->get_accession;
$header->{"Comment[Description]"} = $info->get_description;
$header->{"Comment[SubmittedName]"} = $info->get_name;
$header->{"Comment[Organism]"} = $info->get_species;
+ $header->{"Comment[ArrayExpressReleaseDate]"} = $info->get_comments->{'ArrayExpressReleaseDate'};
+ $header->{"Comment[ArrayExpressSubmissionDate]"} = $info->get_comments->{'ArrayExpressSubmissionDate'};
$header->{"Printing Protocol"} = $info->get_protocol_name.": ".$info->get_protocol_text;
$header->{"Technology Type"} = $info->get_tech_type_value;
$header->{"Surface Type"} = $info->get_surface_type_value;
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/ADFParser.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/ADFParser.pm 2011-09-29 15:12:26 UTC (rev 2379)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/ADFParser.pm 2011-09-29 16:09:00 UTC (rev 2380)
@@ -1547,7 +1547,7 @@
$BIOSEQUENCE_POLYMERTYPE,
$BIOSEQUENCE_TYPE,
$REPORTER_COMMENT,
- $REPORTER_COMMENT_COLUMN_PATTERN,
+ #$REPORTER_COMMENT_COLUMN_PATTERN,
);
if ($self->is_in_adf($COMPOSITE_IDENTIFIER)){
@@ -1797,7 +1797,7 @@
my $error = "Error on line $abs_line_num: Values for \"$value\" in lines $abs_line_num and"
." $orig_line_num do not match in column $ann_col\n";
print $log_fh $error;
- $duplicates_found = 1;
+ $duplicates_found = 1;
$self->set_error_status(1);
}
}
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/AE2Exporter.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/AE2Exporter.pm 2011-09-29 15:12:26 UTC (rev 2379)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/AE2Exporter.pm 2011-09-29 16:09:00 UTC (rev 2380)
@@ -286,7 +286,7 @@
foreach my $col (@prot_columns){
my $prot_name = $row->[$col];
my $acc = $prot_accs->{$prot_name};
- $row->[$col] = $acc;
+ $row->[$col] = $acc || $prot_name; #changed by Natalja from $row->[$col] = $acc;
}
}
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/Spreadsheet.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/Spreadsheet.pm 2011-09-29 15:12:26 UTC (rev 2379)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/Spreadsheet.pm 2011-09-29 16:09:00 UTC (rev 2380)
@@ -1062,7 +1062,7 @@
} },
{ 'Comment[LIBRARY_SOURCE]' => {
usage => $UHTS,
- help => 'one of GENOMIC, TRANSCRIPTOMIC, METAGENOMIC, SYNTHETIC, VIRAL RNA, OTHER',
+ help => 'one of GENOMIC, TRANSCRIPTOMIC, METAGENOMIC, METATRANSCRIPTOMIC, SYNTHETIC, VIRAL RNA, OTHER',
example => 'GENOMIC',
} },
{ 'Comment[LIBRARY_STRATEGY]' => {
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/templates/intro.html
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/templates/intro.html 2011-09-29 15:12:26 UTC (rev 2379)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/templates/intro.html 2011-09-29 16:09:00 UTC (rev 2380)
@@ -1,6 +1,14 @@
<!-- TMPL_INCLUDE HEADER -->
<h1><!-- TMPL_VAR NAME=TITLE --></h1>
+<table class="contenttable_lmenu">
+ <tr>
+ <td>
+<b>The Tab2MAGE format will be phased out for submissions by the end of 2011.<br> The MAGE-TAB format should be used instead.<b>
+ </td>
+ </tr>
+</table>
+
<p class="text">Welcome to the <!-- TMPL_VAR NAME=EXPT_TYPE -->
submissions system. This web page will allow you to submit your
data files with experimental annotation entered in a
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/Common.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/Common.pm 2011-09-29 15:12:26 UTC (rev 2379)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/Common.pm 2011-09-29 16:09:00 UTC (rev 2380)
@@ -1038,8 +1038,11 @@
"ChIP-chip by SNP array",
"ChIP-chip by tiling array",
"ChIP-seq",
+ "RIP-chip by array",
"comparative genomic hybridization by array",
"DNA-seq",
+ "FAIRE-seq",
+ "CLIP-Seq",
"genotyping by array",
"genotyping by high throughput sequencing",
"methylation profiling by array",
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/Database.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/Database.pm 2011-09-29 15:12:26 UTC (rev 2379)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/Database.pm 2011-09-29 16:09:00 UTC (rev 2380)
@@ -249,14 +249,24 @@
sub get_ae2_dbh {
# If we're not yet connected, and we know how to, do so.
- if ( !$AE2_DBH && $CONFIG->get_AE2_DSN() ) {
+ if ( !$AE2_DBH ) {
+ if ( $CONFIG->get_AE2_DSN() ) {
print STDOUT ("Connecting to ArrayExpress 2 database...\n");
$AE2_DBH = _connect_to_oracle(
$CONFIG->get_AE2_DSN(), $CONFIG->get_AE2_USERNAME(),
$CONFIG->get_AE2_PASSWORD(), $CONFIG->get_AE2_DBPARAMS()
);
+ }
}
+ elsif ( ! $AE2_DBH->ping() ) {
+ print STDOUT ("Reconnecting to ArrayExpress database...\n");
+ $AE2_DBH = _connect_to_oracle(
+ $CONFIG->get_AE2_DSN(), $CONFIG->get_AE_USERNAME(),
+ $CONFIG->get_AE2_PASSWORD(), $CONFIG->get_AE_DBPARAMS()
+ );
+ }
+
return $AE2_DBH;
}
@@ -424,30 +434,32 @@
sub retrieve_accno_table {
if ($USE_AE2){
- print STDOUT (
- "\nConnecting to ArrayExpress 2 database...retrieving accession number table.\n"
- );
-
- my $ae2_dbh = get_ae2_dbh()
- or die "Error: Could not connect to ArrayExpress 2 - $!";
+ # Fall back to web access. Note that private arrays are
+ # unavailable here.
+ print STDOUT (
+ "\nConnecting to ArrayExpress ftp site...retrieving accession number table.\n"
+ );
- my $sth = $ae2_dbh->prepare(<<'QUERY');
-select id, acc, name
-from plat_design
-where status=1
-and acc not like '%delete'
-QUERY
+ my $ua = LWP::UserAgent->new();
+ $ua->max_size( $CONFIG->get_MAX_LWP_DOWNLOAD() );
- $sth->execute()
- or croak("Error retrieving array list from database: " . $sth->errstr);
+ my $response = $ua->get( $CONFIG->get_AE_ARRAYDESIGN_LIST() );
- while ( my $result = $sth->fetchrow_arrayref() ) {
- push @AE_ARRAY_ACCNO_TABLE, join("\t", map { $_ || q{} } @{ $result } );
- }
- $ae2_dbh->disconnect or die "Could not disconnect from AE after getting accno table. $!";
-
- # We do not attempt to fall back to arraylist URL as I don't know if this
- # will exist for AE2
+ unless ( $response->is_success ) {
+ croak( "Error connecting to ArrayExpress: "
+ . $response->status_line
+ . "\n" );
+ }
+
+ # If the database goes down, ArrayExpress can return an error
+ # page. We trap this as HTML.
+ if ( $response->content =~ m/^\s*<html>/i ) {
+ croak(
+ "Error connecting to ArrayExpress (web query returned HTML, not text).\n"
+ );
+ }
+
+ @AE_ARRAY_ACCNO_TABLE = split /\n/, $response->content; # split into rows
return;
}
@@ -519,7 +531,7 @@
sub retrieve_public_protocol_list{
print STDOUT (
- "\nConnecting to ArrayExpress web site...retrieving public protocol table.\n"
+ "\nConnecting to ArrayExpress ftp site...retrieving public protocol table.\n"
);
my $ua = LWP::UserAgent->new();
@@ -679,6 +691,10 @@
my ( $db_id, $accession, $error_fh, $skip_adf_download ) = @_;
+ my ($a1,$accession_prefix,$a3) = split(/-/, $accession);
+
+ my $ftp_adf_path = "$accession_prefix/$accession/$accession".".adf.txt";
+
unless ( $accession || $db_id ) {
carp(
"ERROR: Neither array accession nor AE database identifier available "
@@ -706,19 +722,22 @@
my $ua = LWP::UserAgent->new();
$ua->max_size( $CONFIG->get_MAX_LWP_DOWNLOAD() );
- my $source = "ArrayExpress 1 URI";
- my $uri = $CONFIG->get_AE_RETRIEVE_ADF() . $db_id;
+ my $source = "ArrayExpress URI";
+ my $uri = $CONFIG->get_AE_RETRIEVE_ADF() . $ftp_adf_path; # $db_id;
# Get file from AE2 FTP site download if $USE_AE2
if ($USE_AE2){
$accession =~/[A-Z]-([A-Z]{4})-\d*/;
my $pipeline = $1;
- $uri = $CONFIG->get_AE_FTP_URI()
- ."array/"
+ $uri = "file:/".$CONFIG->get_AE_RETRIEVE_ADF() #get_AE_FTP_URI()
+ # ."array/"
.$pipeline."/"
.$accession."/"
."$accession.adf.txt";
- $source = "ArrayExpress FTP site";
+ $source = "ArrayExpress FTP site";
+ print STDOUT (
+ "URI for ADF: $uri \n"
+ );
}
my $temporary_adf_filehandle;
if ( $skip_adf_download ) {
@@ -736,7 +755,7 @@
my $response = $ua->get( $uri );
unless ( $response->is_success ) {
- croak( "Error retrieving ADF from $source: "
+ croak( "Error retrieving ADF from $source $uri: "
. $response->status_line
. "\n" );
}
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/MAGETAB/Checker.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/MAGETAB/Checker.pm 2011-09-29 15:12:26 UTC (rev 2379)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/MAGETAB/Checker.pm 2011-09-29 16:09:00 UTC (rev 2380)
@@ -275,6 +275,16 @@
my %in_relaxed_mode : ATTR( :name<in_relaxed_mode>, :default<0> );
my %in_validate_mode : ATTR( :name<in_validate_mode>, :default<0> );
+my $hts_type = 0;
+
+sub trim($)
+{
+ my $string = shift;
+ $string =~ s/^\s+//;
+ $string =~ s/\s+$//;
+ return $string;
+}
+
sub START {
my ( $self, $id, $args ) = @_;
@@ -541,12 +551,17 @@
$self->set_hybridizations(\%hybridization);
$self->set_scans(\%scan);
$self->set_normalizations(\%normalization);
+
+ #Not needed checks for HTS experiments
+ #Natalja's changes
+ if ($hts_type==0){
- # Check Labeled Extract - Label association is present.
- $self->check_mage_labeledextracts( $bags->{'labeledextract'}->() );
+ # Check Labeled Extract - Label association is present.
+ $self->check_mage_labeledextracts( $bags->{'labeledextract'}->() );
- # Check hybs are created using LabeledExtracts
- $self->check_hybs_use_labeledextracts( $bags->{'pba'}->());
+ # Check hybs are created using LabeledExtracts
+ $self->check_hybs_use_labeledextracts( $bags->{'pba'}->());
+ }
# Check annotation of the Sources.
$self->check_mage_bmchars( $bags->{'source'}->() );
@@ -1254,10 +1269,11 @@
my ( $self, $idf_checker ) = @_;
+ #Publications excluded by Natalja
my %required_list = (
'Experimental Factors' => 'get_factors',
'Contacts (Persons)' => 'get_people',
- 'Publications' => 'get_publications',
+ #'Publications' => 'get_publications',
'Protocols' => 'get_protocols',
);
while ( my ( $needed_items, $method ) = each %required_list ) {
@@ -1479,6 +1495,16 @@
$self->check_ae_expt_types(
@{ $idf_checker->get_comments->{'AEExperimentType'} }
);
+
+ my @types = @{ $idf_checker->get_comments->{'AEExperimentType'} };
+
+ foreach my $type_string (@types){
+ foreach my $type (split /\s*[\t,;]\s*/, $type_string){
+ if ($type =~ /sequencing/){
+ $hts_type = 1;
+ }
+ }
+ }
my %typemap = (
'Quality Control Type' => 'get_qualitycontroltypes',
@@ -1570,9 +1596,10 @@
unless ( $contact->{'email'} ) {
$self->logprint(
'error',
- "Warning: Submitter email address is missing.\n",
+ "Error: Submitter email address is missing.\n",
);
- $self->add_error( $CONFIG->get_ERROR_PARSEBAD() );
+ $self->add_error( $CONFIG->get_ERROR_PARSEFAIL() );
+ #Upgraded to fail http://bar.ebi.ac.uk:8080/trac/ticket/2877-Emma
#$self->add_validation_error("Error: Submitter email address is missing\n");
}
}
@@ -1978,7 +2005,12 @@
# This needs to support one and two-color scans, but we've already
# gathered all the relevant BATs while generating MAGE.
- $self->check_mage_scan_protapps( [ values %{ $self->get_scans() } ] );
+
+ #Not needed checks for HTS experiments
+ #Natalja's changes
+ if ($hts_type==0){
+ $self->check_mage_scan_protapps( [ values %{ $self->get_scans() } ] );
+ }
$self->check_mage_norm_protapps( $bags->{'dbad'}->() );
return;
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/MAGETAB/SDRF.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/MAGETAB/SDRF.pm 2011-09-29 15:12:26 UTC (rev 2379)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/MAGETAB/SDRF.pm 2011-09-29 16:09:00 UTC (rev 2380)
@@ -441,7 +441,7 @@
)
}
elsif ( $termsource ) {
-
+ print STDOUT ("Protocol $name \n");
my $args;
if ( $self->get_in_relaxed_mode() ) {
$args = {name => $termsource,
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/MXExport/Protocol.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/MXExport/Protocol.pm 2011-09-29 15:12:26 UTC (rev 2379)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/MXExport/Protocol.pm 2011-09-29 16:09:00 UTC (rev 2380)
@@ -55,11 +55,11 @@
my $AE_DBH;
# If we're not yet connected, and we know how to, do so.
- if ( !$AE_DBH && $CONFIG->get_AE_DSN() ) {
+ if ( !$AE_DBH && $CONFIG->get_AE2_DSN() ) {
# print STDOUT ("Connecting to ArrayExpress database...\n");
$AE_DBH = _connect_to_oracle(
- $CONFIG->get_AE_DSN(), $CONFIG->get_AE_USERNAME(),
- $CONFIG->get_AE_PASSWORD(), $CONFIG->get_AE_DBPARAMS()
+ $CONFIG->get_AE2_DSN(), $CONFIG->get_AE2_USERNAME(),
+ $CONFIG->get_AE2_PASSWORD(), $CONFIG->get_AE2_DBPARAMS()
);
}
@@ -81,7 +81,7 @@
# Get protocol accession from AE if it has a GLOBID
if ( my $ae_id = $results[0] ){
-
+ #ArrayExpress::Curator::Database->import('get_ae2_dbh');
my $ae_dbh = get_ae_dbh();
# Get from URL if no direct connection to AE available
@@ -92,8 +92,8 @@
# Get from AE directly
my $ae_sth = $ae_dbh->prepare(<<'END');
-select identifier
-from tt_identifiable
+select acc
+from protocol
where id=?
END
$ae_sth->execute($ae_id) or croak ($ae_sth->errstr);
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/ExperimentSet.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/ExperimentSet.pm 2011-09-29 15:12:26 UTC (rev 2379)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/ExperimentSet.pm 2011-09-29 16:09:00 UTC (rev 2380)
@@ -296,10 +296,10 @@
$xml->dataElement($att, $self->get_nvt($extract, $att));
}
$xml->startTag("LIBRARY_LAYOUT");
- if ($layout =~ /single/i){
+ if ($layout =~ /SINGLE/i){
$xml->emptyTag("SINGLE");
}
- elsif ($layout =~ /paired/i){
+ elsif ($layout =~ /PAIRED/i){
$xml->startTag(
"PAIRED",
"ORIENTATION"=> $self->get_nvt($extract,"ORIENTATION"),
@@ -353,7 +353,7 @@
my $default = {};
if ($platform =~ /Illumina/i){
- if ($layout =~ /paired/i){
+ if ($layout =~ /PAIRED/i){
$default = $illumina_paired;
}
else{
@@ -361,7 +361,7 @@
}
}
elsif ($platform =~ /454/){
- if ($layout =~ /paired/i){
+ if ($layout =~ /PAIRED/i){
$default = $ls454_paired;
}
else{
@@ -369,11 +369,11 @@
}
}
elsif ($platform =~ /solid/){
- if ($layout =~ /paired/i){
- $default = $solid_single;
+ if ($layout =~ /PAIRED/i){
+ $default = $solid_paired;
}
else{
- $default = $solid_paired;
+ $default = $solid_single;
}
}
@@ -495,10 +495,10 @@
my %plat_elements = (
- ILLUMINA => ["CYCLE_COUNT","SEQUENCE_LENGTH"],
+ ILLUMINA => ["SPOT_COUNT","SEQUENCE_LENGTH"],
LS454 => ["KEY_SEQUENCE", "FLOW_SEQUENCE","FLOW_COUNT"],
HELICOS => ["FLOW_SEQUENCE","FLOW_COUNT"],
- ABI_SOLID => ["COLOR_MATRIX_CODE","SEQUENCE_LENGTH"],
+ ABI_SOLID => ["SPOT_COUNT","SEQUENCE_LENGTH"],
);
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/SRAtoMAGETAB.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/SRAtoMAGETAB.pm 2011-09-29 15:12:26 UTC (rev 2379)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/SRAtoMAGETAB.pm 2011-09-29 16:09:00 UTC (rev 2380)
@@ -27,6 +27,9 @@
use Date::Parse;
use XML::Simple;
use File::Temp;
+use Date::Manip;
+use DateTime::Format::DateManip;
+use Data::Dumper;
use Class::Std;
@@ -83,6 +86,13 @@
);
+sub trim($)
+{
+ my $string = shift;
+ $string =~ s/^\s+//;
+ $string =~ s/\s+$//;
+ return $string;
+}
sub START{
@@ -91,7 +101,7 @@
# Attempt to download study xml
my $acc = $self->get_study_accession;
my $xml_path = "$acc.study.xml";
-
+
$self->download_xml_for_acc($acc, $xml_path);
# Set up magetab container object and investigation
@@ -103,10 +113,12 @@
$investigation->set_comments( [$comment] );
$self->set_investigation($investigation);
$self->set_magetab($mtab);
+
+ my $sdrf_file_name = $self->get_output_path.".sdrf.txt";
# Create SDRF object
my $sdrf = Bio::MAGETAB::SDRF->new({
- uri => "$acc.sdrf.txt",
+ uri => $sdrf_file_name, #"$acc.sdrf.txt",
});
$self->set_sdrf($sdrf);
@@ -155,7 +167,7 @@
my $url = $self->get_url_template;
$url =~ s/ACCESSION/$acc/;
- $self->log("Getting xml for $acc...");
+ $self->log("Getting xml for $acc from $url ...");
my $rc = getstore($url, $xml_path);
if (is_error($rc)){
die "Could not download $url to $xml_path: $rc";
@@ -164,6 +176,17 @@
return $xml_path;
}
+sub download_file_for_acc{
+ my ($self, $url, $file_path) = @_;
+ $self->log("Getting file from $url ...");
+ my $rc = getstore($url, $file_path);
+ if (is_error($rc)){
+ die "Could not download $url to $file_path: $rc";
+ }
+
+ return $file_path;
+}
+
sub parse_study_xml{
my ($self, $study_xml) = @_;
@@ -186,6 +209,10 @@
}
$self->log("Using study type not mapped to EFO - $study_type", "warn");
}
+ #Added by Natalja
+ if (length($study_type)==0){
+ $study_type = "transcription profiling by high throughput sequencing";
+ }
$self->_add_expt_comment("AEExperimentType", $study_type);
# get additional study types from STUDY_ATTRIBUTE
@@ -213,7 +240,7 @@
# Download and parse submission xml
# FIXME: assuming only 1 submission acc
my ($sub_acc) = @{ $links{"ENA-SUBMISSION"} || []};
- my $submission_xml = "$sub_acc.submission.xml";
+ my $submission_xml = "$sub_acc.submission.xml";
$self->download_xml_for_acc($sub_acc, $submission_xml);
$self->parse_submission_xml($submission_xml);
@@ -239,11 +266,26 @@
my $lab = $xml->{SUBMISSION}->{lab_name};
my $date = $xml->{SUBMISSION}->{submission_date};
- $self->_add_expt_comment("SRASubmissonDate", $date);
+ $self->_add_expt_comment("SRASubmissonDate", substr($date,0,10));
+
+ $self->get_investigation->set_publicReleaseDate(substr($date,0,10));
+ my $e_mail = "";
+
+ if (trim($lab)=="Wellcome Trust Sanger Institute"){
+ $e_mail = "datahose\@sanger.ac.uk";
+ }
+
+ my $role = Bio::MAGETAB::ControlledTerm->new({
+ category=>"Roles",
+ value=>"submitter"
+ });
+
if ($lab){
my $contact = Bio::MAGETAB::Contact->new({
lastName => $lab,
+ email => $e_mail,
+ roles => [$role],
});
$self->get_investigation->set_contacts([$contact]);
}
@@ -289,6 +331,8 @@
foreach my $attr ( @{ $sample->{SAMPLE_ATTRIBUTES}->{SAMPLE_ATTRIBUTE} || [] }){
my $category = $attr->{TAG};
next if $category =~ /^ENA-.*/;
+ my $value = $attr->{VALUE};
+ next if $value !=~ /[A-Z][a-z][a-z]*[0-9][0-9]/;
my $char = Bio::MAGETAB::ControlledTerm->new({
category => $category,
value => $attr->{VALUE},
@@ -347,7 +391,39 @@
my $value = $expt->{DESIGN}->{LIBRARY_DESCRIPTOR}->{$type};
$self->_add_comment($type, $value, $extract);
}
-
+ foreach my $type qw(LIBRARY_LAYOUT){
+ my $value = "SINGLE";
+ if ($expt->{DESIGN}->{LIBRARY_DESCRIPTOR}->{LIBRARY_LAYOUT}->{SINGLE}){
+ $value = "SINGLE";
+ $self->_add_comment($type, $value, $extract);
+ }
+ if ($expt->{DESIGN}->{LIBRARY_DESCRIPTOR}->{LIBRARY_LAYOUT}->{PAIRED}){
+ $value = "PAIRED";
+ $self->_add_comment($type, $value, $extract);
+ my $type_paired = "ORIENTATION";
+ if ($expt->{DESIGN}->{LIBRARY_DESCRIPTOR}->{LIBRARY_LAYOUT}->{PAIRED}->{$type_paired}){
+ $value = $expt->{DESIGN}->{LIBRARY_DESCRIPTOR}->{LIBRARY_LAYOUT}->{PAIRED}->{$type_paired};
+ $self->_add_comment($type_paired, $value, $extract);
+ }
+ $type_paired = "NOMINAL_LENGTH";
+ if ($expt->{DESIGN}->{LIBRARY_DESCRIPTOR}->{LIBRARY_LAYOUT}->{PAIRED}->{$type_paired}){
+ $value = $expt->{DESIGN}->{LIBRARY_DESCRIPTOR}->{LIBRARY_LAYOUT}->{PAIRED}->{$type_paired};
+ $self->_add_comment($type_paired, $value, $extract);
+ }
+ $type_paired = "NOMINAL_SDEV";
+ if ($expt->{DESIGN}->{LIBRARY_DESCRIPTOR}->{LIBRARY_LAYOUT}->{PAIRED}->{$type_paired}){
+ $value = $expt->{DESIGN}->{LIBRARY_DESCRIPTOR}->{LIBRARY_LAYOUT}->{PAIRED}->{$type_paired};
+ $self->_add_comment($type_paired, $value, $extract);
+ }
+ }
+
+ }
+
+ if ($expt->{DESIGN}->{SPOT_DESCRIPTOR}->{SPOT_DECODE_SPEC}->{SPOT_LENGTH}){
+ my $spot_length = $expt->{DESIGN}->{SPOT_DESCRIPTOR}->{SPOT_DECODE_SPEC}->{SPOT_LENGTH};
+ $self->_add_comment("SPOT_LENGTH", $spot_length, $assay);
+ }
+
# Create extraction edge and extraction protocol
my $extraction_edge = Bio::MAGETAB::Edge->new({
inputNode => $sample,
@@ -387,6 +463,23 @@
my $plat_xml = $expt->{PLATFORM};
my ($platform) = keys %$plat_xml;
my $instrument = $plat_xml->{$platform}->{INSTRUMENT_MODEL};
+ if ($plat_xml->{$platform}->{SEQUENCE_LENGTH}){
+ my $value = $plat_xml->{$platform}->{SEQUENCE_LENGTH};
+ $self->_add_comment("SEQUENCE_LENGTH", $value, $assay);
+ }
+ if ($plat_xml->{$platform}->{FLOW_SEQUENCE}){
+ my $value = $plat_xml->{$platform}->{FLOW_SEQUENCE};
+ $self->_add_comment("FLOW_SEQUENCE", $value, $assay);
+ }
+ if ($plat_xml->{$platform}->{FLOW_COUNT}){
+ my $value = $plat_xml->{$platform}->{FLOW_COUNT};
+ $self->_add_comment("FLOW_COUNT", $value, $assay);
+ }
+ if ($plat_xml->{$platform}->{KEY_SEQUENCE}){
+ my $value = $plat_xml->{$platform}->{KEY_SEQUENCE};
+ $self->_add_comment("KEY_SEQUENCE", $value, $assay);
+ }
+
my $seq_protocol;
unless ($seq_protocol = $protocol_with_hardware{$instrument}){
$seq_protocol = Bio::MAGETAB::Protocol->new({
@@ -415,26 +508,58 @@
push @{ $self->get_all_nodes }, $assay;
my $links = $self->_get_links($expt, "EXPERIMENT");
+
+ foreach my $fastq_list (@{ $links->{"ERA-FASTQ-FILES"} || [] }){
+ my $file_fastq = $self->get_output_path.".tmp.txt";
+ $self->download_file_for_acc($fastq_list, $file_fastq);
+ open(DAT, $file_fastq) || die("Could not open file $file_fastq!");
+ my @raw_data=<DAT>;
+ foreach my $line (@raw_data)
+ {
+ my @entries = (split "\t", $line);
+ if ($entries[3] =~ /ERR\d{6}/){
+
+ # Add scan
+ my $fastq_path = trim($entries[16]);
+ my $run_acc = trim($entries[3]);
+ $all_study_runs{$run_acc} = 1;
+
+ my $data = Bio::MAGETAB::DataAcquisition->new({
+ name => $run_acc,
+ });
+ $self->_add_comment("FASTQ_URI", "ftp://ftp.sra.ebi.ac.uk/vol1/$fastq_path", $data);
+ $self->_add_comment("ENA_RUN", $run_acc, $data);
+ my $data_edge = Bio::MAGETAB::Edge->new({
+ inputNode => $assay,
+ outputNode => $data,
+ });
+ push @{ $self->get_all_nodes }, $data;
+
+ }
+
+ }
+ close(DAT);
+ }
- foreach my $fastq_list (@{ $links->{"ERA-FASTQ"} || [] }){
- foreach my $fastq_path (split ",", $fastq_list){
+ #foreach my $fastq_list (@{ $links->{"ERA-FASTQ"} || [] }){
+ # foreach my $fastq_path (split ",", $fastq_list){
# Add scan
- $fastq_path =~ /.*\/(ERR\d{6})\/.*/g;
- my $run_acc = $1;
- $all_study_runs{$run_acc} = 1;
+ # $fastq_path =~ /.*\/(ERR\d{6})\/.*/g;
+ # my $run_acc = $1;
+ # $all_study_runs{$run_acc} = 1;
- my $data = Bio::MAGETAB::DataAcquisition->new({
- name => $run_acc,
- });
- $self->_add_comment("FASTQ_URI", "ftp://ftp.sra.ebi.ac.uk/vol1/$fastq_path", $data);
- $self->_add_comment("ENA_RUN", $run_acc, $data);
- my $data_edge = Bio::MAGETAB::Edge->new({
- inputNode => $assay,
- outputNode => $data,
- });
- push @{ $self->get_all_nodes }, $data;
- }
- }
+ # my $data = Bio::MAGETAB::DataAcquisition->new({
+ # name => $run_acc,
+ # });
+ # $self->_add_comment("FASTQ_URI", "ftp://ftp.sra.ebi.ac.uk/vol1/$fastq_path", $data);
+ # $self->_add_comment("ENA_RUN", $run_acc, $data);
+ # my $data_edge = Bio::MAGETAB::Edge->new({
+ # inputNode => $assay,
+ # outputNode => $data,
+ # });
+ # push @{ $self->get_all_nodes }, $data;
+ # }
+ #}
}
@@ -496,11 +621,19 @@
$self->log("Writing magetab file...");
- open (my $mtab_fh, ">", $self->get_output_path)
- or die "Error: could not open file ".$self->get_output_path." for writing - $!";
+ #open (my $mtab_fh, ">", $self->get_output_path)
+ # or die "Error: could not open file ".$self->get_output_path." for writing - $!";
+ #Natalja's modifications
+ my $idf_name = $self->get_output_path.".idf.txt";
+ my $sdrf_name = $self->get_output_path.".sdrf.txt";
+ open (my $idf_fh, ">", $idf_name)
+ or die "Error: could not open file ".$idf_name." for writing - $!";
+ open (my $sdrf_fh, ">", $sdrf_name)
+ or die "Error: could not open file ".$sdrf_name." for writing - $!";
+
- my $idf_fh = File::Temp->new;
- my $sdrf_fh = File::Temp->new;
+ #my $idf_fh = File::Temp->new;
+ #my $sdrf_fh = File::Temp->new;
$self->get_investigation->set_protocols($self->get_protocols);
@@ -512,34 +645,34 @@
my $idf_writer = Bio::MAGETAB::Util::Writer::IDF->new({
magetab_object => $self->get_investigation,
filehandle => $idf_fh,
- });
+ });
- my $sdrf_writer = Bio::MAGETAB::Util::Writer::SDRF->new({
+ my $sdrf_writer = Bio::MAGETAB::Util::Writer::SDRF->new({
magetab_object => $self->get_sdrf,
filehandle => $sdrf_fh,
});
# Make factor values from characteristics the same way we do for GEO
- $self->_make_fvs_from_chars();
+ $self->_make_fvs_from_chars();
- $self->get_investigation->set_factors( $self->get_factors );
+ $self->get_investigation->set_factors( $self->get_factors );
- $idf_writer->write();
+ $idf_writer->write();
$sdrf_writer->write();
# Combine IDF and SDRF
- seek($idf_fh, 0,0) or die "Could not rewind temp IDF fh - $!";
- seek($sdrf_fh, 0,0) or die "Could not rewind temp SDRF fh - $!";
+ #seek($idf_fh, 0,0) or die "Could not rewind temp IDF fh - $!";
+ #seek($sdrf_fh, 0,0) or die "Could not rewind temp SDRF fh - $!";
- print $mtab_fh "[IDF]\n";
- while (<$idf_fh>){
- print $mtab_fh $_ unless /^\"?SDRF File/i;
- }
- print $mtab_fh "\n\n[SDRF]\n";
- while (<$sdrf_fh>){
- print $mtab_fh $_;
- }
- close $mtab_fh;
+ #print $mtab_fh "[IDF]\n";
+ #while (<$idf_fh>){
+ # print $mtab_fh $_ unless /^\"?SDRF File/i;
+ #}
+ #print $mtab_fh "\n\n[SDRF]\n";
+ #while (<$sdrf_fh>){
+ # print $mtab_fh $_;
+ #}
+ #close $mtab_fh;
}
sub _add_expt_comment{
@@ -607,7 +740,7 @@
# If a category has more than 1 value make it a factor
foreach my $type (keys %types_to_values){
- if ( (keys %{ $types_to_values{$type} }) > 1){
+ #if ( (keys %{ $types_to_values{$type} }) > 1){
$self->log("Adding $type factor values");
@@ -643,7 +776,7 @@
$row->set_factorValues( [ @{ $existing || [] }, $fv ] );
}
}
- }
+ #}
}
return;
}
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <fa...@us...> - 2011-09-29 15:12:35
|
Revision: 2379
http://tab2mage.svn.sourceforge.net/tab2mage/?rev=2379&view=rev
Author: farne
Date: 2011-09-29 15:12:26 +0000 (Thu, 29 Sep 2011)
Log Message:
-----------
some updates following redesign of ArrayExpress database
Modified Paths:
--------------
branches/ae-dev/Tab2MAGE/MANIFEST
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/DB/Protocol.pm
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/Daemon/Exporter.pm
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/Config.pm
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/Database.pm
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/ExperimentChecker.pm
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/MAGETAB/Checker.pm
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/MXExport/SDRF.pm
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Tracking/QueryHandler.pm
branches/ae-dev/Tab2MAGE/t/magetab_checker.t
Modified: branches/ae-dev/Tab2MAGE/MANIFEST
===================================================================
--- branches/ae-dev/Tab2MAGE/MANIFEST 2011-09-22 10:54:12 UTC (rev 2378)
+++ branches/ae-dev/Tab2MAGE/MANIFEST 2011-09-29 15:12:26 UTC (rev 2379)
@@ -24,6 +24,8 @@
automation/admin/app/controllers/pipelines_controller.rb
automation/admin/app/controllers/protocols_controller.rb
automation/admin/app/controllers/quantitation_type_controller.rb
+automation/admin/app/controllers/samplegroups_controller.rb
+automation/admin/app/controllers/samples_controller.rb
automation/admin/app/controllers/tab2mages_controller.rb
automation/admin/app/controllers/taxon_controller.rb
automation/admin/app/controllers/users_controller.rb
@@ -65,6 +67,8 @@
automation/admin/app/models/quality_metric.rb
automation/admin/app/models/quantitation_type.rb
automation/admin/app/models/role.rb
+automation/admin/app/models/sample.rb
+automation/admin/app/models/sample_group.rb
automation/admin/app/models/spreadsheet.rb
automation/admin/app/models/taxon.rb
automation/admin/app/models/user.rb
@@ -105,6 +109,7 @@
automation/admin/app/views/experiments/list.rhtml
automation/admin/app/views/experiments/migrations.rhtml
automation/admin/app/views/experiments/new.rhtml
+automation/admin/app/views/experiments/sample_subs.rhtml
automation/admin/app/views/experiments/show.rhtml
automation/admin/app/views/experiments/show_qm.rhtml
automation/admin/app/views/experiments/today_list.rhtml
@@ -136,6 +141,8 @@
automation/admin/app/views/protocols/list.rhtml
automation/admin/app/views/protocols/new.rhtml
automation/admin/app/views/protocols/show.rhtml
+automation/admin/app/views/samplegroups/list.rhtml
+automation/admin/app/views/samples/list.rhtml
automation/admin/app/views/tab2mages/_aedwinfo.rhtml
automation/admin/app/views/tab2mages/_annotate.rhtml
automation/admin/app/views/tab2mages/_form.rhtml
@@ -177,6 +184,8 @@
automation/admin/db/migrate/005_add_daemon_tables_dec2009.rb
automation/admin/db/migrate/006_add_sample_table.rb
automation/admin/db/migrate/007_add_experiment_qm_table.rb
+automation/admin/db/migrate/008_add_migration_and_ae2_tracking_cols.rb
+automation/admin/db/migrate/009_bio_sample_tracking_changes.rb
automation/admin/db/migration/migrate_2.0_to_2.1.sql
automation/admin/db/migration/migrate_2.1_to_2.2.sql
automation/admin/db/migration/migrate_csv2sqlite.pl
@@ -614,6 +623,7 @@
lib/ArrayExpress/AutoSubmission/DB/Role.pm
lib/ArrayExpress/AutoSubmission/DB/RoleUser.pm
lib/ArrayExpress/AutoSubmission/DB/Sample.pm
+lib/ArrayExpress/AutoSubmission/DB/SampleGroup.pm
lib/ArrayExpress/AutoSubmission/DB/Spreadsheet.pm
lib/ArrayExpress/AutoSubmission/DB/Taxon.pm
lib/ArrayExpress/AutoSubmission/DB/User.pm
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/DB/Protocol.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/DB/Protocol.pm 2011-09-22 10:54:12 UTC (rev 2378)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/DB/Protocol.pm 2011-09-29 15:12:26 UTC (rev 2379)
@@ -11,7 +11,7 @@
use ArrayExpress::Curator::Common qw(date_now);
-my $USE_AE2 = 0;
+my $USE_AE2 = 1;
__PACKAGE__->table('protocols');
__PACKAGE__->columns(
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/Daemon/Exporter.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/Daemon/Exporter.pm 2011-09-22 10:54:12 UTC (rev 2378)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/Daemon/Exporter.pm 2011-09-29 15:12:26 UTC (rev 2379)
@@ -33,7 +33,7 @@
# Temporary flag to turn on/off AE2 and MAGE-ML export
# Eventually mageml export will be retired
-my %export_to_mageml : ATTR( :name<export_to_mageml>, :default<1> );
+my %export_to_mageml : ATTR( :name<export_to_mageml>, :default<0> );
my %export_to_ae2 : ATTR( :name<export_to_ae2>, :default<1> );
sub export_tab2mage : PRIVATE {
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/Config.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/Config.pm 2011-09-22 10:54:12 UTC (rev 2378)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/Config.pm 2011-09-29 15:12:26 UTC (rev 2379)
@@ -221,7 +221,7 @@
MX_ADF_REGEX =>
qr{\b(Meta\s*Column|Reporter\s*Identifier|Composite\s*Sequence\s*Identifier)\b}ixms,
MAGETAB_ADF_REGEX =>
- qr{^(Block\s*Column|Block\s*Row|Column|Row|Reporter\s*Name|Composite\s*Element\s*Name)\b}ixms,
+ qr{^(Block\s*Column|Block\s*Row|Reporter\s*Name|Composite\s*Element\s*Name)\b}ixms,
# List of characters which cannot be used in data file names
FILENAME_FORBIDDEN_CHARS => qr{ [ \: \\ \" ]+ }ixms,
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/Database.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/Database.pm 2011-09-22 10:54:12 UTC (rev 2378)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/Database.pm 2011-09-29 15:12:26 UTC (rev 2379)
@@ -167,7 +167,7 @@
# NB: While we are running both AE1 and AE2 export this AE2 query must be
# turned off because we need to query AE1 to get the feature identifiers
# for feature coordinates in order to create the FeatureDimension MAGE-ML
-my $USE_AE2 = 0;
+my $USE_AE2 = 1;
# Globals to hold AE and AEDW database handles, if needed.
my ($AE_DBH, $AEDW_DBH, $AE2_DBH);
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/ExperimentChecker.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/ExperimentChecker.pm 2011-09-22 10:54:12 UTC (rev 2378)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Curator/ExperimentChecker.pm 2011-09-29 15:12:26 UTC (rev 2379)
@@ -889,6 +889,12 @@
my ( $self, $file ) = @_;
+ # Don't do this test when running in MAGETAB validator mode
+ return if (
+ $self->isa("ArrayExpress::MAGETAB::Checker")
+ and $self->get_in_validate_mode
+ );
+
# We can't sample the data, so we just take the MD5 hash.
$file->set_test_data_line( $file->get_md5_digest() );
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/MAGETAB/Checker.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/MAGETAB/Checker.pm 2011-09-22 10:54:12 UTC (rev 2378)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/MAGETAB/Checker.pm 2011-09-29 15:12:26 UTC (rev 2379)
@@ -273,6 +273,7 @@
my %validation_errors : ATTR( :get<validation_errors>, :default<[]>);
my %in_relaxed_mode : ATTR( :name<in_relaxed_mode>, :default<0> );
+my %in_validate_mode : ATTR( :name<in_validate_mode>, :default<0> );
sub START {
my ( $self, $id, $args ) = @_;
@@ -1572,7 +1573,7 @@
"Warning: Submitter email address is missing.\n",
);
$self->add_error( $CONFIG->get_ERROR_PARSEBAD() );
- $self->add_validation_error("Error: Submitter email address is missing\n");
+ #$self->add_validation_error("Error: Submitter email address is missing\n");
}
}
if ( my $ts = $contact->{'termsource'} ) {
@@ -1591,7 +1592,7 @@
"Warning: No submitters provided in IDF.\n",
);
$self->add_error( $CONFIG->get_ERROR_PARSEBAD() );
- $self->add_validation_error("Error: No submitters provided in IDF\n");
+ #$self->add_validation_error("Error: No submitters provided in IDF\n");
}
return;
@@ -2257,6 +2258,9 @@
my ($self) = @_;
+ # Set a flag to show we are validating (some checks will be skipped)
+ $self->set_in_validate_mode(1);
+
# Read in the spreadsheet; populate array design info.
my ( $filelist, $hyb_ids, $norm_ids, $scan_ids )
= $self->get_files_and_annotation();
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/MXExport/SDRF.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/MXExport/SDRF.pm 2011-09-22 10:54:12 UTC (rev 2378)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/MXExport/SDRF.pm 2011-09-29 15:12:26 UTC (rev 2379)
@@ -468,24 +468,25 @@
# for each line retrieved for SDRF create/modify values
# then store in sdrf_line_values array
+ my ($fgem_converted, $fgem_path);
while ( my $line_ref = $sth->fetchrow_hashref ){
my $submission_id = $line_ref->{TSUBMIS_SYSUID};
my $user = $line_ref->{TSUBMIS_LOGIN};
$line_count{$submission_id}++;
- # check for FGEM and convert to magetab format if possible
- # add to first line for each experiment along with transformation prot
- if (! $fgem_check_done_for{$submission_id} ){
- my ($fgem, $trans_prot) = get_fgem( $submission_id, $dbh );
- if ($fgem){
- $line_ref->{TEXPRFNL_PROTOCOLID} = $trans_prot;
- $fgem = convert_fgem($user, $submission_id, $fgem, $args{mx_path});
- my $fgem_path = File::Spec->catfile($user,"submission$submission_id",$fgem);
- $line_ref->{fgem} = $fgem;
- $args{data_file}->{$fgem_path} = 1;
- }
- $fgem_check_done_for{$submission_id} = 1;
+ # Fixed by Natalja, to have FGEM for each hybridization not only for the first one
+ my ($fgem, $trans_prot) = get_fgem( $submission_id, $dbh );
+ if ($fgem){
+
+ $line_ref->{TEXPRFNL_PROTOCOLID} = $trans_prot;
+ if (! $fgem_check_done_for{$submission_id} ){
+ $fgem_converted = convert_fgem($user, $submission_id, $fgem, $args{mx_path});
+ $fgem_path = File::Spec->catfile($user,"submission$submission_id",$fgem_converted);
+ $fgem_check_done_for{$submission_id} = 1;
+ }
+ $line_ref->{fgem} = $fgem_converted;
+ $args{data_file}->{$fgem_path} = 1;
}
# Get any EXP file associated with the hyb
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Tracking/QueryHandler.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Tracking/QueryHandler.pm 2011-09-22 10:54:12 UTC (rev 2378)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/Tracking/QueryHandler.pm 2011-09-29 15:12:26 UTC (rev 2379)
@@ -28,7 +28,7 @@
my %cached_sth : ATTR( :name<cached_sth>, :default<{}> );
my %atlas_events_cached : ATTR( :name<atlas_events_cached>, :default<0> );
-Readonly my $USE_AE2 => 0;
+Readonly my $USE_AE2 => 1;
Readonly my $AEREP_DB => 'AEPUB1';
Readonly my $AEDW_DB => 'ATLASDR';
Modified: branches/ae-dev/Tab2MAGE/t/magetab_checker.t
===================================================================
--- branches/ae-dev/Tab2MAGE/t/magetab_checker.t 2011-09-22 10:54:12 UTC (rev 2378)
+++ branches/ae-dev/Tab2MAGE/t/magetab_checker.t 2011-09-29 15:12:26 UTC (rev 2379)
@@ -1,4 +1,4 @@
-#!/usr/bin/env perl -wT
+#!/usr/bin/env perl -w
#
# $Id$
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <fa...@us...> - 2011-09-22 10:54:18
|
Revision: 2378
http://tab2mage.svn.sourceforge.net/tab2mage/?rev=2378&view=rev
Author: farne
Date: 2011-09-22 10:54:12 +0000 (Thu, 22 Sep 2011)
Log Message:
-----------
hourly db connection check stops application freezing overnight on some systems
Modified Paths:
--------------
branches/ae-dev/Tab2MAGE/automation/admin/config/environment.rb
Modified: branches/ae-dev/Tab2MAGE/automation/admin/config/environment.rb
===================================================================
--- branches/ae-dev/Tab2MAGE/automation/admin/config/environment.rb 2011-02-03 17:32:02 UTC (rev 2377)
+++ branches/ae-dev/Tab2MAGE/automation/admin/config/environment.rb 2011-09-22 10:54:12 UTC (rev 2378)
@@ -64,3 +64,7 @@
# Pagination now uses an external gem. Apparently
# this is much better than the classic pagination.
require 'will_paginate'
+
+# Check connection once an hour to stop application freezing overnight
+Thread.new{ loop{ sleep(60*60);
+ActiveRecord::Base.verify_active_connections! } }.priority = -10
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <fa...@us...> - 2010-11-30 12:50:31
|
Revision: 2372
http://tab2mage.svn.sourceforge.net/tab2mage/?rev=2372&view=rev
Author: farne
Date: 2010-11-30 12:50:25 +0000 (Tue, 30 Nov 2010)
Log Message:
-----------
bug fixes
Modified Paths:
--------------
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/SRAtoMAGETAB.pm
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/SRAtoMAGETAB.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/SRAtoMAGETAB.pm 2010-11-29 17:28:50 UTC (rev 2371)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/SRAtoMAGETAB.pm 2010-11-30 12:50:25 UTC (rev 2372)
@@ -73,6 +73,17 @@
# Tag for these in SRA xml has not yet been decided
my $STUDY_TYPE_TAG = "tbc";
+my @force_array = qw(
+ SAMPLE_ATTRIBUTE
+ SAMPLE
+ EXPERIMENT
+ STUDY_ATTRIBUTE
+ STUDY_LINK
+ EXPERIMENT_LINK
+ );
+
+
+
sub START{
my ($self, $id, $args) = @_;
@@ -157,7 +168,7 @@
my ($self, $study_xml) = @_;
- my $xml = XMLin($study_xml);
+ my $xml = XMLin($study_xml, ForceArray => \@force_array);
# Set title
my $title = $xml->{STUDY}->{DESCRIPTOR}->{STUDY_TITLE};
@@ -223,7 +234,7 @@
my ($self, $submission_xml) = @_;
- my $xml = XMLin($submission_xml);
+ my $xml = XMLin($submission_xml, ForceArray => \@force_array);
my $lab = $xml->{SUBMISSION}->{lab_name};
my $date = $xml->{SUBMISSION}->{submission_date};
@@ -244,7 +255,7 @@
my ($self, $sample_xml) = @_;
- my $xml = XMLin($sample_xml);
+ my $xml = XMLin($sample_xml, ForceArray => \@force_array);
foreach my $sample(@{ $xml->{SAMPLE} || [] }){
my $acc = $sample->{accession};
@@ -255,7 +266,7 @@
$self->_add_comment("ENA_SAMPLE", $acc, $source);
my $descr = $sample->{DESCRIPTION};
- $source->set_description($descr) if $descr;
+ $source->set_description($descr) if ($descr and ref($descr) eq "SCALAR");
$self->_add_comment("TITLE",$sample->{TITLE},$source)
if $sample->{TITLE};
@@ -267,8 +278,13 @@
my @chars;
my $taxon_id = $sample->{SAMPLE_NAME}->{TAXON_ID};
my $species = $sample->{SAMPLE_NAME}->{SCIENTIFIC_NAME};
- my $species_attr = $self->create_species_attribute($species, $taxon_id);
- push @chars, $species_attr;
+ if ($species or $taxon_id){
+ my $species_attr = $self->create_species_attribute($species, $taxon_id);
+ push @chars, $species_attr;
+ }
+ else{
+ $self->log("no species found for sample $acc","warn");
+ }
foreach my $attr ( @{ $sample->{SAMPLE_ATTRIBUTES}->{SAMPLE_ATTRIBUTE} || [] }){
my $category = $attr->{TAG};
@@ -289,7 +305,7 @@
my ($self, $expt_xml) = @_;
- my $xml = XMLin($expt_xml);
+ my $xml = XMLin($expt_xml, ForceArray => \@force_array);
my %protocol_with_text;
my %protocol_with_hardware;
@@ -312,7 +328,12 @@
my $sample_acc = $expt->{DESIGN}->{SAMPLE_DESCRIPTOR}->{accession};
my ($sample) = grep {$_->get_name eq $sample_acc} $self->get_magetab->get_sources;
- $self->log("Sample $sample_acc found for experiment $acc") if $sample;
+ if ($sample){
+ $self->log("Sample $sample_acc found for experiment $acc")
+ }
+ else{
+ die "Could not find sample $sample_acc for experiment $acc";
+ }
# Make extract name unique by adding experiment accession (just in case..)
my $extract_name = $expt->{DESIGN}->{LIBRARY_DESCRIPTOR}->{LIBRARY_NAME}
@@ -533,6 +554,8 @@
sub _add_comment{
my ($self, $category, $value, $thing) = @_;
+ $value ||= "";
+
if ($category =~ /Date$/g){
# Attempt to convert date to YYYY-MM-DD format
my $time = str2time($value);
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <fa...@us...> - 2010-11-29 17:28:57
|
Revision: 2371
http://tab2mage.svn.sourceforge.net/tab2mage/?rev=2371&view=rev
Author: farne
Date: 2010-11-29 17:28:50 +0000 (Mon, 29 Nov 2010)
Log Message:
-----------
header only ADF now passes validation
Modified Paths:
--------------
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/ADFParser/MAGETAB_ADF.pm
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/ADFParser/MAGETAB_HeaderSection.pm
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/ADFParser.pm
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/ADFParser/MAGETAB_ADF.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/ADFParser/MAGETAB_ADF.pm 2010-11-29 08:39:15 UTC (rev 2370)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/ADFParser/MAGETAB_ADF.pm 2010-11-29 17:28:50 UTC (rev 2371)
@@ -393,9 +393,12 @@
}
}
else{
- $self->add_validation_error("ERROR: Could not prepare file "
- .$self->get_adf_path
- ." for checking\n");
+ # Removed this error to allow for files which
+ # contain ADF header only
+
+ #$self->add_validation_error("ERROR: Could not prepare file "
+ # .$self->get_adf_path
+ # ." for checking\n");
}
# Print validation errors to stdout if we have any
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/ADFParser/MAGETAB_HeaderSection.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/ADFParser/MAGETAB_HeaderSection.pm 2010-11-29 08:39:15 UTC (rev 2370)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/ADFParser/MAGETAB_HeaderSection.pm 2010-11-29 17:28:50 UTC (rev 2371)
@@ -207,14 +207,17 @@
my $line = $csv_parser->getline($fh);
- until ( (join "\t", @$line) =~ $self->get_heading_regex )
+ LINE: until ( (join "\t", @$line ) =~ $self->get_heading_regex )
{
# FIX: add this back in later
#$line =~ s/[\n\r]//g;
#ArrayExpress::ADFParser::escape_html(\$line);
push @header_lines, $line;
- $line = $csv_parser->getline($fh) or die ("Error: Could not identify ADF heading line\n");
+ unless ($line = $csv_parser->getline($fh)){
+ warn ("Warning: Could not identify ADF heading line\n");
+ last LINE;
+ }
}
#store header in array of arrays
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/ADFParser.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/ADFParser.pm 2010-11-29 08:39:15 UTC (rev 2370)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/ADFParser.pm 2010-11-29 17:28:50 UTC (rev 2371)
@@ -603,9 +603,9 @@
$start_line++;
if ( ! ($line = $csv_parser->getline($adf)) ) {
- print "ERROR: Could not identify header row\n";
+ print "Warning: Could not identify header row\n";
if ($fh){
- print $fh "ERROR: Could not identify header row\n";
+ print $fh "Warning: Could not identify header row\n";
}
$self->set_error_status(1);
return 0;
@@ -1028,8 +1028,11 @@
});
if (!$adf){
- print "ERROR: Could not prepare file for checking\n";
- $self->set_error_status(1);
+ # Allow MAGE-TAB ADFs to be header only
+ unless ($self->isa("ArrayExpress::ADFParser::MAGETAB_ADF")){
+ print "ERROR: Could not prepare file for checking\n";
+ $self->set_error_status(1);
+ }
return 0;
}
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <fa...@us...> - 2010-11-29 08:39:21
|
Revision: 2370
http://tab2mage.svn.sourceforge.net/tab2mage/?rev=2370&view=rev
Author: farne
Date: 2010-11-29 08:39:15 +0000 (Mon, 29 Nov 2010)
Log Message:
-----------
fixed regex to allow spaces in FV column names in t2m to magetab conversion
Modified Paths:
--------------
branches/ae-dev/magetab_converter/lib/ArrayExpress/TabConverter/IDF.pm
Modified: branches/ae-dev/magetab_converter/lib/ArrayExpress/TabConverter/IDF.pm
===================================================================
--- branches/ae-dev/magetab_converter/lib/ArrayExpress/TabConverter/IDF.pm 2010-11-26 10:50:40 UTC (rev 2369)
+++ branches/ae-dev/magetab_converter/lib/ArrayExpress/TabConverter/IDF.pm 2010-11-29 08:39:15 UTC (rev 2370)
@@ -77,7 +77,7 @@
my ( @factor_type, @factor_unit, @factor_name, @factor_type_ref );
foreach my $heading ( @{ $row } ) {
- if ( my ( $category ) = ( $heading =~ /FactorValue \[ \s* (.+?) \s* \] /xms ) ) {
+ if ( my ( $category ) = ( $heading =~ /Factor \s* Value \s* \[ \s* (.+?) \s* \] /xms ) ) {
if ( my ( $type, $unit ) = ( $category =~ /(.+?) \s* \( \s* (.+?) \s* \)/xms ) ) {
push( @factor_name, uc($type) );
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <fa...@us...> - 2010-11-26 10:50:47
|
Revision: 2369
http://tab2mage.svn.sourceforge.net/tab2mage/?rev=2369&view=rev
Author: farne
Date: 2010-11-26 10:50:40 +0000 (Fri, 26 Nov 2010)
Log Message:
-----------
use ESD as the experiment type for biosample submissions (this is what the submission page cgi app expects)
Modified Paths:
--------------
branches/ae-dev/Tab2MAGE/automation/admin/app/controllers/experiments_controller.rb
branches/ae-dev/Tab2MAGE/automation/admin/app/views/experiments/sample_subs.rhtml
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/WebForm.pm
Modified: branches/ae-dev/Tab2MAGE/automation/admin/app/controllers/experiments_controller.rb
===================================================================
--- branches/ae-dev/Tab2MAGE/automation/admin/app/controllers/experiments_controller.rb 2010-11-25 19:22:38 UTC (rev 2368)
+++ branches/ae-dev/Tab2MAGE/automation/admin/app/controllers/experiments_controller.rb 2010-11-26 10:50:40 UTC (rev 2369)
@@ -37,7 +37,7 @@
def sample_subs
num_per_page = params[:list_all].to_i.zero? ? 30 : 1000000
- sql_where_clause = "is_deleted=0 and experiment_type='biosample'"
+ sql_where_clause = "is_deleted=0 and experiment_type='ESD'"
@search_term = ""
Modified: branches/ae-dev/Tab2MAGE/automation/admin/app/views/experiments/sample_subs.rhtml
===================================================================
--- branches/ae-dev/Tab2MAGE/automation/admin/app/views/experiments/sample_subs.rhtml 2010-11-25 19:22:38 UTC (rev 2368)
+++ branches/ae-dev/Tab2MAGE/automation/admin/app/views/experiments/sample_subs.rhtml 2010-11-26 10:50:40 UTC (rev 2369)
@@ -32,7 +32,7 @@
<%= link_to "Edit", :action => "edit",
:id => e.id,
:page => params[:page],
- :experiment_type => "biosample",
+ :experiment_type => "ESD",
:search_term => @search_term %>
</td>
<%= content_tag("td", e.accession) %>
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/WebForm.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/WebForm.pm 2010-11-25 19:22:38 UTC (rev 2368)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/WebForm.pm 2010-11-26 10:50:40 UTC (rev 2369)
@@ -518,7 +518,8 @@
my $experiment
= ArrayExpress::AutoSubmission::DB::Experiment->retrieve($id)
or croak("Internal error: No experiment found in database");
- my $allfiles = scalar($experiment->data_files(is_deleted => 0))
+ my $allfiles = (scalar($experiment->data_files(is_deleted => 0))
+ || $experiment->experiment_type eq "ESD" )
&& scalar($experiment->spreadsheets(is_deleted => 0));
return $allfiles ? 1 : 0;
},
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <fa...@us...> - 2010-11-25 19:22:45
|
Revision: 2368
http://tab2mage.svn.sourceforge.net/tab2mage/?rev=2368&view=rev
Author: farne
Date: 2010-11-25 19:22:38 +0000 (Thu, 25 Nov 2010)
Log Message:
-----------
extra tables and views for sample and sample group tracking
Modified Paths:
--------------
branches/ae-dev/Tab2MAGE/automation/admin/app/controllers/experiments_controller.rb
branches/ae-dev/Tab2MAGE/automation/admin/app/views/layouts/admin.rhtml
branches/ae-dev/Tab2MAGE/automation/admin/db/schema.rb
branches/ae-dev/Tab2MAGE/geo/import_geo_platforms.pl
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/AE2Exporter.pm
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/DB/Sample.pm
Added Paths:
-----------
branches/ae-dev/Tab2MAGE/automation/admin/app/controllers/samplegroups_controller.rb
branches/ae-dev/Tab2MAGE/automation/admin/app/controllers/samples_controller.rb
branches/ae-dev/Tab2MAGE/automation/admin/app/models/sample.rb
branches/ae-dev/Tab2MAGE/automation/admin/app/models/sample_group.rb
branches/ae-dev/Tab2MAGE/automation/admin/app/views/experiments/sample_subs.rhtml
branches/ae-dev/Tab2MAGE/automation/admin/app/views/samplegroups/
branches/ae-dev/Tab2MAGE/automation/admin/app/views/samplegroups/list.rhtml
branches/ae-dev/Tab2MAGE/automation/admin/app/views/samples/
branches/ae-dev/Tab2MAGE/automation/admin/app/views/samples/list.rhtml
branches/ae-dev/Tab2MAGE/automation/admin/db/migrate/009_bio_sample_tracking_changes.rb
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/DB/SampleGroup.pm
Modified: branches/ae-dev/Tab2MAGE/automation/admin/app/controllers/experiments_controller.rb
===================================================================
--- branches/ae-dev/Tab2MAGE/automation/admin/app/controllers/experiments_controller.rb 2010-11-25 16:00:07 UTC (rev 2367)
+++ branches/ae-dev/Tab2MAGE/automation/admin/app/controllers/experiments_controller.rb 2010-11-25 19:22:38 UTC (rev 2368)
@@ -34,6 +34,24 @@
end
+ def sample_subs
+ num_per_page = params[:list_all].to_i.zero? ? 30 : 1000000
+
+ sql_where_clause = "is_deleted=0 and experiment_type='biosample'"
+
+ @search_term = ""
+
+ @search_term = strip_single_quotes(params[:search_term])
+
+ sql_where_clause += search_sql(@search_term,"accession","comment","name")
+
+ params[:page] ||= 1
+ @experiments = Experiment.paginate :page => params[:page],
+ :per_page => num_per_page,
+ :conditions => sql_where_clause.to_s,
+ :order => 'accession'
+ end
+
def today_list
@time_now = Time.now;
date_today = @time_now.strftime("%Y-%m-%d %H:%M:%S");
Added: branches/ae-dev/Tab2MAGE/automation/admin/app/controllers/samplegroups_controller.rb
===================================================================
--- branches/ae-dev/Tab2MAGE/automation/admin/app/controllers/samplegroups_controller.rb (rev 0)
+++ branches/ae-dev/Tab2MAGE/automation/admin/app/controllers/samplegroups_controller.rb 2010-11-25 19:22:38 UTC (rev 2368)
@@ -0,0 +1,49 @@
+class SamplegroupsController < ApplicationController
+
+ layout "admin"
+ before_filter :login_required
+
+ def index
+ list
+ render :action => 'list'
+ end
+
+ # GETs should be safe (see http://www.w3.org/2001/tag/doc/whenToUseGet.html)
+ verify :method => :post, :only => [ :destroy, :update ],
+ :redirect_to => { :action => :list }
+
+ def list
+
+ sql_where_clause = "is_deleted=0"
+
+ @search_term = ""
+
+ if params[:search_term]
+
+ # Strip single quotes, otherwise they will cause a crash.
+ @search_term = params[:search_term].gsub(/\'/, "")
+
+ # Silently allow asterisk wildcards
+ sql_search = @search_term.gsub(/\*/, "%").gsub(/\?/, "_")
+
+ sql_where_clause += " and (accession like '#{ sql_search }'" +
+ " or user_accession like '#{ sql_search }'" +
+ " or source_repository like '#{ sql_search }'" +
+ " or project_name like '#{ sql_search }'" +
+ " or comment like '%#{ sql_search }%'" +
+ " or submission_accession like '#{ sql_search }')"
+ end
+
+ if params[:submission_accession]
+ sql_where_clause += " and submission_accession = '#{ params[:submission_accession]}'"
+ end
+
+ params[:page] ||= 1
+ @samplegroups = SampleGroup.paginate :page => params[:page],
+ :per_page => 40,
+ :conditions => sql_where_clause.to_s,
+ :order => 'id DESC'
+
+ end
+
+end
\ No newline at end of file
Property changes on: branches/ae-dev/Tab2MAGE/automation/admin/app/controllers/samplegroups_controller.rb
___________________________________________________________________
Added: svn:eol-style
+ native
Added: branches/ae-dev/Tab2MAGE/automation/admin/app/controllers/samples_controller.rb
===================================================================
--- branches/ae-dev/Tab2MAGE/automation/admin/app/controllers/samples_controller.rb (rev 0)
+++ branches/ae-dev/Tab2MAGE/automation/admin/app/controllers/samples_controller.rb 2010-11-25 19:22:38 UTC (rev 2368)
@@ -0,0 +1,48 @@
+class SamplesController < ApplicationController
+
+ layout "admin"
+ before_filter :login_required
+
+ def index
+ list
+ render :action => 'list'
+ end
+
+ # GETs should be safe (see http://www.w3.org/2001/tag/doc/whenToUseGet.html)
+ verify :method => :post, :only => [ :destroy, :update ],
+ :redirect_to => { :action => :list }
+
+ def list
+
+ sql_where_clause = "is_deleted=0"
+
+ @search_term = ""
+
+ if params[:search_term]
+
+ # Strip single quotes, otherwise they will cause a crash.
+ @search_term = params[:search_term].gsub(/\'/, "")
+
+ # Silently allow asterisk wildcards
+ sql_search = @search_term.gsub(/\*/, "%").gsub(/\?/, "_")
+
+ sql_where_clause += " and (accession like '#{ sql_search }'" +
+ " or user_accession like '#{ sql_search }'" +
+ " or source_repository like '#{ sql_search }'" +
+ " or comment like '%#{ sql_search }%'" +
+ " or submission_accession like '#{ sql_search }')"
+ end
+
+ if params[:submission_accession]
+ sql_where_clause += " and submission_accession = '#{ params[:submission_accession]}'"
+ end
+
+ params[:page] ||= 1
+ @samples = Sample.paginate :page => params[:page],
+ :per_page => 40,
+ :conditions => sql_where_clause.to_s,
+ :order => 'id DESC'
+
+ end
+
+end
Property changes on: branches/ae-dev/Tab2MAGE/automation/admin/app/controllers/samples_controller.rb
___________________________________________________________________
Added: svn:eol-style
+ native
Added: branches/ae-dev/Tab2MAGE/automation/admin/app/models/sample.rb
===================================================================
--- branches/ae-dev/Tab2MAGE/automation/admin/app/models/sample.rb (rev 0)
+++ branches/ae-dev/Tab2MAGE/automation/admin/app/models/sample.rb 2010-11-25 19:22:38 UTC (rev 2368)
@@ -0,0 +1,4 @@
+class Sample < ActiveRecord::Base
+ validates_presence_of :user_accession, :submission_accession
+ validates_uniqueness_of :accession
+end
\ No newline at end of file
Property changes on: branches/ae-dev/Tab2MAGE/automation/admin/app/models/sample.rb
___________________________________________________________________
Added: svn:eol-style
+ native
Added: branches/ae-dev/Tab2MAGE/automation/admin/app/models/sample_group.rb
===================================================================
--- branches/ae-dev/Tab2MAGE/automation/admin/app/models/sample_group.rb (rev 0)
+++ branches/ae-dev/Tab2MAGE/automation/admin/app/models/sample_group.rb 2010-11-25 19:22:38 UTC (rev 2368)
@@ -0,0 +1,4 @@
+class SampleGroup < ActiveRecord::Base
+ validates_presence_of :user_accession, :submission_accession
+ validates_uniqueness_of :accession
+end
\ No newline at end of file
Property changes on: branches/ae-dev/Tab2MAGE/automation/admin/app/models/sample_group.rb
___________________________________________________________________
Added: svn:eol-style
+ native
Added: branches/ae-dev/Tab2MAGE/automation/admin/app/views/experiments/sample_subs.rhtml
===================================================================
--- branches/ae-dev/Tab2MAGE/automation/admin/app/views/experiments/sample_subs.rhtml (rev 0)
+++ branches/ae-dev/Tab2MAGE/automation/admin/app/views/experiments/sample_subs.rhtml 2010-11-25 19:22:38 UTC (rev 2368)
@@ -0,0 +1,65 @@
+<%= content_tag("h1", "BioSample Submissions", :class => "title") %>
+
+<% if flash[:notice] %><div class="notice"><%= flash[:notice] %></div><% end %>
+
+<div>
+ <% form_tag do %>
+ <%= content_tag("span", "Search on Accession, Name, Comment (wildcards: *, ?): ") %>
+ <%= text_field_tag("search_term", @search_term) %>
+ <%= submit_tag("Search") %>
+ <% end %>
+</div>
+
+<div>
+ <table>
+ <tr>
+ <%= content_tag("th", " ", { :class => 'actionlist' } ) %>
+ <%= content_tag("th", "Accession") %>
+ <%= content_tag("th", "Directory") %>
+ <%= content_tag("th", "Name") %>
+ <%= content_tag("th", "User") %>
+ <%= content_tag("th", "Status") %>
+ <%= content_tag("th", "Curator") %>
+ <%= content_tag("th", "Date last processed") %>
+ <%= content_tag("th", "Comment") %>
+ <%= content_tag("th", " ", { :class => 'actionlist' } ) %>
+ </tr>
+
+ <% @experiments.each do |e| %>
+
+ <tr>
+ <td>
+ <%= link_to "Edit", :action => "edit",
+ :id => e.id,
+ :page => params[:page],
+ :experiment_type => "biosample",
+ :search_term => @search_term %>
+ </td>
+ <%= content_tag("td", e.accession) %>
+ <%= content_tag("td", e.directory) %>
+ <%= content_tag("td", e.name) %>
+ <%= content_tag("td", e.user ? e.user.login : nil) %>
+ <%= content_tag("td", e.status) %>
+ <%= content_tag("td", e.curator) %>
+ <%= content_tag("td", e.date_last_processed ?
+ e.date_last_processed.localtime.strftime('%Y-%m-%d %H:%M:%S') :
+ "") %>
+ <%= content_tag("td", truncate(e.comment, 100)) %>
+ <td>
+ <%= link_to "Samples", :controller => "samples",
+ :action => "list",
+ :submission_accession => e.accession %>
+ <%= link_to "Groups", :controller => "samplegroups",
+ :action => "list",
+ :submission_accession => e.accession %>
+ </td>
+ </tr>
+ <% end %>
+ </table>
+
+ <%= content_tag("p", "Note that all times are GMT.") %>
+
+ <%= will_paginate @experiments, :class => 'footer',
+ :params => { :search_term => params[:search_term] } %>
+
+</div>
\ No newline at end of file
Property changes on: branches/ae-dev/Tab2MAGE/automation/admin/app/views/experiments/sample_subs.rhtml
___________________________________________________________________
Added: svn:eol-style
+ native
Modified: branches/ae-dev/Tab2MAGE/automation/admin/app/views/layouts/admin.rhtml
===================================================================
--- branches/ae-dev/Tab2MAGE/automation/admin/app/views/layouts/admin.rhtml 2010-11-25 16:00:07 UTC (rev 2367)
+++ branches/ae-dev/Tab2MAGE/automation/admin/app/views/layouts/admin.rhtml 2010-11-25 19:22:38 UTC (rev 2368)
@@ -40,6 +40,21 @@
</div>
<div class="menu-item">
+ <%= link_to "GEO experiments",
+ :controller => "tab2mages",
+ :action => "list",
+ :experiment_type => "GEO",
+ :has_gds => "0" %>
+ </div>
+
+ <div class="menu-item">
+ <%= link_to "MAGE-TAB experiments",
+ :controller => "tab2mages",
+ :action => "list",
+ :experiment_type => "MAGE-TAB" %>
+ </div>
+
+ <div class="menu-item">
<%= link_to "MUGEN experiments",
:controller => "tab2mages",
:action => "list",
@@ -60,23 +75,8 @@
:experiment_type => "GEO",
:has_gds => "1" %>
</div>
-
+
<div class="menu-item">
- <%= link_to "GEO experiments",
- :controller => "tab2mages",
- :action => "list",
- :experiment_type => "GEO",
- :has_gds => "0" %>
- </div>
-
- <div class="menu-item">
- <%= link_to "MAGE-TAB experiments",
- :controller => "tab2mages",
- :action => "list",
- :experiment_type => "MAGE-TAB" %>
- </div>
-
- <div class="menu-item">
<%= link_to "Protocols",
:controller => "protocols",
:action => "list" %>
@@ -208,6 +208,31 @@
</div>
<div class="submenu">
+ <div class="submenutitle">
+ <%= content_tag("h4", "BioSample DB") %>
+ </div>
+
+ <div class="menu-item">
+ <%= link_to "Submissions",
+ :controller => "experiments",
+ :action => "sample_subs" %>
+ </div>
+
+ <div class="menu-item">
+ <%= link_to "Sample Groups",
+ :controller => "samplegroups",
+ :action => "list" %>
+ </div>
+
+ <div class="menu-item">
+ <%= link_to "Samples",
+ :controller => "samples",
+ :action => "list" %>
+ </div>
+
+ </div>
+
+ <div class="submenu">
<% if !session[:user].nil? %>
<div class="submenutitle">
<%= content_tag("h4", "Account") %>
Added: branches/ae-dev/Tab2MAGE/automation/admin/app/views/samplegroups/list.rhtml
===================================================================
--- branches/ae-dev/Tab2MAGE/automation/admin/app/views/samplegroups/list.rhtml (rev 0)
+++ branches/ae-dev/Tab2MAGE/automation/admin/app/views/samplegroups/list.rhtml 2010-11-25 19:22:38 UTC (rev 2368)
@@ -0,0 +1,44 @@
+<%= content_tag("h1", "Sample Groups", :class => "title") %>
+
+<div>
+ <% form_tag do %>
+ <%= content_tag("span", "Search on Accession, User Accession, Submission Accession, Project Name or Source Repository (wildcards: *, ?): ") %>
+ <%= text_field_tag("search_term", @search_term) %>
+ <%= submit_tag("Search") %>
+ <% end %>
+</div>
+
+<div>
+ <table>
+ <tr>
+ <%= content_tag("th", "Accession") %>
+ <%= content_tag("th", "User accession") %>
+ <%= content_tag("th", "Submission accession") %>
+ <%= content_tag("th", "Project name") %>
+ <%= content_tag("th", "Date last processed") %>
+ <%= content_tag("th", "Source repository") %>
+ <%= content_tag("th", "Linking repositories") %>
+ <%= content_tag("th", "Comment") %>
+ </tr>
+
+ <% @samplegroups.each do |p| %>
+ <tr>
+ <%= content_tag("td", p.accession) %>
+ <%= content_tag("td", p.user_accession) %>
+ <%= content_tag("td", p.submission_accession) %>
+ <%= content_tag("td", p.project_name) %>
+ <%= content_tag("td", p.date_last_processed ?
+ p.date_last_processed.strftime('%Y-%m-%d') :
+ "") %>
+ <%= content_tag("td", p.source_repository) %>
+ <%= content_tag("td", p.linking_repositories) %>
+ <%= content_tag("td", p.comment) %>
+ </tr>
+ <% end %>
+ </table>
+
+ <%= will_paginate @samplegroups, :class => 'footer',
+ :params => { :search_term => params[:search_term] } %>
+
+
+</div>
\ No newline at end of file
Property changes on: branches/ae-dev/Tab2MAGE/automation/admin/app/views/samplegroups/list.rhtml
___________________________________________________________________
Added: svn:eol-style
+ native
Added: branches/ae-dev/Tab2MAGE/automation/admin/app/views/samples/list.rhtml
===================================================================
--- branches/ae-dev/Tab2MAGE/automation/admin/app/views/samples/list.rhtml (rev 0)
+++ branches/ae-dev/Tab2MAGE/automation/admin/app/views/samples/list.rhtml 2010-11-25 19:22:38 UTC (rev 2368)
@@ -0,0 +1,40 @@
+<%= content_tag("h1", "Samples", :class => "title") %>
+
+<div>
+ <% form_tag do %>
+ <%= content_tag("span", "Search on Accession, User Accession, Submission Accession or Source Repository (wildcards: *, ?): ") %>
+ <%= text_field_tag("search_term", @search_term) %>
+ <%= submit_tag("Search") %>
+ <% end %>
+</div>
+
+<div>
+ <table>
+ <tr>
+ <%= content_tag("th", "Accession") %>
+ <%= content_tag("th", "User accession") %>
+ <%= content_tag("th", "Submission accession") %>
+ <%= content_tag("th", "Date last processed") %>
+ <%= content_tag("th", "Source repository") %>
+ <%= content_tag("th", "Comment") %>
+ </tr>
+
+ <% @samples.each do |p| %>
+ <tr>
+ <%= content_tag("td", p.accession) %>
+ <%= content_tag("td", p.user_accession) %>
+ <%= content_tag("td", p.submission_accession) %>
+ <%= content_tag("td", p.date_last_processed ?
+ p.date_last_processed.strftime('%Y-%m-%d') :
+ "") %>
+ <%= content_tag("td", p.source_repository) %>
+ <%= content_tag("td", p.comment) %>
+ </tr>
+ <% end %>
+ </table>
+
+ <%= will_paginate @samples, :class => 'footer',
+ :params => { :search_term => params[:search_term] } %>
+
+
+</div>
\ No newline at end of file
Property changes on: branches/ae-dev/Tab2MAGE/automation/admin/app/views/samples/list.rhtml
___________________________________________________________________
Added: svn:eol-style
+ native
Added: branches/ae-dev/Tab2MAGE/automation/admin/db/migrate/009_bio_sample_tracking_changes.rb
===================================================================
--- branches/ae-dev/Tab2MAGE/automation/admin/db/migrate/009_bio_sample_tracking_changes.rb (rev 0)
+++ branches/ae-dev/Tab2MAGE/automation/admin/db/migrate/009_bio_sample_tracking_changes.rb 2010-11-25 19:22:38 UTC (rev 2368)
@@ -0,0 +1,24 @@
+class BioSampleTrackingChanges < ActiveRecord::Migration
+ def self.up
+ add_column :samples, :source_repository, :string
+
+ create_table :sample_groups, :id=>true do |t|
+ t.string :accession
+ t.string :user_accession
+ t.string :submission_accession
+ t.string :project_name
+ t.string :source_repository
+ t.string :linking_repositories
+ t.datetime :date_assigned
+ t.datetime :date_last_processed
+ t.text :comment
+ t.integer :is_deleted
+ end
+
+ end
+
+ def self.down
+ remove_column :samples, :source_repository
+ drop_table :sample_groups
+ end
+end
Property changes on: branches/ae-dev/Tab2MAGE/automation/admin/db/migrate/009_bio_sample_tracking_changes.rb
___________________________________________________________________
Added: svn:eol-style
+ native
Modified: branches/ae-dev/Tab2MAGE/automation/admin/db/schema.rb
===================================================================
--- branches/ae-dev/Tab2MAGE/automation/admin/db/schema.rb 2010-11-25 16:00:07 UTC (rev 2367)
+++ branches/ae-dev/Tab2MAGE/automation/admin/db/schema.rb 2010-11-25 19:22:38 UTC (rev 2368)
@@ -9,7 +9,7 @@
#
# It's strongly recommended to check this file into your version control system.
-ActiveRecord::Schema.define(:version => 8) do
+ActiveRecord::Schema.define(:version => 9) do
create_table "array_designs", :force => true do |t|
t.integer "miamexpress_subid"
@@ -94,6 +94,8 @@
t.string "user"
end
+ add_index "daemon_instances", ["pipeline_id"], :name => "FK_daemon_inst_to_pipeline"
+
create_table "data_files", :force => true do |t|
t.integer "experiment_id", :null => false
t.string "name"
@@ -153,6 +155,9 @@
t.datetime "date_calculated"
end
+ add_index "experiment_quality_metrics", ["quality_metric_id"], :name => "FK_exp_qm_to_qm"
+ add_index "experiment_quality_metrics", ["experiment_id"], :name => "FK_exp_qm_to_exp"
+
create_table "experiments", :force => true do |t|
t.string "accession"
t.string "name"
@@ -357,6 +362,19 @@
add_index "roles_users", ["user_id"], :name => "user_id"
add_index "roles_users", ["role_id"], :name => "role_id"
+ create_table "sample_groups", :force => true do |t|
+ t.string "accession"
+ t.string "user_accession"
+ t.string "submission_accession"
+ t.string "project_name"
+ t.string "source_repository"
+ t.string "linking_repositories"
+ t.datetime "date_assigned"
+ t.datetime "date_last_processed"
+ t.text "comment"
+ t.integer "is_deleted"
+ end
+
create_table "samples", :force => true do |t|
t.string "accession"
t.string "user_accession"
@@ -365,6 +383,7 @@
t.datetime "date_last_processed"
t.text "comment"
t.integer "is_deleted"
+ t.string "source_repository"
end
create_table "spreadsheets", :force => true do |t|
Modified: branches/ae-dev/Tab2MAGE/geo/import_geo_platforms.pl
===================================================================
--- branches/ae-dev/Tab2MAGE/geo/import_geo_platforms.pl 2010-11-25 16:00:07 UTC (rev 2367)
+++ branches/ae-dev/Tab2MAGE/geo/import_geo_platforms.pl 2010-11-25 19:22:38 UTC (rev 2368)
@@ -26,7 +26,7 @@
$| = 1;
-my $file, $want_help;
+my ($file, $want_help);
GetOptions( "f|file=s" => \$file, "h|help" => \$want_help );
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/AE2Exporter.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/AE2Exporter.pm 2010-11-25 16:00:07 UTC (rev 2367)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/AE2Exporter.pm 2010-11-25 19:22:38 UTC (rev 2368)
@@ -95,6 +95,10 @@
my $rc = 0;
my $type = $self->get_type;
+ # Always convert mac files to unix before starting
+ my $ss = $self->get_spreadsheet;
+ $self->mac2unix($ss);
+
if ($type =~/tab2mage/i){
$rc = $self->export_t2m();
}
@@ -211,6 +215,9 @@
$self->log("Rewriting SDRF to load directory");
+ my ($counts,$eol) = check_linebreaks($input);
+ unless($eol){ die "Error: cannot determine line ending type for file $input"};
+
open (my $in_fh, "<", $input) or die "Could not open SDRF $input for reading";
open (my $out_fh, ">", $output) or die "Could not open new SDRF $output for writing";
@@ -221,7 +228,7 @@
quote_char => qq{"}, # default
escape_char => qq{"}, # default
binary => 1,
- eol => "\n",
+ eol => $eol,
allow_loose_quotes => 1,
});
@@ -529,5 +536,18 @@
}
}
+sub mac2unix{
+ my ($self, $file) = @_;
+
+ my ($counts, $le) = check_linebreaks($file);
+
+ if ($counts->{mac}){
+ print "Converting mac line endings to unix for file $file\n";
+ my @args = ('/usr/bin/perl','-i','-pe','s/\r/\n/g',$file);
+ system (@args) == 0
+ or die "system @args failed: $?";
+ }
+ return;
+}
1;
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/DB/Sample.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/DB/Sample.pm 2010-11-25 16:00:07 UTC (rev 2367)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/DB/Sample.pm 2010-11-25 19:22:38 UTC (rev 2368)
@@ -22,6 +22,7 @@
date_last_processed
comment
is_deleted
+ source_repository
)
);
Added: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/DB/SampleGroup.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/DB/SampleGroup.pm (rev 0)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/DB/SampleGroup.pm 2010-11-25 19:22:38 UTC (rev 2368)
@@ -0,0 +1,56 @@
+#!/usr/bin/env perl
+#
+# $Id$
+
+use strict;
+use warnings;
+
+package ArrayExpress::AutoSubmission::DB::SampleGroup;
+use base 'ArrayExpress::AutoSubmission::DB';
+use base 'ArrayExpress::AutoSubmission::DB::Accessionable';
+
+use ArrayExpress::Curator::Common qw(date_now);
+
+__PACKAGE__->table('sample_groups');
+__PACKAGE__->columns(
+ All => qw(
+ id
+ accession
+ user_accession
+ submission_accession
+ project_name
+ source_repository
+ linking_repositories
+ date_assigned
+ date_last_processed
+ comment
+ is_deleted
+ source_repository
+ )
+);
+
+sub reassign_sample_group { # Class method.
+
+ my ( $class, $user_accession, $submission_accession) = @_;
+
+ # Checks for (user_accession eq user_accession and
+ # submission_accession eq submission_accession). Creates a
+ # new sample group in db and assigns accession if not found.
+ my $group_accession;
+
+ my $group = ArrayExpress::AutoSubmission::DB::SampleGroup->find_or_create(
+ user_accession => $user_accession,
+ submission_accession => $submission_accession,
+ is_deleted => 0,
+ );
+
+ $group_accession = $group->get_accession();
+
+ # Record that this group has been used
+ $group->set(
+ date_last_processed => date_now(),
+ );
+ $group->update();
+
+ return $group_accession;
+}
\ No newline at end of file
Property changes on: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/DB/SampleGroup.pm
___________________________________________________________________
Added: svn:eol-style
+ native
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <fa...@us...> - 2010-11-25 16:00:13
|
Revision: 2367
http://tab2mage.svn.sourceforge.net/tab2mage/?rev=2367&view=rev
Author: farne
Date: 2010-11-25 16:00:07 +0000 (Thu, 25 Nov 2010)
Log Message:
-----------
fixed typo. ignore .gz file extension
Modified Paths:
--------------
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/ExperimentSet.pm
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/RunSet.pm
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/ExperimentSet.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/ExperimentSet.pm 2010-11-24 11:55:14 UTC (rev 2366)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/ExperimentSet.pm 2010-11-25 16:00:07 UTC (rev 2367)
@@ -436,7 +436,7 @@
# PROCESSING deprecated in v1.2 (November 2010)
# Empty element still needed
- $xml->emptyTab("PROCESSING");
+ $xml->emptyTag("PROCESSING");
#my $default = $self->get_default_xml($platform,$layout);
#my $processing = $default->{processing};
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/RunSet.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/RunSet.pm 2010-11-24 11:55:14 UTC (rev 2366)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/RunSet.pm 2010-11-25 16:00:07 UTC (rev 2367)
@@ -80,7 +80,11 @@
sub get_type{
my ($self, $name) = @_;
- my @accepted_types = qw(srf sff fastq);
+ my @accepted_types = qw(srf sff fastq bam);
+
+ # Strip off .gz before attempting to get type
+ $name =~ s/\.gz$//g;
+
$name =~ /.*\.([^\.]*)$/g;
my $type = $1;
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <fa...@us...> - 2010-11-24 11:55:21
|
Revision: 2366
http://tab2mage.svn.sourceforge.net/tab2mage/?rev=2366&view=rev
Author: farne
Date: 2010-11-24 11:55:14 +0000 (Wed, 24 Nov 2010)
Log Message:
-----------
changes for SRA XML v 1.2
Modified Paths:
--------------
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/Spreadsheet.pm
branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/ExperimentSet.pm
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/Spreadsheet.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/Spreadsheet.pm 2010-11-19 11:37:07 UTC (rev 2365)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/AutoSubmission/Spreadsheet.pm 2010-11-24 11:55:14 UTC (rev 2366)
@@ -1062,17 +1062,17 @@
} },
{ 'Comment[LIBRARY_SOURCE]' => {
usage => $UHTS,
- help => 'one of GENOMIC, NON GENOMIC, SYNTHETIC, VIRAL RNA, OTHER',
+ help => 'one of GENOMIC, TRANSCRIPTOMIC, METAGENOMIC, SYNTHETIC, VIRAL RNA, OTHER',
example => 'GENOMIC',
} },
{ 'Comment[LIBRARY_STRATEGY]' => {
usage => $UHTS,
- help => 'one of WGS, WCS, CLONE, POOLCLONE, AMPLICON, CLONEEND, FINISHING, ChIP-Seq, MNase-Seq, EST, FL-cDNA, CTS, Bisulfite-Seq, DNAse-Hypersensitivity, OTHER',
+ help => 'one of WGS, WXS, RNA-Seq, WCS, CLONE, POOLCLONE, AMPLICON, CLONEEND, FINISHING, ChIP-Seq, MNase-Seq, DNAse-Hypersensitivity, Bisulfite-Seq, EST, FL-cDNA, CTS, MRE-Seq, MeDIP-Seq, MBD-Seq, OTHER',
example => 'WGS',
} },
{ 'Comment[LIBRARY_SELECTION]' => {
usage => $UHTS,
- help => 'one of RANDOM, PCR, RANDOM PCR, RT-PCR, HMPR, MF, CF-S, CF-M, CF-H, CF-T, MSLL, cDNA, ChIP, MNase, DNAse, Hybrid Selection, other, unspecified',
+ help => 'one of RANDOM, PCR, RANDOM PCR, RT-PCR, HMPR, MF, CF-S, CF-M, CF-H, CF-T, MSLL, cDNA, ChIP, MNase, DNAse, Hybrid Selection, Reduced Representation, Restriction Digest, 5-methylcytidine antibody, MBD2 protein methyl-CpG binding domain, CAGE, RACE, size fractionation, other, unspecified',
example => 'RANDOM',
} },
{ 'Sample Name' => {
Modified: branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/ExperimentSet.pm
===================================================================
--- branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/ExperimentSet.pm 2010-11-19 11:37:07 UTC (rev 2365)
+++ branches/ae-dev/Tab2MAGE/lib/ArrayExpress/SRA_XML/ExperimentSet.pm 2010-11-24 11:55:14 UTC (rev 2366)
@@ -434,17 +434,21 @@
my ($self, $platform, $layout) = @_;
my $xml = $self->get_xml_writer;
- my $default = $self->get_default_xml($platform,$layout);
- my $processing = $default->{processing};
+ # PROCESSING deprecated in v1.2 (November 2010)
+ # Empty element still needed
+ $xml->emptyTab("PROCESSING");
- unless ($processing){
- print "WARNING: no processing xml defined for platform $platform\n";
- $processing = "<!-- No processing xml defined for platorm $platform. This must be completed manually -->\n"
- }
+ #my $default = $self->get_default_xml($platform,$layout);
+ #my $processing = $default->{processing};
+
+ #unless ($processing){
+ # print "WARNING: no processing xml defined for platform $platform\n";
+ # $processing = "<!-- No processing xml defined for platorm $platform. This must be completed manually -->\n"
+ #}
- $xml->startTag("PROCESSING");
- $xml->raw($processing);
- $xml->endTag;
+ #$xml->startTag("PROCESSING");
+ #$xml->raw($processing);
+ #$xml->endTag;
}
sub get_assay_hardware{
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|