Recent changes to SADLFormat

SADLFormat modified by Aleksi Kallio

Aleksi Kallio — Thu, 26 Jun 2014 06:43:10 -0000

--- v22
+++ v23
@@ -1,134 +1 @@
-[TOC]
-
-## Describing tools with SADL
-
-SADL (Simple Analysis Description Language) is a simple notation for describing analysis tools so that they can be used in the Chipster framework. SADL describes what input files the tool takes, what output files it produces, and what parameters are needed to run it. In Chipster inputs are selected by user, clicking on datasets at the GUI. Parameters are used to create the parameter panel, and outputs are the datasets produced by the tool. 
-
-In SADL, each line describes one thing. General format of a line is: 1) what it is 2) optionality 3) internal name 4) display name 5) type 6) type details 7) description. By default inputs and parameters are required to be set by the user, but they can also be declared optional. All display names are in quotes and descriptions are in parentheses, but they can be omitted when the string does not contain whitespace or operator characters. The required order of the lines is: TOOL, INPUT*, OUTPUT*, PARAMETER*. Example of a SADL description for an simple concatenation tool is given below. 
-
-  
-
-    
-    
-    TOOL concat.R: "Concatenate tool" (Concatenates two files.)
-    INPUT file1.txt: "First input" TYPE GENERIC (First file to concatenate.)
-    INPUT file2.txt: "Second input" TYPE GENERIC (Second file to concatenate.)
-    OUTPUT concatenated.txt: "Concatenated file" (The concatenated result file.)
-    
-
-The concatenation tool is very simple. It defines the tool name and description and then the two inputs we are going to concatenate and, finally, the single output. Read further to understand the syntax that is used to define names (first there is the technical name, a colon and then the human readable name). 
-
-  
-
-
-### Names
-
-All names in SADL have same syntax. They can have two parts: ID (technical name) and human readable name (shown in GUI). ID's should not be changed without a very good reason, as they are used to identify tools, parameters etc. in the Chipster framework. Especially for users' workflows to remain valid it is best to not change the ID's. Human readable names can be changed freely. 
-
-Example of name without and with human readable part: 
-    
-    
-    p_value
-    p_value: "The P-value"
-    
-
-The ID part of the name can be followed by colon and human readable name. ID is required, but human readable name is not. Both parts are typically in parentheses, but they can be omitted, if the name is a simple string without spaces or operator characters. 
-
-### Input and output files
-
-Inputs are the data files that are being processed, and outputs are the results that are returned to user as data files. Input and output definition formats are: 
-    
-    
-    INPUT (META) (OPTIONAL) name TYPE type (description)<TYPE>
-    OUTPUT (META) (OPTIONAL) name TYPE type (description)<TYPE>
-    
-
-File names follow the normal conventions, with one addition. File name can contain the special string **{...}**, which makes it an input file set. Chipster binds all matching inputs and gives them names with numbering 1... replacing the special string. 
-
-Type information is not currently used, except for backwards compatibility in some older tools. Currently type **GENERIC** should be used always. In future it will be replaced by a more fine grained and flexible system (type tagging). 
-
-### Parameters
-
-Parameters allow user to tune behavior of an analysis tool. They are shown in the graphical parameter panel in the Chipster user interface and stored to variables or given as arguments when running the tool. 
-
-Parameter definition format is: 
-    
-    PARAMETER (OPTIONAL) name TYPE type FROM min_value TO max_value DEFAULT def_value (description)
-
-**FROM**, **TO** and **DEFAULT** are optional. Description can be left blank. 
-
-Valid parameter types are: 
-
-  * INTEGER 
-    * For integer values 
-    * Represented as a text box in GUI 
-  * DECIMAL 
-    * For decimal values 
-    * Represented as a text box in GUI 
-  * PERCENT 
-    * For percentages (integer from between 0 - 100) 
-    * Might be removed in future, if there is no need for this 
-    * Represented as a slider in GUI 
-  * STRING 
-    * For free string values 
-    * Represented as a text box in GUI 
-  * [key1:val1, key1:val2, key1:val3] 
-    * For enumerated values (selection from a predefined list) 
-    * Valid values are given in block parenthesis 
-    * Represented as a drop-down list in GUI 
-    * First part of the name is the actual technical value of the selection, is second part is given it is used in the GUI 
-  * COLUMN_SEL 
-    * For selecting one column from the input dataset 
-    * Possible values are read from the input dataset 
-      * In case of multiple inputs, present in all of them 
-    * Can also be empty 
-    * Represented as a drop-down list in GUI 
-  * METACOLUMN_SEL 
-    * For selecting one column from the phenodata 
-    * Behaves exactly like COLUMN_SEL, but uses phenodata as input dataset 
-
-Numeric parameters allow also minimum and maximum values to be set, by using keywords **FROM** and **TO** after the parameter type. For enumeration type, **FROM** and **TO** can be used to specify the minimum and maximum number of selections the user can make (by default one selection can be made). 
-
-All parameters allow a default value, which is given by using the keyword **DEFAULT**. The default value must be a valid value for the parameter. User interface implements validity checking in real time, so writing "one" to a **INTEGER** text box or "10" to a **INTEGER** text box with maximum of 5 results in immediate error shown in the parameter panel side and run button being blocked. 
-
-### Advanced example
-
-Below is an example of an imaginery tool that highlights all different features that can be described with the language. 
-    
-    
-    TOOL util-test.R: "Test tool" (This tool description is shown to the user in the GUI (note that certain operators must be escaped\).)
-    INPUT microarray{...}.tsv: "Raw data files" TYPE CDNA (A set of 1 or more raw data files that are given as input.)
-    INPUT META phenodata.tsv: "Experiment description" TYPE GENERIC (Meta-level description of the input files.)
-    OUTPUT result.txt: "Result file" (The output file that this tool always produces.)
-    OUTPUT OPTIONAL warnings.txt: "Warning file" (The tool might produce warnings while running and then they are returned also.)
-    PARAMETER value1: "The first value" TYPE INTEGER FROM 0 TO 200 DEFAULT 10 (Description of this parameter)
-    PARAMETER OPTIONAL value2: "The second value" TYPE DECIMAL FROM 0 TO 200 DEFAULT 20.2 (Description of this parameter)
-    PARAMETER method: "The method" TYPE [linear: "Linear scale", logarithmic: "Logarithmic scale"] DEFAULT logarithmic (Description of this parameter)
-    PARAMETER genename: "Gene name" TYPE STRING DEFAULT at_1234 (Description of this parameter)
-    PARAMETER key: "Key column" TYPE COLUMN_SEL (Which column is used as a key)
-    
-
-### Format of SADL syntax description
-
-For geek users, a more formal syntax defination is below. It is in the form of rewrite rules. First rule in the list is the initial rule where rewriting is started. Quoted texts are snippets of SADL. For example, TOOL is a term that is rewritten using the given rules, but "TOOL" is a string that should be found in the source code. Operators ?, +, * and | have their common semantics. The canonical syntax definition in maintained in the Javadoc documentation of the class SADLSyntax. 
-    
-    
-    -> TOOL+
-    TOOL -> "TOOL" NAME DESCRIPTION INPUT* OUTPUT* PARAMETER*
-    INPUT -> "INPUT" META? OPTIONALITY? NAME "TYPE" TYPE_NAME DESCRIPTION
-    OUTPUT -> "OUTPUT" META? OPTIONALITY? NAME DESCRIPTION
-    PARAMETER -> "PARAMETER" OPTIONALITY? NAME "TYPE" PARAMETER_TYPE PARAMETER_FROM? PARAMETER_TO? PARAMETER_DEFAULT? DESCRIPTION 
-    PARAMETER_TYPE -> TOKEN | PARAMETER_TYPE_ENUM
-    PARAMETER_TYPE_ENUM -> "[" PARAMETER_TYPE_ENUM_ELEMENTS "]"
-    PARAMETER_TYPE_ENUM_ELEMENTS -> NAME | NAME "," PARAMETER_TYPE_ENUM_ELEMENTS
-    PARAMETER_FROM -> "FROM" TOKEN
-    PARAMETER_TO -> "TO" TOKEN
-    PARAMETER_DEFAULT -> "DEFAULT" PARAMETER_DEFAULT_ELEMENT
-    PARAMETER_DEFAULT_ELEMENT -> TOKEN | TOKEN "," PARAMETER_DEFAULT_ELEMENT 
-    OPTIONALITY -> "OPTIONAL"
-    META -> "META"
-    NAME -> TOKEN | TOKEN ":" TOKEN
-    DESCRIPTION -> TOKEN
-    TYPE_NAME -> TOKEN (see SADLSyntax.InputType for declaration, implementations pluggable)
-    TOKEN -> any single token produced by tokeniser
-    
+See [TechnicalManual#Describing_tools_with_SADL].

SADLFormat modified by Aleksi Kallio

Aleksi Kallio — Thu, 26 Jun 2014 06:43:10 -0000

--- v21
+++ v22
@@ -2,23 +2,21 @@

 ## Describing tools with SADL

-SADL (Simple Analysis Description Language) is a simple notation for describing analysis tools so that they can be used in the Chipster 2.0 framework. SADL describes what input files the tool takes, what output files it produces, and what parameters are needed for running it. In Chipster inputs are selected by user, clicking on datasets at the GUI. Parameters are used to create the parameter panel, and outputs are the datasets produced by the tool. 
+SADL (Simple Analysis Description Language) is a simple notation for describing analysis tools so that they can be used in the Chipster framework. SADL describes what input files the tool takes, what output files it produces, and what parameters are needed to run it. In Chipster inputs are selected by user, clicking on datasets at the GUI. Parameters are used to create the parameter panel, and outputs are the datasets produced by the tool. 

-In SADL, each line describes a one thing. General format of a line is: 1) what is it 2) optionality 3) internal name 4) display name 5) type 6) type details 7) description. By default inputs and parameters are required to be set by the user, but they can also be declared optional. All display names are in quotes and descriptions are in parentheses, but they can be omitted when the string does not contain whitespace or operator characters. Order of lines is: TOOL, INPUT*, OUTPUT*, PARAMETER* and the order is enforced. Example of a SADL description for an imaginery tool is given below. 
+In SADL, each line describes one thing. General format of a line is: 1) what it is 2) optionality 3) internal name 4) display name 5) type 6) type details 7) description. By default inputs and parameters are required to be set by the user, but they can also be declared optional. All display names are in quotes and descriptions are in parentheses, but they can be omitted when the string does not contain whitespace or operator characters. The required order of the lines is: TOOL, INPUT*, OUTPUT*, PARAMETER*. Example of a SADL description for an simple concatenation tool is given below. 
+
+  
+

-    TOOL util-test.R: "Test tool" (This tool description is shown to the user in the GUI (note that certain operators must be escaped.\))
-    INPUT microarray{...}.tsv: "Raw data files" TYPE CDNA
-    INPUT META phenodata.tsv: "Experiment description" TYPE GENERIC
-    OUTPUT result.txt: "Result file"
-    PARAMETER value1: "The first value" TYPE INTEGER FROM 0 TO 200 DEFAULT 10 (Description of this parameter)
-    PARAMETER OPTIONAL value2: "The second value" TYPE DECIMAL FROM 0 TO 200 DEFAULT 20.2 (Description of this parameter)
-    PARAMETER method: "The method" TYPE [linear: "Linear scale", logarithmic: "Logarithmic scale"] logarithmic (Description of this parameter)
-    PARAMETER genename: "Gene name" TYPE STRING DEFAULT at_1234 (Description of this parameter)
-    PARAMETER key: "Key column" TYPE COLUMN_SEL (Which column is used as a key)
+    TOOL concat.R: "Concatenate tool" (Concatenates two files.)
+    INPUT file1.txt: "First input" TYPE GENERIC (First file to concatenate.)
+    INPUT file2.txt: "Second input" TYPE GENERIC (Second file to concatenate.)
+    OUTPUT concatenated.txt: "Concatenated file" (The concatenated result file.)

-Note that SADL replaces the previous VVSADL language. The two languages are quite similar, but can be told apart easily by looking at the first line. SADL descriptions start with TOOL, whereas old VVSADL descriptions start with ANALYSIS. SADL parser has backwards compatibility support for parsing VVSADL, but the support will be removed in future. 
+The concatenation tool is very simple. It defines the tool name and description and then the two inputs we are going to concatenate and, finally, the single output. Read further to understand the syntax that is used to define names (first there is the technical name, a colon and then the human readable name). 

@@ -34,20 +32,20 @@
     p_value: "The P-value"

-The ID part of the name can be followed by colon and human readable name. ID is required, but human readable name is not. Both parts are typically in parentheses, but they can be omitted, if the name is a simple string without spaces. 
+The ID part of the name can be followed by colon and human readable name. ID is required, but human readable name is not. Both parts are typically in parentheses, but they can be omitted, if the name is a simple string without spaces or operator characters. 

 ### Input and output files

 Inputs are the data files that are being processed, and outputs are the results that are returned to user as data files. Input and output definition formats are: 

-    INPUT (META) (OPTIONAL) name TYPE (COMMENT)<TYPE>
-    OUTPUT (META) (OPTIONAL) name TYPE (COMMENT)<TYPE>
+    INPUT (META) (OPTIONAL) name TYPE type (description)<TYPE>
+    OUTPUT (META) (OPTIONAL) name TYPE type (description)<TYPE>

 File names follow the normal conventions, with one addition. File name can contain the special string **{...}**, which makes it an input file set. Chipster binds all matching inputs and gives them names with numbering 1... replacing the special string. 

-For information on types and more details on binding of inputs and outputs, see [AnalysisToolInputsAndOutputs] (still in old VVSADL). 
+Type information is not currently used, except for backwards compatibility in some older tools. Currently type **GENERIC** should be used always. In future it will be replaced by a more fine grained and flexible system (type tagging). 

 ### Parameters

@@ -55,7 +53,7 @@

 Parameter definition format is: 

-    PARAMETER (OPTIONAL) name TYPE <TYPE> FROM min_value TO max_value DEFAULT def_value (description)
+    PARAMETER (OPTIONAL) name TYPE type FROM min_value TO max_value DEFAULT def_value (description)

 **FROM**, **TO** and **DEFAULT** are optional. Description can be left blank. 

@@ -91,11 +89,28 @@

 Numeric parameters allow also minimum and maximum values to be set, by using keywords **FROM** and **TO** after the parameter type. For enumeration type, **FROM** and **TO** can be used to specify the minimum and maximum number of selections the user can make (by default one selection can be made). 

-All parameters allow a default value, which is given by using the keyword **DEFAULT**. The default value must be a valid value for the parameter. User interface implements validity checking in real time, so writing "one" to a **INTEGER** text box or "10" to a **INTEGER** text box with maximum 5 results in immediate error shown in the parameter panel side. 
+All parameters allow a default value, which is given by using the keyword **DEFAULT**. The default value must be a valid value for the parameter. User interface implements validity checking in real time, so writing "one" to a **INTEGER** text box or "10" to a **INTEGER** text box with maximum of 5 results in immediate error shown in the parameter panel side and run button being blocked. 

-### Format SADL syntax description
+### Advanced example

-Syntax defination is below. It is in the form of rewrite rules. First rule in the list is the initial rule where rewriting is started. Quoted texts are snippets of SADL. For example, TOOL is a term that is rewritten using the given rules, but "TOOL" is a string that should be found in the source code. Operators ?, +, * and | have their common semantics. The canonical syntax definition in maintained in the Javadoc documentation of the class SADLSyntax. 
+Below is an example of an imaginery tool that highlights all different features that can be described with the language. 
+    
+    
+    TOOL util-test.R: "Test tool" (This tool description is shown to the user in the GUI (note that certain operators must be escaped\).)
+    INPUT microarray{...}.tsv: "Raw data files" TYPE CDNA (A set of 1 or more raw data files that are given as input.)
+    INPUT META phenodata.tsv: "Experiment description" TYPE GENERIC (Meta-level description of the input files.)
+    OUTPUT result.txt: "Result file" (The output file that this tool always produces.)
+    OUTPUT OPTIONAL warnings.txt: "Warning file" (The tool might produce warnings while running and then they are returned also.)
+    PARAMETER value1: "The first value" TYPE INTEGER FROM 0 TO 200 DEFAULT 10 (Description of this parameter)
+    PARAMETER OPTIONAL value2: "The second value" TYPE DECIMAL FROM 0 TO 200 DEFAULT 20.2 (Description of this parameter)
+    PARAMETER method: "The method" TYPE [linear: "Linear scale", logarithmic: "Logarithmic scale"] DEFAULT logarithmic (Description of this parameter)
+    PARAMETER genename: "Gene name" TYPE STRING DEFAULT at_1234 (Description of this parameter)
+    PARAMETER key: "Key column" TYPE COLUMN_SEL (Which column is used as a key)
+    
+
+### Format of SADL syntax description
+
+For geek users, a more formal syntax defination is below. It is in the form of rewrite rules. First rule in the list is the initial rule where rewriting is started. Quoted texts are snippets of SADL. For example, TOOL is a term that is rewritten using the given rules, but "TOOL" is a string that should be found in the source code. Operators ?, +, * and | have their common semantics. The canonical syntax definition in maintained in the Javadoc documentation of the class SADLSyntax. 

     -> TOOL+

SADLFormat modified by Chipster admin

Chipster admin — Thu, 26 Jun 2014 06:43:10 -0000

--- v20
+++ v21
@@ -38,9 +38,7 @@

 ### Input and output files

-Inputs are the data files that are being processed, and outputs are the results that are returned to user as data files. 
-
-Input and output definition formats are: 
+Inputs are the data files that are being processed, and outputs are the results that are returned to user as data files. Input and output definition formats are: 

     INPUT (META) (OPTIONAL) name TYPE (COMMENT)<TYPE>

SADLFormat modified by Eija Korpelainen

Eija Korpelainen — Thu, 26 Jun 2014 06:43:10 -0000

--- v19
+++ v20
@@ -10,13 +10,12 @@
     TOOL util-test.R: "Test tool" (This tool description is shown to the user in the GUI (note that certain operators must be escaped.\))
     INPUT microarray{...}.tsv: "Raw data files" TYPE CDNA
     INPUT META phenodata.tsv: "Experiment description" TYPE GENERIC
-    OUTPUT result{...}.txt: "Result files"
-    OUTPUT OPTIONAL error.txt: "Error, if any"
-    PARAMETER value1: "The first value" TYPE INTEGER FROM 0 TO 200 DEFAULT 10 (the first value of the result set)
-    PARAMETER OPTIONAL value2: "The second value" TYPE DECIMAL FROM 0 TO 200 DEFAULT 20.2 (the second value of the result set)
-    PARAMETER method: "The method" TYPE [linear: "Linear scale", logarithmic: "Logarithmic scale"] logarithmic (which scale to use)
-    PARAMETER genename: "Gene name" TYPE STRING DEFAULT at_1234 (which gene we are interested in)
-    PARAMETER key: "Key column" TYPE COLUMN_SEL (which column we use as a key)
+    OUTPUT result.txt: "Result file"
+    PARAMETER value1: "The first value" TYPE INTEGER FROM 0 TO 200 DEFAULT 10 (Description of this parameter)
+    PARAMETER OPTIONAL value2: "The second value" TYPE DECIMAL FROM 0 TO 200 DEFAULT 20.2 (Description of this parameter)
+    PARAMETER method: "The method" TYPE [linear: "Linear scale", logarithmic: "Logarithmic scale"] logarithmic (Description of this parameter)
+    PARAMETER genename: "Gene name" TYPE STRING DEFAULT at_1234 (Description of this parameter)
+    PARAMETER key: "Key column" TYPE COLUMN_SEL (Which column is used as a key)

 Note that SADL replaces the previous VVSADL language. The two languages are quite similar, but can be told apart easily by looking at the first line. SADL descriptions start with TOOL, whereas old VVSADL descriptions start with ANALYSIS. SADL parser has backwards compatibility support for parsing VVSADL, but the support will be removed in future.

SADLFormat modified by Eija Korpelainen

Eija Korpelainen — Thu, 26 Jun 2014 06:43:10 -0000

--- v18
+++ v19
@@ -13,9 +13,7 @@
     OUTPUT result{...}.txt: "Result files"
     OUTPUT OPTIONAL error.txt: "Error, if any"
     PARAMETER value1: "The first value" TYPE INTEGER FROM 0 TO 200 DEFAULT 10 (the first value of the result set)
-    PARAMETER value2: "The second value" TYPE DECIMAL FROM 0 TO 200 DEFAULT 20 (the second value of the result set)
-    PARAMETER OPTIONAL value3: "The third value" TYPE DECIMAL FROM 0 TO 200 DEFAULT 30.2 (the third value of the result set)
-    PARAMETER method: "The fourth value" TYPE PERCENT DEFAULT 34 (how much we need)
+    PARAMETER OPTIONAL value2: "The second value" TYPE DECIMAL FROM 0 TO 200 DEFAULT 20.2 (the second value of the result set)
     PARAMETER method: "The method" TYPE [linear: "Linear scale", logarithmic: "Logarithmic scale"] logarithmic (which scale to use)
     PARAMETER genename: "Gene name" TYPE STRING DEFAULT at_1234 (which gene we are interested in)
     PARAMETER key: "Key column" TYPE COLUMN_SEL (which column we use as a key)

SADLFormat modified by Eija Korpelainen