You can subscribe to this list here.
2001 |
Jan
(135) |
Feb
(57) |
Mar
(84) |
Apr
(43) |
May
(77) |
Jun
(51) |
Jul
(21) |
Aug
(55) |
Sep
(37) |
Oct
(56) |
Nov
(75) |
Dec
(23) |
---|---|---|---|---|---|---|---|---|---|---|---|---|
2002 |
Jan
(32) |
Feb
(174) |
Mar
(121) |
Apr
(70) |
May
(55) |
Jun
(20) |
Jul
(23) |
Aug
(15) |
Sep
(12) |
Oct
(58) |
Nov
(203) |
Dec
(90) |
2003 |
Jan
(37) |
Feb
(15) |
Mar
(14) |
Apr
(57) |
May
(7) |
Jun
(40) |
Jul
(36) |
Aug
(1) |
Sep
(56) |
Oct
(38) |
Nov
(105) |
Dec
(2) |
2004 |
Jan
|
Feb
(117) |
Mar
(69) |
Apr
(160) |
May
(165) |
Jun
(35) |
Jul
(7) |
Aug
(80) |
Sep
(47) |
Oct
(23) |
Nov
(8) |
Dec
(42) |
2005 |
Jan
(19) |
Feb
(2) |
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
2007 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
(2) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
2010 |
Jan
|
Feb
(2) |
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
From: SourceForge.net <no...@so...> - 2004-04-30 15:56:31
|
Bugs item #932847, was opened at 2004-04-10 10:11 Message generated for change (Settings changed) made by jason_e_stewart You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=932847&group_id=16453 Category: DB Schema Group: Genex-2 >Status: Closed >Resolution: Fixed Priority: 8 Submitted By: Jason E. Stewart (jason_e_stewart) Assigned to: Jason E. Stewart (jason_e_stewart) Summary: Add ArrayManufacture and ManufactureLIMSBiomaterial to DB Initial Comment: These critical MAGE classes were omitted from the schema. These are the tables that link Array Features to specific Biomaterial's during the printing process, e.g. to plate ID, plate row, plate col, etc. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=932847&group_id=16453 |
From: SourceForge.net <no...@so...> - 2004-04-30 15:55:15
|
Bugs item #945364, was opened at 2004-04-30 09:27 Message generated for change (Settings changed) made by jason_e_stewart You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=945364&group_id=16453 Category: Mason GUI Group: Genex-2 >Status: Closed >Resolution: Fixed Priority: 7 Submitted By: Jason E. Stewart (jason_e_stewart) Assigned to: Jason E. Stewart (jason_e_stewart) Summary: copy-to-scratch.html does not reload copy-filter on error Initial Comment: The array of copy filters is not loaded if a required argument (DBA name) is not provided ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=945364&group_id=16453 |
From: SourceForge.net <no...@so...> - 2004-04-30 15:49:23
|
Bugs item #945367, was opened at 2004-04-30 09:30 Message generated for change (Settings changed) made by jason_e_stewart You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=945367&group_id=16453 Category: Mason GUI Group: Genex-2 >Status: Closed >Resolution: Fixed Priority: 7 Submitted By: Jason E. Stewart (jason_e_stewart) Assigned to: Jason E. Stewart (jason_e_stewart) Summary: exp-analyze.mason is still showing mba copy as option Initial Comment: This has been totalled moved to copy-to-scratch.html and no longer works. It's not clear that MBA's should be included here - although perhaps an option to export the data should be available. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=945367&group_id=16453 |
From: SourceForge.net <no...@so...> - 2004-04-30 15:48:34
|
Bugs item #945353, was opened at 2004-04-30 09:02 Message generated for change (Comment added) made by jason_e_stewart You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=945353&group_id=16453 Category: Mason GUI Group: Genex-2 >Status: Closed >Resolution: Fixed Priority: 8 Submitted By: Jason E. Stewart (jason_e_stewart) Assigned to: Jason E. Stewart (jason_e_stewart) Summary: 'no child processes' error in copy-to-scratch.html Initial Comment: When copying an MBA to a Scratch view, an error is caught after copy-to-scratch.pl is called even though the process exits normally. ---------------------------------------------------------------------- >Comment By: Jason E. Stewart (jason_e_stewart) Date: 2004-04-30 09:48 Message: Logged In: YES user_id=85550 It was a valid error. Due to a switch in the CGI parameter name, the hidden variable was no longer being store if the page was called from the old application route through exp-analyze.mason. This old route has been disabled. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=945353&group_id=16453 |
From: SourceForge.net <no...@so...> - 2004-04-30 15:30:30
|
Bugs item #945367, was opened at 2004-04-30 09:30 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=945367&group_id=16453 Category: Mason GUI Group: Genex-2 Status: Open Resolution: None Priority: 7 Submitted By: Jason E. Stewart (jason_e_stewart) Assigned to: Jason E. Stewart (jason_e_stewart) Summary: exp-analyze.mason is still showing mba copy as option Initial Comment: This has been totalled moved to copy-to-scratch.html and no longer works. It's not clear that MBA's should be included here - although perhaps an option to export the data should be available. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=945367&group_id=16453 |
From: SourceForge.net <no...@so...> - 2004-04-30 15:27:40
|
Bugs item #945364, was opened at 2004-04-30 09:27 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=945364&group_id=16453 Category: Mason GUI Group: Genex-2 Status: Open Resolution: None Priority: 7 Submitted By: Jason E. Stewart (jason_e_stewart) Assigned to: Jason E. Stewart (jason_e_stewart) Summary: copy-to-scratch.html does not reload copy-filter on error Initial Comment: The array of copy filters is not loaded if a required argument (DBA name) is not provided ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=945364&group_id=16453 |
From: SourceForge.net <no...@so...> - 2004-04-30 15:02:30
|
Bugs item #945353, was opened at 2004-04-30 09:02 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=945353&group_id=16453 Category: Mason GUI Group: Genex-2 Status: Open Resolution: None Priority: 8 Submitted By: Jason E. Stewart (jason_e_stewart) Assigned to: Jason E. Stewart (jason_e_stewart) Summary: 'no child processes' error in copy-to-scratch.html Initial Comment: When copying an MBA to a Scratch view, an error is caught after copy-to-scratch.pl is called even though the process exits normally. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=945353&group_id=16453 |
From: <ja...@op...> - 2004-04-30 14:37:58
|
Harry J Mangalam <hj...@ta...> writes: > PBA (should be renamed PhysicalBioAssay) > Def: again, I dunno for sure, but it seems that it refers to the general > chip line or product ID or expression technology that is used to quantitate > the assay or the organzation that provided the dataset. > Prerequisites: > Measured BioAssay (optional) > Data Provider of type Person or Organization (optional) > Array (required) The BioAssay's are simply collections of annotations associated with actual data (images are the data for PBA and numerical data matrices for MBA and DBA). This is really one of the nicest parts of the MAGE model. Genex-1 only had the ArrayMeasurement - it sucked, and it sucked well. This was because we attempted to place all data annotation for data matrices in a single table - the problem was there were three distinct types of data that needed annotation: the physical chip data, 'raw' data fresh off the feature extraction program, and derived data which is mathematical munging of other data. MAGE sorts this all out into three types of BioAssay. From the MAGE Object Model: PhysicalBioAssay A PhysicalBioAssay is a BioAssay created by the bioAssayCreation event (e.g. in gene expression analysis this event is represented by the hybridization event). MeasuredBioAssay A MeasuredBioAssay is the result of direct processing of information in a PhysicalBioAssay by the featureExtraction event. A MeasuredBioAssay often uses images that are referenced through the PhysicalBioAssay. DerivedBioAssay A DerivedBioAssay is a BioAssay that is created by the Transformation BioEvent from one or more MeasuredBioAssays or DerivedBioAssays. So the idea is: 1..n 1..n PBA -> MBA -> DBA A single chip (PBA) can produce one or more images (each chip can be filtered and re-hybridized or simply re-scanned at different PMT settings), which when run through a feature extraction software program will produce a 'raw' data matrix (MBA) and when these data matrices are analyzed they produce one or more derived data matrices (DBA). The Physical BioAssay is all the physical stuff associated with a hybridization: the chip, the samples, the protocols for hybridization, washing, and scanning, and finally the images produced are the Physical BioAssay Data (PBA Data). The Measured BioAssay is all the meta data associated with the feature extraction event - it is *not* anything physical, it just records all the protocols and information about where the MBA data matrix came from. This is what many consider 'raw' data - the output of the feature extraction program. MGED and MAGE consider the images to be the only true 'raw' data (because there is already data processing that occurs when producing the MBA Data), and so this BioAssay was called 'Measured' to indicate that it is the 'rawest' possible numerical data. Likewise, the Derived BioAssay is the meta data associated with a data matrix produced by numerical manipulations of other data matrices (either MBA Data or DBA Data). So the BioAssay's are simply ways of assoicating annotations (or meta data) with the actual 'data' (images for PBA or data matrices for MBA and DBA). To say it another way, the BioAssay's give you all the information about how the data was produced - which is seperate from the data itself - by seperating the annotations from the actual data it is much easier to do queries because the data matrices are huge while the meta data is small. > Array > Def: refers to the specific parameters on a per-chip or per-assay basis, > including manufacturer, spot concentration, coating, spotter hardware, > medium, etc > Prerequisites: > Spotter Software of type Software (optional) > Provider Contact of type Person or Organization (optional) > Array Manufacture (required) > Array Design (required) > Spotter Hardware of type Hardware (optional) > Coating (optional) > Technology Type (required, but only selectable) > Medium (optional, selectable) > Default Spot Concentration Units (optional) > > > ArrayManufacture > Def: the date and manufacturers name for backtrack and quality control > purposes. > Prerequisites: > Manufacturer Contact of type Person or Orgazation (optional) This is a bit of an oddity, but it is the link between the physical chip and all the LIMS information that went into creating it. So if a lab wants to track this it is done here. > Sample (how does this differ from BioMaterial?_ This is a term from Genex-1, if it is used anywhere it is a typo. It should be Biomaterial. > Def: another descriptor for the bioThingie from which you derived the cDNA > for the hybridization. Requires a Data Provider (Person or Org) > Prerequisites: > Data Provider of type Person or Organization (optional) > Taxon (optional) > > > FactorValues for PBA > - ???? (and page comes up with an error) These are the values for ExperimentalFactor's for any given PBA that is associated with the Experiment. So if an Experiment is a simple time course it has one ExperimentalFactor, 'time'. Each time point in the time course is going to have a seperate PBA and each of these will have the associated FactorValue which is the time point they were harvested. If an experiment has multiple ExperimentalFactors, so 'time' and 'glucose concentration', then each PBA may have (but is not required to have) a FactorValue for each of the ExperimentalFactor's. In Genex-1 this was called the AM_FactorValues table (in fact MAGE adopted the genex model for this, but changed the names slightly). Linking Tables -- One thing that you've missed (because the insert/edit GUI's can't currently capture them) are the linking tables in the DB. These are represented by the apps under the 'Associate' links: * 'Groups | Users' => GroupSecLink table, which controls what users belong to which groups * 'Roles | Users' => (this is the Pg system table pg_group) this controls table level privelege, i.e 'ADMIN', 'CURATOR', or 'USER'. * 'PBA | Protocols' => ProtocolApplicationLink table, which enables protocols to be associated with a PhysicalBioAssay, e.g. hybridization or washing or scanning (probably wants to be *all* BioAssay's and not just PBA's. * 'PBA | Biomaterials' => BiomaterialLink table, allows Biomaterials to be associated with a PBA * 'Biomaterial | Protocols' => ProtocolApplicationLink table, enables protocols to be associated with biomaterials, e.g. labelling protocol, RNA extraction, cDNA preperation, etc. * 'Experiment | BioAssays' => ExperimentLink table, allows BioAssay's to be added to an Experiment. BioAssay's belong to multiple Experiment's. There are two other linking table that do not have a GUI's: * QuantitationTypeDimensionLink - this specifies what QuantitationType's belong to a QT Dimension - these are entered via the QT Dimension loader and not by hand. * ManufactureLIMSBiomaterials - this enables Biomaterial's to be associated with an ArrayManufacture - so that it is possible to identify which nucleic acids are spotted to any given Feature as well as an optional link to an external LIMS DB for each Feature (just noticed an omission in the schema diagram - this table needs a one-to-one link to Feature). HTH, jas. |
From: <ja...@op...> - 2004-04-30 11:32:40
|
Hey All, I started a data loading workflow, but I thought I would expand on it here in an email. This is totally the product of my own deranged thinking, and this could explain why the DB is so confusing. All ideas here are completely open to be changed by a group discussion - they are simply implemented in their current fashion to provide a useful starting point for enlightened discussion. Once again, none of this is sunk in stone, it is just represents my current level of psychosis. As always, feedback is required... Cheers, jas. -- Assumption: * the basic working model for Genex is to support a medium sized core facility that has lab technician's who generate data, system admins who can install and maintain software, bioinformaticians who can create and install new data procession and data analysis tools, and biologist who design experiments, and analyze data. Perhaps this is wrong-headed and instead we want to focus primarily on small labs. My thinking was that this shouldn't be our main target - because small labs are not likely to have adequate computer admin support and this will create way too many questions about how to install the system. I know that I don't have enough time to do that, but perhaps others do. Workflow: One time work -- 1) System Installation: this is done by a Unix-savvy system admin - 5 years unix experience is a *minimum* for install and maintaining Genex - it is *not* intended to be possible for a novice to maintain the system. 2) Basic system configuration: this is to be done by curators and admins who can load the basic set of information to enable users to begin adding data and using the system: - ArrayDesign's used in the lab - FeatureExtraction Software used in the lab - using existing tools to create dynamic MBA Data tables (table creation requires admin privelege) - Data analysis and processing algorithms - using existing tools to create Scratch views to store the output of the algorithms (view creation requires admin privelege) 3) [Optional] Upgrade system - admins monitor improvements to genex and upgrade when needed. Upgrades *must* be transparent to users. Repitive work -- 4) Arrays are produced or procured by the facility - The Array and ArrayManufacture information is entered into the DB, so that the Array's can be identified when data loading occurs. 5) Sample's (Biomaterial's) are produced by biologists (or lab techs) - they are entered into the DB, and the DB produces an identifier (bar-code) for the sample. 6) A biologist requests a set of hybridizations. They indicate which bar-coded samples (Biomaterial's) should be used with which set of chips (ArrayDesign) and with what Protocol's (washing, hybridization, scanning, etc), and they indicate to which experiment the data should be added [Q: this should be optional, shouldn't it?]. 7) The core facility technicians do the actual work: hybridize, wash, scan. They enter the data (MBA Data, images) into the DB using the data loader. Then they send an email to the biologist acknowledging that the data is ready. 8) The biologist/bioinformatician copies the MBA Data into the Scratch table and chooses a set of processing Protocol's to apply to the new data. 9) The biologist/bioinformatician analyzes the data storing intermediate output into the Scratch table. 10) Once analysis is complete the biologist/bioinformatician copies the data into the archival Derived BioAssay Data table (DBA Data). 11) The biologist/bioinformatician uses the annotation system to record any important information that might have been missed. 12) The biologist/bioinformatician exports the data as MAGE-ML and sends it off to ArrayExpress as a pre-requisite for publication. 13) [Optional] - a bioinformatician adds new processing and analysis Protocols to the DB. |
From: <ja...@op...> - 2004-04-30 11:02:07
|
Hey All, History -- We talked about protecting and archiving raw expression data in Genex a *long* time ago and based on that discussion I made a dictatorial decision. What we discussed was that raw data should be protected and stored, and ordinary users should not be allowed to delete the raw data - that only someone with privelege (PI or curator or admin) should have that power. This was so that an institution or PI could *always* go back and re-analyze the *original* data from any scientist. The rawest data we get are the images - that's what MAGE calls the Physical BioAssay Data. But the first set of numbers that is generated comes after the FeatureExtraction Software (FE Software) generates what MAGE calls the Measured BioAssay Data (MBA Data). Current implementation -- The dictatorial decision I took was that MBA Data cannot be modified by users that only curators could modify it - so whenever we create an MBA Data table (what I call the 'dynamic data' tables in the schema diagram) it is given CURATOR privelege, so only curators can INSERT, UPDATE, or DELETE data from it. I think protecting the data for archival reasons is good, so stopping UPDATE's and DELETE's is good. But I've been really bothered by not allowing normal users to INSERT the data. Proposal -- There are already two other tables that are very similar: ChangeLog and Audit. Both of these tables allow INSERT's by users, but on an admin can modify an existing entry - they are archive only. I propose to add the dynamic data tables to this same category. This means that normal users will be able to insert data into the MBA data tables. Pros and Cons -- Pros: * Users will be happy - they can add data whenever they want * Curators will be happy - they don't have to listen to users complain, "why didn't you enter my data??" Cons: * Users will be unhappy - they will load data whenever they want and this will slow down the DB *a lot* * Curators will be unhappy - they will have to listen to users complain, "Why is this DB so %#!$!! slow??" Workaround -- Since the only big problem I foresee is a slowdown of the DB, it can easily be solved in two ways by the system admins: * Use a queuing system - data is loaded during a pre-appointed time of the day or night when load is low * Use a preemptive scheduler - data loading is given a low priority, and so it can be prempted by a high priority task like analysis * Use a seperate machine for loading and a seperate machine for analysis - this is a bit hairier but we can pretty easily set up the DB to be a multi-machine system. There are three main tasks in the DB: * Data Loading - ArrayDesigns and MBA data * Data processing and analysis - number crunching * Annotation - get biologists to explain what they *actually* did Annotation is a low bandwidth task, it will not be affected by the speed of the computer. Data loading is a highly disk-intensive task - all those INSERT's really grind the disk to a halt. Data processing is a highly CPU intensive task. It is possible for each of these to co-exist happily, but we haven't tested the reality. If data loading interferes with data analysis, then we may be forced to use one of the above workarounds. My original solution of making the tables CURATOR privelege was basically to enforce a queuing system - the curator decides when to load the data. Thoughts? Comments? Cheers, jas. |
From: <ja...@op...> - 2004-04-30 09:59:25
|
Harry J Mangalam <hj...@ta...> writes: > From: > http://bodi.tacgi.com/genex/mason/login/annotate/annotate.html > hitting the Insert -> FactorValues for PBA > I get: > ERROR: Page: edit/pba-factorvalue-insert.html not found > on top of a table that shows the experiments that I've entered. Right, this is a typo, edit => insert. All the navbar URL's are hand-coded so if one is wrong - please file a bug on it so that we don't lose track of it. Give it a priority of 8, so that it get's fixed before Beta Release. Cheers, jas. |
From: <ja...@op...> - 2004-04-30 09:53:32
|
Hey All, Here's the synopsis: 1) We're getting ready for a big release. 2) There are a lot of long emails on the dev list describing problems - this is excellent - go testers, RAH, RAH!! 3) I want to ensure is that we retain *all* the details - if things get lost, a real user out there is going to find it and get annoyed. 4) We need to use the SF tracking system for bugs and feature requests - I've been making requests, but I'm going to make this manditory - I'm not fixing any more problems unless they have bug ids. We need to coordinate what bugs exist, what bugs get fixed, and when they get fixed. Why enter bugs? -- I feel that email is really good to alert others that a problem exists and it's good for discussion about how to fix a problem, but it is very poor for tracking problems - these emails contain anywhere from a single bug to 20 bugs, and I want to make sure that we retain a record of their existence so that we can give them a priority and decide when to fix them (pre-beta, post-beta, pre-release, post-release). When we make a release, it is very, *very* important to be able to track what bugs were fixed in the release, and what the resolution was. For a first release like genex-2.0 it isn't so critical, but in all subsequent release it is *critical* users will want to know if their important bug got fixed and how, so we must begin tracking them know, rather than later so we get in the habit. So, I'll repeat - I'm not fixing another issue until it has a SF Bug id. How to enter bugs? -- To enter a bug: http://sourceforge.net/tracker/?func=add&group_id=16453&atid=116453 To enter a feature request: http://sourceforge.net/tracker/?func=add&group_id=16453&atid=366453 I often find having to point and click everythin really irritating so I wrote a command line tool for entering (and resolving) SF bugs. I also wrote a tool so that you can make an excel spreadsheet with bugs, and batch enter them. I will make these available to anyone that wants it. ... So entering bugs is really simple ... What is a bug and what isn't? -- The only question is what is a bug, what is a feature request, and what is user error. In the beginning it's not easy to tell what is what. If you're in doubt, send an email to the list and we can discuss it, and then I can say: 'ok, it's a bug, please file a bug report'. I will be *totally* thrilled if we get *too* many bug reports, so don't be worried about filing something as a bug. I can always mark it as invalid later on. My rule of thumb is that if it is something that doesn't yet exist in the system, it's a feature request. If it's something that already exists in the system, it's a bug. Don't worry about the 'Group', 'Priority', or 'Assigned To' menus - I'll handle that. The most import parts are: 1) Summary - keep it short but descriptive 2) Details - make sure it describes the problem in a way that someone else can reproduce it (give URL's for example). 3) Category - what part of the system is affected. Don't worry too much about the category, it can always be changed. Here's the current list of categories and what they mean: * Administrative Apps - anything that lives in Perl/scripts * Analysis Tools - (we don't have any yet) * Bio::Genex Perl API - Something with Perl/Bio-Genex, or Perl/Class-ObjectTemplate, or Perl/Class-ObjectTemplate-DB, or Perl/Bio-MAGE. * DB Schema - something is not stored properly in the DB (e.g. some table needs an extra data column, or a foreign key to another table) * Documentation - something wrong for docs that already exists (use a feature request for docs that need to be added) * Installer - something that goes wrong during installation * Mason GUI - anything using the local WWW pages * Sample Data - the sample data loaded during DB installation * SourceForge WWW - http://genex.sf.net/ If you think there needs to be more, or we need to change these, let me know. Cheers, jas. |
From: SourceForge.net <no...@so...> - 2004-04-30 08:15:38
|
Bugs item #945131, was opened at 2004-04-30 02:15 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=945131&group_id=16453 Category: Mason GUI Group: Genex-2 Status: Open Resolution: None Priority: 8 Submitted By: Jason E. Stewart (jason_e_stewart) Assigned to: Jason E. Stewart (jason_e_stewart) Summary: bad URL for pba-factorvalue-insert.html Initial Comment: The navbar entry for Insert FactorValue for PBA in the 'Data Annotation' navbox in 'Annotate' tab is wrong - the url-base is edit/ and it should be insert/ ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=945131&group_id=16453 |
From: Harry J M. <hj...@ta...> - 2004-04-30 03:11:00
|
From: http://bodi.tacgi.com/genex/mason/login/annotate/annotate.html hitting the Insert -> FactorValues for PBA I get: ERROR: Page: edit/pba-factorvalue-insert.html not found on top of a table that shows the experiments that I've entered. I've also attached a slightly modified version of the DataLoading html doc. It's far from finished, but I include so you can see the approach - the copyright/version stuff has been moved to the bottom and the internal breakup has been changed a bit - I think Jason did it in emacs which probably parses it into stanzas and therefore doesn't care about actual text formatting, so I've broken it up a bit so that it's easier to look at in a text editor. The last bit I include for now is a starter glossary for GeneX for y'all to correct, validate, or extend. Here are some glossary definitions I've come up with. Jason, could you correct or approve them. MY idea was that for each of the itms on the Annotation tab::'Insert' menu, we have the items show up as now, except with an appended: (#) Def where: (#) = the number of entries currently in the DB Def = a short definition of what the items are and how they relate to each other. The #s come from a DB query at the page load (unless it takes too much time) and the Defs are as below. Also included are prerequisites of type (other than self) that must be filled in beforehand. If a prerequisite can only be selected from a pulldown and the user cannot add it himself (if in a predefined pulldown), it's omitted. Experiment Def: an EXPERIMENT is the name given to a set of related arrays, normalizations, and analyses that purports to describe an entire experimental procedure. For example, an experiment could be the 20 control and treatment arrays, the numerical transforms, and the analyses that show that water deprivation causes a specific set of gene expression changes in the meristematic tissues of sunflower shoots. It could also be used to describe a much larger set of arrays - if a group was profiling the same treatment on 50 tissues instead of just one. Prerequisites: Data Provider of type Person or Organization (optional) Citation Person Def: a PERSON is, strangely enough, what it sounds like - a human individual that needs to be identified to the GeneX system - it can be a person who provided a cell line, a sales rep, a PI, a contracts staffer overseeing a related grant, a sysadmin in charge of the data flow machines, or (unlikely), George W Bush. A User (see below) must be a PERSON, but a PERSON doesn't have to be a User (and usually isn't, certainly not George W Bush). Prerequisites: Affiliated Organization of type Organization (optional) User (not in the Insert menu just yet, but maybe soon...?) Def: a USER is a user of the GeneX database system, much like a user on a UNix system. It has certain restrictions and abilities to read and write data to and from the database. USERs can have at least 3 levels of administrative power - in order of increasing GeneX permissions, they are: a normal user, a PI (Primary Investigator) and a GeneX administrator (aka Curator). Prerequisites: Person Group Def: a GROUP is one of the few descriptive names in GeneX that is close to the normal English usage (in the Unix sense anyway) - a group of people who need to have the same read and write permissions to the data. A GROUP is made up of a number of PERSONS. (HOWEVER from the <<GeneX DB GroupSec Edit Page>>, it seems that the way you make groups is that you create a new group out of other groups. Is this right or do you make them de novo?) Organization Def: an organization is the name applied to any formal institution (government, academic, non-profit, or commercial) that can contribute data, material, services, personnel, etc to the GeneX DB. The National Cancer Institute is an ORGANIZATION, as is Vanderbilt University, or Affymetrix. ORGANIZATIONS can also be hierarchical in that a Department of Vanderbilt could be an ORGANIZATION as well as Vanderbilt itself. Prerequisites: Parent Organization (if required) ExperimentalFactor (vs ExperimentFactor?) Def: this I don't know - what is this and how does it differ from ExperimentFactor? I'm guessing here - an ExperimentalFactor is the primary treatment that is used to characterize an EXPERIMENT? An example would be that in an experiment used to examine the growth characteristics of treating a cell culture with a steroid, the steroid would be the EXPERIMENTALFACTOR. Prerequisites: Experiment Set Citation Def: a citation is the formal research publication that describes the EXPERIMENT entered here. It does not have to be a refereed publication, but can also be a technical note, a white paper, or a methods publication. Prerequisites: none, but lots of per-Citation info. BioMaterial Def: a mushy MAGE term that means exactly what you want it to mean, but in essence it is used to describe the biological material contributing the RNA that is used in the hybridization reaction. It includes BioSource (the source of the BioMaterial such as a cell line, tissue sample, or entire organism), the BioSample (an intermediate preparation step such as dissection or filtration), and LabelledExtract (the tagged cDNA that is applied to the chip or other substrate). Prerequisites: Data Provider of type Person or Organization (optional) ExperimentFactor Def: I dunno - see above. PBA (should be renamed PhysicalBioAssay) Def: again, I dunno for sure, but it seems that it refers to the general chip line or product ID or expression technology that is used to quantitate the assay or the organzation that provided the dataset. Prerequisites: Measured BioAssay (optional) Data Provider of type Person or Organization (optional) Array (required) Array Def: refers to the specific parameters on a per-chip or per-assay basis, including manufacturer, spot concentration, coating, spotter hardware, medium, etc Prerequisites: Spotter Software of type Software (optional) Provider Contact of type Person or Organization (optional) Array Manufacture (required) Array Design (required) Spotter Hardware of type Hardware (optional) Coating (optional) Technology Type (required, but only selectable) Medium (optional, selectable) Default Spot Concentration Units (optional) ArrayManufacture Def: the date and manufacturers name for backtrack and quality control purposes. Prerequisites: Manufacturer Contact of type Person or Orgazation (optional) Sample (how does this differ from BioMaterial?_ Def: another descriptor for the bioThingie from which you derived the cDNA for the hybridization. Requires a Data Provider (Person or Org) Prerequisites: Data Provider of type Person or Organization (optional) Taxon (optional) FactorValues for PBA - ???? (and page comes up with an error) -- Cheers, Harry Harry J Mangalam - 949 856 2847 (v&f) - hj...@ta... <<plain text preferred>> |
From: <ja...@op...> - 2004-04-29 12:18:19
|
Hey All, The installer now creates a test experiment and loads three test data matrices into it. I only had it load the very short (20 Features) artifical data set (just so I could test out the code). Once we settle on a better test data set to load, I can switch to loading it instead. You can see it on genex2: http://genex2.binf.gmu.edu/genex/mason/login/workspace/workspace.html login as user 'test' with the same password. Cheers, jas. |
From: <jw...@gm...> - 2004-04-29 03:24:54
|
Hi All, Karen Schlauch and Brandon Higgs (GMU) have been working on various data processing scripts (in R) for the analysis toolkit. The scripts have been documented and tested, and in the spirit of transparency we would like them to be available for inspection. Where should they be checked in, and how should we make this branch distinct and obvious to interested developers and users? We could post the location in the genex-users forum, which currently does not get much action. I guess I am thinking a site map of some kind. Suggestions? Cheers, Jennifer |
From: Muhammad <muh...@si...> - 2004-04-27 20:28:12
|
Hallo, I want to insert Gene experiment data into GeneX Database. for this purpose, i have choosen a small schema. As attechment you will find choosed schema, where i am intended to insert data. We have gene experiment data in two flate files. The first file contains clinical information and the scecond spots information. In first file, we have four following clinical properties. 1. sample name 2. Translocation 3. Treatment type 4. Outcome information like following formate: +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Sample name translocateion type outcome BCR-ABL-#1 BCR-ABL T13A Censored BCR-ABL-#2 BCR-ABL T13B Censored BCR-ABL-#3 BCR-ABL T13B Censored BCR-ABL-#4 BCR-ABL T11 NA BCR-ABL-#5 BCR-ABL T12 NA +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ And in the other flate file, we have following spots information. 4. spot_name 5. Normalized intesity (expression) of on a spot with its belonging Sample_name. like following formate: +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Gene_name BCR.ABL..1 BCR.ABL..2 BCR.ABL..3 BCR.ABL..4 BCR.ABL..5 1000_at 1.44316890296237 1.34035075057766 1.48303546191075 1.46672518104584 1. 48830338653164 1001_at 0.978063915142567 1.08129954743597 1.09047133840959 0.966733966954692 0. 92340706624928 1002_f_at 0.888879088423292 0.891881799999218 0.79746788484822 0. 95309567091632 0.785177495990561 1003_s_at 1.22144402768475 1.27740380645519 1.14955589812956 1.28440406219124 1. 17954677735482 1004_at 1.19813226946593 1.11859689741790 1.11275118624824 1.16928072863073 1. 11182581513464 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ I am intended to insert data as follow: -"Sample name" and "Translocation" will be insert into Title Attribute of table Protocol. -In type Attribute will be insert treatment type. -In Text Attribute will insert the values of "Sample_id" and "translocation". -Gene_id will insert into "spot identifier" Attribute of table AL_Spots. -Intensity values will insert into "spot_value" Attribute of table AM_Spot. -ArrayLayout table will contain the name of Array in "name" Attribute. -"Comment_prot" of Sample table will contain the "outcome" values e.g Censored. i want to know, is the choosed DB-Schema suitable for inserting the above data? or i should consider also other tables or other links table? I will be thank full for your kindly reply and worthfull comments. With best regards, Muhammad Ghiyas |
From: Muhammad <muh...@si...> - 2004-04-27 20:24:51
|
Hallo, I want to insert Gene experiment data into GeneX Database. for this purpose, i have choosen a small schema. As attechment you will find choosed schema, where i am intended to insert data. We have gene experiment data in two flate files. The first file contains clinical information and the scecond spots information. In first file, we have four following clinical properties. 1. sample name 2. Translocation 3. Treatment type 4. Outcome information like following formate: +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Sample name translocateion type outcome BCR-ABL-#1 BCR-ABL T13A Censored BCR-ABL-#2 BCR-ABL T13B Censored BCR-ABL-#3 BCR-ABL T13B Censored BCR-ABL-#4 BCR-ABL T11 NA BCR-ABL-#5 BCR-ABL T12 NA +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ And in the other flate file, we have following spots information. 4. spot_name 5. Normalized intesity (expression) of on a spot with its belonging Sample_name. like following formate: +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Gene_name BCR.ABL..1 BCR.ABL..2 BCR.ABL..3 BCR.ABL..4 BCR.ABL..5 1000_at 1.44316890296237 1.34035075057766 1.48303546191075 1.46672518104584 1. 48830338653164 1001_at 0.978063915142567 1.08129954743597 1.09047133840959 0.966733966954692 0. 92340706624928 1002_f_at 0.888879088423292 0.891881799999218 0.79746788484822 0. 95309567091632 0.785177495990561 1003_s_at 1.22144402768475 1.27740380645519 1.14955589812956 1.28440406219124 1. 17954677735482 1004_at 1.19813226946593 1.11859689741790 1.11275118624824 1.16928072863073 1. 11182581513464 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ I am intended to insert data as follow: -"Sample name" and "Translocation" will be insert into Title Attribute of table Protocol. -In type Attribute will be insert treatment type. -In Text Attribute will insert the values of "Sample_id" and "translocation". -Gene_id will insert into "spot identifier" Attribute of table AL_Spots. -Intensity values will insert into "spot_value" Attribute of table AM_Spot. -ArrayLayout table will contain the name of Array in "name" Attribute. -"Comment_prot" of Sample table will contain the "outcome" values e.g Censored. i want to know, is the choosed DB-Schema suitable for inserting the above data? or i should consider also other tables or other links table? I will be thank full for your kindly reply and worthfull comments. With best regards, Muhammad Ghiyas =================================================================== EASY and FREE access to your email anywhere: http://Mailreader.com/ =================================================================== |
From: <ja...@op...> - 2004-04-27 16:33:49
|
Hey All, Here is the list of sample data that Jennifer asked for. Some of these files were recently added to SVN, so do an 'svn update' to ensure all file exist. QT Dimension files: # these were created by Brandon King for the Caltech data DB/curated_data/agilent-5.1.1.xml DB/curated_data/genexpix-gpr-1.0.xml # these are example files created by me - all but qtdim-cel.xml # are loaded during the installation process DB/curated_data/qtdim-cel.xml DB/curated_data/qtdim-mas4.xml DB/curated_data/qtdim-mas5-short.xml DB/curated_data/qtdim-mas5.xml DB/curated_data/quantarray-jcush.xml DB/curated_data/quantarray.xml ArrayDesign files: DB/curated_data/200-AD-short.xml DB/curated_data/JCush-AD.xml DB/curated_data/1200-mangle-AD.txt - this is a AD precursor file DB/curated_data/1_3-A-AD.xml.gz - these are big files, so they are compressed DB/curated_data/1_3-B-AD.xml.gz DB/curated_data/YE6100.xml.gz Data Files: DB/curated_data/1200-mangle.txt DB/curated_data/200-short.txt DB/curated_data/HS_120.1.txt DB/curated_data/Cushman1.time0h.txt Data Sets: -- 1) Artificial test data - this is a tiny set with one chip that has 20 spots. I use this to test data loading * ArrayDesign: DB/curated_data/200-AD-short.xml * FE Software: quantarry.xml - loaded during installation, choose 'QuantArray:3.0' from the drop-down menu * Data Files: DB/curated_data/200-short.txt 2) Yease Heat Shock data - there are more data files on genex2 not in SVN * ArrayDesign: DB/curated_data/YE6100.xml.gz (gunzip this file before loading) * FE Software: qtdim-mas4.xml - loaded during installation, choose 'MicroArraySuite:4.0' from the drop-down menu * Data Files: DB/curated_data/HS_120.1.txt 3) JCushman data - the data file is out of sync with the AD file so loading the data file generates an error * Array Design: DB/curated_data/JCush-AD.xml * FE Software: quantarray-jcush.xml - loaded during installation, choose QuantArray:3.0.JCushman' from the drop-down menu * Data Files: DB/curated_data/Cushman1.time0h.txt 4) Artificial test data #2 - to be useful the tab-delimited ArrayDesign must first be converted to a MAGE-ML file using one of my convertors. * ArrayDesign: DB/curated_data/1200-mangle-AD.txt - this is a AD precursor file and it needs to be converted to a MAGE-ML file * FE Software: quantarray.xml - loaded during installation, choose QuantArray:3.0 from the drop-down menu * Data Files: DB/curated_data/1200-mangle.txt This is what we have for testing at the moment. Cheers, jas. |
From: <ja...@op...> - 2004-04-27 15:54:33
|
Hey All, After thinking about it for a few days, I decided upon a strategy for implementing the float => int mapping. I copied the generic security view code and special cased it for the scratch view float mapping. I hard-coded an INSERT rule to multiply all floats by a constant (set at 1000 for now) and to truncate to an integer using floor(). Then the SELECT rule does the opposite, it divides by the constant to give back the float. Here's an example - select directly from the Scratch table: genex2_test=# select * from genex_scratch; spot_identifier | dba_fk | view_num | float1 | float2 | float3 | float4 | int1 | int2 | int3 | int4 | bool1 | varchar1 -----------------+--------+----------+--------+--------+--------+--------+------+------+------+------+-------+---------- 971 | 1324 | 11 | 323484 | 130494 | | | | | | | | 973 | 1324 | 11 | 316664 | 127414 | | | | | | | | 975 | 1324 | 11 | 300468 | 149744 | | | | | | | | 977 | 1324 | 11 | 360255 | 160090 | | | | | | | | the float columns are actually stored as ints But if we do a select from the scratch view we actually get floats: genex2_test=# select * from jas3; dba_fk | spot_identifier | ch1_background | ch2_background --------+-----------------+----------------+---------------- 1324 | 971 | 130.494 | 323.484 1324 | 973 | 127.414 | 316.664 1324 | 975 | 149.744 | 300.468 I now need to make the floatX columns be stored as integers - the schema is still creating floating point columns. The results of this bit show that it is easy to do the low-level mapping I was talking about - but the drawback is there is no way around it. Any INSERT will *always* multiply the data and truncate the result. Any SELECT will always divide the number by 1000. Allowing users to define input or output protocols on the fly is much more flexible - but my only concern is that most users really don't care how the internals are stored - they just want it to be fast. And advertising all this float => int stuff is just going to irritate them. Cheers, jas. |
From: SourceForge.net <no...@so...> - 2004-04-27 06:23:49
|
Bugs item #942815, was opened at 2004-04-27 00:23 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=942815&group_id=16453 Category: Mason GUI Group: Genex-2 Status: Open Resolution: None Priority: 8 Submitted By: Jason E. Stewart (jason_e_stewart) Assigned to: Jason E. Stewart (jason_e_stewart) Summary: array-design-load.html should allow user to set name Initial Comment: The ArrayDesign loader currently sets the name using the MAGE identifier specifiec in the XML file. Since there is a unique constraint on the name of an ArrayDesign, the user should be able to over-ride this setting. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=942815&group_id=16453 |
From: SourceForge.net <no...@so...> - 2004-04-27 06:15:22
|
Bugs item #942793, was opened at 2004-04-26 23:23 Message generated for change (Comment added) made by jason_e_stewart You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=942793&group_id=16453 Category: Administrative Apps Group: Genex-2 >Status: Closed >Resolution: Fixed Priority: 8 Submitted By: Jason E. Stewart (jason_e_stewart) Assigned to: Jason E. Stewart (jason_e_stewart) Summary: eliminate warnings on array-design-load.pl Initial Comment: The program spews huge amounts of warnings about not finding location for feature or not finding zone for feature for every spot on the chip. ---------------------------------------------------------------------- >Comment By: Jason E. Stewart (jason_e_stewart) Date: 2004-04-27 00:15 Message: Logged In: YES user_id=85550 These warnings are only emitted if --debug is set. Committed to r1149 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=942793&group_id=16453 |
From: SourceForge.net <no...@so...> - 2004-04-27 05:46:19
|
Bugs item #942796, was opened at 2004-04-26 23:29 Message generated for change (Settings changed) made by jason_e_stewart You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=942796&group_id=16453 Category: Mason GUI Group: Genex-2 >Status: Closed >Resolution: Fixed Priority: 9 Submitted By: Jason E. Stewart (jason_e_stewart) Assigned to: Jason E. Stewart (jason_e_stewart) Summary: ERROR: Page: qtdim-load.html:title not found Initial Comment: > Loading qt-dim file > I copied the qtdim-mas5-short.xml file to the upload dir and could see it as > one of the choices, but when I tried to load it, I got the following error: > ERROR: Page: qtdim-load.html:title not found > positioned above the expts that I had already loaded > does this error mean that it really failed or is this more like a compiler > warning? This is a wierd Mason problem (can't call it a bug). For all the <form>'s, I used to hard-code that 'action' URL, and so whenever I had to move things around, I had to go and change all the URL's - this gets to be a pain. So I switched to using some of the Mason internals that in turn use the mod_perl internals to tell you what the current component path is. This helped a lot. Unfortunately, I just discovered that Mason seems to be adding extra information to the component path, so instead of 'qtdim-load.html' you get 'qtdim-load.html:title' ... because the qtdim-load.html component defines a 'title' method??!!? So I have to use a different Mason approach that doesn't add this extra info to the path. ---------------------------------------------------------------------- Comment By: Jason E. Stewart (jason_e_stewart) Date: 2004-04-26 23:45 Message: Logged In: YES user_id=85550 The problem was that $action was located in a <%shared> section. These sections are aparently run within the context of subcomponents, and since qtdim-load.html has a subcomponent (the method 'title') it was showing that as it's path. To fix it I moved the initialization of $action to <%init> ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=942796&group_id=16453 |
From: SourceForge.net <no...@so...> - 2004-04-27 05:45:39
|
Bugs item #942796, was opened at 2004-04-26 23:29 Message generated for change (Comment added) made by jason_e_stewart You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=942796&group_id=16453 Category: Mason GUI Group: Genex-2 Status: Open Resolution: None Priority: 9 Submitted By: Jason E. Stewart (jason_e_stewart) Assigned to: Jason E. Stewart (jason_e_stewart) Summary: ERROR: Page: qtdim-load.html:title not found Initial Comment: > Loading qt-dim file > I copied the qtdim-mas5-short.xml file to the upload dir and could see it as > one of the choices, but when I tried to load it, I got the following error: > ERROR: Page: qtdim-load.html:title not found > positioned above the expts that I had already loaded > does this error mean that it really failed or is this more like a compiler > warning? This is a wierd Mason problem (can't call it a bug). For all the <form>'s, I used to hard-code that 'action' URL, and so whenever I had to move things around, I had to go and change all the URL's - this gets to be a pain. So I switched to using some of the Mason internals that in turn use the mod_perl internals to tell you what the current component path is. This helped a lot. Unfortunately, I just discovered that Mason seems to be adding extra information to the component path, so instead of 'qtdim-load.html' you get 'qtdim-load.html:title' ... because the qtdim-load.html component defines a 'title' method??!!? So I have to use a different Mason approach that doesn't add this extra info to the path. ---------------------------------------------------------------------- >Comment By: Jason E. Stewart (jason_e_stewart) Date: 2004-04-26 23:45 Message: Logged In: YES user_id=85550 The problem was that $action was located in a <%shared> section. These sections are aparently run within the context of subcomponents, and since qtdim-load.html has a subcomponent (the method 'title') it was showing that as it's path. To fix it I moved the initialization of $action to <%init> ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=942796&group_id=16453 |
From: SourceForge.net <no...@so...> - 2004-04-27 05:29:17
|
Bugs item #942796, was opened at 2004-04-26 23:29 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=942796&group_id=16453 Category: Mason GUI Group: Genex-2 Status: Open Resolution: None Priority: 9 Submitted By: Jason E. Stewart (jason_e_stewart) Assigned to: Jason E. Stewart (jason_e_stewart) Summary: ERROR: Page: qtdim-load.html:title not found Initial Comment: > Loading qt-dim file > I copied the qtdim-mas5-short.xml file to the upload dir and could see it as > one of the choices, but when I tried to load it, I got the following error: > ERROR: Page: qtdim-load.html:title not found > positioned above the expts that I had already loaded > does this error mean that it really failed or is this more like a compiler > warning? This is a wierd Mason problem (can't call it a bug). For all the <form>'s, I used to hard-code that 'action' URL, and so whenever I had to move things around, I had to go and change all the URL's - this gets to be a pain. So I switched to using some of the Mason internals that in turn use the mod_perl internals to tell you what the current component path is. This helped a lot. Unfortunately, I just discovered that Mason seems to be adding extra information to the component path, so instead of 'qtdim-load.html' you get 'qtdim-load.html:title' ... because the qtdim-load.html component defines a 'title' method??!!? So I have to use a different Mason approach that doesn't add this extra info to the path. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=942796&group_id=16453 |