Menu
▾
▴
genex-dev — GeneX developers
You can subscribe to this list here.
| 2001 |
Jan
(135) |
Feb
(57) |
Mar
(84) |
Apr
(43) |
May
(77) |
Jun
(51) |
Jul
(21) |
Aug
(55) |
Sep
(37) |
Oct
(56) |
Nov
(75) |
Dec
(23) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2002 |
Jan
(32) |
Feb
(174) |
Mar
(121) |
Apr
(70) |
May
(55) |
Jun
(20) |
Jul
(23) |
Aug
(15) |
Sep
(12) |
Oct
(58) |
Nov
(203) |
Dec
(90) |
| 2003 |
Jan
(37) |
Feb
(15) |
Mar
(14) |
Apr
(57) |
May
(7) |
Jun
(40) |
Jul
(36) |
Aug
(1) |
Sep
(56) |
Oct
(38) |
Nov
(105) |
Dec
(2) |
| 2004 |
Jan
|
Feb
(117) |
Mar
(69) |
Apr
(160) |
May
(165) |
Jun
(35) |
Jul
(7) |
Aug
(80) |
Sep
(47) |
Oct
(23) |
Nov
(8) |
Dec
(42) |
| 2005 |
Jan
(19) |
Feb
(2) |
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2007 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
(2) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2010 |
Jan
|
Feb
(2) |
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: Tom L. <tw...@vi...> - 2004-05-17 15:26:14
|
Jason, Todd, We're chaning our name to Gene Expression Open Source System or GEOSS to avoid confusion. Jason, I've got a description of Genex-Lite from Todd. If you send me one for GeneX 2, I'll add that (and a link) to our "background" page. Cheers! Tom Todd F. Peterson wrote: > We still do have a version at NCGR. > > Todd > > On Mon, 17 May 2004, Jason E. Stewart wrote: > > >>Hey All, >> >>I did a google search on genex and found a number of pages that are >>still referring to the project as part of NCGR. So I wrote to about >>five of the webmasters and let them know of the SF site. They have all >>written back thanking me for the update, and have said they have >>already or the will update their pages. >> >>Cheers, >>jas. >> >> >>------------------------------------------------------- >>This SF.Net email is sponsored by: SourceForge.net Broadband >>Sign-up now for SourceForge Broadband and get the fastest >>6.0/768 connection for only $19.95/mo for the first 3 months! >>http://ads.osdn.com/?ad_id=2562&alloc_id=6184&op=click >>_______________________________________________ >>Genex-dev mailing list >>Gen...@li... >>https://lists.sourceforge.net/lists/listinfo/genex-dev >> > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: SourceForge.net Broadband > Sign-up now for SourceForge Broadband and get the fastest > 6.0/768 connection for only $19.95/mo for the first 3 months! > http://ads.osdn.com/?ad_id=2562&alloc_id=6184&op=click > _______________________________________________ > Genex-dev mailing list > Gen...@li... > https://lists.sourceforge.net/lists/listinfo/genex-dev > -- Tom Laudeman mailto:tw...@vi... (434) 924-2456 http://www.people.virginia.edu/~twl8n/ http://laudeman.com/ |
|
From: Todd F. P. <tf...@nc...> - 2004-05-17 14:37:28
|
We still do have a version at NCGR. Todd On Mon, 17 May 2004, Jason E. Stewart wrote: > Hey All, > > I did a google search on genex and found a number of pages that are > still referring to the project as part of NCGR. So I wrote to about > five of the webmasters and let them know of the SF site. They have all > written back thanking me for the update, and have said they have > already or the will update their pages. > > Cheers, > jas. > > > ------------------------------------------------------- > This SF.Net email is sponsored by: SourceForge.net Broadband > Sign-up now for SourceForge Broadband and get the fastest > 6.0/768 connection for only $19.95/mo for the first 3 months! > http://ads.osdn.com/?ad_id=2562&alloc_id=6184&op=click > _______________________________________________ > Genex-dev mailing list > Gen...@li... > https://lists.sourceforge.net/lists/listinfo/genex-dev > |
|
From: SourceForge.net <no...@so...> - 2004-05-17 06:35:16
|
Bugs item #955104, was opened at 2004-05-17 00:35 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=955104&group_id=16453 Category: Mason GUI Group: Genex-2 Status: Open Resolution: None Priority: 7 Submitted By: Jason E. Stewart (jason_e_stewart) Assigned to: Jason E. Stewart (jason_e_stewart) Summary: objs2table.mason: OTM fkeys should fetch count only Initial Comment: currently this component is doing a full DB lookup for all rows in all OTM fkeys. With an empty DB this is doable, but later it will be slow and useless. Instead, all that is needed is the number of rows to be fetched. This can be done simply after fixing get_all_objects() to return a row count in a scalar context. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=955104&group_id=16453 |
|
From: <ja...@op...> - 2004-05-17 04:18:15
|
Hey All, I did a google search on genex and found a number of pages that are still referring to the project as part of NCGR. So I wrote to about five of the webmasters and let them know of the SF site. They have all written back thanking me for the update, and have said they have already or the will update their pages. Cheers, jas. |
|
From: <ja...@op...> - 2004-05-16 16:27:03
|
Hey All, Just a heads up. I've finished the final schema pieces that were remaining: * ControlledVocab => OntologyEntry * Image support * few minor pieces My network here keeps going up and down, so I haven't been able to commit the changes. I'll do so as soon as I'm able. I'd like for us to test this for a few days in any way we can, and see if there are any remaining pieces we need to add, and if not, we set a schema freeze and go into intensive bug testing and release a beta soon after. I'll send out a message tomorrow with a proposed testing plan that we can kick around. Cheers, jas. |
|
From: SourceForge.net <no...@so...> - 2004-05-16 06:29:01
|
Bugs item #953954, was opened at 2004-05-14 07:15 Message generated for change (Comment added) made by jason_e_stewart You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=953954&group_id=16453 Category: Mason GUI Group: Genex-2 Status: Open Resolution: None Priority: 8 Submitted By: Jason E. Stewart (jason_e_stewart) Assigned to: Jason E. Stewart (jason_e_stewart) Summary: analyze-experiment.html cannot export data Initial Comment: mason error: Error during compilation of /var/www/genex/mason/login/workspace/comps/export-bad-tab-file.mason: Global symbol "$data_table" requires explicit package name at /var/www/genex/mason/login/workspace/comps/export-bad-tab-file.mason line 47. ---------------------------------------------------------------------- >Comment By: Jason E. Stewart (jason_e_stewart) Date: 2004-05-16 00:29 Message: Logged In: YES user_id=85550 export of MBA data is now working, but exporting of DBA data is still broken ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=953954&group_id=16453 |
|
From: SourceForge.net <no...@so...> - 2004-05-16 06:16:15
|
Bugs item #818257, was opened at 2003-10-05 13:47 Message generated for change (Comment added) made by jason_e_stewart You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=818257&group_id=16453 Category: Administrative Apps Group: Genex-2 Status: Open Resolution: None Priority: 7 Submitted By: Jason E. Stewart (jason_e_stewart) Assigned to: Jason E. Stewart (jason_e_stewart) Summary: certain tables should not have security views Initial Comment: The Scratch table should not have a view. Instead it should only have specific data views with lookup table security. ---------------------------------------------------------------------- >Comment By: Jason E. Stewart (jason_e_stewart) Date: 2004-05-16 00:16 Message: Logged In: YES user_id=85550 This is false. The scratch table requires a security view in order to handle the lookup table security for the other views. However, other tables, such as the abstract tables, need better view handling. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=818257&group_id=16453 |
|
From: <ja...@op...> - 2004-05-15 17:02:49
|
ja...@op... (Jason E. Stewart) writes: > I did notice one (bug < feature) thing tho. Mason enforces it's own >styles on referenced docs so the miame_glossary that this URL now >points to is stripped of the borders, making it a bit difficult to >read. Can we give all tables a visible borders of '1' ? It's not Mason - it's the rather elaborate set of CSS stylesheets that I stole/borrowed from the style.tigris.org project. They are a bit of a mixed blessing. It's incredible the power they have stored in them, but actually dissecting them and understanding all their complexity is a bit of a pain. So the short answer is 'no' - it would be bad for *all* tables to have visible borders - there are a lot of tables, and most want to stay hidden. One thing to do is surround the glossary with a <div class="give-me-borders-or-give-me-death"> tag, and set a style for that class to always have borders. Cheers, jas. |
|
From: <ja...@op...> - 2004-05-15 16:58:32
|
ja...@op... (Jason E. Stewart) writes:
>> You can force it to use this by creating a 2-level index:
>>
>> create index fk_int2_idx on scratch(dba_fk, int2)
>
> Yup, that seems to work.
Spoke too soon. It only uses the index of the number of rows returned
is small:
genex2=# explain select count(*) from genex_scratch where dba_fk=966 and int2>1000000 and int3>50;
QUERY PLAN
-----------------------------------------------------------------------------
Aggregate (cost=59562.05..59562.05 rows=1 width=0)
-> Seq Scan on genex_scratch (cost=0.00..56836.96 rows=1090035 width=0)
Filter: ((dba_fk = 966) AND (int2 > 1000000) AND (int3 > 50))
(3 rows)
However, if I force the query planner to use the index it does seem to
speed things up by about 50%
genex2=# set enable_seqscan to off;
SET
Time: 1.49 ms
genex2=# explain select count(*) from genex_scratch where dba_fk=966 and int2>1000000 and int3>int2;
QUERY PLAN
-------------------------------------------------------------------------------------------------------
Aggregate (cost=3691049.01..3691049.01 rows=1 width=0)
-> Index Scan using fk_int2_int3_idx on genex_scratch (cost=0.00..3690140.56 rows=363382 width=0)
Index Cond: ((dba_fk = 966) AND (int2 > 1000000))
Filter: (int3 > int2)
(4 rows)
Time: 3.97 ms
genex2=# select count(*) from genex_scratch where dba_fk=966 and int2>1000000 and int3>int2;
count
-------
0
(1 row)
Time: 8980.08 ms
genex2=# set enable_seqscan to on;
SET
Time: 6.49 ms
genex2=# explain select count(*) from genex_scratch where dba_fk=966 and int2>1000000 and int3>int2;
QUERY PLAN
----------------------------------------------------------------------------
Aggregate (cost=57745.42..57745.42 rows=1 width=0)
-> Seq Scan on genex_scratch (cost=0.00..56836.96 rows=363382 width=0)
Filter: ((dba_fk = 966) AND (int2 > 1000000) AND (int3 > int2))
(3 rows)
Time: 3.17 ms
genex2=# select count(*) from genex_scratch where dba_fk=966 and int2>1000000 and int3>int2;
count
-------
0
(1 row)
Time: 12441.70 ms
So that's 8.9s with the index and 12.4s doing the sequential scan. Not
sure what to say...
Cheers,
jas.
|
|
From: <ja...@op...> - 2004-05-15 16:35:59
|
yeah - sorry - my mistake for not checking it in correctly. Just checking it in now. I did notice one (bug < feature) thing tho. Mason enforces it's own styles on referenced docs so the miame_glossary that this URL now points to is stripped of the borders, making it a bit difficult to read. Can we give all tables a visible borders of '1' ? hjm Jason E. Stewart wrote: > Hey Harry, > > I'm still getting errors on make install: > > Handling /var/www/genex2/mason/nologin/docs/templates > docs/Security.txt > docs/Array-model.txt > cp: cannot stat `docs/Array-model.txt': No such file or directory > > !! System Error: No such file or directory @ line: 286 (cp docs/Array-model.txt /var/www/genex2/mason/nologin/docs/templates) > > Cheers, > jas. > > > ------------------------------------------------------- > This SF.Net email is sponsored by: SourceForge.net Broadband > Sign-up now for SourceForge Broadband and get the fastest > 6.0/768 connection for only $19.95/mo for the first 3 months! > http://ads.osdn.com/?ad_id=2562&alloc_id=6184&op=click > _______________________________________________ > Genex-dev mailing list > Gen...@li... > https://lists.sourceforge.net/lists/listinfo/genex-dev > -- Cheers, Harry Harry J Mangalam - 949 856 2847 (v&f) - hj...@ta... <<plain text preferred>> |
|
From: <ja...@op...> - 2004-05-15 16:29:01
|
Harry Mangalam <hj...@ta...> writes: > but maybe it needs the 2-level index: > > > You can force it to use this by creating a 2-level index: > > create index fk_int2_idx on scratch(dba_fk, int2) Yup, that seems to work. > and then query it, but I'll have to look at this in more detail - it > really should use both indices if they're available separately. You > can also cluster data so that identical values are grouped. I don't know about clusters. > Can I use this test DB to try out some things? Where is it? Send > info to me personally, if you don;t want others to hit it). I have it on my laptop. I just installed it on genex2, though so you can access it there now, too. > Incidentally, these were good, fast reads on indexing and improving > perf in postgres: > http://techdocs.postgresql.org/redir.php?link=/techdocs/pgsqladventuresep2.php > http://techdocs.postgresql.org/redir.php?link=/techdocs/pgsqladventuresep3.php > http://www.argudo.org/postgresql/soft-tuning.html Ah, useful! Thanks. Too bad the series of four articles stopped at 2, it was most illuminating. Cheers, jas. |
|
From: Harry M. <hj...@ta...> - 2004-05-14 21:27:52
|
Hi Jason, I think PG ought to realize that there are 2 indices that it can and should use, but it may be that it's using the sequential scan b/c it's a complex query (a pretty simple one, but has more than 1 term). It looks like you notified pg that you had built the index vacuum analyze scratch but maybe it needs the 2-level index: You can force it to use this by creating a 2-level index: create index fk_int2_idx on scratch(dba_fk, int2) and then query it, but I'll have to look at this in more detail - it really should use both indices if they're available separately. You can also cluster data so that identical values are grouped. Can I use this test DB to try out some things? Where is it? Send info to me personally, if you don;t want others to hit it). Incidentally, these were good, fast reads on indexing and improving perf in postgres: http://techdocs.postgresql.org/redir.php?link=/techdocs/pgsqladventuresep2.php http://techdocs.postgresql.org/redir.php?link=/techdocs/pgsqladventuresep3.php http://www.argudo.org/postgresql/soft-tuning.html hjm Jason E. Stewart wrote: > Hey Harry, > > I'm testing performance on the scratch table. I made an application > that would insert arbitrary rows into the scratch table, and had it > insert 2,000,000 rows. > > I then created two indices, one on the dba_fk and another on one of > the data columns, int2. I did some queries and checked out the query > path using 'explain'. > > Then I ran 'vacuum analyze' to update the statistics and see if it > changed the query path. Here's my results: > > genex2=# explain select int2,int3 from genex_scratch where dba_fk=926 and int2>1000000; > QUERY PLAN > ------------------------------------------------------------------------------------------ > Index Scan using genex_scratch_dba_fk on genex_scratch (cost=0.00..2.02 rows=1 width=8) > Index Cond: (dba_fk = 926) > Filter: (int2 > 1000000) > (3 rows) > > This is a DBA with only 20 rows. In this case Pg uses the dba_fk > index. The next DBA, 966, has 2,000,000 rows and for this, Pg does not > use *any* of indices... > > genex2=# explain select count(*) from genex_scratch where dba_fk=966 and int2>1000000; > QUERY PLAN > ----------------------------------------------------------------------------- > Aggregate (cost=54046.01..54046.01 rows=1 width=0) > -> Seq Scan on genex_scratch (cost=0.00..51335.96 rows=1084021 width=0) > Filter: ((dba_fk = 966) AND (int2 > 1000000)) > > So, how do I get Pg to actually use the index? One of the reasons that > we switched the columns from floats to ints was to enable using > indices on the columns to speed up the queries. But if Pg isn't going > to actually use the indices we create, then that isn't so helpful. > > Perhaps there is a more complex query I should be using? > > Cheers, > jas. > > > ------------------------------------------------------- > This SF.Net email is sponsored by: SourceForge.net Broadband > Sign-up now for SourceForge Broadband and get the fastest > 6.0/768 connection for only $19.95/mo for the first 3 months! > http://ads.osdn.com/?ad_id=2562&alloc_id=6184&op=click > _______________________________________________ > Genex-dev mailing list > Gen...@li... > https://lists.sourceforge.net/lists/listinfo/genex-dev > -- Cheers, Harry Harry J Mangalam - 949 856 2847 (v&f) - hj...@ta... <<plain text preferred>> |
|
From: SourceForge.net <no...@so...> - 2004-05-14 13:24:04
|
Bugs item #953958, was opened at 2004-05-14 07:24 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=953958&group_id=16453 Category: Installer Group: Genex-2 Status: Open Resolution: None Priority: 7 Submitted By: Jason E. Stewart (jason_e_stewart) Assigned to: Harry Mangalam (mangalam) Summary: Errors on make install Initial Comment: Array-model.txt was removed but the entry in the MANIFEST.in is still there ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=953958&group_id=16453 |
|
From: <ja...@op...> - 2004-05-14 13:22:49
|
Hey Harry, I'm still getting errors on make install: Handling /var/www/genex2/mason/nologin/docs/templates docs/Security.txt docs/Array-model.txt cp: cannot stat `docs/Array-model.txt': No such file or directory !! System Error: No such file or directory @ line: 286 (cp docs/Array-model.txt /var/www/genex2/mason/nologin/docs/templates) Cheers, jas. |
|
From: SourceForge.net <no...@so...> - 2004-05-14 13:15:58
|
Bugs item #953954, was opened at 2004-05-14 07:15 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=953954&group_id=16453 Category: Mason GUI Group: Genex-2 Status: Open Resolution: None Priority: 8 Submitted By: Jason E. Stewart (jason_e_stewart) Assigned to: Jason E. Stewart (jason_e_stewart) Summary: analyze-experiment.html cannot export data Initial Comment: mason error: Error during compilation of /var/www/genex/mason/login/workspace/comps/export-bad-tab-file.mason: Global symbol "$data_table" requires explicit package name at /var/www/genex/mason/login/workspace/comps/export-bad-tab-file.mason line 47. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=953954&group_id=16453 |
|
From: <ja...@op...> - 2004-05-14 13:01:19
|
Hey Harry, I just moved some files so that there is fewer things in the top level: * Install => Perl/scripts * Configure => Perl/scripts * make-dist.pl => Perl/scripts * InstallUtils.pm => perl/lib * GenexInstall.pm => perl/lib I also deleted all the old CVS ChangeLog files. I Haven't been using these since we switched to SVN. This gives the following files with a fresh system: Bio/ G2G/ Java/ Mason/ R/ apache/ etc/ xml/ DB/ GeneX.config LICENSE Perl/ README* css/ graphics/ DTD/ INSTALL Makefile Python/ SQL/ docs/ include/ Pretty reasonable, don't you think? Cheers, jas. |
|
From: SourceForge.net <no...@so...> - 2004-05-14 08:19:14
|
Bugs item #953804, was opened at 2004-05-14 02:19 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=953804&group_id=16453 Category: Bio::Genex Perl API Group: Genex-2 Status: Open Resolution: None Priority: 9 Submitted By: Jason E. Stewart (jason_e_stewart) Assigned to: Jason E. Stewart (jason_e_stewart) Summary: Scratch table: inserting float fields into int colums Initial Comment: create_scratch_view_sql() is doing the wrong transformation. it is placing all the i_ columns into int fields and not float fields ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=953804&group_id=16453 |
|
From: SourceForge.net <no...@so...> - 2004-05-14 08:14:20
|
Bugs item #952386, was opened at 2004-05-12 00:44 Message generated for change (Settings changed) made by jason_e_stewart You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=952386&group_id=16453 Category: DB Schema Group: Genex-2 >Status: Closed >Resolution: Fixed Priority: 9 Submitted By: Jason E. Stewart (jason_e_stewart) Assigned to: Jason E. Stewart (jason_e_stewart) Summary: All data tables must store floats as ints Initial Comment: We need to track which columns are intended to hold floating point values. Step 1) make all floating point columns be float columns with an 'i_' as prefix, e.g. column 'foo' becomes 'i_foo'. Step 2) make a view on top of this that provides the original column, 'foo', as 'i_foo / 1000'. The issue with Step 2 is how to do it in such a way that genex_admin users don't create problems. ---------------------------------------------------------------------- Comment By: Jason E. Stewart (jason_e_stewart) Date: 2004-05-12 03:08 Message: Logged In: YES user_id=85550 stage 1) is complete. Modified qtdim-insert.pl to rename columns to 'i_foo' and create_scratch_view_sql() to remove the old multiplication attempt. Made system wide config variable GENEX_FLOAT_MULT ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=952386&group_id=16453 |
|
From: SourceForge.net <no...@so...> - 2004-05-14 08:14:20
|
Bugs item #848040, was opened at 2003-11-24 00:26 Message generated for change (Settings changed) made by jason_e_stewart You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=848040&group_id=16453 Category: DB Schema Group: Genex-2 >Status: Closed Resolution: Fixed Priority: 9 Submitted By: Nobody/Anonymous (nobody) Assigned to: Jason E. Stewart (jason_e_stewart) Summary: Scratch wants to have it's float fields turned into ints Initial Comment: Scratch wants to have it's float fields turned into ints - this is for indexing performance reasons (how many do we need? Just one I think) perhaps create a SELECT trigger so that the columns are returned correctly as floats? Probably not, since that would probably really confuse the DBI. ---------------------------------------------------------------------- Comment By: Jason E. Stewart (jason_e_stewart) Date: 2004-05-14 02:13 Message: Logged In: YES user_id=85550 ok all float columns are now ints ---------------------------------------------------------------------- Comment By: Jason E. Stewart (jason_e_stewart) Date: 2004-05-10 07:48 Message: Logged In: YES user_id=85550 Whoops! This didn't get fixed properly. It's back and this time the DB needs a major rehaul. We're going to store all data table floats as ints. The data loader needs to do the conversion from floats => ints at load time. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=848040&group_id=16453 |
|
From: SourceForge.net <no...@so...> - 2004-05-14 08:13:55
|
Bugs item #848040, was opened at 2003-11-24 00:26 Message generated for change (Comment added) made by jason_e_stewart You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=848040&group_id=16453 Category: DB Schema Group: Genex-2 Status: Open >Resolution: Fixed Priority: 9 Submitted By: Nobody/Anonymous (nobody) Assigned to: Jason E. Stewart (jason_e_stewart) Summary: Scratch wants to have it's float fields turned into ints Initial Comment: Scratch wants to have it's float fields turned into ints - this is for indexing performance reasons (how many do we need? Just one I think) perhaps create a SELECT trigger so that the columns are returned correctly as floats? Probably not, since that would probably really confuse the DBI. ---------------------------------------------------------------------- >Comment By: Jason E. Stewart (jason_e_stewart) Date: 2004-05-14 02:13 Message: Logged In: YES user_id=85550 ok all float columns are now ints ---------------------------------------------------------------------- Comment By: Jason E. Stewart (jason_e_stewart) Date: 2004-05-10 07:48 Message: Logged In: YES user_id=85550 Whoops! This didn't get fixed properly. It's back and this time the DB needs a major rehaul. We're going to store all data table floats as ints. The data loader needs to do the conversion from floats => ints at load time. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=116453&aid=848040&group_id=16453 |
|
From: <ja...@op...> - 2004-05-14 06:42:40
|
ja...@op... (Jason E. Stewart) writes: > From: Harry Mangalam <hj...@ta...> > Subject: Re: [GeneX-dev] ctid and oid now in views > To: "Jason E. Stewart" <ja...@op...> > Date: Thu, 13 May 2004 09:28:19 -0700 > > In reverse order of your emails - Yeehah! 'tworks!! We have views > in view! Excellent. The fix is a bit crufty, but now that the columns are in the schema we can modify how that API handles them later without fear of creating a total DB reload. Cheers, jas. |
|
From: <ja...@op...> - 2004-05-14 06:40:52
|
Hey Harry,
I'm testing performance on the scratch table. I made an application
that would insert arbitrary rows into the scratch table, and had it
insert 2,000,000 rows.
I then created two indices, one on the dba_fk and another on one of
the data columns, int2. I did some queries and checked out the query
path using 'explain'.
Then I ran 'vacuum analyze' to update the statistics and see if it
changed the query path. Here's my results:
genex2=# explain select int2,int3 from genex_scratch where dba_fk=926 and int2>1000000;
QUERY PLAN
------------------------------------------------------------------------------------------
Index Scan using genex_scratch_dba_fk on genex_scratch (cost=0.00..2.02 rows=1 width=8)
Index Cond: (dba_fk = 926)
Filter: (int2 > 1000000)
(3 rows)
This is a DBA with only 20 rows. In this case Pg uses the dba_fk
index. The next DBA, 966, has 2,000,000 rows and for this, Pg does not
use *any* of indices...
genex2=# explain select count(*) from genex_scratch where dba_fk=966 and int2>1000000;
QUERY PLAN
-----------------------------------------------------------------------------
Aggregate (cost=54046.01..54046.01 rows=1 width=0)
-> Seq Scan on genex_scratch (cost=0.00..51335.96 rows=1084021 width=0)
Filter: ((dba_fk = 966) AND (int2 > 1000000))
So, how do I get Pg to actually use the index? One of the reasons that
we switched the columns from floats to ints was to enable using
indices on the columns to speed up the queries. But if Pg isn't going
to actually use the indices we create, then that isn't so helpful.
Perhaps there is a more complex query I should be using?
Cheers,
jas.
|
|
From: <ja...@op...> - 2004-05-14 06:24:27
|
In reverse order of your emails - Yeehah! 'tworks!! We have views in view!
Excellent!
hjm
Jason E. Stewart wrote:
> Hey Harry,
>
> As of r1253 ctid and oid are selectable from security views.
>
> Please try hooking up OO.o and ensure that it actually works.
>
> Cheers,
> jas.
>
>
--
Cheers, Harry
Harry J Mangalam - 949 856 2847 (v&f) - hj...@ta...
<<plain text preferred>>
|
|
From: <ja...@op...> - 2004-05-14 03:24:47
|
Harry Mangalam <hj...@ta...> writes: > Corrected this & will keep this in mind for later stirring of the > docs pot. Ok, thanks. Just wanted to be clueful early on the process so that you understood how some of the bits were glued together. I find that running a 'make install' before I commit is critical - if the errors file comes up empty, I didn't break anything, so I can commit. Cheers, jas. |
|
From: <ja...@op...> - 2004-05-14 03:20:01
|
Harry Mangalam <hj...@ta...> writes: > Jason - I can see how files are copied via the MANIFEST.in -> > MANIFEST, but I'm not clear on how files make their way from > ...nologin/docs/templates (where the Array-model file was referenced) > to ...nologin/docs/ which is where the URL is pointing to. Obviously > my deletion of it in the doc/ dir caused the failure of the URL, but I > don't follow the process, as I didn't touch the reference to it in the > .../nologin/docs stanza in the MANIFEST.in file. > > Is there a 'copy-on-installation' that is triggered when you run the > make install? Just to be clear - once MANIFEST is created from MANIFEST.in the section for the docs directory looks like: '/var/www/genex2/mason/nologin/docs/templates' => [ 'docs/Security.txt', 'docs/Array-model.txt', 'docs/ArrayDesignCreation-HOWTO.html', 'docs/DataLoading-HOWTO.html', 'R/analysis/script.1factor.ANOVA.txt', 'R/analysis/script.2factor.ANOVA.txt', 'R/analysis/script.corr.heatmap.txt', 'R/analysis/script.mvr.txt', 'R/analysis/script.pca.txt', 'R/analysis/script.qqplot.txt', ], The hash key is the *destination* directory, and the files are the source files. All the source files will be copied into the destination directory on a 'make install_files'. > Aha! I see - you've wrapped the plain text file with <pre> tags, that > why it's called a template. this is sort of cool, but it also makes > changing docs a multi-stage gymnastic. Yes, I just wanted to start putting some docs up so there was something there. This was more of an illustration for you and others how we can add docs if we want to - i.e. if all we have is plain text, then it works. > Don't most browsers return text as plain text anyway? Mozilla and > Konqueror do. Yes, but look at the source of the WWW page you get back - there's a lot of HTML with a little bit of plain text. All the header and tabs and navbar and footer stuff is HTML, just the main panel is plain text. All the other stuff is handled automatically by a chain of Mason 'autohandlers': * /autohandler * /nologin/autohandler * /nologin/docs/autohandler That's the Mason component chain in action. The actual WWW page requested, array-model.html is the part that renders in the main window, all the other bits are handled by the autohandlers. HTH, jas. |
54 messages has been excluded from this view by a project administrator.
