Menu
▾
▴
lxr-developer — Don't use, active developers only
You can subscribe to this list here.
| 2001 |
Jan
|
Feb
|
Mar
|
Apr
|
May
(11) |
Jun
(21) |
Jul
(14) |
Aug
(83) |
Sep
(23) |
Oct
(37) |
Nov
(52) |
Dec
(10) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2002 |
Jan
(28) |
Feb
(40) |
Mar
(21) |
Apr
(8) |
May
(21) |
Jun
(13) |
Jul
(9) |
Aug
(5) |
Sep
(8) |
Oct
(7) |
Nov
(2) |
Dec
|
| 2003 |
Jan
(2) |
Feb
(1) |
Mar
(11) |
Apr
(4) |
May
(6) |
Jun
(15) |
Jul
(4) |
Aug
(4) |
Sep
(9) |
Oct
(1) |
Nov
(1) |
Dec
(1) |
| 2004 |
Jan
(4) |
Feb
|
Mar
(4) |
Apr
(12) |
May
(5) |
Jun
(9) |
Jul
(47) |
Aug
(1) |
Sep
(1) |
Oct
(7) |
Nov
|
Dec
(1) |
| 2005 |
Jan
(4) |
Feb
(2) |
Mar
(3) |
Apr
(10) |
May
(9) |
Jun
(15) |
Jul
(3) |
Aug
(1) |
Sep
(8) |
Oct
(9) |
Nov
(10) |
Dec
(4) |
| 2006 |
Jan
(1) |
Feb
|
Mar
(9) |
Apr
(5) |
May
(1) |
Jun
(6) |
Jul
(2) |
Aug
|
Sep
(5) |
Oct
(2) |
Nov
|
Dec
(3) |
| 2007 |
Jan
(2) |
Feb
(1) |
Mar
(32) |
Apr
(3) |
May
(3) |
Jun
(16) |
Jul
(1) |
Aug
|
Sep
|
Oct
(2) |
Nov
(4) |
Dec
(3) |
| 2008 |
Jan
|
Feb
(1) |
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2009 |
Jan
|
Feb
|
Mar
(46) |
Apr
(70) |
May
(15) |
Jun
(13) |
Jul
(1) |
Aug
|
Sep
(7) |
Oct
|
Nov
|
Dec
|
| 2010 |
Jan
(5) |
Feb
(4) |
Mar
|
Apr
|
May
(2) |
Jun
(1) |
Jul
(1) |
Aug
|
Sep
|
Oct
(7) |
Nov
(6) |
Dec
|
| 2011 |
Jan
(1) |
Feb
|
Mar
(85) |
Apr
(18) |
May
(4) |
Jun
(3) |
Jul
(4) |
Aug
(1) |
Sep
|
Oct
(2) |
Nov
(2) |
Dec
(20) |
| 2012 |
Jan
(17) |
Feb
(16) |
Mar
(13) |
Apr
(18) |
May
|
Jun
(6) |
Jul
(6) |
Aug
(10) |
Sep
(15) |
Oct
(10) |
Nov
(25) |
Dec
(1) |
|
From: Arne G. G. <ar...@gl...> - 2007-06-18 14:46:00
|
Paul Smith wrote: > I would love to do some more work on LXR but I really don't want to get > too far out in front of the official codebase; my experiences with this > have been universally negative (more work for everyone, in the best > case). On this tangent: in the process of trying to resolve my own scalability issues with lxr.linux.no, I have a private lxr codebase that has diverged non-trivially from mainline. (And I do regret that, my experience is in line with Paul's here...) Still, I do hope to put this into production on lxr.linux.no in the near future. Once that happens, I'd be interested in assessing the actual gap to mainline, but in the mean time the code lives in a private repository of mine under the name "lxrng". If any of the movers and shakers in the lxr-mainline camp feel uncomfortable with me using this name in public as well, I'd like to know. I do feel I have a certain historical claim to the "lxr" name, but at the same time I'd like to avoid stepping on peoples toes. -- Arne. |
|
From: Malcolm B. <mal...@gm...> - 2007-06-14 23:02:45
|
Hi, Paul Smith wrote: > Hi all; > > Is there any news on a new release, or applying any of the patches, etc. > that are available for LXR? The last time I posted here there was a > large flurry of excitement that seemed promising, but it settled down > and it's been largely quiet ever since. Indeed > Are there plans? Are more hands needed to make releases/apply > patches/update bugs/perform drudge work? I'd be willing to put in some > elbow grease, esp. short term, to get the wheels turning again (I'd need > access in SF for these tasks of course). Plans yes, action no :-( I'm afraid I have very little time for LXR these days, hence the lack of releases, patches etc. If you'd like to start doing some of this then you'd be very, very welcome. Drop me a mail with your SF details and I can get you set up. > > We just installed a new LAMP box and I'm about to start migrating my > test install of LXR over to this system to be a "real" environment, and > I'd sure like to have some of the enhancements I've mentioned before > (not the least because we use an SCM tool which is not currently > supported by LXR and I'm toying with the idea of implementing a backend > for it). More backends welcome - some of the recent work on BK/Git should make implementing this easier, since at least what has to be done is now documented! Malcolm |
|
From: Paul S. <ps...@ne...> - 2007-06-14 01:24:27
|
Hi all; Is there any news on a new release, or applying any of the patches, etc. that are available for LXR? The last time I posted here there was a large flurry of excitement that seemed promising, but it settled down and it's been largely quiet ever since. Are there plans? Are more hands needed to make releases/apply patches/update bugs/perform drudge work? I'd be willing to put in some elbow grease, esp. short term, to get the wheels turning again (I'd need access in SF for these tasks of course). We just installed a new LAMP box and I'm about to start migrating my test install of LXR over to this system to be a "real" environment, and I'd sure like to have some of the enhancements I've mentioned before (not the least because we use an SCM tool which is not currently supported by LXR and I'm toying with the idea of implementing a backend for it). I would love to do some more work on LXR but I really don't want to get too far out in front of the official codebase; my experiences with this have been universally negative (more work for everyone, in the best case). Let me know; cheers! |
|
From: SourceForge.net <no...@so...> - 2007-06-13 23:15:02
|
Bugs item #1720862, was opened at 2007-05-17 10:07 Message generated for change (Comment added) made by dmulter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=390117&aid=1720862&group_id=27350 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: David Multer (dmulter) Assigned to: Nobody/Anonymous (nobody) Summary: Filename with space reports does not exist Initial Comment: Any files in my CVS repository that contain spaces in the file name, report "file XXX does not exist" when clicking on them in Source Navigation. The actual filename displayed for XXX appears to have the name truncated at the space character. Not sure if similar problems happen when traversing files like this using Identifier or General Search. For reference, the file displays correctly when using ViewVC. ---------------------------------------------------------------------- >Comment By: David Multer (dmulter) Date: 2007-06-13 16:15 Message: Logged In: YES user_id=1344296 Originator: YES Problem reported on the latest released source version: 0.9.5 tarball. ---------------------------------------------------------------------- Comment By: Malcolm Box (mbox) Date: 2007-06-13 15:47 Message: Logged In: YES user_id=215386 Originator: NO What version of LXR are you using? This problem should be fixed in the latest version in CVS. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=390117&aid=1720862&group_id=27350 |
|
From: SourceForge.net <no...@so...> - 2007-06-13 23:01:24
|
Bugs item #1009433, was opened at 2004-08-15 06:49 Message generated for change (Comment added) made by mbox You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=390117&aid=1009433&group_id=27350 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Browsing Group: v0.9.1 >Status: Closed >Resolution: Fixed Priority: 5 Private: No Submitted By: Hideaki Suzuki (h1suzuki) Assigned to: Nobody/Anonymous (nobody) Summary: extra </html> Initial Comment: the current lxr seems to have an extra html anchor, failing HTML 4.01TR validation, because both Common.pm and html-head.html have that anchor. I'm using lxr-0.9.2 and according to CVS head, the problem looks like still exists. I'm not sure which part of the files is guilty, but I wanted to let templates to start AND to end an html. so here is a fix attempt. ---------------------------------------------------------------------- Comment By: Malcolm Box (mbox) Date: 2007-06-14 00:01 Message: Logged In: YES user_id=215386 Originator: NO This has now been fixed in CVS. If you can install the new version and check that it solves your problem, then it would be very useful. Thanks for reporting this defect and helping to make LXR better. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=390117&aid=1009433&group_id=27350 |
|
From: SourceForge.net <no...@so...> - 2007-06-13 22:47:06
|
Bugs item #1720862, was opened at 2007-05-17 18:07 Message generated for change (Comment added) made by mbox You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=390117&aid=1720862&group_id=27350 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: David Multer (dmulter) Assigned to: Nobody/Anonymous (nobody) Summary: Filename with space reports does not exist Initial Comment: Any files in my CVS repository that contain spaces in the file name, report "file XXX does not exist" when clicking on them in Source Navigation. The actual filename displayed for XXX appears to have the name truncated at the space character. Not sure if similar problems happen when traversing files like this using Identifier or General Search. For reference, the file displays correctly when using ViewVC. ---------------------------------------------------------------------- >Comment By: Malcolm Box (mbox) Date: 2007-06-13 23:47 Message: Logged In: YES user_id=215386 Originator: NO What version of LXR are you using? This problem should be fixed in the latest version in CVS. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=390117&aid=1720862&group_id=27350 |
|
From: SourceForge.net <no...@so...> - 2007-05-17 17:07:51
|
Bugs item #1720862, was opened at 2007-05-17 10:07 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=390117&aid=1720862&group_id=27350 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: David Multer (dmulter) Assigned to: Nobody/Anonymous (nobody) Summary: Filename with space reports does not exist Initial Comment: Any files in my CVS repository that contain spaces in the file name, report "file XXX does not exist" when clicking on them in Source Navigation. The actual filename displayed for XXX appears to have the name truncated at the space character. Not sure if similar problems happen when traversing files like this using Identifier or General Search. For reference, the file displays correctly when using ViewVC. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=390117&aid=1720862&group_id=27350 |
|
From: SourceForge.net <no...@so...> - 2007-05-13 00:44:24
|
Bugs item #1209273, was opened at 2005-05-26 18:33 Message generated for change (Comment added) made by ipo23 You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=390117&aid=1209273&group_id=27350 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Database interface Group: None Status: Open Resolution: Accepted Priority: 5 Private: No Submitted By: Jeff Warnica (jeffwarnica) Assigned to: Malcolm Box (mbox) Summary: "release" a reserved word in MySQL 5.x Initial Comment: And perhaps earlier versions too. The table lxr_releases can not be created.. The MySQL docs claim that you can use reserved words as identifiers if you quote them, but testing the CREATE TABLE with "release" quoted also failed. ---------------------------------------------------------------------- Comment By: aaron (ipo23) Date: 2007-05-13 02:44 Message: Logged In: YES user_id=1619342 Originator: NO same here - aaron@texas:/usr/share/doc/lxr-cvs/examples$ mysql --version mysql Ver 14.12 Distrib 5.0.38, for pc-linux-gnu (x86_64) using readline 5.2 aaron@texas:/usr/share/doc/lxr-cvs/examples$ release needs to be quoted with backquotes as a field definition as well as in the primary key clause then it works ---------------------------------------------------------------------- Comment By: Brian St. Pierre (bstpierre) Date: 2006-10-06 19:40 Message: Logged In: YES user_id=10357 Looking in CVS, it doesn't appear that the patch was applied to Mysql.pm. (I do see the change in initdb-mysql.) ---------------------------------------------------------------------- Comment By: Danny (perunaion) Date: 2006-09-16 04:20 Message: Logged In: YES user_id=1047608 Release 1559658, attempts to address this issue. Its a patch of the two modified files (Mysql.pm and initdb-mysql). Hopefully that will help fix this. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2006-07-14 01:59 Message: Logged In: NO You need to change BOTH the initdb-mysql script and the Mysql.pm module (8 occurrences total). You can either change the name of the 'release' field (which was the route I took), or quote it with back-ticks. ---------------------------------------------------------------------- Comment By: Laurence Passmore (lmop) Date: 2006-06-07 10:25 Message: Logged In: YES user_id=1410237 I think you need to change lib/LXR/Index/Mysql.pm thus: 1) s/r.release/r.`release`/g 2) s/release =/`release` =/g 3) s/ release\)/ `release`\)/g (I say think because I haven't tested it; I currently have other issues with my LXR installation that is preventing me from getting it working at all...) ---------------------------------------------------------------------- Comment By: Malcolm Box (mbox) Date: 2006-06-07 00:24 Message: Logged In: YES user_id=215386 Apparently not a good fix - reopening ---------------------------------------------------------------------- Comment By: Malcolm Box (mbox) Date: 2006-06-05 12:16 Message: Logged In: YES user_id=215386 Now fixed in CVS. Thanks to lmop for the fix. ---------------------------------------------------------------------- Comment By: S Melody (prplehaze2) Date: 2006-05-31 17:37 Message: Logged In: YES user_id=911986 I used this fix to create the tables, but indexing seems to fail, most likely because release is a reserved word: DBD::mysql::st execute failed: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'release = 'head'' at line 1 at lib/LXR/Index/Mysql.pm line 209. DBD::mysql::st execute failed: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'release) values ('387', 'head')' at line 1 at lib/LXR/Index/Mysql.pm line 213. ---------------------------------------------------------------------- Comment By: Laurence Passmore (lmop) Date: 2005-12-21 18:20 Message: Logged In: YES user_id=1410237 This is a simple fix. initdb-mysql should be changed as follows (basically, backtick (`) the column name `release` to escape/quote it): create table lxr_releases ( fileid int not null references lxr_files, `release` char(255) binary not null, primary key (fileid, `release`) ); ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=390117&aid=1209273&group_id=27350 |
|
From: SourceForge.net <no...@so...> - 2007-05-09 22:19:18
|
Bugs item #1716172, was opened at 2007-05-09 15:19 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=390117&aid=1716172&group_id=27350 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Documentation Group: current cvs Status: Open Resolution: None Priority: 5 Private: No Submitted By: Nobody/Anonymous (nobody) Assigned to: Malcolm Box (mbox) Summary: small typo in INSTALL Initial Comment: >From the CVS version of INSTALL, I think line 112: 112 'ectagsconf' => \ '/usr/lib/perl5/site_perl/Lang/ectags.conf', should read: 112 'ectagsconf' => \ '/usr/lib/perl5/site_perl/LXR/Lang/ectags.conf', missing 'LXR' -- tehmasp ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=390117&aid=1716172&group_id=27350 |
|
From: SourceForge.net <no...@so...> - 2007-04-13 23:31:48
|
Feature Requests item #1691372, was opened at 2007-03-30 16:31 Message generated for change (Comment added) made by mbox You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=390120&aid=1691372&group_id=27350 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: General Group: None Status: Open Priority: 5 Private: No Submitted By: Paul D. Smith (psmith) Assigned to: Nobody/Anonymous (nobody) Summary: Rearchitect the LXR::Files class to LXR::File Initial Comment: The current LXR::Files class has a single global object that represents a backend (plain, CVS, etc.) Methods of this object take a filename/release pair and return various information about that file using that backend. I don't think this is the proper organization and this should be rearchitected. I propose the following: 1) The current LXR::Files class should be reworked to be a very small class, that just does whatever setup is needed for that backend type. It could be renamed to something like LXR::FileType or something, maybe. It's possible we won't even need this: we'll have to see. 2) A new LXR::File class should be created. An object of this class will represent ONE file/release. It will have various methods to retrieve the path, content, size, etc.; these don't need to take any arguments (generally). LXR::File is probably going to be a superclass with subclasses for each backend, and the actual objects returned would be objects of the subclass. The LXR::File class would provide basic method implementations (probably similar to today's LXR::Files::Plain class) which various other backends could override. Once this is done I think a lot of other things, like the web caching described in another feature request, become more straightforward to implement. ---------------------------------------------------------------------- >Comment By: Malcolm Box (mbox) Date: 2007-04-14 00:31 Message: Logged In: YES user_id=215386 Originator: NO Excellent idea, this would make a bunch of things easier and be higher performance with certain SCM systems. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=390120&aid=1691372&group_id=27350 |
|
From: Paul S. <ps...@ne...> - 2007-04-03 22:36:37
|
FYI, as a response to the idea that we should just change the schema rather than trying to quote everything (to fix the "release" issue), I've just loaded a patch that does this fairly comprehensively, including updating existing schemas so they will work with the new code. Tested on MySQL and PostgreSQL. I read the docs for Oracle so I *hope* that one works as well :). I hope there's still some interest in getting an LXR 1.0 released... It's been pretty quiet for the last week or so. I have other projects I'd like to do, such as add a new SCM backend for our system, etc. but I think some general cleanup is required before that can happen... and also I think the caching stuff I mentioned before (and I filed on the SF site) would be pretty important for performance here. I don't want to get too far ahead of the "real" codebase, so there are a number of steps that need to be taken. Cheers! -- ----------------------------------------------------------------------------- Paul D. Smith <ps...@ne...> http://netezza.com "Please remain calm--I may be mad, but I am a professional."--Mad Scientist ----------------------------------------------------------------------------- These are my opinions--Netezza takes no responsibility for them. |
|
From: SourceForge.net <no...@so...> - 2007-04-03 22:30:26
|
Patches item #1693932, was opened at 2007-04-03 18:30 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=390119&aid=1693932&group_id=27350 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Paul D. Smith (psmith) Assigned to: Nobody/Anonymous (nobody) Summary: Rename "release" to "stream" in SQL interface Initial Comment: In a discussion on the LXR mailing list the proposal was made to avoid the whole "quoting `release` keyword" problem in MySQL 5 by simply renaming the column to something that's not a keyword (rather than going through and remembering to quote it everywhere). This patch is a comprehensive attempt to do that, renaming the lxr_releases table to lxr_streams and the release column in that table to "stream". It also renames the "release" references in the DB backends to "stream" so that they're consistent. It does NOT attempt to rename the upper-level uses of the term "release", although that's something that might be worth doing for consistency's sake. Finally, it provides some scripts to update existing databases and perform those renames on them, so you don't have to dump/reload your data, and gives instructions in the INSTALL file on how to use them. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=390119&aid=1693932&group_id=27350 |
|
From: SourceForge.net <no...@so...> - 2007-03-30 16:04:54
|
Bugs item #1691407, was opened at 2007-03-30 12:04 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=390117&aid=1691407&group_id=27350 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: genxref Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Paul D. Smith (psmith) Assigned to: Nobody/Anonymous (nobody) Summary: genxref binary file detection flawed Initial Comment: Genxref uses File::MMagic to determine what kind of file it's indexing, and that's great. However, I noticed that a number of binary files in my tree were not being detected properly as "binary". Looking more closely I see that Perl's File::MMagic module uses an older, less complete table of file types as its hardcoded defaults, which is causing it to not detect some types of non-text files. However, it turns out that you can tell File:MMagic to use an external definition table for file types rather than the default builtin table. So, I got the latest version of magic.mime from my Ubuntu system and added it to the LXR directory. I needed to uncomment one or two lines, which magic.mime had commented out with the note 'Formats for "compress" proper have been moved into "compress.c"', which we don't have in File::MMagic apparently. This helped a LOT in classifying my files. It still gets a few wrong, though: apparently its method for detecting binary-ness of files that don't match any of the mime types is more liberal than the file(1) method. But, this works much better than before (detects .z compressed files, RPMs, etc.) Patch attached. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=390117&aid=1691407&group_id=27350 |
|
From: SourceForge.net <no...@so...> - 2007-03-30 15:55:55
|
Bugs item #1691391, was opened at 2007-03-30 11:55 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=390117&aid=1691391&group_id=27350 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: genxref Group: current cvs Status: Open Resolution: None Priority: 5 Private: No Submitted By: Paul D. Smith (psmith) Assigned to: Nobody/Anonymous (nobody) Summary: genxref/swish-e/binary files is VERY inefficient Initial Comment: I have a number of very large binary files (cross-compilers, etc.) in my source trees, and when I genxref (using swish-e) it sucks up way more CPU/memory/time than I would expect. I examined the code and found an algorithmic error. For swish-e, genxref reads the contents of the files into memory (in Perl), THEN it examines the file to determine whether it's binary or not using File::MMagic::checktype_contents(). If the file is determined to be binary (or empty) we skip it. This is obviously extremely inefficient, since File::MMagic never needs to see the entire file to know its file type: it needs to look at the filename (which it doesn't have here!) and possibly the first 8K or so of the file. So, I rewrote this section of genxref to get a filehandle for the file, then test to see if it's binary, and only if it's not do we read the contents of the file into memory. This sped up my genxref significantly and greatly reduced the load on my server during indexing. Patch from latest CVS attached. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=390117&aid=1691391&group_id=27350 |
|
From: SourceForge.net <no...@so...> - 2007-03-30 15:38:00
|
Feature Requests item #1691372, was opened at 2007-03-30 11:31 Message generated for change (Settings changed) made by psmith You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=390120&aid=1691372&group_id=27350 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. >Category: General Group: None Status: Open Priority: 5 Private: No Submitted By: Paul D. Smith (psmith) Assigned to: Nobody/Anonymous (nobody) Summary: Rearchitect the LXR::Files class to LXR::File Initial Comment: The current LXR::Files class has a single global object that represents a backend (plain, CVS, etc.) Methods of this object take a filename/release pair and return various information about that file using that backend. I don't think this is the proper organization and this should be rearchitected. I propose the following: 1) The current LXR::Files class should be reworked to be a very small class, that just does whatever setup is needed for that backend type. It could be renamed to something like LXR::FileType or something, maybe. It's possible we won't even need this: we'll have to see. 2) A new LXR::File class should be created. An object of this class will represent ONE file/release. It will have various methods to retrieve the path, content, size, etc.; these don't need to take any arguments (generally). LXR::File is probably going to be a superclass with subclasses for each backend, and the actual objects returned would be objects of the subclass. The LXR::File class would provide basic method implementations (probably similar to today's LXR::Files::Plain class) which various other backends could override. Once this is done I think a lot of other things, like the web caching described in another feature request, become more straightforward to implement. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=390120&aid=1691372&group_id=27350 |
|
From: SourceForge.net <no...@so...> - 2007-03-30 15:37:27
|
Feature Requests item #1691378, was opened at 2007-03-30 11:37 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=390120&aid=1691378&group_id=27350 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: General Group: None Status: Open Priority: 5 Private: No Submitted By: Paul D. Smith (psmith) Assigned to: Nobody/Anonymous (nobody) Summary: Rearchitect the DB backends Initial Comment: The LXR index DB backends all use DBI, which is perfect, but they don't take enough advantage of it and thus end up duplicating a significant amount of code with all the problems that generally brings. Perl's DBI is ALREADY a generic front-end for SQL databases of all different types, so having separate LXR::Index::* subclasses for each database is largely unnecessary: all databases can use the DBI interface almost exactly the same way. There are minor differences, though, so it's not true that we can get rid of the LXR::Index::* database specific classes entirely. I propose that we make LXR::Index a proper superclass for the DB-specific subclasses, and we push up as much functionality as possible into the superclass. The subclasses should be pretty small: they will deal only with DB-specific issues (for example, auto-incremented fields which work differently in PostgreSQL and MySQL). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=390120&aid=1691378&group_id=27350 |
|
From: SourceForge.net <no...@so...> - 2007-03-30 15:31:07
|
Feature Requests item #1691372, was opened at 2007-03-30 11:31 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=390120&aid=1691372&group_id=27350 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Priority: 5 Private: No Submitted By: Paul D. Smith (psmith) Assigned to: Nobody/Anonymous (nobody) Summary: Rearchitect the LXR::Files class to LXR::File Initial Comment: The current LXR::Files class has a single global object that represents a backend (plain, CVS, etc.) Methods of this object take a filename/release pair and return various information about that file using that backend. I don't think this is the proper organization and this should be rearchitected. I propose the following: 1) The current LXR::Files class should be reworked to be a very small class, that just does whatever setup is needed for that backend type. It could be renamed to something like LXR::FileType or something, maybe. It's possible we won't even need this: we'll have to see. 2) A new LXR::File class should be created. An object of this class will represent ONE file/release. It will have various methods to retrieve the path, content, size, etc.; these don't need to take any arguments (generally). LXR::File is probably going to be a superclass with subclasses for each backend, and the actual objects returned would be objects of the subclass. The LXR::File class would provide basic method implementations (probably similar to today's LXR::Files::Plain class) which various other backends could override. Once this is done I think a lot of other things, like the web caching described in another feature request, become more straightforward to implement. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=390120&aid=1691372&group_id=27350 |
|
From: SourceForge.net <no...@so...> - 2007-03-30 15:18:47
|
Feature Requests item #1691362, was opened at 2007-03-30 11:18 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=390120&aid=1691362&group_id=27350 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: General Group: None Status: Open Priority: 5 Private: No Submitted By: Paul D. Smith (psmith) Assigned to: Nobody/Anonymous (nobody) Summary: Add caching of annotated files to speed up access Initial Comment: Currently all pages displayed by LXR are dynamically generated. However, most of the content displayed is relatively static. I propose that LXR create a cache of annotated files to speed up browsing. Here are some specifics: 1) We already have a fileid, so the cache should be organized by fileid with an appropriate directory fan-out. Since the fileid format is specific to the file backend, this might be a bit tricky--one possibility is to let the file backend do the fan-out with a special method. At any rate, this ensures that all releases that use the same file version share the cache, while releases with the same file but different versions have different cached contents. 2) Only the actual annotated file should be cached: we should not cache the entire page (header/footer/etc.) So, we don't have a completely static page, but close: we just have to generate the header/footer etc., and insert the contents of the cached file at the appropriate place. This allows us to avoid the cache even when the list of releases/architectures/etc. changes. 3) If the fileid is truly unique (which I believe it has to be for LXR to work properly) then we don't really need to worry about a stale cache: it can never be stale because a different file will have a different fileid. If, for some reason, that's not sufficient it's trivial enough to detect stale data using creation time on the cached file. 4) Cleaning the cache is a trickier problem. There are a number of options here. For simple disk space reclamation we can, for example, examine the time last accessed and clean up the cached files older than a certain date. A cleanup script could be run daily or weekly, with various options such as a maximum cache size, or a fixed "clean anything older than X days". An alternative to using access time (which can be problematic since backups can modify it, or maybe your filesystem is mounted noatime for efficiency reasons) is to use time last modified, and have LXR touch the file to update the modification time every time it gets the file from the cache. Another option is to simply not clean the cache, since we don't clean out the database normally. We should add in some code to clean the cache when we DO clean out the DB: there are options to genxref to remove releases etc. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=390120&aid=1691362&group_id=27350 |
|
From: SourceForge.net <no...@so...> - 2007-03-30 14:37:49
|
Feature Requests item #489929, was opened at 2001-12-06 12:53 Message generated for change (Comment added) made by psmith You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=390120&aid=489929&group_id=27350 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Browsing Interface Group: None Status: Open Priority: 5 Private: No Submitted By: David Oleszkiewicz (poppinfresh) Assigned to: Nobody/Anonymous (nobody) Summary: function index (speedbar) Initial Comment: I use the speedbar in xemacs alot to help be jump straight down to a function in a source file. Well i actually map it to the mouse...but it would be nice to have a speebar (html frame) on the the left or right hand side of the source file which has links to all of the functions declared in a source file. presumably this information is already parsed in a manner that you know it is a function def. this could be ignored in header files or contain the index of typedefs and #defines or something. ---------------------------------------------------------------------- Comment By: Paul D. Smith (psmith) Date: 2007-03-30 10:37 Message: Logged In: YES user_id=30808 Originator: NO <Edna Most> No Frames! </Edna Most> Frames are a horrible, horrible invention and should be banned from the web forever. And even longer. Having a function list is not a bad idea, but a simple drop-down box will work. Or, if we really want something on the sidebar, please use CSS/tables. BTW, in the SQL query provided by mbox, why check both the fileid AND the filename? It seems like just the fileid would be sufficient (and a lot faster!) ---------------------------------------------------------------------- Comment By: Malcolm Box (mbox) Date: 2002-01-23 11:08 Message: Logged In: YES user_id=215386 Wouldn't be hard. The sql to find all the symbols defined in a file is: select s.symname, i.line, d.declaration from indexes i, symbols s, declarations d, files f where s.symi d=i.symid and i.type = d.declid and i.fileid=f.fileid and f.filename = "filename"; >From there, it should be easy to spin some HTML and links round the results and shove it in a sidebar. Presumably this would be linked off the source script, so that it would first generate the source listing, and then add the sidebar panel. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=390120&aid=489929&group_id=27350 |
|
From: Paul S. <ps...@ne...> - 2007-03-23 15:12:07
|
On Thu, 2007-03-22 at 21:41 +0100, Jan-Benedict Glaw wrote: > On Thu, 2007-03-22 16:12:23 -0400, Paul Smith <ps...@ne...> wrote: > > > > Genxref performance improvement: > > Especially when adding a new release, genxref can be very slow. I think > > it's because of all the indexing that goes on, and the DB re-indexes > > after every insert. A common method of adding a lot of data to an SQL > > DB is to put the commands into a file, then load the file all at once > > with indexing disabled, then re-index everything at the end. > > The basic concept could be simplified to drop PRIMARY KEY constraints > and indexes and add them back afterwards. (Though you may fail in case > the PRIMARY KEY constraints were ignored in parts of the data...) Not > a bright idea... Yes, and there's an even more important problem I realized after I sent this: many of LXR's tables are indexed through auto-increment values. Of course we cannot know what values these will have until the table is loaded, so we can't really write out a file to load all that data at once. The best we could do would be to update the tables one at a time, being sure to create the basic tables first then reading the key values out of them to create the next table, etc. Seems overly complex, so this project is probably not worth it as things stand. > > Web performance improvement: > > I'm sure someone else has already thought of this, but right now we > > generate our HTML dynamically every time. It seems to me that this is a > > prime candidate for caching! Especially for backends that support > > annotate/blame etc. The annotations on a file won't change unless the > > contents of the file changes, for the most part (the other possibility > > is that the symbols in the database changes--as far as I can tell this > > shouldn't happen normally but if people are worried we can have genxref > > flush the cache). > > Output shouldn't change at all :) Well, unless the templates get > modified. Hopefully most/all of those types of changes can be handled through CSS; that would be my preferred way to do it anyway (I think many already are). I was thinking about symbols: if we re-index and some symbol that we didn't used to know about suddenly becomes available in the database, then any cached files that reference that symbol will not be updated to have a link. For static source trees this can't happen, of course, but LXR is also used to index trees that are still changing (a daily update of the HEAD of some stream/branch for example). Admittedly it's pretty hard to think of how this could happen without the relevant source files changing as well, which would obviously flush the cache. The only way I can see it offhand is if a symbol which used to be contained outside the tree (and so not indexed), suddenly were moved inside. This is such a corner case I'm not sure it's worth catering to at the expense of much performance. I was imagining how this might be implemented and I think that the first step is to create an LXR::File (one object per file) class. That would make the caching interface and code much simpler. This is definitely a post-1.0 thing. > > I'd be happy with that. It's not actually such a big deal to fix this; > > you just need a series of ALTER TABLE operations. It's easy enough to > > add to the readme, or even write an update script. > > I'm not sure how happy MySQL will be with its foreign keys... Hm. It definitely needs to be tested. -- ----------------------------------------------------------------------------- Paul D. Smith <ps...@ne...> http://netezza.com "Please remain calm--I may be mad, but I am a professional."--Mad Scientist ----------------------------------------------------------------------------- These are my opinions--Netezza takes no responsibility for them. |
|
From: Jan-Benedict G. <jb...@lu...> - 2007-03-22 20:44:59
|
On Thu, 2007-03-22 16:20:18 -0400, Paul Smith <ps...@ne...> wrote:
> On Thu, 2007-03-22 at 20:09 +0100, Jan-Benedict Glaw wrote:
> > Since I was about to loose overview over the patches and suggestions,
> > I've pushed the recent stuff into a Wiki page.
>=20
> I loves me a wiki, but we already have a sourceforge project and I
> wonder if this kind of thing isn't better handled using their trackers?
Sure. Just put it in there (-:
MfG, JBG
--=20
Jan-Benedict Glaw jb...@lu... +49-172-7608481
Signature of: Wenn ich wach bin, tr=C3=A4ume ic=
h.
the second :
|
|
From: Jan-Benedict G. <jb...@lu...> - 2007-03-22 20:41:49
|
On Thu, 2007-03-22 16:12:23 -0400, Paul Smith <ps...@ne...> wrote:
> I had two other performance-related ideas:
>=20
> Genxref performance improvement:
> Especially when adding a new release, genxref can be very slow. I think
> it's because of all the indexing that goes on, and the DB re-indexes
> after every insert. A common method of adding a lot of data to an SQL
> DB is to put the commands into a file, then load the file all at once
> with indexing disabled, then re-index everything at the end. I know
> both MySQL and Postgres support this model (although most likely it's
> accomplished in different ways) although I've not investigated it
> thoroughly.
The basic concept could be simplified to drop PRIMARY KEY constraints
and indexes and add them back afterwards. (Though you may fail in case
the PRIMARY KEY constraints were ignored in parts of the data...) Not
a bright idea...
> So, my idea was changing genxref to do this: instead of adding things to
> the DB one at a time, it would write out the statements to a file and at
> the end, import that file with indexing disabled.
COPY could be an alternative, but that requires a file being read
directly by the PostgreSQL server. (Don't know how, or whether at all,
this is implemented by the other DB backends.)
Also, the \copy directive of psql is worth being mentioned.
> Web performance improvement:
> I'm sure someone else has already thought of this, but right now we
> generate our HTML dynamically every time. It seems to me that this is a
> prime candidate for caching! Especially for backends that support
> annotate/blame etc. The annotations on a file won't change unless the
> contents of the file changes, for the most part (the other possibility
> is that the symbols in the database changes--as far as I can tell this
> shouldn't happen normally but if people are worried we can have genxref
> flush the cache).
Output shouldn't change at all :) Well, unless the templates get
modified.
> We have a unique file id already, so we can cache the content using the
> file id with a typical span out to avoid any single directory being too
> large. We can compare creation time of the cached file vs. the source
> file to tell when it's out of date. We can use the access time to clean
> out old, unused cache entries if we want.
>=20
> Also, we can just cache the actual file content, and leave off the
> header information; the header info can be added dynamically when the
> user browses it. That way the same cached copy of a file can be used
> for different releases, if they all share that fileid.
Hopefully, everybody has a nice robots.txt to forbid Google et al. to
index the whole thing, once...
> > Heck, this f*ing column name caused so much grief, lets just rename
> > it! Yes, that's somewhat painful and we need a Big Fat Warning in the
> > v1.0 docs that the column needs to be renamed, but I'm all for doing
> > that.
>=20
> I'd be happy with that. It's not actually such a big deal to fix this;
> you just need a series of ALTER TABLE operations. It's easy enough to
> add to the readme, or even write an update script.
I'm not sure how happy MySQL will be with its foreign keys...
> > > Actually there is an "ANSI_QUOTES" mode that we could set, that lets =
you
> > > (among other things) use standard "-quoting in MySQL. That might be a
> > > valid thing to do.
> >=20
> > Is this in the DBI backend or configury on the MySQL server side?
>=20
> It can be set globally or per-session. We'd use per-session obviously.
> It's set from the client side.
>=20
> I also discovered that there's a DBI method that quotes identifiers like
> this for you:
>=20
> my $release =3D $dbh->quote_identifier('release');
>=20
> That would be the safest way to go, although it's annoyingly verbose.
> And, there's a DBI get_info() method that lets you ask about all kinds
> of features of the server, and one of those is the quoting character, so
> we could get that and use it instead of quotes.
>=20
> But, changing the name sounds good to me! :)
Lets just change the name. I actually don't think such a kludge is
worth being done while an easy solution is available.
MfG, JBG
--=20
Jan-Benedict Glaw jb...@lu... +49-172-7608481
Signature of: Alles sollte so einfach wie m=C3=B6glich gemacht=
sein.
the second : Aber nicht einfacher. (Einstein)
|
|
From: Paul S. <ps...@ne...> - 2007-03-22 20:20:31
|
On Thu, 2007-03-22 at 20:09 +0100, Jan-Benedict Glaw wrote: > Since I was about to loose overview over the patches and suggestions, > I've pushed the recent stuff into a Wiki page. I loves me a wiki, but we already have a sourceforge project and I wonder if this kind of thing isn't better handled using their trackers? It's easier to add comments, upload attachments, track the status, etc. You can configure it to send you mail when "interesting things" happen. And, it's all combined with the rest of the project info. I've used these kinds of sites with other projects and actually you can get a pretty handy release/management process using them, if you ignore the stuff you don't need and concentrate on a few things. -- ----------------------------------------------------------------------------- Paul D. Smith <ps...@ne...> http://netezza.com "Please remain calm--I may be mad, but I am a professional."--Mad Scientist ----------------------------------------------------------------------------- These are my opinions--Netezza takes no responsibility for them. |
|
From: Paul S. <ps...@ne...> - 2007-03-22 20:13:09
|
On Thu, 2007-03-22 at 20:30 +0100, Jan-Benedict Glaw wrote:
> What may be different is the way to get the newly auto-generated
> number back again.
Yes, this is what I meant.
> But since the INSERT pathes aren't time-critical, we'd just allow to
> SELECT for the value instead of playing tricks to get it.
Hm. Interesting idea. I'm not so sure they aren't time-critical. They
don't impact the web user of course, but genxref does take a while to
index stuff already :).
I had two other performance-related ideas:
Genxref performance improvement:
Especially when adding a new release, genxref can be very slow. I think
it's because of all the indexing that goes on, and the DB re-indexes
after every insert. A common method of adding a lot of data to an SQL
DB is to put the commands into a file, then load the file all at once
with indexing disabled, then re-index everything at the end. I know
both MySQL and Postgres support this model (although most likely it's
accomplished in different ways) although I've not investigated it
thoroughly.
So, my idea was changing genxref to do this: instead of adding things to
the DB one at a time, it would write out the statements to a file and at
the end, import that file with indexing disabled.
Web performance improvement:
I'm sure someone else has already thought of this, but right now we
generate our HTML dynamically every time. It seems to me that this is a
prime candidate for caching! Especially for backends that support
annotate/blame etc. The annotations on a file won't change unless the
contents of the file changes, for the most part (the other possibility
is that the symbols in the database changes--as far as I can tell this
shouldn't happen normally but if people are worried we can have genxref
flush the cache).
We have a unique file id already, so we can cache the content using the
file id with a typical span out to avoid any single directory being too
large. We can compare creation time of the cached file vs. the source
file to tell when it's out of date. We can use the access time to clean
out old, unused cache entries if we want.
Also, we can just cache the actual file content, and leave off the
header information; the header info can be added dynamically when the
user browses it. That way the same cached copy of a file can be used
for different releases, if they all share that fileid.
> Heck, this f*ing column name caused so much grief, lets just rename
> it! Yes, that's somewhat painful and we need a Big Fat Warning in the
> v1.0 docs that the column needs to be renamed, but I'm all for doing
> that.
I'd be happy with that. It's not actually such a big deal to fix this;
you just need a series of ALTER TABLE operations. It's easy enough to
add to the readme, or even write an update script.
> > Actually there is an "ANSI_QUOTES" mode that we could set, that lets you
> > (among other things) use standard "-quoting in MySQL. That might be a
> > valid thing to do.
>
> Is this in the DBI backend or configury on the MySQL server side?
It can be set globally or per-session. We'd use per-session obviously.
It's set from the client side.
I also discovered that there's a DBI method that quotes identifiers like
this for you:
my $release = $dbh->quote_identifier('release');
That would be the safest way to go, although it's annoyingly verbose.
And, there's a DBI get_info() method that lets you ask about all kinds
of features of the server, and one of those is the quoting character, so
we could get that and use it instead of quotes.
But, changing the name sounds good to me! :)
--
-----------------------------------------------------------------------------
Paul D. Smith <ps...@ne...> http://netezza.com
"Please remain calm--I may be mad, but I am a professional."--Mad Scientist
-----------------------------------------------------------------------------
These are my opinions--Netezza takes no responsibility for them.
|
|
From: Arne G. G. <ar...@gl...> - 2007-03-22 20:03:01
|
Jan-Benedict Glaw wrote: > What may be different is the way to get the newly auto-generated > number back again. But since the INSERT pathes aren't time-critical, > we'd just allow to SELECT for the value instead of playing tricks to > get it. While indexing is a batch job and as such not time-critical interactive-wise, there's a fair amount of inserts going on when you add a new release of a large project. I don't think the added simplicity is really worth de-optimizing this path. While it probably makes sense to add general versions to the DBI base class, I'd keep the specialized versions in the db-specific classes, at least for the larger tables. -- Arne. |
6 messages has been excluded from this view by a project administrator.
