From: Adrian I. <adr...@ho...> - 2009-05-10 12:02:37
|
Hi All, I've just made a commit that probably needs a bit more than normal explanation. The issue I set out to solve was to further standarise the SQL used by each of the DBI implementations; in particular to use the same column names in the lxr_releases table. Malcolm correctly fixed bug 1209273 (release a reserved word in MySQL 5.x) just after v0.9.5 of lxr was released by changing the release column to be the rel column. However, this was done just for MySql so something similar needed to be done for the Postgres and Oracle implementations too. I felt that rel wasn't as immediately understandable as releaseid so I changed all the columns to that. While doing a search and replace it struck me that the code would be cleaner if all the $release variables were renamed to $releaseid too to match up with the SQL so I've also done that which meant a whole lot of files have changed superficially. Anyhow, if you don't like the new name, feel free to suggest an alternative and I'd be happy to make the changes. Cheers, Adrian BTW, when we release v0.9.6, as result of the fix for bug 1209273, we're going to need to tell people that their existing databases won't work out-of-the-box with the new version ... > From: adr...@us... > Date: Sun, 10 May 2009 11:54:31 +0000 > To: lxr...@li... > Subject: [Lxr-commits] CVS: lxr/lib/LXR/Index Mysql.pm, 1.30, 1.31 Oracle.pm, 1.20, 1.21 DB.pm, 1.17, 1.18 Postgres.pm, 1.31, 1.32 > > Update of /cvsroot/lxr/lxr/lib/LXR/Index > In directory ddv4jf1.ch3.sourceforge.com:/tmp/cvs-serv3821/lib/LXR/Index > > Modified Files: > Mysql.pm Oracle.pm DB.pm Postgres.pm > Log Message: > Feature 1691378 (Rearchitect the DB backends) Part X > * Renamed lxr_release.rel -> releaseid for Mysql > * Renamed lxr_release.release -> releaseid for Oracle and Postgres > * Renamed $release -> $releaseid generally > > Index: Mysql.pm > =================================================================== > RCS file: /cvsroot/lxr/lxr/lib/LXR/Index/Mysql.pm,v > retrieving revision 1.30 > retrieving revision 1.31 > diff -u -d -r1.30 -r1.31 > --- Mysql.pm 9 May 2009 21:57:34 -0000 1.30 > +++ Mysql.pm 10 May 2009 11:54:29 -0000 1.31 > @@ -73,7 +73,7 @@ > . "where s.symid = i.symid and i.fileid = f.fileid " > . "and f.fileid = r.fileid " > . "and i.langid = d.langid and i.type = d.declid " > - . "and s.symname = ? and r.rel = ? " > + . "and s.symname = ? and r.releaseid = ? " > . "order by f.filename, i.line, d.declaration"); > $self->{indexes_insert} = > $self->{dbh}->prepare( > @@ -81,9 +81,9 @@ > ); > > $self->{releases_select} = > - $self->{dbh}->prepare("select * from ${prefix}releases where fileid = ? and rel = ?"); > + $self->{dbh}->prepare("select * from ${prefix}releases where fileid = ? and releaseid = ?"); > $self->{releases_insert} = > - $self->{dbh}->prepare("insert into ${prefix}releases (fileid, rel) values (?, ?)"); > + $self->{dbh}->prepare("insert into ${prefix}releases (fileid, releaseid) values (?, ?)"); > > $self->{status_select} = > $self->{dbh}->prepare("select status from ${prefix}status where fileid = ?"); > @@ -100,7 +100,7 @@ > . "where s.symid = u.symid " > . "and f.fileid = u.fileid " > . "and u.fileid = r.fileid " > - . "and s.symname = ? and r.rel = ? " > + . "and s.symname = ? and r.releaseid = ? " > . "order by f.filename, u.line"); > $self->{decl_select} = > $self->{dbh}->prepare( > @@ -113,24 +113,24 @@ > $self->{dbh}->prepare("delete from i " > . "using ${prefix}indexes i, ${prefix}releases r " > . "where i.fileid = r.fileid " > - . "and r.rel = ?"); > + . "and r.releaseid = ?"); > $self->{delete_usage} = > $self->{dbh}->prepare("delete from u " > . "using ${prefix}usage u, ${prefix}releases r " > . "where u.fileid = r.fileid " > - . "and r.rel = ?"); > + . "and r.releaseid = ?"); > $self->{delete_status} = > $self->{dbh}->prepare("delete from s " > . "using ${prefix}status s, ${prefix}releases r " > . "where s.fileid = r.fileid " > - . "and r.rel = ?"); > + . "and r.releaseid = ?"); > $self->{delete_releases} = > - $self->{dbh}->prepare("delete from ${prefix}releases where rel = ?"); > + $self->{dbh}->prepare("delete from ${prefix}releases where releaseid = ?"); > $self->{delete_files} = > $self->{dbh}->prepare("delete from f " > . "using ${prefix}files f, ${prefix}releases r " > . "where f.fileid = r.fileid " > - . "and r.rel = ?"); > + . "and r.releaseid = ?"); > > return $self; > } > @@ -185,13 +185,13 @@ > } > > sub setfilerelease { > - my ($self, $fileid, $release) = @_; > + my ($self, $fileid, $releaseid) = @_; > > - my $rows = $self->{releases_select}->execute($fileid + 0, $release); > + my $rows = $self->{releases_select}->execute($fileid + 0, $releaseid); > $self->{releases_select}->finish(); > > unless ($rows > 0) { > - $self->{releases_insert}->execute($fileid, $release); > + $self->{releases_insert}->execute($fileid, $releaseid); > } > } > > @@ -251,10 +251,10 @@ > } > > sub symdeclarations { > - my ($self, $symname, $release) = @_; > + my ($self, $symname, $releaseid) = @_; > my ($rows, @ret, @row); > > - $rows = $self->{indexes_select}->execute("$symname", "$release"); > + $rows = $self->{indexes_select}->execute("$symname", "$releaseid"); > while (@row = $self->{indexes_select}->fetchrow_array) { > $row[3] &&= $self->symname($row[3]); # convert the symid > push(@ret, [@row]); > @@ -272,10 +272,10 @@ > } > > sub symreferences { > - my ($self, $symname, $release) = @_; > + my ($self, $symname, $releaseid) = @_; > my ($rows, @ret, @row); > > - $rows = $self->{usage_select}->execute("$symname", "$release"); > + $rows = $self->{usage_select}->execute("$symname", "$releaseid"); > > while (@row = $self->{usage_select}->fetchrow_array) { > push(@ret, [@row]); > @@ -293,15 +293,15 @@ > } > > sub issymbol { > - my ($self, $symname, $release) = @_; # TODO make full use of $release > + my ($self, $symname, $releaseid) = @_; # TODO make full use of $releaseid > my ($symid); > > - $symid = $symcache{$release}{$symname}; > + $symid = $symcache{$releaseid}{$symname}; > unless (defined($symid)) { > $self->{symbols_byname}->execute($symname); > ($symid) = $self->{symbols_byname}->fetchrow_array(); > $self->{symbols_byname}->finish(); > - $symcache{$release}{$symname} = $symid; > + $symcache{$releaseid}{$symname} = $symid; > } > > return $symid; > @@ -363,15 +363,15 @@ > } > > sub purge { > - my ($self, $release) = @_; > + my ($self, $releaseid) = @_; > > # we don't delete symbols, because they might be used by other versions > # so we can end up with unused symbols, but that doesn't cause any problems > - $self->{delete_indexes}->execute($release); > - $self->{delete_usage}->execute($release); > - $self->{delete_status}->execute($release); > - $self->{delete_releases}->execute($release); > - $self->{delete_files}->execute($release); > + $self->{delete_indexes}->execute($releaseid); > + $self->{delete_usage}->execute($releaseid); > + $self->{delete_status}->execute($releaseid); > + $self->{delete_releases}->execute($releaseid); > + $self->{delete_files}->execute($releaseid); > } > > 1; > > Index: Oracle.pm > =================================================================== > RCS file: /cvsroot/lxr/lxr/lib/LXR/Index/Oracle.pm,v > retrieving revision 1.20 > retrieving revision 1.21 > diff -u -d -r1.20 -r1.21 > --- Oracle.pm 9 May 2009 21:57:34 -0000 1.20 > +++ Oracle.pm 10 May 2009 11:54:29 -0000 1.21 > @@ -71,7 +71,7 @@ > . "where s.symid = i.symid and i.fileid = f.fileid " > . "and f.fileid = r.fileid " > . "and i.langid = d.langid and i.type = d.declid " > - . "and s.symname = ? and r.release = ? " > + . "and s.symname = ? and r.releaseid = ? " > . "order by f.filename, i.line, d.declaration"); > $self->{indexes_insert} = > $self->{dbh}->prepare( > @@ -79,9 +79,9 @@ > ); > > $self->{releases_select} = > - $self->{dbh}->prepare("select * from ${prefix}releases where fileid = ? and release = ?"); > + $self->{dbh}->prepare("select * from ${prefix}releases where fileid = ? and releaseid = ?"); > $self->{releases_insert} = > - $self->{dbh}->prepare("insert into ${prefix}releases (fileid, release) values (?, ?)"); > + $self->{dbh}->prepare("insert into ${prefix}releases (fileid, releaseid) values (?, ?)"); > > $self->{status_select} = > $self->{dbh}->prepare("select status from ${prefix}status where fileid = ?"); > @@ -100,7 +100,7 @@ > . "where s.symid = u.symid " > . "and f.fileid = u.fileid " > . "and u.fileid = r.fileid " > - . "and s.symname = ? and r.release = ? " > + . "and s.symname = ? and r.releaseid = ? " > . "order by f.filename, u.line"); > $self->{decl_select} = > $self->{dbh}->prepare( > @@ -112,21 +112,21 @@ > $self->{delete_indexes} = > $self->{dbh}->prepare("delete from ${prefix}indexes " > . "where fileid in " > - . " (select fileid from ${prefix}releases where release = ?)"); > + . " (select fileid from ${prefix}releases where releaseid = ?)"); > $self->{delete_usage} = > $self->{dbh}->prepare("delete from ${prefix}usage " > . "where fileid in " > - . " (select fileid from ${prefix}releases where release = ?)"); > + . " (select fileid from ${prefix}releases where releaseid = ?)"); > $self->{delete_status} = > $self->{dbh}->prepare("delete from ${prefix}status " > . "where fileid in " > - . " (select fileid from ${prefix}releases where release = ?)"); > + . " (select fileid from ${prefix}releases where releaseid = ?)"); > $self->{delete_releases} = > - $self->{dbh}->prepare("delete from ${prefix}releases where release = ?"); > + $self->{dbh}->prepare("delete from ${prefix}releases where releaseid = ?"); > $self->{delete_files} = > $self->{dbh}->prepare("delete from ${prefix}files " > . "where fileid in " > - . " (select fileid from ${prefix}releases where release = ?)"); > + . " (select fileid from ${prefix}releases where releaseid = ?)"); > > return $self; > } > @@ -181,13 +181,13 @@ > } > > sub setfilerelease { > - my ($self, $fileid, $release) = @_; > + my ($self, $fileid, $releaseid) = @_; > > - my $rows = $self->{releases_select}->execute($fileid + 0, $release); > + my $rows = $self->{releases_select}->execute($fileid + 0, $releaseid); > $self->{releases_select}->finish(); > > unless ($rows > 0) { > - $self->{releases_insert}->execute($fileid, $release); > + $self->{releases_insert}->execute($fileid, $releaseid); > } > } > > @@ -247,10 +247,10 @@ > } > > sub symdeclarations { > - my ($self, $symname, $release) = @_; > + my ($self, $symname, $releaseid) = @_; > my ($rows, @ret, @row); > > - $rows = $self->{indexes_select}->execute("$symname", "$release"); > + $rows = $self->{indexes_select}->execute("$symname", "$releaseid"); > while (@row = $self->{indexes_select}->fetchrow_array) { > $row[3] &&= $self->symname($row[3]); # convert the symid > push(@ret, [@row]); > @@ -268,10 +268,10 @@ > } > > sub symreferences { > - my ($self, $symname, $release) = @_; > + my ($self, $symname, $releaseid) = @_; > my ($rows, @ret, @row); > > - $rows = $self->{usage_select}->execute("$symname", "$release"); > + $rows = $self->{usage_select}->execute("$symname", "$releaseid"); > > while (@row = $self->{usage_select}->fetchrow_array) { > push(@ret, [@row]); > @@ -289,15 +289,15 @@ > } > > sub issymbol { > - my ($self, $symname, $release) = @_; # TODO make full use of $release > + my ($self, $symname, $releaseid) = @_; # TODO make full use of $releaseid > my ($symid); > > - $symid = $symcache{$release}{$symname}; > + $symid = $symcache{$releaseid}{$symname}; > unless (defined($symid)) { > $self->{symbols_byname}->execute($symname); > ($symid) = $self->{symbols_byname}->fetchrow_array(); > $self->{symbols_byname}->finish(); > - $symcache{$release}{$symname} = $symid; > + $symcache{$releaseid}{$symname} = $symid; > } > > return $symid; > @@ -359,15 +359,15 @@ > } > > sub purge { > - my ($self, $release) = @_; > + my ($self, $releaseid) = @_; > > # we don't delete symbols, because they might be used by other versions > # so we can end up with unused symbols, but that doesn't cause any problems > - $self->{delete_indexes}->execute($release); > - $self->{delete_usage}->execute($release); > - $self->{delete_status}->execute($release); > - $self->{delete_releases}->execute($release); > - $self->{delete_files}->execute($release); > + $self->{delete_indexes}->execute($releaseid); > + $self->{delete_usage}->execute($releaseid); > + $self->{delete_status}->execute($releaseid); > + $self->{delete_releases}->execute($releaseid); > + $self->{delete_files}->execute($releaseid); > } > > 1; > > Index: DB.pm > =================================================================== > RCS file: /cvsroot/lxr/lxr/lib/LXR/Index/DB.pm,v > retrieving revision 1.17 > retrieving revision 1.18 > diff -u -d -r1.17 -r1.18 > --- DB.pm 9 May 2009 21:57:34 -0000 1.17 > +++ DB.pm 10 May 2009 11:54:29 -0000 1.18 > @@ -54,14 +54,14 @@ > > # Returns array of (fileid, line, type) > sub symdeclarations { > - my ($self, $symname, $release) = @_; > + my ($self, $symname, $releaseid) = @_; > > my (@d, $f); > foreach $f (split(/\0/, $$self{'indexes'}{ $self->symid($symname) })) { > my ($fi, $l, $t, $s) = split(/\t/, $f); > > my %r = map { ($_ => 1) } split(/;/, $self->{'releases'}{$fi}); > - next unless $r{$release}; > + next unless $r{$releaseid}; > > push(@d, [ $self->filename($fi), $l, $t, $s ]); > } > @@ -69,7 +69,7 @@ > } > > sub symreferences { > - my ($self, $symname, $release) = @_; > + my ($self, $symname, $releaseid) = @_; > return (); > } > > @@ -98,11 +98,11 @@ > return 1; > } > > -# Indicate that this filerevision is part of this release > +# Indicate that this filerevision is part of this releaseid > sub setfilerelease { > - my ($self, $fileid, $release) = @_; > + my ($self, $fileid, $releaseid) = @_; > > - $self->{'releases'}{$fileid} .= $release . ";"; > + $self->{'releases'}{$fileid} .= $releaseid . ";"; > } > > sub symid { > @@ -113,7 +113,7 @@ > } > > sub issymbol { > - my ($self, $symname, $release) = @_; > + my ($self, $symname, $releaseid) = @_; > > return $$self{'indexes'}{ $self->symid($symname) }; > } > > Index: Postgres.pm > =================================================================== > RCS file: /cvsroot/lxr/lxr/lib/LXR/Index/Postgres.pm,v > retrieving revision 1.31 > retrieving revision 1.32 > diff -u -d -r1.31 -r1.32 > --- Postgres.pm 9 May 2009 21:57:34 -0000 1.31 > +++ Postgres.pm 10 May 2009 11:54:29 -0000 1.32 > @@ -77,14 +77,14 @@ > . "where s.symid = i.symid and i.fileid = f.fileid " > . "and f.fileid = r.fileid " > . "and i.langid = d.langid and i.type = d.declid " > - . "and s.symname = ? and r.release = ? " > + . "and s.symname = ? and r.releaseid = ? " > . "order by f.filename, i.line, d.declaration"); > $self->{indexes_insert} = > $self->{dbh}->prepare("insert into ${prefix}indexes (symid, fileid, line, langid, type, relsym) " > . "values (?, ?, ?, ?, ?, ?)"); > > $self->{releases_select} = > - $self->{dbh}->prepare("select * from ${prefix}releases where fileid = ? and release = ?"); > + $self->{dbh}->prepare("select * from ${prefix}releases where fileid = ? and releaseid = ?"); > $self->{releases_insert} = > $self->{dbh}->prepare("insert into ${prefix}releases values (?, ?)"); > > @@ -103,7 +103,7 @@ > . "where s.symid = u.symid " > . "and f.fileid = u.fileid " > . "and u.fileid = r.fileid " > - . "and s.symname = ? and r.release = ? " > + . "and s.symname = ? and r.releaseid = ? " > . "order by f.filename, u.line"); > > $self->{declid_nextnum} = > @@ -119,21 +119,21 @@ > $self->{delete_indexes} = > $self->{dbh}->prepare("delete from ${prefix}indexes " > . "where fileid in " > - . " (select fileid from ${prefix}releases where release = ?)"); > + . " (select fileid from ${prefix}releases where releaseid = ?)"); > $self->{delete_usage} = > $self->{dbh}->prepare("delete from ${prefix}usage " > . "where fileid in " > - . " (select fileid from ${prefix}releases where release = ?)"); > + . " (select fileid from ${prefix}releases where releaseid = ?)"); > $self->{delete_status} = > $self->{dbh}->prepare("delete from ${prefix}status " > . "where fileid in " > - . " (select fileid from ${prefix}releases where release = ?)"); > + . " (select fileid from ${prefix}releases where releaseid = ?)"); > $self->{delete_releases} = > - $self->{dbh}->prepare("delete from ${prefix}releases where release = ?"); > + $self->{dbh}->prepare("delete from ${prefix}releases where releaseid = ?"); > $self->{delete_files} = > $self->{dbh}->prepare("delete from ${prefix}files " > . "where fileid in " > - . " (select fileid from ${prefix}releases where release = ?)"); > + . " (select fileid from ${prefix}releases where releaseid = ?)"); > > return $self; > } > @@ -199,15 +199,15 @@ > } > > sub setfilerelease { > - my ($self, $fileid, $release) = @_; > + my ($self, $fileid, $releaseid) = @_; > > - $self->{releases_select}->execute($fileid + 0, $release); > + $self->{releases_select}->execute($fileid + 0, $releaseid); > my $firstrow = $self->{releases_select}->fetchrow_array(); > > # $self->{releases_select}->finish(); > > unless ($firstrow) { > - $self->{releases_insert}->execute($fileid + 0, $release); > + $self->{releases_insert}->execute($fileid + 0, $releaseid); > $self->_commitIfLimit(); > } > } > @@ -270,10 +270,10 @@ > } > > sub symdeclarations { > - my ($self, $symname, $release) = @_; > + my ($self, $symname, $releaseid) = @_; > my ($rows, @ret, @row); > > - $rows = $self->{indexes_select }->execute("$symname", "$release"); > + $rows = $self->{indexes_select }->execute("$symname", "$releaseid"); > > while (@row = $self->{indexes_select }->fetchrow_array) { > $row[3] &&= $self->symname($row[3]); # convert the symid > @@ -298,10 +298,10 @@ > } > > sub symreferences { > - my ($self, $symname, $release) = @_; > + my ($self, $symname, $releaseid) = @_; > my ($rows, @ret, @row); > > - $rows = $self->{usage_select}->execute("$symname", "$release"); > + $rows = $self->{usage_select}->execute("$symname", "$releaseid"); > > while (@row = $self->{usage_select}->fetchrow_array) { > push(@ret, [@row]); > @@ -320,7 +320,7 @@ > } > > sub issymbol { > - my ($self, $symname, $release) = @_; # TODO make full use of $release > + my ($self, $symname, $releaseid) = @_; # TODO make full use of $releaseid > > unless (exists($symcache{$symname})) { > $self->{symbols_byname}->execute($symname); > @@ -383,15 +383,15 @@ > } > > sub purge { > - my ($self, $release) = @_; > + my ($self, $releaseid) = @_; > > # we don't delete symbols, because they might be used by other versions > # so we can end up with unused symbols, but that doesn't cause any problems > - $self->{delete_indexes}->execute($release); > - $self->{delete_usage}->execute($release); > - $self->{delete_status}->execute($release); > - $self->{delete_releases}->execute($release); > - $self->{delete_files}->execute($release); > + $self->{delete_indexes}->execute($releaseid); > + $self->{delete_usage}->execute($releaseid); > + $self->{delete_status}->execute($releaseid); > + $self->{delete_releases}->execute($releaseid); > + $self->{delete_files}->execute($releaseid); > > $self->{dbh}->commit() or die "Commit failed: $DBI::errstr"; > $self->{dbh}->begin_work() or die "begin_work failed: $DBI::errstr"; > > > ------------------------------------------------------------------------------ > The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your > production scanning environment may not be a perfect world - but thanks to > Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700 > Series Scanner you'll get full speed at 300 dpi even with all image > processing features enabled. http://p.sf.net/sfu/kodak-com > _______________________________________________ > Lxr-commits mailing list > Lxr...@li... > https://lists.sourceforge.net/lists/listinfo/lxr-commits _________________________________________________________________ View your Twitter and Flickr updates from one place – Learn more! http://clk.atdmt.com/UKM/go/137984870/direct/01/ |
From: Adrian I. <adr...@ho...> - 2009-05-12 20:48:36
|
> I suppose for extra points we could provide a migration script This did occur to me but I must admit hoped we'd not need to go for it. > We've broken db compatibility in the past without anyone complaining, I suspect this is because a common use of lxr that I've seen is to run genxref on the latest source and show that and old versions of the code in the database don't get kept around for long (less than a month). Hence no database files are kept around that long anyhow. Cheers, Adrian ----- Original Message ----- From: "Malcolm Box" <mal...@gm...> To: "Adrian Issott" <adr...@ho...> Cc: <lxr...@li...> Sent: Monday, May 11, 2009 12:18 PM Subject: Re: [Lxr-dev] FW: [Lxr-commits] CVS: lxr/lib/LXR/Index Mysql.pm, 1.30, 1.31 Oracle.pm, 1.20, 1.21 DB.pm, 1.17, 1.18 Postgres.pm, 1.31, 1.32 > Hi, > > On Sun, May 10, 2009 at 1:02 PM, Adrian Issott <adr...@ho...> > wrote: >> >> I felt that rel wasn't as immediately >> understandable as releaseid so I changed all the columns to that. While >> doing a search and replace it struck me that the code would be cleaner if >> all the $release variables were renamed to $releaseid too to match up >> with >> the SQL so I've also done that which meant a whole lot of files have >> changed >> superficially. > > releaseid is a better name - so this change looks fine to me. > >> >> BTW, when we release v0.9.6, as result of the fix for bug 1209273, we're >> going to need to tell people that their existing databases won't work >> out-of-the-box with the new version ... > > Yes, must remember to flag that up in the release notes and > announcement. We've broken db compatibility in the past without > anyone complaining, so I guess it will be OK. > > I suppose for extra points we could provide a migration script, but my > preference would be to only do that for stable versions (i.e. 1.0 > onwards). > > Malcolm > |
From: Malcolm B. <mal...@gm...> - 2009-05-11 11:19:15
|
Hi, On Sun, May 10, 2009 at 1:02 PM, Adrian Issott <adr...@ho...> wrote: > > I felt that rel wasn't as immediately > understandable as releaseid so I changed all the columns to that. While > doing a search and replace it struck me that the code would be cleaner if > all the $release variables were renamed to $releaseid too to match up with > the SQL so I've also done that which meant a whole lot of files have changed > superficially. releaseid is a better name - so this change looks fine to me. > > BTW, when we release v0.9.6, as result of the fix for bug 1209273, we're > going to need to tell people that their existing databases won't work > out-of-the-box with the new version ... Yes, must remember to flag that up in the release notes and announcement. We've broken db compatibility in the past without anyone complaining, so I guess it will be OK. I suppose for extra points we could provide a migration script, but my preference would be to only do that for stable versions (i.e. 1.0 onwards). Malcolm |