Menu
▾
▴
gnuplot-beta
|
From: Mojca M. <moj...@gm...> - 2017-10-09 07:12:11
|
On 9 October 2017 at 08:30, Daniel J Sebald wrote: > On 10/09/2017 12:15 AM, Mojca Miklavec wrote: >> >> They are actually "lost" is CVS already, while they are (and will) >> still (be) in ChangeLog. >> >> What one *can* do with cvs2git conversion is to start with something >> that looks like a reasonable conversion already and then make a map >> >> <commit-shasum> <author> >> >> for all commits where the author doesn't match the committer. Then one >> can run a script that changes the author of each individual commit. > > This is what I was wondering about, i.e., an iterative approach where it > might not be correct on the first pass but then somehow run a script or > program that cross-references comments and adds or removes author info to > keep everything in sync. > > I thought that the git database was encoded such that changing authorship > wouldn't be so easy unless there is a git command that does so. This is one of the reasons why I like git a lot. You can basically do whatever you please with a git repository. In contrast to subversion (and also mercurial ot a large extent), changing the history of a git repository is considered "normal" and there's built-in support to do very strange things. Or, if the support is not built-in, it's "easy" to write a python/perl/bash script to achieve what you want. The main "problem" with assigning the correct author is getting the mapping right. That requires quite some man-hours, I guess. Doing the change on the git end is more or less straightforward. >> PS: w.r.t. the question of what could go wrong with my conversion, >> this is what the "git cvsimport" complains about: >> >> revision 1.3 of file docs/pdffigures.tex is tagged but not present >> revision 1.29 of file src/contour.c is tagged but not present ... >> Skipping #CVSPS_NO_BRANCH I need to add that this comes from "git cvsimport" (which is considered buggy). But just to show a partial answer to "what's wrong with the repo on github". > Hmmm, this probably comes about because of misalignment of changeset > groupings and the converters revision count. That is, the CVS tag numbers > are translated in a way that doesn't agree with the original file versions. > Do all these complaints come at the very end? That is, is the last version > of, say file src/graphics.c, 1.464.2.37? I don't know and I'm not too eager to dig into it as it won't bring anything even if I "debug it". > This makes me think that none of the tags after conversion will be reliable. > That's a pretty crucial issue, i.e., the ability to reconstruct program > versions older than the conversion date/time. Well, one needs to be aware that there is one caveat with cvs -> git conversion. CVS theoretically supports tagging strange combination of files that never existed together as such in the main branch. Getting the exact tag might not always be possible unless one first creates a branch with those exact versions of all files and then makes a tag on that branch. I don't know how Eric's tool handles that. On 9 October 2017 at 08:21, sfeam wrote: > > True. So as long as the original commit dates are retained in some form, > the modified files can still be mapped to the ChangeLog description and > hence the original author. OK. > > Ethan The timestamps are (usually) retained. I'm not 100% sure, but I seem to remember that CVS is not timezone-aware and would accept whatever date the committer has set on his machine. Mojca |
|
From: Daniel J S. <dan...@ie...> - 2017-10-09 18:08:54
|
On 10/09/2017 02:12 AM, Mojca Miklavec wrote:
> On 9 October 2017 at 08:30, Daniel J Sebald wrote:
>> On 10/09/2017 12:15 AM, Mojca Miklavec wrote:
>>>
>>> They are actually "lost" is CVS already, while they are (and will)
>>> still (be) in ChangeLog.
>>>
>>> What one *can* do with cvs2git conversion is to start with something
>>> that looks like a reasonable conversion already and then make a map
>>>
>>> <commit-shasum> <author>
>>>
>>> for all commits where the author doesn't match the committer. Then one
>>> can run a script that changes the author of each individual commit.
>>
>> This is what I was wondering about, i.e., an iterative approach where it
>> might not be correct on the first pass but then somehow run a script or
>> program that cross-references comments and adds or removes author info to
>> keep everything in sync.
>>
>> I thought that the git database was encoded such that changing authorship
>> wouldn't be so easy unless there is a git command that does so.
>
> This is one of the reasons why I like git a lot. You can basically do
> whatever you please with a git repository. In contrast to subversion
> (and also mercurial ot a large extent), changing the history of a git
> repository is considered "normal" and there's built-in support to do
> very strange things. Or, if the support is not built-in, it's "easy"
> to write a python/perl/bash script to achieve what you want.
>
> The main "problem" with assigning the correct author is getting the
> mapping right. That requires quite some man-hours, I guess. Doing the
> change on the git end is more or less straightforward.
As much automation as possible is preferred. A script file to sort
through the changed files and ChangeLog entry might give reasonably good
alignment. Consider the first two examples from the test repository.
The "git log --stat" tells us what files have changed with each changeset:
GIT LOG
-------
commit 0a3035e39a1f9402a474d0ea9300deac4cd9ef33
Author: bbbbbb <bbbbbb>
Date: Fri Oct 6 18:35:09 2017 +0000
Use <sys/wait.h> if available.
Provide centralized fall-back of WEXITSTATUS if <sys/wait.h> does
not supply it.
ChangeLog | 15 +++++++++++++--
configure.ac | 4 +++-
src/command.c | 6 +-----
src/syscfg.h | 20 ++++++++++++++++----
4 files changed, 33 insertions(+), 12 deletions(-)
CHANGELOG
---------
2017-10-06 hhhh bbbbbb <xxxxxx@xxxxxx>
* src/command.c: Move WEXITSTATUS fall-back definition away from here.
* src/syscfg.h: Include <sys/wait.h>, if it exists.
(WEXITSTATUS): Provide fall-back definition, if none in
<sys/wait.h>. Move MS Windows specific replacement from command.c
to here.
* configure.ac: Add call to AC_HEADER_SYS_WAIT
The above is a good match in terms of files that changed and the change
log for that day. Hence, in this case we would substitute Author hhhh
bbbbbb <xxxxxx@xxxxxx> for bbbbbb <bbbbbb>
But then the second example isn't so simple:
GIT LOG
-------
commit 1455f9768f84f061aade6a9d52c29dfd92f621db
Author: mmmmmm <mmmmmm>
Date: Fri Oct 6 07:52:24 2017 +0000
Add menu items to edit gnuplot.ini and wgnuplot.ini
ChangeLog | 6 ++++++
src/win/wgnuplot.mnu | 9 ++++++++-
2 files changed, 14 insertions(+), 1 deletion(-)
commit c5031aa2e31adb4b98569da80f2eb65abd282310
Author: mmmmmm <mmmmmm>
Date: Fri Oct 6 07:43:05 2017 +0000
Build and install support files for lua/tikz
config/msvc/Makefile | 32 ++++++++++++++++++++++++++++++--
1 file changed, 30 insertions(+), 2 deletions(-)
commit 6078650b9396c309b593f0395cb0ef6fa552329a
Author: mmmmmm <mmmmmm>
Date: Fri Oct 6 07:36:50 2017 +0000
Add a note on secure APIs
ChangeLog | 3 +++
config/mingw/Makefile | 4 +++-
2 files changed, 6 insertions(+), 1 deletion(-)
commit b720450b7620be7a5946b851411e49eb8b791521
Author: mmmmmm <mmmmmm>
Date: Fri Oct 6 07:28:39 2017 +0000
Default to Mingw-w64 and Direct2D v1.1
ChangeLog | 4 ++++
config/mingw/Makefile | 6 +++---
2 files changed, 7 insertions(+), 3 deletions(-)
commit 14ed6d64ad2375f33868ae9603fae9ce17f087c0
Author: mmmmmm <mmmmmm>
Date: Fri Oct 6 07:23:19 2017 +0000
Add helpfiles to 'all' target
ChangeLog | 5 +++++
config/mingw/Makefile | 10 +++++-----
2 files changed, 10 insertions(+), 5 deletions(-)
CHANGELOG
---------
2017-10-06 bbbbbb mmmmmm <xxxxxx@xxxxxx>
* config/mingw/Makefile: Add helpfiles to "all" target, including
the japanese version. Remove helpfile from default target.
* config/mingw/Makefile: Default to Mingw-w64 and Direct2D v1.1.
Note that building using Mingw32 currently does not work anyway
due to missing headers libraries for newer Windows APIs.
* config/mingw/Makefile: Add a note that secure APIs are required
(pointed out by Allin Cottrell on the mailing list).
* config/msvc/Makefile: Build and install support files for the
lua/tikz terminal.
src/win/wgnuplot.mnu: Add menu items to edit gnuplot.ini and
wgnuplot.ini to 'Help' menu.
So, this second example is sort of a series of commits in which
ChangeLog entry was constructed piecemeal over time. In that case, we
probably wouldn't substitute bbbbbb mmmmmm <xxxxxx@xxxxxx> for mmmmmm.
But in all likelihood it is the maintainer who is building the changes
and ChangeLog entry over the course of a day in this way, so the Author
should be correct already.
And I'm going to guess that, generally, when Ethan has put together a
ChangeLog entry for contributed work from a contributed patch, he pretty
much does it as one CVS "commit" (i.e., more like the first example
above). He does go back and change typos and such after the fact, but
such clean-ups can be left as is and attributed to Ethan.
So, if one were to write a script that goes through the "git log --stat"
and whenever on a matching day there is a "ball of files" in a commit
that matches to 90% a "ball of files" in a ChangeLog, substitute the
Author info, otherwise leave the maintainer as Author; might that be
good enough?
>> Hmmm, this probably comes about because of misalignment of changeset
>> groupings and the converters revision count. That is, the CVS tag numbers
>> are translated in a way that doesn't agree with the original file versions.
>> Do all these complaints come at the very end? That is, is the last version
>> of, say file src/graphics.c, 1.464.2.37?
>
> I don't know and I'm not too eager to dig into it as it won't bring
> anything even if I "debug it".
I think doing a parallel reconstruction of a tagged version in both CVS
and git (or SVN) with the idea in mind of making them file-by-file exact
could lead to the most insight as far as alignment. Those tags being
correct is actually a little higher priority than having the exact
contribution attributes by user per commit.
>> This makes me think that none of the tags after conversion will be reliable.
>> That's a pretty crucial issue, i.e., the ability to reconstruct program
>> versions older than the conversion date/time.
>
> Well, one needs to be aware that there is one caveat with cvs -> git conversion.
>
> CVS theoretically supports tagging strange combination of files that
> never existed together as such in the main branch. Getting the exact
> tag might not always be possible unless one first creates a branch
> with those exact versions of all files and then makes a tag on that
> branch. I don't know how Eric's tool handles that.
So, when a release is done currently, are a collection of files grouped
into an archive file and that's the official release, sitting on a
server somewhere? It's not the tagged files in the repository that is
considered the release?
Dan
|
|
From: Petr M. <mi...@ph...> - 2017-10-09 20:59:32
|
> As much automation as possible is preferred. Few impressions: If someone wants to look to a patch description of the (current) cvs period, then just look into the ChangeLog, thus slight misalignments in local description is not a big issue. Thanks for explaing why it is desperate to search for ChangeLog's in some non-cvs projects :-) I see there is no cvs command to list all available branches; it is necessary to look into version.c,h, for example. That's nice that git has such a command. Looking to a calendar, 2017-10-17 may be a nice date for the switch :-) Getting the cvs repository backup, it has "just" 183 MB complete and 18 MB compressed, with the oldiest entry in April 1998 (import from beta340). Good job for 20 years, right? --- Petr Mikulik |
|
From: Hans-Bernhard B. <HBB...@t-...> - 2017-10-09 19:54:55
|
Am 09.10.2017 um 07:15 schrieb Mojca Miklavec: > PS: w.r.t. the question of what could go wrong with my conversion, > this is what the "git cvsimport" complains about: > [...] ESR's tool, cvs-fast-import, which "man git-cvsimport" itself recommends to be used instead, shows none of those messages, so I believe that these can be fixed by a change towards cvs-fast-import. I've tried out reposurgen on our repository (with the "faq" module removed), and it seems to work almost perfectly. "make allcompare" showed nothing missing from the git version. The original import pseudo-branch GNUPLOT_BETA is, correctly, dropped, all other tags and branch tips compare equal, and .cvsignore files are also taken care of. All in all, I'm quite impressed by this tool. |
|
From: Hans-Bernhard B. <HBB...@t-...> - 2017-10-09 20:05:40
|
Am 09.10.2017 um 20:08 schrieb Daniel J Sebald: > The above is a good match in terms of files that changed and the change > log for that day. Hence, in this case we would substitute Author hhhh > bbbbbb <xxxxxx@xxxxxx> for bbbbbb <bbbbbb> It's also kind of a non-example, given that with an authormap file in place, there would be nothing left to substitute: i.e. my full name and address would already be in git log. [...] > I think doing a parallel reconstruction of a tagged version in both CVS > and git (or SVN) with the idea in mind of making them file-by-file exact > could lead to the most insight as far as alignment. Those tags being > correct is actually a little higher priority than having the exact > contribution attributes by user per commit. reposurgeon will do these checks automatically, and in my trial run there were essentially no differences found. > So, when a release is done currently, are a collection of files grouped > into an archive file and that's the official release, sitting on a > server somewhere? It's not the tagged files in the repository that is > considered the release? They both are. It's basically a required step while making a release to 1) check in what needs to be, 1a) repeat 1) until "make distcheck" finishes without errors 2) tag that state from your local working copy 3) check out a clean copy at that tag 4) "make dist" in there, too 5) verify that both tarballs are effectively identical. |
|
From: Daniel J S. <dan...@ie...> - 2017-10-09 21:14:50
|
On 10/09/2017 03:05 PM, Hans-Bernhard Bröker wrote:
> Am 09.10.2017 um 20:08 schrieb Daniel J Sebald:
>
>
>> The above is a good match in terms of files that changed and the
>> change log for that day. Hence, in this case we would substitute
>> Author hhhh bbbbbb <xxxxxx@xxxxxx> for bbbbbb <bbbbbb>
>
> It's also kind of a non-example, given that with an authormap file in
> place, there would be nothing left to substitute: i.e. my full name and
> address would already be in git log.
Yes, in this case the Author info is replace with the same exact info.
But I was thinking generally. I.e., most non-maintainer contributions
would also be a group of files. Ethan would have the most insight on
how this is characterized. The problem is that contributions which only
change a single file could, with higher probability, be on a day when
someone else made a modification to the same file.
BUT, I may have hit upon a more reliable and straightforward approach
from an automated script standpoint. It's based on what I pointed out
about the ChangeLog file typically being listed in the diff hunks.
Experiment with the following git command:
git log -p --unified=20 ChangeLog
That gives us the log record for the ChangeLog, printing out the diff
hunks in the ChangeLog and with enough context lines such that it
displays back to the Date/Contributor/Address line. (Use --unified=30
if one thinks there could be that length of comment.)
With such a record, the script can search for the changes in the file
(i.e., "+" as character in the first location of the line), and then
search backward for the Date/Contributor/Address info. With a few
heuristic rules, this might be rather accurate, in theory. Let's look
at some examples.
Here's one that's pretty simple, the Author info is the first modified line:
-------------------------
commit 87ca52fd56cf60be023642ad2664390affd5d896
Author: sfsfsf <sfsfsf>
Date: Mon Sep 4 05:57:37 2017 +0000
Use AC_MSG_RESULT rather than AC_MSG_WARN
diff --git a/ChangeLog b/ChangeLog
index f771b54..8cc8ab9 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,20 +1,25 @@
+2017-09-03 Daniel J Sebald <xxxxxx@xxxxxx>
+
+ * configure.ac: Use AC_MSG_RESULT rather than AC_MSG_WARN to report
+ which Qt version will be used.
+
2017-09-03 Ethan Mmmm <xxxxxx@xxxxxx>
* src/time.c (xstrftime): The variant format specs for time
%tH %tM did not behave as documented in that hours wrapped at 24 and
minutes at 60 if the decimal precision modifier was missing.
2017-09-02 Ethan Mmmm <xxxxxx@xxxxxx>
* src/axis.c (parse_range eval_link_function) docs/gnuplot.doc:
Attempt to handle the case of linked axes (x+x2 or y+y2) and an in-line
range specifier in the plot statement. This fix only addresses simple
cases, such as
set link x2; plot [x=min:max] something-using-x2 axes x2y1
Recommend to use separate "set xrange ... set yrange ..." instead.
2017-09-01 Bastian Mmmm <xxxxxx@xxxxxx>
* src/win/winmain.c (ConsoleHandler): Install the console handler
also for wgnuplot and wgnuplot_pipes. Avoids segfaults when closing
the wgnuplot_pipes console window or the caca terminal console window.
-------------------------
How about the scenario of my previous post where the commits are a
series of small changes where the ChangeLog entry is gradually assembled
piecemeal?
-------------------------
commit 1455f9768f84f061aade6a9d52c29dfd92f621db
Author: mmmmmm <mmmmmm>
Date: Fri Oct 6 07:52:24 2017 +0000
Add menu items to edit gnuplot.ini and wgnuplot.ini
diff --git a/ChangeLog b/ChangeLog
index 9fe292c..45b8ac3 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,32 +1,38 @@
2017-10-06 Bastian Mmmm <xxxxxx@xxxxxx>
* config/mingw/Makefile: Add helpfiles to "all" target, including
the japanese version. Remove helpfile from default target.
* config/mingw/Makefile: Default to Mingw-w64 and Direct2D v1.1.
Note that building using Mingw32 currently does not work anyway
due to missing headers libraries for newer Windows APIs.
* config/mingw/Makefile: Add a note that secure APIs are required
(pointed out by Allin Cottrell on the mailing list).
+ * config/msvc/Makefile: Build and install support files for the
+ lua/tikz terminal.
+
+ src/win/wgnuplot.mnu: Add menu items to edit gnuplot.ini and
+ wgnuplot.ini to 'Help' menu.
+
2017-10-05 Bastian Mmmm <xxxxxx@xxxxxx>
* src/win/wd2d.cpp: Enable color font support. This enables colored
emojis, which can be used e.g. as point symbols. Due to the default
font-fallback to "Segoe UI Emoji", this might lead to unexpected
results if a plot relied on non-colored character fallbacks.
2017-10-03 Ethan Mmmm <xxxxxx@xxxxxx>
* term/gd.trm: Report number of frames in completed animation sequence.
2017-10-01 Bastian Mmmm <xxxxxx@xxxxxx>
* src/win/winmain.c|h src/command.c: For wgnuplot, open or attach to a
console when executing system commands so we can its the output. That
in my option eliminates the last benefit of wgnuplot_pipes over
wgnuplot. Note that the process terminates if the new console is
closed, just as is the case with wgnuplot_pipes, but the console is
only opened when actually required.
-------------------------
Notice in the above how having enough context lines shows us the Author
information (searching backward from the first changed line in the
ChangeLog file) even though Bastian appended to the end of the previous
ChangeLog entry.
Does that approach seem fairly reliable and easy to automate?
>> I think doing a parallel reconstruction of a tagged version in both
>> CVS and git (or SVN) with the idea in mind of making them file-by-file
>> exact could lead to the most insight as far as alignment. Those tags
>> being correct is actually a little higher priority than having the
>> exact contribution attributes by user per commit.
>
> reposurgeon will do these checks automatically, and in my trial run
> there were essentially no differences found.
That's good. To me, this seemed like the most "surgical" work.
Dan
|
|
From: Daniel J S. <dan...@ie...> - 2017-10-10 07:58:11
|
On 10/09/2017 04:14 PM, Daniel J Sebald wrote: > BUT, I may have hit upon a more reliable and straightforward approach > from an automated script standpoint. It's based on what I pointed out > about the ChangeLog file typically being listed in the diff hunks. > Experiment with the following git command: > > git log -p --unified=20 ChangeLog > > That gives us the log record for the ChangeLog, printing out the diff > hunks in the ChangeLog and with enough context lines such that it > displays back to the Date/Contributor/Address line. (Use --unified=30 > if one thinks there could be that length of comment.) > > With such a record, the script can search for the changes in the file > (i.e., "+" as character in the first location of the line), and then > search backward for the Date/Contributor/Address info. With a few > heuristic rules, this might be rather accurate, in theory. Mojca, HBB, I've written a short C++ utility to extract the authorship info from the ChangeLog diff hunks per changeset and posted the code here: https://sourceforge.net/p/gnuplot/patches/763/ along with a file containing the g++ command to compile it. Once built, type git_changelog_author --help for a description of how to use it. Basically git log -p --unified=50 ChangeLog > ChangeLog.diff git_changelog_author ChangeLog.diff > author.txt will give a condensed list of git ID and authorship information, whitespace reduce to a single character and no whitespace within the email address. Give it a try. It's very fast. Commit the program to the repository in some utility directory, if you like, then we can tweak the code for your use. Or, just post new versions to the patch report. Dan |
|
From: sfeam <sf...@us...> - 2017-10-11 03:57:39
|
On Monday, 09 October 2017 21:54:42 Hans-Bernhard Bröker wrote: > > ESR's tool, cvs-fast-import, which "man git-cvsimport" itself recommends > to be used instead, shows none of those messages, so I believe that > these can be fixed by a change towards cvs-fast-import. > > I've tried out reposurgen on our repository (with the "faq" module > removed), and it seems to work almost perfectly. "make allcompare" > showed nothing missing from the git version. The original import > pseudo-branch GNUPLOT_BETA is, correctly, dropped, all other tags and > branch tips compare equal, and .cvsignore files are also taken care of. > > All in all, I'm quite impressed by this tool. It looks to me that there is general consensus that - we should move to git, - that Eric Raymond's toolset is the best way to do that, and that - if Eric himself is willing to guide the conversion it has the greatest chance of success. So let's do that. There is no obvious consensus on where the repository should live. I'm going to step in and vote to stay with SourceForge for now because I know how to manage their infrastructure for issue trackers and putting out a release. Eric - several people responded with bits and pieces of meta-info that you asked for. Do you have everything you need? Do I understand correctly that the git repository can be created initially anywhere that you find convenient and then replicated to SourceForge afterwards? If not then maybe we do need additional discussion about where to do this. Should we set a freeze date for the existing CVS source [*]? Anything else that needs to be done in preparation? Ethan [*] freeze concerns - 5.2: I want to package up the current 5.2 branch as an incremental release, just so that's out of the way and there is no time pressure after the conversion while we deal with any cleanup or fallout. That takes very little time, but I need to know when to do it. 5.3: So far as I know this is as good a time as any to freeze. The current build is usable as it stands. Older branches: There should not be any activity there anyhow. |
|
From: Clark G. <cga...@vt...> - 2017-10-11 08:00:04
|
I agree with freeze. I suggest that if Eric is willing to perform the final
production migration then he should control that timing (else Mojca,
probably best for you to take the helm?) We may have a few days where the
change is happening, etc that we're frozen (could be just one day, but
giving a little buffer).
After the migration I'll plan to grab a final CVS copy to put into
historical.
Yes one of the really great things about git/if is that repo clone is a
first class citizen, so we can (indeed *will*) have several places the repo
resides. So yes the conversion could happen anywhere. Similarly, merge is
not nearly the horror you would probably associate it with if you've ever
done it with CVS or svn; I still feel my stomach knot up thinking of the
pain unborking a CVS repo following a merge attempt gone bad many years ago.
I'm a little concerned with the longevity of SourceForge, but from our
original discussion of the matter I'll recall the critical decision rule:
support what Ethan wants to work with. :-) Certainly keeping the
environment on SF is the easiest option for me.
[Btw Eric: I *loved* your article on heirloom software and ADVENT last
month in Linux Journal! Thanks for that.]
Regards
Clark
--
Clark Gaylord
cga...@vt...
... Autocorrect may have improved this message
Brevity should not be interpreted as curtness ...
On Oct 10, 2017 23:57, "sfeam" <sf...@us...> wrote:
> On Monday, 09 October 2017 21:54:42 Hans-Bernhard Bröker wrote:
> >
> > ESR's tool, cvs-fast-import, which "man git-cvsimport" itself recommends
> > to be used instead, shows none of those messages, so I believe that
> > these can be fixed by a change towards cvs-fast-import.
> >
> > I've tried out reposurgen on our repository (with the "faq" module
> > removed), and it seems to work almost perfectly. "make allcompare"
> > showed nothing missing from the git version. The original import
> > pseudo-branch GNUPLOT_BETA is, correctly, dropped, all other tags and
> > branch tips compare equal, and .cvsignore files are also taken care of.
> >
> > All in all, I'm quite impressed by this tool.
>
> It looks to me that there is general consensus that
>
> - we should move to git,
>
> - that Eric Raymond's toolset is the best way to do that, and that
>
> - if Eric himself is willing to guide the conversion it has the greatest
> chance of success.
>
> So let's do that.
>
> There is no obvious consensus on where the repository should live.
> I'm going to step in and vote to stay with SourceForge for now
> because I know how to manage their infrastructure for issue trackers and
> putting out a release.
>
> Eric - several people responded with bits and pieces of meta-info that
> you asked for. Do you have everything you need?
>
> Do I understand correctly that the git repository can be created
> initially anywhere that you find convenient and then replicated to
> SourceForge afterwards? If not then maybe we do need additional
> discussion about where to do this.
>
> Should we set a freeze date for the existing CVS source [*]?
> Anything else that needs to be done in preparation?
>
> Ethan
>
>
> [*] freeze concerns -
>
> 5.2: I want to package up the current 5.2 branch as an incremental
> release, just so that's out of the way and there is no time pressure after
> the conversion while we deal with any cleanup or fallout. That takes
> very little time, but I need to know when to do it.
>
> 5.3: So far as I know this is as good a time as any to freeze.
> The current build is usable as it stands.
>
> Older branches: There should not be any activity there anyhow.
>
>
|
|
From: Eric S. R. <es...@th...> - 2017-10-11 13:16:46
|
sfeam <sf...@us...>: > It looks to me that there is general consensus that > > - we should move to git, > > - that Eric Raymond's toolset is the best way to do that, and that > > - if Eric himself is willing to guide the conversion it has the greatest chance of success. > > So let's do that. This is a good time for me to do it. NTPsec 1.0 just shipped and our PM said he didn't want to see any commits for a week. :-) > Eric - several people responded with bits and pieces of meta-info that > you asked for. Do you have everything you need? Not yet. 1. Here's the committer map: janert = Philipp K. Janert <ja...@us...> uid225733 = Ethan Merritt <merritt@u.washington.edu> uid93776 = Ethan Merritt <merritt@u.washington.edu> mikulik = Petr Mikulik <mi...@ph...> markisch = Bastian Maerkisch <bma...@we...> tlecomte = Timothee Lecomte <tim...@en...> persquare = Per Persson <per...@us...> amai = Alexander Mai <am...@us...> lhecking = Lars Hecking <lhe...@us...> cgaylord = Clark Gaylord <cga...@vt...> uid26705 = Petr Mikulik <mi...@ph...> juhaszp = Peter Juhasz <ju...@us...> vanzandt = James R. Van Zandt <van...@us...> sfeam = Ethan Merritt <merritt@u.washington.edu> lodewyck = J´erˆome Lodewyck <lod...@us...> joze = Joze Duhovnik <jo...@us...> Ideally, we'd enhance this in two ways. First, those of you with preferred email addresses should make sure this points at the one you want. Second, we'd add timezone offsets. 2. If you want the authorship data to be right, somebody's going to have to write code to grovel through the ChangeLog(s) and generate date/author/filelist triples. (Then I'll have to write some custom Python to use those.) Though maybe this isn't worth it - the only ChangeLog I can see only seems to cover 1998-2000. 3. There was some talk of gluing to the history old releases that only exist as tarballs. That can be done, but I need those tarballs. 3. I'm not clear on what ought to be excised from the repo. HBBroeker mentions dropping the faq module. Someone else (I think Mojca) had a pre-conversion sript that stripped out some stuff. A set of excisions you guys agree on - ideally expressed as a pre-conversion script modifying the repo - is one thing we need. 4. The following paragraph from HBBroeker worries me a little: > I've tried out reposurgen on our repository (with the "faq" module > removed), and it seems to work almost perfectly. "make allcompare" > showed nothing missing from the git version. The original import > pseudo-branch GNUPLOT_BETA is, correctly, dropped, all other tags and > branch tips compare equal, and .cvsignore files are also taken care of. I didn't use reposurgeon, I used cvsconvert. This is a wrapper script in the reposurgeon distribution that also uses cvs-fast-export as an engine, but is specialized for CVS and does more detailed correctness checking than allcompare. What worries me is that *my* conversion didn't look quite so smooth - it was not clear that GNUPLOT_BETA was dropped, and there were some files on ther gitspace side that weren't in CVS (probably due to a botched CVS delete). This needs to be further investigated. > Do I understand correctly that the git repository can be created > initially anywhere that you find convenient and then replicated to > SourceForge afterwards? That understanding is correct. > Should we set a freeze date for the existing CVS source [*]? That's not important yet. Once I have the conversion process scripted, you can basically choose any time to cut over and it will all get done in a time on the close order of two hours. > Anything else that needs to be done in preparation? For the conversion itself, no. You guys need to make a hosting site decision, then I need to have push and force-push privileges so I can drop the git repo in place. -- <a href="http://www.catb.org/~esr/">Eric S. Raymond</a> My work is funded by the Internet Civil Engineering Institute: https://icei.org Please visit their site and donate: the civilization you save might be your own. |
|
From: Daniel J S. <dan...@ie...> - 2017-10-11 15:50:00
|
On 10/11/2017 08:16 AM, Eric S. Raymond wrote: > sfeam <sf...@us...>: >> It looks to me that there is general consensus that >> >> - we should move to git, >> >> - that Eric Raymond's toolset is the best way to do that, and that >> >> - if Eric himself is willing to guide the conversion it has the greatest chance of success. >> >> So let's do that. > > This is a good time for me to do it. NTPsec 1.0 just shipped and our PM said > he didn't want to see any commits for a week. :-) > >> Eric - several people responded with bits and pieces of meta-info that >> you asked for. Do you have everything you need? > > Not yet. > > 1. Here's the committer map: > [snip] > > Ideally, we'd enhance this in two ways. First, those of you with preferred > email addresses should make sure this points at the one you want. Second, > we'd add timezone offsets. I didn't know timezone offsets for individuals is possible. How is that advantageous in terms of git usage? Is it that when git displays stuff to the user it adjusts date/time? Does that happen in viewers like gitg, "git gui", etc.? > 2. If you want the authorship data to be right, somebody's going to have to > write code to grovel through the ChangeLog(s) and generate > date/author/filelist triples. (Then I'll have to write some custom Python > to use those.) Though maybe this isn't worth it - the only > ChangeLog I can see only seems to cover 1998-2000. I've written a utility that constructs the list you are looking for by examining all the ChangeLog diff hunks after the repository is translated. The code is here https://sourceforge.net/p/gnuplot/patches/763/ and the algorithm is based on searching the diff hunks for the first changed line (i.e., "+" in the first character) after the "+++" of the headers. After finding that first change, it searches backward for the author info of the ChangeLog on that particular version. Providing enough context lines in the diff hunks should allow capturing that authorship line. Example usage is: git log -p --unified=50 ChangeLog > ChangeLog.diff git_changelog_author ChangeLog.diff > author.txt (I suppose I could have written the utility to allow pipe redirection to skip the intermediate file.) Examining the ChangeLog.diff file in an editor should go a long way to understanding how the algorithm works. The output is gitID/Date/Author/Address separated by space character (allowing the Author to have spaces as well). The date information is of no use to reconstruction in the git repository--as I explained in a previous email, any particular ChangeLog entry can be constructed piecemeal over the course of a day. (But the utility should properly assign the same author info to all the incremental changes.) I would think a python script could step through the list and retroactively call some git command to correct the authorship. That would be nice. I hope it doesn't make an entry in the git repository for every authorship change somewhere, though, as there are over 6000 authorship lines. (BTW, those changesets that had no modification to ChangeLog go unrecognized by the utility, but that's fine.) I notice in the list of authorship there are some variations on names/address. For example, Ethan is most often E. M., but often it appears E. A. M. Do we want to make names consistent? Those sort of variations are probably going to happen going forward in git as well, seeing as any particular person might do mods from different computers with slightly different user info configuration. The list of files would come from the git repository changeset itself, but honestly I don't think that's of any benefit without any comments to go along with it. Two reasons: 1) If it is just a list of files, that's already gotten from something like gitg viewer which lists all the files modified in a particular changeset, and 2) conveniently the name "ChangeLog" often being the first in the list puts the diff hunk for the ChangeLog right after the description so we sort of have the info format we want already, e.g., from gitg: broeker <broeker> 10/06/2017 06:35:09 PM +0000 Use if available. Provide centralized fall-back of WEXITSTATUS if does not supply it. Expand all 0a3035e39a1f9402a474d0ea9300deac4cd9ef33 .... .... ▼15 ChangeLog .... .... @@ -1,3 +1,14 @@ 1 + 2017-10-06 Xxxxxx Xxxxxx <xxxxxx@xxxxxx> 2 + 3 + * src/command.c: Move WEXITSTATUS fall-back definition away from here. 4 + 5 + * src/syscfg.h: Include <sys/wait.h>, if it exists. 6 + (WEXITSTATUS): Provide fall-back definition, if none in 7 + <sys/wait.h>. Move MS Windows specific replacement from command.c 8 + to here. 9 + 10 + * configure.ac: Add call to AC_HEADER_SYS_WAIT 11 + 1 12 2017-10-06 Xxxxxx Xxxxxx <xxxxxx@xxxxxx> 2 13 3 14 * config/mingw/Makefile: Add helpfiles to "all" target, including The above works for me, as far as retrieving a more detailed description of the changeset, i.e., no need to consult historical ChangeLog on this one. A discussion worth having (gnuplot group) is what to do with the ChangeLog going forward. Rather than ChangeLog ChangeLog.0 ChangeLog.1 ChangeLog.2 ChangeLog.3 ChangeLog.4 ChangeLog.5 it might be nice to put all those in some file ChangeLog_CVS or ChangeLog_historical--something that indicates this is really no longer an active file and all such info (similarly formatted descriptions) will be in the git changeset comment going forward. Also, leaving said file in the root directory is best because I often use "grep */*" to quickly search for information in the source tree and prefer not all sorts of entries from the ChangeLog appearing in the search list. > 3. There was some talk of gluing to the history old releases that only > exist as tarballs. That can be done, but I need those tarballs. If you were to first create a draft posted somewhere, we could clone the repository and test whether the tags for a particular version will retrieve code that matches the tarball. If so, then we don't need tar balls within the repository, do we (group)? Dan |
|
From: Ethan A M. <sf...@us...> - 2017-10-11 17:04:12
|
On Wednesday, 11 October, 2017 09:16:39 Eric S. Raymond wrote: > sfeam <sf...@us...>: > > It looks to me that there is general consensus that > > > > - we should move to git, > > > > - that Eric Raymond's toolset is the best way to do that, and that > > > > - if Eric himself is willing to guide the conversion it has the greatest chance of success. > > > > So let's do that. > > This is a good time for me to do it. NTPsec 1.0 just shipped and our PM said > he didn't want to see any commits for a week. :-) > > > Eric - several people responded with bits and pieces of meta-info that > > you asked for. Do you have everything you need? > > Not yet. > > 1. Here's the committer map: > > janert = Philipp K. Janert <ja...@us...> > uid225733 = Ethan Merritt <merritt@u.washington.edu> > uid93776 = Ethan Merritt <merritt@u.washington.edu> > mikulik = Petr Mikulik <mi...@ph...> > markisch = Bastian Maerkisch <bma...@we...> > tlecomte = Timothee Lecomte <tim...@en...> > persquare = Per Persson <per...@us...> > amai = Alexander Mai <am...@us...> > lhecking = Lars Hecking <lhe...@us...> > cgaylord = Clark Gaylord <cga...@vt...> > uid26705 = Petr Mikulik <mi...@ph...> > juhaszp = Peter Juhasz <ju...@us...> > vanzandt = James R. Van Zandt <van...@us...> > sfeam = Ethan Merritt <merritt@u.washington.edu> > lodewyck = J´erˆome Lodewyck <lod...@us...> > joze = Joze Duhovnik <jo...@us...> ^^^^ No. this one is not correct. "joze" refers to Johannes Zellner Bastian Maerkisch sent an updated list, which I copy below I do not know which email address Hans-Bernhard Broeker would prefer amai = Alexander Mai <st0...@hr...> Europe/Berlin broeker = Hans-Bernhard Broeker <br...@ph...> Europe/Berlin cgaylord = Clark Gaylord <cga...@vt...> US/Eastern janert = Philipp K. Janert <ja...@ie...> US/Pacific joze = Johannes Zellner <joh...@ze...> juhaszp = Peter Juhasz <ju...@us...> lhecking = Lars Hecking <lhe...@us...> lhecking = Lars Hecking <lhe...@nm...> Europe/Dublin lodewyck = Jérôme Lodewyck <lod...@us...> markisch = Bastian Maerkisch <bma...@we...> Europe/Berlin mikulik = Petr Mikulik <mi...@ph...> Europe/Prague persquare = Per Persson <per...@ma...> sfeam = Ethan A Merritt <merritt@u.washington.edu> US/Pacific tlecomte = Timothee Lecomte <tim...@en...> Europe/Paris vanzandt = James R. Van Zandt <jr...@va...> vanzandt = James R. Van Zandt <jr...@de...> uid26705 = Petr Mikulik <mi...@ph...> uid93776 = Ethan A Merritt <merritt@u.washington.edu> > Ideally, we'd enhance this in two ways. First, those of you with preferred > email addresses should make sure this points at the one you want. Second, > we'd add timezone offsets. I don't think a timezone correction is appropriate. My experience in committing to SourceForge is that the timestamp is recorded in UMT rather than as my local time. So at least for my commits it would be incorrect to apply a shift based on the original location of the author. > 2. If you want the authorship data to be right, somebody's going to have to > write code to grovel through the ChangeLog(s) and generate > date/author/filelist triples. (Then I'll have to write some custom Python > to use those.) Though maybe this isn't worth it - the only > ChangeLog I can see only seems to cover 1998-2000. ChangeLog 2014-08-21 - 2017-10-09 ChangeLog.5 2014-03-15 - 2014-08-21 (yes this one is redundant) ChangeLog.4 2011-11-22 - 2014-08-20 ChangeLog.3 2009-10-18 - 2011-11-22 ChangeLog.2 2006-10-01 - 2009-10-17 ChangeLog.1 2004-04-17 - 2006-10-01 ChangeLog.0 1998-04-09 - 2004-04-15 > 3. There was some talk of gluing to the history old releases that only > exist as tarballs. That can be done, but I need those tarballs. I will Email a copy privately. I leave it to others to respond to points 3 + 4 Ethan > 3. I'm not clear on what ought to be excised from the repo. HBBroeker > mentions dropping the faq module. Someone else (I think Mojca) had > a pre-conversion sript that stripped out some stuff. A set of excisions > you guys agree on - ideally expressed as a pre-conversion script modifying > the repo - is one thing we need. > > 4. The following paragraph from HBBroeker worries me a little: > > > I've tried out reposurgen on our repository (with the "faq" module > > removed), and it seems to work almost perfectly. "make allcompare" > > showed nothing missing from the git version. The original import > > pseudo-branch GNUPLOT_BETA is, correctly, dropped, all other tags and > > branch tips compare equal, and .cvsignore files are also taken care of. > > I didn't use reposurgeon, I used cvsconvert. This is a wrapper script > in the reposurgeon distribution that also uses cvs-fast-export as an > engine, but is specialized for CVS and does more detailed correctness > checking than allcompare. What worries me is that *my* conversion didn't > look quite so smooth - it was not clear that GNUPLOT_BETA was dropped, and > there were some files on ther gitspace side that weren't in CVS (probably > due to a botched CVS delete). This needs to be further investigated. > > > Do I understand correctly that the git repository can be created > > initially anywhere that you find convenient and then replicated to > > SourceForge afterwards? > > That understanding is correct. > > > Should we set a freeze date for the existing CVS source [*]? > > That's not important yet. Once I have the conversion process > scripted, you can basically choose any time to cut over and it > will all get done in a time on the close order of two hours. > > > Anything else that needs to be done in preparation? > > For the conversion itself, no. You guys need to make a hosting site > decision, then I need to have push and force-push privileges so I can drop > the git repo in place. > |
|
From: Eric S. R. <es...@th...> - 2017-10-11 18:24:34
|
Ethan A Merritt <sf...@us...>: > Bastian Maerkisch sent an updated list, which I copy below Saved. > I do not know which email address Hans-Bernhard Broeker would prefer I guess he'll tell us. > vanzandt = James R. Van Zandt <jr...@va...> > vanzandt = James R. Van Zandt <jr...@de...> Can someone find out which he prefers? > > Ideally, we'd enhance this in two ways. First, those of you with preferred > > email addresses should make sure this points at the one you want. Second, > > we'd add timezone offsets. > > I don't think a timezone correction is appropriate. > My experience in committing to SourceForge is that the > timestamp is recorded in UMT rather than as my local time. > So at least for my commits it would be incorrect to apply a shift > based on the original location of the author. You are right. In the conversion process he timezone is *not* used to apply a time shift. Rather, it is used to set the time zone offset that git will use for *display* of the time. This is really just a cosmetic feature; the times are stored internally as UTC and without a per-committer timezone they're displayed as UTC too. > > 2. If you want the authorship data to be right, somebody's going to have to > > write code to grovel through the ChangeLog(s) and generate > > date/author/filelist triples. (Then I'll have to write some custom Python > > to use those.) Though maybe this isn't worth it - the only > > ChangeLog I can see only seems to cover 1998-2000. > > ChangeLog 2014-08-21 - 2017-10-09 > ChangeLog.5 2014-03-15 - 2014-08-21 (yes this one is redundant) > ChangeLog.4 2011-11-22 - 2014-08-20 > ChangeLog.3 2009-10-18 - 2011-11-22 > ChangeLog.2 2006-10-01 - 2009-10-17 > ChangeLog.1 2004-04-17 - 2006-10-01 > ChangeLog.0 1998-04-09 - 2004-04-15 That's odd. The ChangeLog.[0-5] files don't show in my conversion. I will investigate. > > 3. There was some talk of gluing to the history old releases that only > > exist as tarballs. That can be done, but I need those tarballs. > > I will Email a copy privately. OK. -- <a href="http://www.catb.org/~esr/">Eric S. Raymond</a> My work is funded by the Internet Civil Engineering Institute: https://icei.org Please visit their site and donate: the civilization you save might be your own. |
|
From: Hans-Bernhard B. <HBB...@t-...> - 2017-10-11 20:10:29
|
Am 11.10.2017 um 19:01 schrieb Ethan A Merritt: > I do not know which email address Hans-Bernhard Broeker would prefer > > broeker = Hans-Bernhard Broeker <br...@ph... > <mailto:br...@ph...>> Europe/Berlin My physik.rwth-aachen.de address is gone, and has been for a while. I've kept using it as the check-in address out of tradition, mainly. Please use broeker .at. users.sourceforge.net, instead. > > 3. I'm not clear on what ought to be excised from the repo. HBBroeker > > mentions dropping the faq module. Someone else (I think Mojca) had > > a pre-conversion sript that stripped out some stuff. A set of excisions > > you guys agree on - ideally expressed as a pre-conversion script > > modifying the repo - is one thing we need. Using reposurgeon, the 'faq' module is already excluded because its generated Makefile rule for SourceForge inputs only pulls one module at a time, so there's nothing to excise. If anything, this module might best be converted independently, and the resulting git repo plunked into the new gnuplot git repo, e.g. into docs/old. > > there were some files on ther gitspace side that weren't in CVS (probably > > due to a botched CVS delete). This needs to be further investigated. I didn't mention those before because I'm convinced the reconstruction is actually better than the original in this case. There are a date range of old tags/branches (BETA_347_980818 ... BETA_349_990114, GNUPLOT_RELEASE_3_7_0, GNUPLOT_3_7_0_2 to GNUPLOT_3_7_0_3, and GNUPLOT_990126 ... GNUPLOT_990317) for which the reconstructed git repo has a subdirectory docs/ps But the comparison didn't find it in its CVS checkouts. That folder did exist between 1998-07-21 and 1999-03-18, then it was copied to docs/psdoc, and the original dropped. So the archives for these files are now in gnuplot/docs/Attic/ps/Attic/*,v Looks like CVS can't really cope with subdirectories inside an Attic. FWIW, cvs2git also rejects this: Directory cvs/gnuplot/docs/Attic/ps found within Attic; ignoring cvsconvert has effectively the same result: git tags/branches from that date range report five gitspace-only files in that folder because CVS fails to express files that should have been there. |
|
From: Eric S. R. <es...@th...> - 2017-10-12 10:57:28
|
Hans-Bernhard Bröker <HBB...@t-...>: > Am 11.10.2017 um 19:01 schrieb Ethan A Merritt: > > >I do not know which email address Hans-Bernhard Broeker would prefer > > > >broeker = Hans-Bernhard Broeker <br...@ph... > ><mailto:br...@ph...>> Europe/Berlin > > My physik.rwth-aachen.de address is gone, and has been for a while. I've > kept using it as the check-in address out of tradition, mainly. > > Please use broeker .at. users.sourceforge.net, instead. Done. > > > 3. I'm not clear on what ought to be excised from the repo. HBBroeker > > > mentions dropping the faq module. Someone else (I think Mojca) had > > > a pre-conversion sript that stripped out some stuff. A set of excisions > > > you guys agree on - ideally expressed as a pre-conversion script > > >modifying the repo - is one thing we need. > > Using reposurgeon, the 'faq' module is already excluded because its > generated Makefile rule for SourceForge inputs only pulls one module at a > time, so there's nothing to excise. If anything, this module might best be > converted independently, and the resulting git repo plunked into the new > gnuplot git repo, e.g. into docs/old. OK, this can be done. I have successfully fetched the faq module > > > there were some files on ther gitspace side that weren't in CVS (probably > > > due to a botched CVS delete). This needs to be further investigated. > > I didn't mention those before because I'm convinced the reconstruction is > actually better than the original in this case. That is a happy accident. >cvsconvert gives me a state of the converted repository like this: > > > $ git status > > HEAD detached at pre-pm3d-11 > > nothing to commit, working tree clean > >Back when that branch was current, there were indeed no log-rotated >ChangeLogs yet. > > git checkout master > >makes ChangeLog.[0-6] appear. Yes, it does. It sounds, then, as though there are zero problems with the cvs-fast-export conversion itself as it is. The contributor map is complete; the author map still needs to be merged. I still need the ancient-release tarballs. -- <a href="http://www.catb.org/~esr/">Eric S. Raymond</a> My work is funded by the Internet Civil Engineering Institute: https://icei.org Please visit their site and donate: the civilization you save might be your own. |
