gnuplot-beta

Another git question

[Ethan A M. <sf...@us...>]

Here's part of the problem I'm having with tracking ChangeLog in git.

Suppose that a git format-patch file contains a ChangeLog entry that is
supposed to go at the top of the current ChangeLog file.  Like this:

diff --git a/ChangeLog b/ChangeLog
index 9ca7eee..285597c 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -0,0 +1,4 @@
+2017-11-17  Ethan A Merritt  <merritt@u.washington.edu>
+
+	git does not like this diff format
+


The entire patch file containing this piece is rejected by "git apply"
or "git am", apparently because it does not like the syntax "@@ -0,0"
which diff/patch use to indicate "top of file".

I can apply the patch manually using "patch" and then run "git commit",
but this loses the commit message and author information that was in
the original formatted patch.

Is there some config option for "git am" and/or "git apply" that tells
it to accept this syntax in a patch file?

Note: "patch" will also accept "@@ -1,0" rather than "@@ -0,0" but
git doesn't like this form either.

	Ethan

Re: Another git question

[Dima K. <gn...@di...>]

Ethan A Merritt via gnuplot-beta <gnu...@li...> writes:

> The entire patch file containing this piece is rejected by "git apply"
> or "git am", apparently because it does not like the syntax "@@ -0,0"
> which diff/patch use to indicate "top of file".

-0,0 doesn't just indicate "top of file". It indicates that the file was
empty prior to this patch. Was that the case here?


> I can apply the patch manually using "patch" and then run "git commit",
> but this loses the commit message and author information that was in
> the original formatted patch.
>
> Is there some config option for "git am" and/or "git apply" that tells
> it to accept this syntax in a patch file?

Don't know off the top of my head. Generall 'git am' is used as the
complement to 'git format', but that's not what you're doing. If you
just did a 'git format' and then a 'git am' on the tree at the same
revision, it MUST work. Otherwise you found a bug. I'm wondering if an
empty ChangeLog file was committed when you did a 'git format', but no
ChangeLog was committed at all on your 'git am' side. Is that it?

Re: Another git question

[Achim G. <Str...@ne...>]

Ethan A Merritt via gnuplot-beta writes:
> Unfortunately (4) will never work as-is because the ChangeLog is
> very different between the two branches.  On the other hand I know
> that the modification to the ChangeLog is always at the top,
> so I edit the formatted patch as I showed so that the ChangeLog diffs
> refer specifically to the top of the file.

There are specialized merge drivers for ChangeLog files and similar
things you can tell git to use that use those peculiarities to produce
less merge conflicts.

> Is there some totally different recommended way to replicate
> changes previously made to one branch and apply them to a 
> different branch?

What's wrong with cherry-pick, specifically the form that uses a range
expression for the commits?  If the changes are already in the repo, I
never bother with git format-patch / am.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: Another git question

[Tait <gnu...@t4...>]

Ethan A Merritt via gnuplot-beta <gnu...@li...> said (on 2017/11/17):
> Here's part of the problem I'm having with tracking ChangeLog in git.
> 
> Suppose that a git format-patch file contains a ChangeLog entry that is
> supposed to go at the top of the current ChangeLog file.  Like this:
> 
> diff --git a/ChangeLog b/ChangeLog
> index 9ca7eee..285597c 100644
> --- a/ChangeLog
> +++ b/ChangeLog
> @@ -0,0 +1,4 @@
> +2017-11-17  Ethan A Merritt  <merritt@u.washington.edu>
> +
> +	git does not like this diff format
> +
> 
> The entire patch file containing this piece is rejected by "git apply"
> or "git am", apparently because it does not like the syntax "@@ -0,0"
> which diff/patch use to indicate "top of file".
> 
> I can apply the patch manually using "patch" and then run "git commit",
> but this loses the commit message and author information that was in
> the original formatted patch.
> 
> Is there some config option for "git am" and/or "git apply" that tells
> it to accept this syntax in a patch file?
> 
> Note: "patch" will also accept "@@ -1,0" rather than "@@ -0,0" but
> git doesn't like this form either.
> 
> 	Ethan

How was this patch-file generated?

Git refuses to apply a patch with no context, and usually does
not generate one. So in the ChangeLog example, the patch,
whether generated by "git diff" or "git format-patch" would
normally be generated with three context lines, and look
something like:

diff --git a/ChangeLog b/ChangeLog
index 9ca7eee..285597c 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,7 @@
+2017-11-17  Ethan A Merritt  <merritt@u.washington.edu>
+
+ git does not like this diff format
+
 ===============================================================================
 If you are seeing this you must have obtained a snapshot of gnuplot development
 from the development cvs repository on sourceforge.net.  This is out of date!

You can reduce it down to a single line of context, and the
patch will still apply:

diff --git a/ChangeLog b/ChangeLog
index 9ca7eee..285597c 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,1 +1,5 @@
+2017-11-17  Ethan A Merritt  <merritt@u.washington.edu>
+
+ git does not like this diff format
+
 ===============================================================================

But using zero lines of context, as in "@@ -0,0 +1,4 @@", does
not apply. Even generating the patch with "git format-patch -U0"
and applying it verbatim on the same commit it was generated
from, will fail. There is an option to git-apply,
"--unidiff-zero", to force application even with no context, but
it does not appear that this option has been extended to git-am.

If you are adding a new file, then obviously there is no context
to provide. In this case, git uses the special "old" filename of
/dev/null as an indicator that there is no prior context, and
this seems to be required. Example:

diff --git a/ChangeLog b/ChangeLog
new file mode 100644
index 9ca7eee..285597c 100644
--- /dev/null
+++ b/ChangeLog
@@ -0,0 +1,4 @@
+2017-11-17  Ethan A Merritt  <merritt@u.washington.edu>
+
+ git does not like this diff format
+

You'll notice it also adds the "new file mode" above the index
line, but the mode line does not seem to be strictly required
for the patch to apply.

Re: Another git question

[Ethan A M. <sf...@us...>]

On Friday, November 17, 2017 1:39:17 PM PST you wrote:
> Ethan A Merritt via gnuplot-beta <gnu...@li...> writes:
> 
> > The entire patch file containing this piece is rejected by "git apply"
> > or "git am", apparently because it does not like the syntax "@@ -0,0"
> > which diff/patch use to indicate "top of file".
> 
> -0,0 doesn't just indicate "top of file". It indicates that the file was
> empty prior to this patch. Was that the case here?

-0,0 +1,4 means replace 0 lines starting at line 0 with 4 lines starting at line 1.
This is true whether or not the file was originally empty.
-1,0 +1,4 means the same except that it fails or warns if applied against
an empty file.

> > I can apply the patch manually using "patch" and then run "git commit",
> > but this loses the commit message and author information that was in
> > the original formatted patch.
> >
> > Is there some config option for "git am" and/or "git apply" that tells
> > it to accept this syntax in a patch file?
> 
> Don't know off the top of my head. Generall 'git am' is used as the
> complement to 'git format', but that's not what you're doing. If you
> just did a 'git format' and then a 'git am' on the tree at the same
> revision, it MUST work.

The point of the operation is that it is not at the same revision.
What I'm trying to do is convert my usual workflow into git-speak.
New features are developed and tested against the development branch.
If they pan out, I go back and apply them to the stable branch.

I figured that in git this is equivalent to
1) git checkout origin
2)  git format-patch --start-number 1 XXX..YYY
where XXX was the commit in the development branch just before
addition of the feature in question and YYY the commit just after
3)  git checkout branch-5-2-stable
4)  git am < 0*.patch

Unfortunately (4) will never work as-is because the ChangeLog is
very different between the two branches.  On the other hand I know
that the modification to the ChangeLog is always at the top,
so I edit the formatted patch as I showed so that the ChangeLog diffs
refer specifically to the top of the file. 
The equivalent has always worked for me in the past for patches
generated by "diiff".  And it still works with the output from
"git format-patch"  so long as I apply the patch with "patch" rather than
with "git".  

The closest I have come is

  git am <0*.patch    # fails with error about patch format
  patch --no-backup < 0*patch
  git stage --all          # register the patched files in the staging area
  git am --continue  # works if I'm lucky but not always

Is there some totally different recommended way to replicate
changes previously made to one branch and apply them to a 
different branch?

	Ethan

Re: Another git question

[Tait <gnu...@t4...>]

> ...
> What I'm trying to do is convert my usual workflow into git-speak.
> New features are developed and tested against the development branch.
> If they pan out, I go back and apply them to the stable branch.
> ...
> Is there some totally different recommended way to replicate
> changes previously made to one branch and apply them to a 
> different branch?

You want to use merge as much as possible, but even when that
doesn't work, I think you'll find "cherry-pick" is a much easier
choice than format-patch/am.

What I'd try to do, were I you, is branch each new feature off
the development branch. The feature-branch can be developed and
thrown away or merged back into the development branch. At some
point the development branch looks okay, and it gets merged back
into master. Rinse, repeat.

Implied in this flow is that everything that ultimately survives
on the development branch wants to be part of master. I don't
know how true that is.

If you want to selectively choose commits to copy from the
development branch to master, then check out master (the
destination branch), open up gitk to identify the commits you
want, and "git cherry-pick <sha1>" to copy those commits from
where they are onto master. This is essentially a one-commit
"rebase" operation. It's less preferred than merge, because you
have two different SHA1 that reference the same changes in
different places now, and that's more potential for conflict in
a future merge, although git is pretty good about recognizing
it's the same thing. But in any case, it's much easier than the
round-trip to *.patch and back when you already have the source
data in the repository.

The cherry-pick might conflict, especially with the ChangeLog.
Git will add the usual conflict markers, but it should be easy
to resolve, then "git add" the ChangeLog and "git commit" will
be pre-populated with the author/commit information. (I think
there's a "cherry-pick --abort" to back up and pretend you were
never doing the cherry-pick in the first place.)

Re: Another git question

[Ethan A M. <sf...@us...>]

On Friday, November 17, 2017 2:38:58 PM PST Tait wrote:
> > ...
> > What I'm trying to do is convert my usual workflow into git-speak.
> > New features are developed and tested against the development branch.
> > If they pan out, I go back and apply them to the stable branch.
> > ...
> > Is there some totally different recommended way to replicate
> > changes previously made to one branch and apply them to a 
> > different branch?
> 
> You want to use merge as much as possible, but even when that
> doesn't work, I think you'll find "cherry-pick" is a much easier
> choice than format-patch/am.

Thanks for the pointer.  I will look into that next.

> What I'd try to do, were I you, is branch each new feature off
> the development branch. The feature-branch can be developed and
> thrown away or merged back into the development branch. At some
> point the development branch looks okay, and it gets merged back
> into master. Rinse, repeat.

That would be a drastically different work flow.

> Implied in this flow is that everything that ultimately survives
> on the development branch wants to be part of master. I don't
> know how true that is.

Under the model we have been using, most of the changes to the
development branch never appear in the current stable branch. 
Instead at some point we create a new stable branch and essentially
freeze it except for bugfixes.  OK that's a bit of an overstatement. 
Some relatively self-contained features can be added to the current
stable branch, but normally only after they've been in the
development branch for a while to shake out problems.
That's the part I'm trying to figure out right now. 

Are you thinking there is (or could be) a "master" branch that is
different from "development"?  What would that gain us?

> If you want to selectively choose commits to copy from the
> development branch to master, then check out master (the
> destination branch), open up gitk to identify the commits you
> want, and "git cherry-pick <sha1>" to copy those commits from
> where they are onto master. 

There is no separate "master".  Should there be?

> This is essentially a one-commit
> "rebase" operation. It's less preferred than merge, because you
> have two different SHA1 that reference the same changes in
> different places now, and that's more potential for conflict in
> a future merge, although git is pretty good about recognizing
> it's the same thing. But in any case, it's much easier than the
> round-trip to *.patch and back when you already have the source
> data in the repository.

This is all new to me, but I think that the merge you describe is in
a sense the  opposite of what I want to do.
I don't want to take (cherry-pick?) commits from a separate branch
and merge them into what will be the main trunk going forward. 
Instead I want to copy a few commits that are already part of the
main trunk and graft them back onto an old branch that will never
be merged.  Am I describing this correctly?

The man page for git-cherry-pick does not show any examples
of this, or at least I don't recognize how the examples it does show
would apply to this case.

> The cherry-pick might conflict, especially with the ChangeLog.
> Git will add the usual conflict markers, but it should be easy
> to resolve,  

Hmm. What I've seen so far is that if it fails I don't get conflict
markers.  I get error messages like "index match failed" and a
dump of the original patch.  But I've tried enough things at this
point that they blur together in my mind. 

> then "git add" the ChangeLog and "git commit" will
> be pre-populated with the author/commit information. (I think
> there's a "cherry-pick --abort" to back up and pretend you were
> never doing the cherry-pick in the first place.)

I know how to edit + add  + commit + re-edit + commit --amend
to fix up the conflicts. 
I was kind of hoping there were git tricks to make it easier. 

	Ethan

Re: Another git question

[Tait <gnu...@t4...>]

Ethan A Merritt <sf...@us...> said (on 2017/11/17):
> Are you thinking there is (or could be) a "master" branch that is
> different from "development"?  What would that gain us?

Sorry, I should have gone to look at the branch structure of the
existing repository on SF. "Master" is normally the main branch,
whatever that means. In my head, I was imagining that master was
the current release branch based on what you'd written before.

Having cloned https://git.code.sf.net/p/gnuplot/git-trunk now,
I'll try to use the branch names that are actually there. What
I see is a master branch, and *-stable branches corresponding to
each release, so "master" is the development branch, it looks
like. (Correct me if I'm wrong.)

> Under the model we have been using, most of the changes to the
> development branch never appear in the current stable branch. 
> Instead at some point we create a new stable branch and essentially
> freeze it except for bugfixes.  OK that's a bit of an overstatement. 
> Some relatively self-contained features can be added to the current
> stable branch, but normally only after they've been in the
> development branch for a while to shake out problems.
> That's the part I'm trying to figure out right now. 

When you say "development branch" I think (now) that you mean
master. Then, as with the branch-5-2-stable, you branch from
master and changes unique to 5.2 go on that branch. If some bug
fix on 5.2 also affects master, how does that fix get from 5.2
back to master? What kinds of things go onto branch-5-2-stable
that shouldn't go back to master?

> This is all new to me, but I think that the merge you describe is in
> a sense the  opposite of what I want to do.

Any of what I describe is bidirectional. Git does not care
whether you merge or cherry-pick from stable to development, or
from development to stable. Cherry-pick is really nothing more
than "copy and paste this commit". The only thing to keep
straight is that cherry-pick will "paste" the commit onto
whatever is currently checked out when you issue the command.

Re: Another git question

[Dima K. <gn...@di...>]

Ethan A Merritt via gnuplot-beta <gnu...@li...> writes:

> The point of the operation is that it is not at the same revision.
> What I'm trying to do is convert my usual workflow into git-speak.
> New features are developed and tested against the development branch.
> If they pan out, I go back and apply them to the stable branch.

I don't have time to do a lot of typing right now, but I'll answer the
actual question.

You don't want to deal with patches to do this; you COULD, but it's more
trouble than it's worth.

If you have two different trees (accessible as branches, tags, on a
remote somehere, whatever), you can pull in individual arbitrary commits
with 'git cherry-pick', or pull in whole chunks of the tree with 'git
rebase' or 'git rebase --onto' if you need to be fancy. Clearly these
operations can fail due to a conflict. It'll then stop and tell you what
to do.

I'll type more tomorrow if you don't get it going by then.

Re: Another git question

[sfeam <sf...@us...>]

On Friday, 17 November 2017 16:55:29 Dima Kogan wrote:
> Ethan A Merritt via gnuplot-beta <gnu...@li...> writes:
> 
> > The point of the operation is that it is not at the same revision.
> > What I'm trying to do is convert my usual workflow into git-speak.
> > New features are developed and tested against the development branch.
> > If they pan out, I go back and apply them to the stable branch.
> 
> I don't have time to do a lot of typing right now, but I'll answer the
> actual question.
> 
> You don't want to deal with patches to do this; you COULD, but it's more
> trouble than it's worth.
> 
> If you have two different trees (accessible as branches, tags, on a
> remote somehere, whatever), you can pull in individual arbitrary commits
> with 'git cherry-pick', or pull in whole chunks of the tree with 'git
> rebase' or 'git rebase --onto' if you need to be fancy. Clearly these
> operations can fail due to a conflict. It'll then stop and tell you what
> to do.
> 
> I'll type more tomorrow if you don't get it going by then.

I think I've got it now.
The ChangeLog file is still a problem, but yes 'get cherry-pick'
takes fewer steps than what I was doing before.  

The key was realizing that it was OK to specify commit id's that
were not currently showing in the display, since a different
branch was selected.

   thanks,

		Ethan

Re: Another git question

[Tait <gnu...@t4...>]

sfeam <sf...@us...> said (on 2017/11/18):
> ...
> The key was realizing that it was OK to specify commit id's that
> were not currently showing in the display, since a different
> branch was selected.

I almost always invoke gitk as "gitk --all" so that I can see
all the branches instead of just what's currently checked-out.