#340 project history

[Lee Worden]

We have Special:RecentChanges to list the edit history of the whole wiki, action=history to list the history of a single page, and Special:Contributions for a given user's edits, but we will also be wanting a way to review the changes to all the files in a project.

[Lee Worden]

This will probably share code with

https://sourceforge.net/tracker/?func=detail&aid=3178719&group_id=366300&atid=1527388
past revisions of projects

and

https://sourceforge.net/tracker/?func=detail&aid=3178714&group_id=366300&atid=1527388
git checkout / checkin

[Anonymous]

This and the git problem have especially a lot in common: they both require a chronological list of changes to the project and its files. Special:RecentChangesInProject may include more events:

• adding a source file
• removing a source file

• updating a source file's contents

• adding an archived project file

• removing an archived project file

• updating an archived project file

• adding a regular project-file tag to a page

• removing a regular project-file tag from a page

• editing project attributes (whether to use standard makefiles, prerequisite projects, etc.)

The git export will use a subset of these - primarily the source files and maybe archived project files.

Anyway, there should be a single function to fetch this chronology in some simple data structure that both features can use.

The past revisions of projects issue just requires the state of the project at one past time, which is a single time-slice of the above data. The chronological reconstruction requires knowing the state of the project at each moment along the way, so there might be some code sharing there as well.

[Lee Worden]

Actually the granularity of events is a little odd, for instance adding a source file often involves two events: first adding it to a wiki page and then adding it to the project description (or vice versa). In this case, the revision that adds it to the wiki page won't be listed because it isn't part of the project yet.

But I also notice there are two ways for a file to be removed from a project: unlisting in the project description and removing from its wiki page. Either ought to do the trick from Git's point of view, but they'll be considered separate events from the special page's point of view.

I was thinking of a structure like:

{ ("add source file", "fnug.pl", 201),
  ("edit source file", "fnug.pl", 203),
  ("edit project description", null, 205),
  ...
}

but it seems messy because sometimes "add source file" will refer to a revision that changes a wiki page and sometimes to a revision that changes a project description? Maybe only "edit project description" should refer to revisions to project descriptions, and any time such an edit includes other kinds of changes they should be listed separately? If so, I can repeatedly add and remove a source file, and each time it is added I'll be referring back to the same earlier revision of the page where it's stored. This is odd, but maybe correct? When removing a file, I guess I could always refer to the current rev of the page where it dwells, in the same way. (If the project lists it but it isn't actually there, then removing it from the PD isn't actually removing it, it's just editing the PD, I think.)

This is not entirely great, because I want to capture the edit comments, and sometimes the edit comment relating to adding/removing will be on the PD, not on the wiki page. But I guess I'll ignore that for now (hand-editing the PD is an edge case).

Including the revision number has (at least) two purposes: to identify where to get the file contents from, and to identify where to get the edit comment from. I guess it also is important in sorting things from earliest to latest - that's what we want the revision numbers of the PD for.

So maybe there should actually be two revision id columns in the list - one for the edit that does the action, and one for the revision where the file contents are.

{ (431, "edit project description", null, 431)
  (431, "add source file", "sqap.py", 422)
  (439, "edit source file", "sqap.py", 439),
  ...
}

[Lee Worden]

No, this is messy. Maybe a slightly more structured form, with one clear entry per revision, and let the consumer of the data figure out which comprises "adding a file" or whatever:

{ 431 => ("add source file to project", "sqap.py"),
  439 => ("edit source file", "sqap.py"),
  ...
}

But if I do that, I should look backwards in time and include the edit that put the file on the page before it became part of the project, because git needs the file contents:

{ 420 => ("add source file to page", "sqap.py"),
  431 => ("add source file to project", "sqap.py"),
  439 => ("edit source file", "sqap.py"),
  ...
}

Then the consumer has to keep track of something like 420 and hold it until the file gets added to the project. I don't think this is so good either. There must be a simpler way. I should probably work out exactly what the git consumer and the special-page consumer each want to end up with, and work backward from that.

[Lee Worden]

For git I want a series of

• add file
• edit file
• remove file

together with file contents, edit comments, and revision numbers. (I want the rev number so that when a single edit changes multiple files I can record them as a single commit.)

Right now, I'm not going to include the project description as a file whose history is tracked, but it could happen in the future. I could also include the wiki pages as files, as I used to in the export package.

So I don't care whether it's added when the PD is edited or when the wiki page is edited, but I generally want the edit comment from the wiki page, even if it was made before the file was added to the PD.

For the special page, I may want

• changes to the project description
  • including itemization of what was changed: add source file, change source file location, etc.
• changes to relevant pages
  • including itemization of what source files and other things were changed

Also, when a source file is added to the project, people will want to know what page it's on.

The main issue here seems to be that sometimes you have to look backward in time to find a source file when the project starts using it. This is true when the file's location changes as well as when it's added.

So maybe I just handle that. I record changes to both PD and wiki pages, and when a PD change needs an associated wiki page location I provide it.

{ 431 => ( ( "add source file to project", "qnaf.py", 420 ) ),
  439 => ( ( "edit source file", "qnaf.py", 439 ),
           ( "edit source file", "gspr.mk", 439 ) ),
  ...
}

There will be times when the location doesn't exist, for instance if one adds a source file with a certain page location before creating the page. Then the revision will have to be null, and the file will appear in a later event.

Likewise it might be removed from its page before it is removed from the PD (using the new automatic removals, it will be normal to remove it from the PD just after removing from its page). I think in this case as in the delayed-add case it'll be up to the git client to figure out when the add/remove happens.

[Lee Worden]

Also, if it works well, the unfortunate sequence Preview, Add source file on preview page, File gets added to permanent project description, Abandon preview, File gets removed from PD without ever actually existing in the wiki storage will not have any effect on the git history because the file never actually appears and disappears.

[Lee Worden]

So I am working with a simple project on my private testing wiki. Here are the relevant page histories, with comments:

4825 create project description, with source file included
4826 create page, including source file text

so let's start with making sure the software can export that appropriately.

[Lee Worden]

Now about data structures. WW currently maintains some cached data so that for instance, when there are four source files on a page, we don't load the page text from the database 4 times when we retrieve all the files' contents. Now that we want to retrieve historical contents of source files as well as current ones, there's a question of what to cache and how to organize the related data structures and functions.

I'll make notes about that at [http://lalashan.mcmaster.ca/theobio/projects/index.php/WorkingWiki/Developer_Documentation/Fetching_source_file_contents].

[Lee Worden]

Ticket moved from /p/workingwiki/feature-requests/34/