USAGE

sfwiki -c|-d|-h [SUBJECT]|-p|-r|-s|-t PAGE TITLE|-v

This SF Wiki Helper script enables off-line editing and git version control of
markdown files for the standard SourceForge wiki tool. In this way you can
prepare changes in documentation while working on a release and publish them at
the moment your release becomes available to the public.

You need to run sfwiki from a git repository and from the directory where you
keep or are going to keep the markdown files for your project's wiki.

When prompted for project and wiki name, enter them exactly as they appear in
the URL to your project's wiki.

sfwiki uses the 'master' branch as a reference to the current on-line contents
of the wiki. If you use another default branch, e.g. 'trunk', then you can
change it with 'sfwiki -c'.

For publishing pages you need to enter a bearer token. See the FAQ (sfwiki -h
faq) for how to obtain one and other questions about bearer tokens.

Use one option at a time:
    -c  Configures default branch, 'detect renamed pages' mode  and the
        similarity threshold for detecting renamed pages.
    -d  Downloads markdown, labels and attachments for all pages of project wiki
        tool. Only goes ahead if current directory is empty or if there are no
        differences between current directory and corresponding directory in 
        master.
    -h  Shows this help text or help on optional SUBJECT:
        - requirements
        - configuration
        - workflow
        - offline (off-line editing of wiki pages)
        - rename (renaming pages)
        - faq
        - history and
        - all (show all subjects).
    -p  Publish changed markdown and labels to project wiki tool. List changes
        that you need to do manually after publishing.
    -r  Review renamed pages and manual actions prior to and after publishing.
    -s  See a preview of what is going to be published. Same output as -p
        without actually publishing.
    -t  Test publish page with given PAGE TITLE. Publishes markdown without
        labels. After reviewing the published page you need to manually revert
        to the previous version or delete the page if it did not exist before.
    -v  Show sfwiki version.


REQUIREMENTS

In order to run this script you need to have the following command line tools
installed:
- curl
- jq
- git

Your project's wiki should allow the following permissions
CONFIGURE to a group your user account belongs to
CREATE to a group your user account belongs to
EDIT to a group your user account belongs to
READ to *anonymous (i.e. any user including those not logged in)


CONFIGURATION

SF Wiki Helper uses the local git configuration file to store settings. In that
way you can have separate configurations for different projects and wiki tools.
It also means you need to run sfwiki from the repository where you keep or are
going to keep your wiki's markdown files to be able to read and modify your
configuration.

The following items can be configured:

- Project and wiki name
  You will be asked to enter your project and wiki name when you download or
  publish your wiki pages. You have to enter the names exactly as they appear
  in the URL to your wiki. They are saved automatically. It is possible to
  download your wiki outside a git repository but in that case project and
  wiki name won't be saved
- Default branch name
  SF Wiki Helper assumes that the last commit of the default branch
  represents the current online contents of your wiki. By default it uses the
  'master' branch. Suppose your are using 'main', 'trunk' or another default
  branch name you can change it with sfwiki -c.
- 'Detect renamed pages' mode
  By default 'detect renamed pages' mode is off, i.e. when you rename a page
  it is published under the new name and you have to manually remove the page
  with the original name afterwards. When you turn it on you get the
  opportunity to manually rename renamed pages before publishing so you do not
  need to delete the original page afterwards. This has the advantage that the
  on-line revision history is preserved. For more information see RENAME PAGES
  (sfwiki -h rename). You can turn renamed page detection on or off with
  sfwiki -c. 
- Similarity threshold for 'detect renamed pages' mode
  For detecting renamed files git looks at the similarity between deleted and
  added files. A similarity threshold of 100 means the files need to be
  exactly the same. The lower the threshold, the more differences are
  allowed. So if you notice that renamed pages are not detected you can lower
  the threshold. If you detect false positives you can increase the
  threshold. The default threshold is 50. You can change it to any integer
  from 1 to 100 with sfwiki -c.

There are alternative ways to change these settings. Be careful though
because your input will not be validated. You can edit .git/config in your
git repository. Here is an example of how the sfwiki section can look like.

    [sfwiki]
        project = yourproject
        wiki = yourwiki
        defaultbranch = master
        detectrenamed = off
        similaritythreshold = 50

In addition you could use git config to change the sfwiki configuration, for
example:

    $ git config sfwiki.similaritythreshold 60

This will set the configuration locally. It is possible to set it globally,
for example:

    $ git config --global sfwiki.detectrenamed on

Don't do this for the project and wiki name though. And in all cases make
sure that detectrenamed has a value 'on' or 'off' and similaritythreshold is an
integer between 1 and 100.


WORK FLOW

All example commands are run from the working directory that you choose for
editing your wiki pages off-line.

A. One time step: download current wiki contents, create local git repository,
   make an initial commit of current wiki contents and start working on a new
   release branch
    $ git init
    $ git config --global user.email 'you@example.com'
    $ git config --global user.name 'Your Name'
    $ sfwiki -d
    $ git add .
    $ git commit
    $ git checkout -b release1

B. Edit wiki pages
    Current checked out branch should be release1
    See OFF-LINE EDITING (sfwiki -h offline)

C. Add interim on-line changes to master and rebase release branch onto
   master
    Current checked out branch should be release1
    $ git add .
    $ git commit
    $ git checkout master
    $ sfwiki -d
    $ git add .
    $ git commit
    $ git push
    $ git checkout release1
    $ git rebase master

D. Publish off-line changes to wiki and merge release branch into master
    Current checked out branch should be release1
    $ sfwiki -p
    - Carry out listed manual on-line changes as needed.
    $ git add .
    $ git commit
    $ git checkout master
    $ git merge release1

E. Start working on next release: commit on-line changes to master, delete old
   release branch and start working on new release branch.
    Current checked out branch should be master
    $ sfwiki -d
    $ git add .
    $ git commit
    $ git branch -D release1
    $ git checkout -b release2

This is the most simple flow, i.e. for one person exclusively working with a
local git repository. The key attention point is that sfwiki uses the most
recent commit of the master branch as reference to what is currently on-line
and compares it to the current work directory, no matter whether your changes
are untracked, unstaged, staged or committed. If you are already working with a
distributed work flow with a central repository, then you should not have much
of a problem incorporating SF Wiki Helper. You can create a separate repository
or include it in a subdirectory of your main git repository, e.g. 'docs'. in
the latter case you need to run sfwiki always from this same subdirectory. The
rebase in step C can only work with a local release branch not yet pushed to
the central repository. In case of a distributed work flow it is probably
better to replace it with a merge.


OFF-LINE EDITING

It goes without saying that you should write your markdown according to the
SourceForge Markdown Syntax Guide <https://sourceforge.net/nf/markdown_syntax>,
and not any markdown guide. There is no official SourceForge/Allura markdown
flavour specification, but it seems a superset of the CommonMark specification.
That means that many tools can highlight and preview the basic formatting of
your locally written markdown. Typical SourceForge additions like Artifact
Links, User Mentions and Table of Contents are of course not supported.

Before editing anything make sure you have downloaded your current on-line wiki
pages and committed them to your master branch. For further information read
the section WORK FLOW.

After downloading the wiki pages (sfwiki -d) to your working directory the file
and directory names correspond with the page titles in your wiki. So if you
have a wiki page titled 'Demo example', there should be a file 'Demo
example.md' which contains de markdown you can edit. In addition there should
be a directory 'Demo example' containing a file labels.txt. This file contains
a comma separated list of the labels of the 'Demo example' page. Optionally the
directory 'Demo example' can have a subdirectory attachment that contains the
attachments of the page. These attachments are usually images that are shown on
the page.

The following scenarios will illustrate how to use these local files for
off-line editing. All example commands are run from the working directory that
you choose for editing your wiki pages off-line.


Attached images with Allura img macro

Suppose you want to add image 'ExampleScreenschot.png' to a page titled
'Demo example'.
    $ mkdir -p 'Demo example/attachment'
    - Put 'Example screenschot.png' in the created directory
    - Edit 'Demo example.md' and add the placeholder:
        [[img src=ExampleScreenschot.png]]
    At the time of publishing
    $ sfwiki -p
    - sfwiki will remind you to manually attach 'ExampleScreenschot.png' to
      the on-line wiki page titled 'Demo example'

Likewise you can add more images and rename, replace and remove images. You
cannot locally preview the images, you will just see the img macro placeholder.
sfwiki cannot automatically publish the changed attachments, but will add any
manual actions with regard to attachments to the todo list based on what you
have changed in the attachment directory compared to the master branch.


Testing a page

Suppose the page titled 'Demo example' has attached images and is not yet
present on the current wiki.  Suppose your local tools do not give you enough
insight on how it will look like and you want to test it on the wiki.
    $ sfwiki -t Demo example
    - Manually add the attachments to the published 'Demo example' page
    - Refresh and review the looks of the 'Demo example' page
    - Remove the published 'Demo example' page (click Edit and then the trash
      can)

Suppose the page was already present on the wiki, then you would have to
revert to the previous version afterwards. You also might have to restore
the attachments as they were before.

Alternatively, you can commit and push the changes to the page titled 'Demo
example'. In that way you can preview the page in the on-line repository at
Sourceforge. Except for the title and the images it will show the page exactly
as it will appear on the wiki.


Adding a new page

Suppose you want to add a page titled 'Demo example'.
    $ mkdir -p 'Demo example'
    $ touch 'Demo example/labels.txt'
    $ touch 'Demo example.md'
    At the time of publishing:
    $ sfwiki -p
    Result: the new page 'Demo example' is published on the wiki


Renaming a page

Suppose you would like to rename the title of a page from 'Demo example' to
'Example demo'.
    $ mv 'Demo example' 'Example demo'
    $ mv 'Demo example.md' 'Example demo.md'
    At the time of publishing:
    $ sfwiki -p
    Result: the renamed page 'Example demo' is published on the wiki
    - sfwiki reminds you to manually remove page 'Demo example'
    - If any attachments, you need to manually add them to page 'Example
      demo'

The last two actions are listed in the output of sfwiki -p.

There is an alternative mode where you manually rename the page on-line prior
to publishing, see RENAME PAGES (sfwiki -h rename). Advantage of this alternative
mode is dat you keep the on-line revision history of the orignal page and that
on average you will reduce the number of manual actions necessary.


Deleting a page

Suppose you would like to delete the page titled 'Demo example'
    $ rm 'Demo example.md'
    $ rm -rf 'Demo example'
    At the time of publishing
    $ sfwiki -p
    - sfwiki reminds you to manually remove the on-line wiki page titled 'Demo
      example'

If you add one or more wiki pages in the same publication, sfwiki will prompt you
to check whether you renamed a wiki page. If apart from deleting pages you did
not rename a page, you can ignore this prompt.


Changing labels

Suppose you want to change the labels or the order in which they appear for the
page titled 'Demo example'.
    - Edit 'Demo example/labels.txt', change the comma separated list of labels
      according to your preferences and save the file.
    At the time of publishing
    $ sfwiki -p
    Result: the labels of page 'Demo example' will exactly match the comma
      separated list in 'Demo example/labels.txt'.

Do not enclose labels with quotes unless you want the quotes to be part of the
label. Emptying labels.txt will remove all labels from the corresponding page.
If you do not need labels for your wiki pages, just leave the labels.txt files
empty and ignore them.


RENAME PAGES

TLDR: You can switch SF Wiki Helper to a mode in which it detects renamed
pages, enabling you to manually rename pages prior to publication. The
advantage of doing that is that you can preserve the on-line revision history
of the renamed page and in most cases the number of manual actions is equal or
less compared to regular mode. On the other hand it entails a small learning
curve and you need to put some effort in checking whether the renamed page
detection is correct in your particular situation. If the advantages appeal
to you, I recommend to continue reading and give it a try.

OFF-LINE EDITING (sfwiki -h offline) contains a description of how to rename a
page off-line.


Regular versus 'detect renamed pages' mode

Both modes are the same except when you rename a page and publish.

When you rename a page off-line in regular mode, publishing will lead to a new
on-line page under the new name. Afterwards you need to manually delete the
page with the old name. With this you also delete the on-line revision history
of that page. If the renamed page has attachments you have to manually add them
to the new page.

When you rename a page off-line in 'detect renamed pages' mode, SF Wiki
Helper will prompt you to rename the page on-line before publishing. In this
way you do not have to delete the original page afterwards and you preserve
the on-line revision history. If the renamed page has attachments, they are
still with the renamed page. Only if you have added, deleted, modified or
renamed attachments, you need to carry out manual actions after publication.

In both modes SF Wiki Helper will list all appropriate manual actions.

By defaul SF Wiki Helper is in regular mode. sfwiki -c allows you to switch
from one to the other mode.


Review renamed pages

You can review renamed pages with sfwiki -r. You might want to use this if the
detected renamed pages shown prior to publishing are not as expected or just
when you want to make very sure that the renamed pages are correctly detected.
It shows a list of detected renamed pages, a list of deleted pages and a list
of added pages. The two latter lists are additional, i.e. they are not already
covered by the list of renamed pages.

You can check if any of the detected pages are not a renamed page and if any
pair of a deleted and added page form a undetected renamed page.

If SF Wiki Helper is in regulare mode, sfwiki -r will just show a list of deleted
and added pages.


Errors in renamed page detection

It is unlikely that renamed pages are detected in a wrong way, but it is
possible. In case of this unlikely event, the first step is that you understand
the source of incorrect renamed page detection.

SF Wiki Helper uses git to compare the markdown of deleted pages with the
markdown of added pages. If the markdown of a deleted page is very similar to
the markdown of an added page, it is detected as a renamed page. So if you
rename a page and you change the markdown of this page a lot, it will not be
detected as a renamed page. If you delete a page and then add a page which is
by coincidence very similar to the deleted page, it will be detected as a
renamed page. There is also a very small possibility that you renamed a page
and that an unrelated deleted or added page is more similar to the current or
original page than the actual original or current page, causing a mixup in the
detected renamed page.

There are several things you can do or try to get it right:

1. You can (temporarily) change the similarity threshold
   Git calculates the similarity between the deleted and added page as a
   percentage. If this percentage is below a certain threshold, the page will
   not be detected as renamed. The default threshold is 50%. So in case a
   renamed page is not detected, you can lower this percentage. In case a
   deleted and added page are wrongful detected as a renamed page you can
   increase the threshold. In the latter case you can see the similarity
   percentage in the output of sfwiki -r, so you can increase the threshold to
   just above this percentage. Increasing the threshold to 100% results only in
   detection of a renamed page if the markdown of deleted and added page is
   identical. You can change the similarity threshold with sfwiki -c, but leaf
   out the percentage sign. See also CONFIGURATION (sfwiki -h configuration).

2. You can (temporarily) switch to regular mode. You will then, of course 
   loose the advantages of the 'detect renamed pages' mode.

3. You can diverge from SF Wiki Helper's instructions to get the correct
   result.

   In case of a detected renamed page that is not a renamed page, you can
   ignore the instruction to rename the page on-line and afterwards you
   manually delete the page with the supposedly original name. If the added
   page has any off-line attachments, you add them on-line afterwards.

   If you have spotted an undetected renamed page, you can do the following:
   Prior to publication:
   - Edit the original page on-line and rename it according to the off-line
     name.
   The actions after publication depend on the situation.
   * The renamed page does not have any attachments:
     - No further actions are required
   * The renamed page has attachments and while editing off-line you did not
     add, delete, modify or rename attachments in relation to this page:
     - No further actions are required
   * The renamed page has attachments and while editing off-line you have
     added, deleted, modified or renamed attachments in relation to this page:
     - Edit the renamed page on-line and change the attachments according to
       the off-line changes
   * The renamed page has attachments and you can't remember if you changed the
     attachments off-line or what you have changed to the attachments:
     - Edit the renamed page on-line and remove all attachements
     - Edit the renamed page on-line againn and add all local attachments.


FAQ

Taking SF Wiki Helper into use

I am starting from scratch with an empty project wiki. Is it a good idea to
    start using SF Wiki Helper right from the start?

    I would say no. In the first place it is not necessary. There is no
    existing documentation, so no conflict. If you do not want users to see
    the documentation yet, you can simply restrict access to your wiki until
    the time it can be published. In addition you miss the opportunity to
    learn the SourceForge markdown flavour. This is so much easier on-line
    because you can preview everything. Off-line you will not be able to
    preview typical SourceForge markdown like artifact links. Another advantage
    is that after finalising the first version of your wiki on-line, sfwiki can
    download your wiki and you immediately have the right set-up for your
    off-line working directory.

Publishing wiki pages

Why do I have to do some things manually when publishing my wiki pages?
    Can't sfwiki do that for me? This concerns:
    - renaming and deleting pages
    - changing attachments

    sfwiki is using the Allura API. The only things you can change by using this
    API are the contents (written in markdown) and the labels of a wiki page.
    You cannot rename the title of a page, change a page's  attachments or
    delete a page. But if you do all these things offline in advance, sfwiki
    will generate a todo list of changes you have to change manually when
    publishing the changes in contents and labels.

Security

Why do I have to enter a bearer token?

    A bearer token is the only way to give sfwiki access to your user account,
    so it is authorised to use the Allura API to update your wiki pages. The
    alternative is to turn this script into an application that can do proper
    OAuth negotation.

How do I obtain a bearer token?

    You must first register for an OAuth consumer token at
    <https://sourceforge.net/auth/oauth/>. Once you have registered, the
    consumer token allows you to generate a bearer token that gives sfwiki
    access to your user account without having to perform the OAuth
    negotiation.

Is it safe to use a bearer token?

    There is certainly a risk. If somebody gets hold of your bearer token, a
    REST application using the Allura API can give him or her potentially full
    access to your account. To reduce this risk you could revoke the bearer
    token after use and generate a new one when needed. The sfwiki script itself
    does not display the bearer token and will not retain it in any way after
    publishing your wiki pages.


HISTORY

v0.0.3, 19-08-2024
    Improved documentation
    - Corrected spelling 'git branch'
    The following bug is fixed:
    - Display of v0.0.0 instead of latest version

v0.0.2, 17-05-2023
    Improved documentation:
    - Added push of master after interim update
    - Added possibility to preview page in on-line repository

v0.0.1, 21-08-2022
    The following bugs have been fixed:
    - Modified pages were published twice
    - An extra line feed was added to downloaded markdown and labels
    - sfwiki -p logged publication of page OK while it was NOK

v0.0.0, 20-08-2022
    This is the first release. It is an alpha release and it still can contain
    bugs.