roundup-users

[Roundup-users] Write-access to wiki -- Directions on how to document

[<r2...@mi...>]

Hi!

I am new to public work on software, so please correct any misconceptions I have,
thanks in advance!


I am evaluating roundup for the use as an inhouse maintenance tracker for around
100 facilities / sites (hence the need for recurring tasks).

Currently I think the "Roundup TODO Template" (which I quickly got working yesterday)
is a good starting point. (I tried around 20 different solutions, and not one fit -
some would be passable but renting an cloud service, with no option to self host
for an essential inhouse service, would be insane!)

An estimate for the total numer of tasks does not exist but pessimistic me
thinks there may be around 100 per site, so 100 x 100 = 10.000 - this is why I
like the "Roundup TODO Template":
 > # By default, only the upcoming issue is added to the tracker.


Going forward I am uncertain on how to document the proccess and modifications
so others can use this.

Would the https://wiki.roundup-tracker.org/ be a good place?

Or would a development forge be a better choice? I find the interface of
https://sourceforge.net/ to be quite confusing, but there seems to be alternatives,
 - https://sourcehut.org/ and
 - https://codeberg.org/
would either be a choice to track the changes?


--
Have a fine day!

r2d2

Re: [Roundup-users] Write-access to wiki -- Directions on how to document

[John P. R. <ro...@cs...>]

Hello r2d2:

In message <d180d6111cb3129abfcb88d9015ce297101f497e@mirakel.one>,
r2d2@mirakel.one writes:
>I am evaluating roundup for the use as an inhouse maintenance tracker
>for around 100 facilities / sites (hence the need for recurring tasks).
>
>Currently I think the "Roundup TODO Template" (which I quickly got
>working yesterday) is a good starting point. [...]
>
>An estimate for the total numer of tasks does not exist but
>pessimistic me thinks there may be around 100 per site, so 100 x 100
>= 10.000

Is this per year or over the lifetime? If per year, you may want to
consider using either the postgresql or mysql database backends. I
have used the SQLite backend with a few thousands of issues and it
handles it fine. If you have a complex schema however, you might have
to migrate to one of the other db backends after a few years.

>- this is why I like the "Roundup TODO Template":
> > # By default, only the upcoming issue is added to the tracker.
>
>Going forward I am uncertain on how to document the proccess and
>modifications so others can use this.
>
>Would the https://wiki.roundup-tracker.org/ be a good place?

Having a pointer from:

  https://wiki.roundup-tracker.org/TrackerTemplates

to a development forge would be a better choice. Most of the tracker
templates listed on TrackerTemplates have links to some VCS repo (svn,
hg, git, fossil, ...). Documenting tracker development on a wiki is
going to be tedious as you have to attach files and update inline
code/docs in the browser. (This can be scripted to some extent, but
it's still tedious.)

We do archive some trackers using a zip or tarball where the originals
have been removed. But that is for archive purposes not ongoing
development/documentation.

>Or would a development forge be a better choice? I find the interface of
>https://sourceforge.net/ to be quite confusing, but there seems to be
>alternatives,
> - https://sourcehut.org/ and
> - https://codeberg.org/
>would either be a choice to track the changes?

Managing the development of your tracker using a VCS (git, mercurial,
fossil) would be easier using a forge. That's why my sysadmin tracker
is managed using Fossil.

You will want to put your tracker (but not the db subdirectory) under
VCS control anyway. Also you might not want to put (any of) your
config.ini files in a public VCS as it can have secrets/sensitive info
in the file. True secrets can often be specified in separate files so
the config.ini could be stored in a VCS. However sensitive but not
secret items like:

  the web URL for your tracker,
  the number of rounds used for pbkdf2 derivation,
  the location of your database server,
  etc.

are located in config.ini. For public consumption, adding a clean
config.ini, as config.ini.sample or using the tracker standard of
config_ini.ini to set required fields (see:
https://roundup-tracker.org/docs/tracker_templates.htm) are preferred.

Hope this helps.

--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.

Re: [Roundup-users] Write-access to wiki -- Directions on how to document

[<r2...@mi...>]

Hello rouilj,

thanks for your advice regarding source control and using a forge!

Please have a look at
https://codeberg.org/R2D2/roundup_todo_tracker_template
there is the original TODO Tracker from Nikolaus Rath with my changes
for the current roundup version 2.3.0.

I would be happy if this todo template could be integrated into
roundup.

I tested it with the steps "Install and (quick) setup" from
DESCRIPTION.org .


> 
> > 
> > I am evaluating roundup for the use as an inhouse maintenance tracker
> > for around 100 facilities / sites (hence the need for recurring tasks).
> > Currently I think the "Roundup TODO Template" (which I quickly got
> > working yesterday) is a good starting point. [...]
> > An estimate for the total numer of tasks does not exist but
> > pessimistic me thinks there may be around 100 per site, so 100 x 100
> > = 10.000
> > 
> 
> Is this per year or over the lifetime? If per year, you may want to
> consider using either the postgresql or mysql database backends. I
> have used the SQLite backend with a few thousands of issues and it
> handles it fine. If you have a complex schema however, you might have
> to migrate to one of the other db backends after a few years.
> 


Thanks for your advice! Since I have no real numbers I planned to start
with sqlite and fill the tracker step by step for each site.

How hard would an migration from sqlite to postgres be?
(Yeah, typical red flag for pain ;-) )


--
Have a fine day!

r2d2

Re: [Roundup-users] Write-access to wiki -- Directions on how to document

[John P. R. <ro...@cs...>]

Beep boop brrr R2D2:

In message <58a669bb094b59a7eb612250b92fc20c81736fa0@mirakel.one>,
r2d2@mirakel.one writes:
>Please have a look at
>https://codeberg.org/R2D2/roundup_todo_tracker_template
>there is the original TODO Tracker from Nikolaus Rath with my changes
>for the current roundup version 2.3.0.

I have added a link to this repo to
https://wiki.roundup-tracker.org/TrackerTemplates and updated
https://wiki.roundup-tracker.org/TodoTemplate as well.

>I would be happy if this todo template could be integrated into
>roundup.

Adding more templates is problematic as they have to be kept up to
date. The core developer(s) are already dealing with a backlog of
items.

>I tested it with the steps "Install and (quick) setup" from
>DESCRIPTION.org .

That's a great addition to the docs for the tracker.

What version of Python and what release of Roundup are you using?
Having that mentioned in Description.org would be helpful for others.

Also I noticed in some files (check-tracker.py, remind-tracker.py ...)
warnings are disabled.  What are you seeing as depricated modules that
are being reported: cgi.py, cgitb.py, maybe others?

>> > I am evaluating roundup for the use as an inhouse maintenance tracker
>> > for around 100 facilities / sites (hence the need for recurring tasks).
>> > Currently I think the "Roundup TODO Template" (which I quickly got
>> > working yesterday) is a good starting point. [...]
>> > An estimate for the total numer of tasks does not exist but
>> > pessimistic me thinks there may be around 100 per site, so 100 x 100
>> > = 10.000
>> 
>> Is this per year or over the lifetime? If per year, you may want to
>> consider using either the postgresql or mysql database backends. I
>> have used the SQLite backend with a few thousands of issues and it
>> handles it fine. If you have a complex schema however, you might have
>> to migrate to one of the other db backends after a few years.
>
>Thanks for your advice! Since I have no real numbers I planned to start
>with sqlite and fill the tracker step by step for each site.

I assume you are using one tracker and adding a site field to track
issues per site.

>How hard would an migration from sqlite to postgres be?
>(Yeah, typical red flag for pain ;-) )

The directions are at:

  https://roundup-tracker.org/docs/admin_guide.html?highlight=migrate#migrating-backends

Basically:

 * shut down the tracker (including email interfaces).
 * use 'roundup-admin export' to export the db and files.
 * copy the existing tracker home to a new directory but do not
   include the db sub-directory.
 * configure config.ini to use postgres and set up the postgres database
   user. https://roundup-tracker.org/docs/postgresql.html.
 * use 'roundup-admin import' to reload the database and files to a new
   tracker home directory.
 * start the new tracker and test.
 
Doing it this way requires at least 3x the disk space used by the db
sub-directory (original copy, exported copy, new tracker copy). Usually
files are the bulk of the storage space used.

There is a method using 'roundup-admin exporttables' and
'importtables' that doesn't export/import the files stored in the db
sub-directory. The last time I tested this few years ago (on a tracker
with a few hundred issues and maybe 500 messages), it was a bit faster
than the full export/import since it doesn't touch the files. I expect
it to be faster if there are more files. You reuse the existing
tracker home directory including the db sub-directory. This isn't as
well tested as the export/import method though. Also you need to use
'roundup-admin reindex' if you are using the native full text indexer
(i.e. not xapian or whoosh). Since 2022 you can do this in batches.

You might want to try a few export/import cycles to get some idea of
how long it will take. There are too many variables (number of files,
number of issues, total size of the db (including history logging),
speed of the system and its disks) to make an estimate.

Generally if there is pain, it's due to processing time and disk
space.

Release 2.1.0 fixed an import bug if the export had a retired node
exported after an active node with the same unique values. The current
code sorts the export properly and can import an export with the items
in the wrong order.

Also there is some extra support in the current development tree for
postgres to commit a transaction every 10k rows and a pragma to set
savepoint_limit=10000 if 10k rows is too much data. A user was
experiencing out of memory errors with the way Roundup was trying to
speed up data loading after solving the out of order problem.

So hopefully the major pain points have been/are handled.

Have a great weekend.
--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.

Re: [Roundup-users] Write-access to wiki -- Directions on how to document

[John P. R. <ro...@cs...>]

Hi R2D2:

Something I forgot to mention in my last email.

In your Description.org you have:

  * copy folder todo into your Virtual-Env/share/roundup/templates/

You should be able to run 'roundup-admin -i ... install' in the todo
folder. It should recognize it as a tracker template and give you the
option of installing it. If you want to make "install" the template
making it available for a 'roundup-admin install' command started from
any directory, installing it in .../templates/ is the way to go.

Have a great weekend.
--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.

Re: [Roundup-users] Write-access to wiki -- Directions on how to document

[<r2...@mi...>]

Hi rouilj,

thanks for the pointers!

The deprecation warning suppression was unnecessary and is now gone.

I had a quick  isort / black / flake8 session - after a short test everything
is still working :-)


Thanks again for the warm welcome, I will send another mails with new questions!


--
Have a fine day!

r2d2

Re: [Roundup-users] Write-access to wiki -- Directions on how to document

[John P. R. <ro...@cs...>]

Hi R2d2:

In message <4f37d3dc187ffcb7b4a9c3c3030bf785c3300f61@mirakel.one>,
r2d2@mirakel.one writes:
>thanks for the pointers!

Any time.

>The deprecation warning suppression was unnecessary and is now gone.

Good.

>I had a quick  isort / black / flake8 session - after a short test everything
>is still working :-)

I have been working with ruff now that my main development box runs
Python 3.8.  I am quite impressed with it.

>Have a fine day!

Same to you.

--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.