roundup-devel — Discussion on the development of Roundup

[Roundup-devel] [issue2551367] Issues with dbm backend replacement with sqlite.

[John R. <is...@ro...>]

New submission from John Rouillard:

Newer versions of python (3.13+ https://docs.python.org/3/library/dbm.html#module-dbm.sqlite3)
use sqlite as a key/value store replacing the use of (g)dbm, bdb.

This will be a win on windows where dbm backend devolved to dumbdbm which performs
poorly.

However as msg8151 indicates, there may be issues with tests.

So far 3.13 runs seem to complete ok in CI:

   https://github.com/roundup-tracker/roundup/actions/runs/11394478475/job/31704876042

but I can't tell if it's using the sqlite backend.

This issue tracks any changes required to support the back end change and verify
that 3.13 tests are using and working ok with sqlite.

----------
messages: 8156
nosy: rouilj
priority: normal
severity: normal
status: new
title: Issues with dbm backend replacement with sqlite.
type: behavior
versions: 2.5.0

_________________________________________________
Roundup tracker <is...@ro...>
<https://issues.roundup-tracker.org/issue2551367>
_________________________________________________

Re: [Roundup-devel] [issue2551366] Running tests: Need a small howto, probably in README

[John P. R. <ro...@cs...>]

Hi Ralf:

I had a few free minutes to respond to this. Hopefully I'll have time
later today to go through your perf changes.

In message <172...@ro...>,
Ralf Schlatterbeck writes:
>The README in the test directory doesn't contain any hints on what
>is needed to run the tests:
>- What database users do we need
>- What passwords can we use
>- Can we override the default config
>
>I think the tests should either honor settings in the environment or
>have a small (optional, not checked into a version control system) config
>file.

See docs/developer.txt - Testing Notes. It goes over the setup for
developers. live-server tests need the requests package. If you don't
have it the test are (supposed to be) skipped.

Also there are notes in the mysql.txt and postgresql.txt.  Maybe
developer/testing notes in the database docs should be moved to
developers.txt? At least there should be links (currently missing)
from developer.txt to the test sections in the db docs.

>The test *used to* honor the settings RDBMS_USER and RDBMS_PASSWORD
>in the environment. They no longer seem to do this. Note that having
>the password in an environment variable constitutes a security
>risk. But it is still much better than requiring a hard-coded
>password in the source.

Hmm, I never saw/knew that the environment vars were used by the
tests.  I wonder if that changed as part of the patches to use
postgres schemas as well as just a database.

I am a little worried that testing with environment variables that
look like they could be production variables will result in problems
as the tests nuke databases.

>In addition in Postgres when I do have a database user 'rounduptest'
>most tests run for me.
>But a lot of tests seem to use a username 'rounduptest_schema' with
>an unknown/undocumented password. I'm currently not able to make
>these tests run.

See "Running the PostgreSQL unit tests" in ../../doc/postgresql.txt

Are they failing, or are they skipped? I thought I set them up to be
skipped if the schema user/password failed.

References to these docs in a test/README.txt is a good idea.

--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.

[Roundup-devel] [issue2551366] Running tests: Need a small howto, probably in README

[Ralf S. <is...@ro...>]

New submission from Ralf Schlatterbeck:

The README in the test directory doesn't contain any hints on what is needed to run the tests:
- What database users do we need
- What passwords can we use
- Can we override the default config

I think the tests should either honor settings in the environment or have a small (optional, not checked into a version control system) config file.

The test *used to* honor the settings RDBMS_USER and RDBMS_PASSWORD in the environment. They no longer seem to do this. Note that having the password in an environment variable constitutes a security risk. But it is still much better than requiring a hard-coded password in the source.

In addition in Postgres when I do have a database user 'rounduptest' most tests run for me.
But a lot of tests seem to use a username 'rounduptest_schema' with an unknown/undocumented password. I'm currently not able to make these tests run.

----------
components: Documentation
messages: 8145
nosy: rouilj, schlatterbeck
severity: minor
status: new
title: Running tests: Need a small howto, probably in README
type: rfe
versions: devel

_________________________________________________
Roundup tracker <is...@ro...>
<https://issues.roundup-tracker.org/issue2551366>
_________________________________________________

[Roundup-devel] [issue2551365] Document use of a VCS to manage a deployed tracker with upgrades.

[John R. <is...@ro...>]

New submission from John Rouillard:

Issue 2550764 suggested making the login/logout items in page.html conditional
by a setting in config.ini.

Editing the template is the recommended method of doing this and the requestor
does that. However it is not the best when upgrading to a newer version of the
tracker.

There is no documentation (probably should be part of the admin guide) on how
to use a version control system with a vendor branch to make merging new tracker
template releases into an existing tracker easier.

Somebody should write one. While I hate to say it, git would probably be the VCS
that gets used as an example. I think mercurial or fossil would be easier, have fewer
footguns etc. But git is the mainstream tool for this now.

----------
components: Documentation
keywords: Effort-Low, GSOC, StarterTicket
messages: 8144
nosy: rouilj
severity: normal
status: new
title: Document use of a VCS to manage a deployed tracker with upgrades.
type: rfe

_________________________________________________
Roundup tracker <is...@ro...>
<https://issues.roundup-tracker.org/issue2551365>
_________________________________________________

[Roundup-devel] [issue2551364] Improve performance of pagination using key param rather than page/size

[John R. <is...@ro...>]

New submission from John Rouillard:

https://dev.to/scion01/optimizing-pagination-in-postgresql-offsetlimit-vs-keyset-21dp

Discusses changing pagination to use keyset vs offset/limit (aka page number/page size).

The idea is to replace:

  SELECT * FROM table ORDER BY id ASC LIMIT 100 OFFSET 100;

with:

  SELECT * from table where id > x ORDER BY id ASC LIMIT 100

where X was the max id in the prior set of 100 items.

For larger offsets, the database has to retrieve and order all rows up to OFFSET
and then discard them. In the second case, the index prevents retrieving
rows that would be in the OFFSET region. Only rows with an id larger than X
would be processed. This may come in play only for large offsets (high page number)
but is worth investigating as it could improve response times for X.index.html pages.

----------
components: Database
messages: 8141
nosy: rouilj
severity: normal
status: new
title: Improve performance of pagination using key param rather than page/size
type: behavior

_________________________________________________
Roundup tracker <is...@ro...>
<https://issues.roundup-tracker.org/issue2551364>
_________________________________________________

Re: [Roundup-devel] [issue2551363] Create Kanban view

[Norbert S. <nor...@ne...>]

Hi John

great idea...
please have a look to this example https://youtu.be/W-7gdpaySLs?feature=shared&t=200, 3:20

Thanks
Norbert


> John Rouillard <is...@ro...> hat am 18.09.2024 01:55 CEST geschrieben:
> 
>  
> New submission from John Rouillard:
> 
> Some thoughts on a Kanban view for issues.
> 
> We need to specify:
> 
>   items to display
>   columns for the board
>   card format/fields
>   swimlane (optional)
> 
> One mapping from an index display is:
> 
>   items to display - search results
>   columns for the board - @sort field
>   card format/fields - @columns display
>   swimlane (optional) - @group
> 
> An example kanban board for @sort=status, @columns=id,title,assignedto,
> @group=sprint (assuming the issues have a sprint property) like:
> 
>    Status   Backlog       Ready         Open       Testing      Closed
> Sprint    +----------+ +----------+ +----------+ +----------+ +----------+
>           | -------- | |          | |          | | -------- | |          |
> 24-10-02  | 1060     | |          | |          | | 1319     | |          |
>           | new item | |          | |          | | title 4  | |          |
>           | Fred     | |          | |          | | Renee    | |          |
>           | ======== | |          | |          | | -------- | |          |
>           | 2210     | |          | |          | |          | |          |
>           | need fre | |          | |          | |          | |          |
>           | Steve    | |          | |          | |          | |          |
>           | -------- | |          | |          | |          | |          |
>           |          | |          | |          | |          | |          |
>           |          | |          | |          | |          | |          |
>           +----------+ +----------+ +----------+ +----------+ +----------+
>           +----------+ +----------+ +----------+ +----------+ +----------+
>           | -------- | |          | |          | |          | |          |
> 24-10-16  | 1310     | |          | |          | |          | |          |
>           | title 4  | |          | |          | |          | |          |
>           | Renee    | |          | |          | |          | |          |
>           | -------- | |          | |          | |          | |          |
>           |          | |          | |          | |          | |          |
>           |          | |          | |          | |          | |          |
>           +----------+ +----------+ +----------+ +----------+ +----------+
> 
> Each card can be dragged to a new column and on drop, the back end is updated via
> REST request to set the issue to its new status.
> 
> Consider a drag/drop javascript library:
> 
>   * https://github.com/SortableJS/Sortable
>   * https://github.com/atlassian/pragmatic-drag-and-drop
>   * https://github.com/ThibaultJanBeyer/DragSelect
> 
> Maybe use grid for layout of the columns/swimlanes.
> 
> Open issues:
> 
>   How to display on mobile or when width is too narrow.
>   How to configure card layout. Consider a sub-template for each card style.
>     That way an admin only needs to design the card level template similar
>     to any other template.
> 
> ----------
> components: Web interface
> keywords: Effort-Medium, GSOC
> messages: 8126
> nosy: rouilj
> priority: normal
> severity: normal
> status: new
> title: Create Kanban view
> type: behavior
> 
> _________________________________________________
> Roundup tracker <is...@ro...>
> <https://issues.roundup-tracker.org/issue2551363>
> _________________________________________________
> 
> 
> _______________________________________________
> Roundup-devel mailing list
> Rou...@li...
> https://lists.sourceforge.net/lists/listinfo/roundup-devel

[Roundup-devel] [issue2551363] Create Kanban view

[John R. <is...@ro...>]

New submission from John Rouillard:

Some thoughts on a Kanban view for issues.

We need to specify:

  items to display
  columns for the board
  card format/fields
  swimlane (optional)

One mapping from an index display is:

  items to display - search results
  columns for the board - @sort field
  card format/fields - @columns display
  swimlane (optional) - @group

An example kanban board for @sort=status, @columns=id,title,assignedto,
@group=sprint (assuming the issues have a sprint property) like:

   Status   Backlog       Ready         Open       Testing      Closed
Sprint    +----------+ +----------+ +----------+ +----------+ +----------+
          | -------- | |          | |          | | -------- | |          |
24-10-02  | 1060     | |          | |          | | 1319     | |          |
          | new item | |          | |          | | title 4  | |          |
          | Fred     | |          | |          | | Renee    | |          |
          | ======== | |          | |          | | -------- | |          |
          | 2210     | |          | |          | |          | |          |
          | need fre | |          | |          | |          | |          |
          | Steve    | |          | |          | |          | |          |
          | -------- | |          | |          | |          | |          |
          |          | |          | |          | |          | |          |
          |          | |          | |          | |          | |          |
          +----------+ +----------+ +----------+ +----------+ +----------+
          +----------+ +----------+ +----------+ +----------+ +----------+
          | -------- | |          | |          | |          | |          |
24-10-16  | 1310     | |          | |          | |          | |          |
          | title 4  | |          | |          | |          | |          |
          | Renee    | |          | |          | |          | |          |
          | -------- | |          | |          | |          | |          |
          |          | |          | |          | |          | |          |
          |          | |          | |          | |          | |          |
          +----------+ +----------+ +----------+ +----------+ +----------+

Each card can be dragged to a new column and on drop, the back end is updated via
REST request to set the issue to its new status.

Consider a drag/drop javascript library:

  * https://github.com/SortableJS/Sortable
  * https://github.com/atlassian/pragmatic-drag-and-drop
  * https://github.com/ThibaultJanBeyer/DragSelect

Maybe use grid for layout of the columns/swimlanes.

Open issues:

  How to display on mobile or when width is too narrow.
  How to configure card layout. Consider a sub-template for each card style.
    That way an admin only needs to design the card level template similar
    to any other template.

----------
components: Web interface
keywords: Effort-Medium, GSOC
messages: 8126
nosy: rouilj
priority: normal
severity: normal
status: new
title: Create Kanban view
type: behavior

_________________________________________________
Roundup tracker <is...@ro...>
<https://issues.roundup-tracker.org/issue2551363>
_________________________________________________

Re: [Roundup-devel] Fixing crash bug in German translation.

[Ralf S. <rs...@ru...>]

On Thu, Aug 08, 2024 at 10:10:54AM -0400, John P. Rouillard wrote:
> It turns out this translation seems to have the action varible for all
> of the langauges, not just German. It was added in checkin 4083 in Feb
> 2009 and changed in June of 2009 to the current string.
> 
> Not sure how it ended up being incorrect in so many languages.

I think the translation framework tries to guess translations based on
existing translations. That's why it was marked 'fuzzy'.

Thanks for looking into this...
Ralf
-- 
Dr. Ralf Schlatterbeck                  Tel:   +43/2243/26465-16
Open Source Consulting                  www:   www.runtux.com
Reichergasse 131, A-3411 Weidling       email: of...@ru...

Re: [Roundup-devel] Fixing crash bug in German translation.

[John P. R. <ro...@cs...>]

Hi Marcus:


In message <528...@pr...>,
Marcus Priesch via Roundup-devel writes:
>for me the english translation of the german phrase
>[proper analysis of error elided ...]
>according to the error:
>
>> The translation in the following pair references the %(action)s 
>> variable that doesn't exist in the scope of the translation call.
>
>the "action" variable is simply not defined at this point in the code.

Correct. What I needed and Ralf supplied was a German translation of
the english string. I must have been unclear in my request.

It turns out this translation seems to have the action varible for all
of the langauges, not just German. It was added in checkin 4083 in Feb
2009 and changed in June of 2009 to the current string.

Not sure how it ended up being incorrect in so many languages.

I'll have to open an issue for it and check to see what happens in the
i18n subsystem when the printf string generates an error. Hopefully it
will just return the untranslated string.

Thanks for confirming my analysis of the issue. (Feel free to run
through the locale file for other issues if you wish. There are plenty
of fuzzy translation 8-)).

--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.

Re: [Roundup-devel] Fixing crash bug in German translation.

[Marcus P. <ma...@pr...>]

Hi there,

for me the english translation of the german phrase

> "Sie sind nicht berechtigt, die Aktion(en) %(action)s auf die Klasse"
> "%(classname)s anzuwenden."

would be something like this:

"You are not allowed to apply %(action)s on class %(class)s"

So basically it is written to also contain the requested action, whereas
the original string only references the class and hardcodes the actions
"retire" and "restore"

original: "You do not have permission to retire or restore the
%(classname)s class."

according to the error:

> The translation in the following pair references the %(action)s 
> variable that doesn't exist in the scope of the translation call.

the "action" variable is simply not defined at this point in the code.

hope this helps,
marcus.

Re: [Roundup-devel] Fixing crash bug in German translation.

[Ralf S. <rs...@ru...>]

On Wed, Aug 07, 2024 at 08:42:33PM -0400, John P. Rouillard wrote:
> Hi all:
> 
> The template toolkit's pofilter turned up a bug in one of the German
> translations.
> 
> The translation in the following pair references the %(action)s variable
> that doesn't exist in the scope of the translation call.
> 
> #: ../roundup/actions.py:49
> #, fuzzy, python-format
> msgid ""
> "You do not have permission to retire or restore the %(classname)s class."
> msgstr ""
> "Sie sind nicht berechtigt, die Aktion(en) %(action)s auf die Klasse "
> "%(classname)s anzuwenden."
> 
> Google translate on the original english was totally different from
> the translation given above.

I can imagine :-)

> I then tried using google translate to turn the German phrase into
> english.  Then I modifed the english phrase to get something that
> looks similar to the original German translation. However I again
> ended up with something different.

The problem here is probably the word 'retire'. The semantics of
retirement of data is non-existing (so the word-by-word translation 'in
Pension gehen' oder 'pensionieren' does not apply).

In other entries in the german translation the phrase is translated as
'löschen' (which means remove) or 'verbergen' (which means hide). Both
do not fully reflect the meaning in english.

I would tend to translate it with 'als gelöscht markieren' (mark as
deleted), that indicates that the value is still available.

> Can a native speaker give me a translation for the english phrase?
Not so easy :-)

I would tend to write
"Sie haben keine Berechtigung die %(classname)s Klasse als gelöscht zu
markieren oder die Löschmarkierung aufzuheben."

Kind regards,
Ralf
-- 
Dr. Ralf Schlatterbeck                  Tel:   +43/2243/26465-16
Open Source Consulting                  www:   www.runtux.com
Reichergasse 131, A-3411 Weidling       email: of...@ru...

[Roundup-devel] Fixing crash bug in German translation.

[John P. R. <ro...@cs...>]

Hi all:

The template toolkit's pofilter turned up a bug in one of the German
translations.

The translation in the following pair references the %(action)s variable
that doesn't exist in the scope of the translation call.

#: ../roundup/actions.py:49
#, fuzzy, python-format
msgid ""
"You do not have permission to retire or restore the %(classname)s class."
msgstr ""
"Sie sind nicht berechtigt, die Aktion(en) %(action)s auf die Klasse "
"%(classname)s anzuwenden."

Google translate on the original english was totally different from
the translation given above.

I then tried using google translate to turn the German phrase into
english.  Then I modifed the english phrase to get something that
looks similar to the original German translation. However I again
ended up with something different.

Can a native speaker give me a translation for the english phrase?

Thanks.

--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.

[Roundup-devel] [issue2551362] Test submission via email

[Ee D. <is...@ro...>]

New submission from Ee Durbin:

Testing submission via email after updating the TCP routing for SMTP.

----------
messages: 8119
nosy: EWDurbin
severity: normal
status: new
title: Test submission via email

_________________________________________________
Roundup tracker <is...@ro...>
<https://issues.roundup-tracker.org/issue2551362>
_________________________________________________

[Roundup-devel] [issue2551361] Support rate limiting in XMLRPC interface.

[John R. <is...@ro...>]

New submission from John Rouillard:

Continuation of issue 2551036 which put both REST and XMLRPC rate limiting together.
Splitting it out for XMLRPC as rest is done.

----------
components: Web interface
messages: 8116
nosy: rouilj
priority: low
severity: normal
status: new
title: Support rate limiting in XMLRPC interface.
type: resource usage
versions: devel

_________________________________________________
Roundup tracker <is...@ro...>
<https://issues.roundup-tracker.org/issue2551361>
_________________________________________________

[Roundup-devel] [issue2551360] Processing leftover post data when using GET request

[John R. <is...@ro...>]

New submission from John Rouillard:

Using the following CURL command:

   curl -vv -n -p -X GET --header "Content-Type: application/xml"   \
         --header "Accept: application/json" \
         --header 'If-Match: "3cda6983cac9eb51266b6e418141c53d"' \
         --data-urlencode '@op=add' \
         --header "Origin: https://example.net" \
     "https://example.net/demo/rest/data/keyword?name=sv&@verbose=2"

I see the following logged:

443544 127.0.0.1 - - [17/Jul/2024 23:12:04] "GET /demo/rest/data/keyword?
name=sv&@verbose=2 HTTP/1.1" 200 -
443544 127.0.0.1 - - [17/Jul/2024 23:12:04] code 400, message Bad request syntax 
('@op=add')
443544 127.0.0.1 - - [17/Jul/2024 23:12:04] "@op=add" 400 -

So it looks like curl is sending the data @op=add and a GET request isn't draining the
input. So Roundup (with roundup-server behind a proxy) consumes the initial GET 
header line, routes using it. Then when BaseRequestHandler.handle gets
called again, it gets the left over content on the socket.

What should happen here? I don't think it is a security issue as the only
value recognized by handle() at that point is an HTTP header.

AFAICT it just reports an error that might be useful to find poorly behaving
rest clients.

So maybe nothing has to happen?

Thoughts?

(I generated the curl command by mucking with a PATCH command created for
a different purpose and didn't feel like getting rid of the other settings. So sort
of a manual fuzzer.)

----------
components: Web interface
messages: 8114
nosy: rouilj, schlatterbeck
severity: normal
status: new
title: Processing leftover post data when using GET request
type: behavior

_________________________________________________
Roundup tracker <is...@ro...>
<https://issues.roundup-tracker.org/issue2551360>
_________________________________________________

[Roundup-devel] Roundup 2.4.0 released

[John P. R. <ro...@cs...>]

Hi all:

I'm proud to release version 2.4.0 of the Roundup issue
tracker.  This release is a bugfix and feature release, so
make sure to read `docs/upgrading.txt
<https://www.roundup-tracker.org/docs/upgrading.html>`_ to
bring your tracker up to date.

The 79 changes, as usual, include some new features and many
bug fixes.

Version 2.4.0 will be the last release to support Python
2. The next minor release, planned for mid 2025, will occur
5 years after Roundup started supporting Python 3.

Note that you should run ``roundup-admin ... migrate`` to
update the database schema version. Do this before you use
the web, command-line or mail interface and before any users
access the tracker.

You can install it with::

   pip install roundup

(preferably in a virtual environment). To download it, use::

   pip download roundup

then unpack and test/install from the tarball.

Among the notable improvements in 2.4.0 from the 2.3.0
release are:

* three CVE's have been fixed. One requires changes to your
  tracker's home directory. The other two are fixed by
  installing 2.4.0.  See
  https://www.roundup-tracker.org/docs/security.html for
  details and instructions on how to fix these in 2.4.0 and
  earlier releases.

* new classhelper component thanks to a team of students
  from CS682 at U-Mass Boston. This fixes many issues with
  the old classhelper. It is implemented as a web-component
  and needs REST interface access. It will fall back to the
  classic classhelper if REST is not available or if the
  browser does not support web-components.

* fix Windows Python installation using pip. It used to go
  into an infinite loop during install or download. Also fix
  installation of shared files (templates) so roundup-admin
  can find them.

* using ``@current_user`` as a value in a search URL for a
  user property will use the current logged in user. Now you
  can share searches like: "My issues" as "my" will become
  the current logged in user.

* login failures to the REST/XML-RPC interfaces are now rate
  limited to limit password guessing attacks.

* utf8mb4 is the default charset for MySQL. This requires
  migrating your database using the mysql client. You can
  choose to keep the older character set in config.ini.

* PostgreSQL services defined in pg_service.conf can be
  used.  PostgreSQL schemas are supported to eliminate the
  need for the roundup user to have database
  creation/deletion privileges.

* fix out of memory issue when importing larger trackers
  into PostgreSQL.

* multiple roundup-admin improvements: display protected
  properties (like creation date), better formatting of
  output, command history. Also on windows, pyreadline3 is
  supported to provide an editable interactive command line.

* an experimental wsgi performance improvement in 2.3.0 is
  now now the default and is opt-out.

* new template functions: utils.readfile and
  utils.expandfile. Javascript that is included in the
  Python core will be moved to external files and be able to
  have values from Roundup substituted in the Javascript.

* allow content-type of a template to be set from inside the
  template.  This allows returning json or xml from a
  template without a .json or .xml extention.

* fix import/export on windows to use Unix style line
  endings fixing export/import on Windows and making exports
  portable across platforms.

* various other Windows platform fixes including test suite
  fixes.

* sqlite version 1 and StructuredText support removed.

The file CHANGES.txt has a detailed list of feature
additions and bug fixes for each release. The most recent
changes from there are at the end of this announcement. Also
see the information in doc/upgrading.txt.

If you find bugs, please report them to issues AT
roundup-tracker.org or create an account at
https://issues.roundup-tracker.org and open a new ticket. If
you have patches to fix the issues they can be attached to
the email or uploaded to the tracker.

Upgrading
=========

If you're upgrading from an older version of Roundup you
*must* follow all the "Software Upgrade" guidelines given in
the doc/upgrading.txt documentation.

Note that you should run ``roundup-admin ... migrate`` for
all your trackers to update the database schema version. Do
this before you use the web, command-line or mail interface
and before any users access the tracker.

Roundup requires Python 2 newer than version 2.7.12 or Python 3
newer than or equal to version 3.6 for correct operation. (Python
3.4 or 3.5 may work, but are not tested.) Note that Roundup 2.4.0
will be the last release to support Python 2. You should deploy
new trackers with Python 3 and plan on upgrading older trackers
from Python 2 to Python 3. See the upgrade guide.

To give Roundup a try, just download (directions above),
unpack and run::

    python demo.py

then open the url printed by the demo app.

Release info and download page:

     https://pypi.org/project/roundup/

Source and documentation is available at the website:

     https://www.roundup-tracker.org/

Mailing lists - the place to ask questions:

     https://sourceforge.net/p/roundup/mailman/


About Roundup
=============

Roundup is a simple-to-use and install issue-tracking system
with command-line, web and e-mail interfaces. It is based on
the winning design from Ka-Ping Yee in the Software
Carpentry "Track" design competition.

Roundup manages a number of issues (with flexible properties
such as "description", "priority", and so on) and provides
the ability to:

(a) submit new issues,
(b) find and edit existing issues, and
(c) discuss issues with other participants.

The system facilitates communication among the participants
by managing discussions and notifying interested parties
when issues are edited. One of the major design goals for
Roundup that it be simple to get going. Roundup is therefore
usable "out of the box" with any Python 3.6+
installation. It doesn't even need to be "installed" to be
operational, though an install script is provided.

It comes with five basic issue tracker templates

* a classic bug/feature tracker
* a more extensive devel tracker for bug/features etc.
* a responsive version of the devel tracker
* a jinja2 version of the devel template (work in progress)
* a minimal skeleton

and supports four database back-ends (anydbm, sqlite, mysql
and postgresql).

Recent Changes
==============

>From 2.3.0 to 2.4.0

Fixed:

- CVE-2024-39124 - The classhelpers (_generic.help.html) are
  vulnerable to an XSS attack. A specially crafted URL that used
  that endpoint would result in running a script embedded in the
  URL. (Found/reported by Alec Romano (4rdr), fix/tests John
  Rouillard)
- CVE-2024-39125 - If the Referer header is set to a script tag,
  it will be executed when the error in the Referer header is
  reported. (Found/reported by Alec Romano (4rdr), fix/tests John
  Rouillard)
- CVE-2024-39126 - PDF, XML and SVG files attached to an issue can contain
  embedded JavaScript. This JavaScript was executed when the file was
  accessed. PDF files are now downloaded and not displayed in the
  browser. A content security policy is added for all download files
  which prevents code execution in SVG files.  (Found/reported by Alec
  Romano (4rdr), fix/tests John Rouillard)
- issue2551282 - MySQL utf8mb4 issues and
  issue2551115 - Use utf8mb4 as a default for MySQL instead of utf8
  The default database type and collations have been set to:
  utf8mb4, utf8mb4_unicode_ci and utf8mb4_0900_bin. They are (sadly)
  configurable from config.ini. Require directions on upgrading the
  MySQL db have been documented in upgrading.txt.
- issue2551063 - Rest/Xmlrpc interfaces needs failed login protection.
  Failed API login rate limiting with expiring lockout added. (John
  Rouillard)
- issue2551184 - improve i18n handling. Patch to test to make sure it
  uses the test tracker's locale files and not other locale
  files. (Marcus Priesch)
- issue2551283 - fail if version 2.4.9 of markdown2 is used, it broke
  [issue1](issue1) style links. Support markdown2 2.4.8 and earlier
  and 2.4.10 with its new schema filtering method. (John Rouillard)
- multiple flake8 fixes (John Rouillard)
- rename loop variable in 'for sendto in sendto:' (John Rouillard)
- issue2551193 - Fix roundup for removal of cgi and cgitb standard
  python modules (and FieldStorage/MiniFieldStorage). Replaced imports
  from cgi to use roundup.anypy.cgi\_ which will load the system cgi
  unless it is missing. Then it will load roundup.anypy.vendored.cgi
  and make \*FieldStorage symbols available. Roundup uses its own
  cgitb.py and not the system cgitb.py. It looks like it's the
  precursor to the system cgitb.py. (John Rouillard)
- issue2551278 - datetime.datetime.utcnow deprecation. Replace
  calls with equivalent that produces timezone aware dates rather than
  naive dates. (John Rouillard)
- when using "roundup-admin display" indent the listing only if
  headers or protected fields are requested. This makes the output
  look like it did previously to 2.3.0 if the new features aren't
  used.  Roundup-admin output was never meant to be machine parsed, but
  don't break it unless required. (John Rouillard)
- issue2551290 - pip install roundup Hangs on Windows 10
  The install under windows goes into an infinite loop using pip or
  source install. (John Rouillard)
- Document use of pyreadline3 to allow roundup-admin to have CLI editing
  on windows. (John Rouillard)
- issue2551293 - remove schema_hook from Tracker instance. Looks like
  it was an obsolete hook used for testing. Never documented and not
  accessible from schema.py.
- Fix roundup-admin security command. Lowercase its optional
  argument. Roles are indexed by lower case role name. So 'security
  User' and 'security user' should generate the same output. (John
  Rouillard from issue on mailing list by Chuck Cunningham)
- make roundup-server exit more quickly on ^C. This seems to be
  limited to windows. (John Rouillard)
- Fix error handling so failure during import of a non-user item
  doesn't cause a second traceback. (Found by Norbert Schlemmer, fix
  John Rouillard)
- Handle out of memory error when importing large trackers in
  PostgreSQL. (Found by Norbert Schlemmer, extensive testing by
  Norbert, fix John Rouillard)
- use unittest.mock rather than mock for
  test/test_hyperdbvals.py. (found by Ralf Schlatterbeck. Fix John
  Rouillard)
- disable proxy with wget in roundup_healthcheck. (Norbert Schlemmer
  Noschvie on github.com)
- support dicttoxml2.py for Roundup running on 3.7 and
  newer. dicttoxml uses a type alias: collection.Iterator that is
  dropped in Python 3.10. (found by Norbert Schlemmer, fix John
  Rouillard)
- fix duplicate html id 'password' in user.item.html in all templates except
  jinja2. (John Rouillard)
- fix unclosed file when saving index in indexer_dbm.py. (John Rouillard)
- fix task index in devel tracker so it doesn't cause a crash if all
  fields are selected. (John Rouillard)
- fix windows install. When using pip share directory is installed in
  a directory tree under the lib directory. Fix it so that Lib/share
  is used to install the share tree. The lets Roundup find tracker
  templates and translation files. (Found by Simon Eigeldinger, fix
  John Rouillard)
- fix roundup-demo, interactive mode would nuke an existing tracker.
  (Found Tonu Mikk, fix John Rouillard)
- fix detection/reporting when using a SQLite3 library without FTS5
  support. Install docs updated to state that FTS5 support is required
  when using SQLite for back end. (Found Tonu Mikk, fix John
  Rouillard)
- issue2551320: user.help-search.html doesn't respect
  properties. Setting url parameter properties when using the
  classhelp for users now shows the requested properties. (Found by
  Patel Malav and Nikunj Thakkar of the UMass-Boston CS682 Spring
  2024 class; fix John Rouillard)
- use ast.eval_literal() rather than eval() to turn CSV exported
  string values into Python object/values.
- use template's guess at Content-Type in headers only if Content-Type
  is not already set. This allows a template to set its own content
  type. For example: _generic.translate can set content type (via
  request.client.additional_headers) to application/json and return
  json from the template. This json could access the 1i18n functions
  for a javascript helper. (John Rouillard)
- when template processing raises an exception the line number is
  sometimes missing. This causes cgitb to raise a second exception
  which clobbers the info about the template issue. As a stop-gap set
  the line number to -1 so the original traceback can be seen. This
  could be a bug in ZopeTAL. (John Rouillard)
- issue2551328 - REST results show next link if number of results is a
  multiple of page size. There should be no next link. (Found by Patel
  Malav and Bharath Kanama of the UMass-Boston CS682 Spring 2024
  class; fix John Rouillard)
- issue2551264 - REST X-Total-Count header and @total_size count
  incorrect when paginated - correct values are now returned.
  (John Rouillard)
- issue2551331 - Fix repeat first/last methods. (John Rouillard)
- Fix import/export on windows. Use unix line terminating characters.
  (John Rouillard)
- Fix anydbm session/otks clear() method on windows when backed by
  dumbdbm. Also make anydbm detect the initialized database when
  using dumbdbm. (John Rouillard)
- Use of '-' directory in static_files config option under windows
  Python fixed. (John Rouillard)
- issue2551334 - number of test bugs that prevented test suite from
  running under Windows Python are fixed. WIP. (John Rouillard)
- issue2551302 - Remove support for sqlite version 1 from
  back_sqlite.py. We have been using sqlite3 for over a decade. (John
  Rouillard)
- issue2551285 - Remove StructuredText support. reStructuredText is
  still supported. (John Rouillard)
- Use roundup-demo -p option to set listening port. Was ignored
  before. (John Rouillard)
- issue2551346 - Classic tracker's statusauditor raises error if
  detectors/config.ini missing
  STATUSAUDITOR_CHATTING_REQUIRES_TWO_USERS.  The statusauditor.py for
  jinja2 and classic templates has been changed to assume that this
  option is off when the setting is missing from
  detectors/config.ini. Other templates do not implement this option.
  (John Rouillard)
- issue2551350 - Python changes for 3.12 with roundup 2.3.0. Fixes for
  cgitb.py crash due to pydoc.html.header() signature change. (Patch
  by Andrew (kragacles), applied John Rouillard)
- issue2551350 - Python changes for 3.12 with roundup 2.3.0. Fixes for
  mailer.py crash due to change in starttls signature change. (Patch
  by Andrew (kragacles), modified and applied John Rouillard)
- make classhelper link open in a new window by setting
  target="_blank". This prevents overwriting of current page with the
  classhelper if javascript is disabled. (John Rouillard)
- issue2551341 - if @columns missing from an index url, the
  group headers colspan property = 0. Add "or 100" in
  stanza's so headers span all rows (up to 100).
- fix roundup-server response requiring a 301 redirect. Did
  not set content length leading to hang/error. (John
  Rouillard)
- report basename of filename when template file is invalid
  rather than reporting a TypeError. (John Rouillard)
- Make Last-Modified header use GMT not -0000 timezone. Fix error
  reported by redbot testing. (John Rouillard)
- Send Vary: Accept-Encoding on any file that could be compressed
  even if the file is not encoded/compressed. Found by Redbot
  testing. (John Rouillard)
- make If-None-Match work for static file (@@file) case. Found by
  Redbot testing (John Rouillard)
- Send vary: accept-encoding for if-modified-since conditional
  requests where the file is not modified. (John Rouillard)
- Update JWT example in rest.py to use replacement for
  datetime.datetime.utcnow(). (John Rouillard)
- issue2551219 - document requirements of PEM file when using
  roundup-server in SSL/TLS mode. Report better error messages
  when PEM file is missing certificate or private key. (John
  Rouillard)
- Cleanup tracker index generation by roundup-server. Send
  correct Content-Length headers so HTTP/1.1 connections don't
  hang. (John Rouillard)
- Fix delay when using csv export actions. The CSV file is written
  incrementally, so we can't determine the Content-Length. When using
  HTTP/1.1, this causes a delay while the browser waits for a timeout.
  Forcing the connection to close after the CSV file is written
  removes the delay. (John Rouillard)

Features:

- issue2551323 - Remove XHTML support. Disabled option to set
  html_version to xhtml. Running roundup commands with html_version
  set to xhtml will result in an "Invalid value for HTML_VERSION:
  'xhtml'" error. (John Rouillard)
- issue2551103 - add pragma 'display_protected' to roundup-admin. If
  true, print protected attributes like id, activity, actor...
  when using display or specification subcommands. (John Rouillard)
- add -P pragma=value command line option to roundup-admin. Allows
  setting pragmas when using non-interactive mode. (John Rouillard)
- issue685275 - add pragma show_retired to control display of retired
  items when using list/table. Add pragma display_header to print
  headers for display command. Header displays designator and
  retired/active status. (John Rouillard)
- issue2551299 - support config.ini rdbms option 'service'. Allow use
  of a PostgreSQL connection service file (pg_service.conf) for
  configuring database on a per-tracker basis. Also replaces use of
  PGSERVICE env variable for single instance trackers. (From ML
  question by ivanov. John Rouillard)
- issue2550852 - support for specifying a PostgreSQL schema to use for
  the Roundup database. (Patch by Stuart McGraw; slight modifications,
  tests, docs: John Rouillard).
- issue2551274: add configurable logging for REST API when something
  fails, we now log status code and error message.
  (Ralf Schlatterbeck)
- issue2551317 - add some Jinja2 examples to customizing.txt
  document. (John Rouillard)
- multiple scripts/... updates - Python3, linting, enhancements:
  weekly-report,schema-dump.py, roundup-reminder, copy-user.py,
  dump_dbm_sessions_db.py, contributors.py (John Rouillard)
- roundup/msgfile.py can now be called as 'python msgfmt.py de.po de.mo'
  or 'python msgfmt.py -o de.mo de.po' to compile a translation file if
  GNU msgfmt is missing. (John Rouillard)
- save roundup-admin history between sessions. Load
  ~/.roundup_admin_rlrc file to set history-size persistently. Add
  pragma history_length to override for a session. (John Rouillard)
- the roundup-admin history command now dumps the journal entries
  in a more human readable format. Use the raw option to get the older
  machine parsible output. (John Rouillard)
- Multiple JWT secrets are supported to allow key rotation. See
  an updated config.ini for details. (John Rouillard)
- issue2551212 - wsgi performance improvement feature added in 2.2.0
  is active by default. Can be turned off if needed. See upgrading.txt
  for info. (John Rouillard)
- issue2551270 - Better templating support for JavaScript. Add
  utils.readfile(file, optional=False) and utils.expandfile(file,
  token_dict=None, optional=False). Allows reading an external file
  (e.g. JavaScript) and inserting it using tal:contents or equivalent
  jinja function. expandfile allows setting a dictionary and tokens in
  the file of the form "%(token_name)s" will be replaced in the file
  with the values from the dict. (John Rouillard)
- add @group to rest interface collection queries. Useful when using
  optgroup in select elements. (John Rouillard)
- roundup-demo can set the hostname in the URL using the -H
  parameter. So you can start a demo tracker that is available from
  your network using 'roundup-demo ... -B hostname -H hostname'. (John
  Rouillard)
- issue2551347 - make _generic.help.html work without property
  settings. THis applies to classic or minimal trackers. It allows use
  of classhelp without the property seting for informtion only
  (e.g. description of what a priority or status means) without being
  able to select the property in the classhelper.  Good for adding help
  for Link properties. (John Rouilllard)
- issue1525113 - notation to filter by logged-in user. Use
  @current_user with properties that are a Link to the 'user' class to
  match the currently logged in user. Allows sharing of queries like
  "Issues I created" or "Issues I am assigned to" by removing the
  hard coded user id number and replacing it with the current user's
  id. Tracker templates updated to use it. (John Rouillard from a
  patch by Jon C. Thomason)
- Add a /rest/data/user/roles REST endpoint. (John Rouillard)
- issue2551353 - Add roundup-classhelper for 2.4.0
  release. Integrate new classhelper web component to wrap
  existing classhelper link. This fixes a number of
  outstanding bugs against the current classhelper using
  current web features. (Patel Malav, Nikunj Thakkar,
  Bharath Kanama with integration by John Rouillard)
- disable spellcheck on all password fields to try to prevent
  browser from exposing passwords to external servers. (John
  Rouillard)
--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.

Re: [Roundup-devel] FYI I am locked out of sourceforge (resolved)

[John P. R. <ro...@cs...>]

Hi all:

In message <202...@pe...>,
"John P. Rouillard" writes:
>It looks like my account with sourceforge has either been taken over or
>they are having issues.

They were having issues with their authentication system.

Things are working again.

--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.

[Roundup-devel] FYI I am locked out of sourceforge

[John P. R. <ro...@cs...>]

Hi all:

It looks like my account with sourceforge has either been taken over or
they are having issues.

I am contacting sourceforge support now.
--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.

[Roundup-devel] [issue2551359] Add issue activity sparkline for index pages.

[John R. <is...@ro...>]

New submission from John Rouillard:

To provide a quick indication of ticket activity, implement sparklines
in the index page for issues. "Activity" could be changes that adds a message
(or sends a nosy notification) so it would be an indication of interactions. It
could be any change to the issue (e.g. change status or title) even if it doesn't
result in a nosy message. Which is more useful is open to testing and debate.

Add a checkbox to the issue.search.html page to enable an activity sparkline column
in issue.index.html. It could be a fixed duration and number of segments. But supporting
selection/setting of total duration/time and bucketing time (number of segments)
using a duration in a text input or a select box would be nice.

Use cases a sparkline over:

  the last week period with 7 segments (day bucket),
  the last 4 month period with 16 segments (week bucket),
  the last 1 month period with 28 buckets (day period),
  even the last 1 day (24 hours) period with 24 segments (hour bucket), or
  maybe one over the life of the ticket with a duration calculated to provide 30 or
    so segments of reasonable width (24 hours not 23 hours, 1 week not 5.5 days).

Probably will have to limit the number of buckets to some sensible number so the sparkline
isn't too long. Also how many levels to represent (y axis) is TBD. Probably a max of 10
height levels is reasonable. At that point the sparkline could look continuous rather than
discrete (with steps) given its height.

I envision adding a utils.sparkline(id, period_duration,bucket_duration) to be called from the 
templating language that generates a small svg or unicode character sparkline. It could
also be a new method for the Hyperdb item wrapper. Adding it to the wrapper might make it 
easier to use as tal:content="python:i.sparkline('1w','1d')" where i is an iterator
variable over request/batch.

Ref:

  https://docs.agilechilli.com/docs/views/sparkline/
  https://blog.jonudell.net/2021/08/05/the-tao-of-unicode-sparklines/
  https://www.pygal.org/en/stable/documentation/sparks.html

----------
components: Web interface
keywords: StarterTicket
messages: 8094
nosy: rouilj
severity: normal
status: new
title: Add issue activity sparkline for index pages.
type: rfe

_________________________________________________
Roundup tracker <is...@ro...>
<https://issues.roundup-tracker.org/issue2551359>
_________________________________________________

[Roundup-devel] [issue2551358] Remove htmll comments inside of script tags.

[John R. <is...@ro...>]

New submission from John Rouillard:

A lot of the javascript in Roundup looks like:

  <script>
     <!--
    var text_field = document.frm_help.text_preview;
    original_field=form[field].value;
    text_field.value=original_field;
    //-->
  </script>

The html comment marker and (js) commented out end html comment marker
was a trick for netscape 1.0 to not show script code (since it didn't know to
hide stuff inside a script tag). It should be removed everywhere it occurs.

Since we don't have a javascript testing frame work in place, testing the
javascript will have to be manual.

This is a simple set of edits good for a first timer.

ref: https://stackoverflow.com/questions/808816/are-html-comments-inside-script-tags-a-best-
practice

----------
components: Web interface
keywords: Effort-Low, StarterTicket
messages: 8093
nosy: rouilj
severity: minor
status: new
title: Remove htmll comments inside of script tags.
type: behavior

_________________________________________________
Roundup tracker <is...@ro...>
<https://issues.roundup-tracker.org/issue2551358>
_________________________________________________

Re: [Roundup-devel] Wierd interaction when filtering by id and boolean expression

[John P. R. <ro...@cs...>]

Hi Ralf:

In message <202...@ru...>,
Ralf Schlatterbeck writes:
>On Sat, Jun 08, 2024 at 06:35:29AM -0400, John P. Rouillard wrote:
>> 
>> With the following index url:
>> 
>> .../issue?@columns=title,id,creator&@sort=id\
>>    &@filter=id,creator\
>>    &id=1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20\
>>    &creator=3,1,-4,4,-4,-2
>> 
>> I get what I expect. None of the issues that are returned were created
>> by user3, user1, or user4 and all the issues have id's in the set 1..20.
>> 
>> Now if I change this to use: `&creator=3,1,-4,4,-4` I get issues
>> created by user3, user1 or user4. However the issues that are returned
>> include id's greater than 20. I haven't changed the parameters for
>> @filter or id, just creator.
>
>Are you sure that you have full search permissions on everything?
>We're ignoring search parameters where the user doesn't have
>permissions. Silently.

This happens if I am logged in as admin or demo user.

[...]
>> Note that even using: @creator=3,1,-4 fails the same way. So
>> explicitly expressing 'OR' using -4 causes other filters to be
>> discarded somehow. That's my working hypothesis, but how/why, got me.
>
>Yes, this looks like it. I cannot imagine, though, why a set of search
>parameters should affect other search parameters.
>
>> Can somebody try a similar experiment on their tracker and see if this
>> weirdness happens on your tracker? Bonus points if you can figure out
>> what's happening here.
>
>One thing to look into is if you have search permissions (or even better
>view permissions) on all tables involved.

As admin I certainly should and demo looks like it has the correct
perms as well.

Any other ideas?

--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.

Re: [Roundup-devel] Wierd interaction when filtering by id and boolean expression

[Ralf S. <rs...@ru...>]

On Sat, Jun 08, 2024 at 06:35:29AM -0400, John P. Rouillard wrote:
> 
> With the following index url:
> 
> .../issue?@columns=title,id,creator&@sort=id\
>    &@filter=id,creator\
>    &id=1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20\
>    &creator=3,1,-4,4,-4,-2
> 
> I get what I expect. None of the issues that are returned were created
> by user3, user1, or user4 and all the issues have id's in the set 1..20.
> 
> Now if I change this to use: `&creator=3,1,-4,4,-4` I get issues
> created by user3, user1 or user4. However the issues that are returned
> include id's greater than 20. I haven't changed the parameters for
> @filter or id, just creator.

Are you sure that you have full search permissions on everything?
We're ignoring search parameters where the user doesn't have
permissions. Silently.

> Also another wacky thing:
> 
>  &creator=3,1,-4,4,-4 == &creator=3,1,4
> 
> because the default operation is OR. However if I use the query
> parameter withotu '-4', no issues with an id greater than 20 are
> shown.

That's indeed weird.

> I have also tried this with @filter=status,creator and status=open.
> With the @creator=3,1,4 case, I only see issues with status of open. If
> I use @creator=3,1,-4,4,-4, I see issues with status open as well as
> other statuses.
> 
> Note that even using: @creator=3,1,-4 fails the same way. So
> explicitly expressing 'OR' using -4 causes other filters to be
> discarded somehow. That's my working hypothesis, but how/why, got me.

Yes, this looks like it. I cannot imagine, though, why a set of search
parameters should affect other search parameters.

> Can somebody try a similar experiment on their tracker and see if this
> weirdness happens on your tracker? Bonus points if you can figure out
> what's happening here.

One thing to look into is if you have search permissions (or even better
view permissions) on all tables involved.

Thanks
Ralf
-- 
Dr. Ralf Schlatterbeck                  Tel:   +43/2243/26465-16
Open Source Consulting                  www:   www.runtux.com
Reichergasse 131, A-3411 Weidling       email: of...@ru...

[Roundup-devel] Wierd interaction when filtering by id and boolean expression

[John P. R. <ro...@cs...>]

Hi all:

I am working on https://issues.roundup-tracker.org/issue2550698 to
document the Boolean expression syntax.

While I was working on adding examples to the user doc, I came across
an interesting issue.

With the following index url:

.../issue?@columns=title,id,creator&@sort=id\
   &@filter=id,creator\
   &id=1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20\
   &creator=3,1,-4,4,-4,-2

I get what I expect. None of the issues that are returned were created
by user3, user1, or user4 and all the issues have id's in the set 1..20.

Now if I change this to use: `&creator=3,1,-4,4,-4` I get issues
created by user3, user1 or user4. However the issues that are returned
include id's greater than 20. I haven't changed the parameters for
@filter or id, just creator.

Also another wacky thing:

 &creator=3,1,-4,4,-4 == &creator=3,1,4

because the default operation is OR. However if I use the query
parameter withotu '-4', no issues with an id greater than 20 are
shown.

I have also tried this with @filter=status,creator and status=open.
With the @creator=3,1,4 case, I only see issues with status of open. If
I use @creator=3,1,-4,4,-4, I see issues with status open as well as
other statuses.

Note that even using: @creator=3,1,-4 fails the same way. So
explicitly expressing 'OR' using -4 causes other filters to be
discarded somehow. That's my working hypothesis, but how/why, got me.

Can somebody try a similar experiment on their tracker and see if this
weirdness happens on your tracker? Bonus points if you can figure out
what's happening here.

Thanks.
--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.

[Roundup-devel] [issue2551357] rest responses missing cache-control header

[John R. <is...@ro...>]

New submission from John Rouillard:

The missing cache control allows the response to be cached for a longer period of time
than we probably want.

Because all writes to the tracker via REST must include an ETag, there shouldn't be any
cases of lost edits. But users might have their edits rejected if they get a stale response.

Also we need different lifetimes for different levels of the REST hierarchy.

For example non-issue levels: /rest, /rest/data can probably be cached for a while.

/rest/data/issue should probably have the same cache headers as for an index page.

/rest/data/issue/1 or /rest/data/issue/1/title have the same cache headers as an item page.

/rest/data/roles are probably the same as /rest since it won't change without a
server restart.

----------
components: API, Web interface
messages: 8088
nosy: rouilj
severity: normal
status: new
title: rest responses missing cache-control header
type: behavior

_________________________________________________
Roundup tracker <is...@ro...>
<https://issues.roundup-tracker.org/issue2551357>
_________________________________________________

[Roundup-devel] [issue2551356] Add etag header when If-Modified-Since GET request returns not-modified (304)

[John R. <is...@ro...>]

New submission from John Rouillard:

You can do conditional requests in HTTP using an ETag with the If-None-Match header or a date 
with the If-Modified-Since header. This is supported for static files (/@@file/).

if-modified-since (ims) has been supported for a while, but redbot.org reports that it
is not supplying the ETag header when returning a 304 not modified response.
I tried to fix this, but it's turning into a hairball so I am deferring the
fix/reorg until after 2.4.0 is released.

One issue I have is that the etag is different for different Content-Encodings. In
the REST flow, I just append the content-encoding (gzip, zstd, br) to the base etag
(inode-length-modifiedtime).

When I implemented support for If-None-Match, I was able to return the right
encoding sensitive etag because the encoding sensitive etag is sent by the browser.

With IMS, I don't have any hint about the encoding originally used to deliver the file.
I might have to make a best guess by treating it as a non-conditional get. Doing all
the work to figure out what headers and encoding I would send.

----------
components: Web interface
messages: 8087
nosy: rouilj
severity: normal
status: new
title: Add etag header when If-Modified-Since GET request returns not-modified (304)
type: behavior

_________________________________________________
Roundup tracker <is...@ro...>
<https://issues.roundup-tracker.org/issue2551356>
_________________________________________________