Re: [Roundup-users] Missing messages

[John P. R. <ro...@cs...>]

Hi Thomas:

In message <202...@in...>,
Thomas Arendsen Hein writes:
>* Tom Ekberg <te...@uw...> [20221201 23:03]:
>> It looks like user 33 unlinked message 1514. It looks like all
>> this did was to remove it from the issue_messages table. Should
>> roundup have 'retired' message 1514 too?
>
>I consider the message removing mechanism in Roundup to be
>technically correct, but in reality it is broken:

Well unlink would be more accurate than remove, but also require more
knowledge Roundup's internals than a user should have.

>There is a "remove" button next to each message, which

I interpreted the remove button is on the issue item display as
removing the message from the issue. That is different from removing
the message from the system. This makes sense given Roundup's model of
retiring things rather than deleting them. Keeping history always
seemed like a core concept: e.g. immutable messages.

That being said, in my trackers, only certain roles see the Remove
button on a message.

Also similar to Ralf's trackers, my trackers allow files and messages
to be linked to multiple issues. Only if the user has the ability to
see the issue with the link to a file/message can they see the
file/message. If removing a file/message also purged it, it could
delete info from another ticket.

>- doesn't ask if the message should actually be removed, it just
>  removes it from the issue without an easy way to undo mistakes.
>  -> Risk of losing (easy access to) data

I agree with this. See:
https://issues.roundup-tracker.org/issue2550536 for a possible
solution (but with some concerns).

It would be nice to have a popup/redirect to _generic.unlink.html that
says something like:

========================
    Remove file234 <insert filename here> from issue2345. Note this
    will not delete the data for the file. That data can be seen by
    going to <tracker_basepath>/file234.


    [remove file reference from issue] [cancel]

========================

This would be great for the supplied tracker templates.

>- doesn't actually remove the message, just unlinks it from the
>  issue, so if a user posted information that shouldn't be visible
>  in the tracker, it is still there, while the user might believe
>  it has been deleted!
>  -> Risk of sensitive information being stored in the tracker
>     without the user realizing that it is still there
>
>In our trackers we have removed the remove button, and users have to
>contact an admin to edit or remove messages (or files), which also
>includes shredding and truncating the actual file on disk, and in

Shredding might only work on spinning rust storage. Due to wear
leveling on SSD writing "over" bytes on disk may just write somewhere
else and update the block pointer for the content. Leaving the
original data on disk.

>case of attached files with a file name including sensitive
>information (yes, people do that :-/) also editing the journal

See below for addressing this at (1),

>because otherwise the original file name is still visible.

There is a script in the distribution scripts/spam-remover that purges
old files.  I think it also clears the history. That might be useful
to you. But if the file contents have to be shredded, you would need to
do that first.

Overall, this is similar to purging/removing users
(https://wiki.roundup-tracker.org/RemovingUsers).

Have you thought about adding a "Purge File/Message" button on the
msgNN and fileNN pages that calls a purge action. The action does the
following (which I think touches all your points above):

   1. unlink file/message from all associated issues

   2. clean up purged item metadata:
       * for file:
            * set the filename to [purged by userNN on date]. This
              removes the need to remove (1) the journal entry on
              the issue(s). The filename info comes from the file
	       metadata.)
            * set contenttype to to text/plain
       * for msg:
            * maybe empty the recipients
       * for either maybe clear the date to 1970-01-01

   3. truncate file or use code from something like
        https://github.com/r3nt0n/wiper/blob/master/r3ntlib/wiper_ops.py
      or an os.system or subprocess.call(shell=False) to
      shred(1) to overwrite data on disk file before truncation. (*)

   4. purge/clean up journal/history for the file/message. For
      messages this can probably be skipped. For files, you want to
      remove the journal entry that records the rename if it is
      sensitive data.

Note this process does leave the link/unlink history references in the
issue, but looking at them should show something like:

  later
  date user ... files: - [purged by admin on 2021-12-21]

  date user ... files: + [purged by admin on 2021-12-21]

I don't consider that an issue.

There is an open issue

   https://issues.roundup-tracker.org/issue2551133

to enhance roundup-admin pack to include the ability to run

        pack 0d file200

to purge all journals on file200 starting now (0d) back to (but not
including) the original create. I think that would work for your use
case.

Adding a 'purge' command for file based and user classes to
roundup-admin would be useful. With suitable checks (user verifies
filename, creating user at minimum) and warnings I think it would
work.

Adding this as an action to the core accessible by admins, but not
exposed in the UI would be reasonable. The UI exposure would be part
of customizing the tracker.

--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.