Re: [Gramps-devel] relative path

[James G. S. (jim) <jg...@sa...>]

James G. Sack (jim) wrote:
> Benny Malengier wrote:
>> Jim
>>
>> where you serious about the fact you would try on the relative path problem?
> 
> Yes, I would like to help. I will first have a look at your
> notes/outline and see if I can get an overall picture, then get back to
> you, but probably not before 24 hours or so.
> 
>> <snipped outline>

I am anxious to help here, but I'm afraid I still don't _really_
understand just what the problem is. No doubt, a big part is my relative
newness to details of the UI (and maybe development history).

Plunging in regardless...It seems in cases like this, a common
difficulty is that the terms, assumptions or objectives are unclear, and
so I thought it might help (at least me) if I try putting down some some
background ideas, use-cases/user-stories, and remaining questions as I
envision the system interface and behavior.

If this is worthwhile, maybe it should go on a wiki page. If this is all
 irrelevant to the present task, then somebody just tell me I'm looking
in the wrong direction, and I'll stop wasting (everyone's) time.

Anyway, here are some statements of media handling background
assumptions and requirements, as I see it (meaning, of course, subject
to discussion, correction, negotiation, etc).

  Forgive me if I belabor some things which are fairly obvious
  .. I believe this _is_ going somewhere. ;-)

 a1. media objects are not "part of" the Gramps-managed (and perhaps of
most genealogy) databases, rather each media object ultimately refers to
the content of some file in the category commonly called "media", eg, an
image file, a sound file, etc.

 a2. Gramps data does include information about the media, including
(usually) an association with some genealogy entry (eg, a person), and
also (usually) the location of the media file within the computer's file
system.

 a3. location data is sometimes called (filesystem-) "path", or maybe
simply "link".

 a4. Gramps can and will display certain media objects -- at present,
most types of images show as thumbnail images. Full-size display or
presentation of other types of media objects, in general, require
appropriate media tools provided as applications or operating system
components. Furthermore, the media files may be accessed (and
manipulated) directly via usual operating system tools without Gramps
participation in that process. In particular, media files may be erased,
renamed, moved to different locations by operations external to Gramps.

 a5. Media objects can become "missing" by being erased or moved from
the location Gramps has in it's records. And the Gramps location data
may be edited by the Gramps user as desired (including erasing the
location).

==> all the above are possibly useful in a user manual, even though they
    are likely obvious to gramps developers as well as non-novice
    users[0]. (see "Notes", at bottom of email)

==> from the user viewpoint, the handling of media file location is
    somewhat of an unimportant detail (so long as the user can see the
    thumbnails and/or launch an application). I am not even sure there
    are any questions needing user decision on export. I do think there
    are a number of questions to be resolved when it comes to
     importing genealogy data accompanied by media.

==> following is a simplified list of user actions involving paths

 u1. specify path upon media object creation (addmedia)

 u2. change paths for various reasons (media props editor, other?)

 u3. ("batch") change paths by string replacement (media manager)

 u4. export a database with media references; database only, no files

 u5. export a package of database plus media files (WritePkg)
     (respect tree structure seems advisable if not mandatory)

 u6. import database containing media references but no media files

 u7. import a package of database plus media files (ReadPkg)
     (in general, from another computer; another filesystem)
     (where to put these files -- perhaps the big question)
     (handling overwrite is another big concern, I would think)


I can think of additional interesting features/requirements, but this
may be enough to start -- or have I omitted something essential?


==> Technical details that start to make sense and raise questions

 q1. exporting gpkg -- respect tree structure (ok. std archiving [1])

 q2. export empty dirs and leading dir-chain -- keep or prune & trim

 q3. importing gpkg into virgin database -- seems like gpkg extraction
might be easier and more versatile if creation always produced a top
level dir containing all the media files. Then, for instance the extract
operation might more easily extract those media files directly to some
final target location determined via say user-choice.[2]

 q4. importing gpkg into database having live media objects -- need a
policy: overwrite, rename, ask-user.[2][3]


Discussion
----------

In q3, I have injected a personal preference that media should be rooted
in it's own dir within gpkg, so that gpkg always looks like:
  data.gramps
  mediadir
If for no other reason, this appeals to me because mediadir is like a
stand-alone archive that might offer handling or visualization benefits.
To continue my biases, I think I would prefer that any leading empty
dir-chain be trimmed from the tree stored under media, thus the media
dir should have at least one media file and optionally have directories.
I'm also inclined to think that there should normally be no empty leaf
directories. This would be _my_ normalized media tree. I don't know
what, but the gpkg might conceivably contain additional metadata not in
gramps.data or in the media tree.

in q4, I have identified the tarfile module which I would like to study
some more, and maybe write a wrapper and/or some tests once we decide
what the operational overwrite policy should be.

In my way of looking at things, I don't really encounter a relative
versus absolute path question. Perhaps it's because I haven't thought it
through to lowest detail?



- - -
Notes
- - -
[0] I can envision the need and value of advice to the user on how he
might consider organizing his media files. In particular, pointing out
possible organizational benefits when he chooses to import or export things.

[1] standard archiving practices have evolved some security conventions
that I consider sensible and important.  By default, extracting an
archive should not produce any results above the current working
directory, either via absolute paths, backtracking(via "../") or (say)
with symlink tricks. As part of this convention, both the archive
creating and extracting convention strips leading absolute path root
("/") and backtracking-dots (unless told to preserve them). This is how
the *nix tar and similar archivers work. I think many experienced
sysadmins would feel uncomfortable absent that convention. The python
tarfile module seems to operate in -P (preserve absolute paths) by
default, at least in creating archives. I think I'd like to look more at
it to see if I need be worried. ;-)

[2] python module tarfile is presently used for extraction, I believe I
would want to explore whether the overwrite behavior could be
modified/controlled  (or pre-checked for user approval). It's possible
some customization or custom wrapping might be needed to achieve
whatever overwrite behavior we decide to use.

[3] ask-user would seem a friendly option. I might envision either a
pre-scan to get a list of files that would overwrite to get an ok (or
maybe selective checkbox ok?), or a dialog that has a "yes/no" as well
as a "repeat this choice on all further files" option.


So, does any of this make sense?

Regards,
..jim