#2371 4.2b.0 forced re-import destroys media tables

closed-fixed
None
9
2008-11-26
2008-11-24
Gerry Kroll
No

I have just upgraded my test site from version 4.1.6 (SVN) to version 4.2b.0 (SVN).

After the initial start of the newly upgraded 4.2b.0, I am taken to the Manage GEDCOMs page (as documented and expected). I am informed (as documented and expected) that the GEDCOM needs to be re-imported.

After the re-import, I no longer have ANY of the previous media information -- No media objects, and no media object links.

It appears as though the re-import assumes that the GEDCOM being re-imported contains the required media information.

This is NOT the case: My GEDCOM was previously re-uploaded and does NOT contain any media information because the off-line GEDCOM maintenance program (FTM) doesn't understand about media. That stuff is ALL in the database tables. During the re-upload, I'm allowed to tell PGV to preserve existing media tables, and that's the option I take.

Discussion

  • Greg Roach

    Greg Roach - 2008-11-24

    The "auto upgrade" works by simply setting $GEDCOMS[$GEDCOM]['imported'] to false.

    In uploadgedcom.php, at line 438, there is some logic that skips the option to set the $keepmedia option under certain circumstances.

    And since you never got the keep-media option, you wouldn't have been able to select it, so it defaulted to false.

    I need to understand why we have the logic to skip the $keepmedia option - we clearly want it in this case.

    I'm working on this now.

     
  • Gerry Kroll

    Gerry Kroll - 2008-11-24

    This is a knotty subject:

    I think the problem really is that the incoming GEDCOM _may_ contain media information.

    The logic should be such that the media tables are always preserved. When the incoming GEDCOM contains media information, that information should replace whatever is already in the database.

    It's entirely possible for the incoming GEDCOM to contain incomplete media information. This will happen when a GEDCOM containing no media stuff is imported and we then add media items using Manage Media (or whatever). The new media items appear in the GEDCOM, but the existing stuff doesn't. Perhaps we can cure this "oversight" by exporting the imported GEDCOM immediately. In other words, synch the GEDCOM with the database immediately after doing an Import.

    We also have a problem with the forced Import: There's no guarantee that the GEDCOM we're suggesting should be re-imported has been synchronised with the old database. We need to do an automatic Export, followed by an automatic Import.

     
  • Greg Roach

    Greg Roach - 2008-11-24

    If the user isn't sync'ing their edits, then the import page will show a big red warning warning them of this, and telling them to go back and download it. I also made sure that the download is available when the DB is unimported. This covers this case.

    <<The logic should be such that the media tables are always preserved.>>

    It would seem to me that you should *always* be given the option to specify this.

    Which leaves me a little confused as to why lines 438 and 483 deny this option.

     
  • Greg Roach

    Greg Roach - 2008-11-24

    <<Perhaps we can cure this "oversight" by exporting the imported GEDCOM immediately. In other words, synch the GEDCOM with the database immediately after doing an Import.>>

    In this RFE
    https://sourceforge.net/tracker/index.php?func=detail&aid=2271529&group_id=55456&atid=477082
    I proposed an "export" facility, which would be the reverse of import. i.e. it would take the DB contents and overwrite the file.

    Thus the user could export+import in these circumstances.

    (Incidently, I proposed this because it takes half an hour to download/upload my gedcom, which achieves the same thing).

     
  • Gerry Kroll

    Gerry Kroll - 2008-11-24

    Greg:
    I don't know either. You'll have to ask Roland -- the Import is basically his code. John might know too.

    In the meantime, let's just remove the code that prevents the option from displaying. The Help text is quite good at explaining the impact of this decision.

     
  • Greg Roach

    Greg Roach - 2008-11-24

    I can confirm that replacing 438 and 483 with "if (true)" fixes the issue.

    I guess the logic is trying to prevent an unecessary option from being displayed when the gedcom is imported for the first time, into an empty database. Maybe we just need a better test for the empty database. i.e. count the indi records rather than looking at whether it has been imported.

     
  • Gerry Kroll

    Gerry Kroll - 2008-11-24

    It's actually simpler than that:

    If there are no media objects in the database, there's no need to preserve the media and media links tables.

    You can't have media links without matching media objects.

     
  • Greg Roach

    Greg Roach - 2008-11-24

    OK - I've submitted the "if true" changes, because we need a fix ASAP.

    We'll now get a unnecessary (but harmless) option to keep media when importing a new gedcom for the first time.

    That's easily fixed, but I'll do it tomorrow morning, as it's getting late now.

    Sorry for the problems.

     
  • Greg Roach

    Greg Roach - 2008-11-24

    The "outer" checks still need this, as it controls the "delete existing data" option.

    The "inner" check already checks for the existence of media records, as you suggest, so that's OK.

     
  • Greg Roach

    Greg Roach - 2008-11-25
    • status: open --> pending-fixed
     
  • Greg Roach

    Greg Roach - 2008-11-25

    I've just submitted the final part of this. Rather than make assumptions as to the state of the database, we now;

    physically check the DB for the presence of *any* records to decide whether to prompt for overwrite.

    physically check the DB for the presence of *media* records to decide whether to prompt for keep_media.

    I've tested this for new/existing gedcoms, with/without media, and the "auto upgrade" option. All seem OK to me. You might want to check....

     
  • Gerry Kroll

    Gerry Kroll - 2008-11-26
    • status: pending-fixed --> closed-fixed
     

Log in to post a comment.