Newbie question

Help
Marco
2008-09-09
2013-05-28
  • Marco

    Marco - 2008-09-09

    I just found refBase. Before installing it. I'd like to understand if it fits my needs.
    In particular, I would like to know what happens if the same Endnote database file is imported twice.
    Will I have duplicate entries?

    Thanks.
    Best regards,
    Marco

     
    • Matthias Steffens

      Hi Marco,

      > I just found refBase. Before installing it.

      Note that you can try out refbase online (i.e. without installing it):

      current release version:   http://demo.refbase.net/
      latest development version:  http://beta.refbase.net/

      > what happens if the same Endnote database file is imported twice.
      > Will I have duplicate entries?

      Currently yes. Duplicate detection upon import is a planned feature, but it hasn't yet been implemented.

      The latest dev version has a feature that will allow you to find duplicate entries for a given query *after* records have been imported. But, obviously, this isn't the same as dupe detection on import.

      More info about the current & planned dupe detection features is available at:

      http://duplicates.refbase.net/

      Let us know if you've got further questions.

      Matthias

       
      • Marco

        Marco - 2008-09-09

        Hi Matthias,

        thanks for your answer. This feature is essential to me because I am thinking about this scenario.

        My group keep all bibliographic in refBase.
        Some of us use Word, some latex. And this is where refBase could help.
        When a user wants to use the citation data, he export the file to his favorite application.
        When a user wants to add a new bibiographic entry he should import it directly in refBase.
        However, what most certainly will happen, is that some user will use Endnote to download and manage bibliographic information into the same exported endnote file, and only *then* import the modified endnote file with the new addition to refBase.
        This will create the new entries but a lot of duplicate entries.

        Maybe this use scenario is not how refBase was intended to be used, but that's the only solution I could find to allow different users to use different biblio data format/application (Endnote, Bibtex, etc.), and still share a common base of data.

        What other options do I have?

        Kind regards,
        Marco

         
    • Matthias Steffens

      Hi Marco,

      your envisioned scenario makes sense, and is in fact what refbase was made for: manage and share the academic literature of a group, and facilitate the output to different bib & office apps or editors.

      http://sharing.refbase.net/

      > However, what most certainly will happen, is that some user will
      > use Endnote to download and manage bibliographic information into
      > the same exported endnote file, and only *then* import the
      > modified endnote file with the new addition to refBase.
      > This will create the new entries but a lot of duplicate entries.

      Yes. However, from my experience (with the people & the refbase installation of my institute) this is only a problem during the initial phase. Eventually, most people will get accustomed to the new workflow, i.e. they'll learn to import their records into refbase right away. Of course, this depends on the willingness of people to adopt something new, and your own mileage may vary.

      It may also help to make import into refbase as easy as possible. Recently, we've added direct import of basic metadata from DOI(s). And for people who'd be willing to use third-party bibliographic (i.e. non-Endnote) apps, there may be more options. As an example, the "Bookends"[1] reference manager (for the Mac) offers direct "Upload to refbase" functionality from within the program. And we've thought about  something similar for Zotero[2] and other reference managers. But in most cases we'd need the willingness of the respective developers to take this further.

      [1]: http://www.sonnysoftware.com/
      [2]: http://www.zotero.org/

      > Maybe this use scenario is not how refBase was intended to be
      > used, but that's the only solution I could find to allow different
      > users to use different biblio data format/application (Endnote,
      > Bibtex, etc.), and still share a common base of data.

      It is what refbase was intended for. But we're a small development team, so each feature takes some time until it gets implemented.

      > What other options do I have?

      If you haven't already, you may want to take a look at the other web-based apps or centrally-hosted websites listed at:

      http://en.wikipedia.org/wiki/Comparison_of_reference_management_software

      Maybe there's something that better fits your needs?

      Matthias

       
    • Marco

      Marco - 2008-09-11

      Hi Matthias,

      you are probably right, that it's a matter of getting people used to the workflow.
      However I think that it is actually convenient to work only in one tool (say Endnote), on the same exported file (using it and adding new references) and only then import back the modified file into refBase.

      I am not a user of Endnote myself, so I may be wrong, but I think the workflow you're suggesting implies the following:
      1) Export your references to an endnote file
      2) Use it: write part of your paper...
      When you need to add a new reference (this *can happen* while your working on the paper):
      3) import it to refBase
      4) export it to Endnote
      5) merge it into the Endnote file you were using.

      Am I wrong?
      If not, I think most of my colleague would prefer to import the Endnote file at the end of process.

      In this cases (multiple import of the same file) a detection mechanism at import time that detects if a new entry is *exactly* the same as one in the database would solve the problem  (with no need to check if *similar*, which is maybe more complicated).

      I just posted the same problem as a response to a thread in Open discussion. Sorry for cross-posting...

      Kind regards,
      Marco

       
    • Matthias Steffens

      Hi Marco,

      I fully agree with you that refbase should check for duplicates upon import. This will come, but since we're not there yet, I'm trying to suggest some workarounds below. They may require some extra work from your people but I think the benefits of having a shared reference database outweighs the extra effort. Things will get easier in the future.

      > I think that it is actually convenient to work only in one tool
      > (say Endnote), on the same exported file (using it and adding new
      > references) and only then import back the modified file into
      > refBase.

      Yes. But how about only exporting those records from Endnote that weren't already imported into refbase? Users could use the search features of Endnote to identify all records that weren't exported from refbase (see below), then import only found records into refbase. Alternatively, users could be trained to somehow mark all newly imported records in Endnote (e.g. by marking the record's checkbox, tagging them, or adding them to an "import to refbase" group or library). Then, it would be easy to just import those new records into refbase. The involved steps would then become:

      1) export your references from refbase to an Endnote file
      2) use it: write part of your paper...
      When you've added new reference(s) to your Endnote file:
      3) select all new references & export them from Endnote
      4) import them into refbase

      Note that when exporting records to BibTeX/Endnote/RIS, the latest refbase SVN version includes a note with each record that reads something like this:

      exported from refbase (http://.../show.php?record=16197), last updated on Fri, 20 Apr 2007 23:37:31 +0200

      This could be used in Endnote to search for all records that don't contain this string in the "Notes" field. I just tried it in Endnote 10.0.2.2007 and the following search pattern would accomplish this:

      Search for:____________In Field:
      1______________________Record Number__Is greater than or equal to__Not
      exported from refbase__Notes__________Contains

      i.e. "find all records where the Record Number field is equal to or greater than 1 but exclude all records whose Notes field contains the string "exported from refbase".

      The only question would be how to deal with further record additions since one would still need to somehow tag all records that have been imported into refbase already.

      Note that things can be way easier for users who use reference manager apps with text-based source files (such as BibTeX). E.g. I've added a feature to the 'refbase' command line client which allows to append found records to a local BibTeX or MODS/SRW XML file (if they don't yet exist in that file), and update existing records in that file if their modification date on the server is more recent. This allows to keep a local BibTeX file in sync with record additions or changes made in an online refbase database.

      http://cli.refbase.net/

      With the 'refbase' command line client installed, one could execute following command on the command line to update a local 'paper.bib' file with all additions or changes that were made to the refbase literature data of user "msteffens":

      refbase -l=msteffens -F=bibtex -A=paper.bib -B=1

      So, after one has added or changed a record in refbase, one would only need to re-execute this command, and be done with it.

      When writing a paper in LaTeX, one can even pass the 'paper.aux' file to the 'refbase' command line client, and it will extract all cite keys from the 'paper.aux' file, fetch all missing or changed records from refbase, and update the BibTeX file ('paper.bib') accordingly. As an example, for a refbase user with user ID "2", the appropriate 'refbase' command would be:

      refbase -u=2 -E=paper.aux -F=bibtex -A=paper.bib -B=1

      Of course, these options wouldn't work for Endnote users, but I thought I'd mention them since your group seems to have some BibTeX users as well.

      > In this cases (multiple import of the same file) a detection
      > mechanism at import time that detects if a new entry is *exactly*
      > the same as one in the database would solve the problem  (with no
      > need to check if *similar*, which is maybe more complicated).

      But what happens if the record gets changed in refbase which means that the records wouln't be exactly the same anymore. Also, the import process might alter some parts of the record (e.g. punction & whitespace in the 'author' field, etc). So I think, a smart duplicate detection would be needed in any case.

      Note that the latest refbase SVN version does already have a duplicate detection mechanism that can (to some degree) detect records that are similar but not entirely equal. It's just that we haven't integrated this mechanism into the import process.

      If you haven't already, you may want to have a look at:

      http://wiki.refbase.net/index.php/Handling_of_duplicates#Features_in_development

      and try out (play with) the duplicate detection features of the newest refbase version.

      The duplicate detection mechanism & speed could be further improved by including text/record statistics with each record (such as the number of non-stop words or vowels in the 'title' field, etc).

      Matthias

       
      • Marco

        Marco - 2008-09-11

        Hi Matthias,

        thanks again for your answers. I am going to install refBase soon.

        The workflow you depicted for Endnote users looks reasonable and I'll try to get my colleague to get accostumed to it.

        The command line feature for BibteX users is really cool.

        And finally, yes, I already tried the duplicate mechanism in the demo database.
        However I am going to install the release version (unless you suggest me to install the beta): I imagine I'll be able to switch my references to the new version when it'll be released.

        Kind regards,
        Marco

         
    • Matthias Steffens

      Hi Marco,

      > However I am going to install the release version (unless you
      > suggest me to install the beta): I imagine I'll be able to switch
      > my references to the new version when it'll be released.

      I f you need features from the "beta" version, then you might want to look into installing from SVN (see below). OTOH, if you can wait, then of course, it's a good idea to stay with the release version.

      That said, the refbase version that is available in the "trunk" of the refbase Subversion (SVN) directory is stable, and it is in every way better than the current release version (refbase-0.9.0).

      http://svn.refbase.net/

      Following forum thread contains an overview of the major additions, changes and fixes since refbase-0.9.0:

      http://sourceforge.net/forum/forum.php?thread_id=1741877&forum_id=351913

      We hope to release this new refbase version "soonish". But, unfortunately, I cannot promise an exact release date.

      We strive to always keep the refbase version in the SVN trunk stable, so you should be fine to deploy from SVN. Also, it should be always possible to update any refbase version (no matter whether it stems from a file release or from SVN) by executing the 'update.php' script in your browser.

      http://update.refbase.net/

      If you are (or are willing to make yourself) comfortable with getting stuff from SVN, then there isn't really a difference between a file release and an SVN release (as available in the SVN trunk).

      HTH, Matthias

       

Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks