Batch Import

Help
2009-01-31
2013-05-28
  • Colon Grease
    Colon Grease
    2009-01-31

    I would like to be able to batch import RIS or ENW citations from the web interface without having to copy and paste text.

    Is there a way to multi-select RIS citations in a folder and create records for them?

    Furthermore, I would like a way to do this and then easily add associated pdfs to the records without having to locate them individually within the entire library of references.

    Basically, I have stacks of 10-20 journal article pdfs (as do other users over the web) that I'd like to add to my refbase database with citations. Ideally, there would be a tool that searched for citation information and automatically created entries for them, but I won't hold my breath on that. I'd like to be able to have the 10-20 RIS or ENW cite files in one place, upload them, and then link them one-by-one to the pdfs. Doing this one at a time is fairly tedious.

    Suggestions?

     
    • Just concatenate the RIS files together--a single file could hold any number of records.

      As for the PDFs: if you have a systematic naming scheme for them, you could potentially modify the import scripts to use that default scheme & can manually upload the PDFs in one go.

       
      • Colon Grease
        Colon Grease
        2009-01-31

        Concatenating is a pain for users accessing via their browser in Windows.

        Furthermore, this still doesn't allow you to add pdfs for the multiple records without either carefully orchestrating a series of browser "back" commands to get back to the list of multiple records added, or alternately sift through the entire library of records for those that were just added.

        An automatic DOI lookup would be a perfect solution to these problems, and eliminate the need to download RIS files at all, and eliminate the need to go back to the web to get RIS records for pdfs already in one's personal library that they'd like to add to the refbase database.

        Does this make sense?

         
    • > Ideally, there would be a tool that searched for citation
      > information and automatically created entries for them, but
      > I won't hold my breath on that.

      This was actually one reason why we implemented the DOI lookup feature (which is part of refbase-0.9.5). We had plans to let people upload just their PDFs, then scan the beginning of these PDFs for a DOI (or PMID etc), fetch the associated metadata and finally create a record (with the PDF attached). This didn't make it into the refbase-0.9.5 release, but something like this is still on our wish list.

      Adding to Rick's note about the PDFs: if your PDF naming scheme can be generated from the record metadata, then you could also use a batch SQL update action to create the correct file paths & names in the 'file' field.

      Let us know if you need further help.

      Matthias

       
      • Colon Grease
        Colon Grease
        2009-02-01

        >This was actually one reason why we implemented the DOI lookup feature (which is part of refbase-0.9.5). We had plans to let people upload just their PDFs, then scan the beginning of these PDFs for a DOI (or PMID etc), fetch the associated metadata and finally create a record (with the PDF attached). This didn't make it into the refbase-0.9.5 release, but something like this is still on our wish list

        This is exactly what I want. Any idea when this will be implemented?

         
    • Juan Blanco
      Juan Blanco
      2009-01-31

      Perhaps improving the ability of refbase to import zotero (I don't know if others) could be a good solution. Zotero is good collecting references from diverse sources and it is possible to associate files or links to it. Unfortunately, the links are only reported in their "report" generation (a tagged list), and not in other exportation formats as MODS, RIS,... I suppose that it would not be difficult to write some code to import that, but I an not a programmer and it is not my time. So, just to let anyone to consider the possibility.

       
    • Hi Juan,

      thanks for the suggestion. It's a good one IMHO.

      > Perhaps improving the ability of refbase to import zotero
      > (I don't know if others) could be a good solution.

      Right, I'd also love to see an Zotero -> refbase importer.

      Previously, we've talked about this quite a bit among developers. Ideally, I'd like to develop a little "zotero2refbase" Firefox plugin which would hook into Zotero. Within Zotero, the plugin would provide a new menu item which would allow to directly export/upload selected items to refbase.

      The Zotz plugin is doing something similar to establish integration between Zotero and Citeline.

      http://simile.mit.edu/wiki/Zotz
      http://citeline.mit.edu/
      http://simile.mit.edu/wiki/Citeline_User%27s_Guide

      So it may be actually quite easy to adopt this plugin for use with refbase.

      The problem is just that we need to find somebody who is willing to look into this. Unfortunately, I myself lack the time to do so. But I imagine it would be a very nice little project.

      Matthias

       
    • >> We had plans to let people upload just their PDFs, then scan the
      >> beginning of these PDFs for a DOI (or PMID etc), fetch the
      >> associated metadata and finally create a record (with the PDF
      >> attached).

      > This is exactly what I want. Any idea when this will be implemented?

      I cannot give you a reasonable time frame, sorry. My time available for development is currently close to zero. Please don't get me wrong, I'd love to implement this, and I'll try to look into it. But I can't promise anything. Thanks for your understanding!

      Of course, we wholeheartedly welcome anybody who's willing to help out.

      ToDo items would be (I might be overlooking something):

      1. Make the import script handle upload of PDF files

      2. Integrate refbase with the pdftotext utility (which would extract the text from the PDF)

      3. Create a function that extracts the DOI/PMID/etc from the PDF text

      4. Create a wrapper function that initiates the PDF conversion, loops over each PDF to extract the IDs, initiates the fetching of metadada, and finally triggers the creation of records (with the renamed PDFs attached).

      I'm sure it's all doable. But it probably won't be done in an hour.

      Matthias

       
    • Colon Grease
      Colon Grease
      2009-07-28

      Still waiting for this killer feature!

       
      • Which feature, in particular, are you waiting for?  We discussed a handful of features here six months ago.