Year Field in RIS import

Help
Irish
2009-04-14
2013-05-28
  • Irish
    Irish
    2009-04-14

    The records I am trying to import have the Pub Date (Y1) as a full date (mm/dd/yyyy). They are converted properly when exporting from Reference Manager (yyyy/mm/dd/). These records are from newspapers so the Pub Date is the date of the newspaper - we need the entire date to transfer. I've tried tweaking the mySQL database field (year & issue); having Y1 go to the issue field instead of the year field; and I can't get the entire date to import into any field in refbase. What am I missing?

    If I have to put the date into a different field in Reference Manager, I guess it'll have to happen that way. But was sure hoping I could put that date into another field if it won't import completely into the year field. I have two other larger databases to transfer to refbase and will need this information for them as well.

    Thanks for any & all assistance.

    C

     
    • Irish
      Irish
      2009-04-20

      I am under the gun a bit & so want this to work for our Librarian-types. Can anyone give me any information at all? I have tried using several fields to import the Y1 data into but they all simply import the year (notes imports correctly but there is already data in that field so doesn't make much sense (& Y1 will not import into two fields [year & notes] - just one).

      Any communication or ideas or even a "don't have a clue" would be nice. TIA.

      C

       
      • If you wish to have the date placed in the issue field, you should replace "Y1" in the RIS file with "IS."  The issue field should be able to accommodate the full date string.

         
        • Adding to Rick's suggestion: if you're able to export only newspaper articles from Reference Manager (with the full date string given in the RIS 'IS' field), it may be possible to then apply a batch search & replace query to extract & move the year info to the 'year' field (and the month info to the 'volume' field, respectively). I'd be happy to help you with this.

          Best, Matthias

           
    • Hi, apologies for not responding any earlier! I must admit that I somehow overlooked your first post.

      For newspaper (and magazine) articles, you can put the month info of the publication date into the "volume" field. Similarly, you can put the day info into the 'issue' field. That way, the refbase citation styles should output any newspaper dates (year+month+day) correctly. At least, this should work for the APA, Harvard 1/2/3 and AMA styles.

      So, if you can somehow put/move the month (day) info into the volume (issue) field of Reference Manager, then refbase should be able to import the data from the RefMan RIS file.

      It might be also possible to tweak the refbase RIS importer so that it splits year/month/day information into the respective fields. Let me know if you'd still favour this option and I'll try to look into this.

      Let us know how it goes.

      And sorry again for the late response!

      Matthias

       
    • Irish
      Irish
      2009-04-20

      Thanks for the responses. I will be trying some of your ideas today. I'll report back with any successes or failures. Thanks again.

      C

       
    • > It might be also possible to tweak the refbase RIS importer so that
      > it splits year/month/day information into the respective fields.

      I've taken a look at the refbase RIS importer.

      Say you have RIS records of type newspaper ("TY  - NEWS") or magazine article ("TY  - MGZN") with full date info (formatted as "YYYY/MM/DD") similar to these:

      TY  - NEWS
      AU  - Miller, M.
      AU  - Meyer, H.
      PY  - 2007/mar/23
      TI  - Hello Spencer
      JF  - New York Times
      SN  - 03624331
      SP  - 27
      N1  - this is a note
      ID  - newspaperarticle
      ER  -
      TY  - MGZN
      AU  - Thomson, A.
      AU  - Newton, M.
      PY  - 2008/01/12
      TI  - Modern Cell Biology doesn't support Darwinism
      JF  - The Scientific Magazine
      SP  - 37
      EP  - 47
      N1  - this is a note
      ID  - magazinearticle
      ER  -

      Then it should be possible to import these into refbase (with the year/month/day info imported into refbase's 'year/volume/issue' fields).

      To do so, please open file 'includes/import.inc.php' in a text editor, and search for this block of code (in refbase-0.9.5, this is lines 899-906):

      $preprocessorActionsArray = array(
                                          array(
                                                  'match'   => "/&#?\w+;/",
                                                  'actions' => array(
                                                                      "/(&#?\w+;)/e"  =>  "html_entity_decode('\\1', ENT_QUOTES, '$contentTypeCharset')"
                                                                  )
                                              )
                                      );

      and replace it with this code block:

      $preprocessorActionsArray = array(
                                          array(
                                                  'match'   => "/&#?\w+;/",
                                                  'actions' => array(
                                                                      "/(&#?\w+;)/e"  =>  "html_entity_decode('\\1', ENT_QUOTES, '$contentTypeCharset')"
                                                                  )
                                              ),
                                          array(
                                                  'match'   => "#^PY  - \d{4}/.+?/.+$#ms",
                                                  'actions' => array(
                                                                      "#^PY  - (\d{4})/(.+?)/(.+)#ms"  =>  "PY  - \\1//\nVL  - \\2\nIS  - \\3"
                                                                  )
                                              )
                                      );

      Save the file and try again to import your data. Given RIS data similar to the above, the full date information should now get correctly imported into the respective fields.

      Hope this helps,

      Matthias

       
    • Irish
      Irish
      2009-04-20

      Matthias - I am very close. I did what you described above (I replaced Y1 where you had PY as that is how these are exporting). I got the year in the YEAR field and the month and day showed up in the VOLUME Field as: 03 Is - 27/
      instead of the 27 being in the Issue field. The original Y1 field data is 1940/03/27/

      I think this will work but I need to export some records to make sure. Thanks again for all your help.

      C

       
    • Please open up again file 'includes/import.inc.php' and replace lines 899-906 with this code clock:

      $preprocessorActionsArray = array(
                                          array(
                                                  'match'   => "/&#?\w+;/",
                                                  'actions' => array(
                                                                      "/(&#?\w+;)/e"  =>  "html_entity_decode('\\1', ENT_QUOTES, '$contentTypeCharset')"
                                                                  )
                                              ),
                                  array(
                                          'match'   => "#^(?:PY|Y1)  - \d{4}/.+?/.+?(/.*)?#m",
                                          'actions' => array(
                                                              "#^(?:PY|Y1)  - (\d{4})/(.+?)/(.+?)(/.*)?$#m"  =>  "PY  - \\1//\nVL  - \\2\nIS  - \\3"
                                                          )
                                      )
                              );

      Does this work better now?

      If not, please post some sample RIS records so that we have some real data for testing.

      Matthias

       
    • Irish
      Irish
      2009-04-20

      TY  - NEWS
      ID  - 1
      T1  - AAUW gives silver tea for children's library
      N1  - 1922/04/07/
      N1  - Issue ID: 6
      KW  - American Association of University Women
      KW  - children's library
      KW  - women
      KW  - libraries
      KW  - 10
      RP  - NOT IN FILE
      SP  - 2
      JF  - Pullman Herald
      ER  -

      TY  - NEWS
      ID  - 2
      T1  - AAUW has Christmas party
      N1  - 1922/12/22/
      N1  - Issue ID: 2
      KW  - American Association of University Women
      KW  - Christmas
      KW  - women
      KW  - 20
      RP  - NOT IN FILE
      SP  - 11
      JF  - Pullman Herald
      ER  -

      TY  - NEWS
      ID  - 3
      T1  - Banks in strong condition
      Y1  - 1929/07/19/
      N1  - Issue ID: 7
      KW  - banks
      KW  - 30
      RP  - NOT IN FILE
      SP  - 1
      JF  - Pullman Herald
      ER  -

      TY  - NEWS
      ID  - 4
      T1  - AAUW has first banquet
      Y1  - 1921/12/09/
      N1  - Issue ID: 1-2
      KW  - American Association of University Women
      KW  - women
      KW  - 40
      RP  - NOT IN FILE
      SP  - 10
      JF  - Pullman Herald
      ER  -

      TY  - NEWS
      ID  - 5
      T1  - Norman Ageton writes of Porto Rico to Herald
      Y1  - 1911/07/28/
      N1  - Issue ID: 1
      KW  - Puerto Rico
      KW  - travel
      KW  - Ageton,Norman
      KW  - letters
      KW  - 50
      RP  - NOT IN FILE
      SP  - 2
      JF  - Pullman Herald
      ER  -

      TY  - NEWS
      ID  - 6
      T1  - Alfalfa campaign
      Y1  - 1913/10/03/
      N1  - Issue ID: 6
      KW  - Alfalfa
      KW  - 60
      RP  - NOT IN FILE
      SP  - 1
      JF  - Pullman Herald
      ER  -

      TY  - NEWS
      ID  - 7
      T1  - Holden speaks on alfalfa
      Y1  - 1913/10/24/
      N1  - Issue ID: 1
      KW  - Alfalfa
      KW  - hay
      KW  - Holden
      KW  - 70
      RP  - NOT IN FILE
      SP  - 1
      JF  - Pullman Herald
      ER  -

      TY  - NEWS
      ID  - 8
      T1  - E. D. Alvord writes from Japan
      Y1  - 1925/10/16/
      N1  - Issue ID: 1-3
      KW  - Japan
      KW  - travel
      KW  - Alvord,Emory D.
      KW  - 80
      RP  - NOT IN FILE
      SP  - 1
      JF  - Pullman Herald
      M2  - sec. II
      ER  -

      TY  - NEWS
      ID  - 9
      T1  - Emory D. Alvord speaks on missionary work
      Y1  - 1925/06/26/
      N1  - Issue ID: 1-3
      KW  - missionaries
      KW  - Alvord,Emory D.
      KW  - 90
      RP  - NOT IN FILE
      SP  - 7
      JF  - Pullman Herald
      M2  - sec. II
      ER  -

      Here are a few records for you. The outcome was close - the year was put into the Year field and day went into the Issue field but no Volumn data (month went nowhere). I hope this information is helpful to you - thank you so much.
      C

       
      • Thanks for the sample records.

        On my refbase installation (with the hack from my previous post applied), your sample records actually import fine. I.e. also the month got extracted to the 'volume' field.

        Are you sure that you pasted the code from my previous code correctly? In any case, I've uploaded the modified 'import.inc.php' file here:

        http://store.extracts.de/import.inc.php.zip

        so you may want to download and try that file instead.

        Thanks for your patience,

        Matthias

         
        • Irish
          Irish
          2009-04-20

          Got it! Thanks so much. Not sure what little something in my file was out of sorts. Thanks again and I'm sure I'll be back asking more questions. But refbase is an awesome product. While I'm not a huge programmer, I do write technical documentation and would be happy to help in any way I can.

          C

           
          • Great, I'm glad to hear you got it working!

            And many thanks for the kind words and your offer to help out!

            Documentation is always lacking behind, so any help with improving the docs or writing some new bits of documentation would be greatly appreciated (I'd be happy to setup a wiki account for you or anyone else interested in this). Even suggestions where (and how) you'd like to see the docs improved can be very helpful.

            Thanks again, Matthias

             
            • Irish
              Irish
              2009-04-20

              It would be awesome to have an account for the wiki - haven't found anything I'd change but have learned so much during this process - might be helpeful. Thanks.

              Cindy

               
              • Hi Cindy,

                I've created an account for you at our refbase online wiki (I've sent more info to your SF mail address). Thanks again for your interest.

                Best, Matthias