DOI import failures

Help
2009-05-15
2013-05-28
  • Hi,

    I use doi import quite regularly. It often works great but I find it not as reliable as the pubmed id import for example. Some doi just make refbase "crash".

    For example trying to import 10.1109/JPROC.2008.916343 ends up on a blank page at both http://beta.refbase.net/ and http://demo.refbase.net/. On my server (not fully up-to-date), I get the following error:
    Fatal error: Call to a member function on a non-object in /home/www/refbase/includes/import.inc.php on line 378

    Is there a workaround for this? Even if the doi request cannot be parsed, I think that a good thing would be do display a more helpful message to the user.

    Warm regards,
    Tom Vercauteren

     
    • You must request an account from: http://www.crossref.org/requestaccount/ and place the information in include/import.inc.php in the $crossRefReqDat parameter.  Neither sample website has this, apparently.

      Sorry for the inconvenience.  This parameter should be moved into initialize/ini.inc.php in the future & we should improve error handling & documentation of this feature.

      CrossRef.org had granted refbase credentials to use sometime ago, but those credentials were removed when they began to use the individual account model (and this coincided with our release).

      --Rick

       
  • It worked during some time, but now I cannot import data using DOI again.

    > Fatal error: Call to a member function
    > getTagContent() on a non-object in
    > /var/www/refbase/includes/import.inc.php
    > on line 400

     
  • Matt - Zee
    Matt - Zee
    2009-09-17

    My bad, didn't realize you have to look at URL source to see the actual value of the variable.

    Anyway, I found the source of error, in
    $recordArray = splitSourceText($sourceText,…)
    $recordArray contains XML header, and the <doi_records> tag, which is not what's expected later:

    One can

        array_shift($recordArray);
    it out before returning $recordArray.

      : http://pastebin.com/m2bea12cb

     
  • Martin Fluegge
    Martin Fluegge
    2009-09-17

    Hi,

    we also experience some problems when importing DIOs, but I am not sure whether our problem is related to the one mentioned above.

    When importing DIO we get the following error.


    http://www.crossref.org/openurl/?noredirect=true&format=unixref&pid=abc.xyz@123.de&id=doi%3A10.1007%2F978-3-540-68636-1

    Warning: fopen(http://…@123&id=doi%3A10.1007%2F978-3-540-68636-1) : failed to open stream: Connection refused in /var/www/specimenScout/refbase/includes/include.inc.php on line 5213

    Warning: Cannot modify header information - headers already sent by (output started at /var/www/specimenScout/refbase/includes/import.inc.php:2857) in /var/www/specimenScout/refbase/import_modify.php on line 557


    The $crossRefReqDat is set to a valid account. When using to created URL directly in a browser window it works and we can see the according XML. Pinging the crossref-server works also.

    The strange think is that using a PMID works fine. As far as I understood they both are using the same function ($fetchDataFromURL) to retrieve the data.

    We also sniffed the communication. When using a PMID the refBase is sending a request to the correct server, but with the oid it does not seem to establish a connection to corssref.org.

    Anyone experienced the same problem?

    Thanks in advance for any hint.

    Cheers,

    Martin

     
  • Knut Krüger
    Knut Krüger
    2009-09-17

    Hi martin
    seems that
    allow_url_fopen  is set to no because of security issues
    what does phpinfo shows you for allow_url_fopen?
    save this code to an ???.php file and open it with your browser
    <?PHP phpinfo(); ?>

    Thenext one is because there is an error message already sent out:
    Warning: Cannot modify header information - headers already sent

    Regards Knut

     
  • Knut Krüger
    Knut Krüger
    2009-09-17

    Hi Matthias
    Problem …
    allow_url_fopen  is written in the editor as  allow\_url\_fopen
    but to display the \_ we must use now an backslash before

    and <br> is displayd in the preview as linebreak but not in the final text
    Regards Knut

     
  • Martin, sorry I didn't realize that PMID import is working for you correctly.

    I'm not sure what could cause your issue. Have you tried to URL encode the '$crossRefReqDat' data, like this:

        $sourceURL .= "&pid=" . rawurlencode($crossRefReqDat);

    Also, it's probably a good idea to also apply the fix for the DOI import failure mentioned in my previous post.

    Matthias

     
  • Thank you, Matthias, it works for me.

     
  • The openurl doesn't seem to be working. I am using current svn refbase trunk. Pubmed import works fine. However, crossref using DOI fails with "PHP Fatal error:  Call to a member function getTagContent() on a non-object in /var/www/refs/includes/import.inc.php on line 378" while reading upstream". I've read the relevant threads on the mailing list and registered for a crossref account. If I add $crossRefReqDat = "user@site.edu"; to initialize/ini.inc.php crossref import still fails. I checked fopen and it was allowed. Debugging output of print($sourceText); in line 371 of import.inc.php showed a crossref error that said that my login was null. I've read the current http://www.crossref.org/help/CrossRef_Help_Left.htm#CSHID=05_Interfacing_with_the_CrossRef_system%2FUsing_the_Open_URL_Resolver.htm|StartTopic=Content%2F05_Interfacing_with_the_CrossRef_system%2FUsing_the_Open_URL_Resolver.htm|SkinName=CrossRef_Help crossref page and tried modifying the

    // Build query URL:
                                    $sourceURL = "http://www.crossref.org/openurl/"
                                                       . "?pid=user@site.edu"
                                                       . "&noredirect=true"
                                                       . "&format=" . $fetchType;

    but it times out. I hope someone who knows what they are doing could help resolve this.

    Thank you

    Alex

     
  • Some more reading showed that they now use "redirect=false" instead of "noredirect=true", so          

      $sourceURL = "http://www.crossref.org/openurl/"
                                                       . "?pid=registered@user.domain"
                                                       . "&redirect=false"
                                                       . "&format=" . $fetchType;

    now works at leat for the doi ids I tested. Please patch the refbase.

     
  • I also wonder why the developers wouldn't register user@refbase.net on crossref and use that as default in the initialize/ini.inc.php

     
  • CrossRef actually made a refbase-specific account before they would allow individuals to apply for accounts, but they retired it without explanation.  It might be worthwhile to contact them regarding this issue again, but it is easy enough to create your individual account & it is less likely that it will expire without notice & CrossRef could contact you directly regarding any issues.

    -Rick

     
  • Hi Alex,

    adding to Rick's answer:

    I've actually registered info@refbase.net and we might use this by default in the future.

    W.r.t. your error ("PHP Fatal error: Call to a member function getTagContent() on a non-object …"), please see my posts in this thread:

    https://sourceforge.net/projects/refbase/forums/forum/218757/topic/3401506

    and see whether the provided fix works for you as well.

    Matthias

     
  • Since the code in the previously linked thread was garbled by the SourceForge forum software, here it is again:

    Open file 'includes/import.inc.php' and replace this line (in function 'crossrefToRefbase()'):

    $recordDelimiter = "(\s*<doi_records[^<>\r\n]*>)?\s*(?=<doi_record[^<>\r\n]*>)" // splits before '<doi_record>'
    

    with this one:

    $recordDelimiter = "(\s*(<\?xml[^<>]*\?>\s*)?<doi_records[^<>]*>)?\s*(?=<doi_record[^<>]*>)" // splits before '<doi_record>'
    

    HTH, Matthias

     
  • No, the $recordDelimiter wasn't an issue because refbase wasn't getting any data at all due to its inability to pass my credentials to the crossref server just as I wrote above. It seems that crossref changed the request arguments, so now we have to use pid first, then redirect=false instead of noredirect=true.

     
  • Hi Alex,

    actually, this URL (which still uses 'noredirect=true') seems to work just fine for me:

    http://www.crossref.org/openurl/?noredirect=true&format=unixref&pid=info%40refbase.net&id=10.1515%2FBOT.2001.002

    Can you verify that it does also work for you? I.e. you should get back a CrossRef XML record (with 'doi_records' as the root element).

    If that works for you, try changing 'pid=info%40refbase.net' in the above URL to your own CrossRef PID ('pid=user%40site.edu'). Does it still work then?

    If not, verify your CrossRef account, or make a new one. If it does work, enter that PID in function 'fetchDataFromCrossRef()' (in file 'includes/import.inc.php' on line 2826):

    $crossRefReqDat = rawurlencode("user@site.edu");
    

    Future versions will allow you to set that value in 'initialize/ini.inc.php' but these modifications aren't yet in the refbase SVN repository.

    Also, when you've got your CrossRef account working, I'm pretty sure that you will need to apply the patch I've given above.

    If it still doesn't work, give us more info, e.g. what DOI(s) you've been testing it with.

    Matthias

     
  • It appears that a combination of $crossRefReqDat = rawurlencode("user@site.edu"); in the import.inc.php instead of initialize/ini.inc.php and the $recordDelimiter patch resolve the issue. Thank you.

     
  • QL Studio
    QL Studio
    2012-11-29

    yep - I can confirm that this combination fixes up the DOI import - december 2012 - so I guess this app is not in active development any more?

     
  • That fix is in the SVN branch.  refbase development has certainly slowed down, but it has not stopped.