Corssref DIO urls changing to new https://doi.org/...

[Murali Raju]

Hi,

Crossref have changed the official URL for their DOIs to https://doi.org/...

They explain this and other changes in https://www.crossref.org/display-guidelines/

We've noticed that when we try to convert to endnote from a MOD file containing an updated DOI URL, xml2end 6.0 produces both %U lines with the old and the new Crossref URLs. For example:

Given a MOD file with an updated Crossref URL

$ grep --context=2 url 10.1007%2F978-3-319-57937-5_130.mod.xml
    <identifier type="doi">10.3989/ic.13.014</identifier>
    <location>
        <url>https://doi.org/10.3989/ic.13.014</url>
    </location>
</mods>

When we convert it to endnote using bibutils 6.0 we get two %U lines:

$ ./xml2end --version
xml2end, bibutils suite version 6.0 date 2017-07-04
$ ./xml2end <10.1007%2F978-3-319-57937-5_130.mod.xml 2>/dev/null | grep --context=2 '%U'
%F Rabasa Díaz2013
%R 10.3989/ic.13.014
%U https://doi.org/10.3989/ic.13.014
%U http://dx.doi.org/10.3989/ic.13.014

%0 Book

We've written a initial fix which uses the updated URL instead of the old one in the prefixes array in lib/url.c, but you might want to deal with it in a way which also works well with the old urls. The patch is the following:

$ cat ../bibutils-6.0-new-crossref-doi-url.patch 
diff --git a/lib/url.c b/lib/url.c
index e112d8c..8865711 100644
--- a/lib/url.c
+++ b/lib/url.c
@@ -175,7 +175,7 @@ static url_t prefixes[] = {
         /*              00000000001111111112222222222333333333344444444445 */
         /*              12345678901234567890123456789012345678901234567890 */
    { "ARXIV",     "http://arxiv.org/abs/",                     21 },
-   { "DOI",       "http://dx.doi.org/",                        18 },
+   { "DOI",       "https://doi.org/",                          18 },
    { "JSTOR",     "http://www.jstor.org/stable/",              28 },
    { "MRNUMBER",  "http://www.ams.org/mathscinet-getitem?mr=", 41 },
    { "PMID",      "http://www.ncbi.nlm.nih.gov/pubmed/",       35 },

Could bibutils be changed to cope with the new Crossref DOI URLs, please?

Best regards,
Murali Raju

[Chris Putnam]

Version 6.1 now handles the new and old URLs for DOIs. I had to make additional changes to make sure that everything was handled properly.

Thanks for the report.

[Murali Raju]

Hi Chris,

Thank you for the prompt response. We've tried the new version and we've noticed that the endnote conversion now works as we expected but the conversion to bib and ris have lost the url.

Please see our example below of conversions from a MOD file that includes a url, comparing the results of using both bibutils 6.0 and 6.1

$ grep --context=2 url  ../10.1007%2F978-3-319-57937-5_130.mod.xml 
    <identifier type="doi">10.3989/ic.13.014</identifier>
    <location>
        <url>https://doi.org/10.3989/ic.13.014</url>
    </location>
</mods>

With bibutils 6.0, conversion to bib and ris format includes a url:

$ ./xml2bib --version
xml2bib, bibutils suite version 6.0 date 2017-07-04
$ ./xml2bib < ../10.1007%2F978-3-319-57937-5_130.mod.xml | grep --before=6 --after=1 url
xml2bib: Cannot identify TYPE in reference 1  ref1 (defaulting to @Misc)
xml2bib: Cannot identify TYPE in reference 2  ref2 (defaulting to @Misc)
xml2bib: Cannot identify TYPE in reference 3  ref3 (defaulting to @Misc)
xml2bib: Cannot identify TYPE in reference 4  ref4 (defaulting to @Misc)
xml2bib: Cannot identify TYPE in reference 5  ref5 (defaulting to @Misc)
xml2bib: Cannot identify TYPE in reference 6  ref6 (defaulting to @Misc)
xml2bib: Cannot identify TYPE in reference 7  ref7 (defaulting to @Misc)
xml2bib: Cannot identify TYPE in reference 8  ref8 (defaulting to @Misc)
xml2bib: Processed 10 references.
author="Rabasa D{\'i}az, E.",
title="Estereotom{\'i}a: teor{\'i}a y pr{\'a}ctica, justificaci{\'o}n y alarde",
journal="Informes de la construcci{\'o}n",
year="2013",
volume="65",
doi="10.3989/ic.13.014",
url="https://doi.org/10.3989/ic.13.014"
}
$ ./xml2ris --version
xml2ris, bibutils suite version 6.0 date 2017-07-04
$ ./xml2ris < ../10.1007%2F978-3-319-57937-5_130.mod.xml | grep --before=6 --after=1 '10.3989/ic.13.014'
xml2ris: Processed 10 references.
AU  - Rabasa Díaz, E.
PY  - 2013
DA  - 2013//
TI  - Estereotomía: teoría y práctica, justificación y alarde
JO  - Informes de la construcción
VL  - 65
UR  - https://doi.org/10.3989/ic.13.014
DO  - 10.3989/ic.13.014
ID  - Rabasa Díaz2013

With bibutils 6.1, conversion to bib and ris format doesn't includes a url any more:

$ ./xml2bib --version
xml2bib, bibutils suite version 6.1 date 2017-07-31
$ ./xml2bib < ../10.1007%2F978-3-319-57937-5_130.mod.xml | grep --before=6 --after=1 '10.3989/ic.13.014'
xml2bib: Cannot identify TYPE in reference 1  ref1 (defaulting to @Misc)
xml2bib: Cannot identify TYPE in reference 2  ref2 (defaulting to @Misc)
xml2bib: Cannot identify TYPE in reference 3  ref3 (defaulting to @Misc)
xml2bib: Cannot identify TYPE in reference 4  ref4 (defaulting to @Misc)
xml2bib: Cannot identify TYPE in reference 5  ref5 (defaulting to @Misc)
xml2bib: Cannot identify TYPE in reference 6  ref6 (defaulting to @Misc)
xml2bib: Cannot identify TYPE in reference 7  ref7 (defaulting to @Misc)
xml2bib: Cannot identify TYPE in reference 8  ref8 (defaulting to @Misc)
xml2bib: Processed 10 references.
@Article{RabasaDíaz2013,
author="Rabasa D{\'i}az, E.",
title="Estereotom{\'i}a: teor{\'i}a y pr{\'a}ctica, justificaci{\'o}n y alarde",
journal="Informes de la construcci{\'o}n",
year="2013",
volume="65",
doi="10.3989/ic.13.014"
}
$ ./xml2ris --version
xml2ris, bibutils suite version 6.1 date 2017-07-31
$ ./xml2ris < ../10.1007%2F978-3-319-57937-5_130.mod.xml | grep --before=6 --after=1 '10.3989/ic.13.014'
xml2ris: Processed 10 references.
AU  - Rabasa Díaz, E.
PY  - 2013
DA  - 2013//
TI  - Estereotomía: teoría y práctica, justificación y alarde
JO  - Informes de la construcción
VL  - 65
DO  - 10.3989/ic.13.014
ID  - Rabasa Díaz2013

Would it possible to have a new version which includes the url in the ris/bib formats?

Best regards
Murali Raju

[Chris Putnam]

Actually, not outputing a URL for the DOI for bibtex, ris, and some other formats was the expected behavior, as these formats have other ways to output DOI information. (I had a rationale for this behavior at least.) The reason that the older versions output the url was that bibutils wasn't recognizing the new format of DOI urls.

But since you want the redundant information, I've added the url output of DOIs to these formats in the current version that I've just uploaded as bibutils version 6.2.

Bibutils already handles duplicated information like this internally, so I have no problems outputting it if turns out to be useful.

Chris.