I found that bib data exported from some platform/tools is missing some fields, like pages is missing for an article (but it is available if one downloads the pdf file, and checks the page numbers). And bibutils do not check it at all. It does not complain but generates an item. The missing fields are likely to be ignored until the resulting file is checked by some very careful guy. I hope this should be checked eariler. I don't know if such a tool exists or not. But it would be great if bibutils...
The latest version (7.2) has this fix in it. Let me know if you see any more issues. Thanks again. Chris.
Thanks Chris for the quick reply, yes author names are not correct in the file. Actually I exported this file from mendeley and I can not recall where have I imported these references in my mendeley library. I will correct author names in my mendeley library. Also thank you for fixing title issue so fast, can you give any ETA when this fix will be available in latest release? Thanks, Fahad
history_version7
history_version7
history_version7
Thanks for the report. I found the bug that was preventing titles from being output properly. I will upload a fixed version shortly. I've also solved one problem parsing the names in this file. However, I wonder if the names in the Endnote file aren't entered correctly. The format of the names really confuses my name parsing algorithms and I suspect that Endnote will generate odd names from them as well. For example, in the second reference you have: Alexander A. Samuel and Bruin, De Arie and Kornet...
Hello sir, First up, thank you for the awesome bibutils, they are a gem for the community. Secondly I wanted to share a problem related to an Endnote XML file conversion. When I ran endx2xml on the file, it failed to parse the title of the publication. I have attached the xml files for reference. Please help me in this matter. Thanks, Fahad
I've modified the page parsing code so that it will handle this situation (the end tags will contain "32, 64" in this case). This will come out in the next release.
Thanks for catching these. They'll be corrected in the next release.
Hi, Thanks for maintaining bibutils. I am the Debian packager of the software; I want to suggest two corrections in the files: in bibutils.dbk: relased --> released; same file: GNU Public License --> GNU General Public License; in lib/charsets.c: Portugese --> Portuguese. Thanks and best wishes, pgt
Thanks, I fetched 7.1 version, it seems to be working fine apart of minor issue: if there is space between comma and 64 in '24-32, 64' case, then 64 is skipped: <part> <extent unit="page"> <start>24</start> <end>32,</end> </extent> </part>
Thanks, I fetched 7.1 version, it seems to be working fine apart of minor issue: if there is space between comma and 64 in '24-32,64' case, then 64 is skipped: <part> <extent unit="page"> <start>24</start> <end>32,</end> </extent> </part>
Thanks. It built good now.
history_version7
Sorry about that. Please check the current 7.1 release. I accidentally made all of my build changes to the generated Makefile rather than the Makefile.static/Makefile.dynamic files that the Makefile is generated from. Thanks for the report.
Hi. I can't build bibutils 7.0 for Fedora. Bibutils Configuration Operating system: Linux_x86_64 Library and binary type: dynamic Binary installation directory: /builddir/build/BUILDROOT/bibutils-7.0-1.fc36.x86_64/usr/bin Library installation directory: /builddir/build/BUILDROOT/bibutils-7.0-1.fc36.x86_64/usr/lib64 If auto-identification of operating system failed, e-mail cdputnam@ucsd.edu with the output of the command: uname -a Use --static or --dynamic to specify library and binary type; the --static...
history_version7
history_version7
Bibutils
The new version 7.0 should do this. Since the date field wants to be in ISO 8601, I implemented a check to ensure that the year/month/day information can be expressed properly in ISO 8601 (some of my test suite references have somewhat garbled dates like "c1989", etc.). If the year/month doesn't look right for ISO 8601, I'm defaulting to using the year/month tags.
I just uploaded version 7.0 (I made a change to a structure that would break binary compatibility for people using bibutils as a library).
Let me clarify that with team, please.
Thank you for your reply! Do you have some estimates on when new release is planned? So that we would decide whether to wait for it or implement some temporary solution on our side.
My current version of nbib2xml parses both of these. This fix comes from merging all of the page management code from all of the importers, which I was doing more for code cleanliness reasons more than anything else. The "24-32,64-" is still something of an issue, though, as many bibliography formats either want a single page or a consecutive range and just can't express something like the above. I checked the last publically available version and found that nbib2xml did indeed crash on these inputs...
I have no deep understanding of nbib formats, other than looking at examples and websites like you cited. But I think that you are running into the general problem that bibutils has always faced. That is that there are valid reference types (and data) that can be expressed in one format that simply don't exist in another format. Many people have asked that such "unexpressible" content be retained, but my response is always "show me what you would like it to look like". Some people have good ideas,...
We are facing issues with some nbib files having PG values like below: PG - 24-32,64 - ',64' part which represents additional page apart of range seems to be the reason of failure (just '24-32' is processed properly); PG - XC-CIII - when having date range in Roman numbers, conversion fails as well, while single Roman number is parsed without issues to part/detail/number element. Is that expected behavior? We would expect nbib2xml to ignore such cases which cannot be parsed rather than just fail.
Thanks for your reply. Are there any other type apart of Journal article/News/Book that could be present in nbib files and should be considered? It is mentioned here that: Almost all citations have one of these four basic, most frequently used Publication Types applied to them: Journal Article, Letter, Editorial, News. One of the above four Publication Types is applied to more than 99% of all citations indexed for MEDLINE. On this page the full list of publication types is listed (and there is no...
It's expected behavior with the current version of the program. I'd never seen an example of a nbib book before. But in looking at some documentation, the "BTI" tag is "Book Title", so clearly Book is ok. I've just added it to my local source and hope to upload it sometime soon (after I get done with some teaching).
It looks like only "Journal article" and "News" PT field values are recognized as a Genre (both are treated as "journal article"). Other values are not recognized and get defaulted to "journal article". Could you please clarify if this a bug or expected behavior (e.g. because nbib files could contain only journal article references)? Please see test reference of Book type attached
The format used by xml2biblatex for providing dates in biblatex is incorrect. biblatex does support "year" and "month" (mainly for legacy reasons) but not "day". The correct form is to use a "date" (or "origdate", "eventdate", "urldate") field, and format its content according to ISO 8601, a.k.a. EDTF. Using a "date" field is preferred even if a date consists of just a year, or year+month. Dates, date ranges and date/time combos are supported, e.g. 1850 1997/ /1997 1997/.. ../1997 1967-02 2009-01-31...
Hi, I recently ported bibutils to R. This is to thank Chris and the contributors for their work and to keep them informed. The port is in the form of an R package, rbibutils. It is on CRAN at https://cran.r-project.org/package=rbibutils, has a github page at https://github.com/GeoBosh/rbibutils, and a gh-pages website at https://geobosh.github.io/rbibutils/ . rbibutils is under GPL-2. Credits to the authors/contributors are in file DESCRIPTION and from there propagate to the documentation, the website...
Hi Chris, Thanks for your feedback. I'll adress your comments below inline :) So I've never seen a MODS file with a nonSort tag in the title. (I'd love to see an example.) https://lccn.loc.gov/a40003181/mods Based on the way you've written this patch, you are maintaining nonSort titles as a separate entity and it looks like it's just a special form of a title prefix. Essentially my understanding as well. They are used to as a control of sorts, and are designed to be ignored in the sort process. You...
This is a fine way to get patches to me. And your modifications look really good in general. So I've never seen a MODS file with a nonSort tag in the title. (I'd love to see an example.) Based on the way you've written this patch, you are maintaining nonSort titles as a separate entity and it looks like it's just a special form of a title prefix. If this is correct, I think the one change I'll make to your patch is to call the nonSort internally a "title prefix", which is the more general concept...
Proposed patch: http://ix.io/2qvJ diff -Naur ./lib/adsout.c /home/matt/bibutils_6.10/lib/adsout.c --- ./lib/adsout.c 2020-06-29 11:40:25.777497402 -0400 +++ /home/matt/bibutils_6.10/lib/adsout.c 2020-03-23 14:53:54.000000000 -0400 @@ -152,22 +152,21 @@ } static int -append_title( fields *in, char *nsrt, char *ttl, char *sub, char *adstag, int level, fields *out, int *status ) +append_title( fields *in, char *ttl, char *sub, char *adstag, int level, fields *out, int *status ) { - str fulltitle, *nonsort,...
Hi All, When transforming valid mods XML to RIS (or other formats) the nonSort portion of the title is dropped. I would expect it to appear preceding the main title. I believe I have implemented the proper changes to make this happen. But should note that I haven't worked with C since I last contributed to this project in 2014 [1]. Can you inform me of the process of submitting patches? Thanks, Matt [1] - https://sourceforge.net/p/bibutils/discussion/general/thread/cbd9e838/
Thank you for your work. I just compiled the new version and tested it. It meets my requirement for now. But I find some entry type like online (http://ctan.cs.uu.nl/macros/latex/contrib/biblatex/doc/examples/biblatex-examples.bib) is converted into Electronic, and software (https://github.com/JabRef/jabref/issues/5223) is recognized as Article.
Thanks.
Fixed in version 6.10. Thanks for the report.
history_version6
I made sed -e 's|latex.o |latex.o latex_parse.o |' -i lib/Makefile.dynamic and now built good. Please correct it.
I can't build 6.9 for Fedora. Error while linking: /usr/bin/ld: ../lib/libbibutils.so: undefined reference to `latex_parse' /usr/bin/ld: ../lib/libbibutils.so: undefined reference to `latex_tokenize' What should I install to solve this?
Bibutils
history_version6
Version 6.9 (just uploaded) fixes this in my testing. Turns out that the next latex parser wasn't needed for this particular bug, but the new parser does a better job, so I'm keeping it. Sorry for the delay; work has been a little complicated over the last month or so.
Version 6.9 has an initial version of a xml2biblatex converter. Let me know if you find any problems with it.
@cdputnam Is there any progress?
I required this in https://sourceforge.net/p/bibutils/discussion/general/thread/4352613b/ But I think it is better to be a topic to see how many people would have such requirement. For me, I use biblatex more than bibtex. It supports more entry types, such as patent. But with xml2bib, a patent entry will be converted to a misc entry.
I required this in https://sourceforge.net/p/bibutils/discussion/general/thread/4352613b/ But I think it is better to be a topic to see how many people would have such an requirement. For me, I use biblatex more than bibtex. It supports more entry types, such as patent. But with xml2bib, a patent entry will be converted to a misc entry.
Nice to hear that, and looking forward to your new parser.
Yes, it's a bug. The only way I can think to solve it is to build a real parser for LaTeX. Until now I've been cheating by recognizing common LaTeX patterns, but that won't scale to all possible CJK names. The parser won't be that hard to write (I have a plan for it in my head) and in the end ought to simplify the way I've been handling LaTeX. I just need to find a little time to implement it.
I expect the result better be Aksın, Özge and 张三
I use xml2bib -nl. Aks{\i}n, {\"O}zge and 张三 turns into Aksın, Özge and {张三}. The brackets go away for the former name, but come for the latter CJK name. Is this a bug?
I use xml2bib -nl. Aks{\i}n, {\"O}zge and 张三 turns into Aksın, Özge and {张三}. The brackets go away for the former name, but come for the latter. Is this a bug?
Good to know it.
While this could be in Bibutils, it is already available via pandoc-citeproc which uses Bibutils internally via hs-bibutils.
It is very useful to store bib as json, or maybe csl-json.
Yes, it is a request.
The simple answer is that no one has ever requested an xml2biblatex converter. In general, I don't want to write converters that aren't useful to people. So I tend to wait until someone seems interested. For example, I would have never guessed that an xml2isi converter would have been useful, but apparently it was. So is this a request for an xml2biblatex converter? It will probably take a couple of rounds to get one that's mostly "correct" as Biblatex is somewhat more complex than nbib or isi, for...
I maintain unofficial repository github.com/wilx/bibutils-archive where I import release tarballs of Bibutils. I have used it to make it easier to produce diffs and to have some respository where I can create local branches.
I also tried to clone the repo. Hope there will be a public repo, so that more people could help and fix issues faster.
@cdputnam This topic reminds me my long-time-ago wondering why there is biblatex2xml but no xml2biblatex. Biblatex is now widely used. It supports more entry types, such as patent. But with xml2bib, a patent entry will be converted to a misc entry.
There is no publically available git repository. But I'm very happy to take patches. And I'm sorry I haven't taken time to fix the xml2isi issue you've raised yet. My non-hobby programming time has gotten really complicated recently. If you don't have a fix yet, I'd suggest looking at lib/isiout.c and see if you can add an "output_simple()" call that catches the missing elements. Also running the programs with "--debug" gives lots of useful information for each step in the transformation to see what...
I am trying to see if I can implement fixes for the issues I raised after the latest update. (See here) However, when I try to clone the repo, I get the following. git clone git://git.code.sf.net/p/bibutils/code bibutils-code Cloning into 'bibutils-code'... warning: You appear to have cloned an empty repository. Perhaps I'm missing something obvious?
Also, ris entry C2 which appears to be the pubmed ID is being lost when going from ris to isi.
Thanks for implementing those changes. As it stands, I now have a problem with the journal titles. As far as I can tell, xml2isi is incorrectly 1. using SE instead of JI for the ISO source title abbreviation 2. using JI instead of SO for the journal source title. See attached for output from command $ ris2xml scopus.ris | xml2isi > savedrecs.txt
Version 6.8 should properly handle the .isi output issues you found. Let me know if you have any other problems/suggestions. Thanks again.
Sorry, I see the attachments now. I'll take a look. Thanks for being on top of this and sorry for the noise.
Sorry, I see the attachments now. I'll take a look. Thanks for being on top of this and sorry for the noise.
Can you post an .ris example that fails? I see where publisher info and isbn are falling through the cracks, but I would have thought that the url elements would have been preserved. Before I upload a "fixed" version, I'd like to make sure that it's really fixed.
I'm converting some files from .ris to .xml and then to .isi format. However, when I do so, I lose the book publisher information, isbn, and url elements. Is there an option I am missing? (ps many thanks for a great set of tools)
I've already included a modified version of your patch in version 6.7. Let me know if you find that 6.7 doesn't handle things correctly.
It turns out that adsout.c in bibutils 6.6 does not deal well with cases where the initial of the first author is an accented character of the extended code pages of UTF-8. One would like to convert that character to a plain uppercase ASCII letter before adding this in ads2xml. A patch of that kind for adsout.c is attached.
These changes are now added to version 6.7. Note that I didn't comment out append_urls(), but rather removed DOI from that list of tags output by that function so that DOI's won't be repeated in the output.
This is now updated in version 6.7
history_version6
history_version6
Also: here are refinements that would be appropriate for xml2ads in the function append_data() in lib/adsout.c: (i) move the append_Rtag() call ahead in front of the first append_people() call. (ii) In the DOI output, do not just say "%Y" but "%Y DOI:" for the initial string in the output line. (iii) make the call of append_urls() a comment.
It would be nice if ris2xml could scan lines of the format DO - https://doi.org/xx.xxxx/yyyyy.. that appear as doi specifications of ris files of ELSEVIER. Without the 'https://doi.org/' ris2xml renders the input well and puts the xx.xxxx/yyyyy into the XML files.
It actually is fine, though apparently the cast should be ( long long ) under Windows, rather than ( long ), and the ( long long ) cast also works under Linux; long long is 64 bit under both architectures. This should silence the warning. fields_findv_each() will put pointers to str, pointers to char (C-style strings), or indicies of hits into the output void pointer list (vplist), depending on the mode that the user has chosen. In the case of storing indicies in the vplist, I'm abusing the cast...
I am getting the follwing warning in hs-bibutils compilation on 64 bit Windows: bibutils\fields.c: In function 'fields_findv_each_add': C:\hs-bibutils\bibutils\fields.c:459:7: error: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast] v = ( void * )( (long) n ); This looks fishy. The n variable is int and that is only 32 bits wide on Windows. If any pointer gets transported through the integer it will be truncated in v variable.
I just released version 6.6 that has a first stab at a xml2nbib converter. It seems to work pretty well for my testcases, but let me know if it should be behaving differently.
history_version6
Bibutils
Sure. I can tackle it. I have a few upcoming deadlines, but I'll write it for the next release.
Is this possible to do/add? (it seems like ncbi tool SciENcv requires .nbib for importing references) Thanks.
I compiled bibutils for Windows 10 home 64 bit using msys2. This is how: Download bibutils sources and unpack them preserving the directory structure. Download msys2 from https://sourceforge.net/projects/msys2/ and run the installer. (See https://stackoverflow.com/questions/30069830/how-to-install-mingw-w64-and-msys2) The installer creates three bash shells, called MSYS2 MINGW 32-bit MSYS2 MINGW 64-bit MSYS2 MINGW (in the Start menu, under the entry MSYS2 64-bit.) To compile bibutils, run MSYS2 MINGW...
history_version6
history_version6
I found the problem. Turns out that the defines SKIP and SIMPLE accidentally have the same value in lib/reftypes.h. For the mesh heading keywords ( tag "MH" ), the tag needs to be defined in nbibtypes.c The fix will be in the version 6.5 release.