Menu
▾
▴
#1936 dos2unix does not exhibit appropriate filter behaviour
My expectation of dos2unix has always been that it should act, first and foremost, as a filter; the command
dos2unix crlffile >> lfonlyfile
should concatenate the converted content of crlffile to lfonlyfile, leaving crlffile unchanged. This is the behaviour I am accustomed to, on commercial unix platforms with which I have worked in the past. The MSYS implementation does not exhibit this behaviour; rather it converts crlffile in place, to LF only format, (and so corrupting it), while leaving lfonlyfile unchanged.
Granted, there is no definitive standard for dos2unix, but while the MSYS variant continues to exhibit such anomalous behaviour, I shall be removing it from my system, and instead installing the correctly functioning filters, as I contributed them to MSYS-1.0-11.
Who is the maintainer? Do we document the maintainer of packages somewhere? We probably should put it on the ticket list display.
In this case, Erwin Waterlander, but I don't know of any place where that is formally documented. I guess we should create some such documentation somewhere, maybe a wiki page, or maybe better a story page, on MinGW.org? (I know Aaron did start to compile such a list, on the old wiki, but I'm sure it's hopelessly out of date now, if indeed it ever was migrated to the new site).
The ticket list is another possibility, but I'm not convinced it is the best choice.
Or, maybe add a MAINTAINERS file, as a git controlled component of mingw-dist?
I created [#1947] for this. It is unassigned and I've added a novice tag to it.
Related
Issues: #1947
Hi,
Which commercial implementation are you talking about?
This version is based on the SunOS implementation, with one major difference. The SunOS version works by default in paired mode, while this version works by default in in-place mode. Most people prefer in-place mode so they can do
dos2unix *.txt
Before I started maintaining this dos2unix it was already many years part of RedHat, Suse, and Gentoo Linux. Personally I used it many years on HP-UX, because I didn't like the HP-UX implementation.
In the meantime it has also been included in many other Linux distributions and Cygwin. Practically this is now the de facto standard implementation on Linux. So I'm not going to change the standard behaviour. With this implementation msys is compatible with Cygwin and most Linux distributions.
This version of dos2unix can work as a filter.
To do what you want you have to type:
dos2unix < crlffile >> lfonlyfile
regards,
Erwin
Keith's OP states that the app doesn't allow non-in-place action. He stated regardless of the redirect or pipe it still does the in-place action. I find that the following is documented to work and trying it WJFFM.
cat CRLFfile | dos2unix > LFfile
The first implementation I encountered was that of VentureCom's VENIX, (effectively SysV-R2), running on Intel 286 processors, circa 1988; that implementation behaved as any classical Unix filter, such as grep, sed, awk, or even nroff/troff:
processes each specified file in turn, concatenating the processed output to stdout; so...
would concatenate foo.txt, bar.txt, and baz.txt, writing the result with CRLF line endings, to stdout.
Subsequently, I migrated to a SunOS/Solaris system, where the dos2unix implementation differs in a completely incompatible manner; it requires
This no longer acts like a classical Unix filter; it is also dangerously incompatible with your implementation; anyone running
on Solaris, and expecting the behaviour of your implementation may be unpleasantly surprised to discover that their bar.txt file has been trashed.
FWIW, I consider both the Solaris implementation, and yours, to be completely FUBAR. Even if yours does permit use as a filter, it is in an unnatural manner which is incompatible with the usage of classical Unix filters.
Last edit: Erwin Waterlander 2013-03-11
In the example of Keith the input doesn't come from stdin. So I'm wondering which original implementation Keith is familiar to.
When dos2unix doesn't have file names as argument it will process stdin, and send the converted output to stdout.
Hi,
I want to change the resolution of this bug, but I see no option to. Earnie, do I not have sufficient permissions?
Keith, I don't agree that the behaviour is anomalous (as explained), so I'm going to close this bug (if I can), unless you want to discuss this further.
Try it now Erwin. The update permission was set to admin, it is now set to developer.
Hi Earnie,
Thanks. I can now change the bug.
Frankly, I didn't expect you to be interested in fixing it; it is so utterly FUBAR, that you probably couldn't anyway.
My primary intent was to document the nastiness of this particular package. If you wish to close it, please do so, but be sure to classify it as documentation. I have my own implementation, which works correctly, and which I will continue to use in place of yours.
I agree that this is a documentation issue.
Hi,
I don't see why this 'documentation' makes any sense. Dos2unix behaves as documented. You could document a lot more that dos2unix doesn't do. This is more a documentation of Keith's rant.
You should file a bug when dos2unix behaves different than documented.
You should file a feature request if you want new or different behaviour.
best regards,
Erwin
It makes sense, because it documents a departure, (IMO an unfortunate departure), from classical filter behaviour; (classical behaviour which was available in at least one commercially available dos2unix/unix2dos implementation of which I have personal experience).
It is a bug that this behaviour was changed, in an unnatural manner, regardless of whether or not the altered behaviour was subsequently documented as accepted; it remains valuable to document that this new behaviour differs from classical expectation.
Call it a rant if you wish, but frankly, I have no interest in any feature request to change the behaviour of your FUBAR package. I have an alternative implementation which works correctly; I have no need of yours, which I will never again install on any platform.
Do you want to document all differences of msys with commercial Unix of 30 years ago with bug reports. Which commercial Unix anyway? I call this senseless. If I'm looking for documentation about old commercial Unixes, I look on the wikipedia or so. Not in msys bug reports.
I could also start reporting that make an cc are not compatible with HP-UX as I remember it from 20 years ago.
And, as you said yourself, there is no standard that describes how dos2unix should behave.
If it was fundamentally unworkable then I should get lots of bug reports. But I don't.
The dos2unix that I maintain stands on itself. There is no claim that is compatible with the old version you remember. There never was a departure from it. Behaviour was not changed. It isn't broken. It works correctly. It only states that it is derived from SunOS' dos2unix with a difference that it uses by default in-place mode instead of paired. It is just one of the many alternative implementations, which happens to be popular in the Linux world. I underscore that it is alternative, it never tried to be what you expected from it.
Msys and Cygwin have a strong link with Linux, so if you want to stay compatible with the rest you could use the version I offer. But you can also choose different.
Hi,
You may think that the implementation I maintain is utterly fubar, unnatural, anomalous.
Fact is that the classical unix filter style implementations of dos2unix are passé, in oblivion. Most people prefer in-place replacement. All open source implementations, except yours, work in-place. By now your implementation is an anomaly. If old style was really handier (like with grep and sed) then it would not have developed in the in-place direction. But you are free to use your own version.
You may say that, but it is only so because ID-TEN-T lusers such as you have made it so. If in-place conversion is desirable, then it should have been as an option, with an option flag to activate it. As it is, your implementation is fundamentally unusable, in any work flow which requires sane filtering capability.
Thanks, but no thanks; I have no interest in your broken implementation, (either in MSYS, or on my Linux boxes). I will continue to deploy my own sane implementation.
Last edit: Erwin Waterlander 2013-03-11
The behaviour of this package was already so long before I started maintaining it. This implementation has enough legacy on its own to not change the default behaviour any more.
Classical filter behaviour could be made optional, but I have to see more interest in it.