jpeg-9 has been released and the SONAME changed from .8 to .9, so we need ./configure switch for the jpeg-9 features including the SONAME for ABI compability
I'll say it again. First, show me software that is using jpeg-9. There is no use in emulating an API/ABI if no one is using it. If we eventually do decide to implement the jpeg-8 features, it will be some time before that happens.
Anyone who has a definable need to upgrade to jpeg-9, please let me know. I do not understand why anyone would need to upgrade from jpeg-8 to jpeg-9 unless they are either actively taking advantage of the Lossless SmartScale format or unless they are just blindly downloading the latest & greatest version without regard for whether they actually need it.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Within Gentoo Linux we provide package virtual/jpeg, which allows easy swapping between libjpeg-turbo (.so.8) and jpeg (.so.8)
So adding jpeg (.so.9) to the virtual would require the user to rebuild every package linking to the jpeg (.so.8) and easy swapping would no longer be possible
So if user wants jpeg-9 he can have it for whatever personal project he has, and denying that optionality would be just... silly. Alternative would be dropping libjpeg-turbo support, which we don't want either.
Last edit: Samuli Suominen 2013-01-14
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
OK, but does jpeg-9 co-exist with jpeg-8 on the system in that case? Which version is used as the "system" libjpeg? Or is that controlled by switching the virtual? Presumably Gentoo has a bunch of system software that is linked with a particular version of libjpeg, so you can't just swap in a new ABI at the system level without breaking a lot of things. If you're considering upgrading the system version of libjpeg to jpeg-9, then honestly, I think you should wait and see how the community reacts to the new IJG release.
Personally, my feeling is that you're creating a problem where one doesn't exist yet. If users were asking for jpeg-9, I would understand the desire to implement it, as well as an API/ABI compatibility mechanism in libjpeg-turbo. But I mean, it just literally released a few hours ago. I have yet to run across any software that even takes advantage of the jpeg-8 features, other than possibly the memory-based source/destination managers. If you know of anything that is actually using SmartScale, I'd love to hear about it.
What seems silly to me is continuing to view IJG as an authoritative source, when it's really just one guy doing whatever the heck he wants, breaking backward compatibility and introducing new formats whenever he feels the urge, regardless of the fact that these formats have not been accepted by any standards committee. libjpeg-turbo is a community project. If enough people want me to do something, I'll do it, but not without first evaluating the need for it and making sure we're all on the same page.
At least with jpeg-8, the backward-incompatible ABI came with a lot of new features, but jpeg-9 does little more than introduce yet another non-standard format for lossless encoding whose usefulness has not been proven (and, in fact, my research will show that it is always possible to find a mode in libpng that compresses better and faster.) The purpose of libjpeg-- and this is straight from Tom Lane's mouth, because he and I spoke extensively this week-- was to encourage convergence around a standard format, not to introduce new non-standard formats.
jpeg-8 API/ABI compatibility was originally introduced so that people who had already upgraded to jpeg-8 but weren't taking advantage of the features could take advantage of libjpeg-turbo without recompiling. libjpeg-turbo has never endorsed the new features introduced by jpeg-7 and jpeg-8 and won't do so until/unless their effectiveness can be demonstrated.
I mean, I'll say again-- these are non-standard formats he's introducing. He could decide next year in jpeg-10 to completely change them up, which would create not just an ABI incompatibility but also a format incompatibility as well. Enough is enough.
I am not saying "no" to this. I'm saying "let's wait a bit". See how the community reacts. Personally, I think there will be close to zero demand for jpeg-9 except by those who haven't bothered to look at its change log.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Well, user filed the request for jpeg-9, that's how I knew it was even released.
We have jpeg-6b which will only install libjpeg.so.62 for binary-only packages for backwards compability. It is possible to install this package co-existing with system jpeg.
System jpeg is either libjpeg-turbo-1.2.1 (so.8) or jpeg-8d (.so.8). They both install all the required headers as well as the actual .so symlink for linking. Default is libjpeg-turbo, but user can just uninstall it, and immediately after install jpeg-8d and there is no need to rebuilt anything.
Currently I've added jpeg-9 to the repository 'masked' so it won't get pulled in by default, and the status is pending on libjpeg-turbo gaining .so.9 support OR dropping libjpeg-turbo support entirely. So this is the part of where I'm saying "we are waiting".
I don't see a problem adding new features to libjpeg, it should be treated like any other shared library, when API changes then SONAME raises... It's no way special. It might be problematic for binary-only distros which aren't using -Wl,--as-needed or the GNU gold linker yet, where overlinking is overwhelming. The actual packages using libjpeg isn't really that high, it's an distro-created problem.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
OK, so if a user filed the request, then you should ask the user why he/she needs jpeg-9. If they can't provide a good answer, then deny the request. It isn't rocket science.
What you seem to be saying to me is:
"implement the jpeg-9 API/ABI because some user of mine requested it, and if you don't, we're dropping libjpeg-turbo."
How is that ultimately serving your users? Presumably a lot more of them requested libjpeg-turbo than requested jpeg-9.
I don't know why you think threatening me is going to make me want to help you.
Again, you are really jumping the gun with a library that literally released hours ago. You seem to be expecting me to jump immediately the instant IJG says "jump", and it just doesn't work like that. Like I said, libjpeg-turbo is a community project. I listen to the community, not to just one voice. I want to hear from others before deciding whether this is something we want to do or not.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Oops, maybe that came out wrong but I'm really not threatening anything and I will have to ask others if we want turbo or 9, since it's not completely in my hands (propably good that way ;-)
Anyway, I'm fine with waiting for now since the issue is so new.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
It basically demonstrates how the new features of jpeg-8 and jpeg-9, while clever, do not generally accomplish anything that can't already be accomplished with higher performance using baseline JPEG or PNG.
It isn't the adding of features to libjpeg that I have a problem with. What I have a problem with is the fact that libjpeg v8 (and now v9 as well) introduce new image formats that have not been accepted by any standards body and, as far as I know, have not undergone significant peer review (the article above is my attempt at some form of the latter.) Introducing a new image format is a big deal. That's why we have standards bodies, and the IJG has unfortunately chosen to short-circuit that process and attempt to use the reputation Tom Lane built up in order to push out a format that its maintainer concocted himself. Other than the memory source/destination managers, all of the other features in jpeg-8 and jpeg-9 were prompted by these new formats. People are jumping on the jpeg-8 or jpeg-9 bandwagon without understanding that there may never be a well-defined need for the jpeg-8+ features in any software.
Personally, I believe that, while implementing the jpeg-9 API/ABI would be an easy thing to do, I really want to use this as an opportunity to force the community's hand on the issue. I think too many people don't understand that the Independent JPEG Group is neither Independent, JPEG, nor a Group. It's one guy pushing a non-standard new format because he thinks it's clever, but he hasn't demonstrated why it's necessary.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Given the results of the research above, as well as extensive discussions on the libjpeg-turbo-users and libjpeg-turbo-devel mailing lists, it is our belief that there are other lossless formats (JPEG-LS and webp, namely) that are industry standards and do a much better job than jpeg-9 of compressing content losslessly. Thus, we believe that the new, non-standard lossless SmartScale format introduced by jpeg-9 is unnecessary. Further, even if it were necessary, the additional color transform could have been implemented using an extension to the existing colorspace enum without breaking ABI compatibility.
Given these facts, it is our belief that there is no technical justification for using jpeg-9 over jpeg-8. It provides no other major features, and the fixes it provides are minor, they are already in libjpeg-turbo, and distribution maintainers could easily apply a patch to jpeg-8 to provide those fixes, if so desired.
We do not believe that there is any need to implement the jpeg-9 API/ABI at this time, given the lack of justifiable reasons for upgrading from jpeg-8 to jpeg-9 and given our lack of desire to show support for the direction in which the IJG is moving the libjpeg product.
Last edit: DRC 2013-01-17
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I'll say it again. First, show me software that is using jpeg-9. There is no use in emulating an API/ABI if no one is using it. If we eventually do decide to implement the jpeg-8 features, it will be some time before that happens.
Anyone who has a definable need to upgrade to jpeg-9, please let me know. I do not understand why anyone would need to upgrade from jpeg-8 to jpeg-9 unless they are either actively taking advantage of the Lossless SmartScale format or unless they are just blindly downloading the latest & greatest version without regard for whether they actually need it.
Within Gentoo Linux we provide package virtual/jpeg, which allows easy swapping between libjpeg-turbo (.so.8) and jpeg (.so.8)
So adding jpeg (.so.9) to the virtual would require the user to rebuild every package linking to the jpeg (.so.8) and easy swapping would no longer be possible
So if user wants jpeg-9 he can have it for whatever personal project he has, and denying that optionality would be just... silly. Alternative would be dropping libjpeg-turbo support, which we don't want either.
Last edit: Samuli Suominen 2013-01-14
OK, but does jpeg-9 co-exist with jpeg-8 on the system in that case? Which version is used as the "system" libjpeg? Or is that controlled by switching the virtual? Presumably Gentoo has a bunch of system software that is linked with a particular version of libjpeg, so you can't just swap in a new ABI at the system level without breaking a lot of things. If you're considering upgrading the system version of libjpeg to jpeg-9, then honestly, I think you should wait and see how the community reacts to the new IJG release.
Personally, my feeling is that you're creating a problem where one doesn't exist yet. If users were asking for jpeg-9, I would understand the desire to implement it, as well as an API/ABI compatibility mechanism in libjpeg-turbo. But I mean, it just literally released a few hours ago. I have yet to run across any software that even takes advantage of the jpeg-8 features, other than possibly the memory-based source/destination managers. If you know of anything that is actually using SmartScale, I'd love to hear about it.
What seems silly to me is continuing to view IJG as an authoritative source, when it's really just one guy doing whatever the heck he wants, breaking backward compatibility and introducing new formats whenever he feels the urge, regardless of the fact that these formats have not been accepted by any standards committee. libjpeg-turbo is a community project. If enough people want me to do something, I'll do it, but not without first evaluating the need for it and making sure we're all on the same page.
At least with jpeg-8, the backward-incompatible ABI came with a lot of new features, but jpeg-9 does little more than introduce yet another non-standard format for lossless encoding whose usefulness has not been proven (and, in fact, my research will show that it is always possible to find a mode in libpng that compresses better and faster.) The purpose of libjpeg-- and this is straight from Tom Lane's mouth, because he and I spoke extensively this week-- was to encourage convergence around a standard format, not to introduce new non-standard formats.
jpeg-8 API/ABI compatibility was originally introduced so that people who had already upgraded to jpeg-8 but weren't taking advantage of the features could take advantage of libjpeg-turbo without recompiling. libjpeg-turbo has never endorsed the new features introduced by jpeg-7 and jpeg-8 and won't do so until/unless their effectiveness can be demonstrated.
I mean, I'll say again-- these are non-standard formats he's introducing. He could decide next year in jpeg-10 to completely change them up, which would create not just an ABI incompatibility but also a format incompatibility as well. Enough is enough.
I am not saying "no" to this. I'm saying "let's wait a bit". See how the community reacts. Personally, I think there will be close to zero demand for jpeg-9 except by those who haven't bothered to look at its change log.
Well, user filed the request for jpeg-9, that's how I knew it was even released.
We have jpeg-6b which will only install libjpeg.so.62 for binary-only packages for backwards compability. It is possible to install this package co-existing with system jpeg.
System jpeg is either libjpeg-turbo-1.2.1 (so.8) or jpeg-8d (.so.8). They both install all the required headers as well as the actual .so symlink for linking. Default is libjpeg-turbo, but user can just uninstall it, and immediately after install jpeg-8d and there is no need to rebuilt anything.
Currently I've added jpeg-9 to the repository 'masked' so it won't get pulled in by default, and the status is pending on libjpeg-turbo gaining .so.9 support OR dropping libjpeg-turbo support entirely. So this is the part of where I'm saying "we are waiting".
I don't see a problem adding new features to libjpeg, it should be treated like any other shared library, when API changes then SONAME raises... It's no way special. It might be problematic for binary-only distros which aren't using -Wl,--as-needed or the GNU gold linker yet, where overlinking is overwhelming. The actual packages using libjpeg isn't really that high, it's an distro-created problem.
OK, so if a user filed the request, then you should ask the user why he/she needs jpeg-9. If they can't provide a good answer, then deny the request. It isn't rocket science.
What you seem to be saying to me is:
"implement the jpeg-9 API/ABI because some user of mine requested it, and if you don't, we're dropping libjpeg-turbo."
How is that ultimately serving your users? Presumably a lot more of them requested libjpeg-turbo than requested jpeg-9.
I don't know why you think threatening me is going to make me want to help you.
Again, you are really jumping the gun with a library that literally released hours ago. You seem to be expecting me to jump immediately the instant IJG says "jump", and it just doesn't work like that. Like I said, libjpeg-turbo is a community project. I listen to the community, not to just one voice. I want to hear from others before deciding whether this is something we want to do or not.
Oops, maybe that came out wrong but I'm really not threatening anything and I will have to ask others if we want turbo or 9, since it's not completely in my hands (propably good that way ;-)
Anyway, I'm fine with waiting for now since the issue is so new.
This article should give you more insight into why I really want to draw a line in the sand on this:
http://www.libjpeg-turbo.org/About/SmartScale
It basically demonstrates how the new features of jpeg-8 and jpeg-9, while clever, do not generally accomplish anything that can't already be accomplished with higher performance using baseline JPEG or PNG.
It isn't the adding of features to libjpeg that I have a problem with. What I have a problem with is the fact that libjpeg v8 (and now v9 as well) introduce new image formats that have not been accepted by any standards body and, as far as I know, have not undergone significant peer review (the article above is my attempt at some form of the latter.) Introducing a new image format is a big deal. That's why we have standards bodies, and the IJG has unfortunately chosen to short-circuit that process and attempt to use the reputation Tom Lane built up in order to push out a format that its maintainer concocted himself. Other than the memory source/destination managers, all of the other features in jpeg-8 and jpeg-9 were prompted by these new formats. People are jumping on the jpeg-8 or jpeg-9 bandwagon without understanding that there may never be a well-defined need for the jpeg-8+ features in any software.
Personally, I believe that, while implementing the jpeg-9 API/ABI would be an easy thing to do, I really want to use this as an opportunity to force the community's hand on the issue. I think too many people don't understand that the Independent JPEG Group is neither Independent, JPEG, nor a Group. It's one guy pushing a non-standard new format because he thinks it's clever, but he hasn't demonstrated why it's necessary.
Given the results of the research above, as well as extensive discussions on the libjpeg-turbo-users and libjpeg-turbo-devel mailing lists, it is our belief that there are other lossless formats (JPEG-LS and webp, namely) that are industry standards and do a much better job than jpeg-9 of compressing content losslessly. Thus, we believe that the new, non-standard lossless SmartScale format introduced by jpeg-9 is unnecessary. Further, even if it were necessary, the additional color transform could have been implemented using an extension to the existing colorspace enum without breaking ABI compatibility.
Given these facts, it is our belief that there is no technical justification for using jpeg-9 over jpeg-8. It provides no other major features, and the fixes it provides are minor, they are already in libjpeg-turbo, and distribution maintainers could easily apply a patch to jpeg-8 to provide those fixes, if so desired.
We do not believe that there is any need to implement the jpeg-9 API/ABI at this time, given the lack of justifiable reasons for upgrading from jpeg-8 to jpeg-9 and given our lack of desire to show support for the direction in which the IJG is moving the libjpeg product.
Last edit: DRC 2013-01-17