#555 scintilla shared library

Won't_Implement
closed
Neil Hodgson
Scintilla (356)
3
2014-01-16
2009-01-05
Paul Wise
No

Right now there are several packages in Debian GNU/Linux that duplicate the scintilla code; at least scite, anjuta, qscintilla and geany. It would be great if you could turn scintilla into a shared library (at least on Linux) that applications could link against. At least the Anjuta project doesn't like embedding scintilla and would like this:

http://blogs.gnome.org/johannes/2009/01/05/anjuta-226-preview/

This would require defining a public API for programs interacting with scintilla, creating and tracking an ABI number from that. Some good information about API/ABI and libraries from Debian's perspective is here:

http://www.netfort.gr.jp/~dancer/column/libpkg-guide/libpkg-guide.html

Here is a guide from GNU about libtool:

http://www.gnu.org/software/libtool/manual/

If you are not interested in turning scintilla into a shared library, adding a static library might be useful to those who currently embed scintilla code.

Discussion

  • Neil Hodgson
    Neil Hodgson
    2009-01-05

    • milestone: --> Won't_Implement
    • priority: 5 --> 3
    • assigned_to: nobody --> nyamatongwe
     
  • Neil Hodgson
    Neil Hodgson
    2009-01-05

    Distributing a shared library version of Scintilla could lead to application breakage when the Scintilla library was updated. One of the things learned from DLL hell on Windows was to always distribute all the code in an application together as a *tested* unit. Most people that would like a shared library want more than the base code so that it fits into their environment such as Python users who want it packaged as a Python extension library.

    QScintilla can not use the same library as Anjuta since it is built for Qt, not GTK+.

    The standard Scintilla build on GTK+ (scintilla/gtk/makefile) builds a static library: scintilla.a.

     
  • Paul Wise
    Paul Wise
    2009-01-06

    I would really like you to reconsider this.

    Firstly, updating the library does not always mean application breakage. The way it works on Linux is that each library has a SONAME (or ABI number). The applications link against libscintilla.so.1 (or scintilla-1.dll) rather than an unversioned interface like libscintilla.so (or scintilla.dll). You then update the SONAME whenever you break ABI compatibility. You then rebuild the applications against the new SONAME. There is also a couple of extra numbers that are changed when new interfaces are added but the ABI compatibility is preserved. These other numbers are used to allow applications to depend on the specific version of the library that introduced the interfaces. In this way, the application can be made to always work because it depends on what it needs.

    Many python extensions are simply wrappers around libraries written in this way.

    About Qt vs GTK, if the dependency is application -> scintilla core -> scintilla gtk, then you could make the gtk bits into a private plugin which is loaded at runtime and exports a standard API/ABI to libscintilla. If the dependency is application -> scintilla gtk -> scintilla core, then you could make libscintillagtk and have it link against libscintillacore.

    Apologies for not checking if there was a static library, I'll ask the anjuta folks if they can use that.

    I hope this helps, if not, sorry for the noise. I completely understand if you don't want to put the effort in to making a shared library.

     
  • Neil Hodgson
    Neil Hodgson
    2009-01-07

    Versioning a component is insufficient to be certain that a different component version will operate bug free with the application. The combination of component version and application must be tested to ensure correctness. Trying to avoid this cost just leads to your customers being turned into testers who then have to cope with any resulting failures.

    This isn't about effort but about achieving a higher quality outcome for users.

     
  • Paul Wise
    Paul Wise
    2009-01-07

    Would you at least be open to providing an off-by-default option to create a shared library with proper versioning? The folks who want one can use it, heed the big warning and accept the risks attached to that and everyone else can link statically as they do now.

    In any case, thanks for your reading this request and your work on scintilla.

     
  • Enrico Tröger
    Enrico Tröger
    2009-01-07

    From a Scintilla user's point of view:
    yes, it is a little uncommon to have a library statically linked only and directly embedded in projects.

    But the big advantage is, as Neil already pointed out, that applications know which Scintilla they are using and it's keeping stable as long as they don't update it themselves. And it decreases the general dependencies count on apps. But yes, using shared libraries is generally still better because of easier updating, less duplicate code in memory and such.

    In Geany, we like to keep the embedded Scintilla copy because we have some small modifications made (mainly column mode editing, C89 comments in header files) and it's way easier to have Scintilla embedded in the source code as it is something essential for Geany.
    Depending on an external Scintilla library would probably be fine in Debian once it is packaged and in the archives. But what about the dozens of other distributions? Users then might experience problems with getting a Scintilla library package for their distribution or need just another package to compile from source. This would be annoying.

    Just my 2cents.

     
  • Neil Hodgson
    Neil Hodgson
    2009-01-08

    I'll accept a makefile target that produces a .so if it is reasonably simple and requires zero maintenance effort. The current version number is found in scintilla/version.txt and contains 2 (not 3) parts: a single digit major version number and a 2 digit minor version number.

     
  • Paul Wise
    Paul Wise
    2009-01-14

    Shared libraries are never zero maintenance. You need to at least keep track of your ABI. So nevermind about this request, I'll try to get the scintilla users to try the static library.

    eht16: how about getting those patches included in scintilla so that you can simply compile against the scintilla static library?

    eht16: I note that you don't embed glibc, gtk, cairo, pango, libvte etc in geany. Just depend on the scintilla static library and then the other distros will add it.

     
  • Enrico Tröger
    Enrico Tröger
    2009-01-17

    @Paul:
    We want to get rid off of custom changes and use stock Scintilla, no doubt.
    The mentioned C89 comments problem has been solved, Neil removed them.
    The column mode editing probably needs more work to get Neil's concerns resolved but unless there is someone who want to work on it, it's unlikely that we find a solution. The full discussion is in https://sourceforge.net/tracker/?func=detail&atid=352439&aid=545068&group_id=2439, we are using the patch in the report since about a year and are happy with it.
    Then there is another small issue with the properties lexer, I probably will provide a patch soon for it.

    After all, our goal is to use a stock Scintilla copy.

    > I note that you don't embed glibc, gtk, cairo, pango, libvte etc in geany.

    Sure, it would be silly to do that, obviously.

    > Just depend on the scintilla static library and then the other distros will add it.

    Maybe. Maybe not. Then users need to compile everything from source and already get frustrated before they used anything of the software at all.
    But well, if Scintilla is provided as a shared lib, we (Geany) can think about that again. I'm not saying it will never happen but for now it's just unlikely we will switch soon.

     
  • Distributions like shared libraries because it makes life easier for system administrators. If you leave it up to each application to bundle and verify that their application works with an exact copy of scintilla, then the system administrator has to trust that each application developer is on top of the problems that can occur in scintilla. If, for instance, scintilla were to announce a security fix for an issue, the system administrator would have to check that none of the applications he's using bundle an old scintilla version and if they do, whether they have in fact forked the code, like Geany, and if they've forked, whether the fork fixes the security issue.

    This is a nightmare when there's multiple applications on a system bundling different versions of the library.

    With a single shared library being distributed by their OS Vendor, it's simple for the system administrator to check that the scintilla package has been updated to a version that is no longer vulnerable. If that's the case, they can be assured that the applications on the system have been protected, simply because they installed the updated scintilla package.

    With a single static library being distributed in a separate package, the system administrator is not able to verify easily but the OS vendor is. The OS vendor can port and rebuild the applications that depend on the scintilla library against the new static package that they've built. If there's packages left that are linked against the insecure version of the library, the system administrators have a single point of contact to hold responsible, their vendor.

    If you want to listen to a half hour video, I gave a talk at PyCon about this subject and in what situations bundling makes sense: http://pycon.blip.tv/file/2072580/

    Note that in python, every module can be considered a "shared library", it's just a matter of whether the application authors are bundling or relying on the system version of the module.

    If you'd rather read something than listen to a long talk, I've drafted rationale for the rule against bundling libraries in Fedora: https://fedoraproject.org/wiki/No_Bundled_Libraries

     
  • Neil Hodgson
    Neil Hodgson
    2009-07-07

    What tests will these distributions perform to ensure compatibility between a new release of Scintilla and each application that uses the shared library?

     
  • Enrico Tröger
    Enrico Tröger
    2009-07-07

    Just for clarity:
    In Geany, we don't have a forked Scintilla nor do we intend to fork it in any way. Instead, we intend to use stock Scintilla and avoid any custom modifications.
    The current changes we have in out copy are documented in http://geany.svn.sourceforge.net/viewvc/geany/trunk/scintilla/scintilla_changes.patch?revision=3911&view=markup, the only real change code-wise is the earlier mentioned column mode editing patch. The other two changes are only removing lexers we don't use and a newly generated marshaller file but this isn't strictly necessary at all.

    Wrt updates and security related fixes: we updated our copy two days after the latest Scintilla release. And in between releases, we do apply important fixes from Scintilla CVS to our copy if necessary.
    I doubt distribution packagers would be that fast, not all of them.

    And the previous arguments mentioned in this report are still valid, e.g. maximum compatibility as application developers know which Scintilla version they use and so they know it works with the app.

     
  • Neil Hodgson
    Neil Hodgson
    2009-07-07

    If some individual or group wants to take on the job of providing shared libraries of Scintilla then I won't try to obstruct them. I will point to them from the Scintilla web site as the place to get shared libraries from.

    There has not been a pure bug-fix release of Scintilla since 1.32 in September 2000. Almost every release is supposed to be a superset of the previous releases but improvements can cause compatibility problems. For example, a patch may create a separate lexical class for Unicode literals in Python code but this may confound code (such as a localization helper) that searches for code only in the set of string classes from earlier versions. Even bug fixes can cause compatibility failures.

    The shared library builder could produce a sequence of releases with fewer behaviour changes by omitting the lexers from the library with each lexer also being distributed as a separately versioned shared library.

     
  • Neil Hodgson
    Neil Hodgson
    2014-01-16

    • status: open --> closed