java-gnome-hackers

Re: [java-gnome-hackers] Libraries vs Packaging (1): Linkage

[Philipp H. <phi...@gm...>]

Hallo everyone,

First of all: thank you for a great library. I wouldn't know what to
do without it :-)

I am referring to the issue of "optional dependencies" discussed in
[1]: I can definitely understand the discussed problems: many
developers probably only use a fraction of the built-in functionality.
Reducing dependencies and having optional dependencies is certainly a
must in big projects like this. However, in my opinion, an issue like
this should not stand in the way of making progress and including new
functionalities: Francisco implemented support for App Indicators over
a year ago [2] and even though is works like a charm, it is still not
in the library.

In the long run, I would love a dynamic loader that only allows the
use of classes for which the native libraries are available. For
example, if libappindicate is present, an application may use the
corresponding Java classes. If not, an exception is thrown.

However, to avoid getting stuck in an endless discussion about how to
implement this, I'd suggest to adjust the configure script to allow
choosing libraries at build-time. This is exactly what Serkan
suggested (2011-05-05 07:17). Something like: ./configure
--with-module=libappindicator

That way, if the "default version" in the distros' repositories is not
built with the desired functionality, one could check out the source
of the current stable version (4.0.19) and simply re-compile it
--with-module=...

What do you think of that?

Cheers
Philipp

[1] https://sourceforge.net/mailarchive/message.php?msg_id=27435400
[2] https://sourceforge.net/mailarchive/message.php?msg_id=25397649

Re: [java-gnome-hackers] Libraries vs Packaging (1): Linkage

[Philipp H. <phi...@gm...>]

Hello again,

I sort of implemented the changes I suggested in the post from
earlier. It allows to optionally compile the library with different
modules. Please try it and let me know what you think:

$ bzr co --lightweight lp:~binwiederhier/java-gnome/modules java-gnome-modules
$ cd java-gnome-modules
$ ./configure --with-module=appindicator-0.1
$ make
$ ldd tmp/*.so
--> Appindicator included

Changes I made:
- moved the directory "src/defs" to "src/defs/core"
- moved the directory "src/bindings" to "src/bindings/core"
- added one directory for each module:
  1. "src/defs/appindicator-0.1" and "src/bindings/appindicator-0.1"
  2. "src/defs/indicate-gtk" and "src/bindings/indicate-gtk" (*** see below)
- adjusted all scripts (mainly configure, faster, Makefile and
BindingsGenerator) to support more than one "defs" and "bindings"
folder

Note: when you recompile, make sure to delete the tmp-folder since it
might contain "modules" that are not defined in the --with-module
thing arguments anymore. "make clean" is not enough here.

What do you think? It's not a dynamic loader, but it's a start and it
doesn't break the old stuff...

Cheers,
Philipp

*** On Natty, use 'indicate-gtk-0.5' instead. It didn't compile for me
though. I think because of the major version jump ...

On Tue, May 31, 2011 at 10:40 PM, Philipp Heckel
<phi...@gm...> wrote:
> Hallo everyone,
>
> First of all: thank you for a great library. I wouldn't know what to
> do without it :-)
>
> I am referring to the issue of "optional dependencies" discussed in
> [1]: I can definitely understand the discussed problems: many
> developers probably only use a fraction of the built-in functionality.
> Reducing dependencies and having optional dependencies is certainly a
> must in big projects like this. However, in my opinion, an issue like
> this should not stand in the way of making progress and including new
> functionalities: Francisco implemented support for App Indicators over
> a year ago [2] and even though is works like a charm, it is still not
> in the library.
>
> In the long run, I would love a dynamic loader that only allows the
> use of classes for which the native libraries are available. For
> example, if libappindicate is present, an application may use the
> corresponding Java classes. If not, an exception is thrown.
>
> However, to avoid getting stuck in an endless discussion about how to
> implement this, I'd suggest to adjust the configure script to allow
> choosing libraries at build-time. This is exactly what Serkan
> suggested (2011-05-05 07:17). Something like: ./configure
> --with-module=libappindicator
>
> That way, if the "default version" in the distros' repositories is not
> built with the desired functionality, one could check out the source
> of the current stable version (4.0.19) and simply re-compile it
> --with-module=...
>
> What do you think of that?
>
> Cheers
> Philipp
>
> [1] https://sourceforge.net/mailarchive/message.php?msg_id=27435400
> [2] https://sourceforge.net/mailarchive/message.php?msg_id=25397649
>

Re: [java-gnome-hackers] Libraries vs Packaging (1): Linkage

[Andrew C. <an...@op...>]

On Tue, 2011-05-31 at 22:40 +0200, Philipp Heckel wrote:

> What do you think of that?

I think it's all good.

I've written down some responses below. My only real interest is helping
the project move forward, so please take everything I've written with
that in mind.

++

> However, to avoid getting stuck in an endless discussion about how to
> implement this,

We *do* have to implement $crazy_idea, and hand-waving away engineering
problems isn't really facing up to the challenges involved. In almost
every aspect java-gnome is the way it is because of such limitations. 

Foremost, however, is developer experience: what is it like to use
java-gnome? Thus far we have provided a unified experience to people
using the library; if I write code on Debian you can run it on Gentoo.
I've felt that was important for the last 5 years, but other people can
say what they think.


> many
> developers probably only use a fraction of the built-in functionality.

By the time you're doing anything non-trivial you rapidly find yourself
using the bulk of what's in the library. And that's only going to
increase as GNOME 3 brings increasing convergence between the various
libraries in the stack. The only real exception I'm aware of are those
using it exclusively to do doing drawing with Cairo.

++


>I am referring to the issue of "optional dependencies" discussed in
> [1]: I can definitely understand the discussed problems: 
> Reducing dependencies and having optional dependencies is certainly a
> must in big projects like this.

Well, it is and it isn't. Sure, I raised this topic because it's been on
my mind for a while. And definitely it'd be fantastic to support obscure
corner cases. However, the build-time dependency of java-gnome is "the
GNOME desktop" and the purpose of the library is to allow people to
write applications for the GNOME desktop.

Taking that notion and making it the actual code dependency sure as hell
saved a lot of complication over the years.

As a technical matter, the build system is works by building everything;
if there is a declaration for GtkSomething, Java code and C code are
emitted and that C code will expect to find gtk_somethin_do_whatever()
in headers and at link time. It's (absurdly) monolithic, but as a
practical matter is did mean we got on with it over the past 5 years
instead of wasting time making something that isn't the same for
everyone, and since the build- & run- time requirement is "a GNOME
desktop" I still believe it wasn't actually unreasonable to go this way.

Anyway, that was then, this is now.


> In the long run, I would love a dynamic loader that only allows the
> use of classes for which the native libraries are available. For
> example, if libappindicate is present, an application may use the
> corresponding Java classes. If not, an exception is thrown.

That shouldn't be that hard; I can see a number of ways of doing this
where the result is UnsatisfiedLinkError.

This, however, is not that cool. Consider the current and normal case:
you only get that terminal exception thrown if there's some installation
problem; your app uses java-gnome and java-gnome can either find its
native half or it can't. Very important that it can.

So expecting the library to cope with linking to some things and not
linking to other things is going to get messy. The next matter is pubic
API: do you *really* want to put a try-catch block around every library
call? Surely not. But where else is it going to go? Silently fail? I
don't think so. Uncaught exception? Hardly. So what then? The fact that
I don't have an answer to that is one of the reasons we didn't do this
years ago.

We don't have #ifdef in Java. I personally consider that a feature, not
a bug, but regardless it's not up to me; that's just the way it is and
java-gnome merely conforms accordingly. So enabling or disabling at
configure time as in the C paradigm is only part of the solution, since
we also need to decide what to do on the Java side.

Anyway, I daresay that a uncaught runtime exception is what we're going
to do. I presume UnsupportedOperationException or something custom
parallel to org.gnome.glib.FatalError, maybe
org.gnome.glib.UnsatisfiedFeatureError. whatever. That leaves it up to
the app developer using the bindings to deal with the feature being
present or not? I guess.

++

Another question is API documentation: "this feature will only be
available if java-gnome was built with $X feature enabled" in every
method of the relevant JavaDoc sounds painful - and inelegant.

++


> That way, if the "default version" in the distros' repositories is not
> built with the desired functionality, one could check out the source
> of the current stable version (4.0.19) and simply re-compile it
> --with-module=...

Sure, but I think the real issue is that something like appindicator
that is never going to *be* a part of GNOME and is only available in one
downstream distro is perhaps not entirely suitable for an upstream
library like java-gnome.

Note that this is all same engineering challenge as e.g. WebKit, so the
discussion isn't wasted at all. But first we need to make some design
decisions about what our public API & developer experience is in the
face of optional {upstream,downstream} dependencies.

++

As an aside, but I think a relevant case: I'd observe in passing that I
don't have appindicator packages installed on my Ubuntu system. I'm
running GNOME 3 of course, and have no need of Canonical's Unity stuff.
I would be quite upset if installing Guillaume's java-gnome package
suddenly dragged all those packages in.

For which, of course, there is a standard answer: different packages
depend on different dependencies. Sure. This was actually going to be
the topic of a "(2)" series of emails, but for now consider a few basic
questions:

1. We have the lovely design of only needing the developer to include
one .jar file. That's nice. If we have lots of packages we implicitly
force developers to include zillions of .jar files. That would suck.

2. What about Gentoo? Unlike Debian, they doesn't build multiple binary
packages for a single source package. They do have a wonderful build
time optional dependency mechanism though.

++

If the code is present in the library but not to be built, how [this is
an implementation issue but a huge one: like I said, right now all the
code builds - the list of .java and .c files to be built is simply the
result of `find`. There's no manually articulated list of classes. There
could be. It's been nice not to have all the bugs that happen because
some manual file list is wrong. Perennial nightmare in other projects]

++

> However, in my opinion, an issue like
> this should not stand in the way of making progress and including new
> functionalities:

I agree.

> Francisco implemented support for App Indicators over
> a year ago [2] and even though is works like a charm, it is still not
> in the library.

So we have some design and engineering issues to consider. What do you
want to happen if you're on any Linux other than Ubuntu-with-Unity and
there are no app indicators?

Should the class not exist [which would mean conditionally including or
not including code in the .jar file. Very hard]?

Should the C code not be emitted [which would mean a call down that code
path would hit the native declarations and would result in
UnsatifiedLinkError]?


++

Whew!

I'm sitting outside at a cafe in the early morning shivvering my ass
off. :) so I think I'll just send this now.

To reiterate what I said at the beginning, I am really happy to be
having this discussion. I think you'll agree that the implications are
non-trivial. And as long as people are thinking about that, and keeping
in mind the experience for

        developers [our audience],

and not forgetting

        hackers [the people who have to a) do the work now, b) package
        the resultant mess and c) & d) maintain it in the future],

and finally most important of all

        users [make sure that applications written against java-gnome
        provide a good experience for the users of those applications]

then I'm sure that we'll do the right thing.

If people decide that any or all of these concerns are not important,
then that's fine by me.

AfC
Sydney

Re: [java-gnome-hackers] Libraries vs Packaging (1): Linkage

[Philipp H. <phi...@gm...>]

Hi again,

As you might have seen, I've implemented some changes which I think
solve many of the problems you mention below: only the modules you
define with the "--with-module=" parameter are compiled in the library.
If you leave out a module, the JAR will not contain the Java classes and
the *.so will not be compiled with useless stuff. So no
UnsatisfiedLinkExceptions.

I could even imagine something like "--without-module=(..)" so one could
exclude default modules (such as rsvg, or even cairo?!). This is
particularly useful if you don't want to use the whole thing, but only
very limited parts of java-gnome. I believe that could reduce the
filesize (and used memory) drastically.

Most importantly: my adjustments do NOT change the default behavior. So
the distro versions will not be affected by this.

I'd appreciate it if you could take a look at it and tell me whether you
might consider including this in the trunk.

> It's (absurdly) monolithic, but as a
> practical matter is did mean we got on with it over the past 5 years
> instead of wasting time making something that isn't the same for
> everyone, and since the build- & run- time requirement is "a GNOME
> desktop" I still believe it wasn't actually unreasonable to go this way.

It definitely wasn't. That's why we're talking about /optional/ modules,
because they are not for everyone. Those who want them can hence simply
recompile the library.

>> In the long run, I would love a dynamic loader that only allows the
>> [..]
> 
> So expecting the library to cope with linking to some things and not
> linking to other things is going to get messy. The next matter is pubic
> API: do you *really* want to put a try-catch block around every library
> call? Surely not. But where else is it going to go? Silently fail? I
> don't think so. Uncaught exception? Hardly. So what then? The fact that
> I don't have an answer to that is one of the reasons we didn't do this
> years ago.

True. That's definitely not how you'd want it. However, I could imagine
something like a "hasModule(..)" method for which an application could
check in the beginning if all required modules are available. If they
are, everything is good. If they aren't, the application exists. That
way, the java-gnome library could indeed simply throw an
UnsatisfiedLinkException if a non-present module if about to be used.

In my opinion, the build time module inclusion/exclusion is much cleaner.


> We don't have #ifdef in Java. I personally consider that a feature, not
> a bug, but regardless it's not up to me; that's just the way it is and
> java-gnome merely conforms accordingly. So enabling or disabling at
> configure time as in the C paradigm is only part of the solution, since
> we also need to decide what to do on the Java side.

See above. I did address this by simply leaving out both Java and C code
for the modules that are not in the --with-module parameter.

> Another question is API documentation: "this feature will only be
> available if java-gnome was built with $X feature enabled" in every
> method of the relevant JavaDoc sounds painful - and inelegant.

That's true if it is method specific, but it is not if you do it for a
whole package (cp. appindicator).

> Sure, but I think the real issue is that something like appindicator
> that is never going to *be* a part of GNOME and is only available in one
> downstream distro is perhaps not entirely suitable for an upstream
> library like java-gnome.
> [..]
> As an aside, but I think a relevant case: I'd observe in passing that I
> don't have appindicator packages installed on my Ubuntu system. I'm
> running GNOME 3 of course, and have no need of Canonical's Unity stuff.
> I would be quite upset if installing Guillaume's java-gnome package
> suddenly dragged all those packages in.

That's why I think it should be an optional module. It's in the code,
but only for those who want/need it.


> [...]
> So we have some design and engineering issues to consider. What do you
> want to happen if you're on any Linux other than Ubuntu-with-Unity and
> there are no app indicators?
> 
> Should the class not exist [which would mean conditionally including or
> not including code in the .jar file. Very hard]?
> 
> Should the C code not be emitted [which would mean a call down that code
> path would hit the native declarations and would result in
> UnsatifiedLinkError]?

I think if the *.so and *.jar is compiled with appindicator support, it
is not the responsibility of java-gnome to make sure that the native
library exists -- it's the responsibility of the application using it.

Don't you think?

> Whew!
> 
> I'm sitting outside at a cafe in the early morning shivvering my ass
> off. :) so I think I'll just send this now.

:-)


I hope I'm not the only one commenting on this thread, since I might
have a different point of view than others. My main goal is to help the
java-gnome library to get more flexible when it comes to optional
functionalities. As I intend to include/use only small parts of it in a
project of mine, it should be able to reduce the file size of both JAR
and SO to a minimum.

Cheers,
Philipp

Re: [java-gnome-hackers] Libraries vs Packaging (1): Linkage

[Francisco O. <fr...@pu...>]

Hi all,

First of all, thanks Philipp to bring this back, I really love to see
the indicator patch in trunk too.
Not only for the particular functionality but to reopen the discussion
about the optional dependencies.

Putting the topic about including non-pure-gnome dependencies aside,

Let me give you my point of view (see comments below)

> Hi again,
>
> As you might have seen, I've implemented some changes which I think
> solve many of the problems you mention below: only the modules you
> define with the "--with-module=" parameter are compiled in the library.
> If you leave out a module, the JAR will not contain the Java classes and
> the *.so will not be compiled with useless stuff. So no
> UnsatisfiedLinkExceptions.

I really like the compile-time flags proposal
...but not include the java classes in the JAR is BIG problem from
(library) users perspective. You will end up with a non consistent API
from java side, you will have version java-gnome 4.0.20 with the
library or without it and the user will not know it. Or based on the
distro will have it or not. Same java program will throw a
ClassNotFoundException in the platform not supporting the
functionality.

>
> I could even imagine something like "--without-module=(..)" so one could
> exclude default modules (such as rsvg, or even cairo?!). This is
> particularly useful if you don't want to use the whole thing, but only
> very limited parts of java-gnome. I believe that could reduce the
> filesize (and used memory) drastically.
>
> Most importantly: my adjustments do NOT change the default behavior. So
> the distro versions will not be affected by this.
>
> I'd appreciate it if you could take a look at it and tell me whether you
> might consider including this in the trunk.
>
> > It's (absurdly) monolithic, but as a
> > practical matter is did mean we got on with it over the past 5 years
> > instead of wasting time making something that isn't the same for
> > everyone, and since the build- & run- time requirement is "a GNOME
> > desktop" I still believe it wasn't actually unreasonable to go this way.
>
> It definitely wasn't. That's why we're talking about /optional/ modules,
> because they are not for everyone. Those who want them can hence simply
> recompile the library.
>
> >> In the long run, I would love a dynamic loader that only allows the
> >> [..]
> >
> > So expecting the library to cope with linking to some things and not
> > linking to other things is going to get messy. The next matter is pubic
> > API: do you *really* want to put a try-catch block around every library
> > call? Surely not. But where else is it going to go? Silently fail? I
> > don't think so. Uncaught exception? Hardly. So what then? The fact that
> > I don't have an answer to that is one of the reasons we didn't do this
> > years ago.
>
> True. That's definitely not how you'd want it. However, I could imagine
> something like a "hasModule(..)" method for which an application could
> check in the beginning if all required modules are available. If they
> are, everything is good. If they aren't, the application exists. That
> way, the java-gnome library could indeed simply throw an
> UnsatisfiedLinkException if a non-present module if about to be used.
>
> In my opinion, the build time module inclusion/exclusion is much cleaner.
>

I was thinking about this for more than a year too. Here are 3 options:

- leave it to the user: hasModule() solution Philipp suggested and
throw the runtime exception if its invoked anyway (hey, users call)

- check on every method call: if dependency is not satisfied continue
silently. The problem with this is if the response trigger some
condition, and that will depend on the library :(

- using a factory: its a derivative of the previous one, but cleaner.
The factory is responsible of provide the correct implementation, it
will return a class that implements an interface which can be the real
implementation or the mocked/not supported by the system one. Again ,
its more code but cleaner.


>
> > We don't have #ifdef in Java. I personally consider that a feature, not
> > a bug, but regardless it's not up to me; that's just the way it is and
> > java-gnome merely conforms accordingly. So enabling or disabling at
> > configure time as in the C paradigm is only part of the solution, since
> > we also need to decide what to do on the Java side.
>
> See above. I did address this by simply leaving out both Java and C code
> for the modules that are not in the --with-module parameter.
>
> > Another question is API documentation: "this feature will only be
> > available if java-gnome was built with $X feature enabled" in every
> > method of the relevant JavaDoc sounds painful - and inelegant.
>
> That's true if it is method specific, but it is not if you do it for a
> whole package (cp. appindicator).
>
> > Sure, but I think the real issue is that something like appindicator
> > that is never going to *be* a part of GNOME and is only available in one
> > downstream distro is perhaps not entirely suitable for an upstream
> > library like java-gnome.
> > [..]
> > As an aside, but I think a relevant case: I'd observe in passing that I
> > don't have appindicator packages installed on my Ubuntu system. I'm
> > running GNOME 3 of course, and have no need of Canonical's Unity stuff.
> > I would be quite upset if installing Guillaume's java-gnome package
> > suddenly dragged all those packages in.
>
> That's why I think it should be an optional module. It's in the code,
> but only for those who want/need it.
>
>
> > [...]
> > So we have some design and engineering issues to consider. What do you
> > want to happen if you're on any Linux other than Ubuntu-with-Unity and
> > there are no app indicators?
> >
> > Should the class not exist [which would mean conditionally including or
> > not including code in the .jar file. Very hard]?
> >
> > Should the C code not be emitted [which would mean a call down that code
> > path would hit the native declarations and would result in
> > UnsatifiedLinkError]?
>
> I think if the *.so and *.jar is compiled with appindicator support, it
> is not the responsibility of java-gnome to make sure that the native
> library exists -- it's the responsibility of the application using it.
>
> Don't you think?

I agree the user should check it...or use the mocked version.
The problem with the later is big libraries (appindicator is small)
will create more code.

>
> > Whew!
> >
> > I'm sitting outside at a cafe in the early morning shivvering my ass
> > off. :) so I think I'll just send this now.
>
> :-)
>
>
> I hope I'm not the only one commenting on this thread, since I might
> have a different point of view than others. My main goal is to help the
> java-gnome library to get more flexible when it comes to optional
> functionalities. As I intend to include/use only small parts of it in a
> project of mine, it should be able to reduce the file size of both JAR
> and SO to a minimum.
>
> Cheers,
> Philipp
>

Thanks,

Fran

Re: [java-gnome-hackers] Libraries vs Packaging (1): Linkage

[Andrew C. <an...@op...>]

On Wed, 2011-06-01 at 14:19 -0300, Francisco Ortiz wrote:
...
> I really like the compile-time flags proposal
> ...but not include the java classes in the JAR is BIG problem from
> (library) users perspective. You will end up with a non consistent API
> from java side, you will have version java-gnome 4.0.20 with the
> library or without it and the user will not know it. Or based on the
> distro will have it or not. Same java program will throw a
> ClassNotFoundException in the platform not supporting the
> functionality.

Yes, that's my concern exactly. We *did* have that situation in the 2.x
days, because we had Cairo bindings that needed PDF support, but at the
time some distros were shipping Cairo built with --enable-pdf and some
were not.

Made it damn near impossible to package libcairo-java at the time.

And made it useless to try writing a program against it, because you
[the developer] had no idea if your it would work when installed on a
users' system...

Pain & suffering.

        [There was a lot of that in the 2.x days. When I inherited
        java-gnome, there were 9+ upstream modules and 6+ downstream
        packages (that were alive). Trying to do a point release just to
        fix an autotools problem and then ship those packages in a
        distro was insanely painful. Took 9 hours, I kid you not (and
        people wonder why I hate automake so much)]

Anyway.

> Putting the topic about including non-pure-gnome dependencies aside,

That actually *is* a significant engineering question. I think Serkan
wants this too. We should talk about it. I'll start a new thread for
that part.

AfC
Sydney

Re: [java-gnome-hackers] Libraries vs Packaging (1): Linkage

[Andrew C. <an...@op...>]

As an aside,

On Wed, 2011-06-01 at 14:19 -0300, Francisco Ortiz wrote:

> - using a factory: its a derivative of the previous one, but cleaner.
> The factory is responsible of provide the correct implementation, it
> will return a class that implements an interface which can be the real
> implementation or the mocked/not supported by the system one. Again ,
> its more code but cleaner.

If we had done an interface based design for 4.0, as I originally
considered, then this would be a snap. But interfaces didn't work out,
and changing our entire public API to interfaces-only at this point
would be ... messy. Like 5.0 messy.
http://java-gnome.sourceforge.net/4.0/doc/design/5a-Architecture.html

I mean, go ahead if you want, but it's about 2 years work. Just thought
I'd let you know. :)

AfC
Sydney

Re: [java-gnome-hackers] Libraries vs Packaging (1): Linkage

[Philipp H. <phi...@gm...>]

Hey there,

I was just wondering, since there is no "best" option in sight ...
What's the harm for you guys to add the --with-module=(..) option? It
doesn't change the default behavior of the library...

Cheers,
Philipp

On Thu, Jun 2, 2011 at 1:53 AM, Andrew Cowie
<an...@op...> wrote:
> As an aside,
>
> On Wed, 2011-06-01 at 14:19 -0300, Francisco Ortiz wrote:
>
>> - using a factory: its a derivative of the previous one, but cleaner.
>> The factory is responsible of provide the correct implementation, it
>> will return a class that implements an interface which can be the real
>> implementation or the mocked/not supported by the system one. Again ,
>> its more code but cleaner.
>
> If we had done an interface based design for 4.0, as I originally
> considered, then this would be a snap. But interfaces didn't work out,
> and changing our entire public API to interfaces-only at this point
> would be ... messy. Like 5.0 messy.
> http://java-gnome.sourceforge.net/4.0/doc/design/5a-Architecture.html
>
> I mean, go ahead if you want, but it's about 2 years work. Just thought
> I'd let you know. :)
>
> AfC
> Sydney
>
>
> ------------------------------------------------------------------------------
> Simplify data backup and recovery for your virtual environment with vRanger.
> Installation's a snap, and flexible recovery options mean your data is safe,
> secure and there when you need it. Data protection magic?
> Nope - It's vRanger. Get your free trial download today.
> http://p.sf.net/sfu/quest-sfdev2dev
> _______________________________________________
> java-gnome-hackers mailing list
> jav...@li...
> https://lists.sourceforge.net/lists/listinfo/java-gnome-hackers
>
>

Re: [java-gnome-hackers] Libraries vs Packaging (1): Linkage

[Francisco O. <fr...@pu...>]

Continuing with the aside
> As an aside,
>
> On Wed, 2011-06-01 at 14:19 -0300, Francisco Ortiz wrote:
>
>> - using a factory: its a derivative of the previous one, but cleaner.
>> The factory is responsible of provide the correct implementation, it
>> will return a class that implements an interface which can be the real
>> implementation or the mocked/not supported by the system one. Again ,
>> its more code but cleaner.
>
> If we had done an interface based design for 4.0, as I originally
> considered, then this would be a snap. But interfaces didn't work out,
> and changing our entire public API to interfaces-only at this point
> would be ... messy. Like 5.0 messy.
> http://java-gnome.sourceforge.net/4.0/doc/design/5a-Architecture.html

Its definitely a 5.0 kind of mess. I remember i read that part one
year ago and you get stuck with the JavaDoc issue. Did you check the
{@inheritDoc} tag?.
Check this article:
http://firstclassthoughts.co.uk/java/javadoc_documentation_inheritance.html
maybe can be useful for that.

> My thinking about this has come from the other side. My guess was that
> the thing to do would be to put an individual loadNativeLibrary() call
> in each package's Plumbing class's static { ... } block, rather than the
> single all-encompassing master one down in
> org.freedesktop.bindings.Plumbing's.
>
> That would mean that the only libMODULENAMEjni.so that get loaded would
> be those corresponding to modules you actually use as a developer, which
> corresponds to the lazy loading of Java classes on the JVM side. Good.
>

My concern is the case where the user tries to use the java class
without the library. Should we create a mocked implementation o throws
the runtime exception?

> It means more .so shared libraries. Lots more. Some people tell me
> that's bad. Other people tell me not to worry about it.
>
> So I was thinking about our Java package space, but a better level would
> presumably be pkg-config space.

I didn't check how bad that is, but I think not *all* the dependencies
should be modularized, only the optional ones.


Anyway need to get my hands in the code to get more answers.

Re: [java-gnome-hackers] Libraries vs Packaging (1): Linkage

[Andrew C. <an...@op...>]

On Wed, 2011-06-01 at 17:58 +0200, Philipp Heckel wrote:

> As you might have seen, I've implemented some changes...

So if you only need this for your direct use to slim the library down
for your situation, then by all means, go for it, and I'm glad its
working out.

> 
> Most importantly: my adjustments do NOT change the default behavior. So
> the distro versions will not be affected by this.
> 

Well, the moment one distro does --enable-something and other distro
does --disable-soemthing in their packages then we have an inconsistent
API for developers.

And (this is the part that you may not be seeing directly) that's the
part we, upstream, get blamed for. A developer doesn't care that the
"fault" is two different distros doing things differently. They just
know that java-gnome doesn't work. That's a serious piss off for them,
and it makes it very difficult for us to support users. Common problem
in Open Source, to be sure, but in our little corner of it we've avoided
it thus far.

Anyway, you get that. I just wanted to be clear that I'm not trying to
make life difficult for you on purpose. We just need to figure out the
experience we want for users of the library, then the design and
engineering that supports that.

>
> I'd appreciate it if you could take a look at it and tell me whether you
> might consider including this in the trunk.
> 

> > - added one directory for each module:

So that's an obvious one, now that you say it. Good call.

My thinking about this has come from the other side. My guess was that
the thing to do would be to put an individual loadNativeLibrary() call
in each package's Plumbing class's static { ... } block, rather than the
single all-encompassing master one down in
org.freedesktop.bindings.Plumbing's.

That would mean that the only libMODULENAMEjni.so that get loaded would
be those corresponding to modules you actually use as a developer, which
corresponds to the lazy loading of Java classes on the JVM side. Good.

It means more .so shared libraries. Lots more. Some people tell me
that's bad. Other people tell me not to worry about it.

So I was thinking about our Java package space, but a better level would
presumably be pkg-config space.

so,

src/defs/gtk-3.0/GtkButton.defs 

sure maybe.

> > - moved the directory "src/bindings" to "src/bindings/core"

This worried me, though. Carrying your example further,

> > + src/bindings/indicate-gtk
> > + src/bindings/appindicator-0.1

and so on. There will be 15 or 20 of these, I'd guess. The catch is that
src/bindings/ is a source package root in Eclipse and in the build
system. It's bad enough right now that we have 

src/gnerator/
src/bindings/
tests/generator/
tests/bindings/
tests/screenshots/
tests/prototype/
doc/examples/

all being source packages. So we would need

src/bindings/MODULENAME/
tests/bindings/MODULENAME/

for each variation, I guess. That's not very tidy. Maybe we just do

src/MODULENAME
tests/MODULENAME

going forward, although that's still very messy for Eclipse. Hm.

Or maybe we don't do that at all, build all the sources regardless, and
deal with it at C compile / link time and/or at Plumbing's static
{...} . That would save smashing the Java side of the build system so
much, at the cost of greater complexity on the C side.

++

We're on the right track, discussing this. It's something we've avoided
until now in this project but it's time to figure it out. Lovely to have
your interest and involvement.

AfC
Sydney


P.S. I just checked out your branch, and discovered
ubuntu-appindicator.patch in the root directory. What is that? And then
worse I discovered that you had .so and .jar files added in your
history. Eeek. You don't version control build products and patches :/ I
see that you discovered your mistake, but we're not going to [ever] be
able to merge this branch because it mean bloating everyone's
repositories with this history. Also please fix `bzr whoami` before you
redo these commit(s).

Please join us in #java-gnome I'm sure someone will be happy to walk you
through using Bazaar. There's a trick with bzr merge and revert that can
compress this all into a single patch which might just be the thing.





-- 
Andrew Frederick Cowie

Operational Dynamics is an operations and engineering consultancy
focusing on IT strategy, organizational architecture, systems
review, and effective procedures for change management: enabling
successful deployment of mission critical information technology in
enterprises, worldwide.

http://www.operationaldynamics.com/

Re: [java-gnome-hackers] Libraries vs Packaging (1): Linkage

[Philipp H. <phi...@gm...>]

Hallo Andrew,


> Well, the moment one distro does --enable-something and other distro
> does --disable-soemthing in their packages then we have an inconsistent
> API for developers.

I believe I might have underestimated the impact of a decision like
this. Having a library that offers different classes depending on the
distro is definitely not a good option.

All the more interesting is the idea Francisco suggested. But if you
say this is 2yrs work this is not an option for 4.0.x.

> My thinking about this has come from the other side. My guess was that
> the thing to do would be to put an individual loadNativeLibrary() call
> in each package's Plumbing class's static { ... } block, rather than the
> single all-encompassing master one down in
> org.freedesktop.bindings.Plumbing's.
>
> That would mean that the only libMODULENAMEjni.so that get loaded would
> be those corresponding to modules you actually use as a developer, which
> corresponds to the lazy loading of Java classes on the JVM side. Good.

I find that so far the best idea, and as far as I see it, it could be
implemented with only little changes to the current code and no (?)
impact on the current API.

Forgive me if I take appindicator as example again, but it's the
easiest to imagine for me. How about something like this. As you said,
in every optional module Plumbing class, do a static block which loads
the library as you said:

static {
   try {
     loadNativeLib();
     available = true;
   } catch (...) {
     available = false;
   }
}

In the constructor of the class, check if available is true. And if it
isn't, throw a ModuleNotAvailableException (extends RuntimeEx..). To
avoid try/catch blocks for the developer, you could additionally offer
a static moduleAvailable() method to check that beforehand. Applying
this to my appindicator example, one could write an application that
uses appindicators (if they are available), and statusicons if they
are not, something like this:

if (Indicator.moduleAvailable()) {
  Indicator i = new Indicator(...);
  ...
}
else {
  StatusIcon i = new StatusIcon(..);
}

> It means more .so shared libraries. Lots more. Some people tell me
> that's bad. Other people tell me not to worry about it.
> [...]
> going forward, although that's still very messy for Eclipse. Hm.

Don't worry about it :-)
By the way, isn't splitting things into modules "separation of
concerns". I don't see how that can be bad. Besides, if it's just
about Eclipse, you could still create a source directory and symlink
the module folders into it (ugglllyy, but hey ...) :-)

How much work would it be to externalize a module in a separate .so?
Is it feasable to do that for 4.0.x?

> P.S. I just checked out your branch, and discovered
> ubuntu-appindicator.patch in the root directory. What is that? And then
> worse I discovered that you had .so and .jar files added in your
> history. Eeek. You don't version control build products and patches :/ I
> see that you discovered your mistake, but we're not going to [ever] be
> able to merge this branch because it mean bloating everyone's
> repositories with this history. Also please fix `bzr whoami` before you
> redo these commit(s).
>
> Please join us in #java-gnome I'm sure someone will be happy to walk you
> through using Bazaar. There's a trick with bzr merge and revert that can
> compress this all into a single patch which might just be the thing.

The ubuntu-appindicator patch is the one Francisco posted in his post
a year ago.
As you might have guessed, I'm rather new to bzr, so forgive me my
kindergarten mistakes :-)

Cheers,
Philipp

Re: [java-gnome-hackers] Libraries vs Packaging (1): Linkage

[Andrew C. <an...@op...>]

On Thu, 2011-06-02 at 15:05 +0200, Philipp Heckel wrote:
> How about something like this. As you said,
> in every optional module Plumbing class, do a static block which loads
> the library as you said:
> 
> static {
>    try {
>      loadNativeLib();
>      available = true;
>    } catch (...) {
>      available = false;
>    }
> }
> 
> In the constructor of the class,

Careful; Plumbing classes [and their subclasses] never get instantiated.
We'd have to manually put in a check somewhere (everywhere?) else.
That's doable, but error prone. :(

In fact, only place we've done it is in Cairo. [And one of these days
I'm going to move the checkStatus() calls down to generated C code.
Better stack traces.]

>  check if available is true. And if it
> isn't, throw a ModuleNotAvailableException (extends RuntimeEx..). 


> To
> avoid try/catch blocks for the developer, you could additionally offer
> a static moduleAvailable() method to check that beforehand.

Yeah, that would be sensible, wouldn't it.

We tend to have a "static class" [sic, my own term] that goes with each
package. org.gnome.glib.Glib, org.gnome.gtk.Gtk, etc. So we could put
moduleAvailable() [or maybe isAvailable()?] there on each of those.

        [Java behaves a bit funny about static methods with the same
        name appearing in subclasses. It doesn't do inheritance quite
        the way that I though, I suspect because the compiler generates
        an invoke static with a given class name, not invoke virtual].


I'd almost be more tempted to suggest that public isModuleAvailable()
work by invoking the code path that leads to the loadNativeLibrary()
call.  After that, if you have the temerity to call something its either
there like it should be or your app is broken like it should be.

Hm.


> By the way, isn't splitting things into modules "separation of
> concerns". I don't see how that can be bad.

It's bloody horrible when someone wants to use your libappindicator-java
only to discover they've got to install 4 other modules.

Java's lack of a dependency system in its shared libraries (which is a
different but parallel problem during development and at deployment
runtime) is horrible here. Hence the one source project (developers) and
[at the moment] one resultant .jar [& .so] (deployment).

Anyway.

>  Besides, if it's just
> about Eclipse, you could still create a source directory and symlink
> the module folders into it (ugglllyy, but hey ...) :-)

Horrid and manual experience for developers. Java is already hard
enough. Eclipse for newbies is insane. Costs us people all the time.
They show up in #java-gnome wanting to "try it" because "they've heard
Eclipse is cool" and an hour later are lost for good.

> As you might have guessed, I'm rather new to bzr, so forgive me my
> kindergarten mistakes :-)

Not at all. Takes time to learn the ins and outs of anything you haven't
worked with before.

AfC
Sydney

Re: [java-gnome-hackers] Libraries vs Packaging (1): Linkage

[Vreixo F. L. <met...@ya...>]

Hi!

I have been reading this thread the last days and I would like to tell you all 
my opinion, as I don't really agree with many of the options discussed so far.

First, I'd think we should distinguish between gnome stuff, that is present in 
most (if not all) gnome-based distros, and other stuff that only some distros 
provides (I'm thinking of things like appindicator).

In the first case (pure gnome stuff) it MAY BE interesting to java a single jar 
file and it MAY BE useful to have a single .so. I will discuss about this later.

For now, let me focus in the second case: libraries that are not part of gnome 
or that they are not installed in most distros.
In this particular case, in my opinion it MAKES NO SENSE to include it in 
java-gnome (I mean in the java-gnome jar), neither always nor conditionally by 
means of a compile time flag.
In the first case (always including that sutff) it would be a pain in the ass 
for users of distros that do not include it.
I don't like to install things that require too many dependencies to stuff I'm 
not going to use, and I think most people don't like it neither. So I think it 
would be a terrible idea to do so.
On the other side, a compile-time flag is also, imho, a bad idea. Distros will 
surely include different flavours of java-gnome, leading to an inconsistent API 
across distros. A pain for developers and a terrible approach, too (in fact, it 
is a terrible solution also in C, although it is very common).
Runtime checks like hasModule(xxx) are also a bad idea imho, as aplications will 
either behave very differently across distros or fail at runtime.
If an application feature is going to be enabled only on some distros, it is 
better to leave that for compile time, not at runtime.

But, does this mean that this no-gnome stuff should not be include in 
java-gnome? Not, at least it shouldn't. 

I my opinion the best approach is to include it with java-gnome sources (maybe 
in an /src/opt folder), but never include then in the main java-gnome jar.
Instead, we should generate different JARs (and JNI so) for each independient 
stuff (for example, java-gnome-appindicator, java-gnome-whatever...).
A compile-time flag should be used to choose whether to build those optional 
jars.
Then distros can package them separately.

In this case, applications that depend on, let's say, appaindicator, should add 
a dependency to the desired jar, and check for the presence of that jar at 
compile time.
This way we keep a common API, but also offer support for non-gnome stuff. We 
should provide some way for developers to know with of our classes require 
additional dependencies, by means of documentation, and maybe naming conventions 
(for example, including them under an org.gnome.optional package?).

So, at this point, let me go back to gnome stuff.
In this case, it is good to have a single jar and single API, but we could 
consider generate separate jars for stuff that is not very commonly used, or 
stuff that is sometimes used independently of other parts of the library (I 
think you had mentioned cairo...).
But all of them should be compiled and distributed always, for the sake of a 
common API across distros, and the only purpose of separation is efficiency, 
memory usage, etc.
Some experiments should be carried out in order to check if there is actually a 
significant impact, anyway.
Another idea is to have a single jar, but different .so, that are loaded only 
when needed.

So, what do you think? Any other idea?

Best regards,
Vreixo