xmlvm-users

[xmlvm-users] Collections Framework Design

[Gergely K. <ger...@ma...>]

Hi,

There are already multiple patches in the review queue for the Collections
Framework. Following my previous suggestion, I am now trying to open a
discussion regarding the design of the collections framework. In our patch (
http://xmlvm-reviews.appspot.com/18002) you can already see a partial
implementation of this suggestion.

Requirements:
1. Need to support seamless integration between regular ObjC types (NSArray
... etc.) and Java classes -> use categories
2. Need to support subclassing for non-final classes. This is a problem with
using class clusters as the basis.

The proposal:
1. Create the interfaces as protocols (java_util_List, java_util_Map ...
etc.)
2. Implement the interfaces as categories of the non-mutable ObjC collection
classes. (NSArray, NSSet, NSDictionary)
- The mutator methods are implemented to throw
UnsupportedOperationException. In essence you receive the same behavior as
if you used Collections.unmodifiable*() methods
3. Only implement the mutator methods in the NSMutable* collection classes.

At this point any ObjC collection can be used as a regular Java collection.
Now we need to implement the actual concrete classes.

However class clusters cannot be subclassed easily.
The solution is to implement the concrete classes using NSProxy. (See how
java_util_Vector is implemented in our patch).
However, this implementation adds an unnecessary indirection and performance
overhead. So the idea is to detect in the proxy class init function whether
a subclass is instantiated.
If it is not a subclass, then simply return the appropriate NSMutable*
instance. Otherwise return the proxy.

What do you think? If you agree with this approach we can clean up our
implementation, and submit it separately.

Best Regards,
Gergely

-- 
Kis Gergely
MattaKis Consulting
Email: ger...@ma...
Web: http://www.mattakis.com
Phone: +36 70 408 1723
Fax: +36 27 998 622

Re: [xmlvm-users] Collections Framework Design

[Paul P. <bay...@gm...>]

Resending to everyone on the distribution list instead of just Gergely:


I am not sure I fully understand the need to use categories, since the only
code that should ever access the Java API in Objective-C is in fact the same
API.  To put it another way, once the Java API is fully implemented in
Obj-C, there would not ever be a case that I would explicitly reference a
Java class in Obj-C.


However, I am happy to conform to this, as there seems to be a consensus
that this is the way to go.


I will be looking further into the details tomorrow, such as how you
mentioned proxies, so please excuse any oversight for the time being.  I
would just like to keep up the conversation.


That said, I am not sure how to have both a category and a subclass.  This
brings up two issues (proxy solves these?):

1) HashMap should be a subclass of java.lang.Object, but I do not know how
to do this using categories.

2) If I need more specific member variables, I cannot add them.


Here's an example that will NOT work:


typedef NSMutableDictionary java_util_HashMap;


@interface NSMutableDictionary (cat_java_util_HashMap) : java_lang_Object {

NSObject* someObject;

}



Above, both extending java_lang_Object and attempting to add a member
variable will cause a compile error.


To further expand upon my concerns, in my proposed HashMap implementation
(in patch 32001), I used member variables, but this isn't possible with
categories to my understanding.  Technically the HashMap should be backed by
an entrySet member variable (it isn't because it's backed by an
NSMutableDictionary).


As XMLVM is a work in progress, I have found that not all of the implemented
API in Obj-C follows the Java contracts exactly, so I don't mind ignoring
this minor blip in HashMap.  As a side note, the primary issue I'm referring
to is classes with synchronization lacking said implementation.  I am trying
to follow the Java API as much as possible.


Gergely,

I assume my patch (32001) is what inspired your e-mail.  Although we're
working in the same area, I don't think we're conflicting in a horrible way
(correct me if you think I am wrong).  E.g. see HashSet in both of our
patches.


Could you elaborate on your comment "Need to support subclassing for
non-final classes"?  It sounds like you're expressing the same issue I am.


I plan on taking a better look at this tomorrow to make sure we're on the
same page.


Per Wolfgang's comments, I will be refactoring HashMap & HashSet to use
categories.  Please feel free to comment on my updated patch when I complete
it.

On Fri, Mar 26, 2010 at 12:58 PM, Gergely Kis <ger...@ma...>wrote:

> Hi,
>
> There are already multiple patches in the review queue for the Collections
> Framework. Following my previous suggestion, I am now trying to open a
> discussion regarding the design of the collections framework. In our patch (
> http://xmlvm-reviews.appspot.com/18002) you can already see a partial
> implementation of this suggestion.
>
> Requirements:
> 1. Need to support seamless integration between regular ObjC types (NSArray
> ... etc.) and Java classes -> use categories
> 2. Need to support subclassing for non-final classes. This is a problem
> with using class clusters as the basis.
>
> The proposal:
> 1. Create the interfaces as protocols (java_util_List, java_util_Map ...
> etc.)
> 2. Implement the interfaces as categories of the non-mutable ObjC
> collection classes. (NSArray, NSSet, NSDictionary)
> - The mutator methods are implemented to throw
> UnsupportedOperationException. In essence you receive the same behavior as
> if you used Collections.unmodifiable*() methods
> 3. Only implement the mutator methods in the NSMutable* collection classes.
>
> At this point any ObjC collection can be used as a regular Java collection.
> Now we need to implement the actual concrete classes.
>
> However class clusters cannot be subclassed easily.
> The solution is to implement the concrete classes using NSProxy. (See how
> java_util_Vector is implemented in our patch).
> However, this implementation adds an unnecessary indirection and
> performance overhead. So the idea is to detect in the proxy class init
> function whether a subclass is instantiated.
> If it is not a subclass, then simply return the appropriate NSMutable*
> instance. Otherwise return the proxy.
>
> What do you think? If you agree with this approach we can clean up our
> implementation, and submit it separately.
>
> Best Regards,
> Gergely
>
> --
> Kis Gergely
> MattaKis Consulting
> Email: ger...@ma...
> Web: http://www.mattakis.com
> Phone: +36 70 408 1723
> Fax: +36 27 998 622
>
>
> ------------------------------------------------------------------------------
> Download Intel&#174; Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> xmlvm-users mailing list
> xml...@li...
> https://lists.sourceforge.net/lists/listinfo/xmlvm-users
>
>

Re: [xmlvm-users] Collections Framework Design

[Arno P. <ar...@pu...>]

Consider class UIView in Cocoa. It has a property 'subviews' that allows 
you access to all children of a given UIView. This list of children is 
returned as NSArray*. By using categories, we can make java_util_List a 
category of NSArray. The huge advantage is that you can simply pass 
NSArray* through to the Java side where it can be transparently accessed 
as java.util.List. If it were anything else, you would have to first 
convert the data structure.

Arno


On 3/26/10 7:40 PM, Paul Poley wrote:
> Resending to everyone on the distribution list instead of just Gergely:
>
>
> I am not sure I fully understand the need to use categories, since the
> only code that should ever access the Java API in Objective-C is in fact
> the same API.  To put it another way, once the Java API is fully
> implemented in Obj-C, there would not ever be a case that I would
> explicitly reference a Java class in Obj-C.
>
>
> However, I am happy to conform to this, as there seems to be a consensus
> that this is the way to go.
>
>
> I will be looking further into the details tomorrow, such as how you
> mentioned proxies, so please excuse any oversight for the time being.  I
> would just like to keep up the conversation.
>
>
> That said, I am not sure how to have both a category and a subclass.
> This brings up two issues (proxy solves these?):
>
> 1) HashMap should be a subclass of java.lang.Object, but I do not know
> how to do this using categories.
>
> 2) If I need more specific member variables, I cannot add them.
>
>
> Here's an example that will NOT work:
>
>
> typedef NSMutableDictionary java_util_HashMap;
>
>
> @interface NSMutableDictionary (cat_java_util_HashMap) : java_lang_Object {
>
> NSObject* someObject;
>
> }
>
>
>
> Above, both extending java_lang_Object and attempting to add a member
> variable will cause a compile error.
>
>
> To further expand upon my concerns, in my proposed HashMap
> implementation (in patch 32001), I used member variables, but this isn't
> possible with categories to my understanding.  Technically the HashMap
> should be backed by an entrySet member variable (it isn't because it's
> backed by an NSMutableDictionary).
>
>
> As XMLVM is a work in progress, I have found that not all of the
> implemented API in Obj-C follows the Java contracts exactly, so I don't
> mind ignoring this minor blip in HashMap.  As a side note, the primary
> issue I'm referring to is classes with synchronization lacking said
> implementation.  I am trying to follow the Java API as much as possible.
>
>
> Gergely,
>
> I assume my patch (32001) is what inspired your e-mail.  Although we're
> working in the same area, I don't think we're conflicting in a horrible
> way (correct me if you think I am wrong).  E.g. see HashSet in both of
> our patches.
>
>
> Could you elaborate on your comment "Need to support subclassing for
> non-final classes"?  It sounds like you're expressing the same issue I am.
>
>
> I plan on taking a better look at this tomorrow to make sure we're on
> the same page.
>
>
> Per Wolfgang's comments, I will be refactoring HashMap & HashSet to use
> categories.  Please feel free to comment on my updated patch when I
> complete it.
>
>
> On Fri, Mar 26, 2010 at 12:58 PM, Gergely Kis <ger...@ma...
> <mailto:ger...@ma...>> wrote:
>
>     Hi,
>
>     There are already multiple patches in the review queue for the
>     Collections Framework. Following my previous suggestion, I am now
>     trying to open a discussion regarding the design of the collections
>     framework. In our patch (http://xmlvm-reviews.appspot.com/18002) you
>     can already see a partial implementation of this suggestion.
>
>     Requirements:
>     1. Need to support seamless integration between regular ObjC types
>     (NSArray ... etc.) and Java classes -> use categories
>     2. Need to support subclassing for non-final classes. This is a
>     problem with using class clusters as the basis.
>
>     The proposal:
>     1. Create the interfaces as protocols (java_util_List, java_util_Map
>     ... etc.)
>     2. Implement the interfaces as categories of the non-mutable ObjC
>     collection classes. (NSArray, NSSet, NSDictionary)
>     - The mutator methods are implemented to throw
>     UnsupportedOperationException. In essence you receive the same
>     behavior as if you used Collections.unmodifiable*() methods
>     3. Only implement the mutator methods in the NSMutable* collection
>     classes.
>
>     At this point any ObjC collection can be used as a regular Java
>     collection.
>     Now we need to implement the actual concrete classes.
>
>     However class clusters cannot be subclassed easily.
>     The solution is to implement the concrete classes using NSProxy.
>     (See how java_util_Vector is implemented in our patch).
>     However, this implementation adds an unnecessary indirection and
>     performance overhead. So the idea is to detect in the proxy class
>     init function whether a subclass is instantiated.
>     If it is not a subclass, then simply return the appropriate
>     NSMutable* instance. Otherwise return the proxy.
>
>     What do you think? If you agree with this approach we can clean up
>     our implementation, and submit it separately.
>
>     Best Regards,
>     Gergely
>
>     --
>     Kis Gergely
>     MattaKis Consulting
>     Email: ger...@ma... <mailto:ger...@ma...>
>     Web: http://www.mattakis.com
>     Phone: +36 70 408 1723
>     Fax: +36 27 998 622
>
>     ------------------------------------------------------------------------------
>     Download Intel&#174; Parallel Studio Eval
>     Try the new software tools for yourself. Speed compiling, find bugs
>     proactively, and fine-tune applications for parallel performance.
>     See why Intel Parallel Studio got high marks during beta.
>     http://p.sf.net/sfu/intel-sw-dev
>     _______________________________________________
>     xmlvm-users mailing list
>     xml...@li...
>     <mailto:xml...@li...>
>     https://lists.sourceforge.net/lists/listinfo/xmlvm-users
>
>
>
>
> ------------------------------------------------------------------------------
> Download Intel&#174; Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
>
>
>
> _______________________________________________
> xmlvm-users mailing list
> xml...@li...
> https://lists.sourceforge.net/lists/listinfo/xmlvm-users

Re: [xmlvm-users] Collections Framework Design

[Gergely K. <ger...@ma...>]

Hi Paul,

2010/3/27 Paul Poley <bay...@gm...>

> Resending to everyone on the distribution list instead of just Gergely:
>
Could someone change the reply-to in the list config to point to the mailing
list?


>
> I am not sure I fully understand the need to use categories, since the only
> code that should ever access the Java API in Objective-C is in fact the same
> API.  To put it another way, once the Java API is fully implemented in
> Obj-C, there would not ever be a case that I would explicitly reference a
> Java class in Obj-C.
>
I am not sure that I understand what you mean. Using categories enables us
to extend existing ObjC collection classes with the Java interface. This
means, that we don't have to reimplement List and Map functionality, we only
have to create a wrapper for the native NS* implementations. Categories are
the easiest way to do that in ObjC.



>
> That said, I am not sure how to have both a category and a subclass.  This
> brings up two issues (proxy solves these?):
>
> 1) HashMap should be a subclass of java.lang.Object, but I do not know how
> to do this using categories.
>
Well, in the ObjC compat lib we try to map as many Java types directly to
ObjC types, as possible. If you look at the java_lang_Object implementation:
it is just a typedef of NSObject, and NSObject is extended with the
necessary methods using the cat_java_lang_Object category.

Since ObjC is a dynamically typed OO environment, where the fact whether a
message (selector) is understood by an object is evaluated at runtime, any
NSObject derivative will behave like a java_lang_Object. The only part which
needs special treatment is the reflection and instanceof handling, so it
will correctly determine the type of such classes.



> 2) If I need more specific member variables, I cannot add them.
>
>
> Here's an example that will NOT work:
>
>
> typedef NSMutableDictionary java_util_HashMap;
>
>
> @interface NSMutableDictionary (cat_java_util_HashMap) : java_lang_Object {
>
> NSObject* someObject;
>
> }
>
>
> Yes, this is an issue with categories. If you look at the patch I
referenced, you will see how we solved this problem with regards to the
synchronized implementation, where we need to store a NSCondition for each
java_lang_Object. A similar method has been proposed by Panayotis.

However, in case of implementing java.util.HashMap, you don't need to use
member variables. In fact using a set of Entries in HashMap to store
elements is an implementation detail. You can either:
1. generate Entries for entrySet() on demand.
2. store the Entries in the underlying NSDictionary.

Probably solution 2 is the best, and makes it easy to support the semantics
of entrySet.

>
> As XMLVM is a work in progress, I have found that not all of the
> implemented API in Obj-C follows the Java contracts exactly, so I don't mind
> ignoring this minor blip in HashMap.  As a side note, the primary issue I'm
> referring to is classes with synchronization lacking said implementation.  I
> am trying to follow the Java API as much as possible.
>
Yes. Currently we implement the Java API on a "on demand" basis.
Unfortunately we also don't have a test suite yet, so there are probably
many nuances that don't follow the standard Java API behavior. I was
thinking about making sure that we can use JUnit, and then start using it
for creating tests. Internally we created some basic tests for our
implementations, but it does not use a framework currently.

I saw your patch in the review queue, but I had the outlined approach
implemented previously, as you can see from the patch. I wanted to bring
this issue up on the mailing list earlier, so it is purely a coincidence
with your patch. :)

Best Regards,
Gergely

-- 
Kis Gergely
MattaKis Consulting
Email: ger...@ma...
Web: http://www.mattakis.com
Phone: +36 70 408 1723
Fax: +36 27 998 622

Re: [xmlvm-users] Collections Framework Design

[Arno P. <ar...@pu...>]

that sounds like a fascinating solution to an old problem! (although I 
have to admit I had to read your post twice to understand it :) )

Gergely: would you mind for the benefit of the people on the mailing 
list to post some code sniplet that shows the general approach? Kind of 
a distilled template to show how you do it? Probably easier than to go 
into your patch.

But I like your proposal and I would like to make that change. I think 
NSProxy might even help us with some other problems in XMLVM (@Wolfgang, 
Panayotis: overloading of UIView.drawRect). Nice idea!

Arno


On 3/26/10 10:58 AM, Gergely Kis wrote:
> Hi,
>
> There are already multiple patches in the review queue for the
> Collections Framework. Following my previous suggestion, I am now trying
> to open a discussion regarding the design of the collections framework.
> In our patch (http://xmlvm-reviews.appspot.com/18002) you can already
> see a partial implementation of this suggestion.
>
> Requirements:
> 1. Need to support seamless integration between regular ObjC types
> (NSArray ... etc.) and Java classes -> use categories
> 2. Need to support subclassing for non-final classes. This is a problem
> with using class clusters as the basis.
>
> The proposal:
> 1. Create the interfaces as protocols (java_util_List, java_util_Map ...
> etc.)
> 2. Implement the interfaces as categories of the non-mutable ObjC
> collection classes. (NSArray, NSSet, NSDictionary)
> - The mutator methods are implemented to throw
> UnsupportedOperationException. In essence you receive the same behavior
> as if you used Collections.unmodifiable*() methods
> 3. Only implement the mutator methods in the NSMutable* collection classes.
>
> At this point any ObjC collection can be used as a regular Java collection.
> Now we need to implement the actual concrete classes.
>
> However class clusters cannot be subclassed easily.
> The solution is to implement the concrete classes using NSProxy. (See
> how java_util_Vector is implemented in our patch).
> However, this implementation adds an unnecessary indirection and
> performance overhead. So the idea is to detect in the proxy class init
> function whether a subclass is instantiated.
> If it is not a subclass, then simply return the appropriate NSMutable*
> instance. Otherwise return the proxy.
>
> What do you think? If you agree with this approach we can clean up our
> implementation, and submit it separately.
>
> Best Regards,
> Gergely
>
> --
> Kis Gergely
> MattaKis Consulting
> Email: ger...@ma... <mailto:ger...@ma...>
> Web: http://www.mattakis.com
> Phone: +36 70 408 1723
> Fax: +36 27 998 622
>
>
>
> ------------------------------------------------------------------------------
> Download Intel&#174; Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
>
>
>
> _______________________________________________
> xmlvm-users mailing list
> xml...@li...
> https://lists.sourceforge.net/lists/listinfo/xmlvm-users

Re: [xmlvm-users] Collections Framework Design

[Panayotis K. <pan...@pa...>]

> But I like your proposal and I would like to make that change. I think
> NSProxy might even help us with some other problems in XMLVM (@Wolfgang,
> Panayotis: overloading of UIView.drawRect). Nice idea!
> 
> Arno


Although the idea is really interesting, I belive we should add this functionality when categories or inheritance is not sufficient enough.
For example, UIView.drawRect is only needed for Android. There we can simply create a new object which overwrites the original behavior of drawRect and use this derived object under Android compatibility library (as I said in a previous post)