You can subscribe to this list here.
2011 |
Jan
|
Feb
|
Mar
(38) |
Apr
(34) |
May
(20) |
Jun
(46) |
Jul
(6) |
Aug
(13) |
Sep
(50) |
Oct
(27) |
Nov
(10) |
Dec
(3) |
---|---|---|---|---|---|---|---|---|---|---|---|---|
2012 |
Jan
(7) |
Feb
(3) |
Mar
(4) |
Apr
|
May
|
Jun
(2) |
Jul
(4) |
Aug
(3) |
Sep
(1) |
Oct
(2) |
Nov
|
Dec
|
2013 |
Jan
(1) |
Feb
|
Mar
(4) |
Apr
|
May
(1) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
2015 |
Jan
|
Feb
(1) |
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
From: Shai <sha...@ya...> - 2011-10-17 19:02:04
|
I don't think I know much on the subject, I've ported VM's but usually treated the GC as a "black box" sort of thing. This past week I did some digging in the code (trying to pinpoint the issue) and that's the source of most of my understanding in Boehm. The issue of a deadlock or problem in a finalizer is probably the mistake of the finalizer author, many VM's fail for this use case. This can pose a problem for cases such as a finalizer trying to close a stream which might call flush() hence causing a delay or even failure. But that use case would probably be problematic with every VM. Looking at it I find the code a bit hard to follow and I'm not exactly sure its properly configured for threading to begin with. E.g. for thread support GC_stop_world() needs to be defined via GC_set_stop_func() (there is such a method in the pthread version but no one is invoking it). Anyway, if the world would be stopped during GC it might actually improve the performance and fix some of the issues you were seeing in the past. Or am I missing something here? I don't think there would be a major difference between invoking directly from the no-order method you are currently using and the method I suggested (it just seemed more appropriate since its designed for Java. I don't know how to determine the GC thread as the finalizer thread but I think setting the finalizer to invoke directly will solve this issue just as well. Thanks. |
From: Arno P. <ar...@pu...> - 2011-10-17 18:29:06
|
right now we use GC_register_finalizer_no_order() because otherwise there would be cycles in objects that have a finalizer (which is common in Java). If you are interested in the details, read the section of topological ordering: http://www.hpl.hp.com/personal/Hans_Boehm/gc/finalization.html The method you suggested (GC_register_finalizer_unreachable()) is an optimization but potentially introduces finalizer cycles again. I don't see a way how the code generating backend can distinguish when to use GC_register_finalizer_unreachable() and when to use GC_register_finalizer_no_order(). Arno On 10/17/11 10:10 AM, Shai wrote: > Thanks, > I don't have a choice about using finallizers since I'm porting an existing API and the complexity of monitoring object lifecycle will make the API considerably more complex (not to mention break existing 3rd party code). > > Finalizers have the drawback of "double cycle" gc, but for this particular case I don't see another option. E.g. they are used within Java for stream closing and component/image peer disposal. Since IO streams have a finalizer and must always be used from a separate thread I'm guessing I'm not the only one who runs into this problem, its probably worse for my case though. > > > The best solution would have beein invoking the finalizers on the GC thread, but since you say the performance is prohibitive (I'm already struggling with performance here) this isn't an option. > > > Java itself doesn't handle the use case of every object having a finalizer well (which isn't my case at all) since it performs finalization instead of collection and only collects in the next cycle. > > I don't quite understand why GC_register_finalizer_unreachable can't be used to register a function that invokes the finalizer directly? > > ------------------------------------------------------------------------------ > All the data continuously generated in your IT infrastructure contains a > definitive record of customers, application performance, security > threats, fraudulent activity and more. Splunk takes this data and makes > sense of it. Business sense. IT sense. Common sense. > http://p.sf.net/sfu/splunk-d2d-oct > _______________________________________________ > Xmlvm-developers mailing list > Xml...@li... > https://lists.sourceforge.net/lists/listinfo/xmlvm-developers |
From: Paul P. <bay...@gm...> - 2011-10-17 18:05:19
|
P.S. I misread it before. I thought you were referring to GC_finalizer_notifier & that's what I meant to type. We currently use GC_REGISTER_FINALIZER_NO_ORDER. Paul On Mon, Oct 17, 2011 at 12:59 PM, Paul Poley <bay...@gm...> wrote: > It sounds like you know what you're talking about, so forgive me if I state > the obvious. > > For our discussion, there are 3 types of threads: > > 1. GC thread - created by Boehm code > 2. Finalizer thread - created by XMLVM > 3. All application threads > > So any cross-compiled Java code besides that invoked from a finalize() will > occur in thread type #3. > > Currently, the actual garbage collection is also done in thread type #3. > Every time memory is allocated, it is a chance for the GC to run in that > same thread. I.e. That thread will first invoke the finalizer notifier and > then run the GC. > > In our case, the finalizer notifier temporarily disables collection & > broadcasts a message to our finalizer thread to begin finalizations in the > thread of type #2. Aside from performance, if the finalize() invocations > were done in the same thread instead of broadcasting to the other thread, > deadlocks can occur. I.e. Consider a synchronized block in a finalize() > method that could be invoked any time you dared use the "new" keyword. > > That to say, if we could move the garbage collection to the same thread as > the finalization (not of thread type #3) so that we didn't have to disable > collection at any point, that would be ideal. Right now I'm at the mercy of > the community for their expertise with the Boehm GC. > > I have only used GC_finalizer_notifier and not > GC_register_finalizer_unreachable so far. Could you explain the difference? > > This is good discussion to have, especially since all iOS instances have > finalize() methods to release the Obj-C counterpart. Thanks, > Paul > > > > On Mon, Oct 17, 2011 at 12:10 PM, Shai <sha...@ya...> wrote: > >> Thanks, >> I don't have a choice about using finallizers since I'm porting an >> existing API and the complexity of monitoring object lifecycle will make the >> API considerably more complex (not to mention break existing 3rd party >> code). >> >> Finalizers have the drawback of "double cycle" gc, but for this particular >> case I don't see another option. E.g. they are used within Java for stream >> closing and component/image peer disposal. Since IO streams have a finalizer >> and must always be used from a separate thread I'm guessing I'm not the only >> one who runs into this problem, its probably worse for my case though. >> >> >> The best solution would have beein invoking the finalizers on the GC >> thread, but since you say the performance is prohibitive (I'm already >> struggling with performance here) this isn't an option. >> >> >> Java itself doesn't handle the use case of every object having a finalizer >> well (which isn't my case at all) since it performs finalization instead of >> collection and only collects in the next cycle. >> >> I don't quite understand why GC_register_finalizer_unreachable can't be >> used to register a function that invokes the finalizer directly? >> >> >> ------------------------------------------------------------------------------ >> All the data continuously generated in your IT infrastructure contains a >> definitive record of customers, application performance, security >> threats, fraudulent activity and more. Splunk takes this data and makes >> sense of it. Business sense. IT sense. Common sense. >> http://p.sf.net/sfu/splunk-d2d-oct >> _______________________________________________ >> Xmlvm-developers mailing list >> Xml...@li... >> https://lists.sourceforge.net/lists/listinfo/xmlvm-developers >> > > |
From: Paul P. <bay...@gm...> - 2011-10-17 17:59:29
|
It sounds like you know what you're talking about, so forgive me if I state the obvious. For our discussion, there are 3 types of threads: 1. GC thread - created by Boehm code 2. Finalizer thread - created by XMLVM 3. All application threads So any cross-compiled Java code besides that invoked from a finalize() will occur in thread type #3. Currently, the actual garbage collection is also done in thread type #3. Every time memory is allocated, it is a chance for the GC to run in that same thread. I.e. That thread will first invoke the finalizer notifier and then run the GC. In our case, the finalizer notifier temporarily disables collection & broadcasts a message to our finalizer thread to begin finalizations in the thread of type #2. Aside from performance, if the finalize() invocations were done in the same thread instead of broadcasting to the other thread, deadlocks can occur. I.e. Consider a synchronized block in a finalize() method that could be invoked any time you dared use the "new" keyword. That to say, if we could move the garbage collection to the same thread as the finalization (not of thread type #3) so that we didn't have to disable collection at any point, that would be ideal. Right now I'm at the mercy of the community for their expertise with the Boehm GC. I have only used GC_finalizer_notifier and not GC_register_finalizer_unreachable so far. Could you explain the difference? This is good discussion to have, especially since all iOS instances have finalize() methods to release the Obj-C counterpart. Thanks, Paul On Mon, Oct 17, 2011 at 12:10 PM, Shai <sha...@ya...> wrote: > Thanks, > I don't have a choice about using finallizers since I'm porting an existing > API and the complexity of monitoring object lifecycle will make the API > considerably more complex (not to mention break existing 3rd party code). > > Finalizers have the drawback of "double cycle" gc, but for this particular > case I don't see another option. E.g. they are used within Java for stream > closing and component/image peer disposal. Since IO streams have a finalizer > and must always be used from a separate thread I'm guessing I'm not the only > one who runs into this problem, its probably worse for my case though. > > > The best solution would have beein invoking the finalizers on the GC > thread, but since you say the performance is prohibitive (I'm already > struggling with performance here) this isn't an option. > > > Java itself doesn't handle the use case of every object having a finalizer > well (which isn't my case at all) since it performs finalization instead of > collection and only collects in the next cycle. > > I don't quite understand why GC_register_finalizer_unreachable can't be > used to register a function that invokes the finalizer directly? > > > ------------------------------------------------------------------------------ > All the data continuously generated in your IT infrastructure contains a > definitive record of customers, application performance, security > threats, fraudulent activity and more. Splunk takes this data and makes > sense of it. Business sense. IT sense. Common sense. > http://p.sf.net/sfu/splunk-d2d-oct > _______________________________________________ > Xmlvm-developers mailing list > Xml...@li... > https://lists.sourceforge.net/lists/listinfo/xmlvm-developers > |
From: Shai <sha...@ya...> - 2011-10-17 17:10:34
|
Thanks, I don't have a choice about using finallizers since I'm porting an existing API and the complexity of monitoring object lifecycle will make the API considerably more complex (not to mention break existing 3rd party code). Finalizers have the drawback of "double cycle" gc, but for this particular case I don't see another option. E.g. they are used within Java for stream closing and component/image peer disposal. Since IO streams have a finalizer and must always be used from a separate thread I'm guessing I'm not the only one who runs into this problem, its probably worse for my case though. The best solution would have beein invoking the finalizers on the GC thread, but since you say the performance is prohibitive (I'm already struggling with performance here) this isn't an option. Java itself doesn't handle the use case of every object having a finalizer well (which isn't my case at all) since it performs finalization instead of collection and only collects in the next cycle. I don't quite understand why GC_register_finalizer_unreachable can't be used to register a function that invokes the finalizer directly? |
From: Paul P. <bay...@gm...> - 2011-10-17 16:29:16
|
Yes, the garbage collector is temporarily disabled by design. Now, it would be wonderful if we could not disable the GC at any point, so if you have some suggestions based on the points below, lets discuss them. I will say that garbage collection can get complicated though. First, I would recommend in general, whether using XMLVM or not, to reduce the number of finalizers to as small as possible. Finalizers cause a lot of overhead & there is no guarantee when they are invoked. That is true in a normal JVM as well. Now to the issue at hand: yes, the GC is multi-threaded (single threading the GC causes other undesirable issues such as deadlocks & performance issues). So when it is determined there are finalizers to run (call this thread 1), the GC is disabled & the separate thread (call this thread 2) is notified to run the finalizers. If the GC were not disabled, thread 1 would then garbage collect the instances likely before thread 2 had a chance to run the finalizers. And you'd get an EXC_BAD_ACCESS error. As it is, thread 2 will reenable the GC once there are NO finalizers left to run. That usually means garbage collection has to wait until the next pass & hope there's not more finalizers temporarily blocking GC. So if every instance had a finalizer, the GC would never get a break to clean up. I agree this is not ideal & would prefer a better solution, but this is what we have for now. We use the Boehm GC & part of this solution is due to our understanding of the Boehm GC. Perhaps there is something available in the Boehm GC to do this better? Thanks, Paul On Mon, Oct 17, 2011 at 11:08 AM, Shai <sha...@ya...> wrote: > Hi Guys, > I've been banging my head against this memory leak issue for the past > couple of weeks trying to resolve it and hopefully you guys can help me > understand it. > > My code which is mostly native iPhone code, uses two threads: main and > logical. > I have quite allot of native objects allocated (large image objects) but I > clear them all up in the finalizer code on the java side (the java object > has a "pointer" and when the finalizer is invoked a native method cleans up > the pointer). > > However, it seems the finalizers are only invoked occasionally. This > effectively causes all the RAM to run out practically immediately and for > some reason xcode's Instruments seems to be completely useless against this > particular leak. > > Debugging the GC it seems to be working as expected and never increases > above 3mb total heap (which makes sense for the app) but the free mem eats > up the 128mb pretty quickly. > > Looking at FinalizerNotifier.java I see that the code uses a finalizerMutex > which disables the GC, however since I have a separate thread which keeps > allocating data and doesn't synchronize against this mutex won't that pose a > problem? > > > Thanks. > Shai. > > > ------------------------------------------------------------------------------ > All the data continuously generated in your IT infrastructure contains a > definitive record of customers, application performance, security > threats, fraudulent activity and more. Splunk takes this data and makes > sense of it. Business sense. IT sense. Common sense. > http://p.sf.net/sfu/splunk-d2d-oct > _______________________________________________ > Xmlvm-developers mailing list > Xml...@li... > https://lists.sourceforge.net/lists/listinfo/xmlvm-developers > > |
From: Shai <sha...@ya...> - 2011-10-17 16:08:10
|
Hi Guys, I've been banging my head against this memory leak issue for the past couple of weeks trying to resolve it and hopefully you guys can help me understand it. My code which is mostly native iPhone code, uses two threads: main and logical. I have quite allot of native objects allocated (large image objects) but I clear them all up in the finalizer code on the java side (the java object has a "pointer" and when the finalizer is invoked a native method cleans up the pointer). However, it seems the finalizers are only invoked occasionally. This effectively causes all the RAM to run out practically immediately and for some reason xcode's Instruments seems to be completely useless against this particular leak. Debugging the GC it seems to be working as expected and never increases above 3mb total heap (which makes sense for the app) but the free mem eats up the 128mb pretty quickly. Looking at FinalizerNotifier.java I see that the code uses a finalizerMutex which disables the GC, however since I have a separate thread which keeps allocating data and doesn't synchronize against this mutex won't that pose a problem? Thanks. Shai. |
From: Panayotis K. <pan...@pa...> - 2011-10-14 08:59:19
|
13 Οκτ 2011, 11:53 π.μ., ο/η Aaron VonderHaar <gru...@gm...> έγραψε: > Do you think it could work simply to use List<SomeType> where > SomeType[] is used in C, and Reference<SomeType> where SomeType* is > used? Even though C compiles them identically, we can still > distinguish the intent in the header files. In most cases, even if the intent is to use tables, the definition is still with * instead. There are only a very few situations where the [] notation is used. So, at the end of the day it is still impossible to guess if it is a reference or a table. I believe that only manual could define what something is (and I'd love to hear your suggestions on how to properly define something like this in the XML file). |
From: Panayotis K. <pan...@pa...> - 2011-10-14 08:54:20
|
13 Οκτ 2011, 12:31 μ.μ., ο/η Aaron VonderHaar <gru...@gm...> έγραψε: > There's the problem though that String.getBytes() returns different > bytes depending on the "the platform's default charset". I'm not > immediately sure what that even refers to in using xmlvm to generate > objc code. Since there is no "default implementation" for Java in iOS, I believe that we can safely assume that the "default charset" is UTF8, and thus solve tons of issues... |
From: Panayotis K. <pan...@pa...> - 2011-10-14 08:52:02
|
-- Panayotis 13 Οκτ 2011, 11:53 π.μ., ο/η Aaron VonderHaar <gru...@gm...> έγραψε: > Do you think it could work simply to use List<SomeType> where > SomeType[] is used in C, and Reference<SomeType> where SomeType* is > used? Even though C compiles them identically, we can still > distinguish the intent in the header files. > > On Mon, Oct 10, 2011 at 4:32 AM, Panayotis Katsaloulis > <pan...@pa...> wrote: >>> ---------------------------------- >>> CGFont.java: >>> /** >>> * bool CGFontGetGlyphBBoxes(CGFontRef font, const CGGlyph glyphs[], >>> size_t count, CGRect bboxes[]) ; >>> */ >>> public boolean getGlyphBBoxes(short[] glyphs, int count, >>> Reference<CGRect> bboxes) >>> >>> The mapping from CGRect[] to Reference<CGRect> at first glance didn't >>> seem correct, but inspecting Reference.java that it is basically a >>> stripped down type of List. Could we use array or ArrayLists instead >>> of making a new container class? If not, could Reference be renamed >>> to make it clear that it represents an array of values rather than a >>> single value. Although I see that both "const SomeType arg[]" and >>> "const SomeType* arg" are translated to "References<SomeType>", since >>> they are functionally identical in C, but are semantically different. >>> (There are many instances of this.) >> >> >> Yes, this is a very serious issue. >> As you said, in terms of C, there is no way to distinguish [] from * >> Not only that, but in many locations the API defines a * for a table, or even a nil terminated list. >> I don't even want to go in details about reference for structs,double reference or even typedefs… >> The whole is a mess and with automatic tools this was the best I could do. >> >> Now, about the Reference object, it is primary a "reference", not an "array". >> It just happens in C that arrays are actually references to the first item in a list. >> References are more common to the iOS API than tables, since NSArray is used usually instead (and is properly mapped to a "List"). >> These references (like NSError **) are used as "call by reference" arguments. >> So it's primary work is to provide a "pointer" and not a "table". >> >> What I actually did to automatically solve this issue, was a "trick", to support both references and arrays. >> The same object has behavior as a reference (with the get() and set() methods, without index) and as an array (the same methods with an index parameter). If you think that an ArrayList will be more appropriate than a fixed array, this is something that could change. >> >> I was also thinking of another solution - make everything as arrays. >> Then the callback variables will be a one-dimension, size-one arrays, but this will look a bit strange at first. >> >> If we want to do something more elegant that this (which I agree), I will be happy to provide an "XML API", in which will specifically define all those methods that need to be fine tuned. >> It won't be an easy job though. |
From: Aaron V. <gru...@gm...> - 2011-10-13 09:31:45
|
There's the problem though that String.getBytes() returns different bytes depending on the "the platform's default charset". I'm not immediately sure what that even refers to in using xmlvm to generate objc code. This gets ugly pretty quickly, but at the moment I don't see a real solution for this. Sticking with byte[] does sound like the best bet for now. Just to add a few more details, here's what the Java code would look like to correctly convert draw some text to a CGContext... myCGContext.selectFont(myFontName.getBytes(Charset.forName("US-ASCII"), 12, UIKit.CGEncodingMacRoman); byte[] bytes = myText.getBytes(Charset.forName("MacRoman")); myCGContext.showText(bytes, bytes.length); The font name is required to be a "postscript name", although it doesn't specifically say what the encoding should be (I'm guessing ASCII here). The text passed to showText needs to be in the encoding specified by the third argument to selectFont. (The only two choices are MacRoman or FontSpecific--not particularly helpful.) The C backend is going to be cross-compiling the Apache Harmony, is the correct? Hopefully that includes an implementation of the MacRoman Charset, which is not required to be implemented by every Java platform. Maybe in practice it won't be that big of a deal, but I'm sure some xmlvm user down the road is going to be thrown for a loop by this. Again, I don't have any other solution to offer, but I think it's interesting to stay aware of this. Cheers, --Aaron V. On Mon, Oct 10, 2011 at 4:32 AM, Panayotis Katsaloulis <pan...@pa...> wrote: >> ---------------------------------- >> CGContext.java: >> /** >> * void CGContextSelectFont(CGContextRef c, const char *name, CGFloat >> size, CGTextEncoding textEncoding) ; >> */ >> public void selectFont(byte[] name, float size, int textEncoding) >> >> I'd really like to see (const char*) parameters changed to String >> instead of byte[], although I wonder if it will be possible to do this >> conversion automatically in the C wrappers in a way that will work in >> all cases. (There are many instances of this.) > > > I understand, and I absolutely disagree :) > The reason is that char* is NOT a String, it is a byte[]. > String is a unicode stream of (Java's) char, while char is 2-bytes variable instead of 1-byte C's char. > In English language there is no real difference, but for every other language this is a serious problem (including my mother tongue). > I wanted with the API to make clear that "char* is not a String - if you want to do something like this, do it with your own responsibility". > > Having said that, it would be indeed nice to have functions that the developer can call and convert between byte[] and String - oh wait, here they are! :) > http://download.oracle.com/javase/1,5.0/docs/api/java/lang/String.html#String%28byte[]%29 > http://download.oracle.com/javase/1,5.0/docs/api/java/lang/String.html#getBytes%28%29 > |
From: Aaron V. <gru...@gm...> - 2011-10-13 08:54:01
|
Do you think it could work simply to use List<SomeType> where SomeType[] is used in C, and Reference<SomeType> where SomeType* is used? Even though C compiles them identically, we can still distinguish the intent in the header files. On Mon, Oct 10, 2011 at 4:32 AM, Panayotis Katsaloulis <pan...@pa...> wrote: >> ---------------------------------- >> CGFont.java: >> /** >> * bool CGFontGetGlyphBBoxes(CGFontRef font, const CGGlyph glyphs[], >> size_t count, CGRect bboxes[]) ; >> */ >> public boolean getGlyphBBoxes(short[] glyphs, int count, >> Reference<CGRect> bboxes) >> >> The mapping from CGRect[] to Reference<CGRect> at first glance didn't >> seem correct, but inspecting Reference.java that it is basically a >> stripped down type of List. Could we use array or ArrayLists instead >> of making a new container class? If not, could Reference be renamed >> to make it clear that it represents an array of values rather than a >> single value. Although I see that both "const SomeType arg[]" and >> "const SomeType* arg" are translated to "References<SomeType>", since >> they are functionally identical in C, but are semantically different. >> (There are many instances of this.) > > > Yes, this is a very serious issue. > As you said, in terms of C, there is no way to distinguish [] from * > Not only that, but in many locations the API defines a * for a table, or even a nil terminated list. > I don't even want to go in details about reference for structs,double reference or even typedefs… > The whole is a mess and with automatic tools this was the best I could do. > > Now, about the Reference object, it is primary a "reference", not an "array". > It just happens in C that arrays are actually references to the first item in a list. > References are more common to the iOS API than tables, since NSArray is used usually instead (and is properly mapped to a "List"). > These references (like NSError **) are used as "call by reference" arguments. > So it's primary work is to provide a "pointer" and not a "table". > > What I actually did to automatically solve this issue, was a "trick", to support both references and arrays. > The same object has behavior as a reference (with the get() and set() methods, without index) and as an array (the same methods with an index parameter). If you think that an ArrayList will be more appropriate than a fixed array, this is something that could change. > > I was also thinking of another solution - make everything as arrays. > Then the callback variables will be a one-dimension, size-one arrays, but this will look a bit strange at first. > > If we want to do something more elegant that this (which I agree), I will be happy to provide an "XML API", in which will specifically define all those methods that need to be fine tuned. > It won't be an easy job though. |
From: Paul P. <bay...@gm...> - 2011-10-10 18:59:07
|
For protocols, I'd recommend we keep them as interfaces. We could optionally add "adapters", which are empty implementations of the interface. That way you could extend the adapter & only add the methods you want, or implement the interface & provide all of them. For char* not being a String, you've got a good point about other languages Panayotis (that it should be byte[] instead - as read in the other email thread). Perhaps when it is always appropriate to be a String, we can add details to the "advisor", so that it will be a String instead of byte[] in certain instances? That may not be worth it though. Paul On Mon, Oct 10, 2011 at 6:39 AM, Panayotis Katsaloulis < pan...@pa...> wrote: > > On 10 Οκτ 2011, at 7:25 π.μ., Arno Puder wrote: > > > > > first of all: very nice work! I like the JavaDoc that makes it easy to > > correlate the Objective-C API with the Java counterpart. > > > > A few things: > > > > (1) I would like to move everything to the namespace org.xmlvm.ios.*. I > > wanted to do that for a long time but with the new API existing apps > > need to be changed anyways. > > I wanted to do that too, but I didn't want to touch this subject :) > > > > > > (2) All classes should have the @XMLVMSkeletonOnly annotation. Perhaps > > add a command line argument that triggers the generation of this > annotation. > > No problem > > > > (3) I wonder if we should address the question of optional methods in > > Objective-C protocols again. A long time ago we had a longer discussion > > about this topic. Right now you map a protocol to a Java interface, > > making all its methods mandatory. > > I am all ears. > Do you think it would be better to produce XML files which could be parsed > and create the actual API? > Thus you can post-process whatever you want. > The procedure though will be much more complicated than what it is now. > > > > (4) The parser tool should use more "meta knowledge" to generate better > > Java API. This should be added to advisor.xml. A couple of examples: > > > > - UIView.getSubviews() should return List<UIView>, not just List > > There is already an API for that in advisor.xml > In the reports there is also a list of methods that reflect this problem. > Well, someone has to write it. > > My idea is to have an internal "advisor.xml" which will deal with all basic > non-apple related stuff, and have an external XML at runtime which will live > in XMLVM repository and deal with all specific issues. > Of course, this means that all these details should be handled after the > tool has been committed to the XMLVM repository. > > > > > - As Aaron pointed out, const char* should be mapped to > > java.lang.String where appropriate. > > I disagree here, and I explained the reasons in the other email. > > > > - UIImage.writeToSavedPhotosAlbum(): the completionTarget and > > completionSelector should be combined to a Runnable. > > or use NSSelector instead? > This is a topic that I didn't want to touch yet, until we have a clear > agreement. > > > > ------------------------------------------------------------------------------ > All the data continuously generated in your IT infrastructure contains a > definitive record of customers, application performance, security > threats, fraudulent activity and more. Splunk takes this data and makes > sense of it. Business sense. IT sense. Common sense. > http://p.sf.net/sfu/splunk-d2dcopy1 > _______________________________________________ > Xmlvm-developers mailing list > Xml...@li... > https://lists.sourceforge.net/lists/listinfo/xmlvm-developers > |
From: Panayotis K. <pan...@pa...> - 2011-10-10 11:40:09
|
On 10 Οκτ 2011, at 7:25 π.μ., Arno Puder wrote: > > first of all: very nice work! I like the JavaDoc that makes it easy to > correlate the Objective-C API with the Java counterpart. > > A few things: > > (1) I would like to move everything to the namespace org.xmlvm.ios.*. I > wanted to do that for a long time but with the new API existing apps > need to be changed anyways. I wanted to do that too, but I didn't want to touch this subject :) > > (2) All classes should have the @XMLVMSkeletonOnly annotation. Perhaps > add a command line argument that triggers the generation of this annotation. No problem > (3) I wonder if we should address the question of optional methods in > Objective-C protocols again. A long time ago we had a longer discussion > about this topic. Right now you map a protocol to a Java interface, > making all its methods mandatory. I am all ears. Do you think it would be better to produce XML files which could be parsed and create the actual API? Thus you can post-process whatever you want. The procedure though will be much more complicated than what it is now. > (4) The parser tool should use more "meta knowledge" to generate better > Java API. This should be added to advisor.xml. A couple of examples: > > - UIView.getSubviews() should return List<UIView>, not just List There is already an API for that in advisor.xml In the reports there is also a list of methods that reflect this problem. Well, someone has to write it. My idea is to have an internal "advisor.xml" which will deal with all basic non-apple related stuff, and have an external XML at runtime which will live in XMLVM repository and deal with all specific issues. Of course, this means that all these details should be handled after the tool has been committed to the XMLVM repository. > - As Aaron pointed out, const char* should be mapped to > java.lang.String where appropriate. I disagree here, and I explained the reasons in the other email. > - UIImage.writeToSavedPhotosAlbum(): the completionTarget and > completionSelector should be combined to a Runnable. or use NSSelector instead? This is a topic that I didn't want to touch yet, until we have a clear agreement. |
From: Panayotis K. <pan...@pa...> - 2011-10-10 11:33:04
|
On 10 Οκτ 2011, at 4:37 π.μ., Aaron VonderHaar wrote: > Wow, that's brilliant work! This is a great direction to be moving in. > > I took a quick look and had a few comments: > > ---------------------------------- > UIColor.java: > /** > * - (UIColor *)initWithHue:(CGFloat)hue > saturation:(CGFloat)saturation brightness:(CGFloat)brightness > alpha:(CGFloat)alpha; > * - (UIColor *)initWithRed:(CGFloat)red green:(CGFloat)green > blue:(CGFloat)blue alpha:(CGFloat)alpha; > */ > public UIColor(float arg1, float arg2, float arg3, float arg4, > UIColor.Colorspace colorspace) {} > > This looks a bit strange to me... it's a ctor that maps to two objc > methods. So apparently the inner enum colorspace is automatically > generated just for the purpose of distinguishing these constructors? > > I think it would make more sense to have the generated colorspace > parameter be the first parameter rather than the last. Exactly. Since we do not have something like "name override" in Java constructors, we need to distinguish them somehow. As you said, these enums are created semi-automatically, with external help (to provide the actual enum type). If you can think of a better approach, I am all ears! Of course, having the enum in the beginning is very easy to do, as long as we all agree to that. > ---------------------------------- > CGFont.java: > /** > * bool CGFontGetGlyphBBoxes(CGFontRef font, const CGGlyph glyphs[], > size_t count, CGRect bboxes[]) ; > */ > public boolean getGlyphBBoxes(short[] glyphs, int count, > Reference<CGRect> bboxes) > > The mapping from CGRect[] to Reference<CGRect> at first glance didn't > seem correct, but inspecting Reference.java that it is basically a > stripped down type of List. Could we use array or ArrayLists instead > of making a new container class? If not, could Reference be renamed > to make it clear that it represents an array of values rather than a > single value. Although I see that both "const SomeType arg[]" and > "const SomeType* arg" are translated to "References<SomeType>", since > they are functionally identical in C, but are semantically different. > (There are many instances of this.) Yes, this is a very serious issue. As you said, in terms of C, there is no way to distinguish [] from * Not only that, but in many locations the API defines a * for a table, or even a nil terminated list. I don't even want to go in details about reference for structs,double reference or even typedefs… The whole is a mess and with automatic tools this was the best I could do. Now, about the Reference object, it is primary a "reference", not an "array". It just happens in C that arrays are actually references to the first item in a list. References are more common to the iOS API than tables, since NSArray is used usually instead (and is properly mapped to a "List"). These references (like NSError **) are used as "call by reference" arguments. So it's primary work is to provide a "pointer" and not a "table". What I actually did to automatically solve this issue, was a "trick", to support both references and arrays. The same object has behavior as a reference (with the get() and set() methods, without index) and as an array (the same methods with an index parameter). If you think that an ArrayList will be more appropriate than a fixed array, this is something that could change. I was also thinking of another solution - make everything as arrays. Then the callback variables will be a one-dimension, size-one arrays, but this will look a bit strange at first. If we want to do something more elegant that this (which I agree), I will be happy to provide an "XML API", in which will specifically define all those methods that need to be fine tuned. It won't be an easy job though. > ---------------------------------- > Constant values are missing, for example > CGContext.java: > /** > * void CGContextSetLineJoin(CGContextRef c, CGLineJoin join) ; > */ > public void setLineJoin(int join) > > The CGLineJoin constants (kCGLineJoinMiter, kCGLineJoinRound, > kCGLineJoinBevel) are not defined anywhere. These are defined in > CGContext.h as enum CGLineJoin. I don't know what all the > implications are, but if this could be made into a Java enum instead > of converting it to int, I think that would be great. (There are many > other such enums.) Yes you are right, they are not in the output, since it is still under discussion (not only enums but also external typedefs to constant values). The first thought (I though I said it myself to this list) was exactly what you said. Create Java enums that will do exactly that. The problem is that, many enums can be combined between them and represent different states. So they need to be left as int. In the sake of universality, I prefer to keep them as int, which will be much easier for the C backend too. > ---------------------------------- > CGContext.java: > /** > * void CGContextSelectFont(CGContextRef c, const char *name, CGFloat > size, CGTextEncoding textEncoding) ; > */ > public void selectFont(byte[] name, float size, int textEncoding) > > I'd really like to see (const char*) parameters changed to String > instead of byte[], although I wonder if it will be possible to do this > conversion automatically in the C wrappers in a way that will work in > all cases. (There are many instances of this.) I understand, and I absolutely disagree :) The reason is that char* is NOT a String, it is a byte[]. String is a unicode stream of (Java's) char, while char is 2-bytes variable instead of 1-byte C's char. In English language there is no real difference, but for every other language this is a serious problem (including my mother tongue). I wanted with the API to make clear that "char* is not a String - if you want to do something like this, do it with your own responsibility". Having said that, it would be indeed nice to have functions that the developer can call and convert between byte[] and String - oh wait, here they are! :) http://download.oracle.com/javase/1,5.0/docs/api/java/lang/String.html#String%28byte[]%29 http://download.oracle.com/javase/1,5.0/docs/api/java/lang/String.html#getBytes%28%29 > ---------------------------------- > UIKit.java: > /** > * CGContextRef UIGraphicsGetCurrentContext(void); > */ > public static CGContext UIGraphicsGetCurrentContext() > > I was expecting this to be UIGraphics.getCurrentContext(), rather than > UIKit.UIGraphicsGetCurrentContext(). But this was discussed > previously on the list, and the size of the UIKit grouping seems good > (not too large). So for this I would just want to make sure there is > some clear documentation about where to find methods like this. Yes, the idea is "if an object exists, put it there, if not, put it in the framework object:. Thank you for the constructive comments! |
From: Arno P. <ar...@pu...> - 2011-10-10 04:25:43
|
first of all: very nice work! I like the JavaDoc that makes it easy to correlate the Objective-C API with the Java counterpart. A few things: (1) I would like to move everything to the namespace org.xmlvm.ios.*. I wanted to do that for a long time but with the new API existing apps need to be changed anyways. (2) All classes should have the @XMLVMSkeletonOnly annotation. Perhaps add a command line argument that triggers the generation of this annotation. (3) I wonder if we should address the question of optional methods in Objective-C protocols again. A long time ago we had a longer discussion about this topic. Right now you map a protocol to a Java interface, making all its methods mandatory. (4) The parser tool should use more "meta knowledge" to generate better Java API. This should be added to advisor.xml. A couple of examples: - UIView.getSubviews() should return List<UIView>, not just List - As Aaron pointed out, const char* should be mapped to java.lang.String where appropriate. - UIImage.writeToSavedPhotosAlbum(): the completionTarget and completionSelector should be combined to a Runnable. Arno |
From: Aaron V. <gru...@gm...> - 2011-10-10 01:37:12
|
Wow, that's brilliant work! This is a great direction to be moving in. I took a quick look and had a few comments: ---------------------------------- UIColor.java: /** * - (UIColor *)initWithHue:(CGFloat)hue saturation:(CGFloat)saturation brightness:(CGFloat)brightness alpha:(CGFloat)alpha; * - (UIColor *)initWithRed:(CGFloat)red green:(CGFloat)green blue:(CGFloat)blue alpha:(CGFloat)alpha; */ public UIColor(float arg1, float arg2, float arg3, float arg4, UIColor.Colorspace colorspace) {} This looks a bit strange to me... it's a ctor that maps to two objc methods. So apparently the inner enum colorspace is automatically generated just for the purpose of distinguishing these constructors? I think it would make more sense to have the generated colorspace parameter be the first parameter rather than the last. ---------------------------------- CGFont.java: /** * bool CGFontGetGlyphBBoxes(CGFontRef font, const CGGlyph glyphs[], size_t count, CGRect bboxes[]) ; */ public boolean getGlyphBBoxes(short[] glyphs, int count, Reference<CGRect> bboxes) The mapping from CGRect[] to Reference<CGRect> at first glance didn't seem correct, but inspecting Reference.java that it is basically a stripped down type of List. Could we use array or ArrayLists instead of making a new container class? If not, could Reference be renamed to make it clear that it represents an array of values rather than a single value. Although I see that both "const SomeType arg[]" and "const SomeType* arg" are translated to "References<SomeType>", since they are functionally identical in C, but are semantically different. (There are many instances of this.) ---------------------------------- Constant values are missing, for example CGContext.java: /** * void CGContextSetLineJoin(CGContextRef c, CGLineJoin join) ; */ public void setLineJoin(int join) The CGLineJoin constants (kCGLineJoinMiter, kCGLineJoinRound, kCGLineJoinBevel) are not defined anywhere. These are defined in CGContext.h as enum CGLineJoin. I don't know what all the implications are, but if this could be made into a Java enum instead of converting it to int, I think that would be great. (There are many other such enums.) ---------------------------------- CGContext.java: /** * void CGContextSelectFont(CGContextRef c, const char *name, CGFloat size, CGTextEncoding textEncoding) ; */ public void selectFont(byte[] name, float size, int textEncoding) I'd really like to see (const char*) parameters changed to String instead of byte[], although I wonder if it will be possible to do this conversion automatically in the C wrappers in a way that will work in all cases. (There are many instances of this.) ---------------------------------- UIKit.java: /** * CGContextRef UIGraphicsGetCurrentContext(void); */ public static CGContext UIGraphicsGetCurrentContext() I was expecting this to be UIGraphics.getCurrentContext(), rather than UIKit.UIGraphicsGetCurrentContext(). But this was discussed previously on the list, and the size of the UIKit grouping seems good (not too large). So for this I would just want to make sure there is some clear documentation about where to find methods like this. Cheers, --Aaron V. On Sun, Oct 9, 2011 at 4:33 PM, Arno Puder <ar...@pu...> wrote: > > Guys, > > as some of you already know, we are working on the automatic generation > of the Java iOS API. So far, the generation of the API is a manual > process. Needless to say that this process is sometimes inconsistent, > error prone and does not scale. Currently XMLVM only supports a fraction > of the complete iOS API. > > Generating the Java iOS API will be a quantum leap for XMLVM. Once we > accomplish this, XMLVM will support 100% of the iOS API. There are two > steps to this process: (1) generating the Java API and (2) generating > the matching C wrappers. Panayotis has just completed the first version > of a tool for (1). Those of you who have knowledge of the iOS API - both > Java and Objective-C - are welcomed to take a look at the generated API. > You can do so with the following instructions: > > git clone https://code.google.com/p/crossmobile/ > cd crossmobile/src/xmbuild > ant skeleton > > Make sure that iOS SDK 4.3 is installed. Under the folder > build/skeleton/src you can find the produced Java files. Please take a > look at the generated Java files and send comments/questions/suggestions > to this mailing list. > > Once we have settled on the Java API, we will work on part (2): > automatically generating the C wrappers. > > Thanks to Panayotis for writing a parser tool to generate the Java API. > > Arno > > > ------------------------------------------------------------------------------ > All of the data generated in your IT infrastructure is seriously valuable. > Why? It contains a definitive record of application performance, security > threats, fraudulent activity, and more. Splunk takes this data and makes > sense of it. IT sense. And common sense. > http://p.sf.net/sfu/splunk-d2dcopy2 > _______________________________________________ > Xmlvm-developers mailing list > Xml...@li... > https://lists.sourceforge.net/lists/listinfo/xmlvm-developers > |
From: Arno P. <ar...@pu...> - 2011-10-09 23:33:18
|
Guys, as some of you already know, we are working on the automatic generation of the Java iOS API. So far, the generation of the API is a manual process. Needless to say that this process is sometimes inconsistent, error prone and does not scale. Currently XMLVM only supports a fraction of the complete iOS API. Generating the Java iOS API will be a quantum leap for XMLVM. Once we accomplish this, XMLVM will support 100% of the iOS API. There are two steps to this process: (1) generating the Java API and (2) generating the matching C wrappers. Panayotis has just completed the first version of a tool for (1). Those of you who have knowledge of the iOS API - both Java and Objective-C - are welcomed to take a look at the generated API. You can do so with the following instructions: git clone https://code.google.com/p/crossmobile/ cd crossmobile/src/xmbuild ant skeleton Make sure that iOS SDK 4.3 is installed. Under the folder build/skeleton/src you can find the produced Java files. Please take a look at the generated Java files and send comments/questions/suggestions to this mailing list. Once we have settled on the Java API, we will work on part (2): automatically generating the C wrappers. Thanks to Panayotis for writing a parser tool to generate the Java API. Arno |
From: Panayotis K. <pan...@pa...> - 2011-10-09 18:07:36
|
There are a bunch of C objects, like the ones mentioned in the title, which are (quote) “toll-free bridged” with its Cocoa Foundation counterpart (end quote). C-type functions use this naming convention, while ObjC use the NS* naming. Practically these are the same but this is not reflected at all in the *.h files. Because of their dual nature, parsing is very hard and the strong typed nature of Java, will make things worse. In order to address this issue, I think the easiest (and hopefully safe) method is to "rename" every mention to CF* object to NS* object. What do you think? |
From: Arno P. <ar...@pu...> - 2011-10-09 00:15:12
|
I'm in favor of the first option (create a framework object). Arno On 10/8/11 3:39 AM, Panayotis Katsaloulis wrote: > In the past you asked me if I will deal with C functions as well. > > I am exactly at this position. > >> From what I see there are 2582 functions (which need to be included in objects :) ) > > 1714 are easy, since it exists an object which has the same name as the beginning of the function. > > The problem is with the remaining 868 functions - what would be the easiest way to group them and make this easy for the developer to find the function he wants. > > Let's take two simple examples > > 1) the functionNSUserName which returns the user name. > This function doesn't match to ANY known object up to now. > It partially matches though the "NSUserDefaults" object. > > 2) A "cluster" of functions, such as > CFNetworkExecuteProxyAutoConfigurationURL > CFNetworkExecuteProxyAutoConfigurationScript > CFNetworkCopyProxiesForURL > CFNetworkCopySystemProxySettings > with (or without in another cases) matching object > > > > So, here is what we can do for these unmatched cases: > > 1) Create a "framework object" and put everything there > (like what we did with NSSearchPathForDirectoriesInDomains function) > > 2) Aggressively match a function to an object. Only the remaining functions will be put in the framework object. > > 3) Create a new "cluster" object (e.g. in the second case described above) which will host these functions. > We can either be aggressive there (i.e. create a CFNetwork object) or reluctant (i.e. create an "CFNetworkExecuteProxyAutoConfiguration" and "CFNetworkCopy" object). > > > The second solution I believe will confuse developers. He'll have to search for a specific function instead of knowing where to look at, if he already knows the name. > > The third solution is problematic, since it would be difficult every time to calculate the exact name. E.g. the functions starting with "CFPlugIn" could not provide a good aggressive name, while being reluctant will produce many small almost-empty objects. > Even if we provide externally the information on how to break the names apart, it still would be difficult for the developer because he won't be inside our mind, when we created this partition scheme. > > For these reasons I am more in favor of the first solution, since it would be clear and obvious to the developer, that if a function does not start with a well defined object (or struct) then it will "live" in the framework object. > > > Next step will also be mapping enum's and where to put them. > Most (all?) of the enumerations correspond to specific objects, although sometimes (for space reasons?) the name of some of them are truncated a bit. > For this reason I am more in favor of the second approach. > > So, what do you think? > ------------------------------------------------------------------------------ > All of the data generated in your IT infrastructure is seriously valuable. > Why? It contains a definitive record of application performance, security > threats, fraudulent activity, and more. Splunk takes this data and makes > sense of it. IT sense. And common sense. > http://p.sf.net/sfu/splunk-d2dcopy2 > _______________________________________________ > Xmlvm-developers mailing list > Xml...@li... > https://lists.sourceforge.net/lists/listinfo/xmlvm-developers |
From: Panayotis K. <pan...@pa...> - 2011-10-08 10:40:00
|
In the past you asked me if I will deal with C functions as well. I am exactly at this position. From what I see there are 2582 functions (which need to be included in objects :) ) 1714 are easy, since it exists an object which has the same name as the beginning of the function. The problem is with the remaining 868 functions - what would be the easiest way to group them and make this easy for the developer to find the function he wants. Let's take two simple examples 1) the functionNSUserName which returns the user name. This function doesn't match to ANY known object up to now. It partially matches though the "NSUserDefaults" object. 2) A "cluster" of functions, such as CFNetworkExecuteProxyAutoConfigurationURL CFNetworkExecuteProxyAutoConfigurationScript CFNetworkCopyProxiesForURL CFNetworkCopySystemProxySettings with (or without in another cases) matching object So, here is what we can do for these unmatched cases: 1) Create a "framework object" and put everything there (like what we did with NSSearchPathForDirectoriesInDomains function) 2) Aggressively match a function to an object. Only the remaining functions will be put in the framework object. 3) Create a new "cluster" object (e.g. in the second case described above) which will host these functions. We can either be aggressive there (i.e. create a CFNetwork object) or reluctant (i.e. create an "CFNetworkExecuteProxyAutoConfiguration" and "CFNetworkCopy" object). The second solution I believe will confuse developers. He'll have to search for a specific function instead of knowing where to look at, if he already knows the name. The third solution is problematic, since it would be difficult every time to calculate the exact name. E.g. the functions starting with "CFPlugIn" could not provide a good aggressive name, while being reluctant will produce many small almost-empty objects. Even if we provide externally the information on how to break the names apart, it still would be difficult for the developer because he won't be inside our mind, when we created this partition scheme. For these reasons I am more in favor of the first solution, since it would be clear and obvious to the developer, that if a function does not start with a well defined object (or struct) then it will "live" in the framework object. Next step will also be mapping enum's and where to put them. Most (all?) of the enumerations correspond to specific objects, although sometimes (for space reasons?) the name of some of them are truncated a bit. For this reason I am more in favor of the second approach. So, what do you think? |
From: Arno P. <ar...@pu...> - 2011-10-03 14:41:59
|
Probably a Java class with all fields plus an explicit discriminator field. Arno On Oct 3, 2011, at 2:18 AM, Panayotis Katsaloulis <pan...@pa...> wrote: > How do you think we should properly map a union in the Java API; > > ------------------------------------------------------------------------------ > All the data continuously generated in your IT infrastructure contains a > definitive record of customers, application performance, security > threats, fraudulent activity and more. Splunk takes this data and makes > sense of it. Business sense. IT sense. Common sense. > http://p.sf.net/sfu/splunk-d2dcopy1 > _______________________________________________ > Xmlvm-developers mailing list > Xml...@li... > https://lists.sourceforge.net/lists/listinfo/xmlvm-developers |
From: Panayotis K. <pan...@pa...> - 2011-10-03 09:18:56
|
How do you think we should properly map a union in the Java API; |
From: Shai <sha...@ya...> - 2011-09-26 10:15:36
|
Minor but rather annoying, the seconds came out as ints which resulted in negative numbers when multiplied (see fix bellow). JAVA_LONG java_lang_System_currentTimeMillis__() { //XMLVM_BEGIN_NATIVE[java_lang_System_currentTimeMillis__] struct timeval now; gettimeofday(&now, NULL); JAVA_LONG msec = ((JAVA_LONG)now.tv_sec) * 1000; msec += now.tv_usec / 1000; return msec; //XMLVM_END_NATIVE } |
From: Paul P. <bay...@gm...> - 2011-09-23 00:16:54
|
Big news indeed! Congratulations! Paul On Thu, Sep 22, 2011 at 1:48 PM, Arno Puder <ar...@pu...> wrote: > > Guys, > > some of you may know that XMLVM features a little showcase application: > Xokoban (sources are in xmlvm/demo/android/xokoban). We've had Xokoban > in the Apple AppStore for over two years and Apple has just accepted an > updated version. The visual has not changed much but the internal > implementation has radically changed. For one, Xokoban now makes use of > advanced Android API (activity lifecycle, layout manager, > internationalization). One benefit of using Android's layout manager is > that Xokoban now is a universal binary that runs on iPhone as well as iPad. > > However the biggest change is that the new version of Xokoban was > cross-compiled with the new C backend. This is a radical departure from > the old Objective-C backend. One of the many changes is that > applications cross-compiled with the C backend now bundle a garbage > collector for better Java compliance. Xokoban is (to my knowledge) the > first app that Apple accepted and that was generated with the new C > backend. An important milestone. > > The new version of Xokoban is the combined effort of many people. If you > are on this mailing list you will know who they are. Many thanks to all > of them! > > New big changes are in the pipeline. Look forward to source-code-level > debugging from Eclipse/Netbeans of cross-compiled Java applications. :) > > Arno > > http://itunes.apple.com/us/app/xokoban/id322302746?ls=1&mt=8 > > > > ------------------------------------------------------------------------------ > All the data continuously generated in your IT infrastructure contains a > definitive record of customers, application performance, security > threats, fraudulent activity and more. Splunk takes this data and makes > sense of it. Business sense. IT sense. Common sense. > http://p.sf.net/sfu/splunk-d2dcopy1 > _______________________________________________ > Xmlvm-developers mailing list > Xml...@li... > https://lists.sourceforge.net/lists/listinfo/xmlvm-developers > |