Flexjson

I'm trying to deserialize a Map<String, Foo>. With

new JSONDeserializer() .use (null, HashMap.class) .deserialize ("{\"1\": {\"class\":\"Foo\"}}")

I expect to get

{1=Foo@}

but I get

{1={class=Foo}}

Any suggestions?

(Most idiomatic for me would've been to simply use new JSONDeserializer() .deserialize ("{\"1\": {\"class\":\"Foo\"}}"), assuming that any "{}" without a "class" is a Map, but that gives me the Missing classname for path [ ]. error)

I solved it for now by using a ClassLocator, see http://www.gulic.org/pastebin/77 - however it only works with maps and breaks with arrays.

So the idiomatic solution you proposed doesn't tell the deserializer what classes it's binding into. Given only that information there's no way to actually figure out the types explicit types to bind into (.e.g. Map<String,Foo>). Using the use() method and classLocators help facilitate that. It just as easily could be a Map<String,Map>. The rule of thumb of using Map whenever nothing is specified for a path is a good one, but you'd still need to specify Foo either in a use() or ClassLocators.

The difficultly comes in when the top level object you're dealing with is a generic collection. Part of the problem is the path syntax flexjson uses it can distinguish between the actual collection vs the types contained within. So hopefully I can come up with a better syntax for specifying not only the Map, but also the key's and value's type. Whatever the solution it must be valid for any generic. For now the ClassLocator is the work around.

Couple of things to note if you can give flexjson a concrete object type in a use() clause that it can figure out every other type's through reflection unless it's referencing an interface, abstract class, or some other polymorphic relationship. At that point you have to use ClassLocators to differentiate between all the possible types that could be used.

Charlie

Charlie, have you missed the two problems I mention? 1) There is {\"class\":\"Foo\"} in the JSON string, however, I see no way to deserialize it into Foo using default Deserializer. 2) ClassLocator have no way to differentiate maps from arrays.

> So the idiomatic solution you proposed > doesn't tell the deserializer what > classes it's binding into.

Wrong. 1) Deserializer can see a "class" field and use it to produce that class instead of a Map. 2) Deserializer can use path-specific hints from "use". The idea is to use Map and List by default, that doesn't harm neither existing code nor future deserialization of specific types in any way.

> The rule of thumb of using Map > whenever nothing is specified for a > path is a good one, but you'd still > need to specify Foo either in a use() > or ClassLocators.

That's right. And also using the "class" field emitted by the Serializer.

Oh I see now that there was a class option in there. I missed that earlier. Yes it should realize the class option is there and use that. There are several unit tests focusing on that and they all pass. I'm not sure why this particular case you outlined didn't work. I'll check by tests and see if I can reproduce your problem.

Thanks Charlie

Ok this has been fixed in 1.9. Please downoad and try it out and see if that works for you.

Thanks! I've tried the deserialization in the new version and it now works as expected!

I'm still using the ClassLocator in my real code though, because I'm not sure about the class-loaded issues (I use Thread.currentThread.getContextClassLoader to obtain the ClassLoader, and I suspect Flexjson uses the default ClassLoader which usually doesn't work for Servlet Containers) and using it allows me to do some space-conserving shortcuts as well.

What do you think about my second issue: a theoretical problem that ClassLocator.locate method doesn't have a way to differentiate between objects {} and arrays []?

I'm not sure I understand what's exactly failing between objects and arrays. The prior version of the deserializer didn't work very well with arrays, but that's been fixed in 1.9. However, I don't think that's exactly the same problem as your describing. Can you give me a more concrete example of where your solutions fails to work with arrays?

As for the class loading issues that probably is true. I suppose I could refactor to use Thread.currentThread().getContextClassLoader(). I'm using flexjson in servlet containers without issue. Although I will admit I don't do a lot of hot deployment.

Charlie

The problem is handling arrays with ClassLocator. Here is an example ClassLocator:

def cl = new flexjson.ClassLocator {
  val classLoader = Thread.currentThread.getContextClassLoader
  override def locate (map: java.util.Map[_,_], path: flexjson.Path): Class[_] = {
    val clazz = map.get ("class") // Flexjson class signature.
    if (clazz.isInstanceOf[String]) classLoader.loadClass (clazz.toString)
    else classOf[java.util.HashMap[_, _]]
  }
}

Theoretical problem: I see no way for the ClassLocator to detect arrays. It receives a Map containing the data, but how would it see whether it is an array data or an object data?

Practical problem. Running:

new flexjson.JSONDeserializer () .use (null, cl) .deserialize ("""[{"foo": "bar"}]""")

gives error:

java.lang.ClassCastException: java.util.ArrayList cannot be cast to java.util.Map
    at flexjson.factories.ClassLocatorObjectFactory.instantiate(ClassLocatorObjectFactory.java:26)
    at flexjson.ObjectBinder.convert(ObjectBinder.java:174)
    at flexjson.ObjectBinder.bind(ObjectBinder.java:79)
    at flexjson.ObjectBinder.bind(ObjectBinder.java:64)
    at flexjson.JSONDeserializer.deserialize(JSON...

Ok I see your point. But what could the ClassLocator do meaningfully to locate a different class in the case of arrays? Objects are complex and can encode type information in unique ways. Lists, Arrays, Sets, etc can't encode type information any differently. I don't see any benefit a ClassLocator can offer over simply stating the type you want (i.e. use( null, LinkedList.class ) ).

Maybe if I fix this issue with class loading you won't have a need to substitute a ClassLocator?

I'd like to address this issue, but it involves changing interfaces which I'm not super excited about doing. I'm going to have to stew on this some.

Charlie

"use (null, cl)" enables ClassLocator for the whole hierarchy of objects present in the JSON string. We might not need the ClassLocator for arrays specifically, but we might very well need it for other objects located therein. For example, my current ClassLocator contains this additional string:

    if (map.containsKey ("actLevStart")) classOf[DenormalizedRating]

From that point of view, i think JSONDeserializer might skip the ClassLocator for arrays but should still use it for objects.

Ok. Well I can say that in 1.9 I've fixed this issue regarding the ClassLocator is used for the Array AND the objects contained within.

Starting in 1.9 you can differentiate between the Collection and the contents by using "values". So you if you want a locator for the contents of an array you append ".values" onto the path. For example,

.use("someobject.path.colleciton", LinkedList.class) .use("someobject.path.collection.values", Foo.class)

For maps you can use the names "keys" and "values" to specify the keys and the values respectively for the Map. Remember this is needed in cases where you don't have generic collections, or you're using polymorphic collections.

Just as an aside, the issue in prior versions where you couldn't differentiate the collection from the values using the path notation only affected those two levels. If you need to specify a class or a ClassLocator deeper in the hierarchy you could. It's a small difference, but an important one.

Charlie

I can use this and that to workaround the problem of using ClassLocator with arrays, but it will complicate things unnecessarily and doesn't fix the present issue.

I don't understand what you have issue with then. How do this not fix your problem? Or how does this "complicate things unnecessarily"?

How do this not fix your problem?

ClassLocator still can't be used with arrays.

Or how does this "complicate things unnecessarily"?

Using a universal ClassLocator with "use (null, cl)" is simpler then trying to match paths for various JSON strings.

We might not need the ClassLocator for arrays specifically

...

ClassLocator still can't be used with arrays

But you admitted just above you aren't sure you even need this. Again I go back to the issue that ClassLocators for Arrays don't provide anymore benefit over specify the type of Class you want instantiated for the array.

Using a universal ClassLocator with "use (null, cl)" is simpler then trying to match paths for various JSON strings.

In your case it would be as easy as:

new JSONDeserializer().use( "values", Foo.class ).deserialize( json );

You don't need to specify the collection class because it will default to ArrayList. If you don't care, and are happy with the List or Collection interface then you're good to go. I'm not sure how using a ClassLocator is easier?

In your case it would be as easy as:
new JSONDeserializer().use( "values", Foo.class ).deserialize( json );

Duh. Of course it wouldn't. Foo is just an example I made up, in a real life I have a complex tree with different types of leafs.

But you admitted just above you aren't sure you even need this.

I don't need ClassLocator to provide different kind of arrays, but I need ClassLocator to work on a JSON string containing arrays. For example, for a string [{"foo": "bar"}] I would expect JSONDeserializer to decide the "[]" automagically, but pass the {} part to ClassLocator to decide.

Duh. Of course it wouldn't. Foo is just an example I made up, in a real life I have a complex tree with different types of leafs.

Remember these path changes extend to ClassLocators as well. You don't have to register on the null anymore. For example,

new JSONDeserizlier().use( "values", cl ).deserialize( json );

Why is it important that you use a ClassLocator rather than a simple class? You mentioned classloader problems earlier, but now it seems your complaint is object complexity and not wanting to use the path mechanisms provided to specify your class types. I'm not sure how you're using ClassLocator to overcome these in a easier way. If you can enlighten me then we can talk about changes.

There's one thing in your post that makes me wonder if we're not on the same page here. You mention registering a null ClassLocator as "Universal", which makes we think you believe this ClassLocator extends to the entire graph. While you might be able to see the entire graph, i.e by digging through that map object, the return type only allows one Class. And this ClassLocator isn't called for lower levels. It's only registered at that top level. It will not be used to resolve classes at lower levels.

That's why I'm not too worried about how complex your objects below the [ { "foo": "bar" } ] are because this ClassLocator doesn't factor into those complexities.

In the example ClassLocator you provided I don't see it dealing with anything below the array's members. It looked like it was just to handle the class loader issues. So, I'm not sure why you want to register the loader on the null instead of "values".

for a string [{"foo": "bar"}] I would expect JSONDeserializer to decide the "[]" automagically, but pass the {} part to ClassLocator to decide.

That might be possible, but I pause only because it's an exception to the rule. I'll have to give this careful consideration. I know now it's a design flaw, but I'm going to weigh that against future changes to see if that's a quick fix vs something more long term.

Thanks, Charlie

You mentioned classloader problems earlier, but now it seems your complaint is object complexity and not wanting to use the path mechanisms provided to specify your class types.

I mentioned both class loading issues AND using the additional features ClassLocator provides, see message 8 of this thread. My "complaint" is that ClassLocator doesn't work well with arrays.

You mention registering a null ClassLocator as "Universal", which makes we think you believe this ClassLocator extends to the entire graph.

That's how it worked with flexjson 1.8, see Example 1.

While you might be able to see the entire graph, i.e by digging through that map object, the return type only allows one Class.

I don't think so: ClassLocator.locate might return a different Class for different parts of the tree. Example 1 shows how this worked with flexjson 1.8, and it still works when using "values" with flexjson 1.9.1.

In the example ClassLocator you provided I don't see it dealing with anything below the array's members.

I mentioned it dealing with object members in the 12th message of this thread.

P.S. Please check examples 3 and 4 to see that ClassLocator still isn't usable with arrays.

artemgr,

The bugs you point out in example 3 and example 4 are fixed in 1.9.2. Please grab the updates and make sure these are working as you need them. Thanks for providing those examples of the problem. I hope these will fix your issues.

Charlie