I saw someone wanted to create this ticket, so I did.
I don't remember if I asked, should we give regions some kind of names to identify them inside HTM and make their functions clearer.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Yes, I believe we should have human readable names for the regions. And yes, this would be the place to put that info. I'll add some more detail to the ticket as we get a better idea of what additional data goes in this file (additional to the raw HTM data consisting of cells, segments, etc.)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I will make the CLA classes serializable, and create a method for extracting the network as a binary serialization. The IDE or any other app that uses the CLA should then be able to load / save networks via the .NET built in serialization classes. This should be Mono compatible too.
If it's not as easy as I think I will let you know and we can discuss alternatives. I believe binary serialization will keep the file size down - there shouldn't be a need for the network file to be human readable anyway.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Disregard my previous report. I did not have binary serialization on, it was XML. Now with binary serialization I am able to save the entire network and after training the simple default network on 2000 iterations, the saved network is 2863 KB. It takes a few seconds to serialize.
Trained networks will be much larger. I am still going to do more tests and focus on optimizing the data structure that gets serialized if it looks like we need to.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
A tip: classes are serialized by default, if you wish save storage, you can use the [NonSerializable] attribute to some not so relevant properties. If you know already this, please ignore the tip.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Apply the SerializableAttribute attribute to a type to indicate that instances of this type can be serialized. The common language runtime throws SerializationException if any type in the graph of objects being serialized does not have the SerializableAttribute attribute applied.
Have you seen the NetConfig class which is des/serialized to Instance? Have you seen any exception due to I don't use [SerializableAttribute]? If I'm saying that, is because I already tested. So, at least in Framework 4, classes seems have serializable behaviour by default or at least some serialization features don't require this attribute (http://msdn.microsoft.com/en-us/library/system.xml.serialization.xmlserializer.aspx ).
Last edit: David Ragazzi 2013-03-20
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Nick, thanks for pointing this out. I did not know about the new DataContract attributes. According to the Microsoft Guidance we would probably want to use that for "general persistence". I also like what they say here:
"DO think about serialization when you design new types. Serialization is an important design consideration for any type, because programs might need to persist or transmit instances of the type."
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The task for me is to either use attributes or some way to control what gets serialized or not, so we get the smallest files and shortest load/save times. So I will do what is easiest first, then test and see if I can optimize. I am not sure if I will use attributes to do that, but will probably start there.
Please keep feedback and ideas coming. This is an area that will need a solid solution before we can release a 1.0 version of CLA/openHtm. So I will first get something simple out there and we can discuss more and add performance / features that get us where we want to be after several beta versions.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
After looking at options I think DataContract attributes are the way to go for persisting the HTM. We want anyone to be able to serialize / deserialize the HTMs for sharing, no matter what platform their code is written in. It looks like DataContract was designed to do just that.
Anyone have thought or other ideas? I plan to implement serialize / deserialize for the HTM and create a helper class to be used by the IDE. I will also try to do an inter-op example in some other language (python? ) that shows how to deserialize an HTM.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I saw someone wanted to create this ticket, so I did.
I don't remember if I asked, should we give regions some kind of names to identify them inside HTM and make their functions clearer.
Thanks for creating the ticket.
Yes, I believe we should have human readable names for the regions. And yes, this would be the place to put that info. I'll add some more detail to the ticket as we get a better idea of what additional data goes in this file (additional to the raw HTM data consisting of cells, segments, etc.)
I will make the CLA classes serializable, and create a method for extracting the network as a binary serialization. The IDE or any other app that uses the CLA should then be able to load / save networks via the .NET built in serialization classes. This should be Mono compatible too.
If it's not as easy as I think I will let you know and we can discuss alternatives. I believe binary serialization will keep the file size down - there shouldn't be a need for the network file to be human readable anyway.
Hi Doug, if you use XmlSerializer (check NetConfig class in our IDE) you can't need make serializable the classes:
http://msdn.microsoft.com/en-us/library/system.xml.serialization.xmlserializer.aspx
However, I don't know say if Mono has implemented this feature.
Last edit: David Ragazzi 2013-03-20
Progress report #2 on saving the HTM Network:
Disregard my previous report. I did not have binary serialization on, it was XML. Now with binary serialization I am able to save the entire network and after training the simple default network on 2000 iterations, the saved network is 2863 KB. It takes a few seconds to serialize.
Trained networks will be much larger. I am still going to do more tests and focus on optimizing the data structure that gets serialized if it looks like we need to.
Hi Doug,
A tip: classes are serialized by default, if you wish save storage, you can use the [NonSerializable] attribute to some not so relevant properties. If you know already this, please ignore the tip.
So what's this?
http://msdn.microsoft.com/en-us/library/system.serializableattribute(v=vs.100).aspx
Have you seen the NetConfig class which is des/serialized to Instance? Have you seen any exception due to I don't use [SerializableAttribute]? If I'm saying that, is because I already tested. So, at least in Framework 4, classes seems have serializable behaviour by default or at least some serialization features don't require this attribute (http://msdn.microsoft.com/en-us/library/system.xml.serialization.xmlserializer.aspx ).
Last edit: David Ragazzi 2013-03-20
OK, I didn't see that code. (I remember some linq queries for Regions were there before)
Yes, there are many ways:
http://msdn.microsoft.com/en-us/library/6exf3h2k.aspx
Nick, thanks for pointing this out. I did not know about the new DataContract attributes. According to the Microsoft Guidance we would probably want to use that for "general persistence". I also like what they say here:
The task for me is to either use attributes or some way to control what gets serialized or not, so we get the smallest files and shortest load/save times. So I will do what is easiest first, then test and see if I can optimize. I am not sure if I will use attributes to do that, but will probably start there.
Please keep feedback and ideas coming. This is an area that will need a solid solution before we can release a 1.0 version of CLA/openHtm. So I will first get something simple out there and we can discuss more and add performance / features that get us where we want to be after several beta versions.
After looking at options I think DataContract attributes are the way to go for persisting the HTM. We want anyone to be able to serialize / deserialize the HTMs for sharing, no matter what platform their code is written in. It looks like DataContract was designed to do just that.
Anyone have thought or other ideas? I plan to implement serialize / deserialize for the HTM and create a helper class to be used by the IDE. I will also try to do an inter-op example in some other language (python? ) that shows how to deserialize an HTM.