umit

Hi Narendran,
      Actually there's not a notification system yet. I know you mean to
implement a notification system at the agent side. The method we are
using now is periodically checking the aggregator. The agent sends
GetEvent messages to the aggregator and gets the events happened
in its region or the global Internet. The notification system might not be
a quite urgent job for now. We can add some code to pop-up a small
window for notification.
      I think the routing strategy is a problem needs to be solved. Also
we need to figure out how to form a stable peer network, in which the
peers could dynamically gather into clusters, and bypass the blockage.
Currently the aggregator url and super agent addresses (the entrance
for a new agent) are written into the db of the agent statically.
We still need to think out more ideas for a new peer to find its neighbors
and the super agents. Although we are using a cloud-based aggregator
 to scale our service, we are building a huge network covering the whole
world! So management of peers should be decentralized, by the peers
 themselves or the super agents. Remember the essential task is to
spread the information in a network with blockage. :)

Best Regards.
Alan

On Sun, Mar 25, 2012 at 12:01 PM, Narendran Thangarajan <
narendran.thangarajan@...> wrote:

> Hi,
>  I looked into the reporting mechanism in ICM-desktop agent. I see
> that once the agent logs in, there are three looping calls that
> perform the following :
>   1. Get peer list and populate the peerinfo.
>   2. Schedule tasks and add reports
>   3. Upload the generated reports.
>
>  But I can't find logic, how the peers make use of the
> Website/service reports generated from other peers or aggregator. If
> its available, where should I find the logic? If not, I would like to
> implement a notification system, which notifies the peer user of any
> network / website outages. This tool will be really helpful for
> admins/end users at various scales (On a global level or within a data
> center - thanks to the flexible design of the OpenMonitor system).
> This can be implemented using an evented I/O (say gevent).
>
> Thanks,
> Narendran.
>
>
> ------------------------------------------------------------------------------
> This SF email is sponsosred by:
> Try Windows Azure free for 90 days Click Here
> http://p.sf.net/sfu/sfd2d-msazure
> _______________________________________________
> Umit-devel mailing list
> Umit-devel@...
> https://lists.sourceforge.net/lists/listinfo/umit-devel
>



-- 
Zhongjie Wang
Master Candidate
Computer System Architecture
Peking University, China

Yeah I get it. So we need to implement a P2P routing overlay so that Peers
can form clusters among themselves. There are a horde of options like
Gnutella, DHT, Semantic routing etc. Since we are focusing on a global
scale, I feel the best option is to go for Distributed hash tables
implementation. It is fast and doesn't flood the network with redundant
requests. Moreover, the system gets decentralized, thus it is easy to scale
on demand.

Among the DHT algorithms, I feel that making a Kademlia based
implementation of DHT is a more viable option in python. Added advantage is
that, certain Kademlia implementations provide out-of-the-box visualizers
for monitoring all the peers. This will greatly enhance the user experience.

Thanks,
Narendran Thangarajan.


On Sun, Mar 25, 2012 at 7:16 PM, Zhongjie Wang <wzj401@...> wrote:

> Hi Narendran,
>       Actually there's not a notification system yet. I know you mean to
> implement a notification system at the agent side. The method we are
> using now is periodically checking the aggregator. The agent sends
> GetEvent messages to the aggregator and gets the events happened
> in its region or the global Internet. The notification system might not be
> a quite urgent job for now. We can add some code to pop-up a small
> window for notification.
>       I think the routing strategy is a problem needs to be solved. Also
> we need to figure out how to form a stable peer network, in which the
> peers could dynamically gather into clusters, and bypass the blockage.
> Currently the aggregator url and super agent addresses (the entrance
> for a new agent) are written into the db of the agent statically.
> We still need to think out more ideas for a new peer to find its neighbors
> and the super agents. Although we are using a cloud-based aggregator
>  to scale our service, we are building a huge network covering the whole
> world! So management of peers should be decentralized, by the peers
>  themselves or the super agents. Remember the essential task is to
> spread the information in a network with blockage. :)
>
> Best Regards.
> Alan
>
> On Sun, Mar 25, 2012 at 12:01 PM, Narendran Thangarajan <
> narendran.thangarajan@...> wrote:
>
>> Hi,
>>  I looked into the reporting mechanism in ICM-desktop agent. I see
>> that once the agent logs in, there are three looping calls that
>> perform the following :
>>   1. Get peer list and populate the peerinfo.
>>   2. Schedule tasks and add reports
>>   3. Upload the generated reports.
>>
>>  But I can't find logic, how the peers make use of the
>> Website/service reports generated from other peers or aggregator. If
>> its available, where should I find the logic? If not, I would like to
>> implement a notification system, which notifies the peer user of any
>> network / website outages. This tool will be really helpful for
>> admins/end users at various scales (On a global level or within a data
>> center - thanks to the flexible design of the OpenMonitor system).
>> This can be implemented using an evented I/O (say gevent).
>>
>> Thanks,
>> Narendran.
>>
>>
>> ------------------------------------------------------------------------------
>> This SF email is sponsosred by:
>> Try Windows Azure free for 90 days Click Here
>> http://p.sf.net/sfu/sfd2d-msazure
>> _______________________________________________
>> Umit-devel mailing list
>> Umit-devel@...
>> https://lists.sourceforge.net/lists/listinfo/umit-devel
>>
>
>
>
> --
> Zhongjie Wang
> Master Candidate
> Computer System Architecture
> Peking University, China
>

Guys, 

You also need to consider the mobile agent. The same DHT algorithm needs to be applicable to the mobile agent as well. First, and foremost, we need to come up with a solution for pure P2P communication assuming that both peers are behind a NAT. One option is to use tcp hole punching. In such an architecture, can some of the desktop agents --with public IPs -- serve as relay servers between other desktop agents and mobile agents? This was a major impediment that we ran into last year. I know of a couple of Kademlia libraries in C++ and Java that have UPnP-enabled NAT traversal. Do the Python implementations consider NAT traversal? In addition, for people interested in working on the mobile agent, are there any Android Java libraries to this end?

Best,
Zubair



On 25 Mar 2012, at 20:48, Narendran Thangarajan wrote:

> Yeah I get it. So we need to implement a P2P routing overlay so that Peers can form clusters among themselves. There are a horde of options like Gnutella, DHT, Semantic routing etc. Since we are focusing on a global scale, I feel the best option is to go for Distributed hash tables implementation. It is fast and doesn't flood the network with redundant requests. Moreover, the system gets decentralized, thus it is easy to scale on demand.
> 
> Among the DHT algorithms, I feel that making a Kademlia based implementation of DHT is a more viable option in python. Added advantage is that, certain Kademlia implementations provide out-of-the-box visualizers for monitoring all the peers. This will greatly enhance the user experience.
> 
> Thanks,
> Narendran Thangarajan.
> 
> 
> On Sun, Mar 25, 2012 at 7:16 PM, Zhongjie Wang <wzj401@...> wrote:
> Hi Narendran,
>       Actually there's not a notification system yet. I know you mean to 
> implement a notification system at the agent side. The method we are
> using now is periodically checking the aggregator. The agent sends 
> GetEvent messages to the aggregator and gets the events happened 
> in its region or the global Internet. The notification system might not be
> a quite urgent job for now. We can add some code to pop-up a small 
> window for notification. 
>       I think the routing strategy is a problem needs to be solved. Also 
> we need to figure out how to form a stable peer network, in which the 
> peers could dynamically gather into clusters, and bypass the blockage.
> Currently the aggregator url and super agent addresses (the entrance 
> for a new agent) are written into the db of the agent statically. 
> We still need to think out more ideas for a new peer to find its neighbors 
> and the super agents. Although we are using a cloud-based aggregator
>  to scale our service, we are building a huge network covering the whole 
> world! So management of peers should be decentralized, by the peers
>  themselves or the super agents. Remember the essential task is to 
> spread the information in a network with blockage. :)
> 
> Best Regards.
> Alan
> 
> On Sun, Mar 25, 2012 at 12:01 PM, Narendran Thangarajan <narendran.thangarajan@...> wrote:
> Hi,
>  I looked into the reporting mechanism in ICM-desktop agent. I see
> that once the agent logs in, there are three looping calls that
> perform the following :
>   1. Get peer list and populate the peerinfo.
>   2. Schedule tasks and add reports
>   3. Upload the generated reports.
> 
>  But I can't find logic, how the peers make use of the
> Website/service reports generated from other peers or aggregator. If
> its available, where should I find the logic? If not, I would like to
> implement a notification system, which notifies the peer user of any
> network / website outages. This tool will be really helpful for
> admins/end users at various scales (On a global level or within a data
> center - thanks to the flexible design of the OpenMonitor system).
> This can be implemented using an evented I/O (say gevent).
> 
> Thanks,
> Narendran.
> 
> ------------------------------------------------------------------------------
> This SF email is sponsosred by:
> Try Windows Azure free for 90 days Click Here
> http://p.sf.net/sfu/sfd2d-msazure
> _______________________________________________
> Umit-devel mailing list
> Umit-devel@...
> https://lists.sourceforge.net/lists/listinfo/umit-devel
> 
> 
> 
> -- 
> Zhongjie Wang
> Master Candidate
> Computer System Architecture
> Peking University, China
> 
> ------------------------------------------------------------------------------
> This SF email is sponsosred by:
> Try Windows Azure free for 90 days Click Here 
> http://p.sf.net/sfu/sfd2d-msazure_______________________________________________
> Umit-devel mailing list
> Umit-devel@...
> https://lists.sourceforge.net/lists/listinfo/umit-devel

Hi guys,
        Yeah, DHT is an option and we may also need to consider those nodes
behind NATs. So I recommend we use the super agents as trackers. The super
agents are those peers who have public IPs and could surely connect to the
aggregator. They can help with forwarding messages between nodes, and also
if an agent can connect to a super agent, then it messages can be sent to
the aggregator. Kad is using some protocol like gossip, we may also bring
this idea in.

Regards.
Alan

On Mon, Mar 26, 2012 at 4:02 AM, Zubair Nabi <zn.zubairnabi@...:

> Guys,
>
> You also need to consider the mobile agent. The same DHT algorithm needs
> to be applicable to the mobile agent as well. First, and foremost, we need
> to come up with a solution for pure P2P communication assuming that both
> peers are behind a NAT. One option is to use tcp hole punching. In such an
> architecture, can some of the desktop agents --with public IPs -- serve as
> relay servers between other desktop agents and mobile agents? This was a
> major impediment that we ran into last year. I know of a couple of Kademlia
> libraries in C++ and Java that have UPnP-enabled NAT traversal. Do the
> Python implementations consider NAT traversal? In addition, for people
> interested in working on the mobile agent, are there any Android Java
> libraries to this end?
>
>  Best,
> Zubair
>
>
>
> On 25 Mar 2012, at 20:48, Narendran Thangarajan wrote:
>
> Yeah I get it. So we need to implement a P2P routing overlay so that Peers
> can form clusters among themselves. There are a horde of options like
> Gnutella, DHT, Semantic routing etc. Since we are focusing on a global
> scale, I feel the best option is to go for Distributed hash tables
> implementation. It is fast and doesn't flood the network with redundant
> requests. Moreover, the system gets decentralized, thus it is easy to scale
> on demand.
>
> Among the DHT algorithms, I feel that making a Kademlia based
> implementation of DHT is a more viable option in python. Added advantage is
> that, certain Kademlia implementations provide out-of-the-box visualizers
> for monitoring all the peers. This will greatly enhance the user experience.
>
> Thanks,
> Narendran Thangarajan.
>
>
> On Sun, Mar 25, 2012 at 7:16 PM, Zhongjie Wang <wzj401@...> wrote:
>
>> Hi Narendran,
>>       Actually there's not a notification system yet. I know you mean to
>> implement a notification system at the agent side. The method we are
>> using now is periodically checking the aggregator. The agent sends
>> GetEvent messages to the aggregator and gets the events happened
>> in its region or the global Internet. The notification system might not be
>> a quite urgent job for now. We can add some code to pop-up a small
>> window for notification.
>>       I think the routing strategy is a problem needs to be solved. Also
>> we need to figure out how to form a stable peer network, in which the
>> peers could dynamically gather into clusters, and bypass the blockage.
>> Currently the aggregator url and super agent addresses (the entrance
>> for a new agent) are written into the db of the agent statically.
>> We still need to think out more ideas for a new peer to find
>> its neighbors
>> and the super agents. Although we are using a cloud-based aggregator
>>  to scale our service, we are building a huge network covering the whole
>> world! So management of peers should be decentralized, by the peers
>>  themselves or the super agents. Remember the essential task is to
>> spread the information in a network with blockage. :)
>>
>> Best Regards.
>> Alan
>>
>>  On Sun, Mar 25, 2012 at 12:01 PM, Narendran Thangarajan <
>> narendran.thangarajan@...> wrote:
>>
>>>  Hi,
>>>  I looked into the reporting mechanism in ICM-desktop agent. I see
>>> that once the agent logs in, there are three looping calls that
>>> perform the following :
>>>   1. Get peer list and populate the peerinfo.
>>>   2. Schedule tasks and add reports
>>>   3. Upload the generated reports.
>>>
>>>  But I can't find logic, how the peers make use of the
>>> Website/service reports generated from other peers or aggregator. If
>>> its available, where should I find the logic? If not, I would like to
>>> implement a notification system, which notifies the peer user of any
>>> network / website outages. This tool will be really helpful for
>>> admins/end users at various scales (On a global level or within a data
>>> center - thanks to the flexible design of the OpenMonitor system).
>>> This can be implemented using an evented I/O (say gevent).
>>>
>>> Thanks,
>>> Narendran.
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> This SF email is sponsosred by:
>>> Try Windows Azure free for 90 days Click Here
>>> http://p.sf.net/sfu/sfd2d-msazure
>>> _______________________________________________
>>> Umit-devel mailing list
>>> Umit-devel@...
>>> https://lists.sourceforge.net/lists/listinfo/umit-devel
>>>
>>
>>
>>
>> --
>> Zhongjie Wang
>> Master Candidate
>> Computer System Architecture
>> Peking University, China
>>
>
>
> ------------------------------------------------------------------------------
> This SF email is sponsosred by:
> Try Windows Azure free for 90 days Click Here
>
> http://p.sf.net/sfu/sfd2d-msazure_______________________________________________
> Umit-devel mailing list
> Umit-devel@...
> https://lists.sourceforge.net/lists/listinfo/umit-devel
>
>
>


-- 
Zhongjie Wang
Master Candidate
Computer System Architecture
Peking University, China

This turned out into a good discussion, and I definitely want to see on the
proposal how students wants to tackle this challenge :)


On Sun, Mar 25, 2012 at 11:43 PM, Zhongjie Wang <wzj401@...> wrote:

> Hi guys,
>         Yeah, DHT is an option and we may also need to consider those
> nodes behind NATs. So I recommend we use the super agents as trackers. The
> super agents are those peers who have public IPs and could surely connect
> to the aggregator. They can help with forwarding messages between nodes,
> and also if an agent can connect to a super agent, then it messages can be
> sent to the aggregator. Kad is using some protocol like gossip, we may also
> bring this idea in.
>
> Regards.
> Alan
>
>
> On Mon, Mar 26, 2012 at 4:02 AM, Zubair Nabi <zn.zubairnabi@...:
>
>> Guys,
>>
>> You also need to consider the mobile agent. The same DHT algorithm needs
>> to be applicable to the mobile agent as well. First, and foremost, we need
>> to come up with a solution for pure P2P communication assuming that both
>> peers are behind a NAT. One option is to use tcp hole punching. In such an
>> architecture, can some of the desktop agents --with public IPs -- serve as
>> relay servers between other desktop agents and mobile agents? This was a
>> major impediment that we ran into last year. I know of a couple of Kademlia
>> libraries in C++ and Java that have UPnP-enabled NAT traversal. Do the
>> Python implementations consider NAT traversal? In addition, for people
>> interested in working on the mobile agent, are there any Android Java
>> libraries to this end?
>>
>>  Best,
>> Zubair
>>
>>
>>
>> On 25 Mar 2012, at 20:48, Narendran Thangarajan wrote:
>>
>> Yeah I get it. So we need to implement a P2P routing overlay so that
>> Peers can form clusters among themselves. There are a horde of options like
>> Gnutella, DHT, Semantic routing etc. Since we are focusing on a global
>> scale, I feel the best option is to go for Distributed hash tables
>> implementation. It is fast and doesn't flood the network with redundant
>> requests. Moreover, the system gets decentralized, thus it is easy to scale
>> on demand.
>>
>> Among the DHT algorithms, I feel that making a Kademlia based
>> implementation of DHT is a more viable option in python. Added advantage is
>> that, certain Kademlia implementations provide out-of-the-box visualizers
>> for monitoring all the peers. This will greatly enhance the user experience.
>>
>> Thanks,
>> Narendran Thangarajan.
>>
>>
>> On Sun, Mar 25, 2012 at 7:16 PM, Zhongjie Wang <wzj401@...> wrote:
>>
>>> Hi Narendran,
>>>       Actually there's not a notification system yet. I know you mean to
>>> implement a notification system at the agent side. The method we are
>>> using now is periodically checking the aggregator. The agent sends
>>> GetEvent messages to the aggregator and gets the events happened
>>> in its region or the global Internet. The notification system might not
>>> be
>>> a quite urgent job for now. We can add some code to pop-up a small
>>> window for notification.
>>>       I think the routing strategy is a problem needs to be solved. Also
>>> we need to figure out how to form a stable peer network, in which the
>>> peers could dynamically gather into clusters, and bypass the blockage.
>>> Currently the aggregator url and super agent addresses (the entrance
>>> for a new agent) are written into the db of the agent statically.
>>> We still need to think out more ideas for a new peer to find
>>> its neighbors
>>> and the super agents. Although we are using a cloud-based aggregator
>>>  to scale our service, we are building a huge network covering the whole
>>> world! So management of peers should be decentralized, by the peers
>>>  themselves or the super agents. Remember the essential task is to
>>> spread the information in a network with blockage. :)
>>>
>>> Best Regards.
>>> Alan
>>>
>>>  On Sun, Mar 25, 2012 at 12:01 PM, Narendran Thangarajan <
>>> narendran.thangarajan@...> wrote:
>>>
>>>>  Hi,
>>>>  I looked into the reporting mechanism in ICM-desktop agent. I see
>>>> that once the agent logs in, there are three looping calls that
>>>> perform the following :
>>>>   1. Get peer list and populate the peerinfo.
>>>>   2. Schedule tasks and add reports
>>>>   3. Upload the generated reports.
>>>>
>>>>  But I can't find logic, how the peers make use of the
>>>> Website/service reports generated from other peers or aggregator. If
>>>> its available, where should I find the logic? If not, I would like to
>>>> implement a notification system, which notifies the peer user of any
>>>> network / website outages. This tool will be really helpful for
>>>> admins/end users at various scales (On a global level or within a data
>>>> center - thanks to the flexible design of the OpenMonitor system).
>>>> This can be implemented using an evented I/O (say gevent).
>>>>
>>>> Thanks,
>>>> Narendran.
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> This SF email is sponsosred by:
>>>> Try Windows Azure free for 90 days Click Here
>>>> http://p.sf.net/sfu/sfd2d-msazure
>>>> _______________________________________________
>>>> Umit-devel mailing list
>>>> Umit-devel@...
>>>> https://lists.sourceforge.net/lists/listinfo/umit-devel
>>>>
>>>
>>>
>>>
>>> --
>>> Zhongjie Wang
>>> Master Candidate
>>> Computer System Architecture
>>> Peking University, China
>>>
>>
>>
>> ------------------------------------------------------------------------------
>> This SF email is sponsosred by:
>> Try Windows Azure free for 90 days Click Here
>>
>> http://p.sf.net/sfu/sfd2d-msazure_______________________________________________
>> Umit-devel mailing list
>> Umit-devel@...
>> https://lists.sourceforge.net/lists/listinfo/umit-devel
>>
>>
>>
>
>
> --
> Zhongjie Wang
> Master Candidate
> Computer System Architecture
> Peking University, China
>
>
> ------------------------------------------------------------------------------
> This SF email is sponsosred by:
> Try Windows Azure free for 90 days Click Here
> http://p.sf.net/sfu/sfd2d-msazure
> _______________________________________________
> Umit-devel mailing list
> Umit-devel@...
> https://lists.sourceforge.net/lists/listinfo/umit-devel
>
>


-- 
Adriano Monteiro Marques

http://www.openmonitor.org
http://www.umitproject.org
http://www.thoughtspad.com

"Don't stay in bed, unless you can make money in bed." - George Burns

The three major python implementations of Kad are Khashmir, SharkyPy and
Entangled. However, all three do not mention about NAT traversal. Even if
we had such an UPnP enabled python implementation, we must look into its
compatibility with our mobile agent too. And as far as I have seen there
are no Android alternatives. And making two bytecode oriented languages
talk with each other is much worse.
The best thing is to go for a NAT traversal enabled C++ implementation (say
maidsafe-dht) connected to our desktop agent (Python) and mobile
agent(Android-java) through native extensions. Though it looks little tough
at the first sight, this gives great flexibility. And since the base
implementation is native C++, its going to be faster too.


On Mon, Mar 26, 2012 at 4:09 PM, Adriano Marques <py.adriano@...:

> This turned out into a good discussion, and I definitely want to see on
> the proposal how students wants to tackle this challenge :)
>
>
> On Sun, Mar 25, 2012 at 11:43 PM, Zhongjie Wang <wzj401@...> wrote:
>
>> Hi guys,
>>         Yeah, DHT is an option and we may also need to consider those
>> nodes behind NATs. So I recommend we use the super agents as trackers. The
>> super agents are those peers who have public IPs and could surely connect
>> to the aggregator. They can help with forwarding messages between nodes,
>> and also if an agent can connect to a super agent, then it messages can be
>> sent to the aggregator. Kad is using some protocol like gossip, we may also
>> bring this idea in.
>>
>> Regards.
>> Alan
>>
>>
>> On Mon, Mar 26, 2012 at 4:02 AM, Zubair Nabi <zn.zubairnabi@...:
>>
>>> Guys,
>>>
>>> You also need to consider the mobile agent. The same DHT algorithm needs
>>> to be applicable to the mobile agent as well. First, and foremost, we need
>>> to come up with a solution for pure P2P communication assuming that both
>>> peers are behind a NAT. One option is to use tcp hole punching. In such an
>>> architecture, can some of the desktop agents --with public IPs -- serve as
>>> relay servers between other desktop agents and mobile agents? This was a
>>> major impediment that we ran into last year. I know of a couple of Kademlia
>>> libraries in C++ and Java that have UPnP-enabled NAT traversal. Do the
>>> Python implementations consider NAT traversal? In addition, for people
>>> interested in working on the mobile agent, are there any Android Java
>>> libraries to this end?
>>>
>>>  Best,
>>> Zubair
>>>
>>>
>>>
>>> On 25 Mar 2012, at 20:48, Narendran Thangarajan wrote:
>>>
>>> Yeah I get it. So we need to implement a P2P routing overlay so that
>>> Peers can form clusters among themselves. There are a horde of options like
>>> Gnutella, DHT, Semantic routing etc. Since we are focusing on a global
>>> scale, I feel the best option is to go for Distributed hash tables
>>> implementation. It is fast and doesn't flood the network with redundant
>>> requests. Moreover, the system gets decentralized, thus it is easy to scale
>>> on demand.
>>>
>>> Among the DHT algorithms, I feel that making a Kademlia based
>>> implementation of DHT is a more viable option in python. Added advantage is
>>> that, certain Kademlia implementations provide out-of-the-box visualizers
>>> for monitoring all the peers. This will greatly enhance the user experience.
>>>
>>> Thanks,
>>> Narendran Thangarajan.
>>>
>>>
>>> On Sun, Mar 25, 2012 at 7:16 PM, Zhongjie Wang <wzj401@...> wrote:
>>>
>>>> Hi Narendran,
>>>>       Actually there's not a notification system yet. I know you mean
>>>> to
>>>> implement a notification system at the agent side. The method we are
>>>> using now is periodically checking the aggregator. The agent sends
>>>> GetEvent messages to the aggregator and gets the events happened
>>>> in its region or the global Internet. The notification system might not
>>>> be
>>>> a quite urgent job for now. We can add some code to pop-up a small
>>>> window for notification.
>>>>       I think the routing strategy is a problem needs to be solved.
>>>> Also
>>>> we need to figure out how to form a stable peer network, in which the
>>>> peers could dynamically gather into clusters, and bypass the blockage.
>>>> Currently the aggregator url and super agent addresses (the entrance
>>>> for a new agent) are written into the db of the agent statically.
>>>> We still need to think out more ideas for a new peer to find
>>>> its neighbors
>>>> and the super agents. Although we are using a cloud-based aggregator
>>>>  to scale our service, we are building a huge network covering the
>>>> whole
>>>> world! So management of peers should be decentralized, by the peers
>>>>  themselves or the super agents. Remember the essential task is to
>>>> spread the information in a network with blockage. :)
>>>>
>>>> Best Regards.
>>>> Alan
>>>>
>>>>  On Sun, Mar 25, 2012 at 12:01 PM, Narendran Thangarajan <
>>>> narendran.thangarajan@...> wrote:
>>>>
>>>>>  Hi,
>>>>>  I looked into the reporting mechanism in ICM-desktop agent. I see
>>>>> that once the agent logs in, there are three looping calls that
>>>>> perform the following :
>>>>>   1. Get peer list and populate the peerinfo.
>>>>>   2. Schedule tasks and add reports
>>>>>   3. Upload the generated reports.
>>>>>
>>>>>  But I can't find logic, how the peers make use of the
>>>>> Website/service reports generated from other peers or aggregator. If
>>>>> its available, where should I find the logic? If not, I would like to
>>>>> implement a notification system, which notifies the peer user of any
>>>>> network / website outages. This tool will be really helpful for
>>>>> admins/end users at various scales (On a global level or within a data
>>>>> center - thanks to the flexible design of the OpenMonitor system).
>>>>> This can be implemented using an evented I/O (say gevent).
>>>>>
>>>>> Thanks,
>>>>> Narendran.
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------------
>>>>> This SF email is sponsosred by:
>>>>> Try Windows Azure free for 90 days Click Here
>>>>> http://p.sf.net/sfu/sfd2d-msazure
>>>>> _______________________________________________
>>>>> Umit-devel mailing list
>>>>> Umit-devel@...
>>>>> https://lists.sourceforge.net/lists/listinfo/umit-devel
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Zhongjie Wang
>>>> Master Candidate
>>>> Computer System Architecture
>>>> Peking University, China
>>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> This SF email is sponsosred by:
>>> Try Windows Azure free for 90 days Click Here
>>>
>>> http://p.sf.net/sfu/sfd2d-msazure_______________________________________________
>>> Umit-devel mailing list
>>> Umit-devel@...
>>> https://lists.sourceforge.net/lists/listinfo/umit-devel
>>>
>>>
>>>
>>
>>
>> --
>> Zhongjie Wang
>> Master Candidate
>> Computer System Architecture
>> Peking University, China
>>
>>
>> ------------------------------------------------------------------------------
>> This SF email is sponsosred by:
>> Try Windows Azure free for 90 days Click Here
>> http://p.sf.net/sfu/sfd2d-msazure
>> _______________________________________________
>> Umit-devel mailing list
>> Umit-devel@...
>> https://lists.sourceforge.net/lists/listinfo/umit-devel
>>
>>
>
>
> --
> Adriano Monteiro Marques
>
> http://www.openmonitor.org
> http://www.umitproject.org
> http://www.thoughtspad.com
>
> "Don't stay in bed, unless you can make money in bed." - George Burns
>
>

Let me first look into the feasibility of using maidsafe-dht as a native
extension with python. Only then we could know the next step. I would love
to hear suggestions in this regard. Should I look into any other specific
implementation? Or has anyone done a study on  this already?

Thanks,
Narendran.

On Mon, Mar 26, 2012 at 9:17 PM, Narendran Thangarajan <
narendran.thangarajan@...> wrote:

> The three major python implementations of Kad are Khashmir, SharkyPy and
> Entangled. However, all three do not mention about NAT traversal. Even if
> we had such an UPnP enabled python implementation, we must look into its
> compatibility with our mobile agent too. And as far as I have seen there
> are no Android alternatives. And making two bytecode oriented languages
> talk with each other is much worse.
> The best thing is to go for a NAT traversal enabled C++ implementation
> (say maidsafe-dht) connected to our desktop agent (Python) and mobile
> agent(Android-java) through native extensions. Though it looks little tough
> at the first sight, this gives great flexibility. And since the base
> implementation is native C++, its going to be faster too.
>
>
> On Mon, Mar 26, 2012 at 4:09 PM, Adriano Marques <py.adriano@...:
>
>> This turned out into a good discussion, and I definitely want to see on
>> the proposal how students wants to tackle this challenge :)
>>
>>
>> On Sun, Mar 25, 2012 at 11:43 PM, Zhongjie Wang <wzj401@...> wrote:
>>
>>> Hi guys,
>>>         Yeah, DHT is an option and we may also need to consider those
>>> nodes behind NATs. So I recommend we use the super agents as trackers. The
>>> super agents are those peers who have public IPs and could surely connect
>>> to the aggregator. They can help with forwarding messages between nodes,
>>> and also if an agent can connect to a super agent, then it messages can be
>>> sent to the aggregator. Kad is using some protocol like gossip, we may also
>>> bring this idea in.
>>>
>>> Regards.
>>> Alan
>>>
>>>
>>> On Mon, Mar 26, 2012 at 4:02 AM, Zubair Nabi <zn.zubairnabi@...:
>>>
>>>> Guys,
>>>>
>>>> You also need to consider the mobile agent. The same DHT algorithm
>>>> needs to be applicable to the mobile agent as well. First, and foremost, we
>>>> need to come up with a solution for pure P2P communication assuming that
>>>> both peers are behind a NAT. One option is to use tcp hole punching. In
>>>> such an architecture, can some of the desktop agents --with public IPs --
>>>> serve as relay servers between other desktop agents and mobile agents? This
>>>> was a major impediment that we ran into last year. I know of a couple of
>>>> Kademlia libraries in C++ and Java that have UPnP-enabled NAT traversal. Do
>>>> the Python implementations consider NAT traversal? In addition, for people
>>>> interested in working on the mobile agent, are there any Android Java
>>>> libraries to this end?
>>>>
>>>>  Best,
>>>> Zubair
>>>>
>>>>
>>>>
>>>> On 25 Mar 2012, at 20:48, Narendran Thangarajan wrote:
>>>>
>>>> Yeah I get it. So we need to implement a P2P routing overlay so that
>>>> Peers can form clusters among themselves. There are a horde of options like
>>>> Gnutella, DHT, Semantic routing etc. Since we are focusing on a global
>>>> scale, I feel the best option is to go for Distributed hash tables
>>>> implementation. It is fast and doesn't flood the network with redundant
>>>> requests. Moreover, the system gets decentralized, thus it is easy to scale
>>>> on demand.
>>>>
>>>> Among the DHT algorithms, I feel that making a Kademlia based
>>>> implementation of DHT is a more viable option in python. Added advantage is
>>>> that, certain Kademlia implementations provide out-of-the-box visualizers
>>>> for monitoring all the peers. This will greatly enhance the user experience.
>>>>
>>>> Thanks,
>>>> Narendran Thangarajan.
>>>>
>>>>
>>>> On Sun, Mar 25, 2012 at 7:16 PM, Zhongjie Wang <wzj401@...:
>>>>
>>>>> Hi Narendran,
>>>>>       Actually there's not a notification system yet. I know you mean
>>>>> to
>>>>> implement a notification system at the agent side. The method we are
>>>>> using now is periodically checking the aggregator. The agent sends
>>>>> GetEvent messages to the aggregator and gets the events happened
>>>>> in its region or the global Internet. The notification system might
>>>>> not be
>>>>> a quite urgent job for now. We can add some code to pop-up a small
>>>>> window for notification.
>>>>>       I think the routing strategy is a problem needs to be solved.
>>>>> Also
>>>>> we need to figure out how to form a stable peer network, in which the
>>>>> peers could dynamically gather into clusters, and bypass the blockage.
>>>>> Currently the aggregator url and super agent addresses (the entrance
>>>>> for a new agent) are written into the db of the agent statically.
>>>>> We still need to think out more ideas for a new peer to find
>>>>> its neighbors
>>>>> and the super agents. Although we are using a cloud-based aggregator
>>>>>  to scale our service, we are building a huge network covering the
>>>>> whole
>>>>> world! So management of peers should be decentralized, by the peers
>>>>>  themselves or the super agents. Remember the essential task is to
>>>>> spread the information in a network with blockage. :)
>>>>>
>>>>> Best Regards.
>>>>> Alan
>>>>>
>>>>>  On Sun, Mar 25, 2012 at 12:01 PM, Narendran Thangarajan <
>>>>> narendran.thangarajan@...> wrote:
>>>>>
>>>>>>  Hi,
>>>>>>  I looked into the reporting mechanism in ICM-desktop agent. I see
>>>>>> that once the agent logs in, there are three looping calls that
>>>>>> perform the following :
>>>>>>   1. Get peer list and populate the peerinfo.
>>>>>>   2. Schedule tasks and add reports
>>>>>>   3. Upload the generated reports.
>>>>>>
>>>>>>  But I can't find logic, how the peers make use of the
>>>>>> Website/service reports generated from other peers or aggregator. If
>>>>>> its available, where should I find the logic? If not, I would like to
>>>>>> implement a notification system, which notifies the peer user of any
>>>>>> network / website outages. This tool will be really helpful for
>>>>>> admins/end users at various scales (On a global level or within a data
>>>>>> center - thanks to the flexible design of the OpenMonitor system).
>>>>>> This can be implemented using an evented I/O (say gevent).
>>>>>>
>>>>>> Thanks,
>>>>>> Narendran.
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------------
>>>>>> This SF email is sponsosred by:
>>>>>> Try Windows Azure free for 90 days Click Here
>>>>>> http://p.sf.net/sfu/sfd2d-msazure
>>>>>> _______________________________________________
>>>>>> Umit-devel mailing list
>>>>>> Umit-devel@...
>>>>>> https://lists.sourceforge.net/lists/listinfo/umit-devel
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Zhongjie Wang
>>>>> Master Candidate
>>>>> Computer System Architecture
>>>>> Peking University, China
>>>>>
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> This SF email is sponsosred by:
>>>> Try Windows Azure free for 90 days Click Here
>>>>
>>>> http://p.sf.net/sfu/sfd2d-msazure_______________________________________________
>>>> Umit-devel mailing list
>>>> Umit-devel@...
>>>> https://lists.sourceforge.net/lists/listinfo/umit-devel
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Zhongjie Wang
>>> Master Candidate
>>> Computer System Architecture
>>> Peking University, China
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> This SF email is sponsosred by:
>>> Try Windows Azure free for 90 days Click Here
>>> http://p.sf.net/sfu/sfd2d-msazure
>>> _______________________________________________
>>> Umit-devel mailing list
>>> Umit-devel@...
>>> https://lists.sourceforge.net/lists/listinfo/umit-devel
>>>
>>>
>>
>>
>> --
>> Adriano Monteiro Marques
>>
>> http://www.openmonitor.org
>> http://www.umitproject.org
>> http://www.thoughtspad.com
>>
>> "Don't stay in bed, unless you can make money in bed." - George Burns
>>
>>
>

In addition, if we native-extensions on the mobile agent, will that require root access? If yes, then I don't think it's a feasible idea. That's actually one of the reasons, we didn't use traceroute last year.

Best,
Zubair



On 27 Mar 2012, at 16:26, Narendran Thangarajan wrote:

> Let me first look into the feasibility of using maidsafe-dht as a native extension with python. Only then we could know the next step. I would love to hear suggestions in this regard. Should I look into any other specific implementation? Or has anyone done a study on  this already?
> 
> Thanks,
> Narendran.
> 
> On Mon, Mar 26, 2012 at 9:17 PM, Narendran Thangarajan <narendran.thangarajan@...> wrote:
> The three major python implementations of Kad are Khashmir, SharkyPy and Entangled. However, all three do not mention about NAT traversal. Even if we had such an UPnP enabled python implementation, we must look into its compatibility with our mobile agent too. And as far as I have seen there are no Android alternatives. And making two bytecode oriented languages talk with each other is much worse.
> The best thing is to go for a NAT traversal enabled C++ implementation (say maidsafe-dht) connected to our desktop agent (Python) and mobile agent(Android-java) through native extensions. Though it looks little tough at the first sight, this gives great flexibility. And since the base implementation is native C++, its going to be faster too. 
> 
> 
> On Mon, Mar 26, 2012 at 4:09 PM, Adriano Marques <py.adriano@...> wrote:
> This turned out into a good discussion, and I definitely want to see on the proposal how students wants to tackle this challenge :)
> 
> 
> On Sun, Mar 25, 2012 at 11:43 PM, Zhongjie Wang <wzj401@...> wrote:
> Hi guys,
>         Yeah, DHT is an option and we may also need to consider those nodes behind NATs. So I recommend we use the super agents as trackers. The super agents are those peers who have public IPs and could surely connect to the aggregator. They can help with forwarding messages between nodes, and also if an agent can connect to a super agent, then it messages can be sent to the aggregator. Kad is using some protocol like gossip, we may also bring this idea in. 
> 
> Regards.
> Alan
> 
> 
> On Mon, Mar 26, 2012 at 4:02 AM, Zubair Nabi <zn.zubairnabi@...> wrote:
> Guys, 
> 
> You also need to consider the mobile agent. The same DHT algorithm needs to be applicable to the mobile agent as well. First, and foremost, we need to come up with a solution for pure P2P communication assuming that both peers are behind a NAT. One option is to use tcp hole punching. In such an architecture, can some of the desktop agents --with public IPs -- serve as relay servers between other desktop agents and mobile agents? This was a major impediment that we ran into last year. I know of a couple of Kademlia libraries in C++ and Java that have UPnP-enabled NAT traversal. Do the Python implementations consider NAT traversal? In addition, for people interested in working on the mobile agent, are there any Android Java libraries to this end?
> 
> Best,
> Zubair
> 
> 
> 
> On 25 Mar 2012, at 20:48, Narendran Thangarajan wrote:
> 
>> Yeah I get it. So we need to implement a P2P routing overlay so that Peers can form clusters among themselves. There are a horde of options like Gnutella, DHT, Semantic routing etc. Since we are focusing on a global scale, I feel the best option is to go for Distributed hash tables implementation. It is fast and doesn't flood the network with redundant requests. Moreover, the system gets decentralized, thus it is easy to scale on demand.
>> 
>> Among the DHT algorithms, I feel that making a Kademlia based implementation of DHT is a more viable option in python. Added advantage is that, certain Kademlia implementations provide out-of-the-box visualizers for monitoring all the peers. This will greatly enhance the user experience.
>> 
>> Thanks,
>> Narendran Thangarajan.
>> 
>> 
>> On Sun, Mar 25, 2012 at 7:16 PM, Zhongjie Wang <wzj401@...> wrote:
>> Hi Narendran,
>>       Actually there's not a notification system yet. I know you mean to 
>> implement a notification system at the agent side. The method we are
>> using now is periodically checking the aggregator. The agent sends 
>> GetEvent messages to the aggregator and gets the events happened 
>> in its region or the global Internet. The notification system might not be
>> a quite urgent job for now. We can add some code to pop-up a small 
>> window for notification. 
>>       I think the routing strategy is a problem needs to be solved. Also 
>> we need to figure out how to form a stable peer network, in which the 
>> peers could dynamically gather into clusters, and bypass the blockage.
>> Currently the aggregator url and super agent addresses (the entrance 
>> for a new agent) are written into the db of the agent statically. 
>> We still need to think out more ideas for a new peer to find its neighbors 
>> and the super agents. Although we are using a cloud-based aggregator
>>  to scale our service, we are building a huge network covering the whole 
>> world! So management of peers should be decentralized, by the peers
>>  themselves or the super agents. Remember the essential task is to 
>> spread the information in a network with blockage. :)
>> 
>> Best Regards.
>> Alan
>> 
>> On Sun, Mar 25, 2012 at 12:01 PM, Narendran Thangarajan <narendran.thangarajan@...> wrote:
>> Hi,
>>  I looked into the reporting mechanism in ICM-desktop agent. I see
>> that once the agent logs in, there are three looping calls that
>> perform the following :
>>   1. Get peer list and populate the peerinfo.
>>   2. Schedule tasks and add reports
>>   3. Upload the generated reports.
>> 
>>  But I can't find logic, how the peers make use of the
>> Website/service reports generated from other peers or aggregator. If
>> its available, where should I find the logic? If not, I would like to
>> implement a notification system, which notifies the peer user of any
>> network / website outages. This tool will be really helpful for
>> admins/end users at various scales (On a global level or within a data
>> center - thanks to the flexible design of the OpenMonitor system).
>> This can be implemented using an evented I/O (say gevent).
>> 
>> Thanks,
>> Narendran.
>> 
>> ------------------------------------------------------------------------------
>> This SF email is sponsosred by:
>> Try Windows Azure free for 90 days Click Here
>> http://p.sf.net/sfu/sfd2d-msazure
>> _______________________________________________
>> Umit-devel mailing list
>> Umit-devel@...
>> https://lists.sourceforge.net/lists/listinfo/umit-devel
>> 
>> 
>> 
>> -- 
>> Zhongjie Wang
>> Master Candidate
>> Computer System Architecture
>> Peking University, China
>> 
>> ------------------------------------------------------------------------------
>> This SF email is sponsosred by:
>> Try Windows Azure free for 90 days Click Here 
>> http://p.sf.net/sfu/sfd2d-msazure_______________________________________________
>> Umit-devel mailing list
>> Umit-devel@...
>> https://lists.sourceforge.net/lists/listinfo/umit-devel
> 
> 
> 
> 
> -- 
> Zhongjie Wang
> Master Candidate
> Computer System Architecture
> Peking University, China
> 
> ------------------------------------------------------------------------------
> This SF email is sponsosred by:
> Try Windows Azure free for 90 days Click Here
> http://p.sf.net/sfu/sfd2d-msazure
> _______________________________________________
> Umit-devel mailing list
> Umit-devel@...
> https://lists.sourceforge.net/lists/listinfo/umit-devel
> 
> 
> 
> 
> -- 
> Adriano Monteiro Marques
> 
> http://www.openmonitor.org
> http://www.umitproject.org
> http://www.thoughtspad.com
> 
> "Don't stay in bed, unless you can make money in bed." - George Burns
> 
> 
>

Native extensions in Android are provided by the Android NDK feature which
comes bundled with Android OS. To be more specific, its the same JNI
implementation as we have in Java. So it does not require root access.

But am afraid it might lead to more complexity in the future. Now I
understand why this issue is puzzling everyone :) In future when iOS
support comes in, the logic should be usable there too. As far as I could
see, native extensions can provide that flexibility.

Thanks,
Narendran.


On Tue, Mar 27, 2012 at 9:01 PM, Zubair Nabi <zn.zubairnabi@...:

> In addition, if we native-extensions on the mobile agent, will that
> require root access? If yes, then I don't think it's a feasible idea.
> That's actually one of the reasons, we didn't use traceroute last year.
>
> Best,
> Zubair
>
>
>
> On 27 Mar 2012, at 16:26, Narendran Thangarajan wrote:
>
> Let me first look into the feasibility of using maidsafe-dht as a native
> extension with python. Only then we could know the next step. I would love
> to hear suggestions in this regard. Should I look into any other specific
> implementation? Or has anyone done a study on  this already?
>
> Thanks,
> Narendran.
>
> On Mon, Mar 26, 2012 at 9:17 PM, Narendran Thangarajan <
> narendran.thangarajan@...> wrote:
>
>> The three major python implementations of Kad are Khashmir, SharkyPy and
>> Entangled. However, all three do not mention about NAT traversal. Even if
>> we had such an UPnP enabled python implementation, we must look into its
>> compatibility with our mobile agent too. And as far as I have seen there
>> are no Android alternatives. And making two bytecode oriented languages
>> talk with each other is much worse.
>> The best thing is to go for a NAT traversal enabled C++ implementation
>> (say maidsafe-dht) connected to our desktop agent (Python) and mobile
>> agent(Android-java) through native extensions. Though it looks little tough
>> at the first sight, this gives great flexibility. And since the base
>> implementation is native C++, its going to be faster too.
>>
>>
>> On Mon, Mar 26, 2012 at 4:09 PM, Adriano Marques <py.adriano@...:
>>
>>> This turned out into a good discussion, and I definitely want to see on
>>> the proposal how students wants to tackle this challenge :)
>>>
>>>
>>> On Sun, Mar 25, 2012 at 11:43 PM, Zhongjie Wang <wzj401@...:
>>>
>>>> Hi guys,
>>>>         Yeah, DHT is an option and we may also need to consider those
>>>> nodes behind NATs. So I recommend we use the super agents as trackers. The
>>>> super agents are those peers who have public IPs and could surely connect
>>>> to the aggregator. They can help with forwarding messages between nodes,
>>>> and also if an agent can connect to a super agent, then it messages can be
>>>> sent to the aggregator. Kad is using some protocol like gossip, we may also
>>>> bring this idea in.
>>>>
>>>> Regards.
>>>> Alan
>>>>
>>>>
>>>> On Mon, Mar 26, 2012 at 4:02 AM, Zubair Nabi <zn.zubairnabi@...:
>>>>
>>>>> Guys,
>>>>>
>>>>> You also need to consider the mobile agent. The same DHT algorithm
>>>>> needs to be applicable to the mobile agent as well. First, and foremost, we
>>>>> need to come up with a solution for pure P2P communication assuming that
>>>>> both peers are behind a NAT. One option is to use tcp hole punching. In
>>>>> such an architecture, can some of the desktop agents --with public IPs --
>>>>> serve as relay servers between other desktop agents and mobile agents? This
>>>>> was a major impediment that we ran into last year. I know of a couple of
>>>>> Kademlia libraries in C++ and Java that have UPnP-enabled NAT traversal. Do
>>>>> the Python implementations consider NAT traversal? In addition, for people
>>>>> interested in working on the mobile agent, are there any Android Java
>>>>> libraries to this end?
>>>>>
>>>>>  Best,
>>>>> Zubair
>>>>>
>>>>>
>>>>>
>>>>> On 25 Mar 2012, at 20:48, Narendran Thangarajan wrote:
>>>>>
>>>>> Yeah I get it. So we need to implement a P2P routing overlay so that
>>>>> Peers can form clusters among themselves. There are a horde of options like
>>>>> Gnutella, DHT, Semantic routing etc. Since we are focusing on a global
>>>>> scale, I feel the best option is to go for Distributed hash tables
>>>>> implementation. It is fast and doesn't flood the network with redundant
>>>>> requests. Moreover, the system gets decentralized, thus it is easy to scale
>>>>> on demand.
>>>>>
>>>>> Among the DHT algorithms, I feel that making a Kademlia based
>>>>> implementation of DHT is a more viable option in python. Added advantage is
>>>>> that, certain Kademlia implementations provide out-of-the-box visualizers
>>>>> for monitoring all the peers. This will greatly enhance the user experience.
>>>>>
>>>>> Thanks,
>>>>> Narendran Thangarajan.
>>>>>
>>>>>
>>>>> On Sun, Mar 25, 2012 at 7:16 PM, Zhongjie Wang <wzj401@...:
>>>>>
>>>>>> Hi Narendran,
>>>>>>       Actually there's not a notification system yet. I know you mean
>>>>>> to
>>>>>> implement a notification system at the agent side. The method we are
>>>>>> using now is periodically checking the aggregator. The agent sends
>>>>>> GetEvent messages to the aggregator and gets the events happened
>>>>>> in its region or the global Internet. The notification system might
>>>>>> not be
>>>>>> a quite urgent job for now. We can add some code to pop-up a small
>>>>>> window for notification.
>>>>>>       I think the routing strategy is a problem needs to be solved.
>>>>>> Also
>>>>>> we need to figure out how to form a stable peer network, in which the
>>>>>> peers could dynamically gather into clusters, and bypass the blockage.
>>>>>> Currently the aggregator url and super agent addresses (the entrance
>>>>>> for a new agent) are written into the db of the agent statically.
>>>>>> We still need to think out more ideas for a new peer to find
>>>>>> its neighbors
>>>>>> and the super agents. Although we are using a cloud-based aggregator
>>>>>>  to scale our service, we are building a huge network covering the
>>>>>> whole
>>>>>> world! So management of peers should be decentralized, by the peers
>>>>>>  themselves or the super agents. Remember the essential task is to
>>>>>> spread the information in a network with blockage. :)
>>>>>>
>>>>>> Best Regards.
>>>>>> Alan
>>>>>>
>>>>>>  On Sun, Mar 25, 2012 at 12:01 PM, Narendran Thangarajan <
>>>>>> narendran.thangarajan@...> wrote:
>>>>>>
>>>>>>>  Hi,
>>>>>>>  I looked into the reporting mechanism in ICM-desktop agent. I see
>>>>>>> that once the agent logs in, there are three looping calls that
>>>>>>> perform the following :
>>>>>>>   1. Get peer list and populate the peerinfo.
>>>>>>>   2. Schedule tasks and add reports
>>>>>>>   3. Upload the generated reports.
>>>>>>>
>>>>>>>  But I can't find logic, how the peers make use of the
>>>>>>> Website/service reports generated from other peers or aggregator. If
>>>>>>> its available, where should I find the logic? If not, I would like to
>>>>>>> implement a notification system, which notifies the peer user of any
>>>>>>> network / website outages. This tool will be really helpful for
>>>>>>> admins/end users at various scales (On a global level or within a
>>>>>>> data
>>>>>>> center - thanks to the flexible design of the OpenMonitor system).
>>>>>>> This can be implemented using an evented I/O (say gevent).
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Narendran.
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------------------------------------------------------
>>>>>>> This SF email is sponsosred by:
>>>>>>> Try Windows Azure free for 90 days Click Here
>>>>>>> http://p.sf.net/sfu/sfd2d-msazure
>>>>>>> _______________________________________________
>>>>>>> Umit-devel mailing list
>>>>>>> Umit-devel@...
>>>>>>> https://lists.sourceforge.net/lists/listinfo/umit-devel
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Zhongjie Wang
>>>>>> Master Candidate
>>>>>> Computer System Architecture
>>>>>> Peking University, China
>>>>>>
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------------
>>>>> This SF email is sponsosred by:
>>>>> Try Windows Azure free for 90 days Click Here
>>>>>
>>>>> http://p.sf.net/sfu/sfd2d-msazure_______________________________________________
>>>>> Umit-devel mailing list
>>>>> Umit-devel@...
>>>>> https://lists.sourceforge.net/lists/listinfo/umit-devel
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Zhongjie Wang
>>>> Master Candidate
>>>> Computer System Architecture
>>>> Peking University, China
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> This SF email is sponsosred by:
>>>> Try Windows Azure free for 90 days Click Here
>>>> http://p.sf.net/sfu/sfd2d-msazure
>>>> _______________________________________________
>>>> Umit-devel mailing list
>>>> Umit-devel@...
>>>> https://lists.sourceforge.net/lists/listinfo/umit-devel
>>>>
>>>>
>>>
>>>
>>> --
>>> Adriano Monteiro Marques
>>>
>>> http://www.openmonitor.org
>>> http://www.umitproject.org
>>> http://www.thoughtspad.com
>>>
>>> "Don't stay in bed, unless you can make money in bed." - George Burns
>>>
>>>
>>
>
>

That is true. Incidentally, TomP2P (http://tomp2p.net/) seems to be a reasonable option for the mobile agent and could possibly be called from within Python through bindings.

Best,
Zubair



On 27 Mar 2012, at 19:48, Narendran Thangarajan wrote:

> Native extensions in Android are provided by the Android NDK feature which comes bundled with Android OS. To be more specific, its the same JNI implementation as we have in Java. So it does not require root access. 
> 
> But am afraid it might lead to more complexity in the future. Now I understand why this issue is puzzling everyone :) In future when iOS support comes in, the logic should be usable there too. As far as I could see, native extensions can provide that flexibility.
> 
> Thanks,
> Narendran.
> 
> 
> On Tue, Mar 27, 2012 at 9:01 PM, Zubair Nabi <zn.zubairnabi@...> wrote:
> In addition, if we native-extensions on the mobile agent, will that require root access? If yes, then I don't think it's a feasible idea. That's actually one of the reasons, we didn't use traceroute last year.
> 
> Best,
> Zubair
> 
> 
> 
> On 27 Mar 2012, at 16:26, Narendran Thangarajan wrote:
> 
>> Let me first look into the feasibility of using maidsafe-dht as a native extension with python. Only then we could know the next step. I would love to hear suggestions in this regard. Should I look into any other specific implementation? Or has anyone done a study on  this already?
>> 
>> Thanks,
>> Narendran.
>> 
>> On Mon, Mar 26, 2012 at 9:17 PM, Narendran Thangarajan <narendran.thangarajan@...> wrote:
>> The three major python implementations of Kad are Khashmir, SharkyPy and Entangled. However, all three do not mention about NAT traversal. Even if we had such an UPnP enabled python implementation, we must look into its compatibility with our mobile agent too. And as far as I have seen there are no Android alternatives. And making two bytecode oriented languages talk with each other is much worse.
>> The best thing is to go for a NAT traversal enabled C++ implementation (say maidsafe-dht) connected to our desktop agent (Python) and mobile agent(Android-java) through native extensions. Though it looks little tough at the first sight, this gives great flexibility. And since the base implementation is native C++, its going to be faster too. 
>> 
>> 
>> On Mon, Mar 26, 2012 at 4:09 PM, Adriano Marques <py.adriano@...> wrote:
>> This turned out into a good discussion, and I definitely want to see on the proposal how students wants to tackle this challenge :)
>> 
>> 
>> On Sun, Mar 25, 2012 at 11:43 PM, Zhongjie Wang <wzj401@...> wrote:
>> Hi guys,
>>         Yeah, DHT is an option and we may also need to consider those nodes behind NATs. So I recommend we use the super agents as trackers. The super agents are those peers who have public IPs and could surely connect to the aggregator. They can help with forwarding messages between nodes, and also if an agent can connect to a super agent, then it messages can be sent to the aggregator. Kad is using some protocol like gossip, we may also bring this idea in. 
>> 
>> Regards.
>> Alan
>> 
>> 
>> On Mon, Mar 26, 2012 at 4:02 AM, Zubair Nabi <zn.zubairnabi@...> wrote:
>> Guys, 
>> 
>> You also need to consider the mobile agent. The same DHT algorithm needs to be applicable to the mobile agent as well. First, and foremost, we need to come up with a solution for pure P2P communication assuming that both peers are behind a NAT. One option is to use tcp hole punching. In such an architecture, can some of the desktop agents --with public IPs -- serve as relay servers between other desktop agents and mobile agents? This was a major impediment that we ran into last year. I know of a couple of Kademlia libraries in C++ and Java that have UPnP-enabled NAT traversal. Do the Python implementations consider NAT traversal? In addition, for people interested in working on the mobile agent, are there any Android Java libraries to this end?
>> 
>> Best,
>> Zubair
>> 
>> 
>> 
>> On 25 Mar 2012, at 20:48, Narendran Thangarajan wrote:
>> 
>>> Yeah I get it. So we need to implement a P2P routing overlay so that Peers can form clusters among themselves. There are a horde of options like Gnutella, DHT, Semantic routing etc. Since we are focusing on a global scale, I feel the best option is to go for Distributed hash tables implementation. It is fast and doesn't flood the network with redundant requests. Moreover, the system gets decentralized, thus it is easy to scale on demand.
>>> 
>>> Among the DHT algorithms, I feel that making a Kademlia based implementation of DHT is a more viable option in python. Added advantage is that, certain Kademlia implementations provide out-of-the-box visualizers for monitoring all the peers. This will greatly enhance the user experience.
>>> 
>>> Thanks,
>>> Narendran Thangarajan.
>>> 
>>> 
>>> On Sun, Mar 25, 2012 at 7:16 PM, Zhongjie Wang <wzj401@...> wrote:
>>> Hi Narendran,
>>>       Actually there's not a notification system yet. I know you mean to 
>>> implement a notification system at the agent side. The method we are
>>> using now is periodically checking the aggregator. The agent sends 
>>> GetEvent messages to the aggregator and gets the events happened 
>>> in its region or the global Internet. The notification system might not be
>>> a quite urgent job for now. We can add some code to pop-up a small 
>>> window for notification. 
>>>       I think the routing strategy is a problem needs to be solved. Also 
>>> we need to figure out how to form a stable peer network, in which the 
>>> peers could dynamically gather into clusters, and bypass the blockage.
>>> Currently the aggregator url and super agent addresses (the entrance 
>>> for a new agent) are written into the db of the agent statically. 
>>> We still need to think out more ideas for a new peer to find its neighbors 
>>> and the super agents. Although we are using a cloud-based aggregator
>>>  to scale our service, we are building a huge network covering the whole 
>>> world! So management of peers should be decentralized, by the peers
>>>  themselves or the super agents. Remember the essential task is to 
>>> spread the information in a network with blockage. :)
>>> 
>>> Best Regards.
>>> Alan
>>> 
>>> On Sun, Mar 25, 2012 at 12:01 PM, Narendran Thangarajan <narendran.thangarajan@...> wrote:
>>> Hi,
>>>  I looked into the reporting mechanism in ICM-desktop agent. I see
>>> that once the agent logs in, there are three looping calls that
>>> perform the following :
>>>   1. Get peer list and populate the peerinfo.
>>>   2. Schedule tasks and add reports
>>>   3. Upload the generated reports.
>>> 
>>>  But I can't find logic, how the peers make use of the
>>> Website/service reports generated from other peers or aggregator. If
>>> its available, where should I find the logic? If not, I would like to
>>> implement a notification system, which notifies the peer user of any
>>> network / website outages. This tool will be really helpful for
>>> admins/end users at various scales (On a global level or within a data
>>> center - thanks to the flexible design of the OpenMonitor system).
>>> This can be implemented using an evented I/O (say gevent).
>>> 
>>> Thanks,
>>> Narendran.
>>> 
>>> ------------------------------------------------------------------------------
>>> This SF email is sponsosred by:
>>> Try Windows Azure free for 90 days Click Here
>>> http://p.sf.net/sfu/sfd2d-msazure
>>> _______________________________________________
>>> Umit-devel mailing list
>>> Umit-devel@...
>>> https://lists.sourceforge.net/lists/listinfo/umit-devel
>>> 
>>> 
>>> 
>>> -- 
>>> Zhongjie Wang
>>> Master Candidate
>>> Computer System Architecture
>>> Peking University, China
>>> 
>>> ------------------------------------------------------------------------------
>>> This SF email is sponsosred by:
>>> Try Windows Azure free for 90 days Click Here 
>>> http://p.sf.net/sfu/sfd2d-msazure_______________________________________________
>>> Umit-devel mailing list
>>> Umit-devel@...
>>> https://lists.sourceforge.net/lists/listinfo/umit-devel
>> 
>> 
>> 
>> 
>> -- 
>> Zhongjie Wang
>> Master Candidate
>> Computer System Architecture
>> Peking University, China
>> 
>> ------------------------------------------------------------------------------
>> This SF email is sponsosred by:
>> Try Windows Azure free for 90 days Click Here
>> http://p.sf.net/sfu/sfd2d-msazure
>> _______________________________________________
>> Umit-devel mailing list
>> Umit-devel@...
>> https://lists.sourceforge.net/lists/listinfo/umit-devel
>> 
>> 
>> 
>> 
>> -- 
>> Adriano Monteiro Marques
>> 
>> http://www.openmonitor.org
>> http://www.umitproject.org
>> http://www.thoughtspad.com
>> 
>> "Don't stay in bed, unless you can make money in bed." - George Burns
>> 
>> 
>> 
> 
>

Thanks a lot Zubair. I have looked into TomP2P. The great part is that it
has NAT traversal and runs on Android without much reconfiguration. Link :
http://tomp2p.net/doc/advanced/android.html
For the desktop agent, the solution is Jython using which we can invoke
Java methods from Python.

So my proposal is to add two new features:
1. Implement inter-peer communication (both desktop and mobile agent) using
TomP2P. (And of course the important part - writing test cases to test all
available usecases)
2. Modify the clustering algorithm of TomP2P so that the root of the
cluster is the Super Peer, thus keeping all the peers under it in sync with
each other.

I am new to Jython. But I have immense interest to learn new concepts and I
have started learning it already. Any suggestion from the mentors would
help me improve :)

Thanks,
Narendran.

On Wed, Mar 28, 2012 at 3:08 AM, Zubair Nabi <zn.zubairnabi@...:

> That is true. Incidentally, TomP2P (http://tomp2p.net/) seems to be a
> reasonable option for the mobile agent and could possibly be called from
> within Python through bindings.
>
> Best,
> Zubair
>
>
>
> On 27 Mar 2012, at 19:48, Narendran Thangarajan wrote:
>
> Native extensions in Android are provided by the Android NDK feature which
> comes bundled with Android OS. To be more specific, its the same JNI
> implementation as we have in Java. So it does not require root access.
>
> But am afraid it might lead to more complexity in the future. Now I
> understand why this issue is puzzling everyone :) In future when iOS
> support comes in, the logic should be usable there too. As far as I could
> see, native extensions can provide that flexibility.
>
> Thanks,
> Narendran.
>
>
> On Tue, Mar 27, 2012 at 9:01 PM, Zubair Nabi <zn.zubairnabi@...:
>
>> In addition, if we native-extensions on the mobile agent, will that
>> require root access? If yes, then I don't think it's a feasible idea.
>> That's actually one of the reasons, we didn't use traceroute last year.
>>
>>  Best,
>> Zubair
>>
>>
>>
>> On 27 Mar 2012, at 16:26, Narendran Thangarajan wrote:
>>
>> Let me first look into the feasibility of using maidsafe-dht as a native
>> extension with python. Only then we could know the next step. I would love
>> to hear suggestions in this regard. Should I look into any other specific
>> implementation? Or has anyone done a study on  this already?
>>
>> Thanks,
>> Narendran.
>>
>> On Mon, Mar 26, 2012 at 9:17 PM, Narendran Thangarajan <
>> narendran.thangarajan@...> wrote:
>>
>>> The three major python implementations of Kad are Khashmir, SharkyPy and
>>> Entangled. However, all three do not mention about NAT traversal. Even if
>>> we had such an UPnP enabled python implementation, we must look into its
>>> compatibility with our mobile agent too. And as far as I have seen there
>>> are no Android alternatives. And making two bytecode oriented languages
>>> talk with each other is much worse.
>>> The best thing is to go for a NAT traversal enabled C++ implementation
>>> (say maidsafe-dht) connected to our desktop agent (Python) and mobile
>>> agent(Android-java) through native extensions. Though it looks little tough
>>> at the first sight, this gives great flexibility. And since the base
>>> implementation is native C++, its going to be faster too.
>>>
>>>
>>> On Mon, Mar 26, 2012 at 4:09 PM, Adriano Marques <py.adriano@...:
>>>
>>>> This turned out into a good discussion, and I definitely want to see on
>>>> the proposal how students wants to tackle this challenge :)
>>>>
>>>>
>>>> On Sun, Mar 25, 2012 at 11:43 PM, Zhongjie Wang <wzj401@...:
>>>>
>>>>> Hi guys,
>>>>>         Yeah, DHT is an option and we may also need to consider those
>>>>> nodes behind NATs. So I recommend we use the super agents as trackers. The
>>>>> super agents are those peers who have public IPs and could surely connect
>>>>> to the aggregator. They can help with forwarding messages between nodes,
>>>>> and also if an agent can connect to a super agent, then it messages can be
>>>>> sent to the aggregator. Kad is using some protocol like gossip, we may also
>>>>> bring this idea in.
>>>>>
>>>>> Regards.
>>>>> Alan
>>>>>
>>>>>
>>>>> On Mon, Mar 26, 2012 at 4:02 AM, Zubair Nabi <zn.zubairnabi@...:
>>>>>
>>>>>> Guys,
>>>>>>
>>>>>> You also need to consider the mobile agent. The same DHT algorithm
>>>>>> needs to be applicable to the mobile agent as well. First, and foremost, we
>>>>>> need to come up with a solution for pure P2P communication assuming that
>>>>>> both peers are behind a NAT. One option is to use tcp hole punching. In
>>>>>> such an architecture, can some of the desktop agents --with public IPs --
>>>>>> serve as relay servers between other desktop agents and mobile agents? This
>>>>>> was a major impediment that we ran into last year. I know of a couple of
>>>>>> Kademlia libraries in C++ and Java that have UPnP-enabled NAT traversal. Do
>>>>>> the Python implementations consider NAT traversal? In addition, for people
>>>>>> interested in working on the mobile agent, are there any Android Java
>>>>>> libraries to this end?
>>>>>>
>>>>>>  Best,
>>>>>> Zubair
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 25 Mar 2012, at 20:48, Narendran Thangarajan wrote:
>>>>>>
>>>>>> Yeah I get it. So we need to implement a P2P routing overlay so that
>>>>>> Peers can form clusters among themselves. There are a horde of options like
>>>>>> Gnutella, DHT, Semantic routing etc. Since we are focusing on a global
>>>>>> scale, I feel the best option is to go for Distributed hash tables
>>>>>> implementation. It is fast and doesn't flood the network with redundant
>>>>>> requests. Moreover, the system gets decentralized, thus it is easy to scale
>>>>>> on demand.
>>>>>>
>>>>>> Among the DHT algorithms, I feel that making a Kademlia based
>>>>>> implementation of DHT is a more viable option in python. Added advantage is
>>>>>> that, certain Kademlia implementations provide out-of-the-box visualizers
>>>>>> for monitoring all the peers. This will greatly enhance the user experience.
>>>>>>
>>>>>> Thanks,
>>>>>> Narendran Thangarajan.
>>>>>>
>>>>>>
>>>>>> On Sun, Mar 25, 2012 at 7:16 PM, Zhongjie Wang <wzj401@...:
>>>>>>
>>>>>>> Hi Narendran,
>>>>>>>       Actually there's not a notification system yet. I know you
>>>>>>> mean to
>>>>>>> implement a notification system at the agent side. The method we are
>>>>>>> using now is periodically checking the aggregator. The agent sends
>>>>>>> GetEvent messages to the aggregator and gets the events happened
>>>>>>> in its region or the global Internet. The notification system might
>>>>>>> not be
>>>>>>> a quite urgent job for now. We can add some code to pop-up a small
>>>>>>> window for notification.
>>>>>>>       I think the routing strategy is a problem needs to be solved.
>>>>>>> Also
>>>>>>> we need to figure out how to form a stable peer network, in which
>>>>>>> the
>>>>>>> peers could dynamically gather into clusters, and bypass the
>>>>>>> blockage.
>>>>>>> Currently the aggregator url and super agent addresses (the entrance
>>>>>>> for a new agent) are written into the db of the agent statically.
>>>>>>> We still need to think out more ideas for a new peer to find
>>>>>>> its neighbors
>>>>>>> and the super agents. Although we are using a cloud-based aggregator
>>>>>>>  to scale our service, we are building a huge network covering the
>>>>>>> whole
>>>>>>> world! So management of peers should be decentralized, by the peers
>>>>>>>  themselves or the super agents. Remember the essential task is to
>>>>>>> spread the information in a network with blockage. :)
>>>>>>>
>>>>>>> Best Regards.
>>>>>>> Alan
>>>>>>>
>>>>>>>  On Sun, Mar 25, 2012 at 12:01 PM, Narendran Thangarajan <
>>>>>>> narendran.thangarajan@...> wrote:
>>>>>>>
>>>>>>>>  Hi,
>>>>>>>>  I looked into the reporting mechanism in ICM-desktop agent. I see
>>>>>>>> that once the agent logs in, there are three looping calls that
>>>>>>>> perform the following :
>>>>>>>>   1. Get peer list and populate the peerinfo.
>>>>>>>>   2. Schedule tasks and add reports
>>>>>>>>   3. Upload the generated reports.
>>>>>>>>
>>>>>>>>  But I can't find logic, how the peers make use of the
>>>>>>>> Website/service reports generated from other peers or aggregator. If
>>>>>>>> its available, where should I find the logic? If not, I would like
>>>>>>>> to
>>>>>>>> implement a notification system, which notifies the peer user of any
>>>>>>>> network / website outages. This tool will be really helpful for
>>>>>>>> admins/end users at various scales (On a global level or within a
>>>>>>>> data
>>>>>>>> center - thanks to the flexible design of the OpenMonitor system).
>>>>>>>> This can be implemented using an evented I/O (say gevent).
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Narendran.
>>>>>>>>
>>>>>>>>
>>>>>>>> ------------------------------------------------------------------------------
>>>>>>>> This SF email is sponsosred by:
>>>>>>>> Try Windows Azure free for 90 days Click Here
>>>>>>>> http://p.sf.net/sfu/sfd2d-msazure
>>>>>>>> _______________________________________________
>>>>>>>> Umit-devel mailing list
>>>>>>>> Umit-devel@...
>>>>>>>> https://lists.sourceforge.net/lists/listinfo/umit-devel
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Zhongjie Wang
>>>>>>> Master Candidate
>>>>>>> Computer System Architecture
>>>>>>> Peking University, China
>>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------------
>>>>>> This SF email is sponsosred by:
>>>>>> Try Windows Azure free for 90 days Click Here
>>>>>>
>>>>>> http://p.sf.net/sfu/sfd2d-msazure_______________________________________________
>>>>>> Umit-devel mailing list
>>>>>> Umit-devel@...
>>>>>> https://lists.sourceforge.net/lists/listinfo/umit-devel
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Zhongjie Wang
>>>>> Master Candidate
>>>>> Computer System Architecture
>>>>> Peking University, China
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------------
>>>>> This SF email is sponsosred by:
>>>>> Try Windows Azure free for 90 days Click Here
>>>>> http://p.sf.net/sfu/sfd2d-msazure
>>>>> _______________________________________________
>>>>> Umit-devel mailing list
>>>>> Umit-devel@...
>>>>> https://lists.sourceforge.net/lists/listinfo/umit-devel
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Adriano Monteiro Marques
>>>>
>>>> http://www.openmonitor.org
>>>> http://www.umitproject.org
>>>> http://www.thoughtspad.com
>>>>
>>>> "Don't stay in bed, unless you can make money in bed." - George Burns
>>>>
>>>>
>>>
>>
>>
>
>

Perfect. That's sounds good. Just add specific details to your proposal. For example, how would you incorporate the current message types that we have, into the overlay network.

Best,
Zubair



On 31 Mar 2012, at 16:16, Narendran Thangarajan wrote:

> Thanks a lot Zubair. I have looked into TomP2P. The great part is that it has NAT traversal and runs on Android without much reconfiguration. Link :http://tomp2p.net/doc/advanced/android.html
> For the desktop agent, the solution is Jython using which we can invoke Java methods from Python. 
> 
> So my proposal is to add two new features:
> 1. Implement inter-peer communication (both desktop and mobile agent) using TomP2P. (And of course the important part - writing test cases to test all available usecases)
> 2. Modify the clustering algorithm of TomP2P so that the root of the cluster is the Super Peer, thus keeping all the peers under it in sync with each other. 
> 
> I am new to Jython. But I have immense interest to learn new concepts and I have started learning it already. Any suggestion from the mentors would help me improve :)
> 
> Thanks,
> Narendran.
> 
> On Wed, Mar 28, 2012 at 3:08 AM, Zubair Nabi <zn.zubairnabi@...> wrote:
> That is true. Incidentally, TomP2P (http://tomp2p.net/) seems to be a reasonable option for the mobile agent and could possibly be called from within Python through bindings.
> 
> Best,
> Zubair
> 
> 
> 
> On 27 Mar 2012, at 19:48, Narendran Thangarajan wrote:
> 
>> Native extensions in Android are provided by the Android NDK feature which comes bundled with Android OS. To be more specific, its the same JNI implementation as we have in Java. So it does not require root access. 
>> 
>> But am afraid it might lead to more complexity in the future. Now I understand why this issue is puzzling everyone :) In future when iOS support comes in, the logic should be usable there too. As far as I could see, native extensions can provide that flexibility.
>> 
>> Thanks,
>> Narendran.
>> 
>> 
>> On Tue, Mar 27, 2012 at 9:01 PM, Zubair Nabi <zn.zubairnabi@...> wrote:
>> In addition, if we native-extensions on the mobile agent, will that require root access? If yes, then I don't think it's a feasible idea. That's actually one of the reasons, we didn't use traceroute last year.
>> 
>> Best,
>> Zubair
>> 
>> 
>> 
>> On 27 Mar 2012, at 16:26, Narendran Thangarajan wrote:
>> 
>>> Let me first look into the feasibility of using maidsafe-dht as a native extension with python. Only then we could know the next step. I would love to hear suggestions in this regard. Should I look into any other specific implementation? Or has anyone done a study on  this already?
>>> 
>>> Thanks,
>>> Narendran.
>>> 
>>> On Mon, Mar 26, 2012 at 9:17 PM, Narendran Thangarajan <narendran.thangarajan@...> wrote:
>>> The three major python implementations of Kad are Khashmir, SharkyPy and Entangled. However, all three do not mention about NAT traversal. Even if we had such an UPnP enabled python implementation, we must look into its compatibility with our mobile agent too. And as far as I have seen there are no Android alternatives. And making two bytecode oriented languages talk with each other is much worse.
>>> The best thing is to go for a NAT traversal enabled C++ implementation (say maidsafe-dht) connected to our desktop agent (Python) and mobile agent(Android-java) through native extensions. Though it looks little tough at the first sight, this gives great flexibility. And since the base implementation is native C++, its going to be faster too. 
>>> 
>>> 
>>> On Mon, Mar 26, 2012 at 4:09 PM, Adriano Marques <py.adriano@...> wrote:
>>> This turned out into a good discussion, and I definitely want to see on the proposal how students wants to tackle this challenge :)
>>> 
>>> 
>>> On Sun, Mar 25, 2012 at 11:43 PM, Zhongjie Wang <wzj401@...> wrote:
>>> Hi guys,
>>>         Yeah, DHT is an option and we may also need to consider those nodes behind NATs. So I recommend we use the super agents as trackers. The super agents are those peers who have public IPs and could surely connect to the aggregator. They can help with forwarding messages between nodes, and also if an agent can connect to a super agent, then it messages can be sent to the aggregator. Kad is using some protocol like gossip, we may also bring this idea in. 
>>> 
>>> Regards.
>>> Alan
>>> 
>>> 
>>> On Mon, Mar 26, 2012 at 4:02 AM, Zubair Nabi <zn.zubairnabi@...> wrote:
>>> Guys, 
>>> 
>>> You also need to consider the mobile agent. The same DHT algorithm needs to be applicable to the mobile agent as well. First, and foremost, we need to come up with a solution for pure P2P communication assuming that both peers are behind a NAT. One option is to use tcp hole punching. In such an architecture, can some of the desktop agents --with public IPs -- serve as relay servers between other desktop agents and mobile agents? This was a major impediment that we ran into last year. I know of a couple of Kademlia libraries in C++ and Java that have UPnP-enabled NAT traversal. Do the Python implementations consider NAT traversal? In addition, for people interested in working on the mobile agent, are there any Android Java libraries to this end?
>>> 
>>> Best,
>>> Zubair
>>> 
>>> 
>>> 
>>> On 25 Mar 2012, at 20:48, Narendran Thangarajan wrote:
>>> 
>>>> Yeah I get it. So we need to implement a P2P routing overlay so that Peers can form clusters among themselves. There are a horde of options like Gnutella, DHT, Semantic routing etc. Since we are focusing on a global scale, I feel the best option is to go for Distributed hash tables implementation. It is fast and doesn't flood the network with redundant requests. Moreover, the system gets decentralized, thus it is easy to scale on demand.
>>>> 
>>>> Among the DHT algorithms, I feel that making a Kademlia based implementation of DHT is a more viable option in python. Added advantage is that, certain Kademlia implementations provide out-of-the-box visualizers for monitoring all the peers. This will greatly enhance the user experience.
>>>> 
>>>> Thanks,
>>>> Narendran Thangarajan.
>>>> 
>>>> 
>>>> On Sun, Mar 25, 2012 at 7:16 PM, Zhongjie Wang <wzj401@...> wrote:
>>>> Hi Narendran,
>>>>       Actually there's not a notification system yet. I know you mean to 
>>>> implement a notification system at the agent side. The method we are
>>>> using now is periodically checking the aggregator. The agent sends 
>>>> GetEvent messages to the aggregator and gets the events happened 
>>>> in its region or the global Internet. The notification system might not be
>>>> a quite urgent job for now. We can add some code to pop-up a small 
>>>> window for notification. 
>>>>       I think the routing strategy is a problem needs to be solved. Also 
>>>> we need to figure out how to form a stable peer network, in which the 
>>>> peers could dynamically gather into clusters, and bypass the blockage.
>>>> Currently the aggregator url and super agent addresses (the entrance 
>>>> for a new agent) are written into the db of the agent statically. 
>>>> We still need to think out more ideas for a new peer to find its neighbors 
>>>> and the super agents. Although we are using a cloud-based aggregator
>>>>  to scale our service, we are building a huge network covering the whole 
>>>> world! So management of peers should be decentralized, by the peers
>>>>  themselves or the super agents. Remember the essential task is to 
>>>> spread the information in a network with blockage. :)
>>>> 
>>>> Best Regards.
>>>> Alan
>>>> 
>>>> On Sun, Mar 25, 2012 at 12:01 PM, Narendran Thangarajan <narendran.thangarajan@...> wrote:
>>>> Hi,
>>>>  I looked into the reporting mechanism in ICM-desktop agent. I see
>>>> that once the agent logs in, there are three looping calls that
>>>> perform the following :
>>>>   1. Get peer list and populate the peerinfo.
>>>>   2. Schedule tasks and add reports
>>>>   3. Upload the generated reports.
>>>> 
>>>>  But I can't find logic, how the peers make use of the
>>>> Website/service reports generated from other peers or aggregator. If
>>>> its available, where should I find the logic? If not, I would like to
>>>> implement a notification system, which notifies the peer user of any
>>>> network / website outages. This tool will be really helpful for
>>>> admins/end users at various scales (On a global level or within a data
>>>> center - thanks to the flexible design of the OpenMonitor system).
>>>> This can be implemented using an evented I/O (say gevent).
>>>> 
>>>> Thanks,
>>>> Narendran.
>>>> 
>>>> ------------------------------------------------------------------------------
>>>> This SF email is sponsosred by:
>>>> Try Windows Azure free for 90 days Click Here
>>>> http://p.sf.net/sfu/sfd2d-msazure
>>>> _______________________________________________
>>>> Umit-devel mailing list
>>>> Umit-devel@...
>>>> https://lists.sourceforge.net/lists/listinfo/umit-devel
>>>> 
>>>> 
>>>> 
>>>> -- 
>>>> Zhongjie Wang
>>>> Master Candidate
>>>> Computer System Architecture
>>>> Peking University, China
>>>> 
>>>> ------------------------------------------------------------------------------
>>>> This SF email is sponsosred by:
>>>> Try Windows Azure free for 90 days Click Here 
>>>> http://p.sf.net/sfu/sfd2d-msazure_______________________________________________
>>>> Umit-devel mailing list
>>>> Umit-devel@...
>>>> https://lists.sourceforge.net/lists/listinfo/umit-devel
>>> 
>>> 
>>> 
>>> 
>>> -- 
>>> Zhongjie Wang
>>> Master Candidate
>>> Computer System Architecture
>>> Peking University, China
>>> 
>>> ------------------------------------------------------------------------------
>>> This SF email is sponsosred by:
>>> Try Windows Azure free for 90 days Click Here
>>> http://p.sf.net/sfu/sfd2d-msazure
>>> _______________________________________________
>>> Umit-devel mailing list
>>> Umit-devel@...
>>> https://lists.sourceforge.net/lists/listinfo/umit-devel
>>> 
>>> 
>>> 
>>> 
>>> -- 
>>> Adriano Monteiro Marques
>>> 
>>> http://www.openmonitor.org
>>> http://www.umitproject.org
>>> http://www.thoughtspad.com
>>> 
>>> "Don't stay in bed, unless you can make money in bed." - George Burns
>>> 
>>> 
>>> 
>> 
>> 
> 
>