[ANet-devel] Enigma

[Benoit N. <be...@ma...>]

I'm sure you are wondering what happens between the time I have an idea
about ANet and when I start writing the doc. Well, you *have* to wonder,
because it takes weeks before I update the web site, or you think I'm
faking having ideas and I stopped working on the project.

Here's below how I write down my "ideas". If you compare my high-level
design with the drawing I did, you'll see some important changes. BTW, "A
picture is worth a thousand word" my a**... I'll remove the
already-outdated drawing and I'll just write.

As you can see, it is pretty cryptic (hence the subject of this email), but
you can start figuring out what is going on... and how it *really* can
work, not just promises. I say that because I read an interview with a guy
that wanted to make "Gnutella 2" in January, saying that he "solved" the
"non-scalability" of Gnutella, that he would release the design "soon", yet
we haven't heard from him since...

And, if you want to know, I'm also looking at the different XML parsers
out there. XML might be something really important in ANet...

- Benad

P.S.: I should be able to start *and* finish TWDT this weekend, plus maybe
Regions, urr, Clusters and Gateways.

----------------------------------

Anonymous Two-Way Data Transfers:
- The idea: proxy paths
- The problem: identifying each other.
  Solution: proxy paths, and identify based on connection protocol
  (assuming protocol that allow unique identifiers, so you have to
  specify what's the subnet...)
- Summary:
  - Both make proxy path (if they want)
  - Both exchange their identifier (connection protocol, subnet & address)
  - Start connection

Regions And Gateways
- RENAME term "region" to cluster
- Cluster: define geographical position and *rules*
- Filtering: defining rules vs. enforcing rules
- Gateway: a node that represents the whole cluster.
  Becomes a "server" of the cluster, and allows "clients"
  to use the cluster as client/server through the gateway.
  That's why it's both a packet protocol module and a client
  connection module.
- Advantage of Gateway over pure client/server:
  resume connection through another gateway is automatic
  and transparent, allowing you to use an ANet cluster as
  if it were a single "server"...
  Could also smartly replace DNS...

High-level design:
(top-down of data movement;
 some can access data from everywhere, ex. band. manager)

Clent SDK

Client Connection Modules:
Transfer commands from/to ANet.
Commands are:
- ANet status
- ANet config read/change (change might be disabled/restricted)
- Get/Send data. (*NOT* "drop": clients can't act as filters...)

Cluster Groups:
Define data duplication between different clusters.

Cluster Filter Modules:
Define data filtering, that is the rules of what should be transferred.
Needs info from bandwidth manager.

Core:
Does the acutal distribution across the different connections.
Contains different Distribution Rule Modules:
- Queries
- Static Data
- TWDT

Handshake Protocol Modules:
Define what and how connections should be made.
Also manages resumed connections.
They do so by fully abstracting the actual connections below:
Same node + different IP address = same ID for Handshake Protocol Module,
different ID for the Connection Protocol Module.
A connection is a combination of both a Packet and a Connection Protocol
(see below).

Data Transform Modules:
Transforms data before sending it (compression, encryption, byte-ordering...).

Packet Protocol Modules:
(WAS "Handshaking Protocol")
Define how the data transfered should be placed in small packets, if any.
Ex: HTML, ANet, Gnutella, Gateway...

Connection Protocol Modules:
Define how the network connection should be made.
Independant of the Packet Protocol Module.
Ex: UDP, TCP, HTTP (it is a connection protocol because it defines how to
connect, contrary to HTML)...

Bandwidth Manager:
"Knows" how much bandwidth is used.
Used by Cluster Filter Modules, but data taken from Connection Protocol,
matched with info from Handshake Protocol.

----------------

Run-Time Wrapper:
Manages the "up-stream" (to clients) and "down-stream" (to network) data
transfers between modules. Has all OS-specific wrappers (file system, memory
management...), and properly does all I/O required outside network I/O.