Re: [Cxtable-devel] A longer reply to "p2p, the next generation"

[David S. W. <xi...@pr...>]

I suppose that while I always wanted the software I was writing to grow
beyond a simple group of people lumped together for similar purposes...a
group of 'known' quantities...  I deep down never really allowed for it with
the algorithm I created...

xTable doesn't scale because it is a _table_, not one of those fictional
"infinite frictionless plane"s that they used to illustrate a frictionless
physics equation....  I think the goal of a scalable true p2p network is to
create said frictionless plane... (Are you familiar with this example?)...
In other words...when they are trying to discuss velocity of a rolling ball,
rolling down a decline to a flat, frictionless, plane...

Ahh..I digress..my point is that xTable was born from a project to create a
6-9 player Dungeons and Dragons chat room, filled with dice, and perhaps
character sheet programs, tournament battles, and maybe even automated
encounters....  An ICQ entirely meant for roleplaying games.  Originally, it
was called CyberGameTable...and it was done by sharing "my" (or the DM's)
ftp-password, either directly, or embedded into a setup file, and posting
via FTP...then the panels each had a URL-reader that read the text file
online associated with it, and downloaded it.. (It skip(long)'d what the
text-area already contained, to speed up the reading)...  It was actually
quite amazing...it was based on the fact that someone said that all dialups
suffered from an inability to create a ServerSocket, so "sockets" would be a
waste of my time... I was green...it made sense to listen at that point...

Then, I (and this might be re-hashing old shit you've read in the doc's
section of the site)... did an experiment, and it worked.... I was both
PISSED, and THRILLED... I then, subsequently, selected 5-10 files of the 70
for CyberGameTable, and started Cyber_x_Table....  The "x" came from the
idea that while this was a table, it wasn't only a game table, but rather,
an amorphic table..., that is, one with many potential purposes..much like
the kitchen table we all played D&D around as adolescents...and
teenagers...  We did homework on the table, we did eating on the table, we
did crafts and/or art on the table...we sat around the table and talked
about nothing in particular...  we played scrabble, and monopoly, around the
table... and yes, we played D&D...

I saw xTable as being the same place where many or few people could gather
and do things...things that required being in some sort of proximity...and I
envisioned extending that proximity to close people (close in the heart, or
close in relation) who were in geographically disparate locations
physically.. .

That is specifically why _two_ persistent connections between each peer is
not so heavy an overhead to place...  because the count of peers, even at 8,
would mean that each user only had 15 connections to manage.... and
technically, 7 of them were only relavent when outgoing things were
occuring, so 8 of them required listeners for inbound messaging...

I am not saying that said design would scale, but it was really never
designed to in the first place...  It was not designed for 30 people, but
rather, for 6-10...

That is why I make no apologies for its design, and at the same time, my own
visions said that it could be greater, that p2p could be done better.... and
I was wondering which way to go..

But if you think about it, I scrapped approximately 70-80 java sources that
had taken me 9-12 months (part-time) to write, in a heart-beat, and
subsequently re-wrote it...incidentally, though, the old one was _REALLY_ a
dog...as at 80 files, it was close to 2/3rds of a meg (ALOT OF WASTE)...
whereas now it is closer to 300 kb with both .java, .class, .bat and .cgt
setup file...

If I get nothing else out of the year I spent on xTable (6-7 months prior to
it appearing on Sourceforge)..it was the more advanced knowledge of .net.*
and sockets in specific...as well as better uses of interfaces...etc...

That is why I think that maybe we could either -1- continue under the
cxtable banner (although maybe something more fitting is required)... or -2-
start the new project...I will still have some work to do and/or try w/
cxtable..until I can safely declare a 4-beta or 5- stabile 1.0
release...then I can see if a user group for the close-knit p2p grows..if it
does./..I maintain it and release additional features and add-ons, if not, I
let it slide away into my educational past...

Idea?

Borne Goodman-Mace wrote:

> There is indeed a huge scalability issue with the current model, and
> with the new model, for those 5 people to communicate, there would be a
> total of 5 connections between them, as they are just all talking to a
> common chat "service".  There could of course be a number of these chat
> "services" which are pulled together, virtually, by a listing service,
> which you could connect to and find out about all the different chat
> services available, but if people don't want to join the "borg", then
> they could also have an independent grouping of these chat services.
>
> Of course if the chat client supports it, you could open up a connection
> to more than one chat service at the same time, giving the same
> abilities as IRC has now, but with better scalability, and security.
> You never have to worry about the whole channel going down and thousands
> of people losing their connections, and with the "backup" system, the
> chat environment could conceptually stay up forever.
>
> I hope, as we are talking this out (and my thoughts are evolving with
> your questions, so this is very helpful for me), that you can see how
> this is FAR more scalable, and bandwidth / diskspace friendly than any
> of the pure p2p systems I have seen.
>
> --bjgm
>
> On Wed, 2002-01-02 at 18:51, xi...@pr... wrote:
> > With relation to "chat"...
> >
> > The way cxtable currently handles this is that it has
> > a Vector of xClientConn objects...
> >
> > When you post, if you wish to send to the whole group,
> > you type your message, press the POST button, and then
> > leave the radio-button chosen to "Main"..
> >
> > Press send (or proceed)
> >
> > If you wish, however, to send to only one (or two, or
> > three, but not all) select whom you wish to send it
> > to, and press send
> >
> > If you have selected "main", then it sends to each
> > xClientConn
> >
> > If you select "other" and then many different people,
> > it polls through the array of xClientConn's, and
> > selects only those to whom you've chosen to send it to
> > the message.
> >
> > currently... 5 people connected look like a weird
> > criss cross..
> >
> > A is connected to B, C, D and E as a client
> > B is connected to A, C, D and E as a client
> > C is connected to A, B, D and E as a client
> > D is connected to A, B, C and E as a client
> > E is connected to A, B, C and D as a client.
> >
> > (By default then, and not being redundant)
> > A has a server connection for B, C, D and E
> > B has a server connection for A, C, D and E
> > C has a server connection for A, B, D and E
> > D has a server connection for A, B, C and E
> > E has a server connection for A, B, C and D
> >
> > So 5 people, to be connected as a network, have 20
> > connections total between all of them.
> >
> > x*(x-1) connections required with X being the value of
> > participants.
> >
> > This style of connections does not scale well to 1000
> > users...
> >
> > 1000 * 999 = 999,000 connections... (999 a piece)
> >
> > But you knew that without me doing the math...
> >
> > However, what the above connections do allow is a
> > certain level of p2p between a limited set of peers.
> >
> > Therefore, a "neighborhood" of peers could then be
> > RELAYED between by registry servers.
> >
> > Let's say, Registry server A handles 10 people.
> > Registry server B handles 10 people.
> >
> > Now... for ALL of A's and B's people to "get together"
> > would take 90 connections.  (an average of 9 a
> > piece...)..  However, if A maintains a persistent
> > connection with B, 45 connections (4.5 per person)
> > would handle it, (actually, allow +1 for the
> > persistent registry server connection)...  and
> > therefore allow multiple neighborhoods to relay...
> > That was how I planned to scale it (and, again,
> > everything remains up for debate)... with N users per
> > registry Server, and relay it while they were up...
> >
> > I think, though, maybe you could see how to create
> > a "virtual" intranet atop of the new "sxtable -->
> > cxtable-->whatever" vision that would allow the same
> > degree of virtual privacy and more flexibility to surf
> > across a distributed internet of nodes?
> >
> > I like your ideas, and sometimes, good ideas are hard
> > to swallow.  Let me digest.
> >
> >
> >
> >
> > --- Original Message ---
> > From: Borne Goodman-Mace <bm...@eg...>
> > To: "cxt...@li..." <cxtable-
> > de...@li...>
> > Subject: Re: [Cxtable-devel] A longer reply to "p2p,
> > the next generation"
> >
> > >A "Server" is really just a piece of code in
> > each "peer".  This part of
> > >the peer software (which each person would run),
> > would include the
> > >server, any services they have installed, and the
> > client code.  A Server
> > >includes registry logic, persistence logic, service
> > replication logic,
> > >and user authentication / authorization logic.  The
> > server is really
> > >just the wrapping through which a service sees the
> > world, which is made
> > >up of it's clients, and it's persistent storage.
> > >
> > >A service is started by the server (when the user
> > commands this to be
> > >done, or by default when the program is started if it
> > is configured in
> > >this way), and then, based on the service's
> > configuration, when it
> > >persists information, the server will send that same
> > information to
> > >other server's which have are designated as "backup"
> > services, to
> > >persist the data there as well.
> > >
> > >Your point about how the client currently works is
> > the largest
> > >divergence from my thoughts, as what I think this
> > idea really turns into
> > >is a kind of hybrid p2p / client-server model, though
> > it is flexible
> > >enough to be a pure p2p, or pure client-server model,
> > based on the
> > >configuration.
> > >
> > >I will now give some examples of usage, which I hope
> > will help to answer
> > >any other questions you might have, but if they
> > don't, please feel free
> > >to ask more! :)
> > >
> > >> Alice starts up the supercharged new cxtable (I
> > will call this sxtable
> > >for now).
> > >> Alice starts up a file sharing service (through the
> > server UI).
> > >> Bob starts up sxtable.
> > >> Bob starts up the file sharing service, but
> > in "backup" mode, and
> > >connects it to Alice's file sharing service (he needs
> > permission to do
> > >this, which he already has from Alice).
> > >
> > >> Charles starts up sxtable.
> > >> Charles has a URL which lists a number of peers
> > which are sometimes up
> > >running the file sharing service and types it into
> > the UI (or it is
> > >saved in his preferences and he doesn't need to type
> > it in).
> > >> Charles gets a list of file sharing services which
> > are currently
> > >available (found through the discovery process).
> > >> Charles starts up his file sharing client, and
> > connects to Alice's
> > >service.  The client may or may not support
> > connecting to more than one
> > >service at a time
> > >> Charle's client finds out that Bob is a "backup"
> > for Alice, and
> > >remembers this.
> > >> Charles downloads some files, and uploads one.
> > This causes Alice's
> > >file sharing service to persist it to disk, at which
> > point the "server"
> > >knows that Bob is a backup, and send the file to him
> > to persist as well.
> > >
> > >> Alice's system crashes.
> > >> Bob's server detects this, and starts up the
> > service.  A service in
> > >backup mode doesn't need to be fully running, the
> > server only needs to
> > >know how to persist the information for that service
> > type.
> > >
> > >> Charles can now connect to Bob's system, and
> > continue uploading /
> > >downloading.
> > >
> > >> Alice's system comes back up, pulls over the
> > persistent data from Bob
> > >and tells it to go back into "backup" mode, once no
> > clients are
> > >connected.
> > >
> > >In this example, if the client software supported it,
> > Charles could
> > >connect to a large number of file sharing systems at
> > once.
> > >
> > >Overall this solves one of the biggest problems of
> > peer networking, in
> > >that you might not be able to get to the data that
> > you want to, but you
> > >don't want to have the data EVERYWHERE because that
> > is a huge waste of
> > >bandwidth / space.
> > >
> > >Also, with things like chatting, rather than having
> > all the messages
> > >sent to everyone, you can have a single system,
> > a "server" of sorts,
> > >which everyone chats through, but if it goes down,
> > other can be set as
> > >"backups", so the conversation can continue
> > seamlessly if the server
> > >goes down.
> > >
> > >One of the guys that invented Lotus Notes is involved
> > in p2p software
> > >now, and I can't remember the name of it now, but I
> > used it for quite a
> > >while, and it was rather nice, with a shared
> > whiteboard, shared
> > >calendar, chat system, and some other stuff
> > (including file sharing),
> > >but it stored everything on each client (they have a
> > version you can pay
> > >for though, that creates a server which holds
> > stuff).  Despite the
> > >ability to "serverize" some of these services, it
> > still lacks a great
> > >deal in the fail-over / reliability area, and
> > everything is encrypted,
> > >but their security scheme still blows, IMHO.
> > >
> > >I want everyone in the world to be able to host a set
> > of services on
> > >their home machine, and with more and more people
> > getting broadband and
> > >leaving their machines on all day long, this makes
> > more and more sense.
> > >
> > >I could then set up some file system space at home,
> > for family pictures,
> > >my mother and aunt could run "backup" services, so
> > that if for some
> > >reason I go off line (which doesn't happen often),
> > everyone can keep
> > >getting the data, and I can get updated when I come
> > back on-line.
> > >
> > >I see this as the ultimate group-ware, since you can
> > still be in
> > >client-server mode if you want, or each person can
> > keep their files on
> > >their local system, or you can tailor make your
> > service network in any
> > >way that is appropriate to your application.
> > >
> > >There could even be a service which acts as a client
> > in a knowledge
> > >"group" to obtain information about all available
> > files in the group.
> > >Then rather than having to connect to each peer, you
> > can just connect to
> > >one of the knowledge managers to get the information.
> > >
> > >There are a nearly infinite number of uses for this
> > technology, I think,
> > >yet I feel that this sort of thing has never been
> > implemented in the
> > >correct way.
> > >
> > >I hope I have not rambled too much, and perhaps even
> > answered some
> > >questions along the way.
> > >
> > >--bjgm
> > >
> > >On Wed, 2002-01-02 at 16:33, Williams, David wrote:
> > >> Borne-
> > >>
> > >> OK....
> > >>
> > >> I am assuming, then, that a Server is a persistent
> > thing...?  It is always available, or this is
> > incorrect?  Either way.... I would make this akin to
> > my mind to the current Registry server process (which
> > also handles relay communications when direct, and
> > peer-enable, both fail..)..  It could handle many
> > other processes as well..but at this time, does only
> > those two...
> > >>
> > >> The Service..  These services, run, then, on the
> > Server, and cache to their backup?  Or did I miss this
> > point?
> > >>
> > >> The Client...  This client has a list of services
> > it can connect to... This is definitely a planned but
> > lacking feature for cxtable.  Cxtable assumes that
> > everyone connected to it wishes to connect to one
> > another... the persistent connection between each peer
> > eventually allows the runner of the Registry server to
> > pull down that server process, and have a set of X
> > people connected with one another, but closed off to
> > the rest of the 'network'..
> > >>
> > >> Service Discovery...
> > >> URL-scanning is good.  I personally like the PHP-
> > piece, as the act of registering a server for the
> > availability of use to clients can be done with a URL
> > scan as well..  Either way...this basic idea does seem
> > to be in line with how I do it (or hope I do)
> > >>
> > >> Persistence...
> > >> I have given no thought to persistence.  These
> > ideas seem valuable, for sure..
> > >>
> > >> Security/Service Ownership..
> > >> Contrary to what the code would tell you.. I have
> > given thought to security...  Relating to public
> > keys...  these keys generally can be published
> > anywhere (hence their name) and could be provided
> > along with the peers other information (IP, port,
> > etc..), unless your intention is to route all
> > communicae through a Server, and then, if course, only
> > the server would need to know it...
> > >>
> > >> ---------------------------------------
> > >>
> > >> Each of the above points has merit with relation to
> > the project I currently have, and at the same time..
> > I pause... as there are one or two points that I am
> > unsure of..
> > >>
> > >> Is the goal of the p2p to have places that relay
> > which are persistent?  Because this vision does give
> > way to a couple of weaknesses...  One of these
> > weaknesses is that if they can pin down where you are,
> > and where you'll always be...  Then they can shut you
> > down....  The other is the need for a certain type of
> > cash-flow to support the persistent servers...
> > >>
> > >> However, as I continue to read through this, I am
> > of the belief that the servers you speak of could be
> > EITHER a persistent server in the traditional sense,
> > OR a peer-type server that is fairly transient...
> > >>
> > >> Change is not a bad thing...and this is definitely
> > the type of thing I set myself up for by inviting new
> > creative blood to come look, to come help....
> > >>
> > >> I think that the changes you propose are worth
> > pursuing.  I think that certain services, under the
> > new modified vision, would allow for the "Intranet
> > over the internet" concept to remain, and at the same
> > time widen the vision to incorporate many other takes..
> > >>
> > >> This is all assuming that you haven't decided to
> > ditch me for Grapevine ;-)
> > >>
> > >> ~Dave
> > >>
> > >>
> > >> David Scott Williams
> > >> Computer Associates
> > >> Marketing Representative-Sales Call Center
> > >> One Computer Associates Plaza
> > >> Islandia, New York  11749
> > >> tel: +1 800-243-9462 ext. 73431
> > >> tel: +1 631-342-3431 (Direct)
> > >> fax: +1 631-342-5734
> > >> wi...@ca...
> > >>
> > >>
> > >> _______________________________________________
> > >> Cxtable-devel mailing list
> > >> Cxt...@li...
> > >>
> > https://lists.sourceforge.net/lists/listinfo/cxtable-
> > devel
> > >>
> > >
> > >
> > >
> > >_______________________________________________
> > >Cxtable-devel mailing list
> > >Cxt...@li...
> > >https://lists.sourceforge.net/lists/listinfo/cxtable-
> > devel
> >

--
~Dave
xi...@pr...
http://sourceforge.net/projects/cxtable >The xTable Project<
http://sourceforge.net/projects/dustyscript  >Dustyscript Child Script
Language<

 |>>weblog: http://www.geocities.com/xiarcel/weblogs/log.html<<|

"He who trades even a small amount of freedom for temporary security
 deserves neither freedom, nor security"
  -Benjamin Franklin

"When I think back on all the crap I learned in High School...
it's a wonder I can think at all..."
  -Paul Simon, "Kodachrome"