blocks-development — Main mailing list for Blocks

Re: [Blocks-development] Mixnets, libblocks, datagrams

[Blat F. <pet...@ho...>]

>From: "Michael Spencer Jr." <m...@ms...>
>To: <blo...@li...>
>Subject: [Blocks-development] Mixnets, libblocks, datagrams
>Date: Sat, 10 Mar 2001 16:45:27 -0600
>
>I've been pouring over the Blocks 0.17 source this morning, and am having
>lots of evil thoughts.  :)
>
>I'm ignoring libsock, because the traffic is supposed to be encrypted
>anyway -- we should never be stepping around libblocks and sending
>unencrypted data.
>
>It looks like GPG source integration is going to be hairy.  They have an
>existing project, gpgme, which is supposed to be a portable "GPG Made Easy"
>library for use in applications.  Unfortunately, it doesn't seem to want to
>compile in MS Visual Studio -- the documentation plainly states that you
>need to use GNU tools for Windows to compile this.  (The rest of Blocks
>compiles just fine in MS VS 6, and even comes with a workspace file
>ready-made for it.)

I have been very unimpressed by the portability of encryption
software in general. Nearly all of it has no platform specific
features beyond a couple of rotate macros, yet it is rare that
you find anything that is not overoptimized, overcomplex
and platform specific. This was one of thereasons why I
implmented my own crypto in Blocks. Its fun as well :-)

>
>A lot of GPG seems to be framework and compatibility code -- figuring out
>what pieces to use where, and encapsulating everything in easy-to-transport
>packets.  Perhaps we can just take the functions we need, and port the code
>to the same multiplatform standards Blocks already uses.  What do we need 
>to
>be able to do?  Generate a keypair, encrypt-with-pubkey,
>decrypt-with-seckey, sign-with-seckey, verify-with-pubkey...correct?  If we
>decide that GPG integration is the way we want to go, should we approach 
>the
>GnuPG dev mailing list with a request for advice?

I would definately go down this route, but be warned, crypto
is a prime fighting ground for people trying to entrench
algorithms and influence on future standards. For instance,
P1363 is largely the result of companies playing tech warfare
for part of the future crypto market.

IMHO all public key crypto is a white elephant. There can
never be trusted certificate authorities over the public
internet, and even the PGP WoT is very limited and subject
to fraud. Security is about guarantees, and IMHO PGP WoT
falls short of pretty good protection. Absolute authentication
based on short memorable secrets is the only obvious
future direction, but this would require the demise of
the current corporate PKI incumbants.

>
>------
>
>I made a post to Infoanarchy last night that contains a rather neat idea.
>Since the content index is the legal target right now, suppose we use 
>mixnet
>routing to hide one or more 'central' content index servers in the Blocks
>network?  One or more nodes would serve as collections of file adverts, and
>search requests and results would go through the Blocks network, using an
>'encrypted tunnel' through the Blocks network.

So you would be using Blocks as like a VPN packet switching
relay network. I think something like AT&T's Crowds project
would probably be a more direct and elegant solution to
hiding central servers, or rather communication beween the
client and server.

>
>For this 'mixnet' protection to work, index servers will need to have
>special anonymity protection from the rest of the network.  Any one
>untrusted user can run three or four blocks nodes on his own machine, 
>making
>a search request appear to have come from farther down the network...so we
>can't chance revealing the index server's identity through an untrusted
>connection.
>
>The buddy-list approach seems useful here.  If one index server has three 
>or
>four peers he knows and trusts, the server can expose its service through
>those peers.  To protect against connection spoofing, trusted peers should
>probably authenticate to each-other with a public-key handshake...although
>an internal list of trusted IPs will probably work for now.
>
>I'm not saying that index servers need to be completely isolated from the
>public network -- if they're worried about their index server getting them
>in trouble, though, they should only expose index services through trusted
>connections.  They're free to route data blocks and datagrams.

Why not make the index servers initiate the connection? They
could have a list of 'clients' they want to connect to and
connect to each in turn. That way ISPs couldnt even accuse
people of running servers at all and its pretty much guaranteed
(bar routing attacks) that noone but the correct audience get
direct connections.

>
>Broadcast traffic is greatly reduced then...because we're only broadcasting
>index server availability and/or index server location requests.  These
>requests can be cached.  It seems that after index server adverts, most of
>the traffic is then point-to-point routed traffic.
>
>Hmm...there's only one problem:  if the server and the client are 
>introduced
>to each other through an index server, how does the index server know what
>the fastest path between the two machines is?  The only path between the 
>two
>machines that's known is the path from server to index concatenated onto 
>the
>path from index to client.

You could gather statistics about the latency between connections
and add them up. Then you would just need to solve the 'travelling
salesman problem' to find the quickest route. Of course, the
men in black might be able to use this information as well :-)

>
>Perhaps the index server can forward a 'download request' from the client 
>to
>the server, and the server could send out a broadcast file advert...but
>encrypted so only the requesting client machine can read it.  The download
>can then continue through the shortest direct route.
>
>------
>
>This seems to call for a couple of changes to the libblocks API, if this is
>a good enough idea to implement:
>
>1)  a send-datagram function and a receive-datagram notification callback.
>2)  a way to negotiate an encrypted datagram tunnel between two hosts
>several hops apart on the network.
>
>...and later on...
>
>3)  broadcast index-server advertisements and index-server requests, and
>internal lists of recent index-servers
>4)  a toggle controlling whether or not to broadcast file adverts in the
>traditional way, and a toggle controlling whether to send file adverts to
>known index-servers.
>5)  a toggle controlling whether to forward search queries to known
>index-servers.
>
>------
>
>What do you guys think?

Blocks is a bit of a mess and any change is probably going to
be more than a minor one. Feel free to change anything you
want. If I get a chance I'll have a look at the code and see
if I can remember anything :-)

ttfn

PG.

_________________________________________________________________________
Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com.

[Blocks-development] Mixnets, libblocks, datagrams

[Michael S. Jr. <m...@ms...>]

I've been pouring over the Blocks 0.17 source this morning, and am having
lots of evil thoughts.  :)

I'm ignoring libsock, because the traffic is supposed to be encrypted
anyway -- we should never be stepping around libblocks and sending
unencrypted data.

It looks like GPG source integration is going to be hairy.  They have an
existing project, gpgme, which is supposed to be a portable "GPG Made Easy"
library for use in applications.  Unfortunately, it doesn't seem to want to
compile in MS Visual Studio -- the documentation plainly states that you
need to use GNU tools for Windows to compile this.  (The rest of Blocks
compiles just fine in MS VS 6, and even comes with a workspace file
ready-made for it.)

A lot of GPG seems to be framework and compatibility code -- figuring out
what pieces to use where, and encapsulating everything in easy-to-transport
packets.  Perhaps we can just take the functions we need, and port the code
to the same multiplatform standards Blocks already uses.  What do we need to
be able to do?  Generate a keypair, encrypt-with-pubkey,
decrypt-with-seckey, sign-with-seckey, verify-with-pubkey...correct?  If we
decide that GPG integration is the way we want to go, should we approach the
GnuPG dev mailing list with a request for advice?

------

I made a post to Infoanarchy last night that contains a rather neat idea.
Since the content index is the legal target right now, suppose we use mixnet
routing to hide one or more 'central' content index servers in the Blocks
network?  One or more nodes would serve as collections of file adverts, and
search requests and results would go through the Blocks network, using an
'encrypted tunnel' through the Blocks network.

For this 'mixnet' protection to work, index servers will need to have
special anonymity protection from the rest of the network.  Any one
untrusted user can run three or four blocks nodes on his own machine, making
a search request appear to have come from farther down the network...so we
can't chance revealing the index server's identity through an untrusted
connection.

The buddy-list approach seems useful here.  If one index server has three or
four peers he knows and trusts, the server can expose its service through
those peers.  To protect against connection spoofing, trusted peers should
probably authenticate to each-other with a public-key handshake...although
an internal list of trusted IPs will probably work for now.

I'm not saying that index servers need to be completely isolated from the
public network -- if they're worried about their index server getting them
in trouble, though, they should only expose index services through trusted
connections.  They're free to route data blocks and datagrams.

Broadcast traffic is greatly reduced then...because we're only broadcasting
index server availability and/or index server location requests.  These
requests can be cached.  It seems that after index server adverts, most of
the traffic is then point-to-point routed traffic.

Hmm...there's only one problem:  if the server and the client are introduced
to each other through an index server, how does the index server know what
the fastest path between the two machines is?  The only path between the two
machines that's known is the path from server to index concatenated onto the
path from index to client.

Perhaps the index server can forward a 'download request' from the client to
the server, and the server could send out a broadcast file advert...but
encrypted so only the requesting client machine can read it.  The download
can then continue through the shortest direct route.

------

This seems to call for a couple of changes to the libblocks API, if this is
a good enough idea to implement:

1)  a send-datagram function and a receive-datagram notification callback.
2)  a way to negotiate an encrypted datagram tunnel between two hosts
several hops apart on the network.

...and later on...

3)  broadcast index-server advertisements and index-server requests, and
internal lists of recent index-servers
4)  a toggle controlling whether or not to broadcast file adverts in the
traditional way, and a toggle controlling whether to send file adverts to
known index-servers.
5)  a toggle controlling whether to forward search queries to known
index-servers.

------

What do you guys think?

--Spence
bl...@ms...

[Blocks-development] Thoughts on preventing 'spam'

[Michael S. <msp...@dp...>]

I've been thinking about what really comprises 'spam', and how to defend
against it.

I want my email box to be a place for a human to send me (personally and
exclusively) a message, and for me to respond back.  I don't want to receive
an automatic message that one person is broadcasting to ten thousand people.

For me, 'spam' represents a message which appears to be person-to-person,
but is really broadcast.

------

Suppose that we're interested in protecting e-mail from spam.

Perhaps someone could build an extra communication step into SMTP:  the
recipient has defined a 'challenge' that requires human thought and
interpretation to respond to, but doesn't require human thought or
interpretation to authenticate the response.

Someone sending an email to me using a special email program and
communicating with my special email server would see a popup, and have to
type in the answer to a question, like "What is the singular form of the
word 'mice'?".  The question would have to be answered before the mail could
be delivered, demonstrating that the sender of the message is paying
specific attention to the recipient.

Back to the land of reality...there's actually an email service that does
something like this.  SpamCop (www.spamcop.net) has a for-pay service that
will capture and hold your email, filter out the spam, and for senders from
ISPs that are known to be frequent spammers, it sends the originator an
email asking them to open a URL in their web browser before the email is
released.

------

Any thoughts about how this affects peer-to-peer filesharing?  File adverts
are most definitely broadcast messages....

--Spence
bl...@ms...

[Blocks-development] Future GPL code in Blocks

[Michael S. <msp...@dp...>]

I remember I had once proposed that we include the GPG code in Blocks, so we
could do public-key crypto.

Blocks is public domain, not GPL.  GPG is GPL.

I wonder about the legality of making two 'packages' of Blocks:

One package is completely licensed under the GPL -- GPG and Blocks code.

Another package is public domain, but contains no GPG code -- merely stub
code that captures GPG calls and returns errors.

---------

Is this violating the GPL?  Is it wrong to offer the Blocks code under two
licenses:  public domain, for people who will not accept the GPL (and who
should not be licensed the GPG code) and GPL, for people who are willing to
accept the restrictions of the different license in exchange for being able
to use the GPG code?

Or more specifically...am I trying to be sneaky and use a loophole?  Or is
this in line with the intent of the GPL?

----------

Or to put it another way -- is it within my rights to re-license the
public-domain Blocks code as GPL?  This does *not* remove the existing
public-domain 'lack of license'...but if someone is willing to accept the
extra weight of the GPL, they can benefit from the extra GPL code included.

What do you guys think?

--Spence

(I know I've been quiet -- I've been working on reading through the Blocks
source code, starting with libsock and libblocks, as well as writing up
pseudocode for the network simulator.  The network simulator is going to be
interactive, because I'm not going to be able to think up every possible
scenario to throw at it.  I'll try to make the code readable enough that
coders can run their own simulations easily...but if you aren't a coder and
want to test a scenario, I'll run your scenario as soon as I can.  I am at
your disposal.)

(There is no code yet.  Just pseudocode.)

Re: [Blocks-development] Tode Node

[Michael S. Jr. <m...@ms...>]

It's easy for me to say, not being able to do the work yet...but...go for
it.  :)

--Spence
bl...@ms...
----- Original Message -----
From: Blat Froop
To: blo...@li...
Sent: Monday, March 05, 2001 4:28 AM
Subject: [Blocks-development] Tode Node


I see ToadNode ( www.toadnode.com ) is soon going to move to a plugin
architecture so it can support more protocols. I think other clients (apart
from the sucky java ones) will probably do similar things. If it doesnt
totally suck I might think about a Blocks plugin.
What do you think?
ttfn
PG.






Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com.

_______________________________________________ Blocks-development mailing
list Blo...@li...
http://lists.sourceforge.net/lists/listinfo/blocks-development

[Blocks-development] Tode Node

[Blat F. <pet...@ho...>]

<html><DIV>
<P>I see ToadNode ( <A href="http://www.toadnode.com">www.toadnode.com</A>&nbsp;) is soon going to move to a plugin architecture so it can support more protocols. I think other clients (apart from the sucky java ones) will probably do similar things. If it doesnt totally suck I might think about a Blocks plugin.</P>
<P>What do you think?</P>
<P>ttfn</P>
<P>PG.</P>
<P>&nbsp;</P>
<P><BR>&nbsp;</P></DIV><br clear=all><hr>Get Your Private, Free E-mail from MSN Hotmail at <a href="http://www.hotmail.com">http://www.hotmail.com</a>.<br></p></html>

[Blocks-development] Future direction: data services need network services need netwo rks

[Michael S. <msp...@dp...>]

I'm a bottom-up design person.  It's difficult for me to think about high
and broad concepts without being able to see details.

Despite this, I would like to start a broad concept discussion, and then tie
the discussion into some immediate low-level network design tasks.

-----

High-level subject:  replacing all client/server tasks with peer-to-peer
service.

I could never quite put my finger on what was bothering me, back in 1995 and
1996.  I knew I didn't like the increasing corporate involvement and the
'tiered' structure the internet was adopting.  I never saw how that affects
peer-to-peer until recently.  That's what makes this technology so exciting.

The whole point of a p2p network is to distribute information...so the
services available on a p2p network must be information services of some
kind.

We've already had many different ideas for the kinds of service that can run
on a peer-to-peer network.  Filesharing is obvious.  Distributed chat and
message boards (moderated even) are also interesting.  A distributed
voice-over-IP gateway is even more interesting.

These are all 'data services'.  Each one uses several 'network services' in
a special way:

Napster uses:  centralized search and indexing, distributed file store, and
direct file data transfer.
Blocks uses:  distributed search and indexing, distributed file store, and
routed file data transfer.
An IRC file-trading room uses:  no content search or indexing, distributed
host directories, centralized file stores, and direct file data transfer.

And that's just filesharing, folks.  ^_^

-----

Mid-level subject:  defining a library of network services which can form
any arbitrary data service

I wonder if we can't create a taxonomy of the properties of a network
service, and then implement a wide range of network services to meet every
possible need.

What properties could a network service have?  Yesterday and the day before
I talked about a network service being 'batch' or 'interactive'.  I'll
clarify that here:

An interactive service is a service that is extremely sensitive to latency.
If someone were to implement an encrypted, tunneled VOIP gateway over a p2p
network, the traffic would be extremely sensitive to latency.  If a node on
the network tries to cache data and retransmit it at a time of lower demand,
several seconds or minutes of silence would be perceived on the telephone
conversation.  Similarly if a 'delayed transmission' trick were done in an
IRC-like chat room, people would be annoyed by the result.

A batch service is a service that isn't sensitive to latency at all.  If I'm
using a 28.8 modem and I'm trying to download a copy of
Crouching_Tiger_Hidden_Dragon.asf, I already know that the transfer is going
to take more than 24 hours.  If someone delays sending me data for a couple
hours (because I'm waiting in a queue, perhaps) then I won't mind that much.

(Both scenarios would actually be pretty silly -- no current p2p network can
guarantee the low latency required for tunneled VOIP, and it's difficult for
a network to guarantee that one large file will be successfully delivered
slowly over a 24-hour-plus period.  But we aren't done designing yet.  It
may be possible in the future for someone to pay a quantity of 'mojo' to all
hosts along a route to get 'guaranteed' data delivery service.)

FIXME:  Can someone help with the classification of other network service
types?  Remember that a network service is what you get when you break apart
an existing data service (Blocks, Gnutella, Napster, ICQ, Slashdot,
whatever) into its requisite parts.

Some other network services:  data storage, indexing, traffic forwarding,
cataloging and grouping, authentication, authorization.

These still need to be classified:  what kinds of data storage are there
(encrypted or not, short term or long term, etc.), what kind of indexing
(how trusted, how distributed), and so forth.

-----

Low-level subject:  expanding the types of network service available through
Blocks.

All of these things require a transport.  I feel (and humbly and
respectfully submit) that our short-term goals should be:

1)  Harden the network underpinnings of Blocks.  They're pretty well
hardened right now.  Testing, simulation, and analysis.
2)  Strengthen the one existing data service we have running on Blocks right
now:  file sharing.  Find weak points and vulnerabilities, scaling problems,
etc.  Don't try to bolt a brand new data service type on top of Blocks just
yet (like I was trying to do with the web-of-trust...sorry...).
3)  Complete an analysis of data service types and the required network
service types.  Drool over all the cool ideas that emerge from this
analysis.
4)  Incrementally design and implement network services, and turn them into
new and innovative data services as quickly as possible.  Once we start
doing this (if we ever get this far) we'll start attracting a lot of
attention.

-----

It is asking too much to think that just because I have a crazy idea and go
type an essay about it, everybody else on the mailing list will immediately
make time to absorb all that verbiage and type a reply the same day.  Sorry
for all the traffic, guys.

-----

We still have our page on Sourceforge.  (Did anyone else notice, humorously,
that www.sourceforget.net points to the Free Haven Project?  ^_^ )

PG, if you still remember your login for SourceForge, could you grant me
admin rights to the (dormant) project?  I already have developer rights
there...but I'd like to add and structure documentation and such.  I'll
figure out how the whole thing works if you'd like, and help people figure
out the site.  A quick link to the project page on SourceForge is:
http://sourceforge.net/projects/blocks

Just for the record...I do not consider myself to have any control
whatsoever over the Blocks project.  I just want to be our SourceForge
janitor/custodian.  :)

-----

When responding with thoughts, please try and keep in mind what 'level' your
thoughts are on.  If someone proposes "I think we should use Java" then
that's a low-level concept.  It's an implementation detail.  After some
introspection the person might realize that there was a high-level concept
he used to arrive at the low-level concept:  "I think the project should use
an open, interpreted language that doesn't require an expensive compiler.
Plus, I already know Java and would like to contribute."

That is...if you say "I think we should use Java" you might get some
resistance, because we haven't had enough high-level discussion to decide
what language would be ideal for the project.  If you say "I think we should
allow people to extend the client in arbitrary ways without needing special
programming experience or a compiler" and people agree, then the people
might be more receptive to the suggestion of Java.

Sorry if that seemed like I was preaching...

-----

Thank you *very* much Erik for giving me your time on IRC last night.  Most
of the collected and organized thought you've seen in this email wouldn't
have come together without the high-level discussion he and I had last
night.  (If my thoughts seem neither collected nor organized...blame it on
him.  ^_^ )

--Spence
bl...@ms...

Re: [Blocks-development] Network simulator

[Blat F. <pet...@ho...>]

>With my new-found confidence in coding, I'd like to implement a network
>simulator.  (I'll probably do it in MFC, because that's what I use at 
>work.)

Cool. I still remember the gold old days when I used to
write code on my Amiga 500.

[Erik's design snipped]

>Erik has a good design, but I don't want to implement most of those 
>features
>yet.  I think that's overkill right now.
>
>I got the motivation to do the simulator a few days ago, after reading 
>about
>a similar test with Freenet:
>
>*digs for a link*
>http://freenet.sourceforge.net/index.php?page=icsi-revised
>
>Scroll about halfway down and look for Performance.

Nice graphs :-)

>I'd like to do those kinds of tests for Blocks.

Cool.

>
>First, I need to understand how Blocks works, in an informal
>protocol-inspecific way.
>(I had just fired up Visio and opened up some of the files in
>libblocks...and had brain-lock.  This is going to take some time.)
>
>At first, I'd like to look at the 'connection' mechanism, irrespective of
>routing blocks or bandwidth constraints.  I want to confirm that, for an
>arbitrary network size and *one* central connection point, can the network
>accommodate that number of hosts?  When hosts start to leave the network,
>will the network fragment?

Possibly. This is tied to the number of interconnections between
nodes.
>
>I'll start writing a simulator to answer these questions.  What rules 
>should
>the simulator use again?  It's been a while:
>
>1)  Messages route no farter than 6 hops

Yup, 6 hops max.

>2)  Hosts advertise themselves to other hosts.  (When?  How far are they
>routed?)

They are advertised by other servers when they receive a
connection, and by themselves once per hour if they have less
than 10 connections. This is to try and stop network
fragmentation.

>3)  When an existing host sees a new host on the network...somebody will
>choose to connect to someone else under some condition.  (When and under
>what conditions?)

If we see a server ad from advertising a server which is not
immediately adjacent we do the following calculation...

x=rand(255)-(16-n)*25

if x>0 we connect to the server

>4)  What a host has ten or more connections already and a new host connects
>to them, the existing host will give the new host a list of possible other
>connections and then ask the other host to disconnect.  (When will the new
>host disconnect, and when will he stay on anyway?)

Whenever any server recieves a connection it sends a list of
recent server ads to its new friend. The server ads are just
a small list (10) of previous servers. The new server ip/port
is added then added to the top of this list.

>
>That should get me started.  I'm going to go try to remember how to do a
>linked list.  :)

The fun stuff is the advert routing, but the connection stuff
is quite interesting as well. Graphs would be cool! :-)

ttfn

PG.

_________________________________________________________________________
Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com.

Re: [Blocks-development] Network simulator

[<lu...@et...>]

> *digs for a link*
> http://freenet.sourceforge.net/index.php?page=icsi-revised
> 
> Scroll about halfway down and look for Performance.
> 
> I'd like to do those kinds of tests for Blocks.

Here's another good one:

http://www.monkey.org/~dugsong/mirror/gnutella.html


A few notes, but no promises to do anything help other than poke holes
in people's ideas.  (One would think that I would figure out that 
working at startups is insane after my first one... *sigh*)


- Any system will be beaten by the most obvious fault.

For Napster it was the central server.  For Gnutella, it's the 
fact that every peer needs to see every message.  If there's a 
big hole staring you in the face, don't ignore it and hope it goes 
away.

- Don't assume that the client can restrict anything.  

Anything that the client permits _will_ be taken advantage of by some
user somewhere.  All security and protection of the system has to be
done at the protocol level.  I'd almost say that you should build 
a working web of trust system that just happens to be able to transfer 
files as well...

- Come up with the protocol first.

The protocol has to be rock-solid before you should even think about
making clients.  It should be taken apart, bit by bit, and looked for 
avenues of attack.   Once it gets out in the world, it's a lot harder
to change things.


(That said, I think the blocks idea is insanely cool and just wish I didn't
have a job to contend with so I could a lot of time on...)

-Luke

[Blocks-development] EFnet #blocks chat tonight

[Michael S. Jr. <m...@ms...>]

I'll try to keep channel #blocks in EFnet open tonight, if anyone wants to
come chat.

--Spence
bl...@ms...

[Blocks-development] Network simulator

[Michael S. Jr. <m...@ms...>]

With my new-found confidence in coding, I'd like to implement a network
simulator.  (I'll probably do it in MFC, because that's what I use at work.)

For review, here's a snippet from Erik's original simulator post back in
August 2000:

----cut from Erik----
Here`s a more detailed proposal:

	- turn based simulation
	- everything that`s unique (adverts, files etc.)
	  should also have unique representations in the
	  simulation
	- Java, AWT GUI (for visual representation:
	  canvas with 1 pixel per node, 1 pixel between
	  each, lines between pixels visualize file
	  transfers / routes)
	- Reason for Java: cross-platform, easy to use

	Required data:

	global:
	- number of good nodes
	- readvertising frequency
	- individual advert size
	- optional: number of bad nodes
	   - flooders
	   - DoSers (don`t use Blocks, but
 	     attack a certain IP-can`t be
         locked out, might focus on
	     prominent nodes)
	   - fakers
	   - filters
	   - analyzers / hoppers
	   - ?

	individual:
	for these numbers we need a distribution table,
	e.g.: 56 kbps; 5%; 64 kbps: 10% ..
    or randomly picked from a range:
	      0-4h:10%; 4-12h:25%
	- speed of nodes
	- uptime
	- number of local files
	- file requests / hour
	- cache size
	- non-ad-routing: how many % don`t route ads?
	- "prominence":
	  how well the nodes are known, e.g.
	  50-100% know this node:1%
      30-50% know this node:4%
	  10-30% know this node:10%
	  0-10% know this node:85%
	  (when a node reconnects, the prominent
	  connection points are connected to more
	  often).
	- cancel frequency: when and how likely
	  will the user cancel the download?
 	  when=at which speed threshold, e.g.
	  at <50% of maximum speed: 1%
	  at 40-50% of max speed: 3%
	  ..
	  at 5-10% of max speed: 40%
	  at 0-5% of max speed: 60%
	- ?

	Optional:
	frustration - after a certain number of cancels
	 and failed downloads, the user is frustrated
     and quits the network. A threshold for this
	 could be defined.

	Implementation:
	The nodes are generated as a vector of objects.
	In a loop, each nodes can generate several
	actions (request, upload, disconnect, continue
	downloading of a requested file, cancel, send
	advert, send file etc.), actions that regard
	other nodes are put in their event queue which
    is processed as soon as possible, according to
	the bandwidth that the node can use per turn.

	Afer one loop is complete, a number of new nodes
	is generated randomly and added to the vector.

	The display is updated. Repeat on user request
	(step by step) or automatically.

	Gatherable data:

	How many nodes d/l at x % of their max speed?
	How many d/ls fail?
	What is the average bandwidth required by adverts?
	How is it distributed?
	Which regions of the network are congested and how
	could this be alleviated?
	How many evil nodes can we tolerate?
	etc.

----end cut from Erik----

Erik has a good design, but I don't want to implement most of those features
yet.  I think that's overkill right now.

I got the motivation to do the simulator a few days ago, after reading about
a similar test with Freenet:

*digs for a link*
http://freenet.sourceforge.net/index.php?page=icsi-revised

Scroll about halfway down and look for Performance.

I'd like to do those kinds of tests for Blocks.

First, I need to understand how Blocks works, in an informal
protocol-inspecific way.
(I had just fired up Visio and opened up some of the files in
libblocks...and had brain-lock.  This is going to take some time.)

At first, I'd like to look at the 'connection' mechanism, irrespective of
routing blocks or bandwidth constraints.  I want to confirm that, for an
arbitrary network size and *one* central connection point, can the network
accommodate that number of hosts?  When hosts start to leave the network,
will the network fragment?

I'll start writing a simulator to answer these questions.  What rules should
the simulator use again?  It's been a while:

1)  Messages route no farter than 6 hops
2)  Hosts advertise themselves to other hosts.  (When?  How far are they
routed?)
3)  When an existing host sees a new host on the network...somebody will
choose to connect to someone else under some condition.  (When and under
what conditions?)
4)  What a host has ten or more connections already and a new host connects
to them, the existing host will give the new host a list of possible other
connections and then ask the other host to disconnect.  (When will the new
host disconnect, and when will he stay on anyway?)

That should get me started.  I'm going to go try to remember how to do a
linked list.  :)

--Spence
bl...@ms...

Re: [Blocks-development] Filesharing concept discussion: limited resources, and queuing...

[Blat F. <pet...@ho...>]

>PG is still around!  Woo hoo!  :)

:-)

>I have a confession to make:  I talked to the press.

Cool. Sounds like you covered most of the good stuff.

>The point of all that was:  I feel we should focus on adding capabilities
>for advanced users.  Batch transfers are useful if there's *no other way* 
>to
>get a file.  Personally...if I find a server on IRC that's hosting every
>single episode of Ranma 1/2 ever made, but only allows one queue entry per
>host at a time, and it takes six hours for the queue to empty out...I'm
>still willing to wait for that slow trickle of file data, because the
>alternative is no data at all.

Gotta get the design right before 'creeping featurism' raises
its ugly head.

>I imagine a *highly protected* but small network of Blocks servers,
>worldwide.  Some (most?) servers only allow direct connections to a small,
>trusted number of other servers.  ...

I see a future where the hardcode runs ftp-rings over encrypted
VPNs and there is no access for normal folks. They will still
get access to pirate stuff via flea markets and CD vendors,
nothing will stop that, its just the legitimate file sharing
folks who will be offlined.

>I would still be afraid to run a Blocks server today.  Despite the fact 
>that
>the content is very well protected, currently people are getting banned and
>attacked because of filename searches.  Currently, filenames aren't
>protected in Blocks.  Someone could reasonably argue that, by seeing a file
>advert for "Metallica - I Disappear.mp3" and forwarding that file advert on
>to my other connections, I'm aiding in the illegal transfer of that file.

I dont think this is illegal, but its enough to get you a
letter and your connection pulled. You can always do the
'hackers must have broken into my system' line, most large
companies are too scared to suggest thats not a possibility.

>
>I don't think this is the end of the game, but it's a pretty big sticking
>point right now.

I agree.

>
>It's possible to create an encrypted tunnel between two nodes on the
>network, creating a 'pseudo-connection'.  Both sides create a PK key pair.
>Both sides exchange public keys with a key exchange mechanism (SNAKE?) and
>begin communication.
>
>With pseudo-connections, someone can send a file advert from one side of me
>to an attacker on the other side of me, and there's no way I could have
>intercepted the message.
>
>So now I'm not guilty of passing on a file advert for "Metallica - I
>Disappear.mp3".  I'm merely forwarding encrypted messages that I can't
>possibly read.
>
>Will I still get shut down?  Probably.

Yup, not reading the content is no different from not being
able to read the content. This reminds me of multicast systems
that disallow you from seeing information that you havent paid
for, even though it has been received and processed by your
computer.

>What's the next link in the protection chain?  We've tried plausible
>deniability before -- legal deniability.  I think we need traffic
>deniability now.

IMHO its ftp-rings over encrypted VPNs... Bladerunner here we
come :-)

>
>Will I have my DSL connection account deactivated because I'm running a
>Gnutella or OpenNap server and somebody complained?  Probably.
>How about if I'm running a Blocks server?  Probably here too.  The person
>making the decision might not know what Blocks is, but will probably trust
>the C&D letter when it says that Blocks is a service similar to Napster.
>How about if I'm running a web server that offers an
>encrypted-message-forwarding service?  Possibly, if it can be shown that 
>the
>service is only really used to support a Napster-like network.
>How about if I'm running an SMTP-based anonymous remailer, and publicly
>promise to make connection logs available to law enforcement upon receipt 
>of
>an original subpoena document?  Now the attacker is on shaky ground.  In my
>opinion, an ISP representative is more likely to discuss the situation with
>his customer.
>
>I think.

Why has IRC not been closed down? Well, because its primary
purpose (for most) is not file sharing. This makes the legal
arguement *much* more complex, and the fact that its primary
goal is disemmination of the spoken/written word attracts
First Amendment lobbyists like flies to dog toffee.


{hometime gotta go]

ttfn

PG.

_________________________________________________________________________
Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com.

Re: [Blocks-development] Filesharing concept discussion: limited resources, and queuing...

[Michael S. <msp...@dp...>]

PG is still around!  Woo hoo!  :)

I have a confession to make:  I talked to the press.

Back when Blocks was still in active development, I was contacted by a W. R.
from Time Digital.  (He saw a post describing the web-of-trust I made to
Slashdot, in an article about people flooding Napster with bogus content.)
He and I talked on the phone...discussed what Blocks' niche was...he even
sat through my web-of-trust explanation, the poor guy...

I think he concluded that Blocks was still pretty far from the general Time
Digital reader's radar scope, and may never appear there anyway.  After
hearing about Blocks, he basically asked me:  when users have Napster and
Gnutella, why would they bother with something like Blocks?  Why spend the
extra effort staying on the network and keeping a cache of encrypted blocks
on your local hard disk?

My answer to him was:  when users have Napster and Gnutella, they won't.  In
a world where Napster servers and users are free to do whatever they want
without coming under attack, there is no reason for a user to suffer through
the decreased convenience that comes with increased security.  However, I
said, I believe there will come a day when even Napster and Gnutella are
no-longer available for use.  I said that I hope the day never comes when
people *must* rely on the technology in Blocks for their freedom...but if
that day ever does come, Blocks will be here waiting.

I think he and I understood each other pretty well.  I think we both
concluded that Blocks should not be mentioned by the press yet.  Hopefully,
though, he came out of the discussion with the idea that volunteers writing
free code *are* thinking about the long-term.

-----

The point of all that was:  I feel we should focus on adding capabilities
for advanced users.  Batch transfers are useful if there's *no other way* to
get a file.  Personally...if I find a server on IRC that's hosting every
single episode of Ranma 1/2 ever made, but only allows one queue entry per
host at a time, and it takes six hours for the queue to empty out...I'm
still willing to wait for that slow trickle of file data, because the
alternative is no data at all.

So with this hypothetical new 'market' focus for Blocks, what would our
place in the p2p landscape be?  What would the world look like five years
from now?

I imagine a *highly protected* but small network of Blocks servers,
worldwide.  Some (most?) servers only allow direct connections to a small,
trusted number of other servers.  The Blocks network of the future has
several hundred nodes, but all of the nodes are very secure -- and very
unreachable.  Some Blocks nodes (located in places with more information
freedom -- someplace out of the reach of oppressive governments or
corporations) choose to use special software to share all of the files on
Blocks out to other p2p services -- the ones with less security but tons of
users.  The people who share files out this way know they're at risk...but
because of their special legal or social arrangement, they're willing to
accept that risk.  If an attacker takes down one of these 'gateway' nodes,
they haven't removed the content from the network completely.  They've only
removed one way to access the content.

-----

I would still be afraid to run a Blocks server today.  Despite the fact that
the content is very well protected, currently people are getting banned and
attacked because of filename searches.  Currently, filenames aren't
protected in Blocks.  Someone could reasonably argue that, by seeing a file
advert for "Metallica - I Disappear.mp3" and forwarding that file advert on
to my other connections, I'm aiding in the illegal transfer of that file.

I don't think this is the end of the game, but it's a pretty big sticking
point right now.

It's possible to create an encrypted tunnel between two nodes on the
network, creating a 'pseudo-connection'.  Both sides create a PK key pair.
Both sides exchange public keys with a key exchange mechanism (SNAKE?) and
begin communication.

With pseudo-connections, someone can send a file advert from one side of me
to an attacker on the other side of me, and there's no way I could have
intercepted the message.

So now I'm not guilty of passing on a file advert for "Metallica - I
Disappear.mp3".  I'm merely forwarding encrypted messages that I can't
possibly read.

Will I still get shut down?  Probably.

-----

What's the next link in the protection chain?  We've tried plausible
deniability before -- legal deniability.  I think we need traffic
deniability now.

Will I have my DSL connection account deactivated because I'm running a
Gnutella or OpenNap server and somebody complained?  Probably.
How about if I'm running a Blocks server?  Probably here too.  The person
making the decision might not know what Blocks is, but will probably trust
the C&D letter when it says that Blocks is a service similar to Napster.
How about if I'm running a web server that offers an
encrypted-message-forwarding service?  Possibly, if it can be shown that the
service is only really used to support a Napster-like network.
How about if I'm running an SMTP-based anonymous remailer, and publicly
promise to make connection logs available to law enforcement upon receipt of
an original subpoena document?  Now the attacker is on shaky ground.  In my
opinion, an ISP representative is more likely to discuss the situation with
his customer.

I think.

-----

As long as we're adding privacy-enhanced advanced-user features, let's
consider what a 'connection' means.

Could it be valuable to consider a path in the routing table to be more than
just a direct IP connection?  Perhaps a connection could be an encrypted
tunnel *through* the Blocks network...or a DNS tunnel...or an
anonymous-remailer tunnel.

Let someone else support the transport mechanism...we'll just build in
support for the protocol.  That way we aren't distributing a 'special use'
server with the software, making all traffic going through that type of
server incriminating.

-----

I know you guys don't have the time to read through and digest all of these
points as soon as you receive them...so I really appreciate the attention
and consideration people have been giving my crazy discussion ideas.

--Spence
bl...@ms...

Re: [Blocks-development] Filesharing concept discussion: limited resources, and queueing...

[Blat F. <pet...@ho...>]

Hey! Good to see there is still some interest in anonymous P2P
and maybe even Blocks :-)

>
>I've had some ideas gnawing at me for the past while, and if any of you are
>still out there...I'd like to bounce the ideas off of you guys before I try
>to post some incoherent ramblings to infoanarchy or shouldexist or
>something.
>
>I'd like to talk about some potential features for filesharing systems:
>
>1)  queues
>

Good idea, but wont people just disconnect when it says its going
to take more than 5 mins? Most users are very impatient, thats
why unfriendly ftp clients which open several connections to
the same file are so popular.

>
>2)  less interactivity of service...batch service?
>

Im not sure batch mode for user interaction is a viable idea,
however, for content replication between servers both
queuing and batch mode are pretty interesting. Obviously
its more elegant to have a single ubiqitous mechanism
for both if at all possible.

>
>3)  More human interaction
>

WoT is the key to stopping spamming, but its needed on the
adverts (or search results) rather than just the data. Since
there will be more info packets than data packets efficiency
is always going to be a problem. Perhaps you could send info
packets in larger 'conatiners' which are signed in some way?

>----
>
>BTW, early February I accepted a position as a programmer at a credit card
>processing company here in town.  I can't believe I was always afraid of
>Windows programming -- now that I have (financial) motivation to sit down
>and figure the stuff out, I'm doing great!
>
>Perhaps it's time to reinvent Blocks...but this time with two or more
>programmers.  ^_^

I think the idea is sound, but competeing with the likes of
BearShare is going to be difficult. What about perhaps
trying to design a distributed anonymous GNUtella backend?
I guess the sucky P2P protocol would allow the men in black
to shutdown the point of content tho :-(

>
>----
>
>With the recent developments in OpenNap server operators coming under
>attack, it's becoming clear:  no matter how legally sound your position is,
>someone with power can still lash out at you if they can find out who you
>are.

You have to make it ineffective cost wise to shut down
servers, and the bad guys have bundles of cash. Not impossible
though, even the old Blocks servers would require a lot
of effort to prove it did anything wrong.

>
>I've been thinking about how Blocks was designed.  If I'm remembering
>correctly...if we don't readvertise files to new connections, and if you
>make *sure* that you aren't connected to someone you don't trust at the 
>time
>you share a file, it's impossible for someone to pull a MediaEnforcer on 
>the
>Blocks network.

Yup, that was the key fact. You never know if the server you
are connected to is just a proxy or in fact the real server.
Assuming not everything gets cached (like freenet) and the
proxy logic has sufficient randomness in data acquisition
(doesnt need much) its a *huge* effort to apply statistical
analysis without being able to directly snoop on the
server in question.

>
>Think about that for a minute:  not even Freenet can claim that level of
>immunity.  On Freenet, the act of requesting content (to check for its
>presence) causes the content to be replicated to that machine.  There's 
>some
>legal protection there, although I'm not a lawyer.  However, the attacker
>can merely go to the ISP and say "I represent this really big organization,
>and we would appreciate it if you stopped this user from running this
>perfectly legal service.  Besides, the user is consuming a much higher than
>average portion of our upstream bandwidth, isn't it?  It'll save you money
>in the long run to remove users like this from your service! *nudge nudge*"
>A Freenet node can be shut down merely because it's a node on an 
>*unpopular*
>network.
>
>Blocks doesn't have established port numbers.  With a little work, it could
>even have a difficult-to-guess protocol, requiring either some 'hash cash'
>or a bit of information that proves you learned about the IP and port
>through the network, not through a host scan.
>
>Ooh...wouldn't that be cool?  Your first connection to the first server
>would require that you connect to a port, and without reading *any* data,
>compute and send 50-million-operations worth of hash-cash to a 'mute'
>socket, before the other end would even talk to you.  That would certainly
>chill efforts to scan hosts for open ports, checking to see if any ports 
>are
>running Blocks nodes.

Just listen to a few (say 5 ports) immediately above and below
the real port. If anyone tries to connects to these, shut down
the main socket and go into stealth mode for a while. That
would defeat 99% or scans. Wont stop people hacking the Blocks
protocol though, any distributed system needs to be able to
locate and connect to other nodes.

Actually thinking about it, locating servers is a task best
done outside of the servers themselves. You just need a
simple server which dishes out the last 100 ips/ports when
it receives a new connection. It can then add the new ip
(and I guess port) for this connection to the list. The
bad guys can shut it down because it has nothing to do
with data.

>
>----
>
>If my tone just changed...it's because within the last three paragraphs I
>suddenly realized that Blocks is providing something that *no other 
>service*
>can provide right now.  It's enough to make me want to pick it up and dust
>it off and start hacking again.

What about encryption? none of the others did that either :-)

>
>I just installed a new 40 GB hard disk.  Once I confirm that UDMA is OFF
>(it's a CRC-skipping Western Digital...it was cheap) I'll put a 30 GB 
>Blocks
>cache up on michael.mspencer.net port 8017 ^_^
>
>Have some anime.  Cowboy Bebop is definitely top-shelf anime, if you don't
>mind reading subtitles.  And it has nothing at all to do with cowboys or
>be-bop.  Ignore the funny name and give it a chance.  ^_^
>
>----
>
>In an attempt to make a 'real programmer' out of me, the CTO has had me
>flowcharting our existing code.  For the past two weeks.  I think I'm
>starting to like it.  *shudders*
>
>I'm going to try to create a flowchart of the current version of Blocks.
>Once I understand the code (and can help others understand the code, 
>through
>flowcharts), I'd like to start working on Blocks myself.  Probably starting
>with a Windows-only ugly stepchild version.

Heh, Blocks was never well designed and is full of kludgy bits.
The network code is quite well contained and very portable tho.
FLTK rocks... check it out before you decend into the hell that
is MFC.

>
>(I can see it now -- instead of falling into the usual trap of making a 
>full
>dialog page for connection information, I'll be like Napster...but I'll say
>"Disconnected"  "Kinda connected"  "Connected"  "Very connected")
>
>That does it.  I need to go to sleep.  Please, let me know what you think
>about both the broad filesharing system ideas and about dusting Blocks off
>and standing it on its feet again.
>
>Also...I've started hosting the Blocks homepage (kinda) at
>http://www.mspencer.net/blocks

Ive got them on CD somewhere... have to check.

I got my own internet connection in my apt now (10mbits!) but
its having a few teething problems, and the next couple of
months are going to be hectic workwise for me. Still, good
to hear there is still some interest in these concepts :-)

ttfn

PG.

_________________________________________________________________________
Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com.

Re: [Blocks-development] Filesharing concept discussion: limited resources, and queueing...

[Michael S. Jr. <m...@ms...>]

Thanks for the files Erik -- http://www.mspencer.net/blocks now feels whole.
(I'm just missing the 0.17 source...can't believe I deleted it...I'll look
tomorrow...)

Sudden idea!  If we decide to resurrect Blocks...

It's pretty much a given that most users don't want to run Blocks.  However,
some users who wish to contribute to the network have lots of technical
ability and lots of resources.  They're also the most likely to be leaned on
by oppressive governments.

"Market" Blocks as a distributed back-end for a filesharing network!  Create
a custom client, or extend an existing client, so an advanced user can join
the Blocks network and automatically share all files he sees on Blocks out
to the GnutellaNet or Napster network or IRC or whatever.

Then almost all Blocks nodes can be perceived as high-value targets, so we'd
need to have pretty good security.

What do you think?

I'll probably forget this idea by morning.  :)

--Spence
bl...@ms...

(BTW...that address, mspencer.net/blocks -- I consider that to be an
'official' homepage mirror.  I promise to never modify that page without the
permission and blessing of you guys.  If I have personal comments or news to
add about Blocks, I'll put that on a separate page.)



----- Original Message -----
From: "Erik Moeller" <mo...@sc...>
To: <blo...@li...>
Sent: Wednesday, February 28, 2001 11:35 PM
Subject: Re: [Blocks-development] Filesharing concept discussion: limited
resources, and queueing...


> On 28 Feb 2001, at 23:19, Michael Spencer Jr. wrote:
>
> > I was originally going to write up a long 'paper' about this...but I'm
> > lazy tonight.
>
> I'm still out there and working heavily on several p2p-related
> projects. I am thinking about building an open-source, uncensorable,
> decentralized general p2p platform, not only for file sharing. Many
> of the things we have talked about (some of which Peter has even
> implemented) in Blocks development will be useful there. But give me
> some time, I need to work out the details of my plan.
>
> I'll check out your posting later. If it's good, I want it at
> infoanarchy.org ;-)
>
> I'll send you the Blocks files I still have in a ZIP.
>
> Regards,
> Erik
>
> --
> Scientific Reviewer, Freelancer, Humanist -- Berlin/Germany
> Phone: +49-30-45491008 - Web: <http://www.humanist.de/erik>
> The Origins of Peace and Violence: <http://www.violence.de>
>
> "The right way to do things is not to try to persuade people you're
> right but to challenge them to think it through for themselves."
> (Noam Chomsky)
>
> _______________________________________________
> Blocks-development mailing list
> Blo...@li...
> http://lists.sourceforge.net/lists/listinfo/blocks-development

Re: [Blocks-development] Filesharing concept discussion: limited resources, and queueing...

[Erik M. <mo...@sc...>]

OK, the old Blocks files are now at

	http://www.scireview.de/p2p/blocks.zip

Regards,
Erik

-- 
Scientific Reviewer, Freelancer, Humanist -- Berlin/Germany
Phone: +49-30-45491008 - Web: <http://www.humanist.de/erik>
The Origins of Peace and Violence: <http://www.violence.de>

"The right way to do things is not to try to persuade people you're 
right but to challenge them to think it through for themselves."
(Noam Chomsky)

Re: [Blocks-development] Filesharing concept discussion: limited resources, and queueing...

[Erik M. <mo...@sc...>]

On 28 Feb 2001, at 23:19, Michael Spencer Jr. wrote:

> I was originally going to write up a long 'paper' about this...but I'm
> lazy tonight.

I'm still out there and working heavily on several p2p-related 
projects. I am thinking about building an open-source, uncensorable, 
decentralized general p2p platform, not only for file sharing. Many 
of the things we have talked about (some of which Peter has even 
implemented) in Blocks development will be useful there. But give me 
some time, I need to work out the details of my plan.

I'll check out your posting later. If it's good, I want it at 
infoanarchy.org ;-)

I'll send you the Blocks files I still have in a ZIP.

Regards,
Erik

-- 
Scientific Reviewer, Freelancer, Humanist -- Berlin/Germany
Phone: +49-30-45491008 - Web: <http://www.humanist.de/erik>
The Origins of Peace and Violence: <http://www.violence.de>

"The right way to do things is not to try to persuade people you're 
right but to challenge them to think it through for themselves."
(Noam Chomsky)

[Blocks-development] Filesharing concept discussion: limited resources, and queueing...

[Michael S. Jr. <m...@ms...>]

I was originally going to write up a long 'paper' about this...but I'm lazy
tonight.

I've had some ideas gnawing at me for the past while, and if any of you are
still out there...I'd like to bounce the ideas off of you guys before I try
to post some incoherent ramblings to infoanarchy or shouldexist or
something.

I'd like to talk about some potential features for filesharing systems:

1)  queues

Does anyone use fserv's on IRC?  I had started to write a technical
description of how IRC fserv's work, so a layperson or an academic could
understand them...someone let me know if you're interested in hearing it and
I'll finish and post it...

I think queuing or time-shifting file requests might help with the problem
of hosts getting swamped with requests, trying to maintain lots of slow
transfers and then failing them all.

I think it'd be nice to see a status of 'queued, 250 min until start'
instead of 'pending' or 'queued'.

This concept could probably also be applied to other services which were
originally designed to provide immediate service:  an ftp server could let a
transfer start but 'queue' it, sending data at 1 byte per 30 seconds, until
it's ready to start the transfer at full speed.

2)  less interactivity of service...batch service?

There's more client demand for data than server bandwidth available to
provide that data.  Servers will get swamped...and even though all users
have the expectation that their requests will all be handled immediately,
some requests will be handled slower than others, and some won't be handled
at all.

Also, there are different kinds of data on a network:  file advertisement or
search request data, network utility data, and file or content data.
Obviously network utility data should have higher priority than other types,
because the network is the transport the other types of data use.

Perhaps we should consider searches that take 24 hours to collect results,
but collect them in a more reliable way?

Perhaps we should consider 'server connections' to not equal TCP
connections?  What's wrong with a 'connection' that sends low volumes of
high-reliability traffic through e-mail, perhaps even using anonymous
remailers?  (Ideas:  with a *very* low volume of traffic, you could encode
your messages so they look human-readable.  With a mail host tolerant of
high volumes, you could have a more 'reliable' high-volume connection, with
a 10 MB buffer on the ISP's mail server, automatic dropping of inbound
connections with the buffer gets full, more tolerance of disconnects and
transient network conditions, etc.)

Perhaps some connections can be through DNS tunnels, etc.

3)  More human interaction

Once you have 'batch' communications, you can process and optimize the
communications in more effective ways...perhaps even involving a human in
some of the processing.

And once you have human interaction...some day in the future...maybe then my
crazy web-of-trust idea becomes valid.  As it was shown to me a while
back...currently there is no need for content sorting or filtering, no
matter how 'strong' the filtering might be.  There isn't a flood of content
out there right now.

What's everyone's thoughts on all this?

----

BTW, early February I accepted a position as a programmer at a credit card
processing company here in town.  I can't believe I was always afraid of
Windows programming -- now that I have (financial) motivation to sit down
and figure the stuff out, I'm doing great!

Perhaps it's time to reinvent Blocks...but this time with two or more
programmers.  ^_^

----

With the recent developments in OpenNap server operators coming under
attack, it's becoming clear:  no matter how legally sound your position is,
someone with power can still lash out at you if they can find out who you
are.

I've been thinking about how Blocks was designed.  If I'm remembering
correctly...if we don't readvertise files to new connections, and if you
make *sure* that you aren't connected to someone you don't trust at the time
you share a file, it's impossible for someone to pull a MediaEnforcer on the
Blocks network.

Think about that for a minute:  not even Freenet can claim that level of
immunity.  On Freenet, the act of requesting content (to check for its
presence) causes the content to be replicated to that machine.  There's some
legal protection there, although I'm not a lawyer.  However, the attacker
can merely go to the ISP and say "I represent this really big organization,
and we would appreciate it if you stopped this user from running this
perfectly legal service.  Besides, the user is consuming a much higher than
average portion of our upstream bandwidth, isn't it?  It'll save you money
in the long run to remove users like this from your service! *nudge nudge*"
A Freenet node can be shut down merely because it's a node on an *unpopular*
network.

Blocks doesn't have established port numbers.  With a little work, it could
even have a difficult-to-guess protocol, requiring either some 'hash cash'
or a bit of information that proves you learned about the IP and port
through the network, not through a host scan.

Ooh...wouldn't that be cool?  Your first connection to the first server
would require that you connect to a port, and without reading *any* data,
compute and send 50-million-operations worth of hash-cash to a 'mute'
socket, before the other end would even talk to you.  That would certainly
chill efforts to scan hosts for open ports, checking to see if any ports are
running Blocks nodes.

----

If my tone just changed...it's because within the last three paragraphs I
suddenly realized that Blocks is providing something that *no other service*
can provide right now.  It's enough to make me want to pick it up and dust
it off and start hacking again.

I just installed a new 40 GB hard disk.  Once I confirm that UDMA is OFF
(it's a CRC-skipping Western Digital...it was cheap) I'll put a 30 GB Blocks
cache up on michael.mspencer.net port 8017 ^_^

Have some anime.  Cowboy Bebop is definitely top-shelf anime, if you don't
mind reading subtitles.  And it has nothing at all to do with cowboys or
be-bop.  Ignore the funny name and give it a chance.  ^_^

----

In an attempt to make a 'real programmer' out of me, the CTO has had me
flowcharting our existing code.  For the past two weeks.  I think I'm
starting to like it.  *shudders*

I'm going to try to create a flowchart of the current version of Blocks.
Once I understand the code (and can help others understand the code, through
flowcharts), I'd like to start working on Blocks myself.  Probably starting
with a Windows-only ugly stepchild version.

(I can see it now -- instead of falling into the usual trap of making a full
dialog page for connection information, I'll be like Napster...but I'll say
"Disconnected"  "Kinda connected"  "Connected"  "Very connected")

That does it.  I need to go to sleep.  Please, let me know what you think
about both the broad filesharing system ideas and about dusting Blocks off
and standing it on its feet again.

Also...I've started hosting the Blocks homepage (kinda) at
http://www.mspencer.net/blocks

I'm missing the graphics though.  Can someone mail those to me, if you have
them?

--Spence
bl...@ms...

Re: [Blocks-development] FAQ entry: What do I do if I get a cease-and-desist letter?

[<lu...@et...>]

* Spencer Jr., Michael wrote:
> Suppose someone is investigating which content is being served on Blocks.
> They connect to me, the 'central server', remain connected for a while, and
> they receive a file advert for a copyrighted file.  Basically, suppose they
> found a reason to send a cease-and-desist letter to someone.
> 
> Since they have no way of figuring out who my connection 'DGAH' is (for
> example), the attacking lawyer sends *me* the cease-and-desist letter.  What
> would I do?

Dont' have the time to go into this in depth (*sigh*), but when I was
tossing this around with a friend we came up with the concept of "null
routes".  Your sever picks a few letters to use as fake routes, using them
to pad out the routes to local blocks to a random large length.  Example:

I've got the password to Bill Clinton's diary on my server.  When advertising
it, my server uses the route XABDADBCA to describe it.  If someone is
interested in the file, my server will see the X and know that it should
ignore the rest of the route and look on the local disk for the file.

(This is vulnerable to some pretty simple attacks that will eventually figure
out what a given server's 'null routes' are, but it's a start...)

I'd still like to take a vacation and take a good hard look at the
protocol, since I think a lot of the problems inherent in P2P will
need to be solved there.

Free time.  Wish I rmemeber what it looked like.

> Goal:  construct a scenario such that the only way to stop a certain
> filesharing network would be to control email, en masse. 

Cute.  Brings to mind the thing that someone did in the UK to point
out how stupid a law was.  (Encrypted a confession and sent it to an MP.
by the laws of the land, the MP was now breaking the law because he had
posession of details about a crime and hadn't told the police.  Or something.)

-Luke

[Blocks-development] FAQ entry: What do I do if I get a cease-and-desist letter?

[Spencer Jr., M. <sp...@ac...>]

No, I haven't actually received a C&D letter from someone.  But I'm curious:

Suppose someone is investigating which content is being served on Blocks.
They connect to me, the 'central server', remain connected for a while, and
they receive a file advert for a copyrighted file.  Basically, suppose they
found a reason to send a cease-and-desist letter to someone.

Since they have no way of figuring out who my connection 'DGAH' is (for
example), the attacking lawyer sends *me* the cease-and-desist letter.  What
would I do?

On one hand, I can insist that even though the attacker is receiving the
file data from me, the data isn't coming from me:  he should really be going
after my connection DGAH.  ('Sure', I'll say over the phone, 'just mail me a
registered copy of the original subpoena from a judge, and I'll tell you
what IP and port my connection D is.')

On the other hand, it's possible that I have four servers set up on my local
machine, and have the other three firewalled so they cannot communicate with
the outside world, except through my main server.  So I really could be
serving the content myself, using the complexities of the Blocks protocol to
hide behind.

Do you guys think I would stand a reasonable chance of withstanding a
cease-and-desist letter?

Or more to the point, shouldn't we make a FAQ entry about 'why you will not
receive a cease-and-desist letter if you run this software' or 'why you will
be able to defend yourself if you receive a cease-and-desist letter'.

---------------------

Lawyer-proofing:

Especially with the MediaEnforcer news lately, it seems like it would be fun
to make software deliberately designed to be a test-case for information
freedom.

Goal:  construct a scenario such that the only way to stop a certain
filesharing network would be to control email, en masse.  Internet email --
POP3 and SMTP servers and connections.  Imagine the media frenzy that would
ensue when our own artist-associations attempt to turn the internet into a
police state -- and fail.  Even grandma can understand what is wrong with a
company that wants to monitor all internet email because some users are
using auto-responding programs to arrange filesharing connections.

Imagine message-passing through email.  Hotmail accounts, even.

Hmm...would we be accused of spamming, as we both send and receive a few
dozen internet emails from a certain host?

Sound like fun?  :)

--Michael Spencer
m...@ms...

RE: [Blocks-development] Ack, it's been three months!

[Spencer Jr., M. <sp...@ac...>]

Not much flying lately -- I've been waiting for the snowy and cloudy weather
to clear up.  Gary's plane isn't through its annual right now -- it needs
some very expensive fireproof carpet in the pilot/copilot seat area before
it'll pass its annual FAA inspection.

(Sorry about the sudden spam everyone, if you forgot about this list.)

I would absolutely love the chance to host SNAKE and Blocks on mspencer.net.
Email me the files to serve and they'll be up in no time.  (Finally...I've
had a dedicated Linux box on mspencer.net for months now, and never had a
real use for it other than DNS and SMTP.)

So it seems the world won't be very interested in a new filesharing
protocol.  Perhaps the Blocks project can move toward layering a new project
on top of the existing Gnutella network.  Or, that is...perhaps we should
re-engineer the Blocks project to sit on top of the Gnutella network (with
all its problems).

(Obviously we don't want our testing network to become flooded, just like
Gnutella...so perhaps we should only connect to authorized clients while
we're still testing...and run a 'public' network on the side when we do
major releases.)

I've been brainstorming lately about this web-of-trust thing living on top
of Gnutella.  I'm a very unpracticed programmer, although I've taken several
classes...so I'm probably going to write something in Win32 to do a
web-of-trust over a local filesystem, not over a filesharing network.  (The
hooks will be there, though, to add a connection to a filesharing system
later.)  If there's any interest in this, I'd love to hear about it.  :)

OK, I'm glad to hear that we all haven't become an X-file.  :)

--Michael Spencer
msp...@ac... (work)
m...@ms... (home)



> -----Original Message-----
> From: Blat Froop [mailto:pet...@ho...]
> Sent: Wednesday, January 10, 2001 8:50 AM
> To: m...@ms...
> Cc: blo...@li...
> Subject: Re: [Blocks-development] Ack, it's been three months!
> 
> 
> >I know we've all had our interests pulled in new directions 
> since we were 
> >working on Blocks.
> 
> Hey Michael, hows things? Still flying?
> 
> >Is there still interest?  Is anyone still interested in 
> continuing the 
> >project?
> 
> Interest has sort of tapered off. It didnt really get enough
> initial momentum, but it was interesting while it lasted.
> I guess the project is dormant now. I will email the
> sourceforge/freshmeat people about its new status.
> 
> >Also, I can host the project page on mspencer.net's web server.
> 
> It would be nice to find a spot to host the Blocks stuff
> and maybe the SNAKE web page as well. Blocks may be
> dormant, but there is some good platform independent code
> in it that might be salvageable for other projects :-)
> 
> Im real busy moving apartment and with work just now, but
> I also plan to build my own PC in the next couple of weeks
> and my new apartment should have an 11Mbits internet connection
> so I should be finally online 24/7. Having said that, as
> soon as the snow goes away its Paragliding time! I still
> havent finished my course, but I now own a paraglider,
> and I need to fly.
> 
> ttfn
> 
> PG.
> 
> PS To everyone who helped out on Blocks...
>    Thanks! It was fun. ;-)
> 
> 
> ______________________________________________________________
> ___________
> Get Your Private, Free E-mail from MSN Hotmail at 
http://www.hotmail.com.


_______________________________________________
Blocks-development mailing list
Blo...@li...
http://lists.sourceforge.net/mailman/listinfo/blocks-development

Re: [Blocks-development] Ack, it's been three months!

[Blat F. <pet...@ho...>]

>I know we've all had our interests pulled in new directions since we were 
>working on Blocks.

Hey Michael, hows things? Still flying?

>Is there still interest?  Is anyone still interested in continuing the 
>project?

Interest has sort of tapered off. It didnt really get enough
initial momentum, but it was interesting while it lasted.
I guess the project is dormant now. I will email the
sourceforge/freshmeat people about its new status.

>Also, I can host the project page on mspencer.net's web server.

It would be nice to find a spot to host the Blocks stuff
and maybe the SNAKE web page as well. Blocks may be
dormant, but there is some good platform independent code
in it that might be salvageable for other projects :-)

Im real busy moving apartment and with work just now, but
I also plan to build my own PC in the next couple of weeks
and my new apartment should have an 11Mbits internet connection
so I should be finally online 24/7. Having said that, as
soon as the snow goes away its Paragliding time! I still
havent finished my course, but I now own a paraglider,
and I need to fly.

ttfn

PG.

PS To everyone who helped out on Blocks...
   Thanks! It was fun. ;-)


_________________________________________________________________________
Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com.

[Blocks-development] Ack, it's been three months!

[Michael S. Jr. <m...@ms...>]

I know we've all had our interests pulled in new directions since we were
working on Blocks.

Is there still interest?  Is anyone still interested in continuing the
project?

(Should I put my server at mspencer.net:8017 back up?  :-) )

Also, I can host the project page on mspencer.net's web server.

--Michael Spencer
m...@ms...
(not hiding behind bl...@ms... any more)

Re: [Blocks-development] Strange direction for Blocks: "building-blocks" :)

[Blat F. <pet...@ho...>]

>From: "Michael Spencer Jr." <bl...@ms...>
>To: <blo...@li...>
>Subject: [Blocks-development] Strange direction for Blocks:  
>"building-blocks" :)
>Date: Mon, 9 Oct 2000 23:50:17 -0500
>
>OK, I had a crazy idea.  This might not be appropriate for Blocks, but it's
>certainly relevant to the problem Blocks tries to solve.
>
>Idea in short:  filesharing system is specified in layers roughly according
>to the OSI model.  Each layer is specified by an interpreted scripting
>language.
>
>Detail:
>
[snip]
>
>What do you guys think?

If I read correctly:

You are making the assuption that since Blocks, GNUtella,
Napster, and maybe even FTP, HTTP, etc. all do the same
thing it would be possible to encpsulate all that
functionality in some common interface. However, since the
interface is a bit fuzzy a scripting language will help
fill in the gaps as well as allow people on the net to
expand the support to other protocols. A sort of file
shareing Emacs without the sandals :)

A few high level observations:

* It would have to be a really high level interface...
  GetFile(), PutFile(), FindFiles() etc, and it would
  be difficult to add complex features like resuming
  without making it pretty complex.

* Why not just write it in perl then perl hackers can
  extend it? That having been said, I have created a
  scripting language and Ive been looking for somewhere
  to use it for ages :)

* In order for the scripting language to be able to have
  customised GUIs for different protocols (config etc)
  it would require the scripting language to have
  GUI commands. In essence we would have to create something
  akin to Visual Basic which would be no trivial task.
  It might be possible to have a static GUI, but if we
  know everything it needs to do then why do we need
  scripting?

* The layers wouldnt be the same as the ISO model, there
  would be less. I'd be iterested to see a list of the
  layers you are suggesting.

* Why not use a browser as the front end and Java as the
  scripting language :)

Just my $0.02 worth,

Its 10am I need coffee :)

ttfn

PG.




_________________________________________________________________________
Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com.

Share information about yourself, create your own public profile at 
http://profiles.msn.com.

[Blocks-development] Strange direction for Blocks: "building-blocks" :)

[Michael S. Jr. <bl...@ms...>]

OK, I had a crazy idea.  This might not be appropriate for Blocks, but it's
certainly relevant to the problem Blocks tries to solve.

Idea in short:  filesharing system is specified in layers roughly according
to the OSI model.  Each layer is specified by an interpreted scripting
language.

Detail:

Like any other filesharing program, this would have an interface for
connecting to the network, searching, downloading, uploading, chatting, and
anything else people can think of.  All of these things are abstracted --
the user doesn't need to know they're communicating with a Napster or a
Gnutella or a Blocks network.

Under the surface, though, the program is using scripted code to work with
each layer of the system.  These scripts have been designed by someone who
is *not* the end-user, and they enable the program to talk to different
networks.  Each script represents a part of the system according to the
seven-layer OSI burrito -- network, protocol, session, application, etc.

This system is beneficial so far because it allows you to easily connect to,
search, and download from multiple network types from one interface.  When a
new network type becomes popular (MojoNation?), simply add scripts for it.
When a proprietary network service tweaks its protocol a bit to break
existing clients, simply tweak your scripts a little bit to match.

The scripting language is what makes this system interesting.  Has anyone
seen Nessus?  It's an open-source security scanner that specifies attack
probes with a custom scripting language called NASL.  You script the attack
with NASL -- what to send, what to expect to receive, what to do in case of
any of a number of responses, etc.  Let's imitate them.  :)

Even though network system design is pretty far out of the reach of most
end-users, I believe the scripting language should be transparent enough for
end-users.

In text form, I imagine the scripts will be BASIC-like (yes, the language
BASIC) and use a very simple set of commands specific to the kind of script.
 In the case of protocols, the script specifies which messages can be
passed, and what the internal format of the messages will be.  For sessions,
the script specifies what actions are taken when each message is received.
(I imagine session and application code will be algorithmic and complex, so
an elegant task-specific language is called for.)

In GUI form, I imagine we can create abstracted pieces and join them
together like flowchart boxes, lego bricks, or any other UI metaphor you
prefer.  Very Hollywood-looking.

If properly implemented, the system gets some interesting properties:
  *  Interoperability with multiple networks and systems!  Want a program
that seamlessly integrates with Napster, iMesh, Gnutella, Blocks, and IRC
channels, with one search interface and one upload/download interface?
Download the program, customize as needed, and you're all set!
  *  Protocol designers who can't code can now design and test new systems.
  *  Therefore, the system can face new threats and new network demands more
rapidly.  Any power-user can make changes:  implementing IRC-style queues in
Gnutella, for example.

Cons?
  *  Chaos!  Ten million little protocols, each with enough insignificant
differences to make them incompatible.  Hmm...maybe we can implement code
signatures and version control.  We need to prevent Joe Sixpack from making
a random change to something and not knowing why his program doesn't work
any more, while still allowing people who know what they're doing to make
necessary design changes.
  *  Script-kiddie heaven!  Attackers can use the network against itself.  I
think there's a 'security-through-obscurity' argument in here somewhere.
This will cause the buggy networks to die and the healthy networks to
thrive.

What do you guys think?

--Michael Spencer
bl...@ms...