libbt-devel — C Library for Bittorrent Developers

Re: [Libbt-devel] restructuring

[Alien <ali...@us...>]

Op donderdag 24 februari 2005 05:33, schreef Elliott Mitchell:
> >From: Alien <ali...@us...>
> > it looks like the benc types are all in types.h and as i understand that
> > most things started there, this is not how things should be:
> >
> > I intend to make the benc types and functions use benc instead of bt
> > prefix. combined with making the bencObject_t a union instead of how it
> > is now. furthermore, it has seemed to me that this allocated item that's
> > in every benc type looks useless. it seems that this type should be
> > filled out in one burst, to improve efficiency, (especially if frontends
> > would use multithreading). of course i could be wrong, so tell me.
>
> Sorry for the slow commentary, my BT energy has been spent elsewhere. My
> copy of the tree is also somewhat behind due to this.
>
> Unfortunatly a nice simple answer to the above two. NO.
>
> bt vs benc, look at what those structures are holding. They're holding
> digested values, not raw benc data. So bt* seems more appropriate to me.
>
>
> You clearly didn't examine how btObject is used closely enough though.
> You missed that the btObject is at the top of the structure, no pointers
> involved. They are allocated as a single blob. btObject has data that is
> required to be present in all of them, for which constructions of this
> sort make sense.
>
> Spend the time to look at them, and don't touch it until you understand
> why it was done. Key idea, this is object-oriented programming in C.
>
> My tendancy is to have the type information be a pointer to a jump table
> (notably the first thing being the deallocation function), rather than an
> enum. The current way works though.

very well, i won't touch it

Re: [Libbt-devel] restructuring

[Elliott M. <eh...@m5...>]

>From: Alien <ali...@us...>
> it looks like the benc types are all in types.h and as i understand that most 
> things started there, this is not how things should be:
> 
> I intend to make the benc types and functions use benc instead of bt prefix. 
> combined with making the bencObject_t a union instead of how it is now. 
> furthermore, it has seemed to me that this allocated item that's in every 
> benc type looks useless. it seems that this type should be filled out in one 
> burst, to improve efficiency, (especially if frontends would use 
> multithreading). of course i could be wrong, so tell me.

Sorry for the slow commentary, my BT energy has been spent elsewhere. My
copy of the tree is also somewhat behind due to this.

Unfortunatly a nice simple answer to the above two. NO.

bt vs benc, look at what those structures are holding. They're holding
digested values, not raw benc data. So bt* seems more appropriate to me.


You clearly didn't examine how btObject is used closely enough though.
You missed that the btObject is at the top of the structure, no pointers
involved. They are allocated as a single blob. btObject has data that is
required to be present in all of them, for which constructions of this
sort make sense.

Spend the time to look at them, and don't touch it until you understand
why it was done. Key idea, this is object-oriented programming in C.

My tendancy is to have the type information be a pointer to a jump table
(notably the first thing being the deallocation function), rather than an
enum. The current way works though.


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \   (    |         EH...@gr... PGP 8881EF59         |    )   /
  \_  \   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
    \___\_|_/82 04 A1 3C C7 B1 37 2A*E3 6E 84 DA 97 4C 40 E6\_|_/___/

[Libbt-devel] restructuring

[Alien <ali...@us...>]

in order to be able to compile a true shared library, i've been adding a few 
header files in include, that'll dictate how the future structure will look 
like.

I started with copy pasting the types from all over and writing some functions 
declarations and how they could look. as this is far from incomplete, i'll 
continue writing on this, until this is done, before i go changing everything 
and probably break cvs HEAD.

I've seen a few header files, and this presents a structural thing that really 
should be changed.

it looks like the benc types are all in types.h and as i understand that most 
things started there, this is not how things should be:

I intend to make the benc types and functions use benc instead of bt prefix. 
combined with making the bencObject_t a union instead of how it is now. 
furthermore, it has seemed to me that this allocated item that's in every 
benc type looks useless. it seems that this type should be filled out in one 
burst, to improve efficiency, (especially if frontends would use 
multithreading). of course i could be wrong, so tell me.

anyway, i expect your thoughts about this...

AL13N

Re: [Libbt-devel] the port [was: header files]

[Alien <ali...@us...>]

Op maandag 14 februari 2005 00:55, schreef Peter Stuge:
> On Sun, Feb 13, 2005 at 11:31:27PM +0100, Alien wrote:
> > Op zaterdag 12 februari 2005 15:15, schreef Peter Stuge:
> > > > If not a single port than atleast a range that can also be
> > > > forwarded.
> > >
> > > There is no reason to use more than one port.
> >
> > and a user defined range to pick one out at random? i'm just saying
> > that the user might want to have 567-574 and have 4 appl using
> > libbt or an appl using 4 instances of libbt just want it?
>
> Fair enough. Four applications with libbt need separate ports, but
> then the port is a setting in the application rather than libbt. And
> if there is no setting, just use any free dynamic port. As for the
> application, it can either inform libbt about the setting, or do the
> work on it's own and provide libbt a socket after bind() and
> listen().
>
> IMHO, an application shouldn't ever need to and possibly not even be
> able to "start" more than one instance of libbt, this is one reason,
> there are probably more.

you have a point, but i'm just saying that if libbt can accept a port, it=20
could very well even accept a port range, where it randomly can choose from=
=2E=20
of course, the library should get everything do everything on one port.

Re: [Libbt-devel] the port [was: header files]

[Peter S. <stu...@cd...>]

On Sun, Feb 13, 2005 at 11:31:27PM +0100, Alien wrote:
> Op zaterdag 12 februari 2005 15:15, schreef Peter Stuge:
> > > If not a single port than atleast a range that can also be
> > > forwarded.
> >
> > There is no reason to use more than one port.
> 
> and a user defined range to pick one out at random? i'm just saying
> that the user might want to have 567-574 and have 4 appl using
> libbt or an appl using 4 instances of libbt just want it?

Fair enough. Four applications with libbt need separate ports, but
then the port is a setting in the application rather than libbt. And
if there is no setting, just use any free dynamic port. As for the
application, it can either inform libbt about the setting, or do the
work on it's own and provide libbt a socket after bind() and
listen().

IMHO, an application shouldn't ever need to and possibly not even be
able to "start" more than one instance of libbt, this is one reason,
there are probably more.


//Peter

Re: [Libbt-devel] the port [was: header files]

[Alien <ali...@us...>]

Op zaterdag 12 februari 2005 15:15, schreef Peter Stuge:
> > If not a single port than atleast a range that can also be
> > forwarded.
>
> There is no reason to use more than one port.

and a user defined range to pick one out at random? i'm just saying that th=
e=20
user might want to have 567-574 and have 4 appl using libbt or an appl usin=
g=20
4 instances of libbt just want it?

Re: [Libbt-devel] the port [was: header files]

[Peter S. <stu...@cd...>]

On Fri, Feb 11, 2005 at 01:38:11PM -0800, Elliott Mitchell wrote:
> >From: Peter Stuge <stu...@cd...>
> > 
> > No, listenport is not required, if we're going to do the networking
> > in libbt I strongly suggest that we only use a single port per
> > application, and by default one in the dynamic port range as defined
> > by IANA, 49152 through 65536. Start from the bottom and try
> > allocating upwards.
> 
> It is standard practice for BT clients to start at 6881 and go up as
> high as 6889. This is semi-silly, but it does make it easier to
> implement MitM caches if so desired.

It's completely silly, since some of those ports are reserved by IANA
and some are unassigned. In either case it is really inappropriate to
use them.


> Failing that, taking whichever port the OS gives to you is standard
> practice. This typically starts at 1024 and goes upwards as various
> programs use temporary ports.

For outgoing connections, yes. For incoming connections, no, not in
Linux at least. I just tried;

stuge@carepad4 ~/a $ cat a.c
#include <sys/socket.h>
#include <sys/types.h>
#include <unistd.h>
#include <stdio.h>
#include <netinet/in.h>

int main() {
  int s,i;  
  struct sockaddr_in sa;
  struct sockaddr *sap=(struct sockaddr *)&sa;
  s=socket(PF_INET,SOCK_STREAM,0);
  listen(s,5);  
  getsockname(s,sap,&i);
  printf("got ip %s port
%d\n",inet_ntoa(sa.sin_addr),ntohs(sa.sin_port));
  close(s);
  return 0;
}

--8<-- ip(7)
When listen(2) or connect(2) are called on a unbound socket the
socket is automatically bound to a random free port with the local
address set to INADDR_ANY.
-->8--

stuge@carepad4 ~/a $ ./a
got ip 0.0.0.0 port 57839

If I run it over and over again it steers clear of ports used by
netfilter:

stuge@carepad4 ~/a $ while :;do ./a;done
[..]
got ip 0.0.0.0 port 60999
got ip 0.0.0.0 port 61000
got ip 0.0.0.0 port 32768
got ip 0.0.0.0 port 32769


On Fri, Feb 11, 2005 at 01:47:34PM -0800, Tyler MacDonald wrote:
> Elliott Mitchell <eh...@m5...> wrote:
> > It is standard practice for BT clients to start at 6881 and go up as
> > high as 6889. This is semi-silly, but it does make it easier to
> > implement MitM caches if so desired.
> 
> 	I vote for *NOT* using those ports by default. Bram Cohen has
> said that it was a bad idea to begin with,

It was.


> 	That said, I think it would be nice to give the application
> the chance to decide what port it wants to listen on, and maybe even
> pass it's own socket fd to the library and say "here, use this"...
> but if it doesnt take that chance, proceed with allocating a random
> port either by not specifying one to bind, or by picking one in the
> IANA range that's available.

This is just a few lines of code we're discussing, but I still think
it's important to get right.

If user specifies port, try to use that. If it fails, give up and
complain to user.

If there's no user setting, try listen() and check that we got a port
in the dynamic range. Linux can fail here, as shown above. If not,
close the socket and make a new one. (How will this affect the IP
stack? This isn't playing really nice with it, I can imagine that
Windows would get upset by this kind of thing.)

If there's still no good socket to accept() on, try bind()ing each
port in the dynamic range.

If that fails, give up and complain to user.


On Fri, Feb 11, 2005 at 05:45:48PM -0500, Nathan Ford wrote:
> I think it would be best for the user to be able to set the port
> (and use this one port for all torrent transfers) so that users
> behind firewalls can forward the port.

Fully agreed. I need that. User setting will always have precedence.


> If not a single port than atleast a range that can also be
> forwarded.

There is no reason to use more than one port.


//Peter

Re: [Libbt-devel] added header files that will be installed

[Nathan F. <zi...@gm...>]

I think it would be best for the user to be able to set the port (and
use this one port for all torrent transfers) so that users behind
firewalls can forward the port. If not a single port than atleast a
range that can also be forwarded.

--Nate

On Fri, 11 Feb 2005 13:47:34 -0800, Tyler MacDonald
<ty...@ac...> wrote:
> Elliott Mitchell <eh...@m5...> wrote:
> > > No, listenport is not required, if we're going to do the networking
> > > in libbt I strongly suggest that we only use a single port per
> > > application, and by default one in the dynamic port range as defined
> > > by IANA, 49152 through 65536. Start from the bottom and try
> > > allocating upwards.
> > It is standard practice for BT clients to start at 6881 and go up as
> > high as 6889. This is semi-silly, but it does make it easier to implement
> > MitM caches if so desired.
> 
>        I vote for *NOT* using those ports by default. Bram Cohen has said
> that it was a bad idea to begin with, azureus and other clients allocate in
> the IANA range now, and some ISPs have begun filtering or severely
> throttling ports in the "standard" BT range.
> 
>        That said, I think it would be nice to give the application the
> chance to decide what port it wants to listen on, and maybe even pass it's
> own socket fd to the library and say "here, use this"... but if it doesnt
> take that chance, proceed with allocating a random port either by not
> specifying one to bind, or by picking one in the IANA range that's
> available.
> 
>        - Tyler
> 
> 
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real users.
> Discover which products truly live up to the hype. Start reading now.
> http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> _______________________________________________
> Libbt-devel mailing list
> Lib...@li...
> https://lists.sourceforge.net/lists/listinfo/libbt-devel
>

Re: [Libbt-devel] added header files that will be installed

[Tyler M. <ty...@Ac...>]

Elliott Mitchell <eh...@m5...> wrote:
> > No, listenport is not required, if we're going to do the networking
> > in libbt I strongly suggest that we only use a single port per
> > application, and by default one in the dynamic port range as defined
> > by IANA, 49152 through 65536. Start from the bottom and try
> > allocating upwards.
> It is standard practice for BT clients to start at 6881 and go up as
> high as 6889. This is semi-silly, but it does make it easier to implement
> MitM caches if so desired.

	I vote for *NOT* using those ports by default. Bram Cohen has said
that it was a bad idea to begin with, azureus and other clients allocate in
the IANA range now, and some ISPs have begun filtering or severely
throttling ports in the "standard" BT range.

	That said, I think it would be nice to give the application the
chance to decide what port it wants to listen on, and maybe even pass it's
own socket fd to the library and say "here, use this"... but if it doesnt
take that chance, proceed with allocating a random port either by not
specifying one to bind, or by picking one in the IANA range that's
available.

	- Tyler

Re: [Libbt-devel] added header files that will be installed

[Elliott M. <eh...@m5...>]

>From: Peter Stuge <stu...@cd...>
> On Fri, Feb 11, 2005 at 09:09:28AM +0100, Alien wrote:
> > bt.h also includes bttypes.h so it's easier not to make mistakes
> 
> >From btTorrent:
> 
>     int listenport;             /* is this required??? */
> 
> No, listenport is not required, if we're going to do the networking
> in libbt I strongly suggest that we only use a single port per
> application, and by default one in the dynamic port range as defined
> by IANA, 49152 through 65536. Start from the bottom and try
> allocating upwards.

It is standard practice for BT clients to start at 6881 and go up as
high as 6889. This is semi-silly, but it does make it easier to implement
MitM caches if so desired.

Failing that, taking whichever port the OS gives to you is standard
practice. This typically starts at 1024 and goes upwards as various
programs use temporary ports.


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \   (    |         EH...@gr... PGP 8881EF59         |    )   /
  \_  \   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
    \___\_|_/82 04 A1 3C C7 B1 37 2A*E3 6E 84 DA 97 4C 40 E6\_|_/___/

Re: [Libbt-devel] RFC: lists

[Elliott M. <eh...@m5...>]

>From: Alien <ali...@us...>
> a list of torrents, a list of files in a torrent, a list of peers per 
> torrent,...
> 
> now i don't want to sound biased, but of course i am; but I have programmed in 
> the not-so-long-ago past a library in C that handles this efficiently.
> 
> dynamic arrays with growing and shrinking. --> http://dynarr.sourceforge.net/
> 
> well, since i'm quite biased (and proud of my creation), i cannot decide to 
> use this. you may feel that you do not want this kind of efficiency or to 
> have another dependency.

No.

We're talking about removing the use of libevent because libevent is
rare. Your library is likely much rarer than libevent. Sure, useful
functionality, but how how many competent programmers haven't
implemented such a thing in the past? This is a grade-school level type
experience.

I also note that glib provides exactly the same functionality, and
libglib is very commonly available. I have to imagine that the common
library plib also likely has it.

No. Not worth the trouble.


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \   (    |         EH...@gr... PGP 8881EF59         |    )   /
  \_  \   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
    \___\_|_/82 04 A1 3C C7 B1 37 2A*E3 6E 84 DA 97 4C 40 E6\_|_/___/

Re: [Libbt-devel] question about btlist

[Peter S. <stu...@cd...>]

(Again, please everyone, check out the list-reply function in your
mailers, it's a great help to limit unneccessary traffic.

--8<-- http://docs.kde.org/en/3.3/kdepim/kmail/menus.html
Message->Reply to Mailing-List... (L)

    Opens up the composer window, inserts the quoted text of the
currently selected message and presets the To: field with the
mailing-list address. If you didn't specify a mailing-list address
for the currently selected folder and KMail can't determine the
posting address from the currently selected message then the To:
field will be empty. Your identity will automatically be set to the
one which this message was sent to.
-->8--

In mutt, there's a list-reply command that I've bound to l in my
setup, and changed limit to L:

--8<-- .muttrc
bind index l list-reply
bind index L limit
bind pager l list-reply
-->8--

)

On Fri, Feb 11, 2005 at 04:38:36PM +0100, Alien wrote:
> > --8<--
> > 'pieces' maps to a string whose length is a multiple of 20. It is to
> > be subdivided into strings of length 20, each of which is the sha1
> > hash of the piece at the corresponding index.
> > -->8--
> >
> > So yes, it's supposed to be long and binary.
> 
> i'm just saying that we could let btlist (or btcheck, i forgot) show
> it as hexadecimal.

Oh, ok! Sure, that could be done, but since the pieces are abstracted
away inside libbt there's limited use for the hashes..


//Peter

Re: [Libbt-devel] RFC: lists

[Peter S. <stu...@cd...>]

On Fri, Feb 11, 2005 at 04:32:50PM +0100, Alien wrote:
> the thing is, i'm relunctant to give it 1.0.0 status, what if
> something didn't work?

Release 1.1! :)


//Peter

Re: [Libbt-devel] question about btlist

[Alien <ali...@us...>]

Op vrijdag 11 februari 2005 15:59, schreef Peter Stuge:
> On Thu, Feb 10, 2005 at 02:42:02PM +0100, Alien wrote:
> >          0x502ab8(STRING)=3D(6)'pieces'
> >          =3D>
> >             0x502a50(STRING)=3D(156040)'[EAd???g?????f????????,1U??
> > ????...'
> >
> >
> > this last string is that normal? a huge string? i thought the
> > pieces were supposed to be there? which it would have to be a
> > number?
> >
> > thanks for any info relating to this
>
> Quoting docs/protocol.txt:
> --8<--
> 'pieces' maps to a string whose length is a multiple of 20. It is to
> be subdivided into strings of length 20, each of which is the sha1
> hash of the piece at the corresponding index.
> -->8--
>
> So yes, it's supposed to be long and binary.

i'm just saying that we could let btlist (or btcheck, i forgot) show it as=
=20
hexadecimal.

Re: [Libbt-devel] RFC: lists

[Alien <ali...@us...>]

Op vrijdag 11 februari 2005 15:41, schreef Peter Stuge:
> On Fri, Feb 11, 2005 at 09:30:28AM +0100, Alien wrote:
> > dynamic arrays with growing and shrinking. -->
> > http://dynarr.sourceforge.net/
>
> Besides being version 0.0.x with comments like "for now" in the
> documentation (sorry if I understood that wrong, only skimmed it)
> it looks useful, but in our case I think that malloc(), realloc()
> and free() of pointers and structs is simple enough not to warrant
> the extra dependency, similar to the select()/libevent case..

well, the documentation might be a little out of date, the last versions ha=
ve=20
been very well tested (the dynarr and the buffers). but even so, a few extr=
a=20
additions have been put into CVS and I was planning to add some functions t=
o=20
make it easier to do operations on dynarrs (eg: seeking). the thing is, i'm=
=20
relunctant to give it 1.0.0 status, what if something didn't work?

> Don't get me wrong, I don't want to shoot anyone down, but zero
> depenencies is really nice. On the other hand, if there's one
> dependency, two isn't all that much worse.

lol

Re: [Libbt-devel] added header files that will be installed

[Peter S. <stu...@cd...>]

On Fri, Feb 11, 2005 at 09:09:28AM +0100, Alien wrote:
> bt.h also includes bttypes.h so it's easier not to make mistakes

From btTorrent:

    int listenport;             /* is this required??? */

No, listenport is not required, if we're going to do the networking
in libbt I strongly suggest that we only use a single port per
application, and by default one in the dynamic port range as defined
by IANA, 49152 through 65536. Start from the bottom and try
allocating upwards.

The common 6881 port and thereabout are listed as:
--8<-- http://www.iana.org/assignments/port-numbers
#               6851-6887   Unassigned
muse            6888/tcp    MUSE
muse            6888/udp    MUSE
#                           Muse Communications Corporation
#                           <ho...@mu...>
#               6889-6960   Unassigned
-->8--

..and hence not really a good fit. Also, there's no telling how many
bt listeners may be active on a single host, so unless there's that
global system mentioned before bt really shouldn't be allocated just
a single port.

And most importantly a dynamic port fits the application. This is not
a server that peers will try to connect blindly to as a result of a
user command, instead peer lists come from the tracker and so ports
do not matter.

--8<-- docs/protocol.txt, about communication between peers
Next comes the 20 byte sha1 hash of the bencoded form of the 'info'
value from the metainfo file. (This is the same value which is
announced as info_hash to the tracker, only here it's raw instead of
quoted here). If both sides don't send the same value, they sever the
connection. The one possible exception is if a downloader wants to do
multiple downloads over a single port, they may wait for incoming
connections to give a download hash first, and respond with the same
one if it's in their list.
-->8--

This sounds good to me.


//Peter

Re: [Libbt-devel] question about btlist

[Peter S. <stu...@cd...>]

On Thu, Feb 10, 2005 at 02:42:02PM +0100, Alien wrote:
>          0x502ab8(STRING)=(6)'pieces'
>          =>
>             0x502a50(STRING)=(156040)'[EAd???g?????f????????,1U?? ????...'
> 
> 
> this last string is that normal? a huge string? i thought the
> pieces were supposed to be there? which it would have to be a
> number?
> 
> thanks for any info relating to this

Quoting docs/protocol.txt:
--8<--
'pieces' maps to a string whose length is a multiple of 20. It is to
be subdivided into strings of length 20, each of which is the sha1
hash of the piece at the corresponding index.
-->8--

So yes, it's supposed to be long and binary.


//Peter

Re: [Libbt-devel] RFC: lists

[Peter S. <stu...@cd...>]

On Fri, Feb 11, 2005 at 09:30:28AM +0100, Alien wrote:
> dynamic arrays with growing and shrinking. -->
> http://dynarr.sourceforge.net/

Besides being version 0.0.x with comments like "for now" in the
documentation (sorry if I understood that wrong, only skimmed it)
it looks useful, but in our case I think that malloc(), realloc()
and free() of pointers and structs is simple enough not to warrant
the extra dependency, similar to the select()/libevent case..

Don't get me wrong, I don't want to shoot anyone down, but zero
depenencies is really nice. On the other hand, if there's one
dependency, two isn't all that much worse.


//Peter

[Libbt-devel] RFC: lists

[Alien <ali...@us...>]

this brings me to another topic: lots of lists are in this:

a list of torrents, a list of files in a torrent, a list of peers per=20
torrent,...

now i don't want to sound biased, but of course i am; but I have programmed=
 in=20
the not-so-long-ago past a library in C that handles this efficiently.

dynamic arrays with growing and shrinking. --> http://dynarr.sourceforge.ne=
t/

well, since i'm quite biased (and proud of my creation), i cannot decide to=
=20
use this. you may feel that you do not want this kind of efficiency or to=20
have another dependency.

I however feel that this would help our library, and make things easier as=
=20
well.

If you feel against it, i will not use it.
=2D-=20
Alien is my name and head-biting is my game.

Re: [Libbt-devel] added header files that will be installed

[Alien <ali...@us...>]

Op vrijdag 11 februari 2005 09:00, schreef Alien:
> I added 2 header files that'll be installed together with the library.
>
> these are the header files that'll have to be used by the frontend to
> control the library.
>
> ALSO!: these header files MUST be used by the library itself as well, so =
no
> future conflicts arise!


forgot to mention that it's bt.h and bttypes.h and will be installed in=20
${include}/bt/

which means that applications will have:

#include <bt/bt.h>

in their header includes.

bt.h also includes bttypes.h so it's easier not to make mistakes
=2D-=20
Alien is my name and head-biting is my game.

[Libbt-devel] added header files that will be installed

[Alien <ali...@us...>]

I added 2 header files that'll be installed together with the library.

these are the header files that'll have to be used by the frontend to contr=
ol=20
the library.

ALSO!: these header files MUST be used by the library itself as well, so no=
=20
future conflicts arise!
=2D-=20
Alien is my name and head-biting is my game.

Re: [Libbt-devel] role of libevent

[Alien <ali...@us...>]

Op vrijdag 11 februari 2005 08:26, schreef Elliott Mitchell:
> >From: Alien <ali...@us...>
> >
> > Op vrijdag 11 februari 2005 01:34, schreef Elliott Mitchell:
> > > >From: Alien <ali...@us...>
> > > >
> > > > Op donderdag 10 februari 2005 05:38, schreef Elliott Mitchell:
> > > > > Does not include Windows though...
> > > >
> > > > i couldn't care less about that
> > >
> > > Despite the annoyance they do exist. To truely takeover the world it =
is
> > > best not to argue with the end users.
> > >
> > > I'm suggesting glib as it is capable of encapsulating all IO in a
> > > cross-platform manner. Also has a nice object system written in C
> > > already put together.
> >
> > but not as efficient as doing it ourselves in a lowleve manner, which is
> > the subtitle of this project, and will be our ticket into being standard
> > distro material.
>
> Being used by lots of program is what makes you into standard material.
> Do you think KDE and GNOME are shipped because they're efficient?

that's frontend stuff, that's different; by comparison, the linux kernel is=
=20
used because it's efficient

> > > > On Thu, Feb 10, 2005 at 11:03:43AM -0500, Emil Sit wrote:
> > > > > On Thu, 10 February 2005 at 16:45 (+0100), Peter Stuge wrote:
> > > > > > libraries. I can certainly live with select() in libbt as for
> > > > > > performance, I think it will be good enough. But, on the other
> > > > > > hand,
> > > > >
> > > > > Performance is not the issue.  The hard problem is how to isolate
> > > > > event-handling code from the details of the library/application so
> > > > > that you have a clean and maintainable program.  Much of the work
> > > > > that I did was to create clean abstractions.
> > > > >
> > > > > The hard part about making libbt a library is in allowing the I/O
> > > > > handling to integrate with the program that it is being linked
> > > > > into. If your application uses X, how do you integrate X event lo=
op
> > > > > and select/poll/libevent-abstracted I/O handling?
> > > > >
> > > > > giFT seems to have the right idea by isolating the I/O into
> > > > > a separate process and communicating using RPCs with front-end
> > > > > applications.
> > > >
> > > > This brings us to the API/internals topic.. Should libbt work on ea=
ch
> > > > torrent in a separate process, or work on all active torrents in a
> > > > libbt process separate from the application, or should we shoehorn
> > > > all the IO into the main application process?
> > > >
> > > > I think either libbt process or torrent process is fine, with the
> > > > former looking nicer and scaling a lot better.
> > >
> > > I think this is an application's choice, not the library's. As such r=
un
> > > everything in the single process, allow for an app to fork() for each
> > > torrent, but do not require that.
> >
> > a thread/torrent is a bad idea, a torrent server cannot handle the
> > overhead of having 5334 threads...
> >
> > but we should allow for a single torrent as well...
>
> Does this mean it is bad to allow an application to make this decision?
>
> Certainly a case like the above an application may not want to have a
> process/thread per torrent, yet it might be perfectly valid for a small
> client to do so.

if they want to, they can just start multiple instances and fork themselves=
,=20
we do not have to think about such things

> > > > I don't know enough X to answer your question, but say libbt provid=
es
> > > > a fd from pipe() back to the application for notification, would th=
at
> > > > be easy enough to check for? If so, libbt can do async however it
> > > > wants in it's own process and just send status back through the pip=
e.
> > >
> > > For every X-server you're talking to you'll have a file-descriptor th=
at
> > > can be watched for being ready for output or having input ready (I ha=
ve
> > > looked at this). Programs in general may need an arbitrary number of
> > > FDs that need monitoring.
> >
> > X-server are not for us to worry about, frontends handle that, the
> > networking has all the fd's we need/want.
>
> Applications worry about front-ends. But libraries do need to consider
> how they're going to be used. If libbt has the poll()/select() call
> internally, an application needs to be able to add FDs to the sets.

i don't think frontends need the FDs if libbt is handling the select/poll=20
itself, but if the frontend is taking care of select/poll, it needs the FDs=
et=20
of course...

but i think you are mixing things here, i do not think we are talking about=
=20
the same things here, whether or not an application uses X is their problem=
,=20
it's like like we're going to communicate over the X protocol to the=20
frontend.

we are losing the focus here:

the idea of the library is:
=2D-> provide efficient functions so that all kinds of frontend can use us=
=20
easily.

in that view: we need:
 - a single function that downloads a torrent and finishes (with a callback=
=20
for progress)
 - a group of functions making daemonizing easy
 - a set of functions that manages all torrents and peers (add, del, info)
 - a set of functions to set options to torrents, connections or the librar=
y=20
itself
 - a set of functions to handle the library itself(init, free, stuff like=20
that)

in any case, this mess needs to be clean up first, so i'm adding a header f=
ile=20
that will be installed and describes the functions to the frontends.
=2D-=20
Alien is my name and head-biting is my game.

Re: [Libbt-devel] role of libevent

[Elliott M. <eh...@m5...>]

>From: Alien <ali...@us...>
> Op vrijdag 11 februari 2005 01:34, schreef Elliott Mitchell:
> > >From: Alien <ali...@us...>
> > >
> > > Op donderdag 10 februari 2005 05:38, schreef Elliott Mitchell:
> > > > Does not include Windows though...
> > >
> > > i couldn't care less about that
> >
> > Despite the annoyance they do exist. To truely takeover the world it is
> > best not to argue with the end users.
> >
> > I'm suggesting glib as it is capable of encapsulating all IO in a
> > cross-platform manner. Also has a nice object system written in C already
> > put together.
> 
> but not as efficient as doing it ourselves in a lowleve manner, which is the 
> subtitle of this project, and will be our ticket into being standard distro 
> material.

Being used by lots of program is what makes you into standard material.
Do you think KDE and GNOME are shipped because they're efficient?

> > > On Thu, Feb 10, 2005 at 11:03:43AM -0500, Emil Sit wrote:
> > > > On Thu, 10 February 2005 at 16:45 (+0100), Peter Stuge wrote:
> > > > > libraries. I can certainly live with select() in libbt as for
> > > > > performance, I think it will be good enough. But, on the other
> > > > > hand,
> > > >
> > > > Performance is not the issue.  The hard problem is how to isolate
> > > > event-handling code from the details of the library/application so
> > > > that you have a clean and maintainable program.  Much of the work
> > > > that I did was to create clean abstractions.
> > > >
> > > > The hard part about making libbt a library is in allowing the I/O
> > > > handling to integrate with the program that it is being linked into.
> > > > If your application uses X, how do you integrate X event loop
> > > > and select/poll/libevent-abstracted I/O handling?
> > > >
> > > > giFT seems to have the right idea by isolating the I/O into
> > > > a separate process and communicating using RPCs with front-end
> > > > applications.
> > >
> > > This brings us to the API/internals topic.. Should libbt work on each
> > > torrent in a separate process, or work on all active torrents in a
> > > libbt process separate from the application, or should we shoehorn
> > > all the IO into the main application process?
> > >
> > > I think either libbt process or torrent process is fine, with the
> > > former looking nicer and scaling a lot better.
> >
> > I think this is an application's choice, not the library's. As such run
> > everything in the single process, allow for an app to fork() for each
> > torrent, but do not require that.
> 
> a thread/torrent is a bad idea, a torrent server cannot handle the overhead of 
> having 5334 threads...
> 
> but we should allow for a single torrent as well...

Does this mean it is bad to allow an application to make this decision?

Certainly a case like the above an application may not want to have a
process/thread per torrent, yet it might be perfectly valid for a small
client to do so.

> > > I don't know enough X to answer your question, but say libbt provides
> > > a fd from pipe() back to the application for notification, would that
> > > be easy enough to check for? If so, libbt can do async however it
> > > wants in it's own process and just send status back through the pipe.
> >
> > For every X-server you're talking to you'll have a file-descriptor that
> > can be watched for being ready for output or having input ready (I have
> > looked at this). Programs in general may need an arbitrary number of FDs
> > that need monitoring.
> 
> X-server are not for us to worry about, frontends handle that, the networking 
> has all the fd's we need/want.

Applications worry about front-ends. But libraries do need to consider
how they're going to be used. If libbt has the poll()/select() call
internally, an application needs to be able to add FDs to the sets.


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \   (    |         EH...@gr... PGP 8881EF59         |    )   /
  \_  \   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
    \___\_|_/82 04 A1 3C C7 B1 37 2A*E3 6E 84 DA 97 4C 40 E6\_|_/___/

Re: [Libbt-devel] what's the typical use-case? [was: mess]

[Tyler M. <ty...@yi...>]

Alien <ali...@us...> wrote:
> > > - functions for converting peer communication into data structures
> i'd rather not, and instead supply a function that gives a the structure 
> that's already in memory

> > > - functions for converting peer responses from data structures into
> > > buffers * both communication of metadata and actual torrnent data
> i don't think this is necessary

	That's great if libbt is handling the actual connections, but that
might not always be the case.

> > > - functions for initiating connections to specific peers
> I'm not sure about this one

	It's neccessary for coding more intelligent connection handling; eg,
seeds preferring sharing with people with higher upload rates...

> wouldn't it be better if the library handled the torrent details, so that 
> frontends were easier to make?

	Depends on the frontend, I guess. :) The library should definately
provide handlers for the torrent details, but I would like it if the "dirty
work" wasn't *completely* concealed from the API...

	- Tyler

Re: [Libbt-devel] role of libevent

[Alien <ali...@us...>]

Op vrijdag 11 februari 2005 01:34, schreef Elliott Mitchell:
> >From: Alien <ali...@us...>
> >
> > Op donderdag 10 februari 2005 05:38, schreef Elliott Mitchell:
> > > Does not include Windows though...
> >
> > i couldn't care less about that
>
> Despite the annoyance they do exist. To truely takeover the world it is
> best not to argue with the end users.
>
> I'm suggesting glib as it is capable of encapsulating all IO in a
> cross-platform manner. Also has a nice object system written in C already
> put together.

but not as efficient as doing it ourselves in a lowleve manner, which is th=
e=20
subtitle of this project, and will be our ticket into being standard distro=
=20
material.

> >From: Peter Stuge <stu...@cd...>
> >
> > On Thu, Feb 10, 2005 at 05:02:09PM +0100, Alien wrote:
> > > > Only reason to use it is the increased performance.
> > >
> > > i sincerely doubt that event would be more performant than using
> > > select directly, really, in fact with using select directly, you
> > > should see a slight performance increase.
> >
> > Well, on reading the libevent page and a few other pages linked from
> > it, I learned that there are other interfaces available that offer
> > even better performance than select(). libevent uses them all. Check
> > out the C10K page for more information. It's about handling thousands
> > of connections on a server, and while that probably wont be happening
> > in Torrent clients anytime soon, spending unneccessary time in the
> > kernel is never useful. :)
>
> glib looks poll() oriented, libcurl is definitely select() oriented
> (wonder how they claim to work on EvilOS?). It appears that select() is
> discouraged in modern programs, with functions like poll() being
> recommended.

i'm not so sure, but i believe it's in our best interest to have this=20
two-ways: one: we should provide this system ourselves (and since the numbe=
r=20
of fd's can be big, i'd suggest select); two: we should also be prepared to=
=20
let the frontend handle that, and give a callback to it instead.

> > On Thu, Feb 10, 2005 at 11:03:43AM -0500, Emil Sit wrote:
> > > On Thu, 10 February 2005 at 16:45 (+0100), Peter Stuge wrote:
> > > > libraries. I can certainly live with select() in libbt as for
> > > > performance, I think it will be good enough. But, on the other
> > > > hand,
> > >
> > > Performance is not the issue.  The hard problem is how to isolate
> > > event-handling code from the details of the library/application so
> > > that you have a clean and maintainable program.  Much of the work
> > > that I did was to create clean abstractions.
> > >
> > > The hard part about making libbt a library is in allowing the I/O
> > > handling to integrate with the program that it is being linked into.
> > > If your application uses X, how do you integrate X event loop
> > > and select/poll/libevent-abstracted I/O handling?
> > >
> > > giFT seems to have the right idea by isolating the I/O into
> > > a separate process and communicating using RPCs with front-end
> > > applications.
> >
> > This brings us to the API/internals topic.. Should libbt work on each
> > torrent in a separate process, or work on all active torrents in a
> > libbt process separate from the application, or should we shoehorn
> > all the IO into the main application process?
> >
> > I think either libbt process or torrent process is fine, with the
> > former looking nicer and scaling a lot better.
>
> I think this is an application's choice, not the library's. As such run
> everything in the single process, allow for an app to fork() for each
> torrent, but do not require that.

a thread/torrent is a bad idea, a torrent server cannot handle the overhead=
 of=20
having 5334 threads...

but we should allow for a single torrent as well...

> Another thing to imagine is that a system might have a global torrent
> handling process, having a unified cache and doing whole bandwidth
> regulation. It would be in the domain of a library to be prepared to
> talk to such a process.

that's unheard of; leave that be, if there is a big system having things li=
ke=20
that, let it be based upon libbt or be silent!

> > I don't know enough X to answer your question, but say libbt provides
> > a fd from pipe() back to the application for notification, would that
> > be easy enough to check for? If so, libbt can do async however it
> > wants in it's own process and just send status back through the pipe.
>
> For every X-server you're talking to you'll have a file-descriptor that
> can be watched for being ready for output or having input ready (I have
> looked at this). Programs in general may need an arbitrary number of FDs
> that need monitoring.

X-server are not for us to worry about, frontends handle that, the networki=
ng=20
has all the fd's we need/want.
=2D-=20
Alien is my name and head-biting is my game.