libbt-devel

[Libbt-devel] News on the Mac front - LibBT 1.03

[Dakidd <da...@so...>]

As of about an hour ago ago, I've got an error and warning free compile,
and functionality of both btcheck.c and btlist.c on a PowerMac 7500 hopped
up to a G4@350 running MacOS 9.1. Testing has been done using
locally-stored .torrent files that I've actually used for live transfers
using the "official" Python implementation of BitTorrent (note 1) in sizes
ranging from a couple hundred K to a couple-few-hundred megs, both
single-file and "directory" modes.

I'm pleased to report that, with the exception of a cosmetic glitch (Looks
like an "LL" suffix is being used, rather than an "L") in a printf format
string (From the debugger, I can see that the correct data is in the
variable, it's just getting printed wonky), execution has been flawless.
And speedy! As an example, hashing torrent pieces - I have a fileset
(.torrent and target file - Target file size just a shade over 175 megs)
that takes well over an hour and a half to hash and decide whether it is or
isn't complete when using the "Official" Python-based client. The same file
going through btCheck (which I'm building to use a slightly adapted public
domain SHA1 implementation called "ReidSHA", since OpenSSL hasn't been
ported to Classic MacOS (note 2) that I know of) is hashed and verified as
complete or not after only 51 seconds! That's with *ZERO* optimizations
being applied - No "hand-crafting ML byte-strings", no compiler
optimizations or use of special processor features, just a straight-up "as
it is written, so shall it be compiled". Once I decide to crank up the
compiler's brainpower by turning on some of the compile optimizations, I
expect the speed will improve even more as it starts sequencing and
pipelining the generated code and making use of currently unused processor
features like predictive branching or the G4's "Velocity Engine"... But I
digress...

The LibBT 1.03 tarball was my base code, and despite my bitching about
Kevin's coding *STYLE*, I have to be fair and say that I can find nothing
to bitch about in his *CODE* once it's been hammered into readability. A
couple of gaffes here and there, which is only to be expected in a project
like this, where the spec is a moving target, and the codebase is evolving
to hit it, but overall, nice and tight. It's just that the voyage to
readability can be frustrating! But, once it gets there, it's a sweet
codebase, with a lot of headroom, and pretty much ALL the toys that should
be needed to follow the protocol wherever it goes. And if it comes up
short? Well, that's the beauty of open source... SOMEBODY will get miffed
at the lack, and fix it! A million monkeys, with a million keyboards,
typing for a million years, just might up and (re)write Linux, too! Or
worse... Windows!!! AUGH!!!! Start shooting monkeys - QUICK! :)


Next step:
Hacking the bejeezus out of the stuff that tries to talk to the 'net. CURL
- FLUSH! Gonna use Apple URLAccess instead. That'll get me talking to the
tracker. Then it's on to try to change from *nix/Windows style networking
using the nice, simple, freindly, reliable, Sockets TCP/IP API, to Mac's
god-awful train-wreck of a TCP/IP implementation called "OpenTransport" so
that my servent can talk to others.

If I'm not back in 6 months, call the mounties and tell 'em I went
thattaway! And tell 'em to bring extra rope - I'll probably be be found
clinging to a knot tied at the end of mine! :)

BTW: Update, since I see I bailed out of Eudora before it could actually
post this message -

It's now several hours later, and the glitch in the printf (turned out to
be exactly what I thought it was: LL in a format string instead of L) has
been fixed. Operation and functionality of both btCheck.c and btList.c are
now 100% with all torrent files I've thrown at them (about 40, all of them
previously used to control actual transfers) so far. Making copies and
moving them around, then mangling the target data of the torrent in some
way to simulate damaged/missing pieces has been detected and reported as
present down to the single changed bit level, so the "Something's wrong
with the file" mode is working just as well as the "Yep, the file's all
there" functionality. I'm officially declaring this portion of the port to
Classic MacOS a success!

Now to get the networking to happen so I can build a client using a REAL
language instead of being stuck with that godforsaken Python interpreter.

Interpreter... <snork> I'd rather compile it! :)

<Ponders Applesoft (Apple IIe BASIC) or Apple Integer BASIC on a Mac>

<flees, screaming for tranquilizers, at the thought>

Anyway...

I *THINK* I've managed to conditionalize my changes in such a way that
Windows/*nix folks can compile my changed code by simply substituting their
own config.h file with the last line before the final #endif reading
"#define TARGET_MACOS 0" or "#undef TARGET_MACOS". My intent was to be sure
that my changes are completely transparent, but I cannot claim with
certainty that I've managed it.

(note 1) Painfully slow, written in an interpreted language (Python) whose
only available (to my knowledge) Mac-hosted interpreter keeps crashing on
me. Python is also a language which bears the second curse of being
"foreign" to me - I don't grok Python (and to be blunt, I don't WANT to
grok Python - it makes my brain itch) so I can't fiddle with it to try to
make it faster, despite having the source code staring me in the face.
Hence, my motivation to get a C-based BitTorrent "toolkit" operational on
the Mac. To make things worse, the interpreter has its own bugs and
glitches, so that about half the time, I wonder whether the most recent
problem I've encountered has been with BitTorrent or a problem with Python.

(note 2) For you non-Mac folks, "Classic" distinguishes the older MacOS
(9.2.x and below) from "The cats" - MacOS X - AKA 10.x.x AKA
Jaguar/Panther/Etc. (I think we're officially up to Tiger now - Or does
that release next week? It's either just released, or releasing very soon)
MacOS X is based on *nix - a sub-flavor of NetBSD, if I recall rightly -
while MacOS 9.x and earlier are now grouped as "Classic", and despite
looking pretty similar on the screen, are pretty significantly different
than X when you crawl under the hood and start banging the bits directly
against the silicon.

Don Bruder -  da...@so...      <--- Preferred Email - unmunged
I will choose a path that's clear: I will choose Free Will! - N. Peart

Re: [Libbt-devel] News on the Mac front - LibBT 1.03

[Peter S. <stu...@cd...>]

Wow, long mail. :)

On Wed, May 11, 2005 at 06:50:56AM -0700, Dakidd wrote:
> As of about an hour ago ago, I've got an error and warning free
> compile, and functionality of both btcheck.c and btlist.c on a
> PowerMac 7500 hopped up to a G4@350 running MacOS 9.1.

Congratualtions.


> Next step:
> Hacking the bejeezus out of the stuff that tries to talk to the
> 'net. CURL - FLUSH! Gonna use Apple URLAccess instead.

Why? Doesn't curl do OpenTransport? If not, are they interested in
having support for it? That's where it belongs anyway.


> I *THINK* I've managed to conditionalize my changes in such a way
> that Windows/*nix folks can compile my changed code by simply




> I don't grok Python (and to be blunt, I don't WANT to grok Python -
> it makes my brain itch)

I think Python is a really nice language and if you ever feel like
learning a new scripting language I recommend it wholeheartedly.

( My very first Python program was something similar to a monochrome
icon editor and it took me 20 minutes to make from start to finish,
GUI and all. :) I also taught Python in a high school programming
class last year and used some material from the OBP:
http://ibiblio.org/obp/thinkCS/python.php )

That said, I agree with you that Python isn't the best language to
use for IO- and CPU-intensive software. I'm on this list for that
particular reason.


> To make things worse, the interpreter has its own bugs and
> glitches, so that about half the time, I wonder whether the most
> recent problem I've encountered has been with BitTorrent or a
> problem with Python.

Is that MacPython-OS9?
http://homepages.cwi.nl/~jack/macpython/macpython-older.html

I'm sure the maintainer would love to hear about any problems you
encounter, especially since most Mac users probably switched over to
OS X and of the few that are still at OS 9, not very many are able or
willing to help fix open source software.


> (I think we're officially up to Tiger now - Or does that release
> next week? It's either just released, or releasing very soon)

Released about two weeks ago.


> MacOS X is based on *nix - a sub-flavor of NetBSD, if I recall
> rightly -

The Unix behind the Finder is called Darwin and can be downloaded
from Apple after registering online, but I don't recall which BSD it
is derived from.


//Peter

Re: [Libbt-devel] News on the Mac front - LibBT 1.03

[Dakidd <da...@so...>]

>Wow, long mail. :)

Heck, for me, that's just barely a baby email :)

>
>On Wed, May 11, 2005 at 06:50:56AM -0700, Dakidd wrote:
>> As of about an hour ago ago, I've got an error and warning free
>> compile, and functionality of both btcheck.c and btlist.c on a
>> PowerMac 7500 hopped up to a G4@350 running MacOS 9.1.
>
>Congratualtions.

I'm lovin' every minute of it :)

It wasn't all that long ago I was getting ready to throw up my hands in
despair at ever getting it running, but after a couple "scrap practically
everything and start over" operations, I've finally made useful progress.
Hanging back there with 0.03 for a long as I did was a good thing - It got
me familiar with the "guts" so that I was comfortable with the idea of
wholesale slash-and-burn alteration where needed, but still able to
preserve (I think) the Windows/Unix compilability/functionality. (I'm
making *NO* claims for any new Windows/Unix code - I'm a Mac geek, so it's
entirely possible I inadvertently screwed something up "on the other side
of the fence" with my Mac-centric hacking.)
>
>
>> Next step:
>> Hacking the bejeezus out of the stuff that tries to talk to the
>> 'net. CURL - FLUSH! Gonna use Apple URLAccess instead.
>
>Why? Doesn't curl do OpenTransport? If not, are they interested in
>having support for it? That's where it belongs anyway.

If it DOES do OT, I haven't heard about it. But to be honest, I haven't
been listening for such news.

In an earlier iteration of "Figure out what works and what needs to be
built as a substitute for Macs", I had operational UrlAccess code for
talking to the tracker. I'll probably dig that out of the code compost
heap, brush the coffee grounds and banana peels off it, and put it back to
work now that I'm playing in the 1.03 codebase. Operation was great, but I
broke something else, and ended up abandoning that branch of the code.

>
>
>> I *THINK* I've managed to conditionalize my changes in such a way
>> that Windows/*nix folks can compile my changed code by simply
>
>
>
>
>> I don't grok Python (and to be blunt, I don't WANT to grok Python -
>> it makes my brain itch)
>
>I think Python is a really nice language and if you ever feel like
>learning a new scripting language I recommend it wholeheartedly.

I've always been uncomfortable - "lost", might be the best way to put it -
in object-oriented languages. For me, a purely procedural language like
plain old C is a better fit for the way my mind works. Sometimes, certain
OOP concepts come through, and make good sense, so they get used (Like the
way that Kevin deals with the "bts" data type, r the various "Python" types
- It only makes sense to work them that way)  Get to going too
object-oriented on me, though, and I start getting confused, and it's not
too long before I throw up my hands in defeat and say "You're the expert,
you do it".

>
>( My very first Python program was something similar to a monochrome
>icon editor and it took me 20 minutes to make from start to finish,
>GUI and all. :) I also taught Python in a high school programming
>class last year and used some material from the OBP:
>http://ibiblio.org/obp/thinkCS/python.php )
>
>That said, I agree with you that Python isn't the best language to
>use for IO- and CPU-intensive software. I'm on this list for that
>particular reason.

I've gotta agree completely with that... Python may be godlike in certain
ways, but for compute and IO intensive software like what we've got with
BT, it's way in the dust behind something like C or even pascal.

>
>
>> To make things worse, the interpreter has its own bugs and
>> glitches, so that about half the time, I wonder whether the most
>> recent problem I've encountered has been with BitTorrent or a
>> problem with Python.
>
>Is that MacPython-OS9?

Yeah...

>http://homepages.cwi.nl/~jack/macpython/macpython-older.html
>
>I'm sure the maintainer would love to hear about any problems you
>encounter, especially since most Mac users probably switched over to
>OS X and of the few that are still at OS 9, not very many are able or
>willing to help fix open source software.

My understanding is that work on the 9 version is officially dead, so I
haven't bothered. I have it installed for *EXACTLY* one purpose: Running
the BitTorrent "Official" code. Sentiment amongst the Python folk I've
talked to is basically "You should be running MacOS X - That's where all
the Python support work is nowdays. Get with the times." To which my reply
is a standard "If you're buying me the hardware to run X on, I'll start
running X. Until then, shuddup yer face about getting with the times."

If it weren't for wanting to run BitTorrent, I wouldn't have Python on my
system at all. (And if/when I get a "real" client coded, believe that I'll
reclaim the disk space wasted on Python and the official BT implementation
quicker than you can say "Are you sure you want to empty the trash?")
>
>
>> (I think we're officially up to Tiger now - Or does that release
>> next week? It's either just released, or releasing very soon)
>
>Released about two weeks ago.

See how enthused about X I am? :)

I've got a full retail install set of 10.3 (not sure which exact version,
but it's labeled as being version 10.3, which makes it "Panther.something")
sitting here on the desk, and the downloaded .dmg file that contains the
"10.3 combination updater". Trouble is, my 7500 won't install it (even with
the help of X-Post Facto - CD-ROM drive hiccups - Sometimes, it isn't a
bootable device, and the install is hosed before I even get a chance to
tell it to happen, other times, it's a bootable device, but then it hangs
early in the install with a repeating "waiting for device" error that's
been diagnosed as meaning "Your CD-ROM drive is too old for Panther's
liking."

>> MacOS X is based on *nix - a sub-flavor of NetBSD, if I recall
>> rightly -
>
>The Unix behind the Finder is called Darwin and can be downloaded
>from Apple after registering online, but I don't recall which BSD it
>is derived from.

I keep getting the impression that Darwin is a NetBSD derivative.

Don Bruder -  da...@so...      <--- Preferred Email - unmunged
I will choose a path that's clear: I will choose Free Will! - N. Peart

Re: [Libbt-devel] News on the Mac front - LibBT 1.03

[Peter S. <stu...@cd...>]

On Wed, May 11, 2005 at 03:27:29PM -0700, Dakidd wrote:
> >Wow, long mail. :)
> 
> Heck, for me, that's just barely a baby email :)

Wordy messages can be nice, but take so long to type..


> >> Next step:
> >> Hacking the bejeezus out of the stuff that tries to talk to the
> >> 'net. CURL - FLUSH! Gonna use Apple URLAccess instead.
> >
> >Why? Doesn't curl do OpenTransport? If not, are they interested in
> >having support for it? That's where it belongs anyway.
> 
> If it DOES do OT, I haven't heard about it. But to be honest, I
> haven't been listening for such news.

Doesn't look like it. Try asking around the curl community to see if
there's interest for supporting OT. If not a neat trick would be a .c
file that uses URLAccess and provides curl-compatible functions that
libbt can use, and only compile and link with it on OS 9.


> In an earlier iteration of "Figure out what works and what needs to be
> built as a substitute for Macs", I had operational UrlAccess code for
> talking to the tracker.

Ah. How does the interface compare to curl or even sockets?


> >I think Python is a really nice language and if you ever feel like
> >learning a new scripting language I recommend it wholeheartedly.
> 
> I've always been uncomfortable - "lost", might be the best way to
> put it - in object-oriented languages. For me, a purely procedural
> language like plain old C is a better fit for the way my mind
> works.

I'm not much of an OO fan either, but I think Python does OO very
well in that it isn't actually required, you can have a completely
flat namespace. I have yet to make my first class and object in
Python.

If you feel like it, take a quick look at that book URL I mentioned.
:)


> I've gotta agree completely with that... Python may be godlike in
> certain ways,

It's incredibly readable, it has a very good set of extensions and
many connections to useful libraries, it has pretty smart handling of
data - many nice tricks you can do that you wish you came up with
yourself, and much more. :) It's fantastic for rapid prototyping.
It's great for making portable GUIs quickly. You can even compile to
standalone binaries on some platforms. (Bytecode+interpreter DLL..)


> but for compute and IO intensive software like what we've got with
> BT, it's way in the dust behind something like C or even pascal.

Anything compiled to machine code is good.


> >> the interpreter has its own bugs and glitches
> >
> >Is that MacPython-OS9?
> 
> Yeah...

Ouch. :(


> Sentiment amongst the Python folk I've talked to is basically "You
> should be running MacOS X - That's where all the Python support
> work is nowdays. Get with the times." To which my reply is a
> standard "If you're buying me the hardware to run X on, I'll start
> running X. Until then, shuddup yer face about getting with the
> times."

Save up and get a Mac Mini. It looks like a nice little box. :)


> If it weren't for wanting to run BitTorrent, I wouldn't have Python
> on my system at all. (And if/when I get a "real" client coded,
> believe that I'll reclaim the disk space wasted on Python and the
> official BT implementation quicker than you can say "Are you sure
> you want to empty the trash?")

Hehe. You can never have too much free disk space. :)


> >Released about two weeks ago.
> 
> See how enthused about X I am? :)

Aye. Hope you have the opportunity to switch over sometime soon, I
think OS X is by far the best modern desktop OS. (And I'm a Linux
advocate.)



> other times, it's a bootable device, but then it hangs early in the
> install with a repeating "waiting for device" error that's been
> diagnosed as meaning "Your CD-ROM drive is too old for Panther's
> liking."

Ouch. No, better not try to shoehorn the wildcats into that system
then.


> I keep getting the impression that Darwin is a NetBSD derivative.

Quite possible. I don't know. :\


//Peter

Re: [Libbt-devel] News on the Mac front - LibBT 1.03

[Alien <ali...@us...>]

Op woensdag 11 mei 2005 15:50, schreef Dakidd:
> lotsa stuff...

I'm working on the CVS HEAD (not stable branch)

would it be possible to send your patch to CVS stable branch, to me? I'd be=
=20
interested in the cURL replacement you have there, since i'm at the moment=
=20
restructuring the cURL stuff and i'm adding local relative files anyway at=
=20
this moment.

I can say that i'm almost back to workable stuff, and i've restructured and=
=20
hopefully added more readability.

when i'm more or less finished with this (when after this i'll be thinking =
of=20
more optimization, and error checking), could you do the same with the=20
regular branch? I would be helping out.

AL13N