Re: [Apt-proxy-users] Performance problems on Debian/mips

[Chris H. <chr...@ni...>]

Hi Manuel, sorry I took a while to get back to you.

On Sat, Mar 02, 2002 at 12:58:09PM +0100, Manuel Estrada Sainz wrote:
> On Thu, Feb 28, 2002 at 09:14:32PM +0100, Chris Halls wrote:
>=20
>  How about 'word by word' rewrite of apt-proxy in perl so we don't have
>  to worry about logic errors and once done we will have plenty of space
>  for improvements and performance enhancements.
>=20
>  By 'word by word' rewrite I mean keeping all funcion names and as much
>  of the structure as posible.

I don't know how much would be left after doing that :-)

There are quite a few of the functions that are very rsync specific and
could do with cleaning up, although I guess it would still at least provide
a framework.  If doing the rewrite myself, I guess I would consider starting
it like that.

> > At the moment, I think aptcached looks the most promising.  The code
> > looks nice and clean but I haven't actually installed it to give it a
> > test run yet.
>=20
>  Well, having apt-proxy around makes me think that aptcached looks a bit
>  'short minded'. And anyway, if we do the rewrite proposed above we
>  could take the nice parts of aptcached and use the in apt-proxy.
>=20
>  Reading the goal's of aptcached I think that we could convince Jason
>  with a 'perl improved'/'http capable' version of apt-proxy. Specially
>  if there is room for some of his code I guess :)
>=20
>  If I do the rewrite and it works, would you adopt it as upstream?

I got hold of Jason today on IRC and we discussed the whole thing.  He's
joined this list so we can discuss here.

It turns out that there are some issues with aptcached that could cause
problems:

 - aptcached depends on POE.  It was so Jason could avoid having to do
   events loops, but POE is still oficially alpha, is not in Potato and has
   a large runtime overhead,  Jason reckoned around 2MB.  I'm not against
   using libraries to help simplify coding and make it more maintainable,
   but I think that POE is too big a price to pay.  Jason understands that
   and is open to dropping POE, as long as someone else does the rewrite :-)

 - It doesn't mirror the directory structure.  That certainly simplifies the
   code but has scalability problems and you don't end up with a partial
   mirror.

 - HTTP 1.1 keep-alives aren't working (yet).

So it looks like a fair amount must be rewritten anyway.  Jason said he's
open to help with whatever we decide to do, which I'm very grateful for.

If you have the time to work on a rewrite that's great by me - I'd rather
have several people involved than just me.  It helps keep bugs shallow and
for the program to grow faster.  The only thing I would ask for is to make
sure others can work with the code too.  In particular, documenting what
code is supposed to do and helping to make sure it is production-ready.

>  And to start contributing on this, attached goes some scripts I wrote
>  to import .deb into a pool structure without Package lists.

Thanks for the scripts.  It's a nice idea, to build the path using dpkg
--info and one I had not thought of.  It only works for mirrors that use the
current Debian mirror structure, and would have to be changed if the mirror
structure were to be changed by the FTP maintainers in the future.

The time taken to write e.g. the apt-proxy-import script was probably less
than half the total time it took to get it into the package, because I had
to write documentation and do stuff like handling typical errors and provide
--help text. =20

It would still be a fair amount of work to get your scripts finished.  I'm
not sure whether you are providing them for comment or as a contribution to
the package.  If you want to help out, I would need you to help get things
nearer to a completed state.  Otherwise, I would be comitting myself to a
lot of extra work which I probably wouldn't have the time to do.

Does that sound OK?  I've set up an #apt-proxy channel on openprojects IRC
so maybe we can all talk there sometime.

Thanks,
Chris (who's glad things are picking up)
--=20
Chris Halls | Frankfurt, Germany
Yahoo:hagga12000 ICQ:36940860 MSN:chr...@ni...