install-log-devel — install-log developers' list

Re: [ID] PackageInfo

[James R. <jr...@fu...>]

On Fri, Aug 24, 2001 at 02:43:57PM -0500, Andy Goth wrote:
> > > I'm glad the problem is on my end, because I can
> > > fix that.  Just tinker with the Makefile, edit config.h, and recompile
> > > myself... uh oh, I forgot, I don't install-log-2 isn't ready yet.  ACK! 
> > > Now I can't go back.
> >
> > Pardon? The latest CVS code seems to compile alright.
> 
> Stupid joke.

oh, ok then :)
 
> > As for the main packageinfo function, perhaps it should really be a parser
> > and the actual types of file names be put in the config file. For example,
> > the default config file could include the patterns (let's ignore patches
> > for the minute):
> >
> > %n%v%m.%e              # e.g. unzip542.tar.gz
> 
> How would it tell where %n stops and %v starts?  (Whoa, this particular 
> package name--unzip542--looks particularly bothersome, especially since there 
> are more like it.)

Assume the %v is a number.
 
> > %n-%v%m.%e             # e.g. sox-12.17.1.tar.gz
> 
> Here we have the - delimiter.
> 
> > where:
> > %n = name
> > %v = version
> > %m = misc (like the h1 in man-1.5h1.tar.gz)
> > %e = extensions
> >
> > And define the valid extensions:
> >
> > EXTENSIONS = tar.gz : tar.bz2 : tgz : bz2 : gz
> 
> That idea is really cool.  I'll just have to steal it. <g>

What else is the GPL for (except call it "borrowing")? :)

Oh, and the extensions probably should have the relevant programs by them, so
that if there's a compression/archiving format that we forget, it can be easily
added in. I can't think of a good format to put this in the config file though.
We would have to put the extension, followed by the unpack command. Something
like

EXTENSIONS = tar.gz, "tar -xzf %p" : tar.bz2, "tar -xyf %p"

> > The %n name field will be tricky though, as a name might include a '-'
> > anyway, like install-log for instance :). I think it would have to be
> > matched last, with an "anything that's left policy":
> 
> Parse from the right.

Yeah, I suppose so...

> > install-log-0.8.tar.gz - %n-%v.%e
> >
> > %e = gz
> > %v = tar  -- ! nope, try again (version must be number, preferably float)
> > %v - 0.8  -- yep!
> > so %e = tar.gz
> > we have: xxxxxxx-%v.%e
> > therefore: %n = install-log
> >
> > Hmmm, still tricky to program, although perhaps we could have a custom
> > config for install-log, like "install-log-%v.%e". Any ideas on how to match
> > '-'s in names?
> 
> We could make %v match only numbers (and %V for stringy versions).

Sounds good.
 
> > Tell me what you think of this parsing idea then.
> 
> It'll be durned hard, but if we parse from the right, then the "install-log" 
> program is solved.  I don't believe I've ever seen the package name anywhere 
> other than the beginning.
> 
> Of course there are packages whose names can't quite be derived from the 
> filename (without wetware).  Think about all those djgpp zip files that cram 
> the GNU development environment into msdog's 8.3 handicap.  Or what about 
> ftp://ftp.uu.net/graphics/jpeg/jpegsrc.v6b.tar.gz?

What about having an optional default name option in the config. So if the
package doesn't include the name (like the Linux kernel patches I seem to
remember), it'll default to the one in the config. This means that a "local"
package config for jpeg6b could look like:

NAME=libjpeg
PATTERN=*.v%V.%e

Everything up to the first '.' is ignored (a * is better than a %i, I think),
then 'v' is matched, after that is the version (up to the next '.') and then
the extension. The name isn't included so the default name is put in. Btw,
another thought: package specific configs should override global configs,
although that's pretty obvious :)

> Okay, we need a plan for the parser, version 0.  And since I'm not in the 
> mood for coding install-log at the moment (too much fun with my 
> lighting/visibility demo, hehehe), either we wait or you do it.  Don't worry, 
> it's easy.

Ok, ok, no need to patronise me :)

> One, support only tarballs.  This means grab the text from the end up to the 
> "t".  It seems easy...
> 
> Two, handle the version as a string.  This means grab text from just to the 
> left of the ".tar." to the first "-".  Of course all this is with reference 
> to the END of the string.
> 
> Three, everything before the "-" is the name.
> 
> Got it?  It'll work for install-log... that's good enough for now.

Ok, I'm doing it now. Unfortunately I'm about to start work on some design work
for a company, and Uni beckons in October, so there'll probably be some reading
materal I have to get through in September. In short, I'll have less time for
install-log, at least in the immediate future.

-- 
James Reeves (Shade)
http://www.arevol.co.uk/

[ID] Linked lists

[Andy G. <unu...@op...>]

Are we happy with our linked lists?  My lighting project requires more 
advanced lists, and maybe I could bring the code into install-log.

lighting's lists are declared thusly:

/* Linked list that tries to cut down on mallocs and frees */
typedef struct List List;
struct List {
	Node* head;		/* Beginning of list */
	Node* tail;		/* End of list */
	int count;		/* Length of list */

	Node** heap;		/* List of pointers to nodes */
	int capacity;		/* Length of node list */
	int step;		/* Number of nodes to allocate when growing */
};

/* Linked list node */
typedef struct Node Node;
struct Node {
	void* data;
	Node* next;
};

The biggest feature here is that a list can be reused without freeing and 
re-malloc'ing every node.  The nodes remain allocated and are written over 
when needed.

A side effect I just now noticed is that "heap" acts like an indexed array 
into the linked list, but it only works right until nodes are shuffled, 
sorted, removed, and so on.

Most likely we don't need this, but keep it in mind.

...

Oh, heh, I just realized I don't need this much functionality myself!  Since 
the list doesn't need to be sorted at all, heap, count, capacity, and step 
are all I need, and heap can just be a void*, pointing to various pieces of 
data.

/me goes back to vim to fix this...
-- 
Andy Goth  |  unu...@op...  |  http://sevatech.com/~andy/

Re: [ID] PackageInfo

[James R. <jr...@fu...>]

On Thu, Aug 23, 2001 at 01:03:50PM -0500, Andy Goth wrote:
> Oops, forgot to respond!!  And to think, I was about to flame you for not 
> exhibiting more interest...

Tsk tsk - I was wondering where you had got to :)

> I'm glad the problem is on my end, because I can 
> fix that.  Just tinker with the Makefile, edit config.h, and recompile 
> myself... uh oh, I forgot, I don't install-log-2 isn't ready yet.  ACK!  Now 
> I can't go back.

Pardon? The latest CVS code seems to compile alright.

> > > let's see how to make things better.  For instance, maybe major and minor
> > > need to be strings rather than integers.
> >
> > Hmmm. Good point - quite a few alpha versions have an "a" after the number,
> > although perhaps this could be better represented with an ALPHA switch. Or
> > a enum for alpha, beta etc.
> 
> But I've seen the letters go quite far.  Take a look at man.  My version is 
> man-1.5h1.tar.bz2.  That means the first release of letter h, version 1.5.  I 
> think there are several newer versions out, though...

Good point again.
 
> > I can't see the part field doing any harm, and it's handy for packages like
> > Vim which don't fit the traditional mould (like Ogg Vorbis, I seem to
> > remember).
> 
> Hey, interesting idea.  We can come up with a filename-->PackageInfo filter 
> that works in almost every case but specially handles the few known weirdo 
> packages.
<snip> 
> I have a feeling that this filename-->PackageInfo be really, really hard, 
> unless we figure out how to take it one piece at a time and know when to (and 
> when not to) special-case a specific package.  This is where user feedback 
> (and building LFS!) will come in handy.

I've been thinking about this (I haven't been completely idle these past
weeks). I think we should first make a simple, temporary package info function
in the form: %n-%v.%e (where %n=name, %v=version, %e=extension), just so that
we can work on other parts of the program as well. The packageinfo function
seems to be proving a bottleneck (can't do anything else without information of
the packages).

As for the main packageinfo function, perhaps it should really be a parser
and the actual types of file names be put in the config file. For example, the
default config file could include the patterns (let's ignore patches for the
minute):

%n%v%m.%e              # e.g. unzip542.tar.gz
%n-%v%m.%e             # e.g. sox-12.17.1.tar.gz

where:
%n = name
%v = version
%m = misc (like the h1 in man-1.5h1.tar.gz)
%e = extensions

And define the valid extensions:

EXTENSIONS = tar.gz : tar.bz2 : tgz : bz2 : gz

In each of the package directories (i.e. /var/install-log/<package>) you could
also have a config file that details patterns for unusual packages, like, er...
cdparanoia-III-alpha9.8.src.tgz:

(In cdparanoia/config)
%n-%m-%m%v.%i.%e

(where %i is a field to be ignored completely)


This way the user could quite easily include usual package names, and the extra
processing time of parsing the expressions should affect the program much, as
they're only done once at the start, and should be fairly simple patterns
anyway.

The %n name field will be tricky though, as a name might include a '-' anyway,
like install-log for instance :). I think it would have to be matched last,
with an "anything that's left policy":

install-log-0.8.tar.gz - %n-%v.%e

%e = gz
%v = tar  -- ! nope, try again (version must be number, preferably float)
%v - 0.8  -- yep!
so %e = tar.gz
we have: xxxxxxx-%v.%e
therefore: %n = install-log

Hmmm, still tricky to program, although perhaps we could have a custom config
for install-log, like "install-log-%v.%e". Any ideas on how to match '-'s in
names?

Tell me what you think of this parsing idea then.

-- 
James Reeves (Shade)
http://www.arevol.co.uk/

Re: [ID] PackageInfo

[Andy G. <unu...@op...>]

Oops, forgot to respond!!  And to think, I was about to flame you for not 
exhibiting more interest...  I'm glad the problem is on my end, because I can 
fix that.  Just tinker with the Makefile, edit config.h, and recompile 
myself... uh oh, I forgot, I don't install-log-2 isn't ready yet.  ACK!  Now 
I can't go back.

Excuse me while I dump core.

On Sunday, August 19, 2001 4:47, James Reeves wrote:
> On Sat, Aug 18, 2001 at 08:13:19PM -0500, Andy Goth wrote:
> > Look this over and comment.  Take a look at a variety of source packages
> > you already have and see how this system fits them.  If it doesn't fit,
> > then
>
> I've been installing quite a lot of new stuff anyway, so I'm sorta doing
> that already. I'll make a note to take a closer look at the package names
> though. Btw, you'll probably be pleased to know that v1.9 is working quite
> well, and hasn't crashed yet in the few weeks I've been using it instead of
> the old v1.5.

Yahoo.  That's really, really good news.

> > let's see how to make things better.  For instance, maybe major and minor
> > need to be strings rather than integers.
>
> Hmmm. Good point - quite a few alpha versions have an "a" after the number,
> although perhaps this could be better represented with an ALPHA switch. Or
> a enum for alpha, beta etc.

But I've seen the letters go quite far.  Take a look at man.  My version is 
man-1.5h1.tar.bz2.  That means the first release of letter h, version 1.5.  I 
think there are several newer versions out, though...

>
> > Or maybe the part field is just gay
> > and needs to be removed.
>
> I can't see the part field doing any harm, and it's handy for packages like
> Vim which don't fit the traditional mould (like Ogg Vorbis, I seem to
> remember).

Hey, interesting idea.  We can come up with a filename-->PackageInfo filter 
that works in almost every case but specially handles the few known weirdo 
packages.

Here are a couple URLs associated with Vorbis:

http://www.xiph.org/ogg/vorbis/download/vorbis-tools-1.0beta4.tar.gz
http://www.xiph.org/ogg/vorbis/download/xmms-vorbis.i386.gz

So, I guess that means we'll need some way to handle prebuilt binary 
packages, too. ;^)

I have a feeling that this filename-->PackageInfo be really, really hard, 
unless we figure out how to take it one piece at a time and know when to (and 
when not to) special-case a specific package.  This is where user feedback 
(and building LFS!) will come in handy.

-- 
Andy Goth  |  unu...@op...  |  http://sevatech.com/~andy/

Re: [ID] PackageInfo

[James R. <jr...@fu...>]

On Sat, Aug 18, 2001 at 08:13:19PM -0500, Andy Goth wrote:
> Look this over and comment.  Take a look at a variety of source packages you
> already have and see how this system fits them.  If it doesn't fit, then

I've been installing quite a lot of new stuff anyway, so I'm sorta doing that
already. I'll make a note to take a closer look at the package names though.
Btw, you'll probably be pleased to know that v1.9 is working quite well, and
hasn't crashed yet in the few weeks I've been using it instead of the old v1.5.

> let's see how to make things better.  For instance, maybe major and minor 
> need to be strings rather than integers.  

Hmmm. Good point - quite a few alpha versions have an "a" after the number,
although perhaps this could be better represented with an ALPHA switch. Or a
enum for alpha, beta etc.

> Or maybe the part field is just gay 
> and needs to be removed.

I can't see the part field doing any harm, and it's handy for packages like Vim
which don't fit the traditional mould (like Ogg Vorbis, I seem to remember).

-- 
James Reeves (Shade)
http://www.arevol.co.uk/

[ID] PackageInfo

[Andy G. <unu...@op...>]

Look this over and comment.  Take a look at a variety of source packages you 
already have and see how this system fits them.  If it doesn't fit, then 
let's see how to make things better.  For instance, maybe major and minor 
need to be strings rather than integers.  Or maybe the part field is just gay 
and needs to be removed.

#define TAR     1
#define ZIP     2
#define RAR     3
#define PATCH   4
#define DIFF    5

#define GZ      6
#define BZ2     7
#define Z       8

typedef struct PackageInfo {
        char* filename; /* man-1.5i2.tar.bz2 | vim-5.7-5.8-rt.diff.bz2 */
        char* name;     /* man               | vim                     */
        char* part;     /*                   | rt                      */

        int major;      /* 1                 | 5                       */
        int minor;      /* 5                 | 7                       */
        char* extra;    /* i2                |                         */

        int to_major;   /*                   | 5                       */
        int to_minor;   /*                   | 8                       */
        char* to_extra; /*                   |                         */

        int type;       /* TAR               | DIFF                    */
        int comp;       /* BZ2               | BZ2                     */
} PackageInfo;

-- 
Andy Goth  |  unu...@op...  |  http://sevatech.com/~andy/

[ID] Re: You back?

[Andy G. <unu...@op...>]

On Saturday, August 18, 2001 12:01, you wrote:
> On Sat, Aug 18, 2001 at 10:22:38AM -0500, Andy Goth wrote:
> > > Yesterday I sent a message with the subject "I'm back". You replied to
> > > it - remember? :)
> >
> > No, sorry.  You must have sent it to somebody else.
>
> Another Andy Goth with your sig, going to the University of Texas at
> Arlington, and has recently been reading a linear algebra book? :)

Odd.

Oh, I get it.  I got your message at exactly the same time as one from James 
Bunting, who I haven't "seen" for a year.  And the wording:

> Back off holiday, got my exam results (mostly what I wanted, but dropped a
> grade on Further Maths, which messes things up a bit). How are you? :)

certainly fits him.  So I thought he goofed and sent two messages, one vague 
and the other with a little more information.

So there.  Hehehe.

> Anyway, what's been happening with install-log while I've been gone?
> (downloading CVS version now)

I didn't make any changes except for the last day you were around.  I started 
poking at a structure to keep track of a package source file.  But the rest 
of the time I wasn't inspired to do anything else.  We must fix this. <g>

-- 
Andy Goth  |  unu...@op...  |  http://sevatech.com/~andy/

Re: [ID] Planning for 2.0

[Andy G. <unu...@op...>]

> >   latest/install-script
> >   latest/source
> >   latest/timestamp
>
> Oops, the timestamp file. Do we need a separate timestamp?

Probably not, but it's good for sorting packages chronologically.  There 
might be a down-the-road ability to list everything along with dates.

> We probably do,
> but my brain isn't working too we as I'm pretty tired.

Trust me, your brain's deficiencies have NOTHING on mine.  Not tonight, at 
least.

> The source &
> install-script can be added in later, was my point. At first lets just get
> the file logging up in the new structure :)

Oh, that's sensible.  Okay.

> > If a file is empty, such as "old", should it be there at all?  I'm not
> > sure, but I lean toward no.
>
> I don't think it really matters much.

Okay, no empty files.

> > Some more complicated ideas...
> >
> > xkobo/latest --> 1.1
>
> <snip>
>
> > xkobo/1.1/0/timestamp
>
> Perhaps any revision to the source could be achieved for longterm storage
> (don't want 10 copies of the source for each package :) as diffs (.bz2).

source isn't the actual source.  It's just a list of filenames.

vim-rt-5.7.tar.bz2
vim-src-5.7.tar.bz2
vim-rt-5.7-5.8.diff.bz2
vim-src-5.7-5.8.diff.bz2

See?

> > > Another point: with the install numbers, do we really need a separate
> > > directory? The only thing that would be changed would be the files,
> > > old, and del.
> >
> > No, the other things can differ as well.  timestamp certainly will. 
> > source might.  install-script might, too.  I mean, additional
> > installations of the same version are for fixing mistakes in the old
> > attempt(s).
>
> Point taken.

Okay, when (if?) I do this stuff, I'll have subdirectories for each different 
installation of the same package.

I say "if" because right now I'm feeling extremely uncertain about 
everything, wondering if I'll die tonight and why I haven't already.  You 
know, bad things. ;^)

Did you see my struct?  Does it look right, or does it look like it needs 
help?

-- 
Andy Goth  |  unu...@op...  |  http://sevatech.com/~andy/

Re: [ID] Planning for 2.0

[James R. <jr...@fu...>]

On Tue, Jul 31, 2001 at 05:50:33PM -0500, Andy Goth wrote:
> > > Are there any other goals we should set for 2.0?
> >
> > I think you've just about covered it. We first need to add in a parsing
> > function to sort out the name of the package, version number etc. before
> > the directory tree can be constructed. I've been out mostly today and I've
> > only just got back, so it'll probably be best for you code it :). From
> > there we can construct the directory.
> 
> Alright, I'll do that.
> 
> > How about first just making:
> >
> > $ ls /var/install-logs/$PACKAGE/*
> >
> > $VERSION/
> >
> > latest --> $VERSION
> > latest/files
> > latest/old
> > latest/del
>   latest/install-script
>   latest/source
>   latest/timestamp

Oops, the timestamp file. Do we need a separate timestamp? We probably do, but
my brain isn't working too we as I'm pretty tired. The source & install-script
can be added in later, was my point. At first lets just get the file logging up
in the new structure :)
 
> If a file is empty, such as "old", should it be there at all?  I'm not sure, 
> but I lean toward no.

I don't think it really matters much.

> Some more complicated ideas...
> 
> xkobo/latest --> 1.1
<snip> 
> xkobo/1.1/0/timestamp

Perhaps any revision to the source could be achieved for longterm storage
(don't want 10 copies of the source for each package :) as diffs (.bz2).

> > Another point: with the install numbers, do we really need a separate
> > directory? The only thing that would be changed would be the files, old,
> > and del.
> 
> No, the other things can differ as well.  timestamp certainly will.  source 
> might.  install-script might, too.  I mean, additional installations of the 
> same version are for fixing mistakes in the old attempt(s).

Point taken.

-- 
James Reeves (Shade)
http://arevol.envy.nu/

[ID] package.c

[Andy G. <unu...@op...>]

Check it out.  It's not much, and I'm sure it needs work, but take a look.  
What more do we need in it, besides some real functions?

-- 
Andy Goth  |  unu...@op...  |  http://sevatech.com/~andy/

Re: [ID] Planning for 2.0

[Andy G. <unu...@op...>]

> Yep, the last plan was a bit rough in places.

It wasn't nearly organized enough to be called a plan.

> > Okay, according to the news entry I posted earlier today, "version 2.0
> > will certainly be a landmark. It will have a new database layout,
> > automation (or at least assistance, depending on your preference) of
> > package installation, upgrading, and removal, and the ability to log more
> > information than just file lists, such as an upgrade history."
>
> Oooo - impressive :)

Call me Mr. PR.

> > That means we need to design the new database layout and write standard
> > shell scripts for installation, upgrading, and removal of packages.  I
> > invite you to take the first stab(s) at these.
>
> The scripts will be ok, I'll get work on them.

Okay, thanks.  Remember, things like unpacking a tarball or applying a diff 
can (and should) be hidden in a function that the install script can call.

I was thinking last night about keeping all the custom install scripts for 
every package we (or any other user) end up needing to tinker with.  We can 
archive them on the site, and maybe there can be some sort of (future) 
ability to download them as needed.  install-log -di xkobo, for instance.  ;^)

> > Are there any other goals we should set for 2.0?
>
> I think you've just about covered it. We first need to add in a parsing
> function to sort out the name of the package, version number etc. before
> the directory tree can be constructed. I've been out mostly today and I've
> only just got back, so it'll probably be best for you code it :). From
> there we can construct the directory.

Alright, I'll do that.

> How about first just making:
>
> $ ls /var/install-logs/$PACKAGE/*
>
> $VERSION/
>
> latest --> $VERSION
> latest/files
> latest/old
> latest/del
  latest/install-script
  latest/source
  latest/timestamp

If a file is empty, such as "old", should it be there at all?  I'm not sure, 
but I lean toward no.

Some more complicated ideas...

xkobo/latest --> 1.1

xkobo/1.0/files          --> latest/files
xkobo/1.0/del            --> latest/del
xkobo/1.0/install-script --> latest/install-script
xkobo/1.0/source         --> latest/source
xkobo/1.0/timestamp      --> latest/timestamp
xkobo/1.0/latest         --> 1

xkobo/1.0/0/files
xkobo/1.0/0/install-sccript
xkobo/1.0/0/source
xkobo/1.0/0/timestamp

xkobo/1.0/1/files
xkobo/1.0/1/del
xkobo/1.0/1/install-script
xkobo/1.0/1/source
xkobo/1.0/1/timestamp

xkobo/1.1/files          --> latest/files
xkobo/1.1/old            --> latest/old
xkobo/1.1/del            --> latest/del
xkobo/1.1/install-script --> latest/install-script
xkobo/1.1/source         --> latest/source
xkobo/1.1/timestamp      --> latest/timestamp
xkobo/1.1/latest         --> 0

xkobo/1.1/0/files
xkobo/1.1/0/old
xkobo/1.1/0/del
xkobo/1.1/0/install-script
xkobo/1.1/0/source
xkobo/1.1/0/timestamp

> Another point: with the install numbers, do we really need a separate
> directory? The only thing that would be changed would be the files, old,
> and del.

No, the other things can differ as well.  timestamp certainly will.  source 
might.  install-script might, too.  I mean, additional installations of the 
same version are for fixing mistakes in the old attempt(s).

> We could always do files --> files-3 to point to the latest
> revision. Hmmm, this needs a bit more thought; we had the general idea
> before, now to figure out the exact details.

Naah, a subdirectory is probably the easiest way to go.  But this is still a 
confusing issue, because I'm trying to do it all at once.

-- 
Andy Goth  |  unu...@op...  |  http://sevatech.com/~andy/

Re: [ID] Planning for 2.0

[James R. <jr...@fu...>]

On Tue, Jul 31, 2001 at 02:41:41AM -0500, Andy Goth wrote:
> /me just updated the website and announced the new version to the 
> http://www.gnu.org/directory/ list.
> 
> Alright, time to do some planning for 2.0.  Yeah, I know we already did it, 
> but we don't have a record.  It'll be good for us to think things through a 
> second time, and this time more realistically.  (Now that we've actually done 
> some programming, we know better the meaning of "real"!)

Yep, the last plan was a bit rough in places.

> Okay, according to the news entry I posted earlier today, "version 2.0 will 
> certainly be a landmark. It will have a new database layout, automation (or 
> at least assistance, depending on your preference) of package installation, 
> upgrading, and removal, and the ability to log more information than just 
> file lists, such as an upgrade history."

Oooo - impressive :)

> That means we need to design the new database layout and write standard shell 
> scripts for installation, upgrading, and removal of packages.  I invite you 
> to take the first stab(s) at these.

The scripts will be ok, I'll get work on them.

> Are there any other goals we should set for 2.0?

I think you've just about covered it. We first need to add in a parsing
function to sort out the name of the package, version number etc. before the
directory tree can be constructed. I've been out mostly today and I've only
just got back, so it'll probably be best for you code it :). From there we can
construct the directory.

How about first just making:

$ ls /var/install-logs/$PACKAGE/*

$VERSION/

latest --> $VERSION
latest/files
latest/old
latest/del

Oh, and looking over some old emails discussing the filestructure, instead of:

5.7-0/ --> 2001.06.08.00.05.07/
5.8-0/ --> 2001.07.13-22.02.07/

latest/ --> 5.8-0/
version --> latest/version

2001.06.08-00.05.00/version
     5.7-0                     // Version 5.7, installation number 0

.

How about:

5.7-0/ (dir not link)
5.8-0/
latest --> 5.8-0/

5.7-0/version
	2001.06.05-00.05.00

Another point: with the install numbers, do we really need a separate
directory? The only thing that would be changed would be the files, old, and
del. We could always do files --> files-3 to point to the latest revision.
Hmmm, this needs a bit more thought; we had the general idea before, now to
figure out the exact details.

-- 
James Reeves (Shade)
http://arevol.envy.nu/

Re: [ID] mk_test

[James R. <jr...@fu...>]

On Mon, Jul 30, 2001 at 05:21:44PM -0500, Andy Goth wrote:
> Here's a handy script for building a test environment for install-log.
<snip> 
> Just execute install-log with the "-r test" option, and it'll (hopefully) use 
> this.

Got my own test environment (what do think I've been testing it in all this
time?). I tend to try it out in there first, then back up /var/install-logs and
use it to install any future software.

-- 
James Reeves (Shade)
http://arevol.envy.nu/

[ID] Planning for 2.0

[Andy G. <unu...@op...>]

/me just updated the website and announced the new version to the 
http://www.gnu.org/directory/ list.

Alright, time to do some planning for 2.0.  Yeah, I know we already did it, 
but we don't have a record.  It'll be good for us to think things through a 
second time, and this time more realistically.  (Now that we've actually done 
some programming, we know better the meaning of "real"!)

Okay, according to the news entry I posted earlier today, "version 2.0 will 
certainly be a landmark. It will have a new database layout, automation (or 
at least assistance, depending on your preference) of package installation, 
upgrading, and removal, and the ability to log more information than just 
file lists, such as an upgrade history."

That means we need to design the new database layout and write standard shell 
scripts for installation, upgrading, and removal of packages.  I invite you 
to take the first stab(s) at these.

Are there any other goals we should set for 2.0?

-- 
Andy Goth  |  unu...@op...  |  http://sevatech.com/~andy/

[ID] segfault fixed!

[Andy G. <unu...@op...>]

Yay!  Time to release 1.9... and it's only an hour late.

-- 
Andy Goth  |  unu...@op...  |  http://sevatech.com/~andy/

[ID] Ack

[Andy G. <unu...@op...>]

Maybe 1.9 won't be released today.  Get the latest cvs and link it with 
efence to see why.  It somehow comes up with a really crazy prev_files linked 
list.  Weirder still, it's fine upon creation, but by the time it gets used 
it has mutated.

next: right arrows
data: down arrows

node1 --> node2 --> node3 ----+
  |         |         |       |
  v         v         v       |
str1      str2      str3      |
            ^                 |
            |                 |
            +-----------------+

What the...?  Imagine what happens when (List*)((&str2) + 4)->data is used... 
 Of course, our program prefers to look at it as node3->next->data.

And I could be looking at this totally wrong, too.  If I could find a copy of 
ddd that isn't corrupted, I'd use it.  Know of any mirrors?  The latest 
version I've heard of us 3.2.1, but when I try to download it from the main 
site I wind up with a corrupt file.

-- 
Andy Goth  |  unu...@op...  |  http://sevatech.com/~andy/

[ID] mk_test

[Andy G. <unu...@op...>]

Here's a handy script for building a test environment for install-log.

#!/bin/sh
rm -rf test
mkdir test test/etc test/usr test/var
touch -d"Jan 1, 1970" test/var/.timestamp
touch test/usr/{01,02,03,04,05,06,07,08,09,10}
cat > test/etc/install-log.rc << EOF
INCLUDE=/usr
LOGDIR=/var
EOF

Just execute install-log with the "-r test" option, and it'll (hopefully) use 
this.

-- 
Andy Goth  |  unu...@op...  |  http://sevatech.com/~andy/

Re: [ID] sort_string_list

[Andy G. <unu...@op...>]

> > Yep, an insertion sort would probably be best (and easiest to implement
> > :)
>
> It's already done.
>
> /* Add a string node, using an insertion sort */
> void insert_string_node(List* list, const char* string)
> {
> 	List* node = list;
> 	List* new_node = make_node(strdup(string));
>
> 	/* Scan for proper position */
> 	for (; node->next != NULL; node = node->next)
> 		if (strcmp((char*)node->next->data, string) > 0) break;
>
> 	/* Insert! */
> 	new_node->next = node->next;
> 	node->next = new_node;
> }
>
> Does that look right?  Notice that it takes a List*, not a List** as the
> sequential add_node functions do.

It works.  James, the version I emailed you says "< 0", which is backwards.  
I committed it to cvs.

-- 
Andy Goth  |  unu...@op...  |  http://sevatech.com/~andy/

Re: [ID] sort_string_list

[James R. <jr...@fu...>]

On Mon, Jul 30, 2001 at 03:26:00PM -0500, Andy Goth wrote:
> > After my A Level course in Decision Maths, and my reading material for my
> > up and coming Uni course, I know quicksort like the back of my hand :)
> 
> You sure do have weird names for courses in school... hehehe.

The Decision Mahs course was also a very annoying one. Over here, the exams are
means-tested (I don't know if this is true over in the US (are you from the
US?)), and the first DM exam I took was extremely easy; I only got a few marks
off for little mistakes, though I got the main questions all ostly right.
Unfortunately every one else found it easy as well so the A grade I wanted
(if you have experiance programming, DM is pretty easy :) was bumped up from an
over 75% pass rate, to an over 96% pass rate - D'oh!

Hopefully my retake will be better. Hmmm, sorry for going a bit off-topic there
:)
 
> > Quicksort is a recusive algorithm, that works by taking a "pivot" (usually
<snip>
> It's weird to watch.  java.sun.com has a little demonstration of quicksort 
> running alongside bubble sort and bidirectional bubble sort.

I'll have to get java up and working again. Now that I switched full-time from
Mandrake to my own LFS, there's still quite a lot of little programs to add in.
Like Realplayer for one...

> Won't this require a doubly-linked list?  We don't have that, and I don't 
> want to implement it.  I know that I've been making a ridiculous amount of 
> industrial-strength stuff in this project, but _this_ is too much!

I tend to agree with you there.

> By the way, this is almost exactly what I saw on the Linux kernel development 
> list.

Great mind think alike - or else we just read the same books :)

> > middle value will have to be found, or bubble sort used. What is the
> > sorting funtion going to be used for? Checking files against the database?
> 
> It's for writing to the database.  The database files were alphabetically 
> sorted in install-log.sh, and I thought that was a nice feature.
> 
> All we really need to do is sort old_files, new_files, and del_files as we 
> generate them.  That's an insertion sort, and I'm about to implement it.

Yep, an insertion sort would probably be best (and easiest to implement :)

-- 
James Reeves (Shade)
http://arevol.envy.nu/

Re: [ID] sort_string_list

[Andy G. <unu...@op...>]

> After my A Level course in Decision Maths, and my reading material for my
> up and coming Uni course, I know quicksort like the back of my hand :)

You sure do have weird names for courses in school... hehehe.

> Quicksort is a recusive algorithm, that works by taking a "pivot" (usually
> the middle item) and then comparing each item in the list with the pivot,
> putting it one side of the pivot or the other. The two separate sides are
> then treated as lists in their own right and the function called again,
> until there are no changes to be made. In psuedo-ish code:

It's weird to watch.  java.sun.com has a little demonstration of quicksort 
running alongside bubble sort and bidirectional bubble sort.

> quicksort(list sort_list)
> {
> 	list lower_list;
> 	list higher_list;
>
> 	if (sort_list->length <= 1)
> 		return;
>
> 	pivot = middle_of(sort_list);
>
> 	foreach (sort_list) {
> 		if (list_item <= pivot)
> 			add_to(lower_list, list_item);
> 		else
> 			add_to(higher_list, list_item);
> 	}
>
> 	quicksort(lower_list);
> 	quicksort(higher_list);
>
> 	sort_list = merge(lower_list, pivot, higher_list);
> }

Won't this require a doubly-linked list?  We don't have that, and I don't 
want to implement it.  I know that I've been making a ridiculous amount of 
industrial-strength stuff in this project, but _this_ is too much!

By the way, this is almost exactly what I saw on the Linux kernel development 
list.

> Note that quicksort is usually faster, but not always. Bubble sort is
> usually faster if the list is nearly sorted already (like 1 2 3 4 8 5 6 7
> 9).

Disk order is pretty much random.

> The main problem is choosing a pivot then. If the list is truely random,
> then the pivot can simply be the first list item. If not, then either the
> middle value will have to be found, or bubble sort used. What is the
> sorting funtion going to be used for? Checking files against the database?

It's for writing to the database.  The database files were alphabetically 
sorted in install-log.sh, and I thought that was a nice feature.

All we really need to do is sort old_files, new_files, and del_files as we 
generate them.  That's an insertion sort, and I'm about to implement it.

-- 
Andy Goth  |  unu...@op...  |  http://sevatech.com/~andy/

Re: [ID] Electric Fence saves the day

[Andy G. <unu...@op...>]

On Monday, July 30, 2001 10:30, you wrote:
> On Mon, Jul 30, 2001 at 03:53:34AM -0500, Andy Goth wrote:
> > Go ahead and get it--it's an excellent program.
> >
> > http://sevatech.com/~andy/files/ElectricFence-2.2.2.tar.bz2
>
> Looks useful! I didn't know that there were tools like this around.

yamd will analyze all of your mallocs and frees.  You might want to get that 
as well.  For catching my bugs, efence worked a little better, though.  yamd 
showed me some minor memory leaks, so it's useful too.

> > We need:
> >
> > - Sorting

We need sorting when we output to the database files.  Currently it outputs 
everything in disk order, which is next to random.  It's semi-sorted by 
directory, but that's about it.

> What for? Not to be rude or anything, it's just that the use determines the
> algorithm used. I looked at designing a quicksort, but on a linked list a
> bubble-sort may be more effective, unless the list is very random. A bubble
> sort is slower generally; it's performance degrades with the greater
> randomness of the list, but it has a few advantages in this case over a
> quicksort:
>
> 1) It's not recursive.
> 2) It uses up less memory.
> 3) You don't need to choose a pivot.
>
> The less random a list, the more of a factor No. 3 becomes (see other email
> for the quicksort algorithm). In a totally random list, the pivot can be at
> the start. In a less random list, it's better for it to be in the middle
> (quite tricky with a linked list), and then it doesn't perform too much
> better than a bubble sort.

After reading some discussion on a Linux kernel development list, I realized 
that the best thing for us would be keeping the list sorted and doing an 
insertion sort (where everything is put in the right place to begin with).

So now I'm going to write a new list function for that. ;^)

> > - Notification in case of old and deleted files
>
> Does this already happen? In process_prev_file there's:
> 	report(5, "Found old file: %s\n", (char*)node->data);
> and
> 	report(5, "Found deleted file: %s\n", (char*)node->data);

From install-log.sh:

[ ! "${QUIET}" ] && grep '^\#' "${PACKAGE}" && \
		utter "Warning: Deletions and/or stale files encountered"

> > ..
> >
> > Hey, I've been asked--sort of--to write a Java chat program for a telecom
> > company in Croatia, but I hope that I can finish install-log first.  If I
> > don't take a programming project seriously, it either doesn't get
> > finished or takes forever.  For instance, install-log.sh was written over
> > a very long period of time, but it was actually done in three, maybe
> > four, bursts of activity.  Judging from the length, each burst couldn't
> > have taken more than a few hours.  But overall it was several months.

One nice thing about the client-server model is that I can write this chat 
client to connect to a Tcl chat server I already wrote.

> I'm the same, or maybe worse - I have a whole collection of unfinished
> pieces of software. One KDE program (designed to communicate with a similar
> program in Windows) I use, but it never got past "beta" stage. Still, we I
> do finally release it, it'll be as stable as a rock :).

Could you tell me a little bit more about it?

> Actually, I found with this project that working with someone else makes me
> take it more seriously, so to speak :)

And you have been a huge help to me, as well.

I've been in failed projects with several other people, but I think that cvs 
is what makes the difference.  Before, it's been pure email.  I didn't even 
know how to download and build the sucker.  So that limited me to only 
writing theoretical stuff, never real source.  And that enabled them to 
ignore me and each other.  It didn't matter how good my ideas were on 
configuration file formats and dynamic module loading... the guy who was 
actually writing stuff made up his own configuration format and considered 
#included source code to be a dynamically loaded module.  "And when the user 
wants to change modules, all he needs to do is adjust some #defines and 
recompile everything!"

-- 
Andy Goth  |  unu...@op...  |  http://sevatech.com/~andy/

Re: [ID] Electric Fence saves the day

[James R. <jr...@fu...>]

On Mon, Jul 30, 2001 at 03:53:34AM -0500, Andy Goth wrote:
> Go ahead and get it--it's an excellent program.
> 
> http://sevatech.com/~andy/files/ElectricFence-2.2.2.tar.bz2

Looks useful! I didn't know that there were tools like this around.
 
> We need:
>
> - Sorting

What for? Not to be rude or anything, it's just that the use determines the
algorithm used. I looked at designing a quicksort, but on a linked list a
bubble-sort may be more effective, unless the list is very random. A bubble
sort is slower generally; it's performance degrades with the greater randomness
of the list, but it has a few advantages in this case over a quicksort:

1) It's not recursive.
2) It uses up less memory.
3) You don't need to choose a pivot.

The less random a list, the more of a factor No. 3 becomes (see other email for
the quicksort algorithm). In a totally random list, the pivot can be at the
start. In a less random list, it's better for it to be in the middle (quite
tricky with a linked list), and then it doesn't perform too much better than a
bubble sort.

> - Notification in case of old and deleted files

Does this already happen? In process_prev_file there's:
	report(5, "Found old file: %s\n", (char*)node->data);
and
	report(5, "Found deleted file: %s\n", (char*)node->data);

> - An editor.c source file

Done.
 
> Can you think of anything else?  If not, then we can release 1.9 very soon.  
> I'd say tomorrow, but it's already 3:50 a.m. here.  So that means later 
> today!  Cool.

The list seems to be complete...

> ..
> 
> Hey, I've been asked--sort of--to write a Java chat program for a telecom 
> company in Croatia, but I hope that I can finish install-log first.  If I 
> don't take a programming project seriously, it either doesn't get finished or 
> takes forever.  For instance, install-log.sh was written over a very long 
> period of time, but it was actually done in three, maybe four, bursts of 
> activity.  Judging from the length, each burst couldn't have taken more than 
> a few hours.  But overall it was several months.

I'm the same, or maybe worse - I have a whole collection of unfinished
pieces of software. One KDE program (designed to communicate with a similar
program in Windows) I use, but it never got past "beta" stage. Still, we I do
finally release it, it'll be as stable as a rock :). 

Actually, I found with this project that working with someone else makes me
take it more seriously, so to speak :)

-- 
James Reeves (Shade)
http://arevol.envy.nu/

Re: [ID] sort_string_list

[James R. <jr...@fu...>]

On Sun, Jul 29, 2001 at 09:01:21PM -0500, Andy Goth wrote:
> We need a sort_string_list function... could you find information on how to 
> do a quicksort?  Bubble sorting sucks.

After my A Level course in Decision Maths, and my reading material for my up
and coming Uni course, I know quicksort like the back of my hand :)

Quicksort is a recusive algorithm, that works by taking a "pivot" (usually the
middle item) and then comparing each item in the list with the pivot, putting
it one side of the pivot or the other. The two separate sides are then treated
as lists in their own right and the function called again, until there are no
changes to be made. In psuedo-ish code:

quicksort(list sort_list)
{
	list lower_list;
	list higher_list;

	if (sort_list->length <= 1)
		return;

	pivot = middle_of(sort_list);

	foreach (sort_list) {
		if (list_item <= pivot)
			add_to(lower_list, list_item);
		else
			add_to(higher_list, list_item);
	}

	quicksort(lower_list); 
	quicksort(higher_list);

	sort_list = merge(lower_list, pivot, higher_list);		
}

Note that quicksort is usually faster, but not always. Bubble sort is usually
faster if the list is nearly sorted already (like 1 2 3 4 8 5 6 7 9).

The main problem is choosing a pivot then. If the list is truely random, then
the pivot can simply be the first list item. If not, then either the middle
value will have to be found, or bubble sort used. What is the sorting funtion
going to be used for? Checking files against the database?

-- 
James Reeves (Shade)
http://arevol.envy.nu/

[ID] Electric Fence saves the day

[Andy G. <unu...@op...>]

Go ahead and get it--it's an excellent program.

http://sevatech.com/~andy/files/ElectricFence-2.2.2.tar.bz2

If install-log exhibits strange behavior (as it did for me--malloc 
crashed!!), use it.

make clean && make CFLAGS=-g LIBS=-lefence && gdb install-log

It's useless outside of a debugger because its purpose is to crash the 
program the instant it tries to do something with memory that it shouldn't.

It helped me to fix extremely subtle bugs in proc_list and 
find_old_and_del_files and other places I forgot.  But there's still work to 
be done in database.c.  Some of it's messy with memory, and I need to fix it 
up.  I'm not sure I can explain in this letter, though, so for now it's best 
you don't worry about it.

We need:
- Sorting
- Notification in case of old and deleted files
- An editor.c source file

Can you think of anything else?  If not, then we can release 1.9 very soon.  
I'd say tomorrow, but it's already 3:50 a.m. here.  So that means later 
today!  Cool.

...

Hey, I've been asked--sort of--to write a Java chat program for a telecom 
company in Croatia, but I hope that I can finish install-log first.  If I 
don't take a programming project seriously, it either doesn't get finished or 
takes forever.  For instance, install-log.sh was written over a very long 
period of time, but it was actually done in three, maybe four, bursts of 
activity.  Judging from the length, each burst couldn't have taken more than 
a few hours.  But overall it was several months.

I actually have some other programming projects, right now, that haven't 
moved at all thanks to install-log.

G'night. ;^)

-- 
Andy Goth  |  unu...@op...  |  http://sevatech.com/~andy/

[ID] sort_string_list

[Andy G. <unu...@op...>]

We need a sort_string_list function... could you find information on how to 
do a quicksort?  Bubble sorting sucks.

-- 
Andy Goth  |  unu...@op...  |  http://sevatech.com/~andy/