integrit-devel — mailing list for discussion of issues related to developing integrit

[Integrit-devel] cosmetic fixes

[Yuri D'E. <wa...@yu...>]

Before the release, I noted there are several cases of warning/error  
messages with extra newlines.
I left just the two cases in option.c which are aesthetically  
pleasing. Let me know if this sounds ok or these newlines were  
actually intended.

Re: [Integrit-devel] documentation update

[Yuri D'E. <wa...@yu...>]

On Sep 23, 2005, at 20:40, Ed L. Cashin wrote:

> On 9/23/05, Yuri D'Elia <wa...@yu...> wrote:
>
>> I finished merging-in the whitespace changes that were lost in the
>> first patch and a i-ls fix included in a later one. I did some
>> documentation for stop_on_err, please proof-read the attached diff.
>
> Looks good to me.  Thanks for keeping the docs
> up to date!

Committed. I did not regenerate the integrit.info file however, I  
still use an older version of makeinfo. I think we're now ready for 3.6.

Re: [Integrit-devel] documentation update

[Ed L. C. <ec...@no...>]

On 9/23/05, Yuri D'Elia <wa...@yu...> wrote:
> I finished merging-in the whitespace changes that were lost in the
> first patch and a i-ls fix included in a later one. I did some
> documentation for stop_on_err, please proof-read the attached diff.

Looks good to me.  Thanks for keeping the docs
up to date!

--
  Ed L. Cashin <ec...@no...>

[Integrit-devel] documentation update

[Yuri D'E. <wa...@yu...>]

I finished merging-in the whitespace changes that were lost in the  
first patch and a i-ls fix included in a later one. I did some  
documentation for stop_on_err, please proof-read the attached diff.

Re: [Integrit-devel] even more integrit patches

[Yuri D'E. <wa...@yu...>]

On Sep 20, 2005, at 3:18, Ed L. Cashin wrote:

> On 9/18/05, Yuri D'Elia <wa...@yu...> wrote:
>
>> [hopefully the last set]
>>
>> - patch2.diff:  introduces the "digest.h" wrapper, making future
>> digest changes trivial. Removed the use of md5, db sums are now in
>> sha1. I'm sorry this patch includes whitespace changes from the
>> previous one. I can re-diff if needed.
>> - patch3.diff: switches integrit to Ripemd-160. Same checksum size as
>> SHA-1 (keeps the db small), almost as fast, included in gnupg stable
>> series (I've used the rmd160.c implementation from gpg 1.4.2), not
>> broken yet. Requires patch2.
>> - patch4.diff: small i-ls fix.
>>
>
> This will be a major version number increase,
> to version 4.0, because it will result in an integrit
> that is not compatible with the older versions.
>
> ... Not that there's anything wrong with that.
> If the old checksums are obsolete, compatibility
> is not desirable.  Plus, users interested in their
> old databases can easily keep an old integrit
> with their databases.

Agreed.

Re: [Integrit-devel] more integrit patches

[Yuri D'E. <wa...@yu...>]

On Sep 20, 2005, at 2:54, Ed L. Cashin wrote:

> On 9/17/05, Yuri D'Elia <wa...@yu...> wrote:
> ...
>
>> I saw "configure" got removed from cvs, but some more files need to
>> be removed consequently:
>>
>> - config.h.in
>> - hashtbl/config.h.in
>> - hashtbl/configure
>>
>
> Those config.h.in files are the templates that
> autoconf uses to generate config.h files, so
> they need to stay.

config.h.in is generated by autoheader directly from configure.in, at  
least in newer versions of autoconf.
I would not mind if it didn't change each time I run the auto* tools.

Re: [Integrit-devel] even more integrit patches

[Ed L. C. <ec...@gm...>]

On 9/18/05, Yuri D'Elia <wa...@yu...> wrote:
> [hopefully the last set]
>
> - patch2.diff:  introduces the "digest.h" wrapper, making future
> digest changes trivial. Removed the use of md5, db sums are now in
> sha1. I'm sorry this patch includes whitespace changes from the
> previous one. I can re-diff if needed.
> - patch3.diff: switches integrit to Ripemd-160. Same checksum size as
> SHA-1 (keeps the db small), almost as fast, included in gnupg stable
> series (I've used the rmd160.c implementation from gpg 1.4.2), not
> broken yet. Requires patch2.
> - patch4.diff: small i-ls fix.

This will be a major version number increase,
to version 4.0, because it will result in an integrit
that is not compatible with the older versions.

... Not that there's anything wrong with that.
If the old checksums are obsolete, compatibility
is not desirable.  Plus, users interested in their
old databases can easily keep an old integrit
with their databases.

--
  Ed L. Cashin <ec...@no...>

Re: [Integrit-devel] more integrit patches

[Ed L. C. <ec...@gm...>]

On 9/17/05, Yuri D'Elia <wa...@yu...> wrote:
...
> I saw "configure" got removed from cvs, but some more files need to
> be removed consequently:
>
> - config.h.in
> - hashtbl/config.h.in
> - hashtbl/configure

Those config.h.in files are the templates that
autoconf uses to generate config.h files, so
they need to stay.

--
  Ed L. Cashin <ec...@no...>

[Integrit-devel] even more integrit patches

[Yuri D'E. <wa...@yu...>]

[hopefully the last set]

- patch2.diff:  introduces the "digest.h" wrapper, making future  
digest changes trivial. Removed the use of md5, db sums are now in  
sha1. I'm sorry this patch includes whitespace changes from the  
previous one. I can re-diff if needed.
- patch3.diff: switches integrit to Ripemd-160. Same checksum size as  
SHA-1 (keeps the db small), almost as fast, included in gnupg stable  
series (I've used the rmd160.c implementation from gpg 1.4.2), not  
broken yet. Requires patch2.
- patch4.diff: small i-ls fix.

[Integrit-devel] more integrit patches

[Yuri D'E. <wa...@yu...>]

Here's some more changes I discussed with Ed against cvs head  
(attached):

- Introduced the new "stop_on_err" directive in the configuration  
file. When stop_on_err is 0, and a non-fatal failure is encountered  
(like inability to perform a checksum or open a directory) integrit  
will warn the user, but continue the process. The exit status will  
still indicate the presence of an error. My statement for the need of  
this option is:

     "You cannot perform static analysis without this option on a  
read-only media. You either need to remove the file that's causing  
the problem or proceed manually."

   Currently defaults to 1 (yes), which makes integrit behave like  
the old versions.

- Disabling checksum for a file would still checksum the file during  
update, contrarily to what the documentation says. This has been  
fixed. Moreover, a race condition when comparing databases generated  
by different rules (with/without checksums for the same files) was  
fixed.

- Since updating and checking on same file is currently not supported  
(I just got a crash), I added a little check that warns the user and  
quits.

I saw "configure" got removed from cvs, but some more files need to  
be removed consequently:

- config.h.in
- hashtbl/config.h.in
- hashtbl/configure

Just for completeness, README states:

   The reason for this limitation is simple: there is no advantage to
   using more than one algorithm.  SHA-1 (Secure Hash Algorithm 1)
   produces a 160-bit checksum.  Since there are no known weaknesses in
   SHA-1, to find a file with the same checksum as a given file on your
   ...

I agree with most parts of the text (one good checksum is  
sufficient), but please note:

- Multiple checksums for a single files reduce the possibility that,  
if a hole is found in one of the algorithms, the same method can be  
used successfully against another algorithm. This is, of course,  
unlikely, but there's enough ground to justify the need of more than  
one algorithm.
- SHA-1 has been broken in 2/2005. We should investigate the newer  
SHA variants.

Regards

Re: [Integrit-devel] Fixing security/integrit build errors on sparc64

[Ed L. C. <ec...@gm...>]

On 9/13/05, Jos Backus <jo...@ca...> wrote:
...
> Oops, I was looking at the 3.0.4 sources where this has been fixed and th=
us
> was drawing the wrong conclusion; the error message was complaining about=
 the
> _presence_ of the paste operators. Hence the paste operators need to go
> instead (they are in the 3.0.2 sources) as is now the case. So I would ex=
pect
> this version to build on sparc64.

Yes, I saw that Yuri made that change and guessed it
was for that reason.  My systems seem happy with it
too, so that's good.

--=20
  Ed L. Cashin <ec...@no...>

Re: [Integrit-devel] integrit patches

[Jos B. <jo...@ca...>]

On Tue, Sep 13, 2005 at 02:12:14PM -0400, Ed L. Cashin wrote:
> On 9/13/05, Jos Backus <jo...@ca...> wrote:
> ...
> > Fwiw: 3.0.4 builds cleanly without warnings on both recent FreeBSD 4-stable
> > and FreeBSD HEAD. This is on i386; I know it doesn't build on sparc64 but I
> > have no means of testing it on that platform. I guess I could ask the ports
> > people what the problems are.
> 
> Thanks!  Did you notice that integrit is still
> recursively descending the filesystem correctly?
> That's the part that was modified recently.
 
I ran `test/test' which says that everything's fine. Do you think that is a
sufficient guarantee that recursively descending the filesystem still works
correctly?

-- 
Jos Backus
jos at catnook.com

Re: [Integrit-devel] Fixing security/integrit build errors on sparc64

[Jos B. <jo...@ca...>]

On Tue, Sep 13, 2005 at 11:24:26AM -0700, Jos Backus wrote:
> See http://pointyhat.freebsd.org/errorlogs/sparc64-errorlogs/e.6.2005060522/integrit-3.02.00.log
> 
> gnupg/md5.c has:
> 
>     p = hd->buf;
>   #ifdef BIG_ENDIAN_HOST
>     #define X(a) do { *p++ = hd->a      ; *p++ = hd->a >> 8;      \
>                       *p++ = hd->a >> 16; *p++ = hd->a >> 24; } while(0)
>   #else /* little endian */
>     /*#define X(a) do { *(u32*)p = hd->##a ; p += 4; } while(0)*/
>     /* Unixware's cpp doesn't like the above construct so we do it his way:
>      * (reported by Allan Clark) */
>     #define X(a) do { *(u32*)p = (*hd).a ; p += 4; } while(0)
>   #endif
>     X(A);
>     X(B);
>     X(C);
>     X(D);
>   #undef X
> 
> I could be wrong but it looks like the BIG_ENDIAN_HOST case is broken (sparc64
> is big endian); there, each occurrence of `hd->a' needs to be replaced with
> `hd->##a' (i.e. use the paste operator). But I have no way of testing this.
 
Oops, I was looking at the 3.0.4 sources where this has been fixed and thus
was drawing the wrong conclusion; the error message was complaining about the
_presence_ of the paste operators. Hence the paste operators need to go
instead (they are in the 3.0.2 sources) as is now the case. So I would expect
this version to build on sparc64.

Sorry for the confusion.

-- 
Jos Backus
jos at catnook.com

Re: [Integrit-devel] Fixing security/integrit build errors on sparc64

[Jos B. <jo...@ca...>]

See http://pointyhat.freebsd.org/errorlogs/sparc64-errorlogs/e.6.2005060522/integrit-3.02.00.log

gnupg/md5.c has:

    p = hd->buf;
  #ifdef BIG_ENDIAN_HOST
    #define X(a) do { *p++ = hd->a      ; *p++ = hd->a >> 8;      \
                      *p++ = hd->a >> 16; *p++ = hd->a >> 24; } while(0)
  #else /* little endian */
    /*#define X(a) do { *(u32*)p = hd->##a ; p += 4; } while(0)*/
    /* Unixware's cpp doesn't like the above construct so we do it his way:
     * (reported by Allan Clark) */
    #define X(a) do { *(u32*)p = (*hd).a ; p += 4; } while(0)
  #endif
    X(A);
    X(B);
    X(C);
    X(D);
  #undef X

I could be wrong but it looks like the BIG_ENDIAN_HOST case is broken (sparc64
is big endian); there, each occurrence of `hd->a' needs to be replaced with
`hd->##a' (i.e. use the paste operator). But I have no way of testing this.

-- 
Jos Backus
jos at catnook.com

Re: [Integrit-devel] integrit patches

[Ed L. C. <ec...@gm...>]

On 9/13/05, Jos Backus <jo...@ca...> wrote:
...
> Fwiw: 3.0.4 builds cleanly without warnings on both recent FreeBSD 4-stab=
le
> and FreeBSD HEAD. This is on i386; I know it doesn't build on sparc64 but=
 I
> have no means of testing it on that platform. I guess I could ask the por=
ts
> people what the problems are.

Thanks!  Did you notice that integrit is still
recursively descending the filesystem correctly?
That's the part that was modified recently.

--=20
  Ed L. Cashin <ec...@no...>

[Integrit-devel] Re: Fixing security/integrit build errors on sparc64

[Jos B. <jo...@ca...>]

On Tue, Sep 13, 2005 at 02:00:40PM -0400, Kris Kennaway wrote:
> On Tue, Sep 13, 2005 at 10:41:02AM -0700, Jos Backus wrote:
> > I am the port maintainer for security/integrit. A new release is about to be
> > made and perhaps the issues with building this port on sparc64 can be
> > addressed if I have some way of knowing what they are. Thanks.
> 
> Start with http://pointyhat.freebsd.org for all your error reporting
> needs.
 
Thanks Kris.

> Kris

-- 
Jos Backus
jos at catnook.com

[Integrit-devel] Re: Fixing security/integrit build errors on sparc64

[Kris K. <kr...@ob...>]

On Tue, Sep 13, 2005 at 10:41:02AM -0700, Jos Backus wrote:
> I am the port maintainer for security/integrit. A new release is about to be
> made and perhaps the issues with building this port on sparc64 can be
> addressed if I have some way of knowing what they are. Thanks.

Start with http://pointyhat.freebsd.org for all your error reporting
needs.

Kris

[Integrit-devel] Fixing security/integrit build errors on sparc64

[Jos B. <jo...@ca...>]

I am the port maintainer for security/integrit. A new release is about to be
made and perhaps the issues with building this port on sparc64 can be
addressed if I have some way of knowing what they are. Thanks.

-- 
Jos Backus
jos at catnook.com

Re: [Integrit-devel] integrit patches

[Jos B. <jo...@ca...>]

On Tue, Sep 13, 2005 at 12:32:03PM -0400, Ed L. Cashin wrote:
> On 9/13/05, Chris Johns <cb...@ca...> wrote:
> > OK Ed, I've updated the Changes file (in chronological order, before
> > Yuri's change) to reflect my March 23rd commit.
> > 
> > To answer your other question: no, the database format did not change.
> > It already contains a 'struct stat' and an SHA-1 checksum string. I just
> > use them differently for comparisons.
> 
> Oh, great.  I'll just release the current CVS as 3.05
> as soon as I can.  
> 
> If anybody can do testing on 3.04 before that, it 
> would be very helpful, because fixes
> could then be incorporated into 3.05.
 
Fwiw: 3.0.4 builds cleanly without warnings on both recent FreeBSD 4-stable
and FreeBSD HEAD. This is on i386; I know it doesn't build on sparc64 but I
have no means of testing it on that platform. I guess I could ask the ports
people what the problems are.

-- 
Jos Backus
jos at catnook.com

Re: [Integrit-devel] integrit patches

[Ed L. C. <ec...@gm...>]

On 9/13/05, Chris Johns <cb...@ca...> wrote:
> OK Ed, I've updated the Changes file (in chronological order, before
> Yuri's change) to reflect my March 23rd commit.
>=20
> To answer your other question: no, the database format did not change.
> It already contains a 'struct stat' and an SHA-1 checksum string. I just
> use them differently for comparisons.

Oh, great.  I'll just release the current CVS as 3.05
as soon as I can. =20

If anybody can do testing on 3.04 before that, it=20
would be very helpful, because fixes
could then be incorporated into 3.05.

--=20
  Ed L. Cashin <ec...@no...>

Re: [Integrit-devel] integrit patches

[Chris J. <cb...@ca...>]

OK Ed, I've updated the Changes file (in chronological order, before 
Yuri's change) to reflect my March 23rd commit.

To answer your other question: no, the database format did not change. 
It already contains a 'struct stat' and an SHA-1 checksum string. I just 
use them differently for comparisons.

Chris

Ed L. Cashin wrote:

>On 9/10/05, Yuri D'Elia <wa...@yu...> wrote:
>  
>
>>Following a private conversation with Ed L. Cashin, I'm posting here
>>some patches against integrit (from cvs) for inspection.
>>    
>>
>
>Hey, Chris.  I applied Yuri's patches tonight.  I have 
>been feeling a bit guilty about failing to do a release
>back when you did your changes in March, but now
>I *really* needed to get something out.
>
>I'm about out of time tonight and I notice that the
>changes you made aren't recorded in the Changes
>file.  I'm going to release anyway, but do you think 
>you'd mind updating that file in CVS?
>
>I'm simplifying the version numbering to reflect my
>current light touch administration of the project.  I'm
>dropping the patch number from "3.04.00" and the
>"-beta" that I used to put on releases that hadn't
>been used much.
>
>  
>

Re: [Integrit-devel] integrit patches

[Ed L. C. <ec...@gm...>]

On 9/10/05, Yuri D'Elia <wa...@yu...> wrote:
> Following a private conversation with Ed L. Cashin, I'm posting here
> some patches against integrit (from cvs) for inspection.

Hey, Chris.  I applied Yuri's patches tonight.  I have=20
been feeling a bit guilty about failing to do a release
back when you did your changes in March, but now
I *really* needed to get something out.

I'm about out of time tonight and I notice that the
changes you made aren't recorded in the Changes
file.  I'm going to release anyway, but do you think=20
you'd mind updating that file in CVS?

I'm simplifying the version numbering to reflect my
current light touch administration of the project.  I'm
dropping the patch number from "3.04.00" and the
"-beta" that I used to put on releases that hadn't
been used much.

--=20
  Ed L. Cashin <ec...@no...>

[Integrit-devel] integrit patches

[Yuri D'E. <wa...@yu...>]

Following a private conversation with Ed L. Cashin, I'm posting here  
some patches against integrit (from cvs) for inspection.

In integrit.diff.gz:

- configure.in: Added some checks whether -static (or other flags)  
can be used. Under at least OSX (and possibly open darwin) -static  
cannot be used. This patch fix the build on those systems.
- elcwft.c: reorganized the walk loop. Ignored directories are now  
_really_ ignored (that is, no more "cannot open directory").
- gnupg/md5.c: fixed broken macro for big endian systems under  
certain compilers.
- other fixes: Assume checksums to be unsigned char as required by  
gnupg/* (eliminates a dozen of warnings).

Side note: config.h.in/configure files should be probably removed  
from the cvs.

Thanks

AW: [Integrit-devel] Improvment of integrit

[<C.S...@mo...>]

Hi,

that's exactly what i want to do.
I will try to have a patch by the end of next week.

cs

-----Urspr=FCngliche Nachricht-----
Von: Ed L Cashin [mailto:ec...@no...]
Gesendet: Sonntag, 22. August 2004 20:44
An: Schwabl Christian
Cc: int...@li...
Betreff: Re: [Integrit-devel] Improvment of integrit


Hi.  I think you are suggesting that integrit could benefit from
ignoring before lstat-ing.  That makes sense to me.

If you want to send a patch, that would be nice.  If not, I'll try to
implement it myself.

--=20
--Ed L Cashin     PGP public key: http://noserose.net/e/pgp/

[Integrit-devel] Improvment of integrit

[<C.S...@mo...>]

Hello,

i am using integrit on many machines and found the following problem with
NFS-mountpoints (both on SunOS an HP-UX):
Whenever there is a hung NFS-mountpoint or a mountpoint where root does not
have access, integrit fails
with a message like this:

Warning: cannot open directory (//nfsmount): Permission denied
integrit (main): Error: walk_file_tree: Permission denied

This even happens if the directory/mountpoint has a ! in the
configurationfile:
!/nfsmount

I took a look at the sourcecode and found that the function
walk_file_tree(..) in elcwft.c
checks if a given path is a directory (is_directory(...)).
Afterwards the function process_file(..) in eachfile.c is called (via
callback).
This function looks into the hashtable if this path should be ignored or
not.

This is a bit problematic if NFS-mounts are "forgotten" or so because
is_directory(..) does a lstat(..) on this 
mountpoint which fails.

Would it be possible to take the part which checks if a given path should be
ignored or not and put it in 
a separate function? (maybe ignore_path(..)?)
Then it would be possible to add a call to ignore_path(..) in the function
walk_file_tree(..) right before
is_directory(..).

This would have the effect that hung NFS-mounts are ignored even if they are
not working.
If they should not be ignored an error-messages would still be generated.

Is there any reason against such a change?

If not, where should the call to ignore_path(..) be added?
I would need more time to figure this out, any help/comment would be
appreciated.

regards,
cs