devil-linux-develop — developer discussions

Re: [Devil-linux-develop] systems with low memory

[Heiko Z. <he...@zu...>]

Heiko Zuerker wrote:

> Hey guys,
>
> I was just thinking if we should modify all the tar commands we use in 
> DL (save-config, init system) to work better on low memory systems.
> According to 'man bunzip2' the '-s' parameter should help us, but will 
> at least take twice as long.
>
> Without this modification, DL 1.2 will not run on 32 MB systems.
>
> What do you think? I think we should do it.


I just finished my testing and checked in the changes into the 1.2 tree. 
1.3 will come later, I have to finish first the test compile.

I didn't not add any intelligence to use he '-s' flag only on low memory 
systems, since the performance impact is much less then I initially thought.
The archives are so small, that it doesn't take much longer to unpack 
them. We still can add this if we have to, but I would suggest to wait 
and see.
Biggest impact was the removal of the clamav virus signatures and the 
squidguard blacklists.

DL can now boot again with only 20 MB of RAM !
I'm sure we could bring it down to 16, but that's not the goal of DL. So 
the official minimum will be 32 MB.

cu
Heiko

Re: [Devil-linux-develop] systems with low memory

[Heiko Z. <he...@zu...>]

Bruce Smith wrote:

>>I was just thinking if we should modify all the tar commands we use in 
>>DL (save-config, init system) to work better on low memory systems.
>>According to 'man bunzip2' the '-s' parameter should help us, but will 
>>at least take twice as long.
>>
>>Without this modification, DL 1.2 will not run on 32 MB systems.
>>
>>What do you think? I think we should do it.
>>    
>>
>
>Good idea, but I don't think it'll work.  At least not with floppies.
>
>We'd have to remove the "-j" from tar and run bzip2 separately.  
>"-s" means something completely different to the tar command.
>And that will cause problems because a floppy isn't big enough to 
>hold an uncompressed tar config image.
>  
>
tar -c /etc | bzip2 -s -c > etc.tar.bz2
should work, since the uncompressed version of the tar archive gets 
directly piped into bzip2

>If we do it anyway for USB drives. I agree with Diego that we should
>make it an option to run it either way.  Or maybe better yet, if we
>could detect the amount of memory and make it default to the correct
>method without specifying an option.
>  
>
As I wrote before, we could just always use the lowmemory option when we 
detect a system with less then i.e. 40 MB of total RAM.

>What about busybox?  Would it do us any good?  
>Does it have tar and/or bzip2 commands?
>  
>

It does, but I'm not sure about the memory requirements.

Heiko

Re: [Devil-linux-develop] usb-install

[Heiko Z. <he...@zu...>]

Diego Torres wrote:

>On Mon, Sep 06, 2004 at 01:28:01AM +0200, Diego Torres wrote:
>  
>
>>sh: line 1: mcopy: command not found
>>which is suprising... could anyone try it?
>>    
>>
>
>fixed by installing the mtools package.
>
>next problem:
>
>Installing syslinux MBR
>0+1 registros leídos
>0+1 registros escritos
>304 bytes transferred in 0,025659 seconds (11848 bytes/sec)
>installing SysLinux boot sector
>copying files
>cp: se omite el directorio `./tmp-install2usb/iso-mnt/boot/grub'
>cp: se omite el directorio `./tmp-install2usb/iso-mnt/boot/syslinux'
>cp: fallo al conservar el propietario de `./tmp-install2usb/disk-mnt/LICENSE': Operación no permitida
>copying source file bootcd.iso to device mounted on ./tmp-install2usb/disk-mnt/bootcd.iso
>
>Copying etc.tar.bz2 to ./tmp-install2usb/etc-mnt/etc.tar.bz2
>syncing and unmounting (this could take a while)
>#
>
>this will cause a failure, because we are using cp -p when copying the
>LICENSE file, and the file owners attribute can't be copied to a fat32
>partition.
>
>should i remove the "-p" from install-usb?
>
>----install-usb-----
>echo "copying files"
># Warnings about not copying subdirs cause spurious eixt here so ignore status...
>cp -p $TMPDIR/iso-mnt/boot/* $TMPDIR/disk-mnt/ &> /dev/null # || return 1
>cp -p $TMPDIR/iso-mnt/{DEVIL-LINUX,LICENSE} $TMPDIR/disk-mnt/ &> /dev/null || return 1
>   ^^
>cat $TMPDIR/iso-mnt/isolinux.cfg | sed -e s%"/boot/"%""% > $TMPDIR/disk-mnt/syslinux.cfg || return 1
>cat $TMPDIR/iso-mnt/boot/message | sed -e s%"/boot/"%""% > $TMPDIR/disk-mnt/message || return 1
>copy_iso_file $DLISO $TMPDIR/disk-mnt/bootcd.iso || return 1
>--------------------
>
>  
>
It would probably be a good idea, to make the script a little more 
intelligent and use the -p only when it's not FAT.


Heiko

Re: [Devil-linux-develop] usb-install

[Heiko Z. <he...@zu...>]

Diego Torres wrote:

>i've been compiling the new 1.3.1 from cvs, and i've found a full new
>interactive script to install the iso on the usb storage device. wow! :)
>
>so, couple of questions:
>
>if i configure an alternative partition to hold the configuration
>scripts, what size should i choose?
>  
>
Something between 10 MB and 10 GB ? ;-)
It needs to hold your etc.tar.bz2, so a couple of MB should be enough.

>why are we supporting 3 boot methods? (syslinux, lilo & grub?) shouldn't
>syslinux be enough?
>  
>
syslinux -> FAT
lilo/grub -> EXT2/3

It's just sometimes you need to use a different bootloader for some 
reason, why basically provide you with that.

>why there are two choices between the initrd fs? cramfs/ext2...
>shouldn't we stay with cramfs?
>  
>
The 2.6 Kernel currently can't boot from a cramfs InitRD, that's the 
reason why we have ext2 in there.

>maybe a little advice about the partition type on the configuration
>partition could be a good thing... :)
>
>  
>
???

Heiko

Re: [Devil-linux-develop] Boom?

[Bruce S. <bw...@ar...>]

> >>>>I just checked http://pax.grsecurity.net/ and there's still no patch for 
> >>>>2.4.27 available.
> >>>>You may want to contact the author.
> >>>>
> >>>How long as 2.4.27 been out?  I hate to be a nag too soon.
> >>>
> >>3 weeks or so
> >
> >I see the website has a later 2.4.26 patch that what we are using.
> >That might be worth a try.

I updated to the latest 2.4.26 patch, and still aborts.  
Hopefully they'll have a newer one soon.

 - BS

Re: [Devil-linux-develop] usb-install

[Diego T. <dt...@co...>]

On Mon, Sep 06, 2004 at 01:28:01AM +0200, Diego Torres wrote:
> sh: line 1: mcopy: command not found
> which is suprising... could anyone try it?

fixed by installing the mtools package.

next problem:

Installing syslinux MBR
0+1 registros leídos
0+1 registros escritos
304 bytes transferred in 0,025659 seconds (11848 bytes/sec)
installing SysLinux boot sector
copying files
cp: se omite el directorio `./tmp-install2usb/iso-mnt/boot/grub'
cp: se omite el directorio `./tmp-install2usb/iso-mnt/boot/syslinux'
cp: fallo al conservar el propietario de `./tmp-install2usb/disk-mnt/LICENSE': Operación no permitida
copying source file bootcd.iso to device mounted on ./tmp-install2usb/disk-mnt/bootcd.iso

Copying etc.tar.bz2 to ./tmp-install2usb/etc-mnt/etc.tar.bz2
syncing and unmounting (this could take a while)
#

this will cause a failure, because we are using cp -p when copying the
LICENSE file, and the file owners attribute can't be copied to a fat32
partition.

should i remove the "-p" from install-usb?

----install-usb-----
echo "copying files"
# Warnings about not copying subdirs cause spurious eixt here so ignore status...
cp -p $TMPDIR/iso-mnt/boot/* $TMPDIR/disk-mnt/ &> /dev/null # || return 1
cp -p $TMPDIR/iso-mnt/{DEVIL-LINUX,LICENSE} $TMPDIR/disk-mnt/ &> /dev/null || return 1
   ^^
cat $TMPDIR/iso-mnt/isolinux.cfg | sed -e s%"/boot/"%""% > $TMPDIR/disk-mnt/syslinux.cfg || return 1
cat $TMPDIR/iso-mnt/boot/message | sed -e s%"/boot/"%""% > $TMPDIR/disk-mnt/message || return 1
copy_iso_file $DLISO $TMPDIR/disk-mnt/bootcd.iso || return 1
--------------------

-- 
-- gnupg keyfingerprint -- 48AF 5BF9 8F54 2966 64CC  2327 7CD0 DD91 B09D 5799
-- Use of a keyboard or mouse may be linked to serious injuries or disorders.
Diego Torres - dtorres at anthalia dot org - Madrid / España

Re: [Devil-linux-develop] usb-install

[Diego T. <dt...@co...>]

On Mon, Sep 06, 2004 at 12:54:50AM +0200, Diego Torres wrote:

when using install-usb with syslinux, on these lines...

---snip snip snip---
       if [ "$BOOTLOADER" = "syslinux" ]; then
           mkdosfs $PARTITION > /dev/null || return 1
           echo "Installing syslinux MBR"
           SIZE=$(du -b $TMPDIR/iso-mnt/boot/syslinux/mbr.bin | cut -f 1 )
           dd if=$TMPDIR/iso-mnt/boot/syslinux/mbr.bin of=$DEVICE count=$SIZE || return 1
           echo installing SysLinux boot sector
           $TMPDIR/iso-mnt/boot/syslinux/syslinux $PARTITION || return 1
--------------------

or just by executing the followin:
./syslinux /dev/scsi/host0/bus0/target0/lun0/part1

i get the same error:

sh: line 1: mcopy: command not found

which is suprising... could anyone try it?
												       
-- 
-- gnupg keyfingerprint -- 48AF 5BF9 8F54 2966 64CC  2327 7CD0 DD91 B09D 5799
-- Use of a keyboard or mouse may be linked to serious injuries or disorders.
Diego Torres - dtorres at anthalia dot org - Madrid / España

[Devil-linux-develop] usb-install

[Diego T. <dt...@co...>]

i've been compiling the new 1.3.1 from cvs, and i've found a full new
interactive script to install the iso on the usb storage device. wow! :)

so, couple of questions:

if i configure an alternative partition to hold the configuration
scripts, what size should i choose?

why are we supporting 3 boot methods? (syslinux, lilo & grub?) shouldn't
syslinux be enough?

why there are two choices between the initrd fs? cramfs/ext2...
shouldn't we stay with cramfs?

maybe a little advice about the partition type on the configuration
partition could be a good thing... :)

-- 
-- gnupg keyfingerprint -- 48AF 5BF9 8F54 2966 64CC  2327 7CD0 DD91 B09D 5799
-- Use of a keyboard or mouse may be linked to serious injuries or disorders.
Diego Torres - dtorres at anthalia dot org - Madrid / España

Re: [Devil-linux-develop] systems with low memory

[Bruce S. <bw...@ar...>]

> I was just thinking if we should modify all the tar commands we use in 
> DL (save-config, init system) to work better on low memory systems.
> According to 'man bunzip2' the '-s' parameter should help us, but will 
> at least take twice as long.
> 
> Without this modification, DL 1.2 will not run on 32 MB systems.
> 
> What do you think? I think we should do it.

Good idea, but I don't think it'll work.  At least not with floppies.

We'd have to remove the "-j" from tar and run bzip2 separately.  
"-s" means something completely different to the tar command.
And that will cause problems because a floppy isn't big enough to 
hold an uncompressed tar config image.

If we do it anyway for USB drives. I agree with Diego that we should
make it an option to run it either way.  Or maybe better yet, if we
could detect the amount of memory and make it default to the correct
method without specifying an option.

What about busybox?  Would it do us any good?  
Does it have tar and/or bzip2 commands?

 - BS

Re: [Devil-linux-develop] systems with low memory

[Heiko Z. <he...@zu...>]

Diego Torres wrote:

>On Sun, Sep 05, 2004 at 10:14:34AM -0400, Heiko Zuerker wrote:
>
>  
>
>>Without this modification, DL 1.2 will not run on 32 MB systems.
>>
>>What do you think? I think we should do it.
>>    
>>
>
>is this change fixing also dl 1.3 on 32mb? if so... could you make it configurable?
>
>  
>
1.2 has a problem too on 32 MB systems.

I'm currently doing some tests, the '-s' flag is really much slower, but 
alone doesn't fix the problem.
I was thinking about checking if the system has less then 64MB and then 
automatically use the '-s' flag.

Additionally I'm right now running a test without the clamav virus 
signatures and without the squidguard blacklist.
I probably remove those completely, since they're always outdated 
anyway. If somebody uses either clamav or squidguard, the files have to 
be replace anyway.

I keep you updated.

cu
Heiko

Re: [Devil-linux-develop] systems with low memory

[Diego T. <dt...@co...>]

On Sun, Sep 05, 2004 at 10:14:34AM -0400, Heiko Zuerker wrote:

> Without this modification, DL 1.2 will not run on 32 MB systems.
> 
> What do you think? I think we should do it.

is this change fixing also dl 1.3 on 32mb? if so... could you make it configurable?

-- 
-- gnupg keyfingerprint -- 48AF 5BF9 8F54 2966 64CC  2327 7CD0 DD91 B09D 5799
-- Use of a keyboard or mouse may be linked to serious injuries or disorders.
Diego Torres - dtorres at anthalia dot org - Madrid / España

Re: [Devil-linux-develop] build iso creates endless amount of processes

[Michael B. <Mic...@ta...>]

just installing suse 9.1 to try again

On Sun, 2004-09-05 at 15:46, Heiko Zuerker wrote:
> Michael Bielicki wrote:
> 
> >On Fri, 2004-08-27 at 22:46, Heiko Zuerker wrote:
> >  
> >
> >>Michael Bielicki wrote:
> >>
> >>    
> >>
> >>>Whiteboxlinux (RHAS 3.0 clone)
> >>> 
> >>>
> >>>      
> >>>
> >>Not sure about it, does it have any kind of special kernel level 
> >>security checks?
> >>    
> >>
> >Probably. I will setup a redhat 7.2 just for the task of compiling devil
> >:) Since we will use it for our PBX systems that we want to sell from
> >next week based on devil and asterisk I need to get it done :)
> >Thx for the nice system, the testing I did was really promising.
> >  
> >
> 
> Is it working now?
> 
> Heiko
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by BEA Weblogic Workshop
> FREE Java Enterprise J2EE developer tools!
> Get your free copy of BEA WebLogic Workshop 8.1 today.
> http://ads.osdn.com/?ad_id=5047&alloc_id=10808&op=click
> _______________________________________________
> Devil-linux-develop mailing list
> Dev...@li...
> https://lists.sourceforge.net/lists/listinfo/devil-linux-develop

[Devil-linux-develop] systems with low memory

[Heiko Z. <he...@zu...>]

Hey guys,

I was just thinking if we should modify all the tar commands we use in 
DL (save-config, init system) to work better on low memory systems.
According to 'man bunzip2' the '-s' parameter should help us, but will 
at least take twice as long.

Without this modification, DL 1.2 will not run on 32 MB systems.

What do you think? I think we should do it.

cu
Heiko

Re: Fw: [Devil-linux-develop] Logfiles from failed Compile

[Heiko Z. <he...@zu...>]

Heiko Zuerker wrote:

> Hey,
>
>> I am willing to use some other distribution also, which one do you 
>> suggest ?
>>
>> I am attaching some info on exec shield.
>>
>> Let me know which path do we take from here on :
>>
>> 1. Different distrubution as per ur suggestion
>>  
>>
> I know that all the SuSE version work fine with DL.
>
>> 2. Tweaking Redhat 9.0 for DEVIL.
>>  
>>
> Let's try this first, since it's easier:
> --- snip ---
>
>> There is a new boot-time kernel command line option called exec-shield=,
>> which has 4 values. Each value represents a different level of security:
>>
>> exec-shield=0    - always-disabled
>> exec-shield=1    - default disabled, except binaries that enable it
>> exec-shield=2    - default enabled, except binaries that disable it
>> exec-shield=3    - always-enabled
>>
>> the current patch defaults to 'exec-shield=2'. The security level can 
>> also
>> be changed runtime, by writing the level into /proc:
>>
>>  echo 0 > /proc/sys/kernel/exec-shield
>>
> --- snap ---
>
> Disable it via one of those methods and try again.
> But please start from scratch ( nwe lfssystem, make mrproper, etc.), 
> since we don't know if the rest of the system is OK.


Is it working now?

Heiko

Re: [Devil-linux-develop] build iso creates endless amount of processes

[Heiko Z. <he...@zu...>]

Michael Bielicki wrote:

>On Fri, 2004-08-27 at 22:46, Heiko Zuerker wrote:
>  
>
>>Michael Bielicki wrote:
>>
>>    
>>
>>>Whiteboxlinux (RHAS 3.0 clone)
>>> 
>>>
>>>      
>>>
>>Not sure about it, does it have any kind of special kernel level 
>>security checks?
>>    
>>
>Probably. I will setup a redhat 7.2 just for the task of compiling devil
>:) Since we will use it for our PBX systems that we want to sell from
>next week based on devil and asterisk I need to get it done :)
>Thx for the nice system, the testing I did was really promising.
>  
>

Is it working now?

Heiko

Re: [Devil-linux-develop] Boom?

[Heiko Z. <he...@zu...>]

Bruce Smith wrote:

>>>>I just checked http://pax.grsecurity.net/ and there's still no patch for 
>>>>2.4.27 available.
>>>>You may want to contact the author.
>>>>        
>>>>
>>>How long as 2.4.27 been out?  I hate to be a nag too soon.
>>>      
>>>
>>3 weeks or so
>>    
>>
>
>I see the website has a later 2.4.26 patch that what we are using.
>That might be worth a try.
>
>Also, I never looked at the pax page before, and it says 
>"you'll need the binutils patch and paxctl to make use of it".
>
>I don't see where we have either, so the PAX patch isn't doing us any
>good anyway?
>  
>

We're still using the old ET_DYN method, so we should be good.

I'm currently preparing to go with the newer 'pie' solution, but for 
that we need the new glibc, for that we need a new lfssystem and when we 
do that we can also switch to nptl, .....

Heiko

Re: [Devil-linux-develop] Boom?

[Bruce S. <bw...@ar...>]

> >>I just checked http://pax.grsecurity.net/ and there's still no patch for 
> >>2.4.27 available.
> >>You may want to contact the author.
> >
> >How long as 2.4.27 been out?  I hate to be a nag too soon.
>
> 3 weeks or so

I see the website has a later 2.4.26 patch that what we are using.
That might be worth a try.

Also, I never looked at the pax page before, and it says 
"you'll need the binutils patch and paxctl to make use of it".

I don't see where we have either, so the PAX patch isn't doing us any
good anyway?

 - BS

Re: [Devil-linux-develop] Boom?

[Heiko Z. <he...@zu...>]

Bruce Smith wrote:

>>>Yup, turning off PAX "fixed" the problem ...
>>>      
>>>
>>  
>>I just checked http://pax.grsecurity.net/ and there's still no patch for 
>>2.4.27 available.
>>You may want to contact the author.
>>    
>>
>
>How long as 2.4.27 been out?  I hate to be a nag too soon.
>  
>

3 weeks or so

Heiko

Re: [Devil-linux-develop] Boom?

[Bruce S. <bw...@ar...>]

> >Yup, turning off PAX "fixed" the problem ...
>   
> I just checked http://pax.grsecurity.net/ and there's still no patch for 
> 2.4.27 available.
> You may want to contact the author.

How long as 2.4.27 been out?  I hate to be a nag too soon.

 - BS

Re: [Devil-linux-develop] Boom?

[Bruce S. <bw...@ar...>]

> >BTW, the prepare build process could use a few more echos to make it
> >easier to tell where it blows up. :)
>
> You got CVS write access..... ;-)

Oh yeah ...  :-)

(done)

 - BS

Re: [Devil-linux-develop] Boom?

[Heiko Z. <he...@zu...>]

Bruce Smith wrote:

>>>>>OK, I'll try to reproduce the problem.
>>>>>          
>>>>>
>>>>FWIW, the problem seems to be in:  linux-2.4.26-i2c-2.8.4.patch
>>>>        
>>>>
>>>It applied just fine....
>>>      
>>>
>>After some more looking, I think the problem is in the PAX patch.
>>src/pax-linux-2.4.26-200404151725.patch.bz2
>>
>>I'll know in a few minutes since I just turned off PAX ...
>>
>>BTW, the prepare build process could use a few more echos to make it
>>easier to tell where it blows up. :)
>>    
>>
>
>Yup, turning off PAX "fixed" the problem ...
>  
>

I just checked http://pax.grsecurity.net/ and there's still no patch for 
2.4.27 available.
You may want to contact the author.

Heiko

Re: [Devil-linux-develop] Boom?

[Heiko Z. <he...@zu...>]

Bruce Smith wrote:

>>>>OK, I'll try to reproduce the problem.
>>>>        
>>>>
>>>FWIW, the problem seems to be in:  linux-2.4.26-i2c-2.8.4.patch
>>>      
>>>
>>It applied just fine....
>>    
>>
>
>After some more looking, I think the problem is in the PAX patch.
>src/pax-linux-2.4.26-200404151725.patch.bz2
>
>I'll know in a few minutes since I just turned off PAX ...
>
>BTW, the prepare build process could use a few more echos to make it
>easier to tell where it blows up. :)
>  
>

You got CVS write access..... ;-)

Heiko

Re: [Devil-linux-develop] Boom?

[Bruce S. <bw...@ar...>]

> > >>OK, I'll try to reproduce the problem.
> > >
> > >FWIW, the problem seems to be in:  linux-2.4.26-i2c-2.8.4.patch
> >
> > It applied just fine....
> 
> After some more looking, I think the problem is in the PAX patch.
> src/pax-linux-2.4.26-200404151725.patch.bz2
> 
> I'll know in a few minutes since I just turned off PAX ...
> 
> BTW, the prepare build process could use a few more echos to make it
> easier to tell where it blows up. :)

Yup, turning off PAX "fixed" the problem ...

 - BS

Re: [Devil-linux-develop] Boom?

[Bruce S. <bw...@ar...>]

> >>OK, I'll try to reproduce the problem.
> >
> >FWIW, the problem seems to be in:  linux-2.4.26-i2c-2.8.4.patch
>
> It applied just fine....

After some more looking, I think the problem is in the PAX patch.
src/pax-linux-2.4.26-200404151725.patch.bz2

I'll know in a few minutes since I just turned off PAX ...

BTW, the prepare build process could use a few more echos to make it
easier to tell where it blows up. :)

 - BS

Re: [Devil-linux-develop] Boom?

[Heiko Z. <he...@zu...>]

Bruce Smith wrote:

>>>>>I started a fresh compile last night.  Updated CVS & source, new LFS,
>>>>>mrproper, and it aborted in prepare:
>>>>>
>>>>>/data/build/tmp/LOGS/build/prepare:
>>>>>
>>>>>applying patch: /data/build/tmp/kernel-patches-2.4/IMQ-01-kernel-2.4.22-through-to-2.4.25-pre6.diff
>>>>>patching file Documentation/Configure.help
>>>>>...
>>>>>patching file include/linux/sysctl.h
>>>>>Hunk #1 FAILED at 128.
>>>>>1 out of 1 hunk FAILED -- saving rejects to file include/linux/sysctl.h.rej
>>>>>patching file kernel/sysctl.c
>>>>>Hunk #1 succeeded at 128 (offset 1 line).
>>>>>...
>>>>>     
>>>>>
>>>>>          
>>>>>
>>>>I did this a couple times lately and it worked fine....
>>>>You're using 1.3, correct?
>>>>   
>>>>
>>>>        
>>>>
>>>Yes, 1.3.  I just did a new LFS, mrproper, and it aborted again in the
>>>same place.
>>> 
>>>
>>>      
>>>
>>OK, I'll try to reproduce the problem.
>>    
>>
>
>FWIW, the problem seems to be in:  linux-2.4.26-i2c-2.8.4.patch
>  
>

It applied just fine....

I attached my config file.

cu
  Heiko