syllable-kernel — The list is for discussion of Kernel issues by developers

Re: [Syllable-kernel] IO operations

[Daniel G. <da...@fp...>]

On Tue, 2004-12-14 at 19:11 -0800, Nathanael Bettridge wrote:

> IOPL is a chiplevel function. It has no idea what root
> even is. It sees only the three rings, and in flat
> paging mode, AFAIK (and definitely in current
> syllable), only ring 0 (kernel) and 3 (user) are
> really relevant. In other words, if its allowed in
> user mode, it can be done, regardless of the user
> account. It might not be easy to get at via libc or
> whatever, but it can still be done. To do things
> properly, we'd have to either shunt hw-accessing code
> to ring 1 or 2, or have it all done via syscalls.
> 
> Disclaimer: of course, I could be wrong ;)

No, it's still easy.  IOPL is included in the saved context state when
transitioning from 3 -> 0 (ie on a syscall or interrupt).  Thus, it's
restored during the 0 -> 3 transition (ie, on iret).  Since Syllable,
like all Unix kernels, stores this state on a per-process basis, it's
easy to enable/disable IOPL on a per-process (or per-user, like most
Unix does) basis.  Obvsiously, Syllable's not currently doing this, but
just allowing all processes to have IOPL enabled.  This doesn't mean it
couldn't be done.

Most likely, we want to enable IOPL on the appserver, and deny it to
anything else, by default.

Linux does this based on it's fine-grained Capabilities.

Daniel

Re: [Syllable-kernel] IO operations

[Nathanael B. <nbe...@ya...>]

--- Daniel Gryniewicz <da...@fp...> wrote:

> On Tue, 2004-12-14 at 19:52 +0100, Jacek wrote:
> > Daniel wrote:
> > > On Tue, 2004-12-14 at 18:21 +0100, Jacek wrote:
> > >> 1. all kernel and application code currently
> runs on privilege level 0
> > >
> > >Nope.  User code runs at level 3.
> > 
> > That's great for me, half of work is done.
> > 
> > > I believe iopl is allowed (optionally) to root. 
> Anything not run as
> > > root does not have iopl access.
> > 
> > Unfortunately you're not right. You can make a
> simple test:
> > 
> > // this will disable your PS/2 mouse
> > int main() {
> >         asm("mov $0xA7, %al;out %al, $0x64");
> > }
> > 
> > sending 0xA8 to the same port will enable your
> mouse back
> > 
> > And the following code will reset your computer
> immediately (run it at your 
> > own risk)
> > int main() {
> >         asm("mov $0xFE, %al;out %al, $0x64");
> > }
> > 
> > What about other questions?
> 
> Yes, but unless you made a new account to log in as,
> you are running as
> root, and therefore have iopl.
> 
> The Appserver, being started by init (which runs as
> root) will always
> have iopl permission.
> 
> I didn't see any other questions in the original
> email?
> 
> Daniel
IOPL is a chiplevel function. It has no idea what root
even is. It sees only the three rings, and in flat
paging mode, AFAIK (and definitely in current
syllable), only ring 0 (kernel) and 3 (user) are
really relevant. In other words, if its allowed in
user mode, it can be done, regardless of the user
account. It might not be easy to get at via libc or
whatever, but it can still be done. To do things
properly, we'd have to either shunt hw-accessing code
to ring 1 or 2, or have it all done via syscalls.

Disclaimer: of course, I could be wrong ;)

Nathanael Bettridge


		
__________________________________ 
Do you Yahoo!? 
Yahoo! Mail - Helps protect you from nasty viruses. 
http://promotions.yahoo.com/new_mail

Re: [Syllable-kernel] IO operations

[Jacek <xo...@o2...>]

Daniel wrote:
> Yes, but unless you made a new account to log in as, you are running as
> root, and therefore have iopl.
You may try it even on the guest account, it will work.

Re: [Syllable-kernel] IO operations

[Daniel G. <da...@fp...>]

On Tue, 2004-12-14 at 19:52 +0100, Jacek wrote:
> Daniel wrote:
> > On Tue, 2004-12-14 at 18:21 +0100, Jacek wrote:
> >> 1. all kernel and application code currently runs on privilege level 0
> >
> >Nope.  User code runs at level 3.
> 
> That's great for me, half of work is done.
> 
> > I believe iopl is allowed (optionally) to root.  Anything not run as
> > root does not have iopl access.
> 
> Unfortunately you're not right. You can make a simple test:
> 
> // this will disable your PS/2 mouse
> int main() {
>         asm("mov $0xA7, %al;out %al, $0x64");
> }
> 
> sending 0xA8 to the same port will enable your mouse back
> 
> And the following code will reset your computer immediately (run it at your 
> own risk)
> int main() {
>         asm("mov $0xFE, %al;out %al, $0x64");
> }
> 
> What about other questions?

Yes, but unless you made a new account to log in as, you are running as
root, and therefore have iopl.

The Appserver, being started by init (which runs as root) will always
have iopl permission.

I didn't see any other questions in the original email?

Daniel

Re: [Syllable-kernel] IO operations

[Jacek <xo...@o2...>]

Daniel wrote:
> On Tue, 2004-12-14 at 18:21 +0100, Jacek wrote:
>> 1. all kernel and application code currently runs on privilege level 0
>
>Nope.  User code runs at level 3.

That's great for me, half of work is done.

> I believe iopl is allowed (optionally) to root.  Anything not run as
> root does not have iopl access.

Unfortunately you're not right. You can make a simple test:

// this will disable your PS/2 mouse
int main() {
        asm("mov $0xA7, %al;out %al, $0x64");
}

sending 0xA8 to the same port will enable your mouse back

And the following code will reset your computer immediately (run it at your 
own risk)
int main() {
        asm("mov $0xFE, %al;out %al, $0x64");
}

What about other questions?

Re: [Syllable-kernel] IO operations

[Daniel G. <da...@fp...>]

On Tue, 2004-12-14 at 18:21 +0100, Jacek wrote:
> Hello,
> I'm writing here for the firs time. Sorry in advance if I'll make any
> mistakes with my english. Please ignore them or you can tell me about them,
> I would be grateful :)
> 
> I've been thinking about disallowing applications to make input/output
> operations on processor's ports.
> I've been looking at the kernel code and here is what I noticed, please tell
> me if I'm right:
> 1. all kernel and application code currently runs on privilege level 0

Nope.  User code runs at level 3.

> 2. syscalls are called via int 0x80
> 3. there are different addres spaces for kernel and user code (different
> segment selectors)
> 4. appserver is also user program
> 5. appserver's drivers also run as user code and perform direct port
> operations
> 
> The easiest way would be to dissallow all user applications making IO
> operations by changing their privilege level to ie 3 and setting the
> processor's IOPL in EFLAGS to ie 1. There is a problem if appservers uses
> direct IO operations. If so either the appsever could run on different IOPL
> or (better I think) the I/O Permission Bit Map could be used. There could be
> even only one bit map for the appserver and all his drivers.
> For now it could allow opperations on all ports. Later there could be
> implemented some functions that would allow registering port ranges that
> driver wants to use, etc.

I believe iopl is allowed (optionally) to root.  Anything not run as
root does not have iopl access.

Daniel

[Syllable-kernel] IO operations

[Jacek <xo...@o2...>]

Hello,
I'm writing here for the firs time. Sorry in advance if I'll make any
mistakes with my english. Please ignore them or you can tell me about them,
I would be grateful :)

I've been thinking about disallowing applications to make input/output
operations on processor's ports.
I've been looking at the kernel code and here is what I noticed, please tell
me if I'm right:
1. all kernel and application code currently runs on privilege level 0
2. syscalls are called via int 0x80
3. there are different addres spaces for kernel and user code (different
segment selectors)
4. appserver is also user program
5. appserver's drivers also run as user code and perform direct port
operations

The easiest way would be to dissallow all user applications making IO
operations by changing their privilege level to ie 3 and setting the
processor's IOPL in EFLAGS to ie 1. There is a problem if appservers uses
direct IO operations. If so either the appsever could run on different IOPL
or (better I think) the I/O Permission Bit Map could be used. There could be
even only one bit map for the appserver and all his drivers.
For now it could allow opperations on all ports. Later there could be
implemented some functions that would allow registering port ranges that
driver wants to use, etc.

What do you think about all this?

Regards
Jacek Kutzmann

PS: If there are any technical documents about Syllable kernel I'd also be
glad if someone told me about them.

[Syllable-kernel] Glibc 2.3.3 source now up

[Kristian V. <kri...@xe...>]

I've uploaded the source for Glibc 2.3.3 to match the binaries
I released last Friday.  Get it from
http://www.liqwyd.com/glibc/glibc-2.3.3-beta1.src.tar.gz  You'll
also need the latest CVS headers for Syllable and
http://www.liqwyd.com/glibc/includes-011104.tgz

How to build it:

o Install Gcc 3.x, Binutils 2.15 and Gawk 3.x
o Untar the source; this will create a directory called "libc"
o Alongside this directory create two more empty directories;
  libc-build and libc-config
o cd into libc-config and run:

  $ ../libc/configure --prefix=/path/to/libc-build
    --enable-hacker-mode

o Once configure has completed:

  $ make

o The build will fail at the point where make will complain that it
  doesn't know how to build "syscall-stubs.o"  I don't know what
  causes this; just run make again and it'll carry on.

o It should build all the way through without a problem, but I
  have noticed that AFS seems to have some problems under very high
  loads E.g. when building Glibc.  Occasionally some files may
  appear to be corrupt and the build will stop.  The copy I built
  this morning is always stopping at libcrypt and complaing about
  undefined symbols, but I know the same source built fine
  previously. let me know if you have any *persistent* problems
  like this.

o Eventually make should exit successfully.  Now run:

  $ make install

  This will place all the files in the libc-build directory.

I'd advise that you install the Glibc 2.3.3 binaries I released on
Friday using the install.sh script and then simply copy the new
libc-2.3.3.so file into /system/libs to overwrite the existing file.
Don't simply copy all of the DSO's that Glibc builds into /system/libs
because you *WILL* break your system.  If you did that, re-install.
Note that if you make changes and copy the new library into place
the old library may be cached so you'll have to reboot to get the
new library to be used.

The API SHOULD be complete; that is, all of the Syllable specific
functions and syscalls should be available and should work as
expected.  The exception to this are stricmp() and strnicmp(); Kurt
added these but I have left them out.  If they're required I'll
implement them.  Let me know if you get any undefined symbols or
unexpected behavour.

C++ isn't working due to some header clashes.  I still need to
solve this one.

I recommend at least 256Mb of memory and 250Mb of disk space,
running sync as often as you can to insure against a crash.  My
development box here is a puny P3 450 and it takes about 2 hours to
do a full build there.  I wouldn't dream of trying on anything
slower!

Be aware that the tarball posted is a CVS working copy with
changes; you can run "cvs update" to pull down the latest changes
in the Glibc CVS server but this MAY cause the build to break.
Breakage is not usually serious but even I've been left
scratching my head at some of them in the past so be warned;
changes can be obscure.

-- 
Vanders
http://www.syllable.org
http://www.liqwyd.com

_____________________________________________________________________
This message has been checked for all known viruses by Xenicom delivered through the MessageLabs Virus Control Centre. 

Re: [Syllable-kernel] Special handling for >1GB RAM and loading apps at 0x04000000 not 0x80000000?

[Jake H. <jh...@an...>]

Kristian Van Der Vliet wrote:

> I think Jake is on the right track, provided we get the issues correctly 
> solved.  We have to be realistic and foreign binary support will be an issue 
> once we hit a certain point.  There is no sense in limiting ourselves.
> 
> Getting create_area() shouldn't be too tough, nor should fixing the device 
> drivers.  It might be a good idea to add a MEMF_DMA flag to kmalloc() that in 
> turn uses AREA_CONTIGOUS, so at least we don't need to make big changes to 
> the existing code base.  Also note that nothing should be calling 
> get_free_pages() directly; the emu10k1 driver hacks it up but shouldn't be 
> doing it.  Henrik used it to get a page aligned area but it should really be 
> using the PAGE_ALIGN() macro..
> 
> We already knew that Syllable wouldn't boot with systems of 2Gb or more, 
> exactly because of that 1:1 kernel memory mapping.  For that reason alone 
> it's good enough to move everything around and load the kernel at 0xc0000000 
> (Could there possibly be a good reason for using more than 1Gb of virtual 
> address space in a desktop OS kernel?  None I can think of).  At todays 
> growth rate 2Gb+ machines as standard are not far off.  There is also some 
> good news; current applications are compiled with -fPIC by default.  
> Theoretically we don't even need any special case handling for old binaries; 
> just load them at 0x08000000.

Cool, now I know what has to be done.  Unfortunately, something else has 
broken since I upgraded my RAM: the vm86 code to call BIOS routines.  At 
least three BIOS calls are now failing for me:


1) the e820 memory map retrieved in init_e820_memory_map() is giving an 
"ERROR : unknown v86 16 bit instruction 7c"

2) the ps2mouse probing gives an "unknown v86 16 bit instruction 2"

3) the APM power-off code has also stopped working.


If anyone is experiencing similar problems after building a recent 
kernel, please let me know.  I hope this isn't something that I broke 
somehow with one of my changes.  I've added some printk() statements so 
you should now see a dump of the BIOS memory map if you search 
/var/log/kernel for "BIOS-e820", and each BIOS call will be traced with 
a line like "sys_realint(21) -> f000:6ac8".

-Jake

Re: [Syllable-kernel] Special handling for >1GB RAM and loading apps at 0x04000000 not 0x80000000?

[Kristian V. D. V. <va...@li...>]

On Saturday 13 November 2004 8:53 am, Arno Klenke wrote:
> Jake Hamby schrieb:
>> After upgrading my laptop to 1.25 GB of RAM (for Windows XP of course,
>> not for Syllable!), I have to force Syllable to use a memsize lower
>> than 1GB in the bootloader or else I get a bunch of fatal areas
>> errors.  I am trying to figure out what needs to be done to fix
>> Syllable to work on machines with up to 4GB of RAM.  Does anyone know
>> if there is any special quirkiness related to Intel-based PC handling
>> of 1GB to 4GB RAM, and if so, how to handle it?
>>
>> <snip>
>>
>> Since we have been talking about glibc and ELF issues this week, I've
>> been thinking that we should consider moving the default application
>> start address from 0x80000000 to the 0x00000000-0x7fffffff range that
>> is used by every other OS I have worked with recently...
>
> There are some problems with this:
>
> Syllable currently maps all the present physical memory at the same
> position in the virtual space. This means in the kernel you have a 1:1
> mapping. This behaviour limits the ram to 2GB but is required by many
> device drivers that use kmalloc() to allocate dma buffers because
> create_area() does not currently have a way to allocate a linear buffer
> of memory. There is a flag AREA_CONTIGUOUS but no implementation for
> this. If we changed all device drivers to use create_area() with this
> flag (and use get_area_physical() to get the physical address) instead
> of calls to kmalloc() or get_free_page() and also change kmalloc() to
> use areas then we could follow your plan.

I think Jake is on the right track, provided we get the issues correctly 
solved.  We have to be realistic and foreign binary support will be an issue 
once we hit a certain point.  There is no sense in limiting ourselves.

Getting create_area() shouldn't be too tough, nor should fixing the device 
drivers.  It might be a good idea to add a MEMF_DMA flag to kmalloc() that in 
turn uses AREA_CONTIGOUS, so at least we don't need to make big changes to 
the existing code base.  Also note that nothing should be calling 
get_free_pages() directly; the emu10k1 driver hacks it up but shouldn't be 
doing it.  Henrik used it to get a page aligned area but it should really be 
using the PAGE_ALIGN() macro..

We already knew that Syllable wouldn't boot with systems of 2Gb or more, 
exactly because of that 1:1 kernel memory mapping.  For that reason alone 
it's good enough to move everything around and load the kernel at 0xc0000000 
(Could there possibly be a good reason for using more than 1Gb of virtual 
address space in a desktop OS kernel?  None I can think of).  At todays 
growth rate 2Gb+ machines as standard are not far off.  There is also some 
good news; current applications are compiled with -fPIC by default.  
Theoretically we don't even need any special case handling for old binaries; 
just load them at 0x08000000.

-- 
Vanders
http://syllable.sourceforge.net/
http://www.liqwyd.com

Re: [Syllable-kernel] Special handling for >1GB RAM and loading apps at 0x04000000 not 0x80000000?

[Arno K. <arn...@ya...>]

Jake Hamby schrieb:

> After upgrading my laptop to 1.25 GB of RAM (for Windows XP of course, 
> not for Syllable!), I have to force Syllable to use a memsize lower 
> than 1GB in the bootloader or else I get a bunch of fatal areas 
> errors.  I am trying to figure out what needs to be done to fix 
> Syllable to work on machines with up to 4GB of RAM.  Does anyone know 
> if there is any special quirkiness related to Intel-based PC handling 
> of 1GB to 4GB RAM, and if so, how to handle it?
>
> One reason I ask is because I recently discovered that Linux's default 
> kernel only supports up to 896MB of physical memory.  You need a 
> special HIGHMEM enabled kernel to go up to 4GB (or 64GB with 36-bit 
> addressing) of RAM.  I've never had that much RAM before so I never 
> noticed.  :-)  I guess I will have to grep through the kernel and see 
> what had to be changed in Linux to support large memory systems.  I 
> had assumed that it would be a simple matter of not accidentally using 
> signed variables for pointer arithmetic and taking care to stuff the 
> addresses into the page table entries in the correct quirky Intel 
> format that the processor can understand.  Apparently it isn't that 
> simple.
>
> Since we have been talking about glibc and ELF issues this week, I've 
> been thinking that we should consider moving the default application 
> start address from 0x80000000 to the 0x00000000-0x7fffffff range that 
> is used by every other OS I have worked with recently... 

There are some problems with this:

Syllable currently maps all the present physical memory at the same 
position in the virtual space. This means in the kernel you have a 1:1 
mapping. This behaviour limits the ram to 2GB but is required by many 
device drivers that use kmalloc() to allocate dma buffers because 
create_area() does not currently have a way to allocate a linear buffer 
of memory. There is a flag AREA_CONTIGUOUS but no implementation for 
this. If we changed all device drivers to use create_area() with this 
flag (and use get_area_physical() to get the physical address) instead 
of calls to kmalloc() or get_free_page() and also change kmalloc() to 
use areas then we could follow your plan.

Arno

Re: [Syllable-kernel] Special handling for >1GB RAM and loading apps at 0x04000000 not 0x80000000?

[William H. <me...@wh...>]

On Saturday 13 Nov 2004 05:34, Jake Hamby wrote:
> After upgrading my laptop to 1.25 GB of RAM (for Windows XP of course,
> not for Syllable!), I have to force Syllable to use a memsize lower than
> 1GB in the bootloader or else I get a bunch of fatal areas errors.  I am
> trying to figure out what needs to be done to fix Syllable to work on
> machines with up to 4GB of RAM.

I have a machine with 1.25GB RAM. Syllable boots and runs fine without any 
special tricks. According to the Sylllable manager all 1.25 GB is correctly 
detected.

I'm not familiar enough with the technical issues to help much, but I can help 
if you need someone to test 1GB+ stuff.

Will H

[Syllable-kernel] Special handling for >1GB RAM and loading apps at 0x04000000 not 0x80000000?

[Jake H. <jh...@an...>]

After upgrading my laptop to 1.25 GB of RAM (for Windows XP of course, 
not for Syllable!), I have to force Syllable to use a memsize lower than 
1GB in the bootloader or else I get a bunch of fatal areas errors.  I am 
trying to figure out what needs to be done to fix Syllable to work on 
machines with up to 4GB of RAM.  Does anyone know if there is any 
special quirkiness related to Intel-based PC handling of 1GB to 4GB RAM, 
and if so, how to handle it?

One reason I ask is because I recently discovered that Linux's default 
kernel only supports up to 896MB of physical memory.  You need a special 
HIGHMEM enabled kernel to go up to 4GB (or 64GB with 36-bit addressing) 
of RAM.  I've never had that much RAM before so I never noticed.  :-)  I 
guess I will have to grep through the kernel and see what had to be 
changed in Linux to support large memory systems.  I had assumed that it 
would be a simple matter of not accidentally using signed variables for 
pointer arithmetic and taking care to stuff the addresses into the page 
table entries in the correct quirky Intel format that the processor can 
understand.  Apparently it isn't that simple.

Since we have been talking about glibc and ELF issues this week, I've 
been thinking that we should consider moving the default application 
start address from 0x80000000 to the 0x00000000-0x7fffffff range that is 
used by every other OS I have worked with recently.  I suspect that the 
reason the kernel is loaded at 0x00100000 today is because it is 
slightly easier to bootstrap when you don't have to move the kernel from 
its BIOS-loaded location to its final destination in virtual memory. 
Since we're using GRUB, I don't think that's an issue any more.  Loading 
the kernel at 0x80000000 (2GB/2GB split) or 0xc0000000 (to give user 
progs. 3GB to the kernel's 1GB) would allow us to consider future 
support for loading foreign binaries, such as Linux programs (we are 
missing quite a few syscalls right now but FreeBSD has shown that it's 
possible) and WINE emulation.  Since the kernel's virtual address space 
is fixed (so that it can copy to/from all user address spaces 
efficiently), making this change sooner rather than later is important, 
even though no-one may get around to porting WINE for years.

I'm assuming that no existing Syllable app requires more than 1GB of 
virtual memory.  Since we don't have swapfile support right now, I think 
we would all notice if there was one!  This makes backwards 
compatibility easy because if we move the Syllable kernel up to 
0xc0000000 then we can give legacy apps that load at 0x80000000 at least 
1GB of address space, and possibly more considering that we now know 
they're actually shared libraries and not ELF executables.  New apps 
could be compiled as ELF executables with a dummy interpreter entry and 
a starting location of let's say 0x0400000 (since this is virtual 
address space, there aren't any holes in it, but it's good to leave the 
initial pages unmapped to catch null-pointer dereferences) and then 
we'll have a 3GB range for virtual memory.  Sound like a reasonable plan?

-Jake

[Syllable-kernel] I win!

[Kristian V. <kri...@xe...>]

[Insert ticker-tape and brass band here]

After something like 14 months of hacking I've finally done
it; Glibc 2.3.3 runs.  I've got the Glibc libraries and
a new libgcc_s.so compiled and all of my current tests compile
and run without crashing, or producing funny results.  Even
Gcc 3.3.4 has been built against the new libc and works; a
pretty good test case.

I'll upload binaries and source tonight; I know there are at
least one or two of you who would like to have a play with
Glibc 2.3.3 and I hope a few more of you are willing to test
it (Don't worry, it installs alongside 2.1.2 so nothing will
stop running)

There is still unfinshed business to attend to:

o The ELF Makefile has been hacked up beyond recognition.  I
  need a clean way to handle ELF stuff on Syllable.
o Ditto for the "Don't use init/fini arrays" quick-patch I did
  yesterday
o Kurt added two functions to libc; stricmp() and strnicmp(),
  which I have not implemented in 2.3.3  I'll probably have to
  add them eventually; I suspect the appserver and libsyllable
  uses them and they have the potential to be generally useful.
o The biggest issue right now is that C++ can't be compiled.
  There are incompatabilities between the C++ headers and the
  2.3.3 headers.  I could do with a hand fixing these; any
  volounters?
o I had to manually hack a Makefile to get libgcc_s.so to
  compile correctly.  I need to figure out how to build a
  working libgcc_s.so without doing that.
o Testing...

For my next trick...I shall go and lie down a bit.  Maybe have
a cup of tea.

-- 
Vanders
http://www.syllable.org
http://www.liqwyd.com

_____________________________________________________________________
This message has been checked for all known viruses by Xenicom delivered through the MessageLabs Virus Control Centre. 

[Syllable-kernel] Re: [Syllable-developer] libc bug redux

[Daniel G. <da...@fp...>]

On Thu, 2004-11-11 at 11:54 +0000, Kristian Vandervliet wrote:
> It's a loader bug folks.  Once I (finally) got libc.so
> recompiled without those symbols and recompiled the
> testcase with that -shared flag intact it runs perfectly.
> Looks like the ELF loader messes up the .bss section on
> non-shared ELF objects, and I'm back to compiling everything
> as ELF DSO's even though it makes no sense.
> 
> This is one bug I'd like to get fixed; Jake?  Anybody else
> looking for an interesting, fairly self-contained project?
> 

Well, first off, the kernel loader ignores the program header, which
will amost certainly break any non-DSO.  It looks like the loader is
extremely incomplete.  Would it be possible to use it to just load a
userspace ld.so, and let that to the real work?  Is the mmap code in the
kernel good enough?  Just speculation here, for the moment.

Daniel

[Syllable-kernel] libc bug redux

[Kristian V. <kri...@xe...>]

It's a loader bug folks.  Once I (finally) got libc.so
recompiled without those symbols and recompiled the
testcase with that -shared flag intact it runs perfectly.
Looks like the ELF loader messes up the .bss section on
non-shared ELF objects, and I'm back to compiling everything
as ELF DSO's even though it makes no sense.

This is one bug I'd like to get fixed; Jake?  Anybody else
looking for an interesting, fairly self-contained project?

-- 
Vanders
http://www.syllable.org
http://www.liqwyd.com

_____________________________________________________________________
This message has been checked for all known viruses by Xenicom delivered through the MessageLabs Virus Control Centre. 

[Syllable-kernel] Re: [Syllable-developer] Need help

[Daniel G. <da...@fp...>]

On Wed, 2004-11-10 at 13:47 +0000, Kristian Vandervliet wrote:
> I'm currently testing Glibc 2.3.3, and as part of that I am trying
> to rebuild Gcc 3.3.4 against the new libs.  I've run into a problem
> which is now beginning to look like a bug with Glibc 2.3.3, but I'm
> totally stumped.
> 
> I've got it down to the following testcase:
> 
> #include <stdlib.h>
> #include <stdio.h>
> 
> FILE *y;
> 
> int main( int argc, char *argv[] )
> {
>   FILE *x;
> 
>   printf("x\n");
>   x = fopen(argv[0],"r");
>   fclose(x);
>   printf("x closed\n");
> 
>   if( NULL == y )
>   {
>     printf("y is NULL\n");
>     return EXIT_FAILURE;
>   }
>   printf("y\n");
>   y = fopen(argv[0],"r");
>   fclose(y);
>   printf("y closed\n");
> 
>   return EXIT_SUCCESS;
> }
> 
> Output is:
> 
> $ ./fopen
> x
> x closed
> y
> Segmentation fault (core dumped)
> $ 

For some strange reason, y is *not* NULL (or it wouldn't have printed
"y").  It's in the BSS, so it *should* be NULL.
> 
> If you're paying attention, it actually crashes when it gets the the
> if( NULL == y ) with a NULL pointer:
> 
> 0:fopen::fopen : Invalid pagefault at 00000000 (NOTP:READ:USER)
> 0:fopen::fopen : EAX = 00000000 : EBX = 8000185c : ECX = 00000009 : EDX =
> a00e4aa0
> 0:fopen::fopen : ESI = 00000001 : EDI = ffffbb2c : EBP = ffffbaac
> 0:fopen::fopen : SS::ESP = 0023::ffffba90
> 0:fopen::fopen : CS::EIP = 0013::800006b5
> 0:fopen::fopen : DS = 0023 : ES = 0023 : FS = 0023 : GS = 00d0
> 0:fopen::fopen : EFLAGS = 00213286 (PF SF IF RF ID )
> 0:fopen::fopen : CPU ID = 0 : kernel stack = 051c5014
> 0:fopen::fopen : 0 -> 800006b5
> 0:fopen::fopen :    fopen + 000006b5 -> __gmon_start__ + 800006b5
> 0:fopen::fopen : 1 -> a00143e1
> 0:fopen::fopen :    libc.so.2 + 000143e1 -> __libc_start_main + 000000a1
> 0:fopen::fopen : 2 -> 80000435
> 0:fopen::fopen :    fopen + 00000435 -> __gmon_start__ + 80000435
> 0:fopen::fopen : verify_area() got kernel address 00000000

This confirms it, as it's crashing in fopen.

My guess is that the loader is somehow not zeroing out the bss, or is
somehow aliasing y to something else.

If I run this code locally on Linux, either with gcc 3.4.3 or with tcc,
it exits with "y is NULL".

Daniel

[Syllable-kernel] Need help

[Kristian V. <kri...@xe...>]

I'm currently testing Glibc 2.3.3, and as part of that I am trying
to rebuild Gcc 3.3.4 against the new libs.  I've run into a problem
which is now beginning to look like a bug with Glibc 2.3.3, but I'm
totally stumped.

I've got it down to the following testcase:

#include <stdlib.h>
#include <stdio.h>

FILE *y;

int main( int argc, char *argv[] )
{
  FILE *x;

  printf("x\n");
  x = fopen(argv[0],"r");
  fclose(x);
  printf("x closed\n");

  if( NULL == y )
  {
    printf("y is NULL\n");
    return EXIT_FAILURE;
  }
  printf("y\n");
  y = fopen(argv[0],"r");
  fclose(y);
  printf("y closed\n");

  return EXIT_SUCCESS;
}

Output is:

$ ./fopen
x
x closed
y
Segmentation fault (core dumped)
$ 

If you're paying attention, it actually crashes when it gets the the
if( NULL == y ) with a NULL pointer:

0:fopen::fopen : Invalid pagefault at 00000000 (NOTP:READ:USER)
0:fopen::fopen : EAX = 00000000 : EBX = 8000185c : ECX = 00000009 : EDX =
a00e4aa0
0:fopen::fopen : ESI = 00000001 : EDI = ffffbb2c : EBP = ffffbaac
0:fopen::fopen : SS::ESP = 0023::ffffba90
0:fopen::fopen : CS::EIP = 0013::800006b5
0:fopen::fopen : DS = 0023 : ES = 0023 : FS = 0023 : GS = 00d0
0:fopen::fopen : EFLAGS = 00213286 (PF SF IF RF ID )
0:fopen::fopen : CPU ID = 0 : kernel stack = 051c5014
0:fopen::fopen : 0 -> 800006b5
0:fopen::fopen :    fopen + 000006b5 -> __gmon_start__ + 800006b5
0:fopen::fopen : 1 -> a00143e1
0:fopen::fopen :    libc.so.2 + 000143e1 -> __libc_start_main + 000000a1
0:fopen::fopen : 2 -> 80000435
0:fopen::fopen :    fopen + 00000435 -> __gmon_start__ + 80000435
0:fopen::fopen : verify_area() got kernel address 00000000
0:fopen::fopen : 
0:fopen::fopen : Areas :
0:fopen::fopen : Area 0000 (10239) -> 80000000-80000fff 04ca7cd8 01 ro_fopen
0:fopen::fopen : Area 0001 (10240) -> 80001000-80001fff 04ca7cd8 01 rw_fopen
0:fopen::fopen : Area 0002 (10244) -> 88000000-88022fff 00000000 01 heap
0:fopen::fopen : Area 0003 (10241) -> a0000000-a00e3fff 04ca7ad8 01
ro_libc.so.2
0:fopen::fopen : Area 0004 (10242) -> a00e4000-a00eafff 04ca7ad8 01
rw_libc.so.2
0:fopen::fopen : Area 0005 (10243) -> ffc00000-ffffffff 00000000 01
main_stack
0:fopen::fopen : 0 -> 800006b5
0:fopen::fopen :    fopen + 000006b5 -> __gmon_start__ + 800006b5
0:fopen::fopen : 1 -> a00143e1
0:fopen::fopen :    libc.so.2 + 000143e1 -> __libc_start_main + 000000a1
0:fopen::fopen : 2 -> 80000435
0:fopen::fopen :    fopen + 00000435 -> __gmon_start__ + 80000435
0:fopen::fopen : verify_area() got kernel address 00000000
0:fopen::fopen : Killed by signal 11

Through experimentation I've found:

o If the declaration of y is moved inside main() it works.
o If y is declared as static FILE *y, it works.

This is what is causing me so much trouble; the scope of the variable is
relevent to the bug.  I could understand it if the crash was being caused
when fopen() tries to assign the value to y, but it crashes *whenever the
code goes anywhere near y*.  This, of course, is impossible.  At least, it
is highly improbable.

I need a linker/asm guru fast.  Can anyone explain why the scope of the
variable would have any relevence, and how it could cause this?

-- 
Vanders
http://www.syllable.org
http://www.liqwyd.com

_____________________________________________________________________
This message has been checked for all known viruses by Xenicom delivered through the MessageLabs Virus Control Centre. 

[Syllable-kernel] New kernel patches

[Jake H. <jh...@an...>]

Jake Hamby wrote:

> I am busy preparing my next "mega-patch" for the kernel which will:
> 
> * add support for SSE/SSE2 and lazy floating-point context switching,
> * inline numerous short assembly language functions from intel.s,
> * fix a few signal handling and fork/exec context initialization bugs,
> * correct numerous typos and grammatical errors in the source code,
> * improve the performance of allocating and freeing memory pages, and
> 
> * integrate the latest version of <atheos/isa_io.h> and "inc/io_ports.h" 
> from Linux (the second one gives us more readable code since I was able 
> to replace "0x20" with "PIC_MASTER_CMD", "0x43" with "PIT_MODE", and so 
> on throughout the i8259A access code).

I've just checked in the kernel improvements I mentioned last week.  The 
main feature is that SSE/SSE2 instructions are now enabled so if you 
have a Pentium 3 or higher, or Athlon with SSE support, you should now 
be able to compile programs such as mplayer with SSE support.  Here's my 
note from ChangeLog.Kernel:

Implement lazy FPU save/restore for faster context switching and save 
the state of XMM registers on Pentium3 and above for SSE support. 
Exception handlers in fault.c send signals (SIGSEGV, SIGFPE, SIGILL) 
rather than exiting the process.  Spelling fixes in comments.  The 
function is_signals_pending() is now correctly spelled 
is_signal_pending(). The new SeqLock_s type from Linux is used in 
timer.c for reading the timer registers without disabling interrupts, 
and the "inc/io_ports.h" header defines constants for the PIC I/O ports, 
which are used for readability in irq.c, smp.c and timer.c.  Other minor 
cleanups include removing useless statistical counters from array.c, 
adding a warning to kmalloc() when 128K or larger allocations are made, 
modifying bcache.c to use an area instead of kmalloc() for its 
hashtable, using g_sSemListSpinLock instead of g_sSchedSpinLock in 
sema.c, a new do_free_pages() function in mman.c that doesn't call 
flush_tlb_global() (used in loops by areas.c), inlined asm functions in 
"inc/intel.h" and atheos/kernel.h, and updated macros from Linux for 
atheos/isa_io.h.  "macros.h" now uses a macro called __likely() on GCC 
3.0 and above as a hint to the compiler in kassertw() that the 
expression asserted is expected to be true.

NOTE: You should install the new header files from the sys/include 
directory with "make dist" and then rebuild your drivers along with the 
kernel.

It takes a day or so for the changes to propagate to anonymous CVS.  If 
you have any problems, please e-mail me or post to syllable-kernel.  If 
the new kernel crashes on your machine (unlikely, I hope!), you can boot 
the previous kernel by editing the kernel line in GRUB to use 
"/atheos/sys/kernel.so.orig".

-Jake

[Syllable-kernel] New developer: Jake Hamby

[Kristian V. D. V. <va...@li...>]

All,

Jake Hamby has officially joined the team and now has CVS commit access.  Some 
of you may have noticed the work that Jake has been doing on the 
long-neglected kernel.  Frankly I was getting tired of checking in all those 
patches, so now Jake is going to do it himself!

Jake is now our official kernel-dude, so any kernel patches should be directed 
in his general direction.  Note that this doesn't really include drivers; 
those patches either should go to the current maintainer, myself, Jake or 
Arno (Depending on who you think is the best match).  If in doubt send it to 
all of us and one of us will deal with it!

If you're interested in the work going into the kernel then please do 
subscribe to the syllable-kernel mailing list.  The traffic is light but 
certainly more focused then it is on syllable-developer.

-- 
Vanders
http://syllable.sourceforge.net/
http://www.liqwyd.com

[Syllable-kernel] Font Tutorial

[Steven P. <lyn...@es...>]

Hi everyone,
    I thought that I would help out a bit by helping Brent in writing
Tutorials on writing Syllable applications, starting with the Font class.

    Could someone explain to me what the style flag OS::FONT_HAS_TUNED_SIZES
implies?

    Thanks,
-Steven

Re: [Syllable-kernel] upcoming Syllable patches..

[Daniel G. <da...@fp...>]

On Sun, 2004-10-31 at 22:19 +1300, Maritimus wrote:

> > >Otherwise, it's probably never going to get fixed.
> > And I think that=E2=80=99s the best.
>=20
> But why not? It's just some lines of code, it isn't of any disadvantage t=
o the system, it's just like removing a minor bug that stops someone's vide=
o card from working.
>=20

Because it's extra code that has to be read and maintained, it causes
complexity to conditionaly compile it, it causes complexity to
distribute it (now we need two sets of kernel downloads), it will be
almost never tested (I have no 486's, and I wouldn't run Syllable on
them even if I did).

I vote no on 486.  It lowers the least common denominator for basically
no benifit.

Daniel

Re: [Syllable-kernel] upcoming Syllable patches..

[Kristian V. D. V. <va...@li...>]

On Sunday 31 October 2004 7:37 am, BurningShadow wrote:
> I got a 486 somewhere (and it should be working), so I can test it.
>
> But who would run it on a 486? The 486 hasnt been in active sale for many
> many years, and I dont think that Syllable should be known as a
> you-can-run-it-on-your-oldest-computer system.

Syllable on anything less than i586 is a total waste of time.  Even if you 
remove the few Pentium-only instructions and actually boot the kernel on a 
i486 Syllable only really supports PCI devices.  Supporting all the ISA 
hardware out there on i486 class machines is a waste of time.  We'd need an 
ISA bus manager with PnP support and changes to a lot of drivers to search 
for and support ISA hardware.

Not only that but the default target for gas is i586; all of the software 
compiled in the last nine months is i586 only.  You'd have to recompile 
everything for i386, and then everyone else looses out because we'd either 
have i386 only binaries on new hardware or we'd have two sets of binaries.

Seriously, forget about old hardware.  We won't support it!

-- 
Vanders
http://syllable.sourceforge.net/
http://www.liqwyd.com

Re: [Syllable-kernel] upcoming Syllable patches..

[BurningShadow <syl...@bu...>]

>> I got a 486 somewhere (and it should be working), so I can test it.
>>
>> But who would run it on a 486? The 486 hasn’t been in active sale for many many years, and I don’t think that Syllable should be known as a you-can-run-it-on-your-oldest-computer system. That’s the way people (in Denmark) see Linux, and now they can’t understand why they can’t run Fedora on their pre-pentium system.
>
>Running Syllable on a 1989 DX25 wouldn't be very practical, but the end of generation 486s and the non-Intel Pentium / P6 clones aren't pre-Pentium in performance, as some of these chips can beat a Pentium 100. When you add in if OpenMOSIX is ported, and specialised tasks like cheap low-power kiosks, they become more
>feasible. Syllable runs fine on a P1 as it is.
>
>If people in developed countries on AMD64 FXs (like you see in the computing forums) think they'd be demasculinated by running an operating system with system requirements that are lower than WinXP... what can be said.
>
You are absolutely right about that, and a lot of people will understand that they should not install Syllable on an old 486/25, but instead choose a 486/100.
But I have seen it a lot of times and of cause the problem might be 100% Danish, although I doubt that. People read an article in Alt Om Data, about how easy it is to make a 486/120 usable, by installing RedHat and OpenOffice. Then they can't understand that Linux/OpenOffice can't run smoothly on their old 486/25, and concludes that Linux sucks.
We can easily agree that a 486/120 can outrun a P1/60, but is this really reason enough to make it run on 486's? 


>
>> In fact I think the exact upper side should be done.10 years from now, there will be almost no 32bit’s left, so at that time it won’t make sense to keep 32bit code. It should be wiped out, in order to keep the system up-to-date.
>
>Intel called AMD64 "Extended Memory Technology" to be derisive to AMD, but that's really all it means for nearly all desktops. If the memory addressing issue wasn't present, then the migration to 64-bit would be very slow because there just isn't any advantage to it unless you're doing specialised number crunching..
>Technologically, support for AMD64 will be driven by applications needing full multigigabyte RAM addressability, not much else.
>
No matter how Intel describes the AMD CPU, it doesn't change the fact that more and more computers are sold with 64bit, and that one day there will be no 32bit machines left, just like there’s no 8bit's left.


>
>> >Otherwise, it's probably never going to get fixed.
>> And I think that’s the best.
>
>But why not? It's just some lines of code, it isn't of any disadvantage to the system, it's just like removing a minor bug that stops someone's video card from working.
>
You might call it a "just some lines of code", but is it positive for the system, in the long term?
7 years ago, some magazines (in Denmark) started to tell people about Linux. At that time Linux didn't have Gnome/KDE. It was configured by using LinuxConf or various command line tools, and everybody wanted to try this cool new system, but they found it hard to use, so they never touched it again.
Today they refuse to try it, because it's "not userfriendly" and "you have to do everything in the console". The very same people can't install a fsck.ing video driver, in WindowsXP (WindowsXP is _very_ user-friendly, you remember? :D ).
By making it possible to use Syllable on the more powerful 486's you will make a lot of people install it on the not-so-powerful 486's, and they will run away, to never return. And they are exactly the people that Syllable was meant for.
You might be able to give Syllable a boost, among the semi-geeks, right now, but then someone writes an article about it, and the people who should be using it 5 years from now, will try it.




I’m not saying, that I got all the right answers, this is just how I see it.

Re: [Syllable-kernel] upcoming Syllable patches..

[Maritimus <rea...@ih...>]

BurningShadow wrote:

> I got a 486 somewhere (and it should be working), so I can test it.
>
> But who would run it on a 486? The 486 hasn’t been in active sale for many many years, and I don’t think that Syllable should be known as a you-can-run-it-on-your-oldest-computer system. That’s the way people (in Denmark) see Linux, and now they can’t understand why they can’t run Fedora on their pre-pentium system.

Running Syllable on a 1989 DX25 wouldn't be very practical, but the end of generation 486s and the non-Intel Pentium / P6 clones aren't pre-Pentium in performance, as some of these chips can beat a Pentium 100. When you add in if OpenMOSIX is ported, and specialised tasks like cheap low-power kiosks, they become more
feasible. Syllable runs fine on a P1 as it is.

If people in developed countries on AMD64 FXs (like you see in the computing forums) think they'd be demasculinated by running an operating system with system requirements that are lower than WinXP... what can be said.


> In fact I think the exact upper side should be done.10 years from now, there will be almost no 32bit’s left, so at that time it won’t make sense to keep 32bit code. It should be wiped out, in order to keep the system up-to-date.

Intel called AMD64 "Extended Memory Technology" to be derisive to AMD, but that's really all it means for nearly all desktops. If the memory addressing issue wasn't present, then the migration to 64-bit would be very slow because there just isn't any advantage to it unless you're doing specialised number crunching..
Technologically, support for AMD64 will be driven by applications needing full multigigabyte RAM addressability, not much else.



> >Otherwise, it's probably never going to get fixed.
> And I think that’s the best.

But why not? It's just some lines of code, it isn't of any disadvantage to the system, it's just like removing a minor bug that stops someone's video card from working.


> >> The NexGen Nx586, AMD/Cyrix 5x86s, 486s etc, don't have it at all. Not that
> >> Syllable can currently run on those because of a couple of lines of Kurt's
> >> i586 assembler enthusiasm, but someone's going to want to sooner or later :)
> >
> >Besides the code which assumes the presence of the cpuid instruction, is
> >there any other assembly language code that anyone knows of that is
> >keeping it from working on 486s?  I know how to fix the cpuid code to
> >detect if the machine doesn't support cpuid, but how many people these
> >days have access to a 486-class machine with math coprocessor that they
> >want to run Syllable on?  If you do, let me know and I may be able to
> >work something out.  Otherwise, it's probably never going to get fixed.
> >