syllable-kernel — The list is for discussion of Kernel issues by developers

Re: [Syllable-kernel] Syllable security and "trusted computing"

[Mark W. <mar...@cl...>]

> would require ring 0.  OTOH, it's quite possible that rings 1 and 2 are
> essentially useless unless you are writing some old-style segmented
> memory or 16/32-bit OS.

I think the usual way to do this is to overlay multiple segments (with the 
same base) with different privilege levels.  I think it can be worked as a 
straightforward extension of the usual x86 tricks: map ring0-only memory at 
the top of memory, then ring 1's private space, then ring 2's private space.  
Then have a set of segments with different privilege levels that permit 
access to the relevant range of linear memory.

Sound sane?

> Perhaps that explains why the only OS I could 
> find which uses more than 2 rings was OS/2, which uses ring 2 for user
> code segments with privileges to access I/O ports directly.

Interesting, I never knew what OS/2 did with those rings :-)  You don't need 
this for user IO privileges, though - can still restrict IO port access on a 
per-process basis (even with IOPL = 3) by setting bits in the TSS IO bitmap.  
With Syllable using hardware task-switching, this should Just Work, rewriting 
the TSS IO bitmap on context switches would be necessary for s/w switching 
(with suitable tracking in place, this needn't have high overhead).

Cheers,
Mark

>
> FWIW, here are the names of the four different privilege modes used by
> VMS.  I still need to learn what the two inner levels are used for, but
> at least the names give a hint as to the level of "need to know"
> privileges that each layer of a more-secure OS might have:
>
> 0 - kernel
> 1 - executive
> 2 - supervisor
> 3 - user
>
> I have a couple of notes here on the requirements needed for a Syllable
> user to be able to "fully control any storage or disclosure" of whatever
> personal info they want to protect:
>
> - user data in the pagefile would be encrypted (using a randomly
> generated key stored in RAM only)
>
> - secure browsing mode:
>    - no caching of pages/images to disk
>    - URL history tracking in memory only
>    - no cookies saved, or optional encrypted cookie file
>
> - support for encrypting any file saved to disk, configured at the file
> or folder level (see NTFS).
>
> - no unauthorized screen grabs or keystroke/mouse loggers.
> - built-in secure delete (see GNU shred).
>
> - "magic key sequence" always pops up the OS-provided task manager or
> login screen (e.g. Windows CTRL-ALT-DEL) and can't be trapped by
> malicious programs to create a fake login screen.
>
> Jake
>
>
> -------------------------------------------------------
> SF.Net email is sponsored by: Tell us your software development plans!
> Take this survey and enter to win a one-year sub to SourceForge.net
> Plus IDC's 2005 look-ahead and a copy of this survey
> Click here to start!  http://www.idcswdc.com/cgi-bin/survey?id=105hix
> _______________________________________________
> Syllable-kernel mailing list
> Syl...@li...
> https://lists.sourceforge.net/lists/listinfo/syllable-kernel

Re: [Syllable-kernel] Syllable security and "trusted computing"

[Daniel G. <da...@fp...>]

On Fri, 2005-04-29 at 17:09 +0100, Mark Williamson wrote:
> Of course non-Intel architectures have a range of different numbers and kinds 
> of protection rings, so it's worth bearing in mind portability whilst 
> architecting a system to use the rings.  Even on x86, a port to something 
> like Xen (or L4) would restrict the rings available for use.
> 

There's a reason why Unix systems only use 2 rings. :)  Minimal subset
makes porting much easier.

Daniel

Re: [Syllable-kernel] Syllable security and "trusted computing"

[Mark W. <mar...@cl...>]

> Aside from that, rings 1 and 2 don't really exist anymore, because
> they're not there in amd64.  This means that, for all intents and
> purposes, you cannot depend on them.  Within 2 years, x86 as such won't
> exist anymore, and all processors sold will be amd64.

Just as a point of interest, IA64 has 4 rings (not that I think a Syllable 
port to Itanium would be very sensible ;-))

Of course non-Intel architectures have a range of different numbers and kinds 
of protection rings, so it's worth bearing in mind portability whilst 
architecting a system to use the rings.  Even on x86, a port to something 
like Xen (or L4) would restrict the rings available for use.

Cheers,
Mark

Re: [Syllable-kernel] Syllable security and "trusted computing"

[Mark W. <mar...@cl...>]

Hi Kip, didn't expect to see you here ;-)

> > First, every UNIX OS as well as Windows uses only two of the four
> > "rings" of protection on x86.  The kernel and all modules run in Ring 0
> > and all user programs run in Ring 3.  Ring 0 can bypass all protections.
> >   What's interesting about the special hardware support that is being
> > added for "Palladium" (what Intel calls "LaGrande Technology", or LT),
> > is that they've essentially had to add a new "Ring -1" that is more
> > privileged than the layer at which the normal operating system runs, in
> > order to be able to assure the privacy and security that Windows itself
> > is apparently not able to provide.  This is stupid:  if we have Rings 1
>
> No it isn't, it is necessary if they are to support running Windows
> _unmodified_ on top of a hypervisor. See VT-x.

VT-x introduces "root mode" and "non-root mode", which is orthogonal to the 
ring you're running in (although root mode can only be entered and left in 
certain restricted ways).  Root mode & ring 0 is where a hypervisor would 
run, non-root mode & rings 0-3 would contain guest OSs.

One thing which I'm curious about is exactly what form "Ring -1" in LT will 
actually take.  Both AMD's and Intel's security extensions are dependent on 
their virtualisation extentions because they use similar mechanisms.  I guess 
it may be that "ring -1" == "root mode ring 0".  Speculation on my part...

Are there any specs available for LT?  I haven't seen any, the deadline for LT 
seems to have moved back relative to what I was expecting.

Cheers,
Mark

Re: [Syllable-kernel] Syllable security and "trusted computing"

[Daniel G. <da...@fp...>]

On Fri, 2005-04-29 at 08:52 -0700, Kip Macy wrote:
> > Microsoft/Intel "trusted computing" initiative.
> > 
> > First, every UNIX OS as well as Windows uses only two of the four
> > "rings" of protection on x86.  The kernel and all modules run in Ring 0
> > and all user programs run in Ring 3.  Ring 0 can bypass all protections.
> >   What's interesting about the special hardware support that is being
> > added for "Palladium" (what Intel calls "LaGrande Technology", or LT),
> > is that they've essentially had to add a new "Ring -1" that is more
> > privileged than the layer at which the normal operating system runs, in
> > order to be able to assure the privacy and security that Windows itself
> > is apparently not able to provide.  This is stupid:  if we have Rings 1
> 
> No it isn't, it is necessary if they are to support running Windows
> _unmodified_ on top of a hypervisor. See VT-x.
> 

Aside from that, rings 1 and 2 don't really exist anymore, because
they're not there in amd64.  This means that, for all intents and
purposes, you cannot depend on them.  Within 2 years, x86 as such won't
exist anymore, and all processors sold will be amd64.

Daniel

Re: [Syllable-kernel] Syllable security and "trusted computing"

[Kip M. <kip...@gm...>]

> Microsoft/Intel "trusted computing" initiative.
>=20
> First, every UNIX OS as well as Windows uses only two of the four
> "rings" of protection on x86.  The kernel and all modules run in Ring 0
> and all user programs run in Ring 3.  Ring 0 can bypass all protections.
>   What's interesting about the special hardware support that is being
> added for "Palladium" (what Intel calls "LaGrande Technology", or LT),
> is that they've essentially had to add a new "Ring -1" that is more
> privileged than the layer at which the normal operating system runs, in
> order to be able to assure the privacy and security that Windows itself
> is apparently not able to provide.  This is stupid:  if we have Rings 1

No it isn't, it is necessary if they are to support running Windows
_unmodified_ on top of a hypervisor. See VT-x.

Re: [Syllable-kernel] High-priority bugs, plus adding a HAL portability layer

[Kristian V. D. V. <va...@li...>]

On Friday 29 April 2005 00:59, Jake Hamby wrote:
> First, the "showstopper" bugs that we need to fix before 0.5.7:
>
> 2) Any other reproducible AFS bugs are high priority.  Fixing our SMP
> issues is high priority.

Both of these are very high priority.  I'd say the SMP issues are higher than 
AFS, but only by a fraction.

> 3) Glibc issues such as DNS resolving, proper timezone handling, and
> proper locale handling should be fixed for 0.5.7. 

I'm all over it, or at least I'll get DNS fixed in the next few days and then 
I can take a look at the tz and locale stuff.

> Currently GNU gettext 
> will crash when building Japanese message catalogs for GNU applications
> (e.g. binutils) due to a library loading bug in the new glibc.

Any stack traces or straces for me?

> I've been thinking about how to separate the x86-specific code from the
> kernel, which would make the code easier to understand as well as to
> port, without compromising the simplicity of the existing code.  The
> model I came up with is based loosely on the Windows NT concept of a
> HAL:  a thin layer beneath the kernel to do things like SMP
> initialization and bus enumeration for a particular platform.  My rough
> sketch looks like this:

<snip>

Sounds like a good plan to me, but you also need to consider possible endian 
issues and memory mapping.  Almost everything assumes IA32 with a 1:1 
virtual->physical mapping in the kernel, which is so unusual it will never be 
true anywhere else.  We need to start worrying about proper virt_to_phys(), 
virt_to_bus(), cpu_to_le() etc. macros and where to insert these into the 
code.  Eek!

-- 
Vanders
http://www.syllable.org
http://www.liqwyd.com

Re: [Syllable-kernel] High-priority bugs, plus adding a HAL portability layer

[Brent P. N. <me...@ot...>]

Quoting Jake Hamby <jh...@po...>:
> 
> First, the "showstopper" bugs that we need to fix before 0.5.7:
> 
> 1) Reformatting AFS partitions doesn't work correctly.
[snip
> Either there is a bug in afs_initialize() or the AFS code is accessing 
> sectors on the disk that it shouldn't be accessing that contain old 
> data.
[snip]

I've added this to my to-do list as the next thing for me to look at in AFS.

-- 
Brent P. Newhall
http://brent.other-space.com/

----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.

[Syllable-kernel] Syllable security and "trusted computing"

[Jake H. <jh...@po...>]

I've made a few notes on potential ways we could tighten up the Syllable 
security model at the kernel level beyond the typical x86 OS.  This is 
based partially on the architecture of OpenVMS and partially on the 
Microsoft/Intel "trusted computing" initiative.

First, every UNIX OS as well as Windows uses only two of the four 
"rings" of protection on x86.  The kernel and all modules run in Ring 0 
and all user programs run in Ring 3.  Ring 0 can bypass all protections. 
  What's interesting about the special hardware support that is being 
added for "Palladium" (what Intel calls "LaGrande Technology", or LT), 
is that they've essentially had to add a new "Ring -1" that is more 
privileged than the layer at which the normal operating system runs, in 
order to be able to assure the privacy and security that Windows itself 
is apparently not able to provide.  This is stupid:  if we have Rings 1 
and 2, as well as (in recent x86 chips) efficient ways to switch between 
rings (e.g. SYSENTER/SYSEXIT on Intel, SYSCALL/SYSRET on AMD), then we 
could theoretically put as little as possible in Ring 0 and have the 
majority of kernel threads running in Ring 1, Ring 2, or Ring 3.

So why use Rings 1 and 2 at all?  Why not have the "microkernel" portion 
in ring 0 and put everything else in ring 3?  Well, from a performance 
standpoint, it may be possible to set up the shared portion of the 
virtual memory space ("kernel space") so that particular areas are 
available to all code running at ring 1, while other data structures 
would require ring 0.  OTOH, it's quite possible that rings 1 and 2 are 
essentially useless unless you are writing some old-style segmented 
memory or 16/32-bit OS.  Perhaps that explains why the only OS I could 
find which uses more than 2 rings was OS/2, which uses ring 2 for user 
code segments with privileges to access I/O ports directly.

FWIW, here are the names of the four different privilege modes used by 
VMS.  I still need to learn what the two inner levels are used for, but 
at least the names give a hint as to the level of "need to know" 
privileges that each layer of a more-secure OS might have:

0 - kernel
1 - executive
2 - supervisor
3 - user

I have a couple of notes here on the requirements needed for a Syllable 
user to be able to "fully control any storage or disclosure" of whatever 
personal info they want to protect:

- user data in the pagefile would be encrypted (using a randomly 
generated key stored in RAM only)

- secure browsing mode:
   - no caching of pages/images to disk
   - URL history tracking in memory only
   - no cookies saved, or optional encrypted cookie file

- support for encrypting any file saved to disk, configured at the file 
or folder level (see NTFS).

- no unauthorized screen grabs or keystroke/mouse loggers.
- built-in secure delete (see GNU shred).

- "magic key sequence" always pops up the OS-provided task manager or 
login screen (e.g. Windows CTRL-ALT-DEL) and can't be trapped by 
malicious programs to create a fake login screen.

Jake

[Syllable-kernel] High-priority bugs, plus adding a HAL portability layer

[Jake H. <jh...@po...>]

First, the "showstopper" bugs that we need to fix before 0.5.7:

1) Reformatting AFS partitions doesn't work correctly.  I ran into this 
problem ysterday when the unzip failed during the 0.5.6 install after 
reformatting my old AFS partition.  Zeroing out the first 100MB of the 
partition with "dd" before reformatting didn't solve the problem. 
Either there is a bug in afs_initialize() or the AFS code is accessing 
sectors on the disk that it shouldn't be accessing that contain old 
data.  I reformatted my partition as FAT (which others have mentioned as 
a workaround for this bug) and will try to reinstall 0.5.6 again.

2) Any other reproducible AFS bugs are high priority.  Fixing our SMP 
issues is high priority.  We should also work properly on large RAM 
systems (>2GB) and I'd like to know if any of Arno's fixes have fixed 
that bug or if it's still present on anyone's system.

3) Glibc issues such as DNS resolving, proper timezone handling, and 
proper locale handling should be fixed for 0.5.7.  Currently GNU gettext 
will crash when building Japanese message catalogs for GNU applications 
(e.g. binutils) due to a library loading bug in the new glibc.  The 
kernel will need to start returning the time in UTC instead of local 
time, and we'll need to adjust by the correct time zone offset when 
loading and saving to the battery backed clock.  If the user is 
dual-booting with Linux, she probably has the h/w clock set to UTC, but 
if dual-booting with Windows, it will be set to local time.  We will 
need a GUI checkbox for the user to select between these two 
possibilities and to select the correct timezone.

I've been thinking about how to separate the x86-specific code from the 
kernel, which would make the code easier to understand as well as to 
port, without compromising the simplicity of the existing code.  The 
model I came up with is based loosely on the Windows NT concept of a 
HAL:  a thin layer beneath the kernel to do things like SMP 
initialization and bus enumeration for a particular platform.  My rough 
sketch looks like this:

All MI (portable) code stays where it is.  Where there is x86-specific 
code now, it will be replaced with calls to hal_xxxx(), functions 
(possibly inline) defined in "hal/xxxx.h".  These functions will be 
implemented in .c and .s files inside a new 'x86' directory and will be 
linked into the kernel (not a module).  In the case of non-portable 
inline functions, the hal header file can include an x86 header file, or 
the appropriate header for another architecture, using #ifdefs to 
determine the appropriate header to include, e.g. #ifdef __i386.  MI 
code can only directly include headers from "inc" or "hal", not from "x86".

Rough sketch of how some existing functions will remap:

kernel_init() [init.c]    ->  hal_kernel_init() + helper funcs
init_kernel_mb() [init.c] ->  hal_early_init(), called from asm startup

spinlock_disable(),       ->  hal_spinlock_disable(),
spinunlock_enable(), etc. ->  hal_spinunlock_enable(), etc.
atomic_add(), etc.        ->  (no change)

cli()/put_cpu_flags()     ->  hal_cli(), hal_put_cpu_flags()

It'd be nice if we had some form of mutex optimized for use inside the 
kernel that would have less overhead than a semaphore and no dynamic 
allocation requirement, like a spinlock.  It might look like this:

create_semaphore(),       -> hal_mutex_init(),
lock_semaphore(),         -> hal_mutex_lock(),
unlock_semaphore(),       -> hal_mutex_unlock(),
delete_semaphore()        ->  (not needed)

dbcon_set_color(),        -> hal_console_set_color(),
dbcon_clear(),            -> hal_console_clear(),
dbcon_write()             -> hal_console_write()

init_smp(),               -> hal_smp_startup()
idle_loop(),              -> hal_idle_loop()
hard_reset(),             -> hal_system_reset()
sys_apm_poweroff(),       -> hal_system_poweroff()
parse_kernel_params()     -> hal_parse_kernel_params()

ISA read/write functions inb(), inb_p(), outb(), etc. can stay the same, 
and the other header files in <atheos/*> should be made portable (using 
inline hal_*() functions for abstraction if necessary).

The kernel header files in 'inc' to move to "x86" are:  cputable.h, 
i82489.h, intel.h, io_ports.h, mc146818.h, mman.h (split into inc/mman.h 
and x86/mman.h), pit_timer.h, smp.h, and virt86.h.

I can figure out the other hal functions as I go through the code, but 
that's the basic idea.  For example in the context switching code, there 
would be a call to hal_swap_context() encapsulating the x86-specific 
register manipulations that are there now.

Some ideas on security coming up..

Jake

[Syllable-kernel] preparing for 64-bit in the kernel (correction)

[Jake H. <jh...@po...>]

As a correction to my last post, a 40-bit linear address can support up 
to 1TB, not 1024TB, of RAM.  At OEMPCworld.com, a 2.0GB PC2-3200 
(400MHz) DDR 2 ECC DIMM costs $499.85.  If your server had 512 DIMM 
sockets to put them in, you could buy 1TB of RAM for your server for 
around US$256,000 and you've just maxed out your memory.

Correcting the other numbers:  with 48 bits, x86-64 can address up to 
256TB of virtual address space.

The DEC Alpha with 8K pages can handle up to 8TB of virtual address 
space and 32TB of physical memory.  By expanding the page size up to 
64K, the architecture can support up to 32768 TB of virtual address 
space and 256TB of RAM.  I don't know what the limits are for PPC or 
Itanium.

-Jake

[Syllable-kernel] preparing for 64-bit in the kernel

[Jake H. <jh...@po...>]

I've just finished taking some notes on how 64-bit virtual memory is 
handled on x86-64.  The good news is that 64-bit virtual memory 
structures are based on the PAE 36-bit extension introduced with the 
Pentium Pro processor to add support for up to 64GB of physical memory. 
If we add support for PAE, we can test it on our existing 32-bit 
machines, gain support for systems with >4GB of physical memory, *and* 
prepare Syllable for full x86-64 support later on.

Here's a quick summary of the existing Intel virtual memory system, then 
PAE-36, and finally the 64-bit "native" memory mapping.  Set your e-mail 
client to use a nonproportional font for the box graphics to render 
correctly.  :-)

The 32-bit x86 CPUs use a page size of 4K.  A 32-bit linear address 
(which is equivalent to a physical memory address as long as your 
segment base is set to 0, which is always true on Syllable) maps to a 
32-bit virtual address like this:

31                22 21                12 11                0
-------------------------------------------------------------
|    Directory      |    Page Table      | Offset w/in page |
-------------------------------------------------------------

The upper 10 bits select one of 1024 page table pointers from the page 
directory (a 4K page pointed to by register CR3) and the middle 10 bits 
select one of 1024 entries in the page table pointed to by the page 
directory entry.

The Pentium Pro introduced the PSE feature, which allows you to use 4K 
pages and large 4MB pages at the same time (one useful optimization we 
could add in the future is to map the kernel and kernel module code into 
a single 4MB page to free up TLB entries for caching user page lookups):

31                22 21                                     0
-------------------------------------------------------------
|    Directory      |         Offset w/in 4MB page          |
-------------------------------------------------------------

In this case, the page directory entries point directly to the page 
instead of pointing to a page table.

Moving right along, the PAE extension, also introduced with the PPro, 
expands the size of each entry in the page directory and the page tables 
from 4 bytes to 8 bytes, so now there are only 512 entries in a 4K table 
instead of 1024.  The extra space can be used to map up to 64GB (36 
bits) of physical memory into the 4GB virtual address space. With 4K 
page mapping, PAE looks like this:

31 30 29                21 20                12 11                0
-------------------------------------------------------------------
|D.P.|     Directory      |    Page Table      | Offset w/in page |
-------------------------------------------------------------------

The page table and page directory indexes are one bit smaller, since 
there are now half the number of entries (512) in each table.  The upper 
two bits are new and add an extra layer of indirection.  With PAE 
enabled, the CR3 register now points to a page-directory-pointer table 
(must be on a 32-byte boundary) and the D.P. bits select one of four 
entries from it.  Each entry is 64 bits in size and the selected entry 
contains a pointer to the 36-bit linear base address of the page-table 
directory to use.

Only one more type of page and then we can move on to x86-64:  PAE mode 
with large pages.  This time the offset within the page is one bit 
smaller than with PSE, so each large page is 2MB instead of 4MB:

31 30 29                21 20                                     0
-------------------------------------------------------------------
|D.P.|     Directory      |        Offset within 2MB page         |
-------------------------------------------------------------------

On to the 64-bit stuff, which Intel calls EM64T and AMD calls AMD64. I 
gathered this info from the Intel docs, available here:
http://www.intel.com/technology/64bitextensions/
If you don't already have a copy of the IA-32 manuals, then you should 
definitely download them from here:
http://developer.intel.com/design/pentium4/manuals/index_new.htm

In PAE, each entry is 8 bytes instead of 4.  The 64-bit extensions take 
advantage of the extra space to allow each entry to point to a 40-bit 
physical address, which allows for up to 1024TB of memory (to paraphrase 
billg: 1024TB of RAM should be enough for anyone!).  Presently, only 48 
bits in a 64-bit linear (virtual) address are used and the page sizes 
remain 4K and 2MB:

47     39 38    30 29         21 20           12 11                0
--------------------------------------------------------------------
|  PML4  |  D.P.  |  Directory  |  Page Table   | Offset w/in page |
--------------------------------------------------------------------

47     39 38    30 29         21 20                                0
--------------------------------------------------------------------
|  PML4  |  D.P.  |  Directory  |     Offset within 2MB page       |
--------------------------------------------------------------------

Compared to 32-bit PAE, the previous page directory pointer (PDP) has 
been expanded from 2 bits to 9, and can now select one of 512 page 
directories from a 4K PDP table.  Like the 32-bit PDP table, each entry 
is 64 bits in size.  x86-64 adds a new table called the page map level 4 
table, or PML4 for short, which works the same way as the PDP table, 
giving an awkward five levels of indirection in total:

PML4 -> PDP -> PDE -> PTE -> physical page

For comparison, the DEC Alpha has one fewer level of page tables to 
decode up to 43 bits (8192TB) of virtual memory, using 8K pages:

63        43 42       33 32       23 22       13 12                0
--------------------------------------------------------------------
|  unused   |  Level 1  |  Level 2  |  Level 3  | Offset w/in page |
--------------------------------------------------------------------

Jake

Re: [Syllable-kernel] AFS corruption with SMP disabled..

[Arno K. <arn...@ya...>]

Jake Hamby schrieb:

>
>
> One scary thing I've noticed is that after there has been heavy disk 
> activity, "sync" will often spend several seconds writing out data 
> even if the system has been idle for a long time since the last disk 
> access.  In other words, there doesn't seem to be any "buffer flush" 
> thread writing out dirty buffers to the disk on a regular basis.  This 
> could definitely be making AFS corruption issues more severe than they 
> otherwise might be.
>
There is a flush thread in bcache.c but it uses quite long snooze times.

Arno

[Syllable-kernel] AFS corruption with SMP disabled..

[Jake H. <jh...@po...>]

Now that Syllable can run the DejaGnu test framework (requires kernel 
build 0009 plus some glibc patches which I've sent to Vanders), I tried 
running the GCC 3.4.3 testsuite, which gives the system a pretty good 
workout.  Unfortunately, after 15 minutes or so, the system locked up, 
and I got these AFS related messages on the next boot.  Now when I tail 
/var/log/kernel, the filesystem seizes up and I can't launch any new 
processes.  Time to restore the partition...

I plan to take a break from low-level kernel hacking for a little while 
in order to study AFS, with the goal (besides fixing any bugs I find!) 
of writing an fsck utility so that we don't have to keep reinstalling/ 
restoring Syllable when these things happen.  Has anyone started to 
write anything along those lines?

Anyway, I wanted to let y'all know that it IS possible to screw up the 
filesystem through heavy activity, even with SMP disabled (although it's 
easier with SMP enabled).  At the time of the crash, it had been doing 
CPU-intensive compiles, but nothing disk intensive.  However, during the 
same session I untarred the GCC 3.4.3 source tree and bootstrapped it, 
so there had been major disk activity at some point prior to the crash.

One scary thing I've noticed is that after there has been heavy disk 
activity, "sync" will often spend several seconds writing out data even 
if the system has been idle for a long time since the last disk access. 
  In other words, there doesn't seem to be any "buffer flush" thread 
writing out dirty buffers to the disk on a regular basis.  This could 
definitely be making AFS corruption issues more severe than they 
otherwise might be.

-Jake

Re: [Syllable-kernel] DLIST -> tailqueue patch

[Jake H. <jh...@po...>]

Daniel Gryniewicz wrote:
> Hi, all.
> 
> I'm back from vacation, and have some time to work on Syllable.
> 
> Attached is a patch against current CVS to turn the DLIST into a tail
> queue.  This allows DLIST_ADDTAIL() and DLIST_LAST() at the cost of a
> pointer in the head (negligible), and the requirement of the head
> pointer when removing or appending.

I looked at your patch and everything seems correct, except that 
DLIST_ON_LIST() should test list_pprev for NULL instead of list_next, 
which could be NULL if the entry is the last item on the list.  For this 
to work, DLIST_REMOVE() needs to reset list_pprev to NULL.  I've 
attached my modified version of your patch.

Unfortunately, when I try to build a kernel with the updated list 
macros, it freezes at the very beginning of the boot sequence, 
immediately after the "do_lock_semaphore_ex( -1 ) called with invalid 
semaphore id!!" line.  So far, I have no idea why this is happening.

-Jake

Re: [Syllable-kernel] Kernel land status..

[Brent P. N. <me...@ot...>]

Quoting Jake Hamby <jh...@po...>:

> So I am going to start writing some unit tests for the kernel that could 
> be optionally compiled to run while the kernel is booting to test the 
> various features.

YAAAAAAAY!

::Brent dances in the streets::

I'm a big believer in unit testing.  I'm thrilled to see it being implemented
here.  This could make our lives much, much easier.

-- 
Brent P. Newhall
http://brent.other-space.com/

----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.

[Syllable-kernel] DLIST -> tailqueue patch

[Daniel G. <da...@fp...>]

Hi, all.

I'm back from vacation, and have some time to work on Syllable.

Attached is a patch against current CVS to turn the DLIST into a tail
queue.  This allows DLIST_ADDTAIL() and DLIST_LAST() at the cost of a
pointer in the head (negligible), and the requirement of the head
pointer when removing or appending.

Daniel

Re: [Syllable-kernel] Kernel land status..

[Saem G. <sae...@gm...>]

On Tue, 22 Feb 2005 12:49:21 -0800, Jake Hamby <jh...@po...> wrote:
> The other link I found was to an interesting project called NetKernel,
> which is a microkernel developed for running web services.  There may be
> one or two concepts that we could integrate into Syllable, for example
> using URIs to address software components:
> 
> http://www.1060research.com/netkernel/
> 
> -Jake

IIRC, zeroinstall does something like this for package management.  It
makes a lot of sense actually, furthermore, I posted a while back on
the Syllable forums about zero-install, I was told that there were
plans to either port it or bring something very similliar in to the
mix for Syllable.

Re: [Syllable-kernel] Kernel land status..

[Kristian V. D. V. <va...@li...>]

On Tuesday 22 February 2005 20:49, Jake Hamby wrote:
> Here's a quick heads-up on my kernel progress.  Yesterday I integrated
> Arno's patches and now I can boot in SMP mode on my P4.  Whoohoo!  The
> bad news is that I'm still getting random lockups every now and then,
> but that's been happening for a while, SMP or no.

Cool :)  The lockups seemed to be worse with the syncronisation primitives 
removed but it's hard to tell.  Obviously we should be aiming for no know 
lock ups no matter what is currently causing them, though!

> My next check-in will be Arno's patches plus commenting out the
> microsecond timer code and the beginnings of the C99 stuff.  I'm pulling
> in the C99 code one file at a time so that I can figure out which one is
> causing the early crash.  The size of the diffs wasn't as big as I was
> afraid of, so I'll probably check in all the rest of the C99/typesafety
> patches in one step, after I verify that I can rebuild all the drivers
> and filesystems, as well as libsyllable and the appserver and run them
> w/o any problems or crashes.

I'd like to hold off on the C99 changes until after 0.5.6, if possible.  It's 
probably best to avoid too many changes all at once, as it may make life 
difficult for us if we need to track down any remaining SMP or lockup bugs.

To the best of my knowledge the appserver currently doesn't build with Gcc 
3.4.3 by the way, although Henrik may have found & fixed this by now.

> There is still one problem with GCC 3.4.3 which is that it occasionally
> generates code which requires memcpy(), which is only available as an
> inline function in kernel land.  If you don't add "-ffreestanding" then
> it sometimes also requires strcpy().

We should modify Gcc 3.4 so that -kernel implies -ffreestanding

> This didn't affect any of the 
> filesystems and only affected one driver (emu10k1) so I just added a
> "gcc34_memcpy.c" file to that build like I did for the kernel.  But the
> -ffreestanding flag is supposed to prevent this kind of code from being
> generated so I would like to know whether it's a bug in GCC itself or in
> our port specifically.

There is nothing about the Gcc port for Syllable that would affect this.  Our 
port just adds configuration files so Gcc builds correctly, but no source 
files are patched.  I had heard of issues with Gcc 3.4 so perhaps it is a 
bug; it may be worth searching GNATS for Gcc?

> So I am going to start writing some unit tests for the kernel that could
> be optionally compiled to run while the kernel is booting to test the
> various features.  I'm also going to add a kassertp() macro to panic if
> the assertion fails, as kassertw() just gives a warning and continues,
> which may not be a good idea if the assertion failure indicates some
> sort of data structure corruption.

Sounds like an excelent idea to me.  kassertp() is probably long overdue, too.

-- 
Vanders (If this email makes it through..)
http://www.syllable.org
http://www.liqwyd.com

[Syllable-kernel] Kernel land status..

[Jake H. <jh...@po...>]

Here's a quick heads-up on my kernel progress.  Yesterday I integrated 
Arno's patches and now I can boot in SMP mode on my P4.  Whoohoo!  The 
bad news is that I'm still getting random lockups every now and then, 
but that's been happening for a while, SMP or no.

My next check-in will be Arno's patches plus commenting out the 
microsecond timer code and the beginnings of the C99 stuff.  I'm pulling 
in the C99 code one file at a time so that I can figure out which one is 
causing the early crash.  The size of the diffs wasn't as big as I was 
afraid of, so I'll probably check in all the rest of the C99/typesafety 
patches in one step, after I verify that I can rebuild all the drivers 
and filesystems, as well as libsyllable and the appserver and run them 
w/o any problems or crashes.

There is still one problem with GCC 3.4.3 which is that it occasionally 
generates code which requires memcpy(), which is only available as an 
inline function in kernel land.  If you don't add "-ffreestanding" then 
it sometimes also requires strcpy().  This didn't affect any of the 
filesystems and only affected one driver (emu10k1) so I just added a 
"gcc34_memcpy.c" file to that build like I did for the kernel.  But the 
-ffreestanding flag is supposed to prevent this kind of code from being 
generated so I would like to know whether it's a bug in GCC itself or in 
our port specifically.

In other news:  two links I found interesting.  First, I started reading 
an O'Reilly book on unit testing and it mentioned this clever example of 
a unit testing framework for C programs in just three lines of code:

http://www.jera.com/techinfo/jtns/jtn002.html

So I am going to start writing some unit tests for the kernel that could 
be optionally compiled to run while the kernel is booting to test the 
various features.  I'm also going to add a kassertp() macro to panic if 
the assertion fails, as kassertw() just gives a warning and continues, 
which may not be a good idea if the assertion failure indicates some 
sort of data structure corruption.

The other link I found was to an interesting project called NetKernel, 
which is a microkernel developed for running web services.  There may be 
one or two concepts that we could integrate into Syllable, for example 
using URIs to address software components:

http://www.1060research.com/netkernel/

-Jake

Re: [Syllable-kernel] ping..

[Daniel G. <da...@fp...>]

On Wed, 2005-02-16 at 19:15 -0800, Jake Hamby wrote:

>If you're running your SMTP server from a cable modem or DSL connection, 
>then it's possible that sourceforge was blocking your entire IP subnet 
>for anti-spam purposes.  I remember the freebsd.org server was 
>particularly rude about blocking me from sending mail using my own SMTP 
>server.  But since it's working now for you, who knows?A

It's DSL, but it's a static, with reverse, and it went away after an
hour or so my guess is it was a transient issue with
lists.sourceforge.net.

>I've been meaning to switch over the server I use for mail from sendmail 
>and uw-imap to postfix and courier imap, but haven't gotten around to it 
>yet.  If you're just starting out, I would only suggest to avoid 
>sendmail because it is a maintenance nightmare!!  Also, while all of the 
>other mail servers support the more efficient Maildir format, sendmail 
>is still putting mail into one big file per user.  I found an 
>interesting sourceforge page describing a complete e-mail solution based 
>on open source that recommended postfix and Courier-IMAP, especially if 
>you are using procmail (which is pretty nice once you understand how to 
>set up the .procmailrc).  Spamassassin is also a must-have and works 
>well with procmail.

I use qmail/courier/spamassassin/clamav.  It's a nice setup.  I tried
postfix/courier/spamassassin/clamav, but I couldn't get postfix working,
and there's a really nice HOWTO for the setup I have on Gentoo.

Daniel

Re: [Syllable-kernel] ping..

[Jake H. <jh...@po...>]

Daniel Gryniewicz wrote:
> 
> My mail server may be flaky (although I hope not, and I was getting mail
> constantly during the time period...), but the error I got from
> sourceforge was definitely their fault:
> 
> <syl...@li...>:
> 66.35.250.206 does not like recipient.
> Remote host said: 550 Administrative prohibition
> Giving up on 66.35.250.206.
> 
> 550 is unknown recipient, so it was refusing mail *destined* to
> syllable-kernel, not *from* my mail server.  Unless, I suppose, they
> lied about the error.  I'm fairly new to this whole mail server thing,
> so I suppose I could have messed something up on my end.

If you're running your SMTP server from a cable modem or DSL connection, 
then it's possible that sourceforge was blocking your entire IP subnet 
for anti-spam purposes.  I remember the freebsd.org server was 
particularly rude about blocking me from sending mail using my own SMTP 
server.  But since it's working now for you, who knows?

I've been meaning to switch over the server I use for mail from sendmail 
and uw-imap to postfix and courier imap, but haven't gotten around to it 
yet.  If you're just starting out, I would only suggest to avoid 
sendmail because it is a maintenance nightmare!!  Also, while all of the 
other mail servers support the more efficient Maildir format, sendmail 
is still putting mail into one big file per user.  I found an 
interesting sourceforge page describing a complete e-mail solution based 
on open source that recommended postfix and Courier-IMAP, especially if 
you are using procmail (which is pretty nice once you understand how to 
set up the .procmailrc).  Spamassassin is also a must-have and works 
well with procmail.

http://jamm.sourceforge.net/howto/html/architecture.html

-Jake

Re: [Syllable-kernel] ping..

[Daniel G. <da...@fp...>]

On Wed, 2005-02-16 at 18:05 -0800, Jake Hamby wrote:
>Daniel Gryniewicz wrote:
>> On Wed, 2005-02-16 at 16:01 -0800, Jake Hamby wrote:
>> 
>>>Testing to see if the list server is still down?
>>>
>>>-Jake
>> 
>> 
>> pong?
>
>P.S.  It must be your mail server that's flaky.. the e-mail I sent to 
>your account is timing out:
>
>----- Transcript of session follows -----
><da...@fp...>... Deferred: Connection timed out with 
>nemesis.fprintf.net.
>Warning: message still undelivered after 4 hours
>Will keep trying until message is 5 days old
>
>

My mail server may be flaky (although I hope not, and I was getting mail
constantly during the time period...), but the error I got from
sourceforge was definitely their fault:

<syl...@li...>:
66.35.250.206 does not like recipient.
Remote host said: 550 Administrative prohibition
Giving up on 66.35.250.206.

550 is unknown recipient, so it was refusing mail *destined* to
syllable-kernel, not *from* my mail server.  Unless, I suppose, they
lied about the error.  I'm fairly new to this whole mail server thing,
so I suppose I could have messed something up on my end.

Daniel

Re: [Syllable-kernel] ping..

[Jake H. <jh...@po...>]

Daniel Gryniewicz wrote:
> On Wed, 2005-02-16 at 16:01 -0800, Jake Hamby wrote:
> 
>>Testing to see if the list server is still down?
>>
>>-Jake
> 
> 
> pong?

P.S.  It must be your mail server that's flaky.. the e-mail I sent to 
your account is timing out:

----- Transcript of session follows -----
<da...@fp...>... Deferred: Connection timed out with 
nemesis.fprintf.net.
Warning: message still undelivered after 4 hours
Will keep trying until message is 5 days old


-Jake

Re: [Syllable-kernel] C99 kernel source preview - needs more work

[Jake H. <jh...@po...>]

Daniel Gryniewicz wrote:
> 
> Sounds good, I'll take a look if I get a chance.  I'm going on vacation
> next week, tho, so don't hold your breath...

No problem.  My reason for posting now was simply to get a "sanity 
check" and make sure I'm not way off target or trying to do something 
crazy.  I've invested quite a few hours into this already, and I have 
quite a few ahead (I deliberately said I was 90% complete knowing the 
old adage that the last 10% takes 90% of the time!), and a simple 
"sounds good" from someone else is a great motivator for me to continue 
compared to working completely in a vacuum with no feedback.

> I've locally modified the spin_lock() setup to automatically
> save/restore interrupt masks, because it's almost always done that way,
> but that can wait.

Yeah.  It's probably good to combine those two functions, b/c even if 
interrupts are already disabled, cli() and put_cpu_flags() can safely be 
nested.  I'm just not very keen on the whole concept of having to 
disable interrupts all the time for extended periods of time, although 
having SMP or HT will alleviate the performance hit somewhat. 
Ultimately, I would like for us to have super fine-grained locking with 
real-time capabilities, like Solaris has, but I don't even understand 
how our existing semaphore code works, much less the reader/writer locks 
that the kernel exports to drivers but that aren't being used at all 
internally.  Once I've finished reading sema.c and can grok what's going 
on and document it, I suspect we'll be able to get rid of 90% of the 
cases where we're currently using spinlocks by switching over to 
semaphores and r/w locks, or possibly the seqlock primitive I copied 
from Linux.

The spinlock problem really bit me when I added the timer offset code to 
get_system_time() b/c it's called a lot internally and disables 
interrupts to acquire the PIT timer spinlock.  Until we have a proper 
high-performance timer (I'll probably copy the Linux code which supports 
several types of timers and chooses the best one at boot-time), I'll 
probably hack this by reverting the kernel version to 1ms accuracy and 
returning the more accurate result in the syscall version.  I assume 
that in the cases where the kernel gets the time (mostly in bcache.c, 
sema.c, and in the TCP/IP stack), it doesn't need to be more accurate 
than it was before or else it would never have worked!

>>/**
>> * Description of function.
>> * \param nFoo param 1
>> * \return the answer to life, the universe, and everything (42).
>> * \pre precondition
>> * \post postcondition
>> * \invariant pre- and post- and in-between condition :-)
>> */
> 
> I've added to this, and it might be generally useful:
>  * \par Locks Required:
>  * \par Locks Taken:
> 
> Then again, it might not, but the scheduler locks all over the place, so
> I think it's usefull there.

Yes, definitely!  This is a big learning experience for me and I want to 
leverage my understanding of each function by documenting its 
assumptions and requirements.  In the short run, we can do a certain 
amount of static analysis by reading the comments and seeing what calls 
what; in the long run, if we follow a consistent notation, we could even 
do an automatic parsing of the code and a full static analysis of the 
call graph and locking requirements.  See, e.g. _Code Generation in 
Action_ by Jack Herrington, a great book with a lot of examples written 
in Ruby.

>>I also have a question for Daniel:  I would like to use the DLIST_ 
>>macros in more places but unfortunately your version doesn't have a 
>>pointer to the tail of the list so there's no quick way to append an 
>>item to the end of a list or get the last item, operations which are 
>>needed by most of the kernel's lists.  Do you have any plans for a new 
>>version of dlist.h that might address this?  If you can add that 
>>functionality to DLIST, then I'll take care of patching up the rest of 
>>the kernel to use it.
> 
> 
> It could definitely be done.  I have a version that does that locally.
> However, there are two ways to do it, and I want an opinion on which to
> use.  The first way is to just make the DLIST macros be a tail-queue.
> The second is to introduce a second set of DLIST macros that are a
> tail-queue, and keep the old ones unchanged.  The problem with a tail
> queue is that you need the head pointer more often (in particular,
> append and remove both need the head pointer in the tail-queue version,
> but not in the current version).  Opinions?

My inclination would be for correctness over optimization, and 
optimization at a gross level before we start fine-tuning.  Right now, 
DLIST is O(n) for appending or removing from the tail, while a 
tail-queue would be O(1), and with only a constant additional overhead 
for the other operations.  What I don't know is whether the additional 
use of the head pointer would screw things up from a caching or SMP 
perspective since it would be one additional cache line being modified 
every time.

But since we aren't at the stage for that type of low-level tuning to be 
productive (too many things in flux), I would argue for the single set 
of DLIST macros to be a tail-queue.  By supporting both kinds of lists, 
we would introduce two problems:  1) for the cases where the head-list 
is used, we will be "locked in" and perhaps miss the opportunity to try 
different algorithms or optimizations requiring tail access out of a 
desire to avoid switching over to the other kind of list, and 2) the 
confusion to others and possible introduction of bugs that could result 
from having two different sets of objects and primitives that do, 
essentially, the same thing.

-Jake