bproc-users

[BProc] bproc+highmem? (kmap_pagetable)

[<ha...@no...>]

Is it hard to reconcile bproc with highmem ?

I found J.A. Magallon's patch collection 2.4.20-pre10-jam1 and wanted
to use it on Abit-IT7-MAX2 based cluster (patches contain both HPT374
support and bproc kernel patch; this made me happy). 

I did few replacements in bproc (using J.A.M.'s hints):

   nice = current->nice  ----->  nice = task_nice(current)
   current->nice = nice  ----->  set_user_nice(current,nice)
               DEF_NICE  ----->  (0) 

and bproc-3.2.2 compiled OK. However depmod complains about unresolved
symbol kmap_pagetable in vmadump.o.

This is caused by new highmem (1G and more memory support) and there
already was lot of buzz regarding many drivers broken by highmem.

There seem to be quick 'fixes' like replacing #include <asm/pgtable.h>
with #include <linux/highmem.h>, but there is more to it:

Andrea Arcangeli May 4 2002:

> You should #include <linux/highmem.h> in those drivers .c files, then it
> will compile, but that's not the right fix, you'd need to add the
> pte_kunmap too or it would deadlock with highmem. The right fix is to
> convert those drivers to vmalloc_to_page, then they will work flawlessy.
> Alan actually has a patch in his -ac that converted most usb and other
> drivers to vmalloc_to_page, I will merge it plus I will convert those
> below drivers if they're not just covered by Alan's patch. Alan could
> you push it to Marcelo?

So, my question is - what does this meen for bproc?

Adding <linux/highmem.h> somewhere, (complicated?) conversion to
vmalloc_to_page or something more?

1G or more memory is certainly reasonable on clusters these days, what
should be done to have bproc working with these?

Or did I miss something and bproc can work with 1Gup without highmem?

Best Regards

Vaclav

Re: [BProc] bproc+highmem? (kmap_pagetable)

[<ha...@no...>]

> Is it hard to reconcile bproc with highmem ?
>
> I found J.A. Magallon's patch collection 2.4.20-pre10-jam1...

Grrrrrrr, just recompiled everything with highmem off and still have
unresolved kmap_pagetable in vmadump.o.

Definitely I screwed something up - maybe my thoughts in previos post?

How is J.A. Magallon's bproc compile ;-) ??

Regards

Vaclav

Re: [BProc] bproc+highmem? (kmap_pagetable)

[<jam...@ab...>]

On 2002.10.30 ha...@no... wrote:
> > Is it hard to reconcile bproc with highmem ?
> >
> > I found J.A. Magallon's patch collection 2.4.20-pre10-jam1...
> 
> Grrrrrrr, just recompiled everything with highmem off and still have
> unresolved kmap_pagetable in vmadump.o.
> 
> Definitely I screwed something up - maybe my thoughts in previos post?
> 
> How is J.A. Magallon's bproc compile ;-) ??
> 

Beacuse I added also #include <hihgmem.h> in vmadump.c. ;)
But I had not posted the corrected bproc sources in the same site, sorry
(task changes, highmen, comment out the scheduling in slave.c...).
I will look at what Andrea says in that mail, also...thanks for the
pointer.

I would not trust too much on my tree wrt bproc (yes, it compiles and
seems to work). We have an older version running in a system used to
solve FEM problems with MPI, and runs well. But as I want the updated VM in
-aa tree, and -aa is not supported by bproc, my kernel will always be
a monster...

BTW, I have just installed 2.4.20-rc1-jam0 (with bproc-3.2.2) and seems to
work. When I can test is seriously I will 'officially' release jam1 and
the modified userspace tree.

Erik, will you admit a patch in the line of

#ifdef NEW_O1_SCHED
	set_user_nice(xxxx)
#else
	current-> nice = xxxx;
#end

TIA


-- 
J.A. Magallon <jam...@ab...>      \                 Software is like sex:
werewolf.able.es                         \           It's better when it's free
Mandrake Linux release 9.1 (Cooker) for i586
Linux 2.4.20-rc1-jam0 (gcc 3.2 (Mandrake Linux 9.0 3.2-2mdk))

Re: [BProc] bproc+highmem? (kmap_pagetable)

[<er...@he...>]

On Wed, Oct 30, 2002 at 05:20:22PM +0100, ha...@no... wrote:
> Is it hard to reconcile bproc with highmem ?
> 
> I found J.A. Magallon's patch collection 2.4.20-pre10-jam1 and wanted
> to use it on Abit-IT7-MAX2 based cluster (patches contain both HPT374
> support and bproc kernel patch; this made me happy). 
> 
> I did few replacements in bproc (using J.A.M.'s hints):
> 
>    nice = current->nice  ----->  nice = task_nice(current)
>    current->nice = nice  ----->  set_user_nice(current,nice)
>                DEF_NICE  ----->  (0) 
> 
> and bproc-3.2.2 compiled OK. However depmod complains about unresolved
> symbol kmap_pagetable in vmadump.o.
> 
> This is caused by new highmem (1G and more memory support) and there
> already was lot of buzz regarding many drivers broken by highmem.

The 2.4.19 highmem is a non-issue for BProc.  I've been running boxes
with >= 1GB of ram and highmem turned on for about a year and I
haven't seen any problems.

What I've seen of highmem so far indicates that it's only an issue
when one thread wants to access user space memory from another thread.
You can see some complication in ptrace as a result of this.  As long
as you're operating on "current" there are no changes.  Vmadump always
operates on "current" so I didn't have to do anything there.  The same
is true for BProc.

If all of that gets changed around in 2.4.20, then I guess I'll have
to revisit it at that point.

As far as other kernel patches are concerned.... I unfortunately don't
have the time to worry about working with anything other than the
stock kernel.

- Erik