linuxsh-dev

[linuxsh-dev] Different old_mmap behavior in 2.4.5 and 2.4.8

[Bao C. H. <ba...@se...>]

We are moving from kernel 2.4.5 to kernel 2.4.8 and above.
One of our applications broke due to different behaviors
of the system call old_mmap.

In kernel 2.4.5:
307   old_mmap(0x7b7f7000, 36864, PROT_READ|PROT_WRITE|PROT_EXEC,
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7b7f7000

In kernel 2.4.8:
[pid   313] old_mmap(0x7b7f7000, 36864, PROT_READ|PROT_WRITE|PROT_EXEC,
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7b7f8000

In 2.4.5, we request 0x7b7f7000 and get the same area back.
In 2.4.8, we also request 0x7b7f7000, but we are getting a
different area pointed by 0x7b7f8000.

Is this supposed to be the correct area?  What changes make
the newer kernels to return different pointers?

Appreciate any pointers/suggestions.

Thanks.
Bao

Re: [linuxsh-dev] Different old_mmap behavior in 2.4.5 and 2.4.8

[David W. <dw...@in...>]

ba...@se... said:
> One of our applications broke due to different behaviors of the system
> call old_mmap. 

You didn't specify MAP_FIXED. If the application relied on getting back the 
address it asked for then it's broken, as you discovered.

>  Is this supposed to be the correct area?  What changes make the newer
> kernels to return different pointers? 

Looking at the numbers, I'd be inclined to guess that it's moved due to
cache aliasing. And hence that it's not very likely to get 'fixed'. 

--
dwmw2

RE: [linuxsh-dev] Different old_mmap behavior in 2.4.5 and 2.4.8

[Bao C. H. <ba...@se...>]

> 
> >  Is this supposed to be the correct area?  What changes 
> make the newer
> > kernels to return different pointers? 
> 
> Looking at the numbers, I'd be inclined to guess that it's 
> moved due to
> cache aliasing. And hence that it's not very likely to get 'fixed'. 
> 
Either the sh4 kernel has to be fixed, or linuxthread has to be fixed.
MAP_FIXED was removed from linuxthreads sometimes earlier this year.
Any serious applications using linuxthreads will exhibit the same
problem.

Bao

RE: [linuxsh-dev] Different old_mmap behavior in 2.4.5 and 2.4.8

[Bao C. H. <ba...@se...>]

I am sending this message also to Ulrich Drepper, since
the ChangeLog indicated that he removed the MAP_FIXED in
the following.  It is causing problems running pthread
applications in the sh4-linux environment.

Following is the relevant segment of linuxthreads that is
broken:

In function pthread_allocate_stack(), file manager.c:
.......
#  ifdef _STACK_GROWS_DOWN
      new_thread = default_new_thread;
      new_thread_bottom = (char *) (new_thread + 1) - stacksize;
      map_addr = new_thread_bottom - guardsize;
      res_addr = mmap(map_addr, stacksize + guardsize,
                      PROT_READ | PROT_WRITE | PROT_EXEC,
                      MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
      if (res_addr != map_addr)
        {
          /* Bad luck, this segment is already mapped. */
          if (res_addr != MAP_FAILED)
            munmap (res_addr, stacksize + guardsize);
          return -1;
        }
.......

We resort to patching the MAP_FIXED back to linuxthreads
until we get a resolution on this problem.

Thanks.
Bao

> -----Original Message-----
> From: lin...@li...
> [mailto:lin...@li...]On Behalf Of Bao C. Ha
> Sent: Tuesday, September 11, 2001 9:55 AM
> To: 'David Woodhouse'
> Cc: lin...@li...; lin...@m1...
> Subject: RE: [linuxsh-dev] Different old_mmap behavior in 2.4.5 and
> 2.4.8 
> 
> 
> 
> > 
> > >  Is this supposed to be the correct area?  What changes 
> > make the newer
> > > kernels to return different pointers? 
> > 
> > Looking at the numbers, I'd be inclined to guess that it's 
> > moved due to
> > cache aliasing. And hence that it's not very likely to get 'fixed'. 
> > 
> Either the sh4 kernel has to be fixed, or linuxthread has to be fixed.
> MAP_FIXED was removed from linuxthreads sometimes earlier this year.
> Any serious applications using linuxthreads will exhibit the same
> problem.
> 
> Bao
> 
> _______________________________________________
> linuxsh-dev mailing list
> lin...@li...
> https://lists.sourceforge.net/lists/listinfo/linuxsh-dev
> 

Re: [linuxsh-dev] Different old_mmap behavior in 2.4.5 and 2.4.8

[Ulrich D. <dr...@re...>]

"Bao C. Ha" <ba...@se...> writes:

> I am sending this message also to Ulrich Drepper, since
> the ChangeLog indicated that he removed the MAP_FIXED in
> the following.  It is causing problems running pthread
> applications in the sh4-linux environment.

Then fix the kernel.  The change is necessary and correct.

-- 
---------------.                          ,-.   1325 Chesapeake Terrace
Ulrich Drepper  \    ,-------------------'   \  Sunnyvale, CA 94089 USA
Red Hat          `--' drepper at redhat.com   `------------------------

Re: [linuxsh-dev] Different old_mmap behavior in 2.4.5 and 2.4.8

[David W. <dw...@in...>]

dr...@re... said:
>  Then fix the kernel.  The change is necessary and correct. 

I believe the change was made for cache aliasing reasons - on SH4 we have a 
virtually-indexed cache with 8KiB-worth of entries, and we avoid having 
alias problems by allowing only even pages to be mapped at even addresses, 
and odd pages at odd addresses.

I haven't checked whether the kernel will make the offending mmap fail if 
you specify MAP_FIXED, or whether it'll fall back into cache-flushing mode, 
map the range uncached or deal with it some other way to give you the 
address you asked for. But surely unless you specify MAP_FIXED, the kernel 
is perfectly entitled to fix up the address to ensure cache sanity?

Why was it necessary to remove MAP_FIXED?

--
dwmw2

Re: [linuxsh-dev] Different old_mmap behavior in 2.4.5 and 2.4.8

[David W. <dw...@in...>]

dr...@re... said:
>  Then fix the kernel.  The change is necessary and correct.

I suppose it is feasible for the kernel to accommodate MAP_ANONYMOUS mmap at
the 'wrong' page addresses. Whether it _should_ do that or not sort of 
depends on how good the reasons are for not specifying MAP_FIXED in 
libpthread when you obviously seem to have meant it that way.

I think a combination of mmap(MAP_ANONYMOUS|MAP_SHARED); fork();
mremap(MAP_FIXED) may still get the SH4 to screw up, but I think that was
the case anyway. Mremap seems to be fundamentally broken in that respect - it
doesn't have the same hooks (arch_get_unmapped_area()) that mmap() has to
allow the arch code to vet the addresses use.

I'll leave that one as an exercise for the reader.

Index: arch/sh/kernel/sys_sh.c
===================================================================
RCS file: /cvsroot/linuxsh/kernel/arch/sh/kernel/sys_sh.c,v
retrieving revision 1.7
diff -u -r1.7 sys_sh.c
--- arch/sh/kernel/sys_sh.c	2001/08/03 23:50:59	1.7
+++ arch/sh/kernel/sys_sh.c	2001/09/11 21:39:00
@@ -68,7 +68,9 @@
 	if (!addr)
 		addr = TASK_UNMAPPED_BASE;
 
-	addr = COLOUR_ALIGN(addr);
+	/* We don't need to do the cache colouring for anonymous pages */
+	if (filp)
+		addr = COLOUR_ALIGN(addr);
 
 	for (vma = find_vma(current->mm, addr); ; vma = vma->vm_next) {
 		/* At this point:  (!vma || addr < vma->vm_end). */
@@ -77,7 +79,8 @@
 		if (!vma || addr + len <= vma->vm_start)
 			return addr;
 		addr = vma->vm_end;
-		addr = COLOUR_ALIGN(addr);
+		if (filp)
+			addr = COLOUR_ALIGN(addr);
 	}
 }
 #endif


--
dwmw2

Re: [linuxsh-dev] Different old_mmap behavior in 2.4.5 and 2.4.8

[NIIBE Y. <gn...@m1...>]

My bad.

Bao, please try version 1.6 (former version) of arch/sh/kernel/sys_sh.c.

It did checks if the page is shared or not.  It does COLOUR_ALIGN the
address, only if it's shared.

At that time, I did find always COLOUR_ALIGN makes the performance
better (because of good cache hit), and didn't know it breaks
semantics and some applications.

I'll revert the change of following:

2001-08-03  NIIBE Yutaka  <gn...@m1...>

	* arch/sh/kernel/sys_sh.c (arch_get_unmapped_area): Always align
	to 16KB.
-- 

RE: [linux-sh:01934] Re: [linuxsh-dev] Different old_mmap behavior in 2.4.5 and 2.4.8

[Bao C. H. <ba...@se...>]

Hi NIIBE,

Thank you.  We will try them.

Hi Ulrich,

Sorry to get you into this.  It looks like we are getting
a satisfactory resolution of consistent kernel behaviors
across platforms now.

Appreciate everybody's help and insight/suggestion.

Regards.
Bao

> -----Original Message-----
> From: NIIBE Yutaka [mailto:gn...@m1...]
> Sent: Tuesday, September 11, 2001 5:42 PM
> To: Ulrich Drepper; Bao C. Ha
> Cc: lin...@li...; lin...@m1...
> Subject: [linux-sh:01934] Re: [linuxsh-dev] Different 
> old_mmap behavior
> in 2.4.5 and 2.4.8
> 
> 
> My bad.
> 
> Bao, please try version 1.6 (former version) of 
> arch/sh/kernel/sys_sh.c.
> 
> It did checks if the page is shared or not.  It does COLOUR_ALIGN the
> address, only if it's shared.
> 
> At that time, I did find always COLOUR_ALIGN makes the performance
> better (because of good cache hit), and didn't know it breaks
> semantics and some applications.
> 
> I'll revert the change of following:
> 
> 2001-08-03  NIIBE Yutaka  <gn...@m1...>
> 
> 	* arch/sh/kernel/sys_sh.c (arch_get_unmapped_area): Always align
> 	to 16KB.
> -- 
> 

[linuxsh-dev] help

[vijay <vi...@kp...>]

hi all,

I understand that this is newbie question.
still i ahve to ask as i am really stuck up..
can any body give me the links to documnets available..

about the infomation of linuxSh...
or any info..
from whre to start.. to understand the linux kernel.. and configuring it
accoording to
sh  target..
(i am familiar with sh processors.. and linux as a end user..)
regards,
vijay

RE: [linuxsh-dev] help

[Bao C. H. <ba...@se...>]

http://linuxsh.sourceforge.net/docs/building-source.php3

Everything is on the homepage of LinuxSH,
http://linuxsh.sourceforge.net.

Bao


> -----Original Message-----
> From: lin...@li...
> [mailto:lin...@li...]On Behalf Of vijay
> Sent: Thursday, September 13, 2001 2:37 AM
> To: NIIBE Yutaka; Ulrich Drepper; Bao C. Ha;
> lin...@li...; lin...@m1...
> Subject: [linuxsh-dev] help
> 
> 
> hi all,
> 
> I understand that this is newbie question.
> still i ahve to ask as i am really stuck up..
> can any body give me the links to documnets available..
> 
> about the infomation of linuxSh...
> or any info..
> from whre to start.. to understand the linux kernel.. and 
> configuring it
> accoording to
> sh  target..
> (i am familiar with sh processors.. and linux as a end user..)
> regards,
> vijay
> 
> 
> _______________________________________________
> linuxsh-dev mailing list
> lin...@li...
> https://lists.sourceforge.net/lists/listinfo/linuxsh-dev
> 

Re: [linuxsh-dev] Different old_mmap behavior in 2.4.5 and 2.4.8

[NIIBE Y. <gn...@m1...>]

We need to implement
	MAP_SHARED: COLOUR_ALIGN
	MAP_PRIVATE: PAGE_ALIGN (Don't COLOUR_ALIGN)
	Others: don't care

I think that it's good to set default to COLOUR_ALIGN, as cache
works well.  Here's a patch.

	* arch/sh/kernel/sys_sh.c (arch_get_unmapped_area): Don't
	COLOUR_ALIGN when it comes with MAP_PRIVATE.

Index: arch/sh/kernel/sys_sh.c
===================================================================
RCS file: /cvsroot/linuxsh/kernel/arch/sh/kernel/sys_sh.c,v
retrieving revision 1.7
diff -u -r1.7 sys_sh.c
--- arch/sh/kernel/sys_sh.c	2001/08/03 23:50:59	1.7
+++ arch/sh/kernel/sys_sh.c	2001/09/13 01:11:11
@@ -68,7 +68,10 @@
 	if (!addr)
 		addr = TASK_UNMAPPED_BASE;
 
-	addr = COLOUR_ALIGN(addr);
+	if (flags & MAP_PRIVATE)
+		addr = PAGE_ALIGN(addr);
+	else
+		addr = COLOUR_ALIGN(addr);
 
 	for (vma = find_vma(current->mm, addr); ; vma = vma->vm_next) {
 		/* At this point:  (!vma || addr < vma->vm_end). */
@@ -77,7 +80,8 @@
 		if (!vma || addr + len <= vma->vm_start)
 			return addr;
 		addr = vma->vm_end;
-		addr = COLOUR_ALIGN(addr);
+		if (!(flags & MAP_PRIVATE))
+			addr = COLOUR_ALIGN(addr);
 	}
 }
 #endif
--