linuxsh-dev

Re[2]: [linuxsh-dev] [RFC] LinuxSH code reorg

[Masahiro A. <m-...@aa...>]

Hello,

On Tue, 31 Jul 2001 21:46:36 -0500
"M. R. Brown" <mr...@0x...> wrote:

> * Masahiro Abe <m-...@aa...> on Wed, Aug 01, 2001:
> 
> > With "Cleanliness", I must admit I'm split-minded. I mean, I'm not with
> > firm opinion on this yet. I can understand your point, but at the same
> > time, "functional but ugly" code may benefit some others who need that
> > feature early. I don't want to debate on this now, but just hope you may
> > understand my uncertain feeling. I'm not a kernel hacker (at least yet,
> > I think) but am an engineer who must release the outcome of my effort at
> > my boss's request 8-< Well, no more personal complaints.
> > 
> 
> Heh, I'm not a kernel hacker yet (I don't think I know enough about kernel
> development yet) either.  But I do want to strive for quality code, and I
> believe that others should at least try to follow the guidelines set forth
> when developing new code.  I do understand the sentiment of getting working
> code out there for everyone to use, but why not spend the extra day or week
> working on the code so that it's better quality?  And while doing that you
> teach yourself more about how the underlying kernel works.

It may not always be possible to spend the extra day, depending on the
situation. I don't say that code quality can be neglected. It should be
high. I may be thinking about the special case...

In some case, bad code can be a starting point of polishment. Oh, this
can be done with "published patch" approach also...

Just my random thoughts, maybe not in a quality for the post ^^;)

=================================
Masahiro ABE, A&D Co., Ltd. Japan

Re: [linuxsh-dev] [RFC] LinuxSH code reorg

[M. R. B. <mr...@0x...>]

* M. R. Brown <mr...@0x...> on Tue, Jul 31, 2001:

> 
> As far as any other special procedures and who will be responsible for
> doing that, well, I know yet :).
> 

Erm, I meant I don't know yet :).

Marcus

[linuxsh-dev] Re: [linux-sh:01916] Re: 2.4.7 unaligned user space access problems

[M. R. B. <mr...@0x...>]

* NIIBE Yutaka <gn...@m1...> on Wed, Aug 01, 2001:

> It seems that you don't like what I did, in last year to this
> April.  OK, it could be understandable.
> 

Ok, I promise I won't bring this up anymore.  I've resolved it w/ myself,
so there's no use in me going over it over and over again.

> Please understand that each person has his own decision.  While you
> and another guy was asking me to release the DC driver, I had friends
> who asked me NOT releasing the driver.  I think that I explained this.
> It was me who decided NOT releasing the driver last year.  This was my
> own decision, as a individual, not as a maintainer.  But I have a
> right not to release the code.
> 

You're right.

> Then, I've been working towards releasing them, and finally with help
> of my friends, we've released.  It took months actually, I spent much
> time to negotiate and make compromise.  I understand that it's quite
> unfortunate for your own purpose, you have complaints.  However, each
> person has his situation to be resolved.  I never intended to confuse
> and bother you.  I did my best, but that was not what you expected, or
> at least not fast enough, clear enough.  Well, finally it's done.
> 

I'm sorry I got so upset as attack you out of character.

> 			*	*	*
> 
> This time, you're quite unfair.  You don't talk on technical bases.
> It seems for me that you currently (temporal-lily) loose a skill to
> read someone's work and accept it or understand related things.
> I hope, someday, you can be good state again and get things straight.

Ok.  I apologize for getting out of hand on this issue.  You're right, my
personal attacks on you are unwarranted, especially since the situation was
outside of my control to begin with.

I will take a closer look at the work you've done, but I'm still under the
impression that there's a better way to implement it, and as you've said
let the code do the talking :).  So I'll stop all the fuss and see if I can
come up with a better implementation :).

Marcus

Re: [linuxsh-dev] [RFC] LinuxSH code reorg

[M. R. B. <mr...@0x...>]

* Masahiro Abe <m-...@aa...> on Wed, Aug 01, 2001:

> I don't know if I'm eligible/qualified to comment, but want to add some
> noise(?) on this issue 8-)
> 

Well, it was addressed to all LinuxSH developers, so that everyone could
voice their opinions.  Please, if you've got an opinion, let's hear it :).

> 
> I like the idea of "drop-in tree" approach, but not sure if it works
> well in current situation. I've seen several files in kernel/ or mm/
> modified for a while, and later that changes go into mainline.
> 

Ah, ok.  I'm sorry NIIBE I think I misunderstood what you wrote in your
reply to the RFC.  As far as these changes to mainline, it's just the issue
of making sure they're synced for every kernel patchlevel, just like you
would the full tree.  If it's a mainline file, like something in kernel/ or
mm/, then it should still be included in the drop-in tree.  As far as any
other special procedures and who will be responsible for doing that, well,
I know yet :).

For example in the ruby tree, there are files that ruby modifies that sit
in other people trees, etc.  Ruby even has an arch/sh/kernel/setup.c :).
The ruby maintainer syncs the ruby drop-in tree with mainline on every
major kernel release, so I would assume we'd do the same thing (or do it
frequently).

> With "Cleanliness", I must admit I'm split-minded. I mean, I'm not with
> firm opinion on this yet. I can understand your point, but at the same
> time, "functional but ugly" code may benefit some others who need that
> feature early. I don't want to debate on this now, but just hope you may
> understand my uncertain feeling. I'm not a kernel hacker (at least yet,
> I think) but am an engineer who must release the outcome of my effort at
> my boss's request 8-< Well, no more personal complaints.
> 

Heh, I'm not a kernel hacker yet (I don't think I know enough about kernel
development yet) either.  But I do want to strive for quality code, and I
believe that others should at least try to follow the guidelines set forth
when developing new code.  I do understand the sentiment of getting working
code out there for everyone to use, but why not spend the extra day or week
working on the code so that it's better quality?  And while doing that you
teach yourself more about how the underlying kernel works.

Marcus

Re: [linuxsh-dev] [RFC] LinuxSH code reorg

[M. R. B. <mr...@0x...>]

* NIIBE Yutaka <gn...@m1...> on Wed, Aug 01, 2001:

> Thanks a lot.  Well written.  I think that change of structure and the
> move towards only maintaining SuperH portion are good idea.  If
> possible, it's good we can adopt such a new way of development for
> 2.5.
> 

Well, I'm willing to be a bit more ambitious and commit to getting the
current tree restructured, not waiting for 2.5.  Sure it will take a lot of
effort, but I think it will be beneficial in the long run.  For August I
have basically unlimited free time to commit to working on this.  I can
come up with a strategy for restructuring, when I get that finished I'll
send it to the list for approval, is that ok?

I can do my work on restructuring the tree in parallel with current LinuxSH
development.  The current LinuxSH code is based in the kernel/ CVS
directory, I can start the restructuring work in a directory called linux/
at the same time.

> 
> (1) "Patch Manager"
> We have "Patch Manager", but it's almost non-functional (no one care
> it).  I think that the future direction is remove using "Patch
> Manager" and require people to send all the changes to review.
> 

I think that's a good idea.  One more thing I wanted to point out but I
forgot: in the mailing list section on the LinuxSH SF website, it mentions
the linux-sh and linux-sh-ja mailing lists, but neglects to mention the
linuxsh-dev and linuxsh-cvs lists.  When I first looked at LinuxSH, I
didn't realize that those lists existed, and I missed a good deal of
discussion that only took place on the linuxsh-dev list.  Could someone
update that section to add those lists?

> (2) Changes done in mainline
> Occasionally, the changes will be done in the mainline source.
> We need to include the changes to our repository.  This'd be a kind
> of special work.  Does it require to send the changes to review?
> Who will do it, and how?  Sometimes we can't identify who does the
> change and don't understand the intention why we need the change.
> 

Well, by using a drop-in tree, we only have to worry about mainline changes
to code that falls under the LinuxSH.  From my understanding, anyone who
attempted to patch LinuxSH from outside the community would have to consult
with you as the LinuxSH maintainer.  That's the advantage of using a
drop-in tree as opposed to including the entire kernel source in CVS - if
there is a "global" mainline change that somehow breaks LinuxSH
functionality, all you have to do is fix that relevant portion in the
drop-in tree - you don't have to patch everything.

It should be easy to pick out individual patches that target LinuxSH code
if that code comes from external sources.  I believe the package named
'patchutils' (in Debian) allows you to pick out individual patches within a
patchset.

> (3) Testing
> It would be good if we can maintain some sort of success/failure table
> with version and/or date.  It is good for tracking bug introduced.
> Performance test is also good.

That's a great idea :).  Based on what you wrote earlier regarding the
xengine performance, I'm eager to build the latest kernel for my Dreamcast
to see how much better it runs compared to 2.4.3 :).

Marcus

Re: [linuxsh-dev] [RFC] LinuxSH code reorg

[Masahiro A. <m-...@aa...>]

I don't know if I'm eligible/qualified to comment, but want to add some
noise(?) on this issue 8-)

I've just read it, and at first glance, basically I'm with your opinion.
I may find some place which I dislike sometime later, but I haven't for
now.

I'm embedded developer, and added code to support for our custom board.
When I did that, I found the file placement in MIPS, and thought about
following that. But as I was lazy, I didn't do what you have done with
this document and just dropped code in existing directories.

I believe your proposal on directory reorganization will benefit many,
if not all.

I like the idea of "drop-in tree" approach, but not sure if it works
well in current situation. I've seen several files in kernel/ or mm/
modified for a while, and later that changes go into mainline.

With "Cleanliness", I must admit I'm split-minded. I mean, I'm not with
firm opinion on this yet. I can understand your point, but at the same
time, "functional but ugly" code may benefit some others who need that
feature early. I don't want to debate on this now, but just hope you may
understand my uncertain feeling. I'm not a kernel hacker (at least yet,
I think) but am an engineer who must release the outcome of my effort at
my boss's request 8-< Well, no more personal complaints.

As a final words, I think your document is constructive, worth
considering for everyone, and can be a good starting point for the
discussion. I hope this discussion will lead us to the new and higher
ground at some time.

=================================
Masahiro ABE, A&D Co., Ltd. Japan

[linuxsh-dev] [RFC] LinuxSH code reorg

[NIIBE Y. <gn...@m1...>]

Thanks a lot.  Well written.  I think that change of structure and the
move towards only maintaining SuperH portion are good idea.  If
possible, it's good we can adopt such a new way of development for
2.5.

I'd like to add some.

(1) "Patch Manager"
We have "Patch Manager", but it's almost non-functional (no one care
it).  I think that the future direction is remove using "Patch
Manager" and require people to send all the changes to review.

(2) Changes done in mainline
Occasionally, the changes will be done in the mainline source.
We need to include the changes to our repository.  This'd be a kind
of special work.  Does it require to send the changes to review?
Who will do it, and how?  Sometimes we can't identify who does the
change and don't understand the intention why we need the change.

(3) Testing
It would be good if we can maintain some sort of success/failure table
with version and/or date.  It is good for tracking bug introduced.
Performance test is also good.
-- 

[linuxsh-dev] [RFC] LinuxSH code reorg

[M. R. B. <mr...@0x...>]

Introduction
============

I've been working with the LinuxSH code for about 10 months now, and in
that time I've come up with a few gripes and ideas concerning the core code
base and the methods developers use to structure and write drivers.  This RFC
attempts to address some of the challenges I've faced writing code for 
LinuxSH, and it hopefully provides some constructive ideas that we can all
use to better the code.  To a lesser extent, it asks about the future of 
LinuxSH, in regards to the imminent release of 2.5.x.

This RFC doesn't cover the specific APIs that would allow for a total code
rewrite across all SH boards/platforms (I'm still working on those :), but
it does talk about driver placement, arch/sh file organization, code reuse,
and developers not being lazy when writing their code.

Some of these ideas are taken from the Linux/MIPS port.  The MIPS port has
thus far been a good model for writing embedded systems code under Linux,
most embedded platforms have homes under this tree.  With the formation of
the linux-mips tree (http://sf.net/projects/linux-mips), their focus is now
on getting additions and patches tested and in the tree in a timely
fashion.  I believe it's beneficial to share concepts and code with MIPS
rather than architectures that aren't traditionally suited for embedded
boards, such as the i386.

Current Situation
=================

Inconsistency
-------------
Looking over the arch/sh subdirectory, I find a number of things disturbing
about the lack of consistency when targetting new boards.  There are two
toplevel board directories, overdrive and stboards.  overdrive appears to
contain code for the support of a single board, as does stboards.  The
remaining boards supported by LinuxSH are in various files in the kernel
directory.  Now, the new-machine.txt document in Documentation/sh does
provide a starting point for adding a new SH board, but it doesn't talk
about where to place other support code for the board, and it fails to
mention the setup_<board>.c convention that the majority of boards follow.

The result of placing board definition and support code in the kernel
directory is that the directory appears cluttered, and this will only
worsen as more boards are added.  This also makes maintenence combersome,
and discourages new developers from hacking on code (I say this because
it's discouraged me a bit :).

Another point of inconsistency was in the addition of the Maple Dreamcast
drivers.  Originally, they were placed in drivers/dreamcast, which
completely goes against standard kernel semantics of driver placement.  The
Maple Bus is an input bus within the Dreamcast, with a variety of
character-style input drivers, such as a mouse, keyboard, or joystick.
Normally, system buses go into a drivers/<bus> subdirectory, and bus
devices go under the respective driver interfaces they support, e.g. the
Maple joystick device driver would live under drivers/char/joystick.

All of the other device drivers that I've seen contributed by LinuxSH
developers have followed the normal convention, so that was probably an
isolated incident.  The misplaced Maple drivers have since been fixed.

Code Cleanliness
----------------
I don't want to take the time to site specific examples, because it would
take a while :).  But overall (just me manually viewing each file) the
source files in arch/sh leave a lot to be desired.  It looks like a lot of
source come from developers who come from embedded backrounds, not
bothering (or not knowing) about the kernel development guidelines.  Of
course, I could be wrong, and it's just laziness.  I'm guilty of it too.

Proposal for Change
===================

Review
------
All code that is submitted to go into LinuxSH should be reviewed by all
developers, where it's assumed that all interested LinuxSH developers are
subscribed to the linuxsh-dev mailing list.  If, after a 1-2 day period no
one objects to the inclusion of the patch, and the submitting developer has
write priveledges, she should feel free to apply the patch.  If someone
does object to the patch, however, then those two developers need to find a
solution, and if that still fails, the group should arbitrate or NIIBE
should step in and make the final call.  The important thing is that *all*
code, no matter how trivial, should be reviewed by the entire community.

Drop-in Tree
------------
While talking to a linux-mips developer while updating the latest revision
of LinuxSH, I came to a realization.  NIIBE spends a significant time
merging changes from upstream back into LinuxSH.  Also, hosting the entire
kernel and updating it is a substantial drain of space and bandwidth,
especially since LinuxSH affects such a small portion of the kernel.
Besides a few device drivers underneath the drivers/ tree, all LinuxSH code
lives under arch/sh and include/asm-sh.

The linux-mips project and ruby project (just two projects I've encountered
that do this) use "drop-in trees".  A drop-in tree only contains the code
relevant to the project, so for LinuxSH, its drop-in tree would look like:

  linux/Documentation/sh/...
  linux/arch/sh/...
  linux/include/asm-sh/...
  linux/drivers/char/sh-sci.[hc]
  linux/drivers/char/ec3104_keyb.c
  linux/drivers/char/maple*
  linux/drivers/char/joystick/maplecontrol.c
  linux/drivers/maple/...

So a developer would download the latest kernel and relevant patches, then
download the LinuxSH drop-in tree.  She runs a script called treelink.sh
that symlinks the files from the drop-in tree on top of the upstream
kernel.  She can then build the kernel for her platform.

This should also make NIIBE's job a lot easier, since he no longer would
have to do branch merges with upstream, but only worry about the overall
"sync" of the LinuxSH tree with the upstream tree.  If there isn't a
kernel-wide change that breaks LinuxSH functionality, then the tree remains
synced.

Drop-in trees also have the benefit of being small (a lot easier to
shuffle than a 136M tree, IMO), and helps developers to be more focused.

No Divergence
-------------
Haven given it minimal thought, it makes no sense for the LinuxDC project
to maintain another divergent tree, especially since the DC is just another
embedded SH board, and SH development is centralized around LinuxSH.  So
all DC drivers will be added directly to the LinuxSH tree.  LinuxDC would
still remain to provide user support, but there's now a central place to
grab and maintain the source.  All SH boards and their drivers should live
within LinuxSH.

Another point about divergence: I've seen/made some changes to the
drivers/net/8139too.c driver so that it can support the Dreamcast's
Broadband Adapter.  If we need to make changes to code that we don't
maintain, we should make the effort to contact the maintainer to ensure that
the code makes it upstream.  If we don't do this, upstream will end up with
shoddy support for our boards, i.e. as it stands the 2.4.7 8139too driver
won't work properly with the DC BBA.

Consistency
-----------
We need to come up with the Standard way of adding new boards to the tree,
while cleaning up the existing mess.  The format that linux-mips finalized
on (and I think it fits just as well with LinuxSH) is this:

- Only processor, kernel, and processor module code should go into
  arch/sh/kernel.  This directory should be "holy ground" - no
  board-specific hacks belong here.

- System controllers, like the hd64465 and hd64461 (please help me out with
  others) that share common code between multiple boards should live in an
  arch/sh/<system controller> directory.  Only boards that share a common
  system controller (does the 7750S fall into this category?  Or is that
  processor-specific code?) should have a system controller placed there.

- Each individual board that LinuxSH runs on should live in
  arch/sh/boards/<board>.  The actual "reference" name of the board should
  be used, not the marketing name.

So, a quick example of the new layout (a lot of files/boards ommitted):

  arch/sh/kernel/
  arch/sh/hd64465/
  arch/sh/hd64461/
  arch/sh/solutionengine/
  arch/sh/boards/hp620/
  arch/sh/boards/hp680/
  arch/sh/boards/hp690/
  arch/sh/boards/dreamcast/
  arch/sh/boards/cqreek/
  ...
  arch/sh/mm/
  arch/sh/lib/
  arch/sh/boot/

I also think we need to split off PCI support into a seperate subdirectory,
i.e. pci/, but I'm still working on that proposal so that can wait.
  
Cleanliness
-----------
C'mon folks, it's not that difficult to follow the guidelines set forth in
CodingStyle and other documents that describe kernel development semantics.
I have also been guilty of not doing this, and we need to make a concerted
effort to make sure our code is clean.  I know that most of us are embedded
systems developers, who may not be as familiar with kernel development, but
it's important that we strive to write high-quality code, especially since
this all goes upstream, and once that happens, your support base gets much,
much larger.

Garbage In, Garbage Out - Just because a piece of code is "functional"
doesn't mean it belongs in the tree, especially if it suffers from critical
design flaws or doesn't follow kernel guidelines.  We should be able to
police ourselves, and we should never settle for shoddy code.

A good place to look for kernel guidelines is, uh, in the kernel
Documentation directory itself.  CodingStyle, kbuild/, DocBook/, and the
files of the subsystems of any drivers we develop should be consulted
before we do *any* code or design.

I've always looked at kernel sub-ports like LinuxSH as "role models" for
young developers like myself who are looking for a fit in a smaller
kernel project with a close-knit community of developers.  We should set
the right example for these developers by making sure we do the right thing
from the start.

The Future
----------
2.5.x is almost upon us.  Are we prepared for it yet?  Besides that, what
are some of our ideas in regards to providing better documentation and tool
support?  I think we should revisit the "roadmap" or "tasklist" and (if
you have the time to be able to) start writing down deadlines for the
things we want done.  I didn't touch documentation at all in this RFC (or
the lack thereof :), or some of the problems with using GNU tools with
LinuxSH, hopefully we can see people getting more proactive around this.

Conclusion
==========

Besides the issues I've raised, I've always been impressed with the rapid
progress of LinuxSH.  The fact that NIIBE was able to quintiple the speed
of his board within 3 kernel revisions is just amazing :).  I think that if
we start making the effort to write better code and establish a consistent
interface, we can make even better progress.  New SH developers looking
to port LinuxSH to their favorite board will have a much easier time doing
so.  Patch review will allow us to ensure that the best code goes in, and
will help minimize conflicts.  Overall, we'll have a much better port.

Comments?

Marcus

[linuxsh-dev] Re: 2.4.7 unaligned user space access problems

[NIIBE Y. <gn...@m1...>]

It seems that you don't like what I did, in last year to this
April.  OK, it could be understandable.

Please understand that each person has his own decision.  While you
and another guy was asking me to release the DC driver, I had friends
who asked me NOT releasing the driver.  I think that I explained this.
It was me who decided NOT releasing the driver last year.  This was my
own decision, as a individual, not as a maintainer.  But I have a
right not to release the code.

Then, I've been working towards releasing them, and finally with help
of my friends, we've released.  It took months actually, I spent much
time to negotiate and make compromise.  I understand that it's quite
unfortunate for your own purpose, you have complaints.  However, each
person has his situation to be resolved.  I never intended to confuse
and bother you.  I did my best, but that was not what you expected, or
at least not fast enough, clear enough.  Well, finally it's done.

			*	*	*

This time, you're quite unfair.  You don't talk on technical bases.
It seems for me that you currently (temporal-lily) loose a skill to
read someone's work and accept it or understand related things.
I hope, someday, you can be good state again and get things straight.
-- 

Re: [linuxsh-dev] Re: [linux-sh:01908] Re: 2.4.7 unaligned user space access problems

[M. R. B. <mr...@0x...>]

* NIIBE Yutaka <gn...@m1...> on Wed, Aug 01, 2001:

> 
> I found that it seems that you don't read the code which I sent, and
> you just asked me not to include it.  At least for me, it's not crap.
> Yes, we have disagreement here.  I haven't though that the
> disagreement is big enough.  I'm sorry.  I think that putting the code
> for the review or use is good thing.
> 

It doesn't matter how many times I read the code, it still sucks.  I read
it when Yaegashi originally posted the link, and it sucked.  I read it when
you sent it to me yesterday, and it still sucked.  Like I said, the
original NetBSD implementation left a lot to be desired as far as
cleanliness of the code.  As a matter of fact, it's mostly based off of the
original Sega BIOS GD-ROM driver, and please, don't get me started on that
one.

Now, you wanna base a Linux driver off a poorly-written NetBSD driver, and
call it quality?  If you seriously think this is a good piece of code, then
there's other issues that need to be addressed.

I'm more of the idea that Function follows Form.

> 
> You can critisize me, because I did wrong thing (not confirm your
> suggestion and do it on my dicision), but please don't insult the work
> others did.
> 

I don't want to dictate anything to anyone, that's not what this is about.
But c'mon, at least hear what I have to say.  I do have a bit of experience
on the Dreamcast side of things, I've been playing with code and
reverse-engineering since late last year.  There are blatant design
violations in Marcus', Bero's and your code in regards to the best way to
use the GDC.  And the fact the code is ugly as hell.  I guess I should've
made that more clear in my inital response, and maybe that would've
strengthed my position.

I'm not going to back down from my stance in regards to listening to
developers because this isn't the first time something like this has happened.
Regarding the situation where you didn't include DC drivers because of
contention at Sega, that did nothing but impede the overall progress of DC
support in the kernel.  One cool thing about Linux is, even if "Yoo hoo"
corporation tries to snuff it on their platform for whatever reason, there
are still others willing to port it anyway.  That's why LinuxDC was able to
add driver support, because I was fueled by the fact that Sega didn't want
it, and that was all the more reason to work on it anyway.

Now, while you were waiting for the Dreamcast to die, I was working on
drivers, and all of the sudden (much to my charign) in pops Yaegashi's DC
drivers - out of the blue.  Imagine how I feel after all those months
working on drivers with not a inch of support (by support I mean
encouragement, blessing, whatever) from you or Yaegashi and my work is
superceded.  It was the fact that you could have easily added that support
months ago, and we could've spent that time improving those drivers, rather
than reinventing the wheel.

Hey, you can say my opinion was irrelevant back then, but the problem was I
wasn't the only one asking for that release.  Sega didn't want it, so you
didn't do it.  (If anyone thinks I'm talking out of my ass here, John Byrd
of Sega specifically said Sega didn't want Linux, because of the GPL, and
that he favored BSD, because of the BSD license - and that's just coming
from Sega of America, I don't know SOJ's position).

> 
> It would be good idea.  Actually, I did follow such a style last year,
> but I thought that it did not functional (most of the time, I wrote, I
> reviewed, I decided).  It made me bit lazy.   Will do.
> 

Tossing the flames aside, I've been working on an RFC over the last few
days that brings some of these issues to light, for everyone to comment on.
I'm trying to get it to the list by this afternoon.

> 
> I didn't see such a change by anyone else.  If there's such a
> breakage, it's almost always my fault most of the time.  When I import
> the changes from mainline (that's my task), sometimes it introduced
> breaks, and ad hoc hack by me is even too bad, I admit it.
> 

Ok, I'll back off on this in the RFC, since it's already been said and
done.

> 
> ??? Do you mean, the feedback from users?  I'm same situation on you.
> I don't remember the feedback from the users of this list these days.
> I didn't get the feedback of 1120 version.  In May 2000 or earlier I
> got the feedback and discussed about the approach.  It seems people
> doesn't ahve enough interest...
>

Heh, well the way around this is to do another survey, explicitly asking
for thoughts and ideas around this.  Then if people don't respond, you have
all the reason/right to do what you will with the patches.

> Or do you mean the patches for GCC?  We're working for 3.0 and
> mainline (3.1) too, and the both patches (you may say it's ad hoc) are
> available on the Web as usual (by Kazmoto Kojima).
> 

What I meant by ad hoc was, I've looked (i.e. read) at the patches, and
some of the fixes, etc. step over certain parts of gcc that it probably
shouldn't.  Also, ad hoc means that I can't build a sh4-linux compiler with
out them, and that makes it more difficult for newbies to start kernel
development, and makes it tougher to do package maintenence for the
toolchain.

> We continue the effort to mainline for GCC, GNU C library, GNU
> Binutils, GNU Libtool, and config.sub/config.guess of GNU.  When I
> think that the patch is good enough, I sent it to mainline and discuss
> it.  It is same thing for Linux kernel mainline merge.

Ok, I'll touch briefly on this in the RFC.

Like I said, it's a RFC so that everyone in the LinuxSH community can
speak, and right now I'm going to stop flaming and finish it up.

Marcus

[linuxsh-dev] Re: [linux-sh:01908] Re: 2.4.7 unaligned user space access problems

[NIIBE Y. <gn...@m1...>]

Sorry about the not interesting topic, folks.

Here's a good news.  My xengine works five times faster now.  It was
around 120rpm (2.4.5), now 600rpm (2.4.8-pre3).  Don't compare it
with your PC, though. :-)
-- 

[linuxsh-dev] Re: [linux-sh:01908] Re: 2.4.7 unaligned user space access problems

[NIIBE Y. <gn...@m1...>]

Well, you have complaints, I see.   I'll think about that.  Thanks.

M. R. Brown wrote:
 > Right.  You sent me an e-mail with the code, and I politely asked you not
 > to include it, the main reason was because the implementation was so
 > horrible.

I found that it seems that you don't read the code which I sent, and
you just asked me not to include it.  At least for me, it's not crap.
Yes, we have disagreement here.  I haven't though that the
disagreement is big enough.  I'm sorry.  I think that putting the code
for the review or use is good thing.

I believe that it's good start, at least.  Could you please read it,
when you have time, please.  It follows new standard of block device
(rely on block device layer do the right thing) and follows Unified
CD-ROM interface.

You can critisize me, because I did wrong thing (not confirm your
suggestion and do it on my dicision), but please don't insult the work
others did.

			*	*	*

 > From my point of view, you should only be concerned about getting
 > the patches to Linus, and if you yourself have code to contribute,
 > then e-mail to the list and let it get reviewed like everyone else.

It would be good idea.  Actually, I did follow such a style last year,
but I thought that it did not functional (most of the time, I wrote, I
reviewed, I decided).  It made me bit lazy.   Will do.

 > I've seen you and others that don't bother with patch review apply
 > a patch that horribly breaks some other part of the tree.  Why?

I didn't see such a change by anyone else.  If there's such a
breakage, it's almost always my fault most of the time.  When I import
the changes from mainline (that's my task), sometimes it introduced
breaks, and ad hoc hack by me is even too bad, I admit it.

 > Even last week, when asked about patches that could help stabilze
 > gcc, you claim to have gathered feedback from others in the
 > community.  Was it public?  I've been on the list since last fall,
 > and I don't seem to remember that.

??? Do you mean, the feedback from users?  I'm same situation on you.
I don't remember the feedback from the users of this list these days.
I didn't get the feedback of 1120 version.  In May 2000 or earlier I
got the feedback and discussed about the approach.  It seems people
doesn't ahve enough interest...

Or do you mean the patches for GCC?  We're working for 3.0 and
mainline (3.1) too, and the both patches (you may say it's ad hoc) are
available on the Web as usual (by Kazmoto Kojima).

We continue the effort to mainline for GCC, GNU C library, GNU
Binutils, GNU Libtool, and config.sub/config.guess of GNU.  When I
think that the patch is good enough, I sent it to mainline and discuss
it.  It is same thing for Linux kernel mainline merge.
-- 

[linuxsh-dev] Re: [linux-sh:01908] Re: 2.4.7 unaligned user space access problems

[M. R. B. <mr...@0x...>]

* NIIBE Yutaka <gn...@m1...> on Tue, Jul 31, 2001:

> M. R. Brown wrote:
>  > That driver is an abomination, there was no patch review, and it silently
>  > went in.
> 
> Calm down, please.  It's just a driver less than 500 lines.  Why don't
> you review it now, it's good opportunity, NOW.  Cooperate together in a
> constructive way.  It's not silently written, his driver has been
> announced here and available for a while (two month or so).  From his
> point of view, it takes long time.
> 

Uh, the driver has always been garbage, it's basically a wrapper around the
NetBSD implementation, and the last time I checked, NetBSD and Linux driver
implementation were completely dissimiliar.  This "hack" has no place in
anyone's kernel.  Even the original NetBSD implementation was bad, I know
because I've been studying it ever since it went into NetBSD, and that's
why I spent months reverse engineering Sega code trying to come up with a
solid implementation.

One of the axioms that I've always tried to follow is "Garbage In, Garbage
Out".  I still don't understand your method of including crappy drivers in
the kernel soley because they're "functional".  There's a proper way to
write things, and this isn't it.

> I have a time yesterday and today to do hacking this (and I need this
> to compare the behavior of SH-4 implementation).  Hence I do that.  I
> let you see the code personally yesterday.  It's only the work of two
> days or so, I don't think we need complex procedure for such a small
> amount of work.  I've asked BERO, Marcus and you for the inclusion and
> ask the comment.  IMHO, it's enough as a maintainer.
> 

Right.  You sent me an e-mail with the code, and I politely asked you not
to include it, the main reason was because the implementation was so
horrible.  I can pull plenty of examples of bad code that has gone into the
LinuxSH tree, not just from the Dreamcast driver side.

I'm not a perfect developer, in fact, I lack a lot of experience and I'm
not ashamed to admit it.  But I do know that there is always a right way to
do things, and I won't rush to develop crappy code when I can take my time
and deliver what everyone would expect from a kernel developer.

One of the things I will address in my RFC today is the lack of patch review 
in LinuxSH.  I understand that you're the LinuxSH maintainer, but does that
mean that your decisions include the finality of everyone else?  I don't
think so.  From my point of view, you should only be concerned about
getting the patches to Linus, and if you yourself have code to contribute,
then e-mail to the list and let it get reviewed like everyone else.  If you
are able to make sweeping decisions even against recommendations then you
aren't really maintaining are you?  You end up with problems that forced
the split off the Linux/MIPS tree.  On more then one occasion (and I will
wait until the RFC to address this), I've seen you and others that don't
bother with patch review apply a patch that horribly breaks some other part
of the tree.  Why?

I have no idea why you just e-mailed the driver code to me personally.
Yes, I'm the lead developer of the ragtag LinuxDC group, but that patch
should have been reviewed by everyone concerned with DC and SH development.
I'm pretty sure that if you did post it for review, it would have been
laughed out of existence.  To me, just e-mailing the code to me and
ignoring my pleas for you not to include it shows me that you don't care
about the opinions of others in the group, because your "maintainer" status
allows you to do whatever the hell you want.  That's BS.

Oh yeah, I didn't want to bring up the ugly situation from late 2000, where
DC code was left out of the tree because you were concerned about how you
looked to Sega.  Did you get the feelings/ideas of the rest of the group,
or did you pay attention when I (among others) asked you to release the
code?

> If you would dislike someone doing something or someone doing similar
> thing (what you want to do in some time) and only want to do whole the
> thing by yourself or under your control, you simplely could not be
> good to be here or any open source world, perhaps.
> 

I don't want it under my control.  Control is not what I'm after.  The
aspiriation to perfection is what I'm after - the kind that you get when
you apply Bazaar techniques to code development.  The simplest one of these
is patch review.  Like you said may the best code win.  I agree with that.
But I don't agree with including code that is piss poor to begin with.  To
me that is the best way to do Open Source development, patch review, with
group arbitration in the case of confilicts.  I have yet to see any
earth-shaking conflicts in LinuxSH besides the whole DC/Sega-release
fiascon, so why is it you who makes the final decision?

Based on your previous actions, I'm not so sure you have the best interests
of the LinuxSH community at heart.  Even last week, when asked about
patches that could help stabilze gcc, you claim to have gathered feedback
from others in the community.  Was it public?  I've been on the list since
last fall, and I don't seem to remember that.  Now, you can criticize me
for taking my SWEET time getting DC work done in the kernel, but I (and
others) have been waiting for months for a SH gcc that we could use that
didn't require ad hoc patches.

> Could you please think about a bit for _his_ or someones point of view.
> 

He needs to read up on some of the files in linux/Documentation/, as does
anyone else serious about doing kernel development work.  That's one of the
first things they tell you in the lkml-FAQ, btw.

> I had believed that this could be the achievement of YOUR project.  I
> still believe that.  It's up to you.

That's not an issue at all for me.  I know one thing, even though I don't
want/care of absolute control over what goes in on the DC driver side, I
do NOT want junk like this GD-ROM driver (I won't cite other examples)
being associated with me or LinuxDC.

Marcus

[linuxsh-dev] [linux-sh:01908] Re: 2.4.7 unaligned user space access problems

[NIIBE Y. <gn...@m1...>]

BTW, BERO also wants to add sound driver support for Linux.  Could you
please help him?  I've recommended ALSA for the start and references,
as the sound drivers in Linux mainline is bit unmaintaind.
-- 

[linuxsh-dev] [linux-sh:01908] Re: 2.4.7 unaligned user space access problems

[NIIBE Y. <gn...@m1...>]

M. R. Brown wrote:
 > That driver is an abomination, there was no patch review, and it silently
 > went in.

Calm down, please.  It's just a driver less than 500 lines.  Why don't
you review it now, it's good opportunity, NOW.  Cooperate together in a
constructive way.  It's not silently written, his driver has been
announced here and available for a while (two month or so).  From his
point of view, it takes long time.

I have a time yesterday and today to do hacking this (and I need this
to compare the behavior of SH-4 implementation).  Hence I do that.  I
let you see the code personally yesterday.  It's only the work of two
days or so, I don't think we need complex procedure for such a small
amount of work.  I've asked BERO, Marcus and you for the inclusion and
ask the comment.  IMHO, it's enough as a maintainer.

If you would dislike someone doing something or someone doing similar
thing (what you want to do in some time) and only want to do whole the
thing by yourself or under your control, you simplely could not be
good to be here or any open source world, perhaps.

Could you please think about a bit for _his_ or someones point of view.

I had believed that this could be the achievement of YOUR project.  I
still believe that.  It's up to you.
-- 

[linuxsh-dev] Re: [linux-sh:01907] Re: 2.4.7 unaligned user space access problems

[M. R. B. <mr...@0x...>]

* NIIBE Yutaka <gn...@m1...> on Tue, Jul 31, 2001:

> To test SH-4 kernel, I've done a little work to include Dreamcast CD-R
> support which is written by BERO.
> 

I'd just like to say publicly that including that driver was a *bad* idea.

That driver is an abomination, there was no patch review, and it silently
went in.

This is one of a few problems in LinuxSH that I plan to address in a RFC,
hopefully I can get it to the relevant list(s) this afternoon (US CST).

M. R.

[linuxsh-dev] Re: SH4. Problem with floating point operations. using sh4-linux-gcc

[kaz K. <kk...@rr...>]

Hi,

Amir Hadar <ami...@is...> wrote:
> As I studied this problem more I found that the when using optimization 
> the assemble don't use the instruction
> 
> fcnvsd  fpul,dr2
> 
> This is a conversion from float to double. This instruction cause the 
> error.

The real problem is the FCNVSD instruction as you've pointed out.
If FPUL register has the denormalized float value, then FCNVCD
generates a fpu exception, though this operation seems to be valid,
at least mathematically. SH-4 manual suggests that the emulation
is needed to handle such situation. I think that the right thing to
do is to add an emulation code to the fpu exception handler.

	kaz

[linuxsh-dev] Re: [linux-sh:01901] FW: A sh4-linux-gcc bug?

[Michael E. <ea...@mv...>]

GAS takes the actual offset and generates the correct machine code
to compute that offset.  Are you seeing incorrect machine code?

Dustin McIntire wrote:
> 
> Hello everyone,
> 
> While examining the assembly output from sh4-linux-gcc I have come
> across a strange error. All the immediate offsets should be x2 for
> mov.w, x4 for mov.l etc. However, gcc lists them all with immediate
> offsets x1.
> 
> For example, compile the following code with the options
> 
> sh4-linux-gcc -S -O3 -m4 -o gcc_bug.S gcc_bug.c
> 
> note the offsets to r14 when fetching Arg5 and Arg6.
> 
> #include <stdio.h>
> 
> int gcc_bugTry1(
>     int Arg1,
>     int Arg2,
>     int Arg3,
>     int Arg4,
>     int Arg5,
>     int Arg6,
>     int *returnValue
>     )
> {
>     int retVal, i;
>     int TheStack [10] ;
> 
>     retVal = Arg1 + Arg2 + Arg3 + Arg4 + Arg5 + Arg6;
> 
>     for (i=0; i<10; i++)
>     {
>         TheStack[i] = retVal + i;
>         returnValue[i] = TheStack[i];
>     }
> 
>     return retVal;
> }
> 
> --
> Jeremy Hoyland
> jer...@su...

--
Michael Eager     ea...@mv...	408-328-8426	
MontaVista Software, Inc. 1237 E. Arques Ave., Sunnyvale, CA  94085

[linuxsh-dev] FW: A sh4-linux-gcc bug?

[Dustin M. <du...@se...>]

Hello everyone,

While examining the assembly output from sh4-linux-gcc I have come
across a strange error. All the immediate offsets should be x2 for
mov.w, x4 for mov.l etc. However, gcc lists them all with immediate
offsets x1.

For example, compile the following code with the options

sh4-linux-gcc -S -O3 -m4 -o gcc_bug.S gcc_bug.c

note the offsets to r14 when fetching Arg5 and Arg6.


#include <stdio.h>

int gcc_bugTry1(
    int Arg1,
    int Arg2,
    int Arg3,
    int Arg4,
    int Arg5,
    int Arg6,
    int *returnValue
    )
{
    int retVal, i;
    int TheStack [10] ;

    retVal = Arg1 + Arg2 + Arg3 + Arg4 + Arg5 + Arg6;

    for (i=0; i<10; i++)
    {
        TheStack[i] = retVal + i;
        returnValue[i] = TheStack[i];
    }

    return retVal;
}


--
Jeremy Hoyland
jer...@su...

Re: [linuxsh-dev] SH4. Problem with floating point operations. using sh4-linux-gcc

[Amir H. <ami...@is...>]

gcc version:

Reading specs from /usr/sh4-linux/lib/gcc-lib/sh4-linux/2.97/specs
Configured with: ../configure --prefix=/usr/sh4-linux --target=sh4-linux 
--enable-nls --with-gnu-as --with-gnu-ld
gcc version 2.97 20001120 (experimental)

As I studied this problem more I found that the when using optimization 
the assemble don't use the instruction

fcnvsd  fpul,dr2

This is a conversion from float to double. This instruction cause the 
error.

Here is the entire assemble code (non optimized):	
------------------------------------------------------------------
.file 
"main.c"
	.little
gcc2_compiled.:
	.text
	.align 5
	.global	main
	.type	main,@function
main:
	mov.l	r14,@-r15
	add	#-20,r15
	mov	r15,r14
	mov.l	r4,@r14
	mov.l	r5,@(4,r14)
	mov	#1,r1
	mov.l	r1,@(8,r14)	! store the 0x1 in r14[8]
	mov	r14,r1		! r14 -> r1
	add	#8,r1		! r1 + 8 -> r1, now r1 points to r14[8]
	fmov.s	@r1,fr1		! *(r1) -> fr1, now bits_of(fr1) = 0x1
	flds	fr1,fpul	! fr1 -> fpul , this is for the conver
	fcnvsd  fpul,dr2	! converting fr1 -> dr2 , ERROR SIGFPE
	mov	r14,r1
	add	#12,r1
	add	#4,r1
	fmov.s	fr2,@r1
	fmov.s	fr3,@-r1
	mov	#0,r0
	add	#20,r14
	mov	r14,r15
	mov.l	@r15+,r14
	rts	
	nop
.Lfe1:
	.size	main,.Lfe1-main
	.ident	"GCC: (GNU) 2.97 20001120 (experimental)"
--------------------------------------------------------------

Amir Hadar

Gareth S. Long wrote:

> In message <3B6...@is...> you wrote:
> 
> 
>>It seems I was mistaken about the assembler part. The problem is somehow 
>>related to compiler optimization. If I use optimization (-O option in 
>>gcc) the program works, If I don't use the optimization it fails.
>>
> 
> What version of gcc are you using, and what patches have been applied?
> 
> 

Re: [linuxsh-dev] SH4. Problem with floating point operations. using sh4-linux-gcc

[Gareth S. L. <ga...@el...>]

In message <3B6...@is...> you wrote:

> 
> It seems I was mistaken about the assembler part. The problem is somehow 
> related to compiler optimization. If I use optimization (-O option in 
> gcc) the program works, If I don't use the optimization it fails.

What version of gcc are you using, and what patches have been applied?

-- 
Gatch

Re: [linuxsh-dev] SH4. Problem with floating point operations. using sh4-linux-gcc

[Amir H. <ami...@is...>]

It seems I was mistaken about the assembler part. The problem is somehow 
related to compiler optimization. If I use optimization (-O option in 
gcc) the program works, If I don't use the optimization it fails.

sorry about the double email thing.

Amir Hadar wrote:

> Hi
> The problem is that when sh4 convert float to double, when the float
> is denormalized, it throws a SIGFPE signal which is illegal arithmetic
> operation.
> 
> The following program shows the problem:
> 
> #include <stdio.h>
> int main(int argc, char* argv[]) {
>   union {
>     float f;
>     int i;
> }tmp;
> 
> tmp.i = 0x1;
> 
>   printf("%f",tmp.f);
>   return 0;
> }
> 
> Now, according to the sh4 programming manual 0x1 is a legal float
> value (IEEE-754). The problem occur with all denormalized float
> values :
> 0x007FFFFF - 00000001  - positive denormalized floats
> 0x80000001 - 807FFFFF  - negative denormalized floats
> 
> The interesting part is that when I compile to assemble (using -S with 
> the gcc) and then compile the assemble to executable, it works. This 
> seems like a compiler bug.
> 
> Any help will be appreciated.
> 
> 
> 
> _______________________________________________
> linuxsh-dev mailing list
> lin...@li...
> http://lists.sourceforge.net/lists/listinfo/linuxsh-dev

[linuxsh-dev] SH4. Problem with floating point operations. using sh4-linux-gcc

[Amir H. <ami...@is...>]

Hi
The problem is that when sh4 convert float to double, when the float
is denormalized, it throws a SIGFPE signal which is illegal arithmetic
operation.

The following program shows the problem:

#include <stdio.h>
int main(int argc, char* argv[]) {
   union {
     float f;
     int i;
}tmp;

tmp.i = 0x1;

   printf("%f",tmp.f);
   return 0;
}

Now, according to the sh4 programming manual 0x1 is a legal float
value (IEEE-754). The problem occur with all denormalized float
values :
0x007FFFFF - 00000001  - positive denormalized floats
0x80000001 - 807FFFFF  - negative denormalized floats

The interesting part is that when I compile to assemble (using -S with 
the gcc) and then compile the assemble to executable, it works. This 
seems like a compiler bug.

Any help will be appreciated.

[linuxsh-dev] 2.4.7 unaligned user space access problems

[NIIBE Y. <gn...@m1...>]

Dustin McIntire wrote:
 > I'm not sure this makes any sense, but one possible reason for seeing the
 > cache troubles on the CqREEK but not on the Solution Engine could be based
 > on the type of IRQ.  All IRQ's on the Solution Engine appear to be IPR IRQ's
 > and would be accessed via P4 (internal registers) while the CqREEK appears
 > to have external IRQ registers and would be accessing P2.

That's true SolutionEngine's IRQ is all IPR based, but it accesses P2
area too.  Look at make_ipr_irq in setup_se.c, the argument BCR_ILCRx
is P2 area.

			*	*	*

Because (my) CqREEK SH-4 is flaky, mine would be hardware problem.
When I run it on heavy load, I see the problem with CqREEK SH-4.  It's
not that the product CqREEK SH-4 board itself is flaky, but my
configuration (with unofficial ISA extention adaptor by CQ publishing)
is surely flaky.

When I set cache-sh4.c:CCR_CACHE_INIT=0x000d (means no Instruction
cache), it runs fine.  CCR_CACHE_INIT=0x0900 (means no Data cache)
also works fine.  It's really wired.  I don't see any reason why
disabling Instruction Cache changes the sitiation.

Perhaps, it just lets slow down the execution, and makes sence for the
hardware.  I'm not sure.

I cannot change bus timing because the ISA adapter depends the bus
clock.  If you can do that and test your board with lower clock, I'd
much appreciate.
-- 

RE: [linux-sh:01865] [linuxsh-dev] 2.4.7 unaligned user space access problems

[Dustin M. <du...@se...>]

I'm not sure this makes any sense, but one possible reason for seeing the
cache troubles on the CqREEK but not on the Solution Engine could be based
on the type of IRQ.  All IRQ's on the Solution Engine appear to be IPR IRQ's
and would be accessed via P4 (internal registers) while the CqREEK appears
to have external IRQ registers and would be accessing P2.

Dustin.


> -----Original Message-----
> From: NIIBE Yutaka [mailto:gn...@m1...]
> Sent: Tuesday, July 24, 2001 3:59 AM
> To: linux-sh; linuxsh-dev
> Subject: [linux-sh:01865] [linuxsh-dev] 2.4.7 unaligned user space
> access problems
>
>
> I've tested some kernels.  My conclusion is: either CqREEK is broken
> or kernel is broken.
>
> Folks, please test the current CVS version and if you have time test
> stock 2.4.6 kernel.  Then, please let me know it works well or not.
>
>
> CVS current (2.4.7+change):
>     CqREEK	SolutionEngine
>     Bad		OK
>
> vanilla 2.4.7+atomic+ksoftirq:
>     CqREEK	SolutionEngine
>     Bad
>
> vanilla 2.4.6
>     CqREEK	SolutionEngine
>     Bad
>
> vanilla 2.4.6-pre6
>     CqREEK	SolutionEngine
>     Bad
>
> vanilla 2.4.6-pre4
>     CqREEK	SolutionEngine
>     Bad
>
> vanilla 2.4.6-pre2+fix(bitops,semaphore,irq.c,entry.S,hardirq.h,softirq.h)
>     CqREEK	SolutionEngine
>     Bad
>
> vanilla 2.4.6-pre1+fix(bitops,semaphore,softirq.h)
>     CqREEK	SolutionEngine
>     Bad
>
> vanilla 2.4.5+fix(bitops,semaphore)
>     CqREEK	SolutionEngine
>     Bad
> --
>