[Kernelbook-feedback] An update on our progress for September 2000

[Gary L. M. <ga...@ca...>]

Yes, this list has been quiet ;) and my apologies for this; here
is a brief update on where we are and where we are going with this
kernelbook project.  

Put another way, here is my List of Top Ten Lame Excuses:

1) We've had several more contributing authors leave us over the
copyright/license issue, and while regrettable, it is an issue out of
my control --- the MCP/OPL was the best we could do, and it is a far
sight better than any previous commercial publishing license. It is
the perogative of all authors to do as their conscience dictates and I
respect their decisions, but we are now short of the expertise needed
for the indepth chapters of the book.

2) My own contributions are stalled due to lack of resources; the time
and hardware commitments exceeded my budgets and as a result, I am
forced to put my involvement on hold while I build up the bank
accounts.  While MCP does fund our writing, we are on our own for our
research, and I have asked our publisher to let us hold off on
committing to a writing schedule until we have a comprehensive
collection of the raw material.  While the Rigi project and others
have been fantastically supportive of my efforts, I can only do so
much on my own and must consider my own bottom line.

3) Linux 2.4 is also maturing much later than anyone expected and
already large parts of what has been contributed to the book are now
obsolete.  Many core sections are still in flux; if I may coin a new
phrase, I'd say they are "pulling a TomCat" ;) (this is an inside joke
to Apache/Tomcat users) To avoid releasing a book only to have it
obsolete before it hits the shelves, I have asked that we wait until
the first public release of 2.4.0 before we run any further analysis.

4) We are not receiving any public contribution to the book.  A
handful of contributors and feedback-list members have provided
fantastic reviews and feedback, and the project is featured on several
kernel websites (such as Kernel Newbies) but I have failed to ignite
the enthusiasm of precisely the people we need. I also have a secret
hope that the kernel developers are all under the gun to complete this
new revision and have little time left to grace us with their support
until 2.4 is released. 

5) Over the delays on the licensing/copyright issues (we have now been
working on this book for just over one year) several contributing
authors have vanished off the face of the earth; only a very few have
actually received and returned their contracts.  We need to mount a
fresh campaign to re-staff the core team.

6) Several excellent free-document initiatives have appeared for the
kernel.  The linux-doc mailing list has resurfaced, kernel-doc and
kernelnewbies have become more active, and the Documentation/Docbook
is growing.  Our book is not a competitor to these projects, since our
scope is to chart the architecture whereas these works chart the APIs
and component-level algorithms.  However, the expanding scope of these
free document projects, and the defection of some of our authors over
to these projects ;) means our table of contents must be rethought to
make this distinction.  I am also talking with MCP about incorporating
the best of these free documents into an appendix of the book so we
can cross reference and index them (also many people cannot print raw
DocBook sources).

        While I applaud and participate in these free document
        initiatives, I personally believe the style and direction of
        such documents are very different from a professionally edited
        trade publication; while we are free to include their material
        into our book, the unfortunate side effect of their free
        document methods is that we cannot apply editorial standards
        to maintain the requisite consistency of scope and style ---
        basically, it is free, it is a gift and you take what you get.

7) On a more pragmatic issue, DocBook SGML proved inadequate to our
process needs.  To go smoothly from authoring through editorial to
typesetting, we must shift to DocBook XML.  Unfortunately, I was only
just beginning to get a grasp of the SGML tools and the fine art of
DSSSL and OpenJade, but now I must re-learn the technology from
scratch ... and here again, I ran out of personal resources to be able
to devote the time to learn the new methods and then to apply them.
Considering the friction I received from most contributing authors
over using DocBook in the first place, I am also not looking forward
to dragging them kicking and screaming into XML ;) One a positive note
for the readers of the feedback-list, moving to XML will also simplify
creating the HTML editions of the chapters, which is the number one
most requested service.

8) Due to (6) and (7) the CVS archive is broken, stuck in a transition
between folding in the kernel Documentation/DocBook appendix (which
requires heavy forking to be incorporated into a book and thus becomes
a maintenance problem) and the transition to XML.  There are
techniques used in our SGML which are unsupported in DocBook XML, and
I have just haven't had the time or money to do all the conversion
work myself.  Since only two contributing authors even attempted to
use this authoring process, it didn't seem a very high priority, but I
do apologize for having frustrated those potential reviewers who went
through the trouble of downloading the book only to discover that it
did not build.

9) Somewhat related to (8) is the issue of diagrams and images.  The
makefile is also broken with regard to images and was not fixed
because my experience working with a large book-sized SGML source
quickly underlined shortcomings in the labelling conventions and
directory structure which require some careful thought.  We have also
learned that the end-to-end XML process is most friendly to postscript
images and since this is an architecture book, line drawings are
probably all that we need, but it does mean we have some bitmaps to
weed out of the text.

10) Last of my lame excuses is psychological: After spending two
months of overtime 7-day-weeks, wasting my summer sifting relational
kernel facts and building LXR tables, I saw many postings on the
linux-kernel list emphatically opposing any such attempts to make the
kernel accessible. Some of the most adamant anti-tools comments come
from Linus himself.  As I began to burn out, and as my bank balance
dwindled, I began to ask "Why am I doing this?"  There's no prize at
the end, the royalties will be minimal (it is not, after all, a new
Harry Potter book), the developers don't want it, the authors don't
like the methodology, the open source people won't endorse it, and
users might buy it, but none are knocking on my door to help make it
happen. Is life too short for all this bother?  As a result, I took
three weeks off to play on the beach with my kids and ponder whether I
had the proper charisma to make this thing work.

Anyway, such as it is, that is the current status of the Kernelbook.
Rumours of its death are greatly exaggerated, but do not expect to see
great leaps of progress any time soon.  Maybe it is like the "Instant
Giant Sequoia Tree" where you just plant the seed, add water, wait 750
years, and then stand back ;)

-- 
Gary Lawrence Murphy <ga...@li...>: office voice/fax: 01 519 4222723
T(!c)Inc Business Innovation through Open Source http://www.teledyn.com
M:I-3 - Documenting the Linux kernel: http://kernelbook.sourceforge.net
"You don't play what you know; you play what you hear." --- Miles Davis