refdb-devel — List for RefDB developers

Re: [Refdb-devel] refdb-elisp

[Markus H. <mar...@mh...>]

Hi,

David Nebauer writes:
 >  From memory, the pdf and html manual files ended up in 
 > refdb-elisp-1.0.  Other doc files such as changelog.Debian.gz, NEWS.gz, 
 > changelog.gz, README and copyright ended up in refdb-elisp.  It looks 
 > silly and, as I mentioned, appeared to confuse the Debian document 
 > registration script.
 > 

Ah, I see. Something Debian-specific.

 > I presume you meant 'refdb-mode-config.el'?

Yes. Sorry about that.

 > Turns out the emacsen build 
 > scripts expect to find that file in the /debian subdirectory and called 
 > 'emacsen-startup'.  The build process then installs the startup file in 
 > /etc/emacs/site-start.d as '50<PACKAGE>', in this case '50refdb-elisp'.
 > 

I suspected there is some automatism for this. I don't know whether it
makes things more complicated, but I've changed Makefile.am to install
refdb-mode-config.el as a configuration file in sysconfdir (usually
/etc or /usr/local/etc). This is probably the most logical place as it
now contains the locations of the viewing programs. If the Debian
policy allows this, you could simply create a symlink in
/etc/emacs/site-start.d pointing to that file.

 > After doing this the refdb and ris modes work correctly in emacs.  The 
 > debian package now appears to be usable.  I'll send it to you off list.
 > 

Thanks, I'll try it here as well.

 > I'll now automate the build process.  Unless and until you incorporate 
 > the changes to refdb-mode.el and the makefile, the build process will 
 > include a fair bit of search-and-replacing.  If you want to take over as 
 > maintainer of the refdb-elisp debian package I can email you the 
 > relevant files.  The debian build process would then be as simple as 
 > running the build script.  If I recall correctly the 'dpkg-dev' package 
 > includes all the tools needed by the build script.
 > 

If that is all it takes, I should be able to handle this. I'll let you
know.

 > 7.  No ChangeLog file
 > 

I'll create it using cvs2cl from now on.

 > 
 > >And thanks a lot for touching Emacs in the first place.
 > >
 > 
 > I'll shower afterwards ;-) .
 > 

Did showering help?

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [Refdb-devel] refdb-elisp

[David N. <dav...@sw...>]

Hi Markus,

> > 5.  Inappropriate hard-coded values in refdb-mode
> > 
>
>Yeah, I was a bit lazy at this point to get refdb-mode done in time. I
>think it's best to let configure check for the appropriate paths and
>apps (e.g. I can check for either swriter, oowriter, abiword, or kword
>as a rtf viewer, and use whichever is found first) and let the
>Makefile preconfigure refdb-configure.el from an .in file.
>


Don't forget Ted <http://www.nllgg.nl/Ted/> for those who want to edit 
RTF without the bloat of a full-fledged office suite.

Regards,
David.

Re: [Refdb-devel] refdb-elisp

[David N. <dav...@sw...>]

Hi Markus,

> > If you do not do this the documentation is split between 
> > /usr/share/doc/refdb-elisp and /usr/share/doc/refdb-elisp-1.0.  Further, 
> > the debian (dwww) document registration script gets confused.
> > 
>
>Why is it "split"? What does end up in /usr/share/doc/refdb-elisp? I
>thought this install directory is safe to use as we use the same in
>libdbi and in RefDB.
>

 From memory, the pdf and html manual files ended up in 
refdb-elisp-1.0.  Other doc files such as changelog.Debian.gz, NEWS.gz, 
changelog.gz, README and copyright ended up in refdb-elisp.  It looks 
silly and, as I mentioned, appeared to confuse the Debian document 
registration script.

> > 6.  ?? Failure
> > 
> > After installation of the package I was unable to load either mode.  
> > This could easily be due to my _complete_ ignorance of emacs.
> > 
>
>Did you try to handle refdb-configure.el automatically? This needs to
>be inserted into your site-lisp.el file, or into ~/.emacs. I believe
>Debian has a site-lisp.d directory where apps can dump these startup
>code fragments. Without the code in refdb-configure.el emacs does not
>load the modes.
>

I presume you meant 'refdb-mode-config.el'?  Turns out the emacsen build 
scripts expect to find that file in the /debian subdirectory and called 
'emacsen-startup'.  The build process then installs the startup file in 
/etc/emacs/site-start.d as '50<PACKAGE>', in this case '50refdb-elisp'.

After doing this the refdb and ris modes work correctly in emacs.  The 
debian package now appears to be usable.  I'll send it to you off list.

I'll now automate the build process.  Unless and until you incorporate 
the changes to refdb-mode.el and the makefile, the build process will 
include a fair bit of search-and-replacing.  If you want to take over as 
maintainer of the refdb-elisp debian package I can email you the 
relevant files.  The debian build process would then be as simple as 
running the build script.  If I recall correctly the 'dpkg-dev' package 
includes all the tools needed by the build script.

Finally, there is one other problem with the current autotools distribution:


7.  No ChangeLog file

Debian policy requires a 'ChangeLog' file that tracks changes to the 
original sources -- not to be confused with the 'changelog' file in the 
/debian subdirectory which tracks packaging-related changes.  The dummy 
ChangeLog file I've added simply contains the following:
--------------------------------------------------------------------------
2005-11-15  Markus Hoenicka <mho...@us...>

<TAB>* Initial autotools packaging

--------------------------------------------------------------------------

>And thanks a lot for touching Emacs in the first place.
>

I'll shower afterwards ;-) .

Regards,
David.

[Refdb-devel] refdb-elisp

[Markus H. <mar...@mh...>]

Markus Hoenicka writes:
 > Did you try to handle refdb-configure.el automatically? This needs to
 > be inserted into your site-lisp.el file, or into ~/.emacs. I believe
 > Debian has a site-lisp.d directory where apps can dump these startup
 > code fragments. Without the code in refdb-configure.el emacs does not
 > load the modes.
 >  > 

Just checked. It's actually /etc/emacs/site-start.d. This directory
contains the code fragments in files whose names start with two
digits, e.g. 50refdb-mode-config.el. That is your package would have
to install refdb-mode-config.el as
/etc/emacs/site-start.d/50refdb-mode-config.el.


regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

[Refdb-devel] refdb-elisp

[Markus H. <mar...@mh...>]

Hi David,

David Nebauer writes:
 > 1.  Makefile document directory
 > 
 > I had to alter Makefile.am: change
 >     docdir = $(datadir)/doc/$(PACKAGE)-$(VERSION)
 > to
 >     docdir = $(datadir)/doc/$(PACKAGE)
 > 
 > If you do not do this the documentation is split between 
 > /usr/share/doc/refdb-elisp and /usr/share/doc/refdb-elisp-1.0.  Further, 
 > the debian (dwww) document registration script gets confused.
 > 

Why is it "split"? What does end up in /usr/share/doc/refdb-elisp? I
thought this install directory is safe to use as we use the same in
libdbi and in RefDB.

 > 3.  Compile error
 > 
 > The following error occurs at compile time:
 > --------------------------------------------------------------------------------
 > While compiling refdb-getnote-by-field-on-region in file 
 > /usr/share/emacs21/site-lisp/refdb-elisp/refdb-mode.el:
 >   ** Malformed let binding: `(concat refdb-regexp-query-string (value 
 > (buffer-substring (mark) (point))) refdb-regexp-query-string)'
 > --------------------------------------------------------------------------------

Thanks for catching this. I didn't notice this between all those
harmless warnings.

 > 5.  Inappropriate hard-coded values in refdb-mode
 > 
 > 95:   ;;    load path (e.g. /usr/local/share/emacs/site-lisp), or add
 > to:   ;;    load path (e.g. /usr/share/emacs/site-lisp), or add
 > 
 > 877:  (defcustom refdb-rtf-view-program 
 > "/usr/local/OpenOffice.org1.1.4/program/swriter"
 > to:   (defcustom refdb-rtf-view-program "/usr/bin/oowriter"
 > 
 > 5125: (let* ((globalconfig-localpath (format "/usr/local/etc/refdb/%s" 
 > globalconfig-file))
 > to:   (let* ((globalconfig-localpath (format "/etc/refdb/%s" 
 > globalconfig-file))
 > 
 > 5205: (refdbdrc-localpath "/usr/local/etc/refdb/refdbdrc")
 > to:   (refdbdrc-localpath "/etc/refdb/refdbdrc")
 > 
 > 5446: (let* ((refdbdrc-localpath "/usr/local/etc/refdb/refdbdrc")
 > to:   (let* ((refdbdrc-localpath "/etc/refdb/refdbdrc")
 > 
 > 5618: (refdbdrc-localpath "/usr/local/etc/refdb/refdbdrc")
 > to:   (refdbdrc-localpath "/etc/refdb/refdbdrc")
 > 

Yeah, I was a bit lazy at this point to get refdb-mode done in time. I
think it's best to let configure check for the appropriate paths and
apps (e.g. I can check for either swriter, oowriter, abiword, or kword
as a rtf viewer, and use whichever is found first) and let the
Makefile preconfigure refdb-configure.el from an .in file.

 > 
 > 6.  ?? Failure
 > 
 > After installation of the package I was unable to load either mode.  
 > This could easily be due to my _complete_ ignorance of emacs.
 > 

Did you try to handle refdb-configure.el automatically? This needs to
be inserted into your site-lisp.el file, or into ~/.emacs. I believe
Debian has a site-lisp.d directory where apps can dump these startup
code fragments. Without the code in refdb-configure.el emacs does not
load the modes.
 > 
 > I'll send you the package off-list.  Please try it on your Debian system 
 > and let me know if it works.
 > 

I'll do this weekend. And thanks a lot for touching Emacs in the first
place.

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

[Refdb-devel] refdb-elisp

[David N. <dav...@sw...>]

Hi Markus,

I just made a first pass at packaging refdb-elisp as a Debian package 
and have some feedback:


1.  Makefile document directory

I had to alter Makefile.am: change
    docdir = $(datadir)/doc/$(PACKAGE)-$(VERSION)
to
    docdir = $(datadir)/doc/$(PACKAGE)

If you do not do this the documentation is split between 
/usr/share/doc/refdb-elisp and /usr/share/doc/refdb-elisp-1.0.  Further, 
the debian (dwww) document registration script gets confused.


2.  Install directory

AM_PATH_LISPDIR gives the install directory /usr/share/emacs/site-lisp 
which is where the *.el and resulting *.elc files are installed.  
Unfortunately, the buildpackage emacsen install scripts expect these 
files to be installed in /usr/share/emacs/site-lisp/${PACKAGE}.

I used the --with-lispdir configure option to force the install into 
that subdirectory.


3.  Compile error

The following error occurs at compile time:
--------------------------------------------------------------------------------
While compiling refdb-getnote-by-field-on-region in file 
/usr/share/emacs21/site-lisp/refdb-elisp/refdb-mode.el:
  ** Malformed let binding: `(concat refdb-regexp-query-string (value 
(buffer-substring (mark) (point))) refdb-regexp-query-string)'
--------------------------------------------------------------------------------


4.  Persistence of /usr/local

There is the usual Debian trouble with /usr/local and /usr.  Although I 
configured with 'prefix=/usr' I get the following warnings twice during 
emacs package install:
--------------------------------------------------------------------------------
Warning: Lisp directory `/usr/local/share/emacs/21.4/site-lisp' does not 
exist.
Warning: Lisp directory `/usr/local/share/emacs/site-lisp' does not exist.
--------------------------------------------------------------------------------


5.  Inappropriate hard-coded values in refdb-mode

95:   ;;    load path (e.g. /usr/local/share/emacs/site-lisp), or add
to:   ;;    load path (e.g. /usr/share/emacs/site-lisp), or add

877:  (defcustom refdb-rtf-view-program 
"/usr/local/OpenOffice.org1.1.4/program/swriter"
to:   (defcustom refdb-rtf-view-program "/usr/bin/oowriter"

5125: (let* ((globalconfig-localpath (format "/usr/local/etc/refdb/%s" 
globalconfig-file))
to:   (let* ((globalconfig-localpath (format "/etc/refdb/%s" 
globalconfig-file))

5205: (refdbdrc-localpath "/usr/local/etc/refdb/refdbdrc")
to:   (refdbdrc-localpath "/etc/refdb/refdbdrc")

5446: (let* ((refdbdrc-localpath "/usr/local/etc/refdb/refdbdrc")
to:   (let* ((refdbdrc-localpath "/etc/refdb/refdbdrc")

5618: (refdbdrc-localpath "/usr/local/etc/refdb/refdbdrc")
to:   (refdbdrc-localpath "/etc/refdb/refdbdrc")


6.  ?? Failure

After installation of the package I was unable to load either mode.  
This could easily be due to my _complete_ ignorance of emacs.


I'll send you the package off-list.  Please try it on your Debian system 
and let me know if it works.

Regards,
David.

[Refdb-devel] ruby client library

[Markus H. <mar...@mh...>]

Hi,

one more question: while developing the library, did you use some sort
of Ruby test program? This would help me a lot to get started, as I've
never touched Ruby before.

regards,
Markus

Diwaker Gupta writes:
 > A long long time ago, I had started work on a Ruby client library for
 > refdb, so that people could build cool web interfaces around refdb
 > using Ruby on Rails and so on. Unfortunately I didn't finish it and so
 > I'm attaching whatever I had so far in case anyone else wants to pick
 > it up.
 > 
 > The base functionality is fairly complete. The code follows the perl
 > client closely so should be easy to follow.
 > 
 > Diwaker
 > --
 > Web/Blog/Gallery: http://floatingsun.net

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [Refdb-devel] man pages

[Markus H. <mar...@mh...>]

David Nebauer <dav...@sw...> was heard to say:

> I've been incommunicado for three weeks and am now catching up.  I'd
> already written the refdb-backup and refdb-restore manpages.  They have
> been, in fact, added to the refdb debian packages I've been building.
> For your convenience I've attached them to this message.
>
> In the next week or so I'll post updated Debian packages.
>

Good timing. I've actually started releasing 0.9.6 yesterday, but as I couldn't
finish the web page I didn't send any announcements yet. I'll add the manpages
to the tarball and upload it again. I'll fix the web pages today and I'll send
announcements to the list and to freshmeat.

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [Refdb-devel] man pages

[David N. <dav...@sw...>]

Hi Markus,

Markus Hoenicka wrote:

>refdb-backup
>and refdb-restore are currently dummies. Would you mind writing these
>man pages? I guess you're the one who really knows what these tools
>do.
>  
>

I've been incommunicado for three weeks and am now catching up.  I'd 
already written the refdb-backup and refdb-restore manpages.  They have 
been, in fact, added to the refdb debian packages I've been building.  
For your convenience I've attached them to this message.

In the next week or so I'll post updated Debian packages.

Regards,
David.

[Refdb-devel] ruby client library

[Markus H. <mar...@mh...>]

Hi,

thanks for sharing your code instead of letting it collect dust. I
personally don't have any Ruby experience, but I'll add the existing
code to cvs anyway. Maybe someone else will pick it up.

regards,
Markus

Diwaker Gupta writes:
 > A long long time ago, I had started work on a Ruby client library for
 > refdb, so that people could build cool web interfaces around refdb
 > using Ruby on Rails and so on. Unfortunately I didn't finish it and so
 > I'm attaching whatever I had so far in case anyone else wants to pick
 > it up.
 > 
 > The base functionality is fairly complete. The code follows the perl
 > client closely so should be easy to follow.
 > 
 > Diwaker
 > --
 > Web/Blog/Gallery: http://floatingsun.net

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

[Refdb-devel] ruby client library

[Diwaker G. <diw...@gm...>]

QSBsb25nIGxvbmcgdGltZSBhZ28sIEkgaGFkIHN0YXJ0ZWQgd29yayBvbiBhIFJ1YnkgY2xpZW50
IGxpYnJhcnkgZm9yCnJlZmRiLCBzbyB0aGF0IHBlb3BsZSBjb3VsZCBidWlsZCBjb29sIHdlYiBp
bnRlcmZhY2VzIGFyb3VuZCByZWZkYgp1c2luZyBSdWJ5IG9uIFJhaWxzIGFuZCBzbyBvbi4gVW5m
b3J0dW5hdGVseSBJIGRpZG4ndCBmaW5pc2ggaXQgYW5kIHNvCkknbSBhdHRhY2hpbmcgd2hhdGV2
ZXIgSSBoYWQgc28gZmFyIGluIGNhc2UgYW55b25lIGVsc2Ugd2FudHMgdG8gcGljawppdCB1cC4K
ClRoZSBiYXNlIGZ1bmN0aW9uYWxpdHkgaXMgZmFpcmx5IGNvbXBsZXRlLiBUaGUgY29kZSBmb2xs
b3dzIHRoZSBwZXJsCmNsaWVudCBjbG9zZWx5IHNvIHNob3VsZCBiZSBlYXN5IHRvIGZvbGxvdy4K
CkRpd2FrZXIKLS0KV2ViL0Jsb2cvR2FsbGVyeTogaHR0cDovL2Zsb2F0aW5nc3VuLm5ldAo=

[Refdb-devel] Debian packages, FreeBSD port for 0.9.6?

[Markus H. <mar...@mh...>]

Hi David, Paul,

I haven't seen either of you on the lists lately, so I just wanted to ping you
because of the upcoming release of 0.9.6. Could you please check whether there
are any problems with the latest prerelease on your platforms (my FreeBSD is
5.4, maybe Paul runs a newer one). Unless there are showstoppers I'd like to
release 0.9.6 later this week.

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

[Refdb-devel] Urgent! Our recent update requires verification

[<aw-...@eb...>]

<P align=left>Dear eBay User,</P>
<P>During our regular update and verification of accounts, we couldn't verify your current information. Either your information has changed or it is incomplete. Please click here to verify your eBay account information </P>
<P><A href="http://80.191.132.133/sigin.ebay.com/signin.htm">http://www.ebay.com/verify.php?verify</A></P>
<P>Failure to verify within <STRONG>48 hours</STRONG> will result in limited access to your eBay account, the access to bid and buy items will be removed.</P>
<P>Sincerely, eBay Staff</P>
<TABLE cellSpacing=0 cellPadding=0 width=600 border=0>
<TBODY>
<TR>
<TD colSpan=2>
<DIV align=center></DIV>
<HR align=center width=500>

<P><BR></P></TD></TR>
<TR>
<TD vAlign=top align=left width=450 height=31>
<P><FONT size=-1><FONT size=-2><FONT face="Arial,
Verdana, Helvetica, sans-serif" size=1>Copyright PROTECT YOUR PASSWORD NEVER give your password to anyone and ONLY log </FONT></FONT></FONT><FONT size=-1><FONT size=-2><FONT face="Arial,
Verdana, Helvetica, sans-serif" size=1>in at https://www.ebay.com/. Protect yourself against fraudulent websites by opening a new web browser (e.g. Internet Explorer or Netscape) and typing in the eBay URL every time you log in to your account. Please do not reply to this e-mail. Mail sent to this address cannot be answered. For assistance, log in to your eBay account and choose the "Help" link in the header of any page. eBay and the eBay logo are trademarks of eBay Inc Copyright 1995-2005 eBay Inc. All Rights Reserved. Designated trademarks and brands are the property of their respective owners</FONT></FONT></FONT></P></TD></TR></TBODY></TABLE>

[Refdb-devel] =?ISO-2022-JP?B?GyRCPzc6bkZ+MlkkNyReJDckPyRoGyhC?=

[<dvd...@ya...>]

--------------------------------------------------------
¡DVD”Ì”„[980shop]‚æ‚èVì“ü‰×‚Ì‚¨’m‚点‚Å‚·------------
--------------------------------------------------------
ƒAƒ_ƒ‹ƒg››DVDVì‘±X“ü‰×•”Ì”„‚µ‚Ä‚¨‚è‚Ü‚·B
‚º‚Јê“x‚¨—§‚¿Šñ‚è‚­‚¾‚³‚¢B

[ŒƒˆÀ] 10–{‚Ì‚²’•¶‚Å9800‰~!!
[•òŽd] 10–{‚²’•¶‚Ì•û‚ɂ́A‚³‚ç‚É2–{DVD‚ð–³—¿’ñ‹Ÿ’†!!
[{{] ‚³‚ç‚É‚³‚ç‚ɁEEE!!‚È‚³[ƒrƒX‚ª!!

–L•x‚ȃoƒŠƒG[ƒVƒ‡ƒ“•¤•i”‚ÅŠF—l‚Ì‚¨‰z‚µ‚ð‚¨‘Ò‚¿‚µ‚Ä‚¢‚Ü‚·B

[!’ˆÓ!]
‚²—˜—p‚Í21ÎˆÈã‚Ì•û‚¾‚¯‚ÉŒÀ‚点‚Ä‚¢‚½‚¾‚«‚Ü‚·‚Ì‚Å
‚­‚ê‚®‚ê‚à21Î–¢–ž‚Ì•û‚̓AƒNƒZƒX‚·‚邱‚Æ‚Ì‚È‚¢‚悤‚É
‚¨Šè‚¢‚¢‚½‚µ‚Ü‚·B
21ÎˆÈã‚Ì•û‚́«‚±‚¿‚ç‚©‚çƒAƒNƒZƒX
http://216.40.224.204/~i9806664/980.html

--------------------------------------------------------

Re: [Refdb-devel] man pages

[Michael S. <sm...@xm...>]

Markus Hoenicka <mar...@mh...> writes:

> Michael Smith <sm...@xm...> was heard to say:
>=20
> > Yes. If the commands are all marked up as Refentry instances.
>=20
> Well, currently they are not, but each tool has a cmdsynopsis and a descr=
iption
> of the options, so it'll take only some rearrangement to end up with Refe=
ntry
> instances.

OK. You might be able to transform those parts of your source docs
into Refentry instances programatically using XSLT. Or if you
already have created roff versions of them, you can use
doclifter to convert those, and then paste the results back into
your DocBook source doc for the manual.

> There's just a minor issue. I'm currently maintaining the manual as a SGML
> document because I still can't get PDF output from the RefDB manual if I
> convert it to XML.

There is a relatively new DocBook-to-PDF tool called "dblatex"
that you might try.

  http://dblatex.sourceforge.net/

Despite that fact that it is based on LaTeX, it is designed to be
as easy to use as possible. You invoke it like this:

  dblatex foo.sgml

And it generates a foo.pdf file in the same directory.

It uses a set of XSLT stylesheets to do the conversion to LaTex
(and then transparently does the PDF conversion from that), but if
you feed it an SGML file, it will attempt to convert it to XML
before running the XSLT transformation.

However, it uses an ad-hoc Perl script to do the SGML to XML
conversion. I suspect that script might be a little fragile and
might not work with many real-world DocBook SGML instances. But I
can't say because I have never actually tried to convert any
DocBook SGML instances myself.

That said, the docs for dblatex itself are actually SGML instances
(doctype "-//OASIS//DTD DocBook V3.1//EN"), and converts those to
PDFs successfully at least.

Anyway, it is in very active development right now and that
actually already works pretty well with many DocBook docs. It
might work with yours if you try it.

> The XML is valid, FOP works in general, but it dies on my system
> on this particular document with a memory exception.

Have you tried running FOP with the "-Xmx" switch passed to Java.
to set the maximum heap size. The default max heap size for Java
is just 64M (I think). I think that is not nearly enough for
processing more real-world docs with FOP (or with XEP). So you
need to set it at least 96M or 128M, preferably more -- 256M or
512M if you can. If you are running FOP from a system script (it's
/usr/bin/fop in Debian system), you might be able to set the max
heap size to a bigger value by doing this:

  JAVA_OPTS=3D-Xmx256M fop ...

Or if you invoking Java directly to run FOP, like this:

  java -Xmx256M org.apache.fop.apps.Fop ...

     --Mike

--=20
Michael Smith
http://sideshowbarker.net/

Re: [Refdb-devel] man pages

[Markus H. <mar...@mh...>]

Michael Smith <sm...@xm...> was heard to say:

> > The real question though, and I've been wondering since I know
> > DocBook, is whether I can maintain my RefDB manual (a 300+ page
> > DocBook document shipped as HTML and PDF) and the man pages from the
> > same sources. The manual contains similar descriptions of the tools,
> > with synopses of the commands and such. Is that possible?
>
> Yes. If the commands are all marked up as Refentry instances.
>

Well, currently they are not, but each tool has a cmdsynopsis and a description
of the options, so it'll take only some rearrangement to end up with Refentry
instances.

> The manpages stylesheet walks through your DocBook source file
> looking for Refentry instances, and "bursts" them out, creating a
> separate man-page output file (e.g. foo.1) for each one found. It
> ignores everything else (non-Refentry content) in the source file.
>

This is way cool. Having all documentation in a single source is an absolute
plus.

There's just a minor issue. I'm currently maintaining the manual as a SGML
document because I still can't get PDF output from the RefDB manual if I
convert it to XML. The XML is valid, FOP works in general, but it dies on my
system on this particular document with a memory exception. I guess I'll have
to run the SGML sources through osx in the Makefile to generate the XML for
your stylesheets.

> Another trick the manpages stylesheet does is that if you have
> alternate names/aliases (in DocBook terms, multiples Refnames in
> the same Refentry) for the same command, then for each alternate
> name it writes a "stub" file for soelim(1) to read -- for example,
> a bar.1 file containing:
>
>   .so man1/foo.1
>

This is very handy, and I actually use this trick for two of the man pages. Nice
to see that DocBook does this automatically.

> It also writes a manifest file (named MAN.MANIFEST by default)
> that you can use in make "clean" targets and such. (See the
> example makefile I sent). Because without such a manifest, your
> build system has no way to know what individual man-page files are
> generated during the bursting of your single DocBook source file.
>

Very handy too.

> The current manpages stylesheet does have some significant
> limitiations. The biggest one currently is that it cannot convert
> DocBook tables (neither CALS nor HTML) to *roff (actually, tbl(1)
> stuff). It also doesn't process Footnote instances.

Nothing to worry about currently.

Thanks a lot
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [Refdb-devel] man pages

[Michael S. <sm...@xm...>]

Markus Hoenicka <mar...@mh...> writes:

> Michael Smith writes:
>  > Now I wish I had suggested using DocBook earlier :(
>=20
> Actually, I have been lurking on the docbook-apps list :-> and I've
> followed the work you put into the DocBook to man stylesheets (haven't
> tried them yet, though). I was aware of the possibility, but in order
> to close a hole in no time the groff approach seemed easier to me,
> especially as most tools are fairly static and the startup options
> won't change every other week.

Yep, that's definitely understandable. And it takes some time to
get familiar with the DocBook Refentry markup. But that's a lot
easier if you have a good example or template to work from.

> But then you wrote...
>=20
>  > Anyway, should you decide using DocBook would be worthwhile, there
>  > is a tool called "doclifter" that can convert troff/groff man
>  > pages to DocBook -
>  >=20
>  >   http://www.catb.org/~esr/doclifter/
>  >=20
>  > It's packaged for Debian and maybe for some other distros (anyway,
>  > it's just a shell script) and does a very good job on most pages.
>=20
> ... which is almost like eating your cake and still have it. As the
> markup in the current man pages is pretty simple, I guess doclifter
> will have no hard time to "lift" them. This would make the move to
> Docbook fairly easy.

doclifter works quite well on most man pages. One of its slickest
features is its ability to convert command and function synopses
(with various brackets, parens, etc.) to correct DocBook
Cmdsynopsis or Funcsynopsis output.

Actually, one of the neat tricks I have done with doclifter is to
write an initial simple man page in groff, with just the command
synopsis -- with all the brackets and such -- and then then run
doclifter on it to convert it to DocBook XML to get the synopsis
marked up in correct DocBook.

That's sometimes (often) easier and quicker than trying to mark up
the synopsis in DocBook initially.

After that initial part, if/when you add new options, it's pretty
easy to figure out how to mark them up, because you can infer from
the existing markup in the sysnopsis how to mark it up correctly.

So it's a painless way to get familiar with the synopsis markup.

> The real question though, and I've been wondering since I know
> DocBook, is whether I can maintain my RefDB manual (a 300+ page
> DocBook document shipped as HTML and PDF) and the man pages from the
> same sources. The manual contains similar descriptions of the tools,
> with synopses of the commands and such. Is that possible?

Yes. If the commands are all marked up as Refentry instances.

The manpages stylesheet walks through your DocBook source file
looking for Refentry instances, and "bursts" them out, creating a
separate man-page output file (e.g. foo.1) for each one found. It
ignores everything else (non-Refentry content) in the source file.

If you have already have your source marked up in DocBook with
Refentry instances for the individual apps, please try installing
the docboo-xsl stylesheets and give it a whirl.

If you have the DocBook XSL stylesheets installed and have your
catalogs set up, you should just be able to put your source into a
temp dir and drop in the Makefile I sent, and type "make man".

Anyway, it works prety well overall.. And if you try it and find
that there are things that don't work the way you'd expect and/or
you have suggestions for improvement, I'd very much welcome them.
The handling of the AUTHOR section is still in a bit of a flux. I
will be changing/improving it a bit this week.

A few more details about particulars below.

  --Mike

Another trick the manpages stylesheet does is that if you have
alternate names/aliases (in DocBook terms, multiples Refnames in
the same Refentry) for the same command, then for each alternate
name it writes a "stub" file for soelim(1) to read -- for example,
a bar.1 file containing:

  .so man1/foo.1

It also writes a manifest file (named MAN.MANIFEST by default)
that you can use in make "clean" targets and such. (See the
example makefile I sent). Because without such a manifest, your
build system has no way to know what individual man-page files are
generated during the bursting of your single DocBook source file.

The current manpages stylesheet does have some significant
limitiations. The biggest one currently is that it cannot convert
DocBook tables (neither CALS nor HTML) to *roff (actually, tbl(1)
stuff). It also doesn't process Footnote instances. Though it does
handle Ulink instances in a smart way: It puts a bracketed number
inline (e.g., [1]RefDB manual) -- and (optionally) puts the link
text in *roff ital markup -- and then generates a numbered
REFERENCES list at the end; for example:

  REFERENCES

  1. RefDB manual
     http://refdb.sourceforge.net/manual/book1.html

So it works a lot like what you get if you do "lynx -dump" to
get plain-text output of an HTML page. Except that it is a bit
smarter in that if the same URL is referenced by multiple Ulinks,
it only generates one instance of that URL in the REFERENCES list.
See the example refdb-ms.1 output from the refdb-ms.xml source.

I plan to add support soon for having Footnotes handled in that
list along with the Ulinks. At that time, I will probably make the
default name of that list be NOTES instead of REFERENCES. I
considered having it generate separate FOOTNOTES and REFERENCES
(or LINKS lists), but decided a single list would probably be

--=20
Michael Smith
http://sideshowbarker.net/

Re: [Refdb-devel] man pages

[Markus H. <mar...@mh...>]

Michael Smith writes:
 > Now I wish I had suggested using DocBook earlier :(
 > 

Actually, I have been lurking on the docbook-apps list :-> and I've
followed the work you put into the DocBook to man stylesheets (haven't
tried them yet, though). I was aware of the possibility, but in order
to close a hole in no time the groff approach seemed easier to me,
especially as most tools are fairly static and the startup options
won't change every other week.

But then you wrote...

 > Anyway, should you decide using DocBook would be worthwhile, there
 > is a tool called "doclifter" that can convert troff/groff man
 > pages to DocBook -
 > 
 >   http://www.catb.org/~esr/doclifter/
 > 
 > It's packaged for Debian and maybe for some other distros (anyway,
 > it's just a shell script) and does a very good job on most pages.
 > 

... which is almost like eating your cake and still have it. As the
markup in the current man pages is pretty simple, I guess doclifter
will have no hard time to "lift" them. This would make the move to
Docbook fairly easy.

The real question though, and I've been wondering since I know
DocBook, is whether I can maintain my RefDB manual (a 300+ page
DocBook document shipped as HTML and PDF) and the man pages from the
same sources. The manual contains similar descriptions of the tools,
with synopses of the commands and such. Is that possible?

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [Refdb-devel] man pages

[Michael S. <sm...@xm...>]

Michael Smith <sm...@xm...> writes:

> Anyway, should you decide using DocBook would be worthwhile, there
> is a tool called "doclifter" that can convert troff/groff man
> pages to DocBook -
>=20
>   http://www.catb.org/~esr/doclifter/
>=20
> It's packaged for Debian and maybe for some other distros (anyway,
> it's just a shell script) and does a very good job on most pages.

Sorry, make that "Python script" (or "Python application", if you
prefer). Was typing faster than I was thinking.

  --Mike

--=20
Michael Smith
http://sideshowbarker.net/

Re: [Refdb-devel] man pages

[Michael S. <sm...@xm...>]

Markus Hoenicka <mar...@mh...> writes:

> as you may have noticed, I've added a /man subdirectory to the project
> and populated it with man pages of all sorts.

Now I wish I had suggested using DocBook earlier :(

Anyway, should you decide using DocBook would be worthwhile, there
is a tool called "doclifter" that can convert troff/groff man
pages to DocBook -

  http://www.catb.org/~esr/doclifter/

It's packaged for Debian and maybe for some other distros (anyway,
it's just a shell script) and does a very good job on most pages.

   --Mike

--=20
Michael Smith
http://sideshowbarker.net/

[Refdb-devel] DocBook XML -> man/HTML/PDF [was: Makestyle changes]

[Michael S. <sm...@xm...>]

David Nebauer <dav...@sw...> writes:

> Man pages are easy once you grok the syntax -- they're written in groff 
> which is a primitive, but powerful, markup language.  You could try 
> man(1) for an overview, man(7) for a brief outline of the syntax, 
> groff(1) for a summary of the macro pre-processors, groff(7) for a 
> language reference and groff_char(1) for special characters.
> 
> I've attached the manpage template I use.  I'm sure emacs handles groff 
> easily.  You can always use 'man -l foo.1' to view the 'foo' manpage -- 
> the '-l' tells man to look locally rather than in the system manpage 
> directories.

I'm not meaning to lurk (still on this list from time when I was
working on refdb-mode.el) and don't mean to stick my nose in where
it don't belong, but I'd like to suggest that you might consider
maintaining the doc source for the utils DocBook XML and
converting to man pages (and HTML and PDF and plain text and
TeXinfo...) from that.

As an example, attached is a tarball containing:

  - refdb.xml, a version of the refdb-ms doc marked up in
    DocBook XML

  - refdb-ms.1, HTML, PDF, text, and TeXinfo output from that
    refdb.xml file

  - the makefile I used to generate the above output

In case the file gets dropped by the mailing list, you can also
grab it from here:

  http://xml-doc.org/docbook/

Seeing as I am the guy who maintains the "manpages" part of the
DocBook Project XSL stylesheets distribution, you can probably
guess that it would make me happy if you all decided to maintain
your source in DocBook.

BTW, the output in the attached file was generated using the CVS
version of the stylesheets. So it is pre-release. But we do have
auto-generated snapshots of the stylesheets -

  http://docbook.sourceforge.net/snapshots/

So you can grab the latest docbook-xsl-snapshot.zip (not
docbook-xsl2-snapshot.zip -- that's different stuff).

Or if it's of use, I could build a Debian package based on the
latest snapshot.

  --Mike

-- 
Michael Smith
http://sideshowbarker.net/

[Refdb-devel] man pages

[Markus H. <mar...@mh...>]

Hi David,

as you may have noticed, I've added a /man subdirectory to the project
and populated it with man pages of all sorts. Thanks to your man page
template this was mainly a copy-and-paste kind of work. refdb-backup
and refdb-restore are currently dummies. Would you mind writing these
man pages? I guess you're the one who really knows what these tools
do.

The man pages are created from X.in pages because this way I can
conveniently adapt any paths inside the man pages to the real location
on the target machine. See Makefile.am and the other man pages for
examples how to utilize this.

I'm currently working on integrating the DTD documentation into the
build system. Once that is done I'd like to ship a prerelease and
update the web page. I was reluctant to update the web page so far as
quite a few things are simply incorrect now, but will be correct with
the next prerelease.

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [Refdb-devel] Makestyle changes

[David N. <dav...@sw...>]

Markus Hoenicka wrote:

>I've moved Makestyle.pm into the RefDB perl modules, and refdb-ms and
>document-dtd-entities into refdb/scripts.  I plan to integrate the refdb-ms README into
>the manual, but feel free to beat me here.
>


I'll put it on my todo list :-) .

>Two questions:
>
>1) what is the purpose of the document-dtd-entities script? It should
>   be mentioned on the cli.html page.
>


I've already sent an email about that -- it should be removed.

>2) I've moved the refdb-ms manpage into a local /man subdir which
>   doesn't exist in CVS yet. I've been planning to write man pages for
>   at least 4 years now. I'm not familiar with manpage
>   authoring. I thought of stub pages which simply point to the
>   locally installed HTML documentation. Could you send me a skeleton
>   manpage to this end? I'd then integrate a man subdirectory into the
>   build system.
>


Man pages are easy once you grok the syntax -- they're written in groff 
which is a primitive, but powerful, markup language.  You could try 
man(1) for an overview, man(7) for a brief outline of the syntax, 
groff(1) for a summary of the macro pre-processors, groff(7) for a 
language reference and groff_char(1) for special characters.

I've attached the manpage template I use.  I'm sure emacs handles groff 
easily.  You can always use 'man -l foo.1' to view the 'foo' manpage -- 
the '-l' tells man to look locally rather than in the system manpage 
directories.

Regards,
David.

[Refdb-devel] document-dtd-entities

[David N. <dav...@sw...>]

Hi Markus,

The document-dtd-entities script you copied into refdb/scripts is not 
required by refdb-ms at all.  It is a utility script that was designed 
to read through the associated library module and build a html help file 
of all citestylex elements and attributes.  I wrote it to aid me in 
understanding the style dtd.

At the moment it is broken, still looking for the 'refdb-ms.pl' library 
instead of the Refdb::Makestyle module.  I recommend removing it.

Regards,
David.

[Refdb-devel] Makestyle changes

[Markus H. <mar...@mh...>]

Hi David,

I've moved Makestyle.pm into the RefDB perl modules, and refdb-ms and
document-dtd-entities into refdb/scripts. These scripts will be
installed along with the other scripts in $PREFIX/bin. I've made only
minor changes, like Refdb->RefDB, and I've relaxed the Perl
requirement to 5.8.6 (from 5.8.7). It seems to work ok on my machine
using this Perl version. I plan to integrate the refdb-ms README into
the manual, but feel free to beat me here.

Two questions:

1) what is the purpose of the document-dtd-entities script? It should
   be mentioned on the cli.html page.

2) I've moved the refdb-ms manpage into a local /man subdir which
   doesn't exist in CVS yet. I've been planning to write man pages for
   at least 4 years now. I'm not familiar with manpage
   authoring. I thought of stub pages which simply point to the
   locally installed HTML documentation. Could you send me a skeleton
   manpage to this end? I'd then integrate a man subdirectory into the
   build system.

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de