refdb-devel — List for RefDB developers

Re: [Refdb-devel] refdbc -C addref & stdin

[Markus H. <mar...@mh...>]

Bruce D'Arcus writes:
 > On Dec 11, 2003, at 7:15 AM, Michael Smith wrote:
 > 
 > > um, has anybody actually had success at adding references to a RefDB
 > > database via the "Add Reference" menu item Emacs?
 > 
 > Well, I just got this:
 > 
 > /bin/bash: line 1: refdbc: command not found
 > 
 > I don't really understand how emacs deals with environment variables, 
 > because refdbc is in my path in the terminal.
 > 
 > Any ideas?
 > 

You'll have to check where your PATH environment variable is set. bash
reads different config files when it is a login shell, an interactive
non-login shell, or a non-interactive shell. Maybe a shell running
within Emacs picks a config file that contains incorrect path
settings. The bash man page has all the gory details.

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

[Refdb-devel] refdbc -C addref & stdin

[Markus H. <mar...@mh...>]

Michael Smith writes:
 > um, has anybody actually had success at adding references to a RefDB
 > database via the "Add Reference" menu item Emacs?
 > 

Yes, me.

 > Or has anybody had success at adding references by piping standard input
 > to 'refdbc -C addref' on the command line?
 > 

Yes, me.

 > That is, when you try the following, does refdbc stop and wait for you
 > to enter data on standard input?
 > 
 >   refdbc -d refdb2  -t ris -C addref
 > 

Yes.

 > Or, if you do something like either of the following, does it work?
 > 
 >   refdbc -d refdb2  -t ris -C addref < data.ris
 > 
 >   cat data.ris | refdbc -d refdb2  -t ris -C addref
 > 

Yes.

 > None of those work for me. My refdbc doesn't seem to be able to read
 > from standard input at all.
 > 

Could it be you're working on Cygwin? The data-waiting-on-stdin
autodetection does not work on Cygwin, unfortunately, so if you want
to do any of the above on Cygwin you have to use the -f stdin switch.

regards,
Markus


-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [Refdb-devel] refdbc -C addref & stdin

[Bruce D'A. <bd...@fa...>]

Well, I just tested with regular ris data, and got the same error.  
Markus?

Bruce

Re: [Refdb-devel] refdbc -C addref & stdin

[Bruce D'A. <bd...@fa...>]

On Dec 11, 2003, at 10:22 AM, Michael Smith wrote:

>> data read error. Stop.
>
> damn. Well. that one is also coming from refdbc too, and I think it's a
> fatal error. If you're able to succesfully add data via the command 
> line
> but not from within Emacs, then either I'm not using the Emacs
> shell-command-on-region function correctly, or there's something wrong
> with the way your Emacs is interacting with the shell.

So the menu is now able to read risx XML data (or should be)?

Bruce

[Refdb-devel] XHTML output type?

[Michael S. <sm...@xm...>]

In an earlier message, Markus mentioned an 'xhtml' output type. But
unless there's some magic that I'm missing somewhere, there isn't
actually any XHTML output option, right?

  --Mike

Re: [Refdb-devel] refdb db in menu?

[Michael S. <sm...@xm...>]

Hi Bruce,

You wrote:

> Question:
> 
> Should the base db (now called "refdb") be included in the list of dbs 
> in the menu?

I don't know the answer to that. The menu is just generated by calling
'refdbc -C listdb'. So, if the based db doesn't show up when you run
that command from the command line, it won't show up in the menu either.

  --Mike

Re: [Refdb-devel] refdbc -C addref & stdin

[Michael S. <sm...@xm...>]

Hi Bruce,

You wrote:

> On Dec 11, 2003, at 7:15 AM, Michael Smith wrote:
> 
> >That is, when you try the following, does refdbc stop and wait for you
> >to enter data on standard input?
> >
> >  refdbc -d refdb2  -t ris -C addref
> >
> >Or, if you do something like either of the following, does it work?
> >
> >  refdbc -d refdb2  -t ris -C addref < data.ris
> >
> >  cat data.ris | refdbc -d refdb2  -t ris -C addref
> >
> >None of those work for me. My refdbc doesn't seem to be able to read
> >from standard input at all.
> 
> OK, I added the path to my .emacs file.  The first does not add 
> anything, and tells me so.  The other two work fine on my installation.

OK. That's good to know. Thanks for checking.

Tonight was the first time I actually tried adding data to the db, and
environment I was trying it on is a Cygwin install. Haven't tried it on
my Linux install yet.

Anyway, got it working on Cygwin by hacking the refdbc source. So the
both the command line and Emacs interaction are working as expected for
me now.

> If I try to do it from the menu though (this with an risx file), I get 
> this:
> 
> Could not set terminal attributes

That's a warning coming from refdbc. I'm not sure what it indicates, but
I don't think it causes any problems.

> data read error. Stop.

damn. Well. that one is also coming from refdbc too, and I think it's a
fatal error. If you're able to succesfully add data via the command line
but not from within Emacs, then either I'm not using the Emacs
shell-command-on-region function correctly, or there's something wrong
with the way your Emacs is interacting with the shell.

  --Mike

[Refdb-devel] refdb db in menu?

[Bruce D'A. <bd...@fa...>]

Question:

Should the base db (now called "refdb") be included in the list of dbs 
in the menu?

Bruce

Re: [Refdb-devel] refdbc -C addref & stdin

[Bruce D'A. <bd...@fa...>]

On Dec 11, 2003, at 7:15 AM, Michael Smith wrote:

> That is, when you try the following, does refdbc stop and wait for you
> to enter data on standard input?
>
>   refdbc -d refdb2  -t ris -C addref
>
> Or, if you do something like either of the following, does it work?
>
>   refdbc -d refdb2  -t ris -C addref < data.ris
>
>   cat data.ris | refdbc -d refdb2  -t ris -C addref
>
> None of those work for me. My refdbc doesn't seem to be able to read
> from standard input at all.

OK, I added the path to my .emacs file.  The first does not add 
anything, and tells me so.  The other two work fine on my installation.

If I try to do it from the menu though (this with an risx file), I get 
this:

Could not set terminal attributes
data read error. Stop.

Bruce

Re: [Refdb-devel] refdbc -C addref & stdin

[Bruce D'A. <bd...@fa...>]

On Dec 11, 2003, at 7:15 AM, Michael Smith wrote:

> um, has anybody actually had success at adding references to a RefDB
> database via the "Add Reference" menu item Emacs?

Well, I just got this:

/bin/bash: line 1: refdbc: command not found

I don't really understand how emacs deals with environment variables, 
because refdbc is in my path in the terminal.

Any ideas?

Bruce

[Refdb-devel] refdbc -C addref & stdin

[Michael S. <sm...@xm...>]

um, has anybody actually had success at adding references to a RefDB
database via the "Add Reference" menu item Emacs?

Or has anybody had success at adding references by piping standard input
to 'refdbc -C addref' on the command line?

That is, when you try the following, does refdbc stop and wait for you
to enter data on standard input?

  refdbc -d refdb2  -t ris -C addref

Or, if you do something like either of the following, does it work?

  refdbc -d refdb2  -t ris -C addref < data.ris

  cat data.ris | refdbc -d refdb2  -t ris -C addref

None of those work for me. My refdbc doesn't seem to be able to read
from standard input at all.

  --Mike

Re: [Refdb-devel] How to handle custom output types?

[Bruce D'A. <bd...@fa...>]

On Dec 10, 2003, at 8:29 PM, Michael Smith wrote:

> As far as this specific case goes, it seems to me like your "MS Word"
> output type is something that ulimately maybe should be added to the
> "standard" RefDB set of supported output types. I think that'd amount  
> to
> getting your stylesheet into the standard RefDB distro, and adjusting
> the makefile template and maybe adjusting refdbxml.

Yeah, that might be good.  For the record, I'm not using this stuff  
with RefDB yet, which is partly why I mention it (in case Markus has  
input on this issue).

Indeed, a lot of my ideas may be more about RefDB than about its menu.   
So, I agree Mike: the menu should not go out of the way to do stuff for  
the user, but should just be a nice front-end for existing  
functionality.

Markus, how would custom stylesheets be used with RefDB?  In the Word  
example, all it does is use modified versions of the block and footnote  
xsl files, and then a special Word-tailored CSS file.  I really wish  
this was included in the default DocBook (x)html stylesheets as a  
parameter option, frankly.

More at:

http://sourceforge.net/tracker/index.php? 
func=detail&aid=810043&group_id=21935&atid=373750

Bruce

[Refdb-devel] How to handle custom output types?

[Michael S. <sm...@xm...>]

Bruce D'Arcus <bd...@fa...> writes:

> On Dec 9, 2003, at 3:22 AM, Michael Smith wrote:
> 
> >>With 2, could you then create the makefile if it didn't exist?  Say 
> >>you
> >>have an existing document that wasn't created with the "Create New
> >>Document" feature you outline: choose process document, and if there
> >>was no makefile, user is prompted for the style file and it's created
> >>and the transformation is run.
> >
> >Yeah, that's doable. Seems like that's the way it should work. Only
> >alternative is to have Emacs just emit a "No makefile found" error with
> >some kind of instructions about what to do in order to create the
> >makefile (which of course would need to be something different than
> >going to the Create New Document" menu),
> 
> Maybe the solution should also be suitable for changing output styles 
> too?  So then perhaps the new doc thing shouldn't specify and output 
> style and create a makefile?

I guess I'm pretty much in agreement with Markus about the value of
keeping everything makefile-driven. Whatever ends up on the menu will
just be a front-end to the command-line toolchain. So, if it's something
that's doable from the command line -- in this case, doable by passing
different target names and variables to 'make' -- then it's something I
can probably have the menu code do also.

But if it's not something you can currently do from the RefDB
command-line toolchain, then I'm reluctant to add it.

But note that it is possible to have the menu code do stuff that would
take multiple steps from the command. For example, what the "Select
Database" does is eliminate the 'listdb' step you'd otherwise need to do
to see the list of available databases in order to know which name to
feed to the selectdb command.

That's probably not the best example (since, realistically, you probably
already know the name of the db you want to use), but you get the idea.

> Let's see:
> 
> As with the database menu, styles are drawn from refdb.  When one needs 
> to process the document but no style is selected, go to the style, 
> choose the output option, and the makefile is created, and run with the 
> appropriate option.
> 
> If later the user needs to change output styles, it involves the same 
> process:  a simple menu click.
> 
> Menu structure might be:
> 
> 	- Process Document
> 		- Style 1
> 			- all
> 			- html
> 			- pdf
> 			- xhtml
> 		- Style 2
> 			- all
> 			- html
> 			- pdf
> 			- xhtml

Well, I think I really don't want to put style names into the menu --
even if the list of styles is made customizable/extensible. I think it'd
be best just to handle it through Emacs completion.

> I'd like output to be customizable.  For example, I sent Bob S. some 
> stylesheet customizations that use (x)html + css to fool MS Word into 
> thinking it is importing a native doc file.  It would be nice, then, 
> for me to be able to customize RefDB and its menu so I could have a "MS 
> Word" output option.

That sounds great. But I don't have a very clear idea at all how you
could generate a custom output type using the current RefDB toolchain.

Can you describe the steps you're using to generate this output type
now ("fake" Word or whatever you call it)? Are you called refdbxml? Or
are you calling xsltproc or whatever directly? If so, how are you
generating the bibliography for it?

Getting back to the menut, it would be hard to make handle this from the
menu unless you had already created a Makefile for it that had an
additional target in it for your custom output type.

What I could do, I think, is an additional "Other Output Type..." menu
item to the "Generate Output" menu, so it'd look like this:

  Generate Output
    HTML
    PDF
    RTF
    Other Output Type...

If you select "Other Output Type", Emacs then prompts you for the name
of the output type, which would need to be a custom target you've added
to your Makefile, and would need to be formatted exactly as it appears
in the Makekfile -- so "doc" or whatever you call the target.

hmm, but looking at the toolchain now, I see that it seems like the
transform scripts (refdbxml and refdbjade) would also need to be made
aware of the stylesheet for your custom output type...

Unless I'm missing something, it's starting to look to me like it'd be
difficult to generate a custom output type using the current RefDB
toolchain. It'll help me if you can explain how you're doing it now.

As far as this specific case goes, it seems to me like your "MS Word"
output type is something that ulimately maybe should be added to the
"standard" RefDB set of supported output types. I think that'd amount to
getting your stylesheet into the standard RefDB distro, and adjusting
the makefile template and maybe adjusting refdbxml.

  --Mike

Re: [Refdb-devel] new-document and doc-processing menu behavior

[Michael S. <sm...@xm...>]

Markus Hoenicka <mar...@mh...> writes:

> Michael Smith writes:
>  > But AFAICT, there is currently no way, from the command line, of
>  > having the RefDB command-line toolchain create a Makefile for an
>  > existing document. Seems like what's needed is a "new Makefile" script
>  > -- 'refdbnm' or whatever.
>  > 
> 
> It shouldn't be too hard to add a command-line switch to refdbnd that
> suppresses the creation of the skeleton document. This way you could
> create a Makefile for an existing document.

That makes sense. As I mentioned, I think also you might need to add
some logic for asking whether the existing doc uses short or full
notations, since there's not guarantee that the user has followed the
'*.short.*' convention that the current generated Makefiles expect for
short-notation documents.

  --Mike

Re: [Refdb-devel] new-document and doc-processing menu behavior

[Michael S. <sm...@xm...>]

Markus Hoenicka <mar...@mh...> writes:

> Michael Smith writes:
>  > It seems like nothing much actually needs to be changed in the RefDB
>  > tool chain to support processing a doc that uses XIncludes.
>  > 
>  > I think the only toolchain fix needed is to change a couple lines in
>  > refdbxml so that it always calls xsltproc with the "--xinclude" option. 
>  > 
> 
> If this is all it takes, then it's a no-brainer. I assume it doesn't
> hurt to use the --xinclude option even if the file does not contain
> Xincludes?

Right -- if you run it on a doc that doesn't actually have any Xincludes
in it, it just processes it the same way it would have if you hadn't
specified the --xinclude option at all.

I guess it may have some small impact on performance, but I don't think
most people care about that too much when generating output -- and
xsltproc is already very fast.

  --Mike

Re: [Refdb-devel] new-document and doc-processing menu behavior

[Markus H. <mar...@mh...>]

Michael Smith writes:
 > But AFAICT, there is currently no way, from the command line, of
 > having the RefDB command-line toolchain create a Makefile for an
 > existing document. Seems like what's needed is a "new Makefile" script
 > -- 'refdbnm' or whatever.
 > 

It shouldn't be too hard to add a command-line switch to refdbnd that
suppresses the creation of the skeleton document. This way you could
create a Makefile for an existing document.

regards,
Markus
-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [Refdb-devel] new-document and doc-processing menu behavior

[Michael S. <sm...@xm...>]

Bruce D'Arcus <bd...@fa...> writes:

> On Dec 9, 2003, at 7:29 PM, Markus Hoenicka wrote:

[...]

> >If we ever get close to the journal support of the likes of RefMan or
> >EndNote, we'd be looking at roughly 700 style entries in this menu,
> >along with 5 submenus each. I don't think this is going to work. I'd
> >rather have the user select the style and, separately from this, have
> >him process the document.

I agree here

> Well, the styles would only be those loaded in the database, and so 
> would be user-defined styles.  A normal user would only ever have 5-10 
> would be my guess.

The way I plan to handle selecting styles is through 'validated' Emacs
completion against whatever list of styles a query to the database
returns. That'll make it easy to use in that you won't need to remember
style names, and won't need to worry about mis-typing a style name,
because it will constrain you to just those style names that are
actually in the db.

How does that sound?

> I think we should open this stuff about user experience up to the user 
> list for comment, BTW.

That sounds like a good idea to me.

  --Mike

Re: [Refdb-devel] new-document and doc-processing menu behavior

[Michael S. <sm...@xm...>]

Markus Hoenicka <mar...@mh...> writes:

> Bruce D'Arcus writes:
>  > > I feel the best way to specify the style from the RefDB menu is to
>  > > pass this as an argument to the make command.
>  > 
>  > Not following.
>  > 
>  > In any case, I think the ideal is to have some flexibility here.  If I 
>  > create a document on a non-refdb system with some other xml editor, I 
>  > want to be able to easily move it to my refdb box fire up emacs, and 
>  > process the document without hassling with the command line.
>  > 
> 
> I wouldn't mind having the menu code create a Makefile if there is
> none as soon as you attempt to process a document.

I guess it's doable to have the menu code do it, but that seems like
a RefDB feature that's useful outside of the menu and outside of Emacs.
I think users working from the command line have the same need.

I'm thinking of the menu mostly just as a user-friendly front-end to the
existing command line tools. In this case, the Create Document menu will
just call refdbnd and the Generate Output menu will call make and pass
the appropriate target name (and variables, if needed) to that.

But AFAICT, there is currently no way, from the command line, of
having the RefDB command-line toolchain create a Makefile for an
existing document. Seems like what's needed is a "new Makefile" script
-- 'refdbnm' or whatever.

I guess the way that would work is, it would prompt you for the same
info that refdbnd does, with the addition of prompting to ask whether
the document uses short or full notation (because it won't necessarily
have the filename clue to go from).

> I am arguing against modifying the Makefile each time you run this
> command. Switching styles is easier by passing the style name as an
> argument on the make command line. The Makefiles are specifically
> designed to facilitate this. This is absolutely transparent to the
> user and faster as you don't have to write the Makefile back to disk.

I'm a 100 percent in agreement on this. It's easy enough to just pass
the style variable to 'make' along with any target names.

  --Mike

[Refdb-devel] libdbi binaries for Cygwin

[Markus H. <mar...@mh...>]

Hi Mike,

I'll fix that asap. Thanks for the hint.

regards,
Markus

Michael Smith writes:
 > Markus,
 > 
 > Something I forgot to mention after I got my Cygwin RefDB install set up:
 > 
 > You might want to a upload a new libdbi binary to replace the one at:
 > 
 >   http://libdbi.sourceforge.net/downloads/
 > 
 > The one that's there now I couldn't get to work. The reason, as far I
 > can see, is that it doesn't seem to include the configure.in
 > library-extension fix you checked into the libdbi CVS back in July.
 > 
 > Behavior I see with that binary is that libdbi can't recognize the
 > libary files in /usr/local/lib/dbd/. But a binary built from the
 > current CVS source, with that configure.in, works as expected.
 > 
 >   --Mike
 > 
 > 

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [Refdb-devel] new-document and doc-processing menu behavior

[Markus H. <mar...@mh...>]

Michael Smith writes:
 > It seems like nothing much actually needs to be changed in the RefDB
 > tool chain to support processing a doc that uses XIncludes.
 > 
 > I think the only toolchain fix needed is to change a couple lines in
 > refdbxml so that it always calls xsltproc with the "--xinclude" option. 
 > 

If this is all it takes, then it's a no-brainer. I assume it doesn't
hurt to use the --xinclude option even if the file does not contain
Xincludes?

regards,
Markus


-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [Refdb-devel] new-document and doc-processing menu behavior

[Markus H. <mar...@mh...>]

Bruce D'Arcus writes:
 > > I feel the best way to specify the style from the RefDB menu is to
 > > pass this as an argument to the make command.
 > 
 > Not following.
 > 
 > In any case, I think the ideal is to have some flexibility here.  If I 
 > create a document on a non-refdb system with some other xml editor, I 
 > want to be able to easily move it to my refdb box fire up emacs, and 
 > process the document without hassling with the command line.
 > 

I wouldn't mind having the menu code create a Makefile if there is
none as soon as you attempt to process a document. I am arguing
against modifying the Makefile each time you run this
command. Switching styles is easier by passing the style name as an
argument on the make command line. The Makefiles are specifically
designed to facilitate this. This is absolutely transparent to
the user and faster as you don't have to write the Makefile back to
disk.

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

[Refdb-devel] libdbi binaries for Cygwin

[Michael S. <sm...@xm...>]

Markus,

Something I forgot to mention after I got my Cygwin RefDB install set up:

You might want to a upload a new libdbi binary to replace the one at:

  http://libdbi.sourceforge.net/downloads/

The one that's there now I couldn't get to work. The reason, as far I
can see, is that it doesn't seem to include the configure.in
library-extension fix you checked into the libdbi CVS back in July.

Behavior I see with that binary is that libdbi can't recognize the
libary files in /usr/local/lib/dbd/. But a binary built from the
current CVS source, with that configure.in, works as expected.

  --Mike

Re: [Refdb-devel] Emacs menu v1.5

[Michael S. <sm...@xm...>]

Hi Bruce,

You wrote:

> 
> On Wednesday, December 10, 2003, at 12:35  PM, Michael Smith wrote:
> 
> >Please try it and let me know if the new stuff works for you and/or if
> >it breaks anything.
> 
> So far, looks really nice Mike!
> 
> Question: how do I get nXML mode to apply to the output buffer?

You can do it manually by switching to the output buffer and typing
M-x nxml-mode. I can maybe add some code later to have Emacs do it
automatically.

  --Mike

Re: [Refdb-devel] Emacs menu v1.5

[Bruce D'A. <bd...@fa...>]

On Wednesday, December 10, 2003, at 12:35  PM, Michael Smith wrote:

> Please try it and let me know if the new stuff works for you and/or if
> it breaks anything.

So far, looks really nice Mike!

Question: how do I get nXML mode to apply to the output buffer?

Bruce

[Refdb-devel] Emacs menu v1.5

[Michael S. <sm...@xm...>]

Attached are a diff and also complete v1.5 refdb-menu.el file.

The main change is just in database selection. (Probably overkill since
it sounds like most people just use one db anyway, and don't switch, but
the idea was just to make it as unobtrusive as possible.)

Details:

  - Removed the "List Databases" menu items; all available databases are
    now shown on the "Select Database" submenu, with a check next to the
    currently selected database.

  - If you haven't specified a default database, the first time you try
    to do anything that requires database interaction, Emacs will prompt
    you for a database name using completion against the list of
    available databases

  - You can limit which databases show up in the menu and completion
    buffer by setting a different value for the
    refdb-menu-listdb-sql-regexp variable; e.g, set it to "refdb%", and
    only those databases that beging with "refdb" will be shown.

I also added an (incomplete) TODO list to the file.

Please try it and let me know if the new stuff works for you and/or if
it breaks anything.

Thanks,

  --Mike