From: Brian C. <B.C...@po...> - 2004-05-16 15:10:47
|
Hi all. I'm a long, long time user of joe and very satisfied. I believe that life is too short to learn Emacs :-) Anyway, I've just started playing with joe-3.0 and its syntax highlighting, built under FreeBSD-5.2.1. The first thing I wanted to do was to make it highlight XML reasonably. My current attempt is attached, along with a test file, but it doesn't work in quite the way I want. In test.xml, the start of comments and the start of CDATA sections don't get coloured correctly. For example: <!--testing comment--> where I've tried to define colours "CommentStart" (<!--) as green underline, "CommentBody" as green, and "CommentEnd" (-->) as green underline again. This renders as: < cyan (wrong) ! green underline - green underline - green underline t cyan (wrong) e green underline (wrong) s green (ok) ... rest is ok So I think I'm getting confused about the interaction between noeat, recolor, and strings matching. I could probably fix it by writing a whole series of states to match < ! - - and < ! [ C D A T A [ explicitly, but I'd like to use the string matching if possible. Any suggestions as to what I'm doing wrong? Incidentally, the attached files or anything derived from them are free to be used in the project. Thanks, Brian. |
From: <ja...@av...> - 2004-05-16 17:00:11
|
I wrote an XML highlighter- you have to get the latest CVS version. I'll take a look at yours, though. I don't use XML and it's always better for someone who actually uses it to make suggestions. I'll think about the XML editor stuff: I'd like the enhance the macro language enough so that this could be implemented as macros. |
From: Brian C. <B.C...@po...> - 2004-05-16 18:28:06
|
On Sun, May 16, 2004 at 01:02:19PM -0400, ja...@av... wrote: > I wrote an XML highlighter- you have to get the latest CVS version. I'll > take a look at yours, though. I don't use XML and it's always better for > someone who actually uses it to make suggestions. Thanks, just pulled it down. Run against joe-3.0 it doesn't seem to work particularly well for me (e.g. tags like <greeting> are not highlighted) so I guess I should recompile the binary from CVS too. Which means go fetch automake/autoconf... OK done that. There is a compilation problem with FreeBSD: if gcc -DHAVE_CONFIG_H -I. -I. -I. -DJOERC="\"/usr/local/etc/joe/\"" -g -O2 -MT tty.o -MD -MP -MF ".deps/tty.Tpo" -c -o tty.o tty.c; then mv -f ".deps/tty.Tpo" ".deps/tty.Po"; else rm -f ".deps/tty.Tpo"; exit 1; fi tty.c:31:17: pty.h: No such file or directory *** Error code 1 It seems that we do have openpty, but not pty.h. From man openpty: LIBRARY System Utilities Library (libutil, -lutil) SYNOPSIS #include <sys/types.h> #include <sys/ioctl.h> #include <termios.h> #include <libutil.h> int openpty(int *amaster, int *aslave, char *name, struct termios *termp, struct winsize *winp); int forkpty(int *amaster, char *name, struct termios *termp, struct winsize *winp); Anyway, sticking those headers in place of pty.h in tty.c made it compile. It didn't like my old joerc so I replaced it. Now, if I do ./joe -syntax xml test.xml I get a core dump: #0 0x08056956 in glopt (s=0xbfbfed9f "syntax", arg=0xbfbfeda6 "xml", options=0x0, set=1) at rc.c:384 384 options->syntax_name = (unsigned char *)strdup((char *)arg); (gdb) bt #0 0x08056956 in glopt (s=0xbfbfed9f "syntax", arg=0xbfbfeda6 "xml", options=0x0, set=1) at rc.c:384 #1 0x08053ea6 in main (argc=4, argv=0xbfbfecb8, envv=0x80908d0) at main.c:296 #2 0x080496c2 in _start () However, "joe test.xml" works and selects the xml syntax mode. It still does not work well though; it appears to be the same as with joe-3.0. Using the 'test.xml' which I posted before, I get cyan for strings: <?xml version="1.0" encoding="UTF-8" ?> ^^^^^ ^^^^^^^ cyan cyan <!ENTITY % draft 'INCLUDE' > ^^^^^^^^^ cyan green for comments and bold cyan for entity references (&), but everything else is black on white. Could you try opening test.xml and see if it's the same for you? I'd still like to understand why my xml.jsf doesn't work completely as expected though. Yours looks for <!-- through separate states, and it would be tedious to do <![CDATA[ like that. Also, I decided to have separate colour names for the start of a comment, the comment, and the end of a comment (ditto for CDATA) mainly because I thought it might be useful for higher-level XML processor to have this level of analysis done already. That needs a bit of recolouring. I could take it out, but I'd still like to know what I did wrong :-) > I'll think about the XML editor stuff: I'd like the enhance the macro > language enough so that this could be implemented as macros. That would certainly be nice, although I'm not sure how it could be done without turning macros into a full programming language - at which point you'd probably be better linking to libperl or libruby rather than writing a new language from scratch. There is an Emacs module called PSGML, but that (a) requires Emacs to use, and (b) requires learning LISP to program/modify. Cheers, Brian. |
From: <ja...@av...> - 2004-05-16 19:21:26
|
Brian Candler <B.C...@po...> wrote: >OK done that. There is a compilation problem with FreeBSD: Thanks for trying this- I use slackware linux, not FreeBSD and I recently applied somebody's patch for openpty. >./joe -syntax xml test.xml >I get a core dump: This and the above should now be fixed in the latest CVS version. Can do a "cvs update ." in the joe-current directory and try it? You have to begin with "aclocal" etc. >It still does not work well though; it appears to be the same as with >joe-3.0. Using the 'test.xml' which I posted before, I get cyan for strings: Oh, I had forgotten that the released joe 3.0 had the xml highlighter. I'll look at your highlighter now :-) Ok, I looked at it. The problem is that you can't have options after strings- you really need recolor and noeat to do what you want. It's a good idea, so I will change the highlighter code to allow for options. It should be ready later today or tomorrow: I'll send mail to you again. |
From: Brian C. <B.C...@po...> - 2004-05-16 21:34:25
Attachments:
xml2.jsf
|
On Sun, May 16, 2004 at 03:23:43PM -0400, ja...@av... wrote: > This and the above should now be fixed in the latest CVS version. Can do a > "cvs update ." in the joe-current directory and try it? You have to begin > with "aclocal" etc. Will do. > >It still does not work well though; it appears to be the same as with > >joe-3.0. Using the 'test.xml' which I posted before, I get cyan for strings: > > Oh, I had forgotten that the released joe 3.0 had the xml highlighter. It didn't - I just pulled the xml.jsf from CVS and tried it with 3.0 before trying it with the CVS version of the binary. > Ok, I looked at it. > > The problem is that you can't have options after strings- you really need > recolor and noeat to do what you want. Thanks. Actually I realised that shortly after posting, and tried introducing an intermediate state to get around it: :decl Decl * decl strings "!--" comment_fix <<<<<<<< "![CDATA[" cdata_fix <<<<<<<< done "<" decl_nest ">" start :cdata_fix CdataStart <<<<<< * cdata_start noeat recolor=-9 <<<<<< :cdata_start CdataStart * cdata noeat ...etc :comment_fix CommentStart <<<<<< * comment_start noeat recolor=-4 <<<<<< :comment_start CommentStart * comment noeat ...etc The full .jsf is attached if you'd like to try it. What I now get is an off-by-one error, for example <!--testing--> < is cyan (wrong) !-- is green underlined (ok) t is green underlined (wrong) esting is green (ok) It looks like "recolor" works from the position *after* the matched character, even if "noeat" was specified. Is that intentional? If so, I can't see how to work around this. > It's a good idea, so I will change > the highlighter code to allow for options. It should be ready later today > or tomorrow: I'll send mail to you again. Excellent, thanks. Best wishes, Brian. |
From: <ja...@av...> - 2004-05-20 03:05:31
|
I've changed the sytnax highlighter so you can have the recolor option after strings (get latest version from CVS). Try this (your) highlighter for xml: (note that recolor=-10 is relative to one character past the end of the string). Send me your finished xml highlighter if you want- I'll include it with JOE. # Define no. sync lines # You can say: # -200 means 200 lines # - means always start parsing from beginning of file when we lose sync # if nothing is specified, the default is -50 - # Define colors # # bold inverse blink dim underline # white cyan magenta blue yellow green red black # bg_white bg_cyan bg_magenta bg_blue bg_yellow bg_green bg_red bg_black # The underlines are here right now just because I want to distinguish which # bits have been coloured (say) CdataStart, CdataBody, CdataEnd. And that's # because I think it may be useful to make that distinction for some people. =Idle =Tag blue =Error red bold =EntityRef magenta =Decl cyan =CommentStart green underline =CommentBody green =CommentEnd green underline =PIStart cyan bold underline =PIBody cyan bold =PIEnd cyan bold underline =CdataStart blue bold underline =CdataBody blue bold =CdataEnd blue bold underline :start Idle * start "<" open_tag recolor=-1 ">" error noeat recolor=-1 "&" entity recolor=-1 :error Error * start :open_tag Tag * tag "?" pi_start recolor=-2 "!" decl recolor=-2 buffer :tag Tag * tag "<" error noeat recolor=-1 "&" entity_attr recolor=-1 ">" start :decl Decl * decl strings "!--" comment_start recolor=-5 "![CDATA[" cdata_start recolor=-10 done "<" decl_nest ">" start # We allow one level of <...> nesting within declarations :decl_nest Decl * decl_nest ">" decl :comment_start CommentStart * comment noeat :comment CommentBody * comment "-" comment2 :comment2 CommentBody * comment "-" comment3 :comment3 CommentBody * comment_error noeat recolor=-3 ">" comment_end recolor=-3 :comment_end CommentEnd * start noeat recolor=-1 # For compatibility, the string "--" (double-hyphen) MUST NOT occur within # comments. [http://www.w3.org/TR/REC-xml/ section 2.5] :comment_error Error * comment "-" comment_error ">" comment_end recolor=-3 :pi_start PIStart * pi noeat recolor=-1 :pi PIBody * pi "?" pi2 :pi2 PIBody * pi ">" pi_end recolor=-2 :pi_end PIEnd * start noeat recolor=-1 :cdata_start CdataStart * cdata noeat :cdata CdataBody * cdata "]" cdata2 :cdata2 CdataBody * cdata "]" cdata3 :cdata3 CdataBody * cdata ">" cdata_end recolor=-3 :cdata_end CdataEnd * start noeat recolor=-1 :entity EntityRef * entity ";" start :entity_attr EntityRef * entity_attr ";" tag |
From: Brian C. <B.C...@po...> - 2004-05-26 17:20:26
|
On Wed, May 19, 2004 at 11:07:41PM -0400, ja...@av... wrote: > I've changed the sytnax highlighter so you can have the recolor option after > strings (get latest version from CVS). Try this (your) highlighter for xml: Apologies for the delay in responding; I have been travelling. As I write this I'm sorting out automake/autoconf on my desktop machine so I can build joe-current. While that's going on: I mentioned before I think some slightly odd behaviour, which is if you match "*" and specify "noeat", then the recolor option works from a position _after_ the matched but uneaten character. If that's intentional, then perhaps it would be useful to have a "match nothing" option, in addition to "match any character". The difference would be that the pointer would not be moved forward in the text, so recolor works from one position earlier. I think this would simplify some of the syntax files. Otherwise, would it make sense that when "noeat" and "recolor" are applied at the same time, that "recolor" applies from the cursor position *before* the matched character? Or would that break too many existing .jsf files? Regards, Brian. |
From: Brian C. <B.C...@po...> - 2004-05-26 21:05:00
|
On Wed, May 26, 2004 at 06:19:17PM +0100, Brian Candler wrote: > Apologies for the delay in responding; I have been travelling. As I write > this I'm sorting out automake/autoconf on my desktop machine so I can build > joe-current. It's built, and the ability to do recolor on a 'strings' match works - thanks! I'd like to do some more work on that XML jsf; your original one did highlighting of attribute strings (attr='abc' attr="abc") which probably makes sense to have, if only for basic syntax checking. Bit busy at the moment though... Cheers, Brian. |
From: Brian C. <B.C...@po...> - 2004-05-27 09:37:18
|
On Wed, May 26, 2004 at 10:00:29PM +0100, Brian Candler wrote: > It's built, and the ability to do recolor on a 'strings' match works - > thanks! One thing I don't understand is why the 'strings' match doesn't seem to take effect until the following character. For example: :decl Decl * decl strings "!--" comment_start recolor=-5 "![CDATA[" cdata_start recolor=-10 done "<" decl_nest ">" start If you type <!-- then it doesn't get coloured until you type the fifth character: <!--h And presumably that's why "recolor=-5" instead of "recolor=-4" is needed. Is that a bug or a feature? Also, I've not tested it, but is 'strings' matching clever enough to work if one string is a prefix of the other? e.g. "foo" "foobar" After typing "foo" then both these will match; so if it were very clever it would defer a decision until sufficient extra characters have been received to make it unambiguous. If that were true, the XML syntax highlighter could be simplified a bit. Final point. "\r" seems to match the letter 'r' rather than ^M. Perhaps either the full set of C-escapes should be implemented, or else unrecognised escapes should be binned. Cheers, Brian. |
From: Brian C. <B.C...@po...> - 2004-05-16 22:20:54
|
On Sun, May 16, 2004 at 03:23:43PM -0400, ja...@av... wrote: > Thanks for trying this- I use slackware linux, not FreeBSD and I recently > applied somebody's patch for openpty. The CVS version compiles cleanly and doesn't dump core - thank you. And I realised that because iconv is no longer used I don't need to put -I/usr/local/include and -L/usr/local/lib in CPPFLAGS/LDFLAGS any more either. Incidentally, joe-3.0 would not compile at all under FreeBSD-4.8 (my laptop), because it required towupper and iswalnum which are not available on that platform. I checked, and found the whole of <wctype.h> is wrapped in #if 0 /* XXX: not implemented */ ... #endif That's old libraries for you; FreeBSD-5.2.1 has these functions. But the CVS version of joe compiles and runs just fine under 4.8. I might even be tempted to replace my trusty joe-2.8 with this :-) Quick question: I notice that when editing a C program, joe now displays the name of the function I'm within (or the text of the previous line which starts in column 1, excluding lines which start '{'). How does that happen? I can't see in either joerc or c.jsf where that's enabled. I see '%x' in the status line definition, but if I copy foo.c to foo.txt then open it, this feature doesn't appear; not even if I subsequently do ^T Y and select 'c' syntax. Final question: the only strange issue I have ever had with joe is when doing paragraph reformatting (^K J). If I have a paragraph where one line happens to start with a non-alpha character, that character gets duplicated or moved. For example: I am writing some text about configuration files, and I would like to mention the one /etc/passwd which is very important in Unix systems. This file contains the system accounts becomes after ^KJ I am writing some text about configuration files, and I would like to /mention the one etc/passwd which is very important in Unix systems. This /file contains the system accounts (/ has become detached from /etc/passwd, and has been duplicated on the line below). Was this meant to help with bulleted lists perhaps? For me, it never seems to do The Right Thing[TM]. It's just a minor annoyance though, typically when composing E-mail. Cheers, Brian. |
From: <ja...@av...> - 2004-05-17 00:36:12
|
Brian Candler <B.C...@po...> wrote: >Quick question: I notice that when editing a C program, joe now displays the >name of the function I'm within (or the text of the previous line which >starts in column 1, excluding lines which start '{'). How does that happen? >I can't see in either joerc or c.jsf where that's enabled. >I see '%x' in the status line definition, but if I copy foo.c to foo.txt >then open it, this feature doesn't appear; not even if I subsequently do ^T >Y and select 'c' syntax. It's enabled when autoindent is enabled. It's not such a great assumption: autoindent implies you have a block structured language, but necessarily one where everything is encapsulated in a function (TCL is like this, for example). I should probably add yet another option. >Final question: the only strange issue I have ever had with joe is when >doing paragraph reformatting (^K J). If I have a paragraph where one line >happens to start with a non-alpha character, that character gets duplicated >or moved. For example: It's for doing paragraph reformat of mail quotes. I think I had limited it to just '>' at one point, but some mailers use different characters for quoting text (plus it's useful for C comments where all lines begin with * and other languages which use '#', '!' or even C++ with //). This needs an overhaul: the occasional /etc/passwd at the beginning of a line is not so bad, but for LaTeX, the frequency of weird characters at the beginnings of lines is very high. |
From: Brian C. <B.C...@po...> - 2004-05-17 09:07:03
|
On Sun, May 16, 2004 at 08:38:21PM -0400, ja...@av... wrote: > It's for doing paragraph reformat of mail quotes. I think I had limited it > to just '>' at one point, but some mailers use different characters for > quoting text (plus it's useful for C comments where all lines begin with * > and other languages which use '#', '!' or even C++ with //). This needs an > overhaul: the occasional /etc/passwd at the beginning of a line is not so > bad, but for LaTeX, the frequency of weird characters at the beginnings of > lines is very high. The things which catch me most often are: *emphasised* text at the start of a line _underlined_ text at the start of a line - a hyphen at the start of a line (I never edit other people's mail quotes - it strikes me as an impolite thing to do...) Cheers, Brian. |
From: Brian C. <B.C...@po...> - 2004-05-27 11:22:53
Attachments:
xml-new2.jsf
test.xml
|
Here's a new version of the XML syntax highlighter. It enforces the spec reasonably strictly, except for internal DTD subsets, <!...> which are too complex to validate properly in a simple state machine. You might want to change the colours, although I think they're not too bad now. One thing I noted is that character classes such as "letter" or "digit" have specific unicode definitions: http://www.w3.org/TR/2004/REC-xml-20040204/#CharClasses So what I've written is not going to work for XML documents where element or attribute names are outside the 'normal' Latin characters. It could be made more lax by listing all unacceptable characters in the Latin character sets, and accepting everything else. Cheers, Brian. |
From: <ja...@av...> - 2004-05-27 13:55:03
|
Thanks. I've checked it in. |