#1210 A lexer for maxima

[Gunter Königsmann]

Maxima is a computer algebra system. That means it can solve all mathematical questions Octave can (even if maxima is slower). But it additionally is specialized in finding an equation that solves the problem.

One example:

(%i1)   eq1:(a+b)^3=c;
(eq1)   (b+a)^3=c
(%i2)   expand(eq1);
(%o2)   b^3+3*a*b^2+3*a^2*b+a^3=c
(%i3)   solve(eq1,b);
(%o3)   [b=((-1)/2-(sqrt(3)*%i)/2)*c^(1/3)-a,b=((sqrt(3)*%i)/2+(-1)/2)*c^(1/3)-a,b=(3*c^(1/3)-3*a)/3]

The maxima Project frequently gets requests if a text editor can be equipped with syntax highlighting for maxima and typically this text editor uses scintilla => attached to this ticket you'll find a shiny new maxima lexer.

Thanks in advance and also thanks to the mailing list that was very helpful in creating this lexer.

Gunter.

[Gunter Königsmann]

The necessary changes for scite, as well.

[Neil Hodgson]

Some of the changes appear to be incomplete, posibly due to being performed manually instead of by running scite/scripts/RegenerateSource.py. For example, the Xcode project has duplicated file ID numbers which means that the references between project sections won't work and Scintilla will fail to link. The necessary changes will be made if you take downloaded copies of Scintilla and SciTE; add LexMaxima.cxx and maxima.properties in the appropriate directories; add IDs to Scintilla.iface; run RegenerateSource.py.

Language-based menus and lists are already very long in SciTE so new languages should be added to the imports.exclude list in SciTEGlobal.properties. Users can then sutomize the set of loaded languages in their user options file.

The file extension "out" is also used by the Spice language which means that they clash: open x.out and either Spice or Maxima will be active for each relevant property. The choices will be arbitrary and may differ between properties. This is another reason for putting languages in imports.exclude but its also worth reevaluating whether "out" is strongly associated with the language or is just a format for listings.

The second argument to *language is a file extension as the Language menu works by appending that 'override extension' to the file name when testing it. Thus this should use "wxm" or whichever extension is most common. Chek it by flipping to another language then back to Maxima.

*language.maxima=Maxima|m.maxima||

Scintilla is supposed to build with no warnings using the 3 popular compilers so these should be fixed by omitting the name of the parameters:

..\lexers\LexMaxima.cxx(38): warning C4100: 'keywordlists': unreferenced formal parameter
..\lexers\LexMaxima.cxx(37): warning C4100: 'initStyle': unreferenced formal parameter

[Neil Hodgson]

Calling isdigit, isalpha, or most other is* library calls is undefined for non-ASCII char, and can lead to exceptions or worse. Each of these calls should be guarded with a call to IsASCII.

[Gunter Königsmann]

Thanks a lot!
* Undid the manual changes and used the Python script
* is*() is now guarded by IsASCII()
* Maxima is now excluded from the languages menu
* the extension .out is rarely used by maxima users as you already suspected: Only one front-end allows to save files as .out-files and this front-end doesn't allow to load these files again => removed it from the list of maxima files.
* The default extension for maxima is .mac
* Thanks for the info on how to avoid the warning!

Now by default scite no more syntax-highlights .mac files. Is this to be expected?

[Gunter Königsmann]

If you want to test the new lexer: Attached a simple test file.

[Gunter Königsmann]

Seems like the last change to the lexer has broken it somehow. Will report back as soon as this is resolved again.

[Gunter Königsmann]

Ok... ...now it works again => Thanks in advance - and thanks for the help,

Gunter.

[Gunter Königsmann]

One last correction...

[Neil Hodgson]

Committed as [7a2aee] and [55aa4d].

There are some issues.

If you type into the 2nd or 3rd line of the initial multi-line comment of the example, that line does not stay in comment colour. The purpose of the initStyle parameter is to remember this sort of multi-line construct when the lexer is asked to process a range from the start of the line that the user edited, not from the start of file.

Line 195:

if(!IsASCII(ch) || !isspacechar(nextNonwhitespace))

A different variable is checked for ASCII than is passed. Its OK to pass non-ASCII to isspacechar as its provided by Scintilla to be a safe version of the C++ library's isspace.

The last line appears to always be lexed continuously. This may be because the assignment of styles stops one byte before the end of the region requested.

Xcode's Analyze thinks the lifetime of chNext is confused as all three instances listed are immediately overwritten by the assignment on line 154.

/Users/neil/merc/scintilla/lexers/LexMaxima.cxx:43:8: warning: Value stored to 'chNext' during its initialization is never read
  char chNext = styler[startPos];
       ^~~~~~   ~~~~~~~~~~~~~~~~
/Users/neil/merc/scintilla/lexers/LexMaxima.cxx:185:6: warning: Value stored to 'chNext' is never read
            chNext = styler.SafeGetCharAt(i + 1);
            ^        ~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/neil/merc/scintilla/lexers/LexMaxima.cxx:206:2: warning: Value stored to 'chNext' is never read
        chNext = styler.SafeGetCharAt(i + 1);
        ^        ~~~~~~~~~~~~~~~~~~~~~~~~~~~
3 warnings generated.

[Gunter Königsmann]

Wow cool! Thanks a lot!

That means I'll try to be as fast to respond as you are and attach the new version of the lexer and the patch that should now be free of the problems you've found. That means if Sourceforge's spambot protection allows me to.

Kind regards,

Gunter

[Gunter Königsmann]

Another small bug, again including a patch against master.

[Neil Hodgson]

The changes don't fix the bad colouring when typing inside a multi-line comment at file start like in the example file. I think the problem is that it should go back one more position to 0.

Since the scope of chNext is just inside the loop, its common to declare it inside the loop just like ch and this leads to fewer warnings.

[Gunter Königsmann]

You are completely right => Attached to this message you'll find the updated patch.

Thanks a lot,

Gunter.

[Neil Hodgson]

Committed as [065b6d].