Rationale-neoteric

Neoteric-Expressions (n-expressions)

Lisp’s standard notation is different from “normal” notation in that the parentheses precede the function name, rather than follow it. Others have commented that it'd be valueable to be able to say name(x) instead of (name x):

• Jorgen ‘forcer’ Schaefer argues that this is a more serious problem than the lack of infix notation; on July 2000 he said “I think most people would like Scheme a lot better if they could say lambda (expression) ... instead of (lambda (expression) ...”
• Peter Norvig had a reader implementation in which “if a function name ends with an open parentheses, move it inside the list (when converting to an s-expression)”. This means that “(fact x)” and “fact(x)” will mean the same thing.
• Skill from Cadence, a proprietary Lisp-based extension language, also supports name-prefixing.

Basic neoteric forms

Neoteric-expressions build on curly-infix's use of {...}. If (...), {...}, or [ ... ] are prefixed with a symbol or list (i.e., have no whitespace between them), they have a new meaning in neoteric-expressions:

1. Prefixed (...). Syntax of the form e(...) - with no whitespace between e and the open parenthesis - are mapped to (e ...). Any parameters in "..." are space-separated. This produces another expression, so this can be repeated (left-to-right). This adds support for traditional function notation. For example, "cos(x)" maps to "(cos x)", "max(3 4)" maps to "(max 3 4)", and "f(x)(a b)" maps to "((f x) a b)". Note that this is especially convenient for certain styles of functional programming, including lambda expressions; in Scheme, lambda((x) {x + x})(4) would compute as 8.
2. Prefixed {...}. A prefixed expression e{...} is an abbreviation for e({...}). ? This rule simplifies combining function calls and infix expressions when there is only one parameter to the function call. This is a common case; for example, "not" (which is normally given only one parameter) often encloses infix "and" and "or". Thus, f{n - 1} maps to (f (- n 1)). When there is more than one function parameter, use the normal term-prefixing format instead, e.g., f({x - 1} {y - 1}) maps to (f (- x 1) (- y 1)).
3. Prefixed [...]. Prefixed square brackets e[...] maps to (bracketaccess e ...). ? Thus, "t[x]" maps to "(bracketaccess t x)". This is intended to simplify use of indexed arrays, associative arrays, and similar constructs. You could even define bracketaccess as a macro that simply returns its arguments; in this case f[5] would eventually map to (f 5).

These combine well with curly-infix forms of {...}. For example, {-(x) * y} maps to (* (- x) y).

Why the (. e) rule?

Neoteric-expressions require that (. x) must mean x.

It is critically important that expressions like read(. port) be supported so you can represent, in the obvious way, (read . port). If (. x) didn't mean x, then it would be easy to get this case wrong. What's more, implementing this would require special treatment. Also, if someone wanted to build on top of an existing reader, they would have to reimplement parts of the list-processing system if this wasn't handled.

This also provides a simple way to escape certain constructs, in particular, some special symbols in sweet-expressions. Sweet-expressions (to be discussed later) need an escape mechanism for characters and symbols like !, \\, and $, so that they can be directly represented. This means that (. $) just means the symbol $, even if $ by itself would normally have some other meaning when being processed by a higher-level reader (such as a sweet-expression reader).

It was far easier to define this escape mechanism as part of neoteric-expressions. This rule could have been defined as part of sweet-expressions, but that would create an implementation problem. We fully expect that implementers will work in stages; in particular, some may not want to build in indentation processing, but they might be willing to build a neoteric-expression system into their reader. If you built a sweet-expression reader on top of a neoteric-expression reader, but that reader didn't implement (. e), then you'd have to re-implement the whole reader underneath anyway. But if all neoteric-expression readers support (. e), then a sweet-expression reader is far more trivial to build on top.

It is already true that (. x) is x in guile, so there was already a working example that this is a reasonable extension. In fact, in a typical implementation of a list reader, it takes extra effort to prevent this extension, so this is a relatively easy extension to include.

Comma-separated parameters

It would be possible to define neoteric-expressions to have comma-separated values in a function call; this would make it even more similar to traditional function call notation. If you simply threw out commas, this would interfere with ,-lifting - and this was quickly rejected.

A better rule, that would indeed work, would be to require each parameter to end with a comma, and then remove that ending comma. However, this rule:

• would obscure any comma used for ,-lifting (making them hard to find).
• is inconsistent with "normal" Lisp lists, which do not use commas this way, as well as being inconsistent with the simple space-separated parameters of sweet-expressions described below. This would make it harder to switch formats and possibly hamper adoption.
• is completely unnecessary. Whitespace is quite sufficient (and clear) for syntactically separating parameters
• Creates clutter. Parameters are very common, so creating an additional extra character for every parameter, to write and read, appeared to be a poor approach.

Many other languages do use commas, but they are required in those languages because infix operators are not surrounded by any marker. Since infix operations are already surrounded by {...} in our notation, there is no need for the additional commas for parameter separation.

Experimentation found that separating parameters solely by whitespace worked well, so that approach was selected.

Neoteric alternatives

Originally the prefix had to be a symbol or list. The theory was that by ignoring others, the sweet-reader would be backwards-compatible with some badly-formatted code, and some errors might not result in incorrectly-interpreted expressions. But this was an odd limitation, and in some cases other prefixes made sense (e.g., for strings). This was changed to eliminate the inconsistency.

At one time it was required that unprefixed [ ... ] be the same as ( ... ), but some Lisps interpret unprefixed [ ... ] specially (e.g., Arc). Thus, it was decided that it'd be better to simply leave [ ... ] unchanged in interpretation. Note that in Scheme R6, [ ... ] does have the same meaning as ( ... ) when unprefixed, but this is a property of Scheme R6 not of neoteric-expressions.

Neoteric-expressions used to be called "modern-expressions". But some people didn't like that name, and obvious abbreviation ("m-expression") was easily confused with the original Lisp M-expression language. So the name was changed to neoteric, which has a similar meaning and abbreviates nicely. It wasn't called "function-expressions" because "f-expressions" are previously used (and can sound bad if said quickly), and they weren't called "prefix-expressions" because "p-expressions" sound like "pee-expressions". It's not called "name-prefix" because the prefix need not be a name.

Comments on neoteric rules

The neoteric rules do introduce the risk of someone inserting a space between the function name and the opening “(”. But whitespace is already significant as a parameter separator, so this is consistent with how the system works anyway... this is not really a change at all.

Obviously, this is trivial to parse. We don’t lose any power, because this is completely optional -- we only use it when we want to, and we can switch back to the traditional s-expression notation if we want to. It’s trivially quoted... if you quote a symbol followed by “(”, just keep going until its matching “)” -- essentially the same rule as before!

Errors that have made people reject this in the past

The article Improving lisp syntax is harder than it looks discusses name-prefixing systems like this, but it makes a number of errors:

• It first claims that this would be hard to integrate with macros, which isn't true. He says, "Under the current syntax macros can treat the code as a series of nested lists, which makes it easy to write intuitive looking macro expansions, for example if a macro expands into '(display "text") it is pretty obvious what it does. Although in theory you could keep this macro system with a new Lisp syntax it would look strange, and basically force users of the language to know both the old syntax and the new syntax. Thus we would expect the macros to read in the function call under this kind of syntax as some new kind of object, with one operation returning the function symbol, and another operation returning the list of parameters. Thus a macro expansion would have to look something like this: build-call('display '("text"))." Absolutely false. There's no reason you have have those kinds of "two object" semantics, and once you realize there are easier ways to handle it, integration with macros is trivial. And "having to know the old and new syntax" is no big deal, it's already true that people need to know that 'x and (quote x) mean the same thing. If the reader transforms a(x) into (a x), then when the macro has a chance to run all it sees is (a x) - exactly what it was expecting to see.
• "The real motivation for leaving Lisp syntax as it is comes from macros. Not only does this expansion fail to visually look the same it also is much more complicated. One could try to get around this by altering the quasiquote operator, as in TwinLisp, so that the expansion becomes `display("text"), but then we have sacrificed the simplicity of the quasiquote, which no longer operates on lists. No matter how you solve the problem you end up in a bit of a bind." Not true. Quasiquote still works on lists, and display("text") is just an alternate way to express a list. After all, (display "text") is actually not the real representation of a list either; a more accurate representation would be (display . ("text" . ())).
• He also says that "Another disadvantage to this change of syntax is that it makes functional programming much more odd looking. Let's say you have a list containing functions and you want to call the first one. In Scheme you write ((car lst) params) and in Common Lisp (funcall (car lst) params). However in our new syntax it looks like: car(lst)(params) and funcall(car(lst) (params)). Neither of these is very elegant, and it only gets worse if that call in turn returns a function, which would look like: car(lst)(params)(params2) and funcall(funcall(car(lst) (params)) (params2))." But I think this argument is backwards; I find this notation remarkably elegant, and better than the traditional notation. This way, to do functional programming, just cuddle up the parentheses. It's much easier to understand sequential parentheses compared to a deeply nested list. But again this misses the point; if you prefer to write ((car lst) params) then do so; these prefixed forms are simply convenient notations that you can use when you find them useful, just as you can always write (quote x) if you don't find it helpful to write 'x.

Technically, this is a change from some official Lisp s-expression notations and implementations. For example, “a(b)” in traditional Scheme or Common Lisp is the same as “a (b)” -- its parser tries to return the value of a, followed by running the function b. But it’s not clear it’s a big change in practice; commonly accepted style always separates parameters (including the first function call name) with whitespace. So normally, what follows a function call’s name is whitespace or “)”, and this is enforced by pretty-printers. Thus, many large existing Lisp programs could go through this kind of parsing without resulting in a change in meaning!

User impact of neoteric-expressions

With neoteric-expressions you can easily use the traditional Lisp read-eval-print loop (at the command line), e.g., as a calculator. Just remember to surround infix expressions with {...} and surround infix operators with whitespace. For example, "{3 + 4}" will be mapped to (+ 3 4), which when executed will produce "7". Use normal function notation for unary functions, e.g., "{-(x) / 2}" maps to "(/ (- x) 2)". Nest {...} when you need to, e.g., "{3 + {4 * 5}}" will map to "(+ 3 (* 4 5))".

Neoteric-expressions are also very compatible with most existing text editors for Lisp. Editors do not "understand" the code, but many work to match (...), {...}, and [ ... ], and that is enough to be useful. After all, Common Lisp readers are designed to allow { ... } to be overridden, so many text editors are designed to support this.

More information

More rationale information is available in SRFI-105.