Recent changes to Retort

Retort modified by David A. Wheeler

David A. Wheeler — Sat, 14 Jun 2014 13:56:55 -0000

--- v36
+++ v37
@@ -162,9 +162,9 @@

 Note that curly-infix-expressions allow users to write it this way:

-    {{1 * 2} + {3 / 4}}.
-
-This is a big deal, but why it's a big deal isn't immediately obvious.  It's a big deal because this notation does **not** depend on any precedence rules, or registration.  It also doesn't depend on knowing what the final language is (or which operators are infix). Instead, **all** operators will work this same way. So the fact that you don't know exactly what the final language will be like, or which operators are infix operators, or what "+" actually means, doesn't matter... and you can change it without trouble. This notation exposes exactly where every list begins and ends, too. Curly infix adds only one abbreviation: if a list is surrounded by {...}, the operators are presented in infix order instead of in prefix order. This is just like 'x, which is just an abbreviation (quote x).  We're adding a small set of additional abbreviations.
+    {{1 * 2} + {3 / 4}}
+
+This is a big deal, but why it's a big deal isn't immediately obvious.  It's a big deal because this notation does **not** depend on any precedence rules, or registration.  It also doesn't depend on knowing what the final language is (or which operators are infix). Instead, **all** operators will work this same way. So the fact that you don't know exactly what the final language will be like, or which operators are infix operators, or what "+" actually means, doesn't matter... and you can change it without trouble. This notation exposes exactly where every list begins and ends, too. Curly infix adds only one abbreviation: if a list is surrounded by {...}, the operators are presented in infix order instead of in prefix order. This is just like 'x, which is just an abbreviation (quote x).  We're adding a small set of additional abbreviations.  And finally, this extension to s-expressions makes the resulting expressions ''much'' easier for humans to read and write, because it allows humans to the use the notation they've been trained to use for 15+ years and that nearly all books use.

 Here are some interesting relevant quotes:

Retort modified by David A. Wheeler

David A. Wheeler — Sat, 14 Jun 2014 13:50:46 -0000

--- v35
+++ v36
@@ -29,7 +29,7 @@

 Now, just because there's a problem, that does not mean there's a solution. It's actually true that lots of people have tried and failed, but we think there's a reason for it: People didn't really understand the requirements. Creating a new syntax is easy, but what people don't appreciate is that Lisp's syntax has some subtle advantages. For example, its syntax is generic (it does not imply a particular semantic) and homoiconic (the mapping of programming construct to syntactic tree is quite obvious). Previous efforts didn't realize that those were requirements, and so they failed.

-But now that we understand the requirements better, we can fix it.  We do *not* need an "Algol-like" syntax, and we are expressly *not* trying to create an "Algol-like" syntax.  What we are trying to create are clearer notations for s-expressions that are general and homoiconic; that's not the same thing.
+But now that we understand the requirements, we can fix the problem.  We do *not* need an "Algol-like" syntax, and we are expressly *not* trying to create an "Algol-like" syntax.  What we are trying to create are clearer notations for s-expressions that are general and homoiconic; that's not the same thing as copying Algol.

 * * * * *

@@ -43,7 +43,7 @@

 *The idea of introducing Algol-like syntax into Lisp keeps popping up and has seldom failed to create enormous controversy between those who find the universal use of S-expressions a technical advantage (and don't mind the admitted relative clumsiness of S-expressions for numerical expressions) and those who are certain that algebraic syntax is more concise, more convenient, or even more natural (whatever that may mean, considering that all these notations are artificial).*

-So even Steele and Gabriel admit that s-expressions are clumsy.  This is again evidence that many people are dissatisfied with traditional Lisp notation.
+So Steele, Gabriel, and some others who argue for traditional s-expression syntax all admit that s-expressions are clumsy.  This is again evidence for our position, that traditional s-expression syntax is a *problem*.  Many people are dissatisfied with traditional Lisp notation, and even its proponents admit that it is clumsy!

 Note that these authors assume that the only two possibilities are s-expressions and an "Algol-like syntax".  They do not consider a third alternative: An improved s-expression notation that is still general and homoiconic.  We advocate this third alternative.

@@ -65,15 +65,13 @@

 This property is typically called "homoiconicity", and is very rare among programming languages. Lisps are one of the very few language groups that are homoiconic, and it's why Lisps are still used, decades after their development.

-Steele and Gabriel are correct in stating that there have been many efforts to create readable Lisp formats, and they all failed because they did not create formats that accurately represented the programs as data structures.  These other notations typically tried to just directly copy existing syntaxes like Fortran's or Algol's, with the notation that they would be "easier to read". They all failed.  One reason was that when the semantics changed underneath, their syntax could not easily access the new capabilities, and the constant maintenance eventually caused the approach to fail.
-
 The whole point of a Lisp-like language is that you *can* treat code as data, and data as code. Any notation that makes this difficult means that you lose Lisp's advantages.  Homoiconicity is critical if you're going to treat a program as data. To do so, you must be able to easily "see" the program's format. If you can, you can do amazing manipulations.

-But what Gabriel and Steele failed to appreciate in their paper is that ***it's possible to have both homoiconicity and a more readable notation***.
-
-At the time, no one understood why the previous efforts at readable Lisps failed. Now that we have a good diagnosis for why these previous efforts failed, we can avoid their mistakes! We thank Gabriel, Steele, and others, who had this key insight. This insight is absolutely necessary for any future effort to even have a chance to succeed.
-
-Unfortunately, their paper presumed that this readability problem was unsolvable, and that strong claim has meant that people who **could** have solved this problem in the past didn't even try. After all, why work on an impossible problem?
+Steele and Gabriel are correct in stating that there have been many efforts to create readable Lisp formats, and they all failed because they did not create formats that accurately represented the programs as data structures.  These other notations typically tried to just directly copy existing syntaxes like Fortran's or Algol's, with the notation that they would be "easier to read". They all failed.  One reason was that when the semantics changed underneath, their syntax could not easily access the new capabilities, and the constant maintenance eventually caused the approach to fail.  There is also a more general problem: It is difficult to manipulate programs-as-data when you cannot easily perceive the programs as data.
+
+Now that we have a good diagnosis for why these previous efforts failed, we can avoid their mistakes! In short, any Lisp notation *must* be homoiconic.  We thank Gabriel, Steele, and others, who had this key insight. This insight is absolutely necessary for any future effort to even have a chance to succeed.
+
+But what Gabriel and Steele failed to appreciate in their paper is that ***it's possible to have both homoiconicity and a more readable notation***. Unfortunately, their paper appears to quietly assume that a homoiconic notation *has* to have poor readability.  That assumption has meant that people who **could** have solved this problem in the past didn't even try. After all, why work on an impossible problem?

 But now that we know why past efforts failed, we have a better chance at solving the problem. So let's solve the problem.

Retort modified by David A. Wheeler

David A. Wheeler — Sat, 14 Jun 2014 13:35:49 -0000

--- v34
+++ v35
@@ -9,11 +9,9 @@

 Q: What's up with all the parentheses? ... Lisp's most superficially obvious characteristic is its extensive use of parentheses to delimit expressions. Unfortunately many would-be Lispers get stuck on the parentheses and never get far enough into Lisp to see that they are a feature, not a bug.

-We disagree.  Most developers do not use Lisp for their real work, and part of that reason is that  Lisp's overuse of parentheses is a bug that many developers perceive as being more of a problem than Lisp's advantages.  Even some people who've gone far into Lisp see that its syntax is a bug, not a feature.  David A. Wheeler started the readable project, after having used Lisp for over 30 years (starting in circa 1982, and having professionally using $120,000 equipment).
-
-Lisp has advantages, sure.  But there's a reason why Lisp is widely referred to as "Lots of Irritating Silly Parentheses" (and worse).
-
-Lots of vendors have tried to tell us that a bug is a really feature.  They are still bugs.  Let's call a spade, a spade, and call a bug, a bug.
+We disagree.  Most developers do not use Lisp for their real work, and part of the reason is that many developers perceive Lisp's poor readability (such as its overuse of parentheses) is more of a problem than Lisp's advantages.  Even some people who've gone far into Lisp believe that its syntax is a bug, not a feature.  David A. Wheeler started the readable project, after having used Lisp for over 30 years (starting in circa 1982, and having professionally using $120,000 equipment).
+
+Lisp has advantages, sure.  But there's a reason why Lisp is widely referred to as "Lots of Irritating Silly Parentheses" (and worse).  Lots of vendors try to claim that a bug is a really feature.  But if a large number of people perceive something as a bug... it is likely to be a bug.

 * * * * *

Retort modified by David A. Wheeler

David A. Wheeler — Mon, 14 Oct 2013 00:09:36 -0000

--- v33
+++ v34
@@ -7,11 +7,7 @@

 * * * * *

-Q: What's up with all the parentheses? ...
-Lisp's most superficially obvious characteristic is its extensive use
-of parentheses to delimit expressions. Unfortunately many would-be
-Lispers get stuck on the parentheses and never get far enough into Lisp
-to see that they are a feature, not a bug.
+Q: What's up with all the parentheses? ... Lisp's most superficially obvious characteristic is its extensive use of parentheses to delimit expressions. Unfortunately many would-be Lispers get stuck on the parentheses and never get far enough into Lisp to see that they are a feature, not a bug.

 We disagree.  Most developers do not use Lisp for their real work, and part of that reason is that  Lisp's overuse of parentheses is a bug that many developers perceive as being more of a problem than Lisp's advantages.  Even some people who've gone far into Lisp see that its syntax is a bug, not a feature.  David A. Wheeler started the readable project, after having used Lisp for over 30 years (starting in circa 1982, and having professionally using $120,000 equipment).

Retort modified by David A. Wheeler

David A. Wheeler — Mon, 14 Oct 2013 00:08:52 -0000

--- v32
+++ v33
@@ -174,14 +174,10 @@

 This is a big deal, but why it's a big deal isn't immediately obvious.  It's a big deal because this notation does **not** depend on any precedence rules, or registration.  It also doesn't depend on knowing what the final language is (or which operators are infix). Instead, **all** operators will work this same way. So the fact that you don't know exactly what the final language will be like, or which operators are infix operators, or what "+" actually means, doesn't matter... and you can change it without trouble. This notation exposes exactly where every list begins and ends, too. Curly infix adds only one abbreviation: if a list is surrounded by {...}, the operators are presented in infix order instead of in prefix order. This is just like 'x, which is just an abbreviation (quote x).  We're adding a small set of additional abbreviations.

-A few quotes are useful, I think:
-
--   "I've used Lisp my whole programming life and I still don't find
-    prefix math expressions natural." - [Paul
-    Graham](http://paulgraham.com/popular.html)
--   "I have more faith that you could convince the world to use
-    esperanto than [to use] prefix notation." - [Paul
-    Prescod](http://people.csail.mit.edu/gregs/ll1-discuss-archive-html/msg01571.h%0Atml)
+Here are some interesting relevant quotes:
+
+-   "I've used Lisp my whole programming life and I still don't find prefix math expressions natural." - [Paul Graham](http://paulgraham.com/popular.html)
+-   "I have more faith that you could convince the world to use esperanto than [to use] prefix notation." - [Paul Prescod](http://people.csail.mit.edu/gregs/ll1-discuss-archive-html/msg01571.h%0Atml)

 * * * * *

Retort modified by David A. Wheeler

David A. Wheeler — Mon, 14 Oct 2013 00:07:45 -0000

--- v31
+++ v32
@@ -211,7 +211,5 @@

 So instead of pretending there is no problem, we've actually worked to solve it.  We already have implementations for Scheme (guile) and Common Lisp, including general-purpose tools that can help with other programming languages.  Please try out our solutions; we think you'll like them.

-The [Solution] page gives a more detailed description of our solution.  For more information, please see .
-
-This page was originally posted as .
-
+The [Solution] page gives a more detailed description of our solution.  For more information, please see .  This page was originally posted as .
+

Retort modified by David A. Wheeler

David A. Wheeler — Mon, 14 Oct 2013 00:07:14 -0000

--- v30
+++ v31
@@ -91,9 +91,7 @@

 * * * * *

-*When Steele and Gabriel are talking about manipulating representations of programs, they are, of course, talking about macros. This is where the s-expression notation really shines. An Algol-style syntax is all well and good for languages that have a finite number of basic constructs - one can define a grammar that specifies how various syntactic constructs get translated into an abstract syntax tree (AST) that can then be processed by an interpreter or compiler. Open any compiler textbook and you'll learn how to write a parser that can turn the infix syntax: 1 \* 2 + 3 / 4 into the AST.... according to the precedence of the operators \*, +, and /. But in a language that supports macros we can't know, at language design time, all the possible legal language constructs. Thus we can't rely on having a grammar that knows how to translate all possible syntactic constructs into ASTs. Rather, we need a way to represent arbitrary ASTs. Which is what s-expressions are. Even if we know nothing about the precedence of \*, +, and / this s-expression: (+ (\* 1 2) (/ 3 4)) unambiguously represents the same AST shown above.  Likewise, when Lisp sees something like this: \
- (with-whatever (something) (do-one-thing) (do-another-thing)) \
- it can parse it into an AST without knowing anything about the internal syntax of the with-whatever construct. If with-whatever is a macro, it will be passed the AST, represented as an s-expression, and is responsible for producing a new s-expression representing the AST that should be used in place of the original with-whatever form.*
+*When Steele and Gabriel are talking about manipulating representations of programs, they are, of course, talking about macros. This is where the s-expression notation really shines. An Algol-style syntax is all well and good for languages that have a finite number of basic constructs - one can define a grammar that specifies how various syntactic constructs get translated into an abstract syntax tree (AST) that can then be processed by an interpreter or compiler. Open any compiler textbook and you'll learn how to write a parser that can turn the infix syntax: 1 \* 2 + 3 / 4 into the AST.... according to the precedence of the operators \*, +, and /. But in a language that supports macros we can't know, at language design time, all the possible legal language constructs. Thus we can't rely on having a grammar that knows how to translate all possible syntactic constructs into ASTs. Rather, we need a way to represent arbitrary ASTs. Which is what s-expressions are. Even if we know nothing about the precedence of \*, +, and / this s-expression: (+ (\* 1 2) (/ 3 4)) unambiguously represents the same AST shown above.  Likewise, when Lisp sees something like this: (with-whatever (something) (do-one-thing) (do-another-thing)) it can parse it into an AST without knowing anything about the internal syntax of the with-whatever construct. If with-whatever is a macro, it will be passed the AST, represented as an s-expression, and is responsible for producing a new s-expression representing the AST that should be used in place of the original with-whatever form.*

 This is accurate as far as it goes, but it misses the point.

@@ -103,15 +101,19 @@

 The current notation for s-expressions was merely the **first** way that was created, one created in the 1950s. It wasn't even intended for use in programming!  It's just that no one created a better one, and so the Lisp programming community quickly ossified on a notation from the 1950s.  There's no reason we must stay there.

-* * * * *
-
-*Q: "I have this cool idea for a new Lisp dialect / Lisp syntax that
-doesn't involve so many parentheses! It uses indentation to show
-structure!" \
- \
- Since the advent of Python, this comes up on c.l.l. with some regularity. We've had 'sweet-expressions' and 'indented Lisp' and probably others.*
-
-In other words, many people see the obvious: Lisp notation is hideously hard to read, and that simple things like the syntactic use of indentation would be a big improvement.
+In our neoteric-expressions, the same things can be expressed as:
+
+* {{1 * 2} + {3 / 4}}
+* with-whatever(something() do-one-thing() do-another-thing())
+
+Note that you can still "parse it into an AST without knowing anything about the internal syntax of the with-whatever construct"... so clearly traditional Lisp notation is *not* required to have a general and homoiconic notation.
+
+
+* * * * *
+
+*Q: "I have this cool idea for a new Lisp dialect / Lisp syntax that doesn't involve so many parentheses! It uses indentation to show structure!"  Since the advent of Python, this comes up on c.l.l. with some regularity. We've had 'sweet-expressions' and 'indented Lisp' and probably others.*
+
+In other words, many people see the obvious: Lisp notation is hideously hard to read, and that syntactic use of indentation would be a big improvement.

 * * * * *

@@ -122,7 +124,7 @@
 Yes, tools help, but tools don't fix bad notation. This answer misses the point, on two levels:

 1.  Tools can make ugly notations easier to manipulate, but they don't do much where it counts: helping the humans understand the code.  Humans have to *read* code to understand it, and a notation that is difficult to read is a serious barrier. Tools can help readability somewhat (e.g., via character dimming, color-coding, etc.), but a good notation makes the workarounds unnecessary.
-2.  Tools can also help easy-to-read notations, so easy-to-read notations always have the advantage. You're much better off starting with a readable notation, and then using tools to help it even further. The idea that ugly notations are *required* is false, and obviously so. Every other popular programming language has exotic support in tools like emacs, vim, and Eclipse. Why settle for ugly when you can have readable **and** manipulable? To get a notation to be supported by tools, you need to have a common notation used by many programs; it does not have to be an ugly notation.
+2.  Tools can also help easy-to-read notations, so easy-to-read notations always have the advantage over bad ones like traditional s-expressions.  You're much better off starting with a readable notation, and then using tools to help it even further. The idea that ugly notations are *required* is false, and obviously so. Every other popular programming language has exotic support in tools like emacs, vim, and Eclipse. Why settle for ugly when you can have readable **and** manipulable? To get a notation to be supported by tools, you need to have a common notation used by many programs; it does not have to be an ugly notation.

 We advocate using a better notation, and developing the tools to match.

@@ -187,7 +189,7 @@

 That's not true - essentially all Lisp code is **already** indented.  Nobody would accept Lisp code today if it wasn't indented! Common Lisp even builds in a pretty-printer, specifically to indent code, and most major Scheme implementations (e.g., guile) include one too.

-Indentation-based approaches basically say that "every time you indent, it's the equivalent of (, and every time you outdent, it's equal to )".  In other words, since Lisp code **already** uses indentation, combined with ugly excess parens, why not get rid of the unneeded baggage?
+Since Lisp code **already** uses indentation, combined with ugly excess parentheses that experts recommend ignoring, let's get rid of the unneeded parentheses.

 * * * * *

@@ -195,7 +197,7 @@

 Here we see the real problem. Almost all software developers have abandoned Lisp, correctly perceiving that Lisp notation is too ugly to be readable... and few want their resulting programs to be that hard to read. Hard-to-read programs are hard to improve later!

-Most of the people left on newsgroups/mailing lists like c.l.l are die-hards who believe that no improvements or innovations can occur in Lisp notation, and are actively hostile to others who **are** interested in improvements.
+As a result, many of the people left on Common Lisp mailing lists are die-hards who believe that no improvements or innovations can occur in Lisp notation, and are actively hostile to others who **are** interested in improvements.

 This is not univerally true, thankfully. Paul Graham is a widely-respected advocate of Lisp, and yet he's willing to publicly say that "Common Lisp and Scheme only directly support s-expressions; disadvantage: long-winded". He recommends [Syntax as abbreviation](http://www.paulgraham.com/arcll1.html), specifically, that additional notation would be an abbreviation for longer still-valid syntax. This just like 'x is today. In particular, he is an advocate of showing structure by indentation instead of parentheses (which would become optional where no ambiguity results). He also admits that infix notation would be nice.

@@ -207,9 +209,9 @@

 No doubt there will be several, but since documents like this will only report people abandoning indentation - and not those who **switch to** indentation - their reporting will be completely biased. It would be more interesting to see, as the alternatives become mature, what the numbers are in both camps.

-Those of us who can perceive the problem clearly are trying to fix it.  If you use Lisp, and are willing to consider or create new ideas (like I-expressions, sweet-expressions, and standardized infix macros), you're welcome to join us. Help us move beyond the current state - please join the "readable" mailing list!
-
-For more, please see .
-
-This was originally posted as .
-
+So instead of pretending there is no problem, we've actually worked to solve it.  We already have implementations for Scheme (guile) and Common Lisp, including general-purpose tools that can help with other programming languages.  Please try out our solutions; we think you'll like them.
+
+The [Solution] page gives a more detailed description of our solution.  For more information, please see .
+
+This page was originally posted as .
+

Retort modified by David A. Wheeler

David A. Wheeler — Sun, 13 Oct 2013 23:58:30 -0000

--- v29
+++ v30
@@ -91,10 +91,7 @@

 * * * * *

-*When Steele and Gabriel are talking about manipulating representations of programs, they are, of course, talking about macros. This is where the s-expression notation really shines. An Algol-style syntax is all well and good for languages that have a finite number of basic constructs - one can define a grammar that specifies how various syntactic constructs get translated into an abstract syntax tree (AST) that can then be processed by an interpreter or compiler. Open any compiler textbook and you'll learn how to write a parser that can turn the infix syntax: 1 \* 2 + 3 / 4 into the AST.... according to the precedence of the operators \*, +, and /. But in a language that supports macros we can't know, at language design time, all the possible legal language constructs. Thus we can't rely on having a grammar that knows how to translate all possible syntactic constructs into ASTs. Rather, we need a way to represent arbitrary ASTs. Which is what s-expressions are. Even if we know nothing about the precedence of \*, +, and / this s-expression: (+ (\* 1 2) (/ 3 4)) unambiguously represents the same AST
-shown above. \
- \
- Likewise, when Lisp sees something like this: \
+*When Steele and Gabriel are talking about manipulating representations of programs, they are, of course, talking about macros. This is where the s-expression notation really shines. An Algol-style syntax is all well and good for languages that have a finite number of basic constructs - one can define a grammar that specifies how various syntactic constructs get translated into an abstract syntax tree (AST) that can then be processed by an interpreter or compiler. Open any compiler textbook and you'll learn how to write a parser that can turn the infix syntax: 1 \* 2 + 3 / 4 into the AST.... according to the precedence of the operators \*, +, and /. But in a language that supports macros we can't know, at language design time, all the possible legal language constructs. Thus we can't rely on having a grammar that knows how to translate all possible syntactic constructs into ASTs. Rather, we need a way to represent arbitrary ASTs. Which is what s-expressions are. Even if we know nothing about the precedence of \*, +, and / this s-expression: (+ (\* 1 2) (/ 3 4)) unambiguously represents the same AST shown above.  Likewise, when Lisp sees something like this: \
  (with-whatever (something) (do-one-thing) (do-another-thing)) \
  it can parse it into an AST without knowing anything about the internal syntax of the with-whatever construct. If with-whatever is a macro, it will be passed the AST, represented as an s-expression, and is responsible for producing a new s-expression representing the AST that should be used in place of the original with-whatever form.*

Retort modified by David A. Wheeler

David A. Wheeler — Sun, 13 Oct 2013 23:57:51 -0000

--- v28
+++ v29
@@ -1,4 +1,4 @@
-Some believe that Lisp notation descended from the gods, and thus cannot be improved on. We think that's nonsense - we believe Lisp notation can be improved, so that it retains its good properties (like homoiconicity and generality) while being easier to read. Here's a retort, for those Lisp users willing to consider alternatives.
+Some believe that traditional Lisp s-expression notation descended from the gods, and thus cannot be improved on. We think that's nonsense.  We believe that Lisp notation can be improved so that it retains its good properties (like homoiconicity and generality) while being easier to read. Here's a retort for our position, for those willing to consider alternatives to traditional Lisp s-expression notation.

 The [perspective of people who think Lisp notation cannot be improved is well expressed in the Common Lisp FAQ](http://www.lispniks.com/faq/faq.html). So, here's a point-by-point rebuttal to that FAQ, as it existed on January 26, 2008, with the FAQ material in italics. The editor of the FAQ is Peter Seibel, who we respect; it's just that on this particular technical issue, we think he (and others) are wrong.  Let's see why we believe that, with the point-by-point rebuttal below.

Retort modified by David A. Wheeler

David A. Wheeler — Sun, 13 Oct 2013 23:54:14 -0000