Introduction

Much of Lisps longevity is due to its metaprogramming, that is code that treats code as data. Lisp uses the same syntax for both code and data, which makes manipulating code as easy as manipulating data.

Macro functions (or just macros) are just like regular functions, except that the arguments aren’t evaluated before calling, and they should return code (which means data as s-expressions which will be interpreted as code) rather than a value. The macro decides if, when, how and how often an argument is evaluated.

This allows macros to remove repetition, extend the semantics of the programming language, or even create new programming languages within the programming language. Most other languages simply do not have the expressive power to do anything like this, or make it different from built-in constructs or difficult to write.

I’ll demo a couple of simple macros to extend control flow and looping constructs, and I’ll use C# as a contrast to the lisp code.

The intention of this post is meant only to pique your interest in lisp. Production quality implementations of concepts presented here is probably available in libraries for pretty much every lisp out there.

Basic building blocks

Before showing the macros, I’ll explain a few of the building blocks we’ll use.

In lisp, as in many other languages, everything is an expression. If you want to execute some effects, you have to wrap them in an expression which somehow returns a value. The most common construct is progn which accepts many expressions as arguments and returns the result of the last expression. Several other languages do this by default, and lisp also has many forms which does this implicitly.

{
    Effect1();
    Effect2();
}

(progn
  (effect1)
  (effect2))

In C#, you can create a new lexical scope using braces. Variables defined within the scope don’t escape the scope. Many programmers don’t use new scopes outside some statements like functions, if, while etc, but they are still available if you want to control visibility of variables, disposing, finalizing etc.

// stuff before
{
    // stuff before available
    // new local scope
}
// stuff before available
// things from local scope is unavailable

A new scope can be created using let in lisp. While progn allows several expressions and returns the last, let also allows creating new variables in this scope. let has an implicit progn so you can pass several expressions instead of explicitly stating a progn.

{
    int x = 1;
    Effect1(x);
    Effect2(x);
}
// x not visible here

(let ((x 1))
  (effect1 x)
  (effect2 x))
;; x not visible here

The fact that everything is an expression means we don’t have to declare variables ahead of initialization with dummy values.

var result = null;
{
    // result visible but not initialized while executing statements here
    Effect1(); // <- we could use result here by mistake
    // then initialize and result
    result = SomeResult();
}
// We can use result here

In lisp, as in other languages with expressions rather than statements, we can write the following.

(let ((result (progn
                ;; Cannot accidentally use result in these expressions
                (effect1) ; <- result undefined
                ;; Then initialize and return
                (some-result))))
  ;; We can use result here
  )

Code as data and code that writes code

That’s about the basics we need before looking into how macros work. Syntax in programming language is parsed into an abstract syntax tree (AST) in order to be further analyzed and manipulated. In lisp we write our programs directly in an AST.

So how can we represent the following code as data?

(let ((x 1))
  (+ x 1))

list creates a list, quote returns the exact symbol/value without evaluating it.

(list (quote let) (list (list (quote x) 1))
      (list (quote +) (quote x) 1))

Fortunately, we don’t have to write code as above. By quoting the first list, everything within is quoted. We can thus rewrite the above with just a single quote at the start, getting the exact same code, but this time as data!

'(let ((x 1))
    (+ x 1))

When creating code that manipulates code, it’s often useful to evaluate certain expressions. When we use the verbose syntax, this is easy. (list (quote +) x 1) will evaluate x as it’s not quoted, but not +.

Let’s say we have the following code, and we want to return the code (+ 1 2). Our first attempt returns (+ x 2), which isn’t what we want.

(let ((x 1))
  '(+ x 2))

We have a form similar to quote called quasiquote or `. When using this form, we can unquote the expression using ,, which will evaluate the expression.

(let ((x 1))
  `(+ ,x 2)) ;; same as (list (quote +) x 2)

There is also unquote-splicing, ,@, which will unquote a list and flatten it (remove the “(list” and “)” part), e.g. `(1 ,@(list 2 3) 4) equals '(1 2 3 4).

With the basics out of the way, let’s start extending the language with some new control flows.

New control flow abstractions

A regular C# if statement looks pretty much the same as a lisp if expression.

if (someTest) {
    Then1();
    Then2();
} else {
    Else1();
    Else2();
}

(if some-test
    (progn
      (then1)
      (then2))
    (progn
      (else1)
      (else2)))

But often, we only care about the “then” or “else” clause.

if (someTest) {
    Then1();
    Then2();
}

The “else” clause is often written by negating test rather than using an empty “then” block

if (!someTest) {
    Else1();
    Else2();
}

Let’s create two new control flow constructs, when for the “then” clause and unless for the “else” clause. This removes some repetition as well as making the intent clearer.

Rather than

(if some-test
    (progn
      (true1)
      (true2)))

We’ll write

(when some-test
  (true1)
  (true2))

It makes it very clear that there is no “else” clause, and we can use an implicit block. Notice that the definition looks pretty much exactly as we would have written the if expression simply because code and data has the same syntax.

(defmacro when (test &body body)
  `(if ,test
       (progn ,@body)
       nil))

We’ll do the same for the “else” case.

if (!someTest) {
    False1();
    False2();
}

(unless some-test
  (false1)
  (false2))

(defmacro unless (test &body body)
  `(if ,test
       nil
       (progn ,@body)))

We could have defined unless in terms of when and negating test, but let’s use if directly to avoid the extra function call.

As a side-note, truthy and falsy is quite different in some Lisps like Common Lisp (CL) which this article is using. In CL, only nil is falsy and every other value is truthy. This means functions can return actual data instead of true / false, and we don’t have to inspect the value or revert to using only booleans.

The pattern below is quite common, but in Lisp we don’t need to store the temporary or checking against a particular value.

var something = GetSomething();
if (something != null) {
    True1();
    True2();
}

(when (get-something)
    (true1)
    (true2))

But what if the block needs the value too? In that case we need to store the value in a variable in order to use it later.

var something = GetSomething();
if (something != null) {
    True1(something);
    True2(something);
}

We can write this using a let

(let ((something (get-something)))
  (when something
    (true1 something)
    (true2 something)))

or is using this fact to return a value rather than evaluating to a boolean. The following is quite common code, and is easily expressed just using or.

var something = GetSomething();
if (something == null) {
    something = CreateSomething();
}
Effect(something);

(let ((something (or (get-something) (create-something))))
  (effect something))

Binding the test only for the success case

The pattern of binding something and then using it within an expression is a common pattern. In other languages we annoyingly often create temporary values which outlive the expression they are used in. But using macros, we can do this The Right Way; we’ll create if-let, when-let and unless-let. Using these, our code just becomes

(when-let (something (get-something))
    (true1 something)
    (true2 something))
;; something unavailable here

The macros are no more difficult than what we’ve already seen.

(defmacro if-let ((id value) then &optional else)
  `(let ((,id ,value))
     (if ,id
         ,then
         ,else)))

(defmacro when-let (expr &body body)
  `(if-let ,expr
     (progn ,@body)
     nil))

(defmacro unless-let (expr &body body)
  `(if-let ,expr
     nil
     (progn ,@body)))

Anaphoric macros

But we’re not finished yet. We don’t really care about the name something in the expression. We could call it it and make it available in the scope. This is called “anaphoric” macros. Our previous example thus becomes

(awhen (get-something)
  (true1 it)
  (true2 it))

(awhen (or (get-something) (create-something))
  (effect it))

(defmacro aif (test then &optional else)
  `(if-let (it ,test)
     ,then
     ,else))

(defmacro awhen (test &body body)
  `(when-let (it ,test)
     ,@body))

(defmacro aunless (test &body body)
  `(unless-let (it ,test)
     ,@body))

Let’s see how a lisp expression using awhen and or would look like in C#.

(awhen (or (get-something) (try-other-thing) (make-thing))
  (do-thing it))

var it = GetSomething();
if (it == null) it = TryOtherThing();
if (it == null) it = MakeThing();
if (it != null) DoThing(it);

Creating new loops

Let’s turn our head over to loops which also has some common patterns; looping until some condition is met. The loop in C# will look different based on what kind of thing we expect. Here is a simple boolean to become true, and other types of checks would look different.

var x = false;
while(!x) {
    // Until x is true
}

(until x
  ;; Until x is non-nil
  )

But any complex expression is possible

(until (> x 10)
  ;; Until x is greater than 10
  )

Looping forever is normal in background processes. It’s easy to express in most languages, but being more explicit could improve readability.

while (true) {
    // Forever do this
}

(forever
  ;; Forever do this
  )

(defmacro until (test &body body)
  `(do ()
       (,test t)
     ,@body))

(defmacro while (test &body body)
  `(until (not ,test)
     ,@body))

(defmacro forever (&body body)
  `(while t
     ,@body))

Conclusion

The examples are quite contrived and don’t show the real power of macros. But hopefully they show that macros are within reach and allow you to extend your language to fit your domain. Building a program bottom-up by creating a language that allows you to express a solution to your problem clearly, is one of the great features of lisp.