Why Macros are cool

A recent discussion on the Clojure Google Group about possible AOT compilation abuse made me think again about VimClojure. There AOT is necessary because we have to generate classes of a known name. That is only possible with AOT compilation and gen-class. The resulting bytecode however is (possibly) tied to a certain Clojure version if something under the hood changes. This is of course annoying if you work with different Clojure versions in different projects.

So I finally decided to make VimClojure independent of the Clojure version used in the project. And while doing so the miracles of macros came upon me...

Anatomy of VimClojure

First we have to understand how the anatomy of VimClojure looks like.

  1. The Clojure part, running under the project's Clojure version providing docstring lookup, omni completion and the like.
  2. The Vim part, „running“ in Vim providing syntax highlighting and indenting.
  3. Nailgun, stitching both parts together.

Now the problem is the last part: nailgun. To provide custom "nails" you have to define a class which has a static nailMain method.

Up to now there was a „nail“ for each command, which could be invoked from Vim. Each "nail" used gen-class to generate the required class. And it exactly this gen-class is the source of the difficulties. For it to work we need AOT compilation and all its problems.

The Redesign

In order to fix this situation I decided to change the interface between the Vim and the Clojure part of VimClojure. To get rid of the gen-class I devised the following layout:

  1. A central nail written in Java which calls into Clojure dispatching on the first argument. This argument now denotes which "real" nail to invoke.
  2. The "real" nails are now simply functions in a Clojure namespace.
  3. A modified defnail macro which instead of calling gen-class and defining a suitable nailMain method, now simply defines a regular Clojure function. It still takes care of boilerplate like argument parsing and striping the -- now -- uninteresting dispatch argument from the arguments.

This approach has several advantages.

  • Since the central nail is written in Java it is largely independent of the Clojure version used. Since only the necessary parts are used (eg. no namespaces), it is much more robust against changes in the Clojure machinery concerning the language itself.
  • AOT compilation for Clojure code is not required anymore making it also largely independent of the Clojure version used in the project.
  • Users may now provide there own nails in their own namespaces without having a clash of names. Previously the package de.kotka.vimclojure.nails was hardwired in the Vim glue. But now arbitrary qualified functions may be provided. (Unqualified symbols still refer to VimClojure's internal namespace for my convenience. :P)

All in all these changes make VimClojure much more modular.

Didn't you say something about miraculous macros?

Yes, I did. Let's see how they fit into the picture.

During working on these changes I had to touch several parts of the system. On the one hand the Vim function which calls into the nailgun system. Here I had to change the invocation to call the central nail and pass the actual nail as argument instead of calling it directly.

Then I had to write the central nail obviously more or less from scratch.

And last but not least I had to change the definition of the already defined nails from generating a class to just defining a function. And here the miraculous macro shows up: defining the classes was done in the defnail macro. So basically I just had to change it to define a function instead of generating a class.

**I didn't have to touch any of the calls to a nail on the Vim side neither the definition of a single nail on Clojure side.**

And this was only due to the defnail macro. Without this macro I would have had to touch every single nail definition. So instead of hours of tedious work, it took only 20 minutes (some quick functional tests inclusive) to improve the design of VimClojure considerably.

*In fact I changed this twice – once with the approach above and once with a central multimethod dispatching on the nail name – before settling on one version.*

Upshot

Macros are a powerful way of abstracting away low-level stuff which cannot be handled in a function. Their ability to take away common boilerplate code makes for very flexible code as long as the macro interface stays the same. I could quickly prototype two designs meaning I could test both before going for the better. I don't really want to work with a language without macros anymore...

*Disclaimer: Unthoughtful use of macros can make your code convoluted and hard to maintain. Let alone the subtle bugs easily introduced by broken macros.*

Published by Meikel Brandmeyer on .