On fast ClojureScript React templates

Abstract: There's a tradeoff between performance and expressiveness in ClojureScript React templating: Default runtime interpretation makes it easy to write slow templates, but without interpretation it's easy to write runtime errors. We motivate a template compiler that enables explicit choice, which nudges one towards performance without risking errors. Unnecessary choices are reduced using type inference.

Background

The Sablono ClojureScript library lets folks use Hiccup markup for React.js. So you can write:

(html [:a {:href "#"} "Link"])

and when ClojureScript compiles this code into JavaScript, the html macro will run, expanding to

(js/React.createElement "a" #js {"href" "#"} "Link")

which will then be emitted as JavaScript:

React.createElement("a", {href: "#"}, "Link");

ready to be run in a browser.

However, Sablono’s html macro can’t translate every form completely to direct React calls during macro expansion (i.e., at compile-time). For example, given the Hiccup template:

[:a (foo)]

Sablono can’t tell whether the call to foo will return:

• a JavaScript string or number primitive value (which can be passed directly to React to render as a child of <a>)
• a ClojureScript map of attributes (which will need to be converted to a JavaScript object for React to set the attributes of <a>)
• a ClojureScript vector of more hiccup markup (which will need to be recursed into)

This isn’t a limitation of Sablono — it could be that foo fetches data over the network or otherwise can only be run at, well, runtime.

If the result only occurs at runtime, then Sablono must emit code to handle it then. Put another way: Sablono translates as much as it can in advance (the vector and map literals; the bodies of common macro forms like if, let, etc.¹), and ships a runtime interpreter to handle everything else.

This is great from a usability perspective, since it means we can write whatever Clojure code we need to express our markup, and it all just works.

However, by freeing us from thinking about whether our templates can be compiled ahead of time or not, Sablono makes it easy to write templates that rely extensively on runtime interpretation.

This initial usability win later becomes a loss when we’ve painted ourselves into a performance corner. When JavaScript profiling indicates lots of sablono.interpreter/interpret calls, we have to tediously try to map them back to ClojureScript templates within our application and refactor them away.

Eliminate interpretation?

For the performance sensitive, one option is eliminate runtime interpretation entirely. That’s the approach the Hicada compiler takes: It just has no interpreter.

However, without an interpreter, any value that Hicada can’t translate at compile-time is a lurking runtime error. (React will throw when it encounters a ClojureScript data structure when it was expecting a React element, JavaScript object, string, etc.)

In a sense, Hicada is just taking the opposite default: Hicada defaults to performance (at the risk of errors) whereas Sablono defaults to correctness (at the risk of poor performance).

Explicit choice

What I’d like is to have the choice. Within the context of my templates, let me choose where to trade between performance and flexibility/expressiveness.

On parts of my application that aren’t rendered often, I don’t care about runtime interpretation; but on something like a list item that’s rendered hundreds of times, I want to eliminate all unnecessary overhead.

However, choice has a cost.

One of my favorite aspects of Clojure and ClojureScript are that they don’t make me choose every last detail about memory allocation or data ownership. Clojure lets me sketch and explore the big ideas, which helps me find better solutions.

So while we want to support explicit choices, we don’t want to force them to be made prematurely — it should be up to the developer when they want to make those choices.

An example of this done right is JVM Clojure’s *warn-on-reflection*. It’s off by default (enabling you to explore ideas and sketch in peace), but once you’ve solidified something and want to improve performance, you can turn on warnings to help you track down and fix performance issues.

We can do the same thing: Allow template writers to turn on warnings about whether forms cannot be translated at compile-time. I.e., update Sablono’s default clause to something like:

(defmethod compile-form :default
  [expr]
  (when *warn-on-interpretation*
    (emit-warning! (str "This might be slow! Consider rewriting this expression: " expr)))
  `(sablono.interpreter/interpret ~expr))

Then allow authors to make an explicit choice by annotating each form with metadata tags: ^:inline (meaning “trust me, pass this directly to React”) or ^:interpret (“Yes, I’m okay with the performance penalty here, silence this warning”).

Reducing interpretation with type inference

When I first enabled warnings, I got annoyingly many of them.

Some warnings occurred where I knew I was using runtime interpretation, and I silenced them by adding ^:inline tags.

But the majority of warnings were about forms where the template compiler just didn’t have enough information — even though the information was theoretically knowable.

For example:

[:p (pr-str :this-will-be-interpreted)]

will be interpreted because the compiler doesn’t know what pr-str returns.

But we know: pr-str always returns a string, so we can annotate it to tell the compiler that we don’t need to pay the cost of interpretation:

[:p ^:inline (pr-str :now-this-will-be-passed-directly-to-react)]

A similar issue occurs with local variables:

(let [text "foo"]
  [:p text])

We can see that the local variable text is a string that can be passed directly to React, but at macro-expansion time the template compiler just sees the symbol text; it doesn’t know anything about what it might be bound to at runtime, and so must emit an interpreter call just to be safe.

Having to annotate all such instances is tedious and demoralizing. In fact, it’s demoralizing because it’s tedious — it’s the sort of work that a computer should be able to do!

Thanks to Mike Fike’s work, the ClojureScript compiler can infer types, which the template compiler can leverage to do this tedious work.

Implementation

To implement this template compiler, I forked Hicada², added back Sablono’s interpreter (see the commit log for details), and wrote a macro to leverage inferred types.

This last part is the interesting bit.

Inferred types are stored on abstract syntax tree nodes, and can be retrieved by calling cljs.analyzer/infer-tag with the node’s analysis environment (a map of symbols to variable bindings).

The analysis environment is accessible within macros as &env, so initially I thought I’d need to write my own top-level compilation macro. E.g.,

(defmacro compile-template
  [template]
  (compile-fn template &env))

where compile-fn would be a regular function that recursively transforms the template.

But there are a few downsides with this approach:

It’s not a drop-in replacement: I’d need to go rewrite all of the Rum defc templates in my projects in terms of this compile-template macro. This would be a lot of tedious work at best, and at worst it risks breaking existing code if I don’t carefully reimplement Rum’s macro functionality around, e.g., mixins.

Incomplete environment: The &env passed to the macro is the environment where the macro is called, but this environment may not have all of the type information defined. Consider this invocation:

(def foo "foo")

(compile-template
  [:p foo (let [text "text"]
           [:span text])])

At the macro invocation site, the var foo is in scope and the analyzer knows that it’s a string. But the local var text is not in scope, so if we just pass &env around through the recursive calls to compile-fn, when it gets to text it won’t know anything about the type and will be forced to interpret. (Exactly the kind of trivial interpretation / warning that what we’d like to avoid.)

Instead, we can go “inside out”: Have compile-fn wrap every form it cannot translate with a macro:

(defmacro interpret-when-necessary
  [expr]
  (if (safe-to-inline? (infer-type expr &env))
    expr
    `(runtime-interpret ~expr)))

Then, when this macro is expanded by the analyzer, its &env will contain exactly the type information we need to determine whether the expression it wraps can be inlined or must be interpreted at runtime.³

What’s better, this macro does not have to be exposed to the template writers at all, which means that the entire template compiler can be “patched in” without any changes to already written templates.

Specifically, since Rum calls sablono.compiler/compile-html, I can exclude Sablono from deps.edn:

{:deps {rum {:mvn/version "0.11.2" :exclusions [sablono/sablono]}}}

and define those namespaces myself:

(ns sablono.compiler
  "Using this namespace so we can overwrite Rum's default usage of the sablono compiler and use our own."
  (:require hicada.compiler))

(defn compile-html
  [body]
  (hicada.compiler/compile body {:create-element 'js/React.createElement}))

and

(ns sablono.core
  "Need this because Rum's core.cljs requires sablono.core"
  (:require hicada.interpreter))

then neither Rum nor my existing application code need know anything about the new template compiler.

Future work

I’m happy using my Hicada fork to see how the design plays out on my own projects.

You’re welcome try it out: See this minimal example repo.

However, there will be no versioned releases as I may wish to adjust the API to suit my needs. I suggest depending on the code via git submodule or pinned deps.edn and reading all commits.

If you enjoy doing open sourcey things, consider these questions and next steps:

• Should this be merged into upstream Hicada / Sablono?
• Hicada itself needs a test suite.
• Can interpolation warnings be displayed in the figwheel heads up display?
• Can this be used “drop-in” by other popular ClojureScript React wrappers (Reagent, others)?
• Incorporate as part of ClojureScript canary tests to detect inference regressions.

See also:

• Nikita Prokopov’s thoughts on hiccup compilation.

• If you like fast things and want to support the author, check out Finda.

Thanks

Thanks to Mike Fikes for discussions around type inference in ClojureScript; Nikita Prokopov, Jamie Brandon, Shaun LeBron, Yuri Vishnevsky, and Andre Rauh for reviewing drafts of this article.