Clojurescript & Node on AWS Lambda

Made easy with Leiningen.

Background

Amazon’s Lambda service executes functions in response to events. Which isn’t all that interesting: an abacus executes functions in response to events. What makes Lambda appealing is the attitude it takes toward disobedience.

I don’t like functions. They’re needy. I experience their suffering as something like music. Through precise constraining of resources, Lambda1 promises an environment in which the flourishing of any number of functions can be inhibited - inexpensively, and repeatably.

And, such is my hope, collaboratively. As a means of attracting potential inquisitors, I’ve written some software which makes the whole process embarrassingly easy. It’s covered step-by-step below.

1 I don’t care where you went to school - you’ll type lambada eventually.

But, The Guacamole Costs Extra

Lambda supports two runtimes - Java 8, and Node. Which is to say, you could run pretty much any language you want. I’m assuming you’d want to run Clojure. So, why not Clojure?

Lambda is metered by a combination of memory allocation and time. Instances (and their JVMs) will sometimes be reused for proximate requests, with no guarantees.

On warmed up instances, Clojure/JVM is clearly going to beat Clojurescript/Node on execution times. That said, Clojure’s warm-up cost on fresh instances is likely to be high1,2. Our uncertainty around the frequency of the warm-ups will trammel our ability to reason about overall performance and cost.

Comparatively, Clojurescript is attractive on Lambda as it allows us to trade a worse best-case runtime3 (warmed-up instance) for greatly improved worst-case runtime (cold instance)4.

1 With ahead-of-time compilation. Without, it’s not worth considering.
2 This was borne out by imprecise and, frankly, distracted experimentation.
3 Like, say, ~20ms for Clojurescript vs. ~1ms for Clojure with the return-immediately example I was benchmarking. This isn’t dramatic, given that Lambda time is billed in 100ms increments.
4 e.g. half a second, rather than multiple seconds, again for the null function.

Leiningen 0day

Enough adjectives, let’s beat up some verbs.

In addition to Leiningen, you’ll need a recent Node runtime, and a local installation of the AWS CLI interface.

We’ll be using the cljs-lambda Lein plugin, which delegates to the AWS CLI. The credentials, target region, etc. will be determined by the CLI’s configuration. So, run aws configure if you haven’t.

Create a Leiningen Project

Let’s pick any old name:

$ lein new cljs-lambda thelema

The cljs-lambda template generates a Leiningen project file, and a Clojurescript module with a single function in it. Let’s look a little closer at the sections in the generated project file (thelema/project.clj):

:plugins

[lein-cljsbuild ...]
[lein-npm ..]
[io.nervous/lein-cljs-lambda ...]

The npm and cljsbuild plugins are used by cljs-lambda to resolve Node dependencies and output Javascript source, respectively.

:cljs-lambda

{:defaults {:role "FIXME"}
 :functions
  [{:name   "work-magic"
    :invoke thelema.core/work-magic}]}

We’re informing the plugin of the functions we’re exposing via Lambda: each function requires the :name, :invoke and :role parameters, with the latter being the ARN of the IAM role we’re running the function under.

Create an IAM Role

If you’ve played around with Lambda before, you may have something to hand - or maybe you fancy your chances with “FIXME”. Otherwise, it’s likely going to be simpler to delegate to the plugin:

~/thelema$ lein cljs-lambda default-iam-role

This creates a role sufficient for a minimal Lambda function (log writing, etc.), using your local AWS CLI install.1

Afterward, the resulting ARN is written into the project file, due to accumulating doubts around your ability to perform remedial tasks.

1 If you’re doing anything other than printing to the console (e.g. trying to connect to another AWS service), the role will require additional privileges.

Inspect the Function

There’s enough to cover here without assuming the burden of writing interesting, or functional code:

(ns thelema.core
  (:require [cljs-lambda.util :refer [async-lambda-fn]])
  (:require-macros [cljs.core.async.macros :refer [go]]))

(def ^:expose work-magic
  (async-lambda-fn
   (fn [{:keys [variety]} context]
     (go
       (js/Error. (str "I don't yet know how to work '"
                       variety
                       "' magic"))))))

Javascript

For those unfamiliar with Lambda’s Node execution model: a Lambda function expects to be pointed at a Javascript function accepting two arguments (which, I guess, is all Javascript functions): event and context.

event is whatever data was passed into the function - it’s either null, or an object constructed from the input JSON. context provides request-specific utility behaviour, such as completion signalling.

Clojurescript

async-lambda-fn, above (take a look, it’s tiny) is allowing an asynchronous channel to take the place of explicit callbacks. If you don’t like it, you can pass context around, and call the explicit cljs-lambda.util/succeed! and fail! functions.

If the value read from the go channel is a Javascript Error (as in the example) the function will fail, and the client’ll get a stacktrace. Otherwise, the value will be serialized to JSON and passed on.

Upload The Function

~/thelema$ lein cljs-lambda deploy

This’ll build your project and smash it into a zip file. A Lambda function named work-magic will be created remotely (if it doesn’t exist), and associated with the source of thelema.core/work-magic from within the uploaded zip.

Subsequent deploy invocations will re-compile, and update the Lambda side.

Run It

For convenience, you can execute any of your Lambda functions remotely using the plugin’s CLI:

~/thelema$ lein cljs-lambda invoke work-magic \
>            '{"variety": "the most black"}'

We’ll see something like this:

REPORT  Duration 18.22 ms ... Max Memory Used 42 MB

{:errorMessage
 "I don't yet know how to work 'the most black' magic",
 :errorType "Error",
 :stackTrace
 [...
  "/var/task/out/thelema/core.cljs:8:6"
  ...
  "processImmediate [as _immediateCallback] (timers.js:354:15)"]}

Which is what we intended. The stacktrace extends from our Clojurescript function down into the Lambda Node dispatcher.

Note the plausible-looking line numbers.

Done! What Else

cljs-lambda supports a bunch more features:

  • Specify execution time & memory limits in your project file
  • Sync function configuration without re-deploying code
  • Define many Lambda functions in a single Clojurescript project

The cljs-lambda README has more details.