Introducing clojure.spec

A presentation by Arne Brasseur

for PolyConf 2016.

30 June 2016

clojure.spec

New library in Clojure 1.9 (now in alpha)

Somebody please think of the types!

I know your type system can do this

Clojure is a dynamic language, for better or worse

Yes, specs are (mostly) checked at runtime, still immensely useful

Interesting to see how a dynlang handles these concerns

Agenda

10 What does Clojure data look like?
20 How do I create and register a spec?
30 What can I do with specs?
40 GOTO 20

Agenda

10 What does Clojure data look like?
20 How do I create and register a spec?
21    Maps
22    Sequences
30 What can I do with specs?
40 GOTO 20

Agenda

10 What does Clojure data look like?
20 How do I create and register a spec?
30 What can I do with specs?
30    Instrumenting Functions & Macros
31    Generative Testing
40 GOTO 20

Data in Clojure

Vectors [] and Maps {} are main data structures

Clojurists like “unadorned” data: structs/records mainly used for interop

Result: maps and keywords everywhere

{:uri "/"
 :method :post
 :headers {"Accept" "*/*"}}

Keywords

Like Ruby’s Symbol, lightweight immutable string

Used for:

:created-on
:put
:cemerick.friend/identity

Keywords everywhere

Chance of collission

Answer: namespaced keywords

{:my.audio.lib/encoding :ogg-vorbis
 :transfer/encoding     :base64}

Namespaced keywords

(ns my.audio.lib
  (:require [transfer :as t]))

Shorthand for a keyword in the current namespace

::encoding        ;;     :my.audio.lib/encoding

Shorthand for a keyword in an aliased namespace

::t/encoding      ;;     :transfer/encoding

More shorthands coming!

clojure.spec

An example (bear with me)

We have a Robot Chef which works with recipes

(def tomato-sauce-recipe
  {:robochef/ingredients [250 :g "peeled tomatoes"
                          3 :clove "garlic"
                          5 :g "pepper"]
   :robochef/steps ["heat a pan"
                    "throw everything in"
                    "stir"]})

Let’s get started!

Load spec in your namespace, aliased to s

(ns robochef
  (:require [clojure.spec :as s]))

An example Spec

;; keep in mind ::recipe == :robochef/recipe

(s/def ::recipe (s/keys :req [::ingredients]
                        :opt [::steps]))

(s/def ::ingredients (s/* (s/cat :amount number?
                                 :unit   keyword?
                                 :name   string?)))

(s/def ::steps ,,,)

This registers specs in a global registry, :robochef/recipe, :robochef/ingredients, and :robochef/steps.

Basic usage

(s/valid? :robochef/ingredients [5 :g "tea"])
;;=> true

(s/conform ::ingredients [5 :g "tea"])
;; [{:amount 5, :unit :g, :name "tea"}]

More interesting features

(s/valid? :robochef/ingredients ["10" :g "tea"])
;;=> false

(s/explain-str ::ingredients ["10" :g "tea"])
;; In: [0] val: "10" fails spec:
;;   :robochef/ingredients at: [:amount] predicate: number?

More interesting features

(s/exercise ::ingredients 2)
;; ([() []]
;;  [(0 :Hi "0") [{:amount 0, :unit :Hi, :name "0"}]])

Spec types

Predicate

map?

Spec object

(s/or :s string?, :n number?)
(s/coll-of number? [])

Name of a registered spec

::ingredients

Creating specs

Two “advanced” types of specs:

Spec’ing maps

Done with s/keys

“same key in different context should have same semantics”

(s/def ::recipe (s/keys :req [::ingredients]))
(s/def ::ingredients ,,,)

(s/valid? ::recipe
          {::ingredients [250 :g "peeled tomatoes"
                          3 :clove "garlic"
                          5 :g "pepper"]})

Spec’ing maps

Most systems for specifying structures conflate the specification of the key set with the specification of the values designated by those keys. This is a major source of rigidity and redundancy.
— Rich Hickey

 

s/keys will look at every key in a map, try to find a spec with that name, use it to validate the corresponding value

s/keys “naturally extensible”

(s/def ::recipe (s/keys))

(def recipe {::ingredients [,,,]
             ::steps [,,,]
             ::cooking-time "30 minutes"})

(s/valid? ::recipe recipe) ;;=> true

(s/def ::cooking-time number?)
(s/valid? ::recipe recipe) ;;=> false

Spec’ing sequences

Clojure data structures all share an underlying “sequence” abstraction.

clojure.spec contains full regular expression engine for dealing with these.

Based on a paper “Parsing with Derivatives”. Powerful enough to parse context-free grammars!

Regexp specs

Five “Regex” operators: *, +, ?, cat, alt

(s/conform (s/* keyword?) [])      ;;=> []
(s/conform (s/* keyword?) [:a])    ;;=> [:a]
(s/conform (s/* keyword?) [:a :b]) ;;=> [:a :b]

(s/conform (s/+ keyword?) [])      ;;=> :clojure.spec/invalid
(s/conform (s/+ keyword?) [:a])    ;;=> [:a]
(s/conform (s/+ keyword?) [:a :b]) ;;=> [:a :b]

(s/conform (s/? keyword?) [])      ;;=> nil
(s/conform (s/? keyword?) [:a])    ;;=> :a
(s/conform (s/? keyword?) [:a :b]) ;;=> :clojure.spec/invalid

Regexp operators: cat

cat “concatentate” sequence items (first this, then that)

(s/conform (s/cat :num number?, :key keyword?) [5 :b])
;;=> {:num 5, :key :b}

Each item gets a name

The conformed result is a map that can easily be consumed with Clojure’s destructuring

Regexp operators: alt

alt distinguishes “alternatives” (either this, or that, like |)

(s/conform (s/alt :num number?,:key keyword?) [5])
;;=> [:num 5]
(s/conform (s/alt :num number?,:key keyword?) [:b])
;;=> [:key :b]

Each alternative gets a name

The conformed result is a two-element vector which can be used with core.match pattern matching

Instrumenting functions

fdef lets you set specs on the arguments, return value, and the relationship between them.

(defn cook! [recipe]
  ,,,)

(s/fdef cook! :args (s/cat :recipe ::recipe)
              :ret  number?)

(s/instrument-all)
;; (s/unstrument-all)

Instrumenting functions

A macro is just a function that takes “code as data” and returns “code as data”

We can instrument it just like function

Macro-expension happens at compile time, so we get compile-time checks!

Test.check

(require '[clojure.test.check :as tc])
(require '[clojure.test.check.generators :as gen])
(require '[clojure.test.check.properties :as prop])

(def positive-recipe-prop
  (prop/for-all [r (s/gen ::recipe)]
                (>= (cook! r) 0)))

(tc/quick-check 100 positive-recipe-prop)
;; {:fail [{:robochef/ingredients (-2.0 :+.j/l*4 "")}],
;;  :smallest [{:robochef/ingredients (-1.0 :A "")}}}

Recap

FIN

@plexus