Chatty checkers

Chatty checkers provide extra information about a failure. There are three ways to create them. Two depend on a checker function having the right "shape". The other is less convenient, but more general.

Chatty results from boolean combinations

Suppose you want to check integer results to see if they're not only primes, but specifically Mersenne primes. Mersenne primes are both Marsenne numbers and primes. You could write such a checker this way:

(defn mersenne-prime [actual]
  (and (= prime? actual)
       (mersenne-number? actual)))

Here's a false fact and its failure report:

user=> (fact 131071 => mersenne-prime)

FAIL at (t_bug.clj:27)
Actual result did not agree with the checking function.
        Actual result: 131071
    Checking function: mersenne-prime

That's not hugely helpful. Did it fail because it's not prime or because it's not Mersenne? To write a more expressive checker, use combining checkers:

user=> (def mersenne-prime (every-checker prime? mersenne-number?))
user=> (fact 131071 => mersenne-prime)

FAIL at (NO_SOURCE_FILE:2)
Actual result did not agree with the checking function.
        Actual result: 131071
    Checking function: mersenne-prime?
    During checking, these intermediate values were seen:
       mersenne-number? => false
false

Chatty results from subcalculations

Generous angel investors are funding our Big Data startup. Key to its success is the ability to crunch data so that the output, when "lateralized" produces a descending sequence of values. Such output is referred to as "nonelian". Put in concrete terms, success depends on implementing code such that facts like the following check out:

(fact (crunch :by-county :gauss 5.0) => nonelian)

Here's the definition of the nonelian checker:

(defn nonelian [actual] (apply > (lateralize actual)))

And here's the result of a check:

user=> (fact (crunch :by-county :gauss 5.0) => nonelian)

FAIL at (NO_SOURCE_FILE:1)
Actual result did not agree with the checking function.
        Actual result: [1.7 8.32 2.0 8.3 0.0 0.0 12.01]
    Checking function: nonelian
false

This isn't horrible information, but it would also be nice to see the intermediate results of lateralize. Fortunately, that's easy to do by using chatty-checker in place of fn:

user=> (def nonelian (chatty-checker [actual] (apply > (lateralize actual))))
user=> (fact (crunch :by-county :gauss 5.0) => nonelian)

FAIL at (NO_SOURCE_FILE:1)
Actual result did not agree with the checking function.
        Actual result: [1.7 8.32 2.0 8.3 0.0 0.0 12.01]
    Checking function: nonelian
    During checking, these intermediate values were seen:
       (lateralize actual) => [5.0 4.98 4.63 4.62 5.0 4.3]
false

chatty-checker converts each top-level function application in its body into a form that reports its results.

It's equally easy to use chatty-checker to make checkers that take arguments. Let's change "nonelian" so that there are variants. The one you've already seen is "Strictly down nonelian" (because it uses >). "Strictly up nonelian" uses <, etc. So here's a checker that's parameterized by direction:

user=> (defn nonelian [comparison]
         (chatty-checker [actual] (apply comparison (lateralize actual))))
user=> (fact (crunch :by-county :gauss 5.0) => (nonelian >))

FAIL at (NO_SOURCE_FILE:1)
Actual result did not agree with the checking function.
        Actual result: [1.7 8.32 2.0 8.3 0.0 0.0 12.01]
    Checking function: (nonelian >)
    During checking, these intermediate values were seen:
       (lateralize actual) => [5.0 4.98 4.63 4.62 5.0 4.3]
false

Adding notes to checker failures

Note: this implementation is nicely functional and all that, but it's actually awkward for the Midje implementation and prone to error. It may be replaced in later versions of Midje.

Midje checkables work with an extended notion of falsehood, called "data-laden falsehood". A data-laden falsehood is a map that can contain a :notes key whose value is a sequence of strings. Those strings are printed alongside the failure.

If, for some strange reason, you wanted a checker that recorded the date of the failure, you could write it like this:

user=> (require '[midje.checking.core :as checking])
user=> (defn chatty-neg? [actual]
         (or (neg? actual)
             (checking/as-data-laden-falsehood {:notes [(str (new java.util.Date))]})))
user=> (fact 1 => chatty-neg?)

FAIL at (NO_SOURCE_FILE:2)
Actual result did not agree with the checking function.
        Actual result: 1
    Checking function: chatty-neg?
    The checker said this about the reason:
        Wed Feb 27 15:33:42 CST 2013
false