Chained assertions

TL;DR: Chain assertions for scannability. Here's some code to help.

I suspect tests are more often scanned than read. When a test unexpectedly fails, I think people often jump to its source. But they don't read that source sequentially from the top; instead, they start at the failing assertion, with the specific goal of understanding what happened. Their eye darts from the assertion to other parts of the test in order to achieve their goal, quite likely skipping over bits of code (like other assertions) that won't contribute.

If that's true, it's useful to arrange tests to make scanning easier. With that in mind, consider an end-to-end-style Phoenix controller test for successfully creating a password by "redeeming" an email token. The test checks two things:

  1. What the externally-visible result is (what page is shown, what's put in the session, what important messages are shown to the user).
  2. Whether the user can now log in. (Having two kinds of check in the same test is arguably wrong. Let's leave that aside.)

Here's the test:

test "the password is acceptable", %{conn: conn, user: user} do
  conn = action__set_fresh_password(conn, @valid_password, @valid_password)
  assert_logged_in(conn, user, @institution)

  assert_info_flash_has(conn, "You have been logged in")

  assert {:ok, _} = Users.attempt_login(user.auth_id, @valid_password, @institution)

I want to say that's less scannable than it should be. In the multi-argument assertions, the conn is the least important argument; it really just gets in the way. And the important functions (action__set_fresh_password and attempt_login) are indented (by different amounts) to the right, which makes their importance not visually obvious.

I propose this instead:

test "the password is acceptable", %{conn: conn, user: user} do
  action__set_fresh_password(conn, @valid_password, @valid_password)
  |> assert_logged_in(user, @institution)
  |> assert_no_token_in_session

  |> assert_redirected_home
  |> assert_info_flash_has("You have been logged in")

  Users.attempt_login(user.auth_id, @valid_password, @institution)
  |> assert_ok

It helps with the above problems:

  1. The assertions that are about the same data structure are clearly grouped together.
  2. The two important functions are clearly visible at the top of the two pipelines. (Notice that I turned the original test's pattern-match of the Users.attempt_login result into a simple assertion that "the login attempt was some variant of :ok".)

There's a more subtle benefit, though. It's really tempting just to use assert throughout a test:

assert redirected_to(conn) == PublicController.path(:index)
assert user_id(conn) ==
assert institution(conn) == @institution

But that puts an extra burden on the reader who scans: what does this collection of equalities mean?

Chaining pushes away from this temptation. To use the |> operator, assertions have to be functions that return their first argument. I find that having to write those functions encourages me to think a bit more abstractly and more "intention-revealingly". I'm more likely to ask why someone should care that the assertion is true and then choose the function name with that in mind.

Because I'm an old Lisper, I wrote a macro to define such assertions, saving myself the unbearable burden of having to explicitly return the first argument. Here's what such definitions look like:

defchain assert_user_sees(conn, claims) when is_list(claims) do
  for claim <- claims, do: assert_user_sees(conn, claim)

defchain assert_user_sees(conn, claim) do
  assert(html_response(conn, 200) =~ claim)

The defchain macro can be found here. (Note: that's the version as of the writing of this post, 2019-12-11. Assuming I don't change its location since then, the trunk version is here.)