Using Snapshots in Your API Integration Tests

A snapshot is a saved test result that we compare future results against. If, after adding new code, the results are equal, then it indicates no new bugs were introduced.

April 5, 2022

Writing automated tests is arguably the most effective way to minimize bugs. In this walkthrough, we’ll discuss how to use snapshots to help write more concise and thorough integration tests. To do so, we’ll implement a tiny Node.js app using Express, Knex.js, and PostgreSQL. We’ll then explore how to add integration tests with snapshots using Mocha, chai, and timekeeper.

What Are Integration Tests and Why Use Them?

With integration tests, we make assertions of how we expect our software to behave in a production-like environment. For example, you might boot up a Node web server and a database server, issue a POST /users request, and check whether the expected user was created.

Integration tests are great at treating our apps as black boxes. As long as an app receives the same input and returns the same output, our tests will happily pass. This means we can change the internal implementation of our code and still use the same tests.

What Are Snapshots and Why Use Them?

A snapshot is a saved test result that we compare future results against. If, after adding new code, the results are equal, then it indicates no new bugs were introduced.

For example, suppose we implemented a POST /users endpoint. When we run our automated tests for the first time, we hit the POST /users endpoint and generate a snapshot by saving the JSON response into a file. We manually check that all the keys and values in the JSON file are correct. Later on, we add some additional code elsewhere, but we want to be confident that POST /users works the same as before. To verify this, we rerun our tests, but this time, we check that the JSON response matches our previously generated snapshot exactly.

Snapshots are great in that we can make deep structural comparisons without adding dozens of lines of code to our test. For example, without snapshots, our test to check if creating a user works correctly might look something like this:

const user = await postUsers(...)
expect(user.email).to.eq(...)
expect(user.name).to.eq(...)
expect(user.createdAt).to.eq(...)
// Repeat the above for all of user's attributes.

With snapshots, it’d look like this:

const user = await postUsers(...)
expect(user).to.matchSnapshot()

As you can see, we bundled all the expect(...).to.eq(...)calls into a single check against our snapshot, which reduced the line count of our test!

Tricky Parts

Our tests must be deterministic, which means that given the same input, we expect the same output. For some fields, this becomes tricky. Consider a JSON response from our POST /users endpoint that contains a createdAt field. If we don’t add some sort of handling for the current time, running the test now and running the same test a few seconds later will result in different values for createdAt.

Another issue that pops up is when our code relies on randomness. With randomness, we have the problem that results will be different from test run to test run. For example, if you’re using UUID for your IDs, which are randomly generated, then you’ll face this issue. If you’re using Math.random(), you’ll face the same issue.

What Can We Do?

For dates, we can “freeze” to a certain point in time (we’ll use a package called timekeeper for this). When we “freeze” time, each call to new Date() will return a date object referencing the frozen point of time. That way, we can avoid the user’s createdAt field changing between test runs.

For code that relies on randomness, we can substitute the random implementation for a deterministic one. We’ll explore how to do that shortly.

Let’s Build This!

Start by bootstrapping your Node project in your terminal:

mkdir tests-walkthrough
cd tests-walkthrough
npm init # You can keep all the default values here.

# Install some dependencies needed for the example.
npm install express body-parser knex pg uuid
npm install --save-dev mocha chai chai-http timekeeper

Next, set up your databases. You’ll have two databases: tests_walkthrough_dev for your “dev” environment, and tests_walkthrough_test for your “test” environment. Make sure you have Postgres installed and running:

createdb tests_walkthrough_dev
psql tests_walkthrough_dev

Enter Postgres’ CLI. From here, you can create your users table:

CREATE TABLE users (
  id UUID PRIMARY KEY,
  email TEXT NOT NULL UNIQUE,
  hashed_password TEXT NOT NULL,
  created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW()  
)

Next, repeat the two steps above, but for tests_walkthrough_test:

createdb tests_walkthrough_test
psql tests_walkthrough_test
CREATE TABLE users (
  id UUID PRIMARY KEY,
  email TEXT NOT NULL UNIQUE,
  hashed_password TEXT NOT NULL,
  created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW()  
)

Now you can start writing some code. Begin by creating a src/stores.js file that will handle the database access logic:

// src/stores.js
const { DatabaseError } = require('pg')

// Add a `src/errors` file in the next step.
const { ConflictError } = require('./errors')

const SQL_CONFLICT_ERROR_CODE = '23505'

class UserStore {
    table = 'users'

    /*
      When instantiating `UserStore`, pass a Knex instance as well as
      the function used to generate UUIDs. By passing `generateUUID`, you'll be
      able to pass a deterministic function during testing and a non-deterministic
      function during dev and production.
    */ 
    constructor(knex, generateUUID) {
        this.knex = knex
        this.generateUUID = generateUUID
    }

    async create({ email, hashedPassword }) {
        try {
            const [row] = await this.knex(this.table)
                .insert({
                    id: this.generateUUID(),
                    email,
                    hashed_password: hashedPassword,
                    created_at: new Date(),
                })
                .returning('*')

            return {
                id: row.id,
                email: row.email,
                hashedPassword: row.hashed_password,
                createdAt: row.created_at,
            }
        } catch (e) {
            if (
                e instanceof DatabaseError &&
                e.code === SQL_CONFLICT_ERROR_CODE
            ) {
                throw new ConflictError('This email is already in use.')
            }
        }
    }
}

module.exports = { UserStore } 

The code above references src/errors.js. This file will contain the custom errors you can catch and throw. Add it now:

// src/errors.js
class ConflictError extends Error {}

module.exports = { ConflictError }

Next, implement a helper to initialize your app, along with a POST /users endpoint. Create a file named src/app.js:

const express = require('express')
const bodyParser = require('body-parser')
const crypto = require('crypto')

const { ConflictError } = require('./errors')

// Use `prepareApp` during dev and testing with different stores. This way, you 
// can have stores referencing the `tests_walkthrough_dev` database in dev and
// `tests_walkthrough_test` in testing. This also opens up a straightforward
// way to inject mocks for your stores.
function prepareApp(stores) {
    const app = express()
    
    // Use `bodyParser.json()` so that `req.body` is set to a parsed JSON object
    // when JSON data is passed to your endpoint.
    app.use(bodyParser.json())

    app.post('/users', async (req, res) => {
        // During snapshot testing, you'll take a snapshot of the response headers.
        // Express automatically includes a `date` header, but it doesn't use
        // `new Date()` to get the date.
        // This is problematic, as your date freezing library won't work,
        // so instead, set it manually here. This can be done in
        // middleware if you have more than a couple endpoints.
        res.header('date', new Date().toUTCString())

        const { email, password } = req.body
        const salt = crypto.randomBytes(64).toString('hex')
        const hash = crypto.scryptSync(password, salt, 64).toString('hex')

        const hashedPassword = salt + hash

        try {
            // Use `var` instead of `const` for declaring the `user` 
            // variable so that you can have access outside of the `try` block.
            var user = await stores.users.create({ email, hashedPassword })
        } catch (e) {
            if (e instanceof ConflictError) {
                return res.status(409).json({ message: e.message })
            }

            throw e
        }

        // Happy path. Return the JSON representation of the user to the client.
        res.json({
            id: user.id,
            email: user.email,
            createdAt: user.createdAt.toISOString(),
        }).status(201)
    })

    return app
}

module.exports = prepareApp

You now have all the pieces to set up and run your Express app. Put it all together in src/index.js and serve your endpoint:

// src/index.js
const Knex = require('knex')
const uuid = require('uuid')

const { UserStore } = require('./store')
const prepareApp = require('./app')

// Pass the Postgres connection string through the `PG_DSN` environment variable.
const knex = Knex({ client: 'pg', connection: process.env['PG_DSN'] })

const stores = {
    // Use non-deterministic UUID generation for the dev environment.
    users: new UserStore(knex, uuid.v4),
}

const app = prepareApp(stores)

app.listen(3000, () => console.log('listening on port 3000'))

Add a scripts entry in your package.json to run your server:

// package.json
{
    ...
    "scripts": {
        "dev": "node src/index.js"
    }
    ...
}

Finally, run your server and issue a curl request to make sure everything is working as intended. In a terminal, run the following (you might need to alter the PG_DSN if you’re running Postgres on a different port):

PG_DSN=postgresql://localhost:5432/tests_walkthrough_dev npm run dev

And, in a separate terminal, run:

curl localhost:3000/users -H 'Content-Type: application/json' -d '{"email": "test@test.com", "password": "abcd1234"}'

The output of the curl request should be a JSON object with the id, email, and createdAt keys.

Time to Write Some Tests!

You did some manual testing, but you want to ensure that as the codebase grows, this endpoint continues to work as expected. To do so, you’ll write some automated tests and leverage snapshots. You’ll first check that there are no changes in the HTTP headers, body, or status code between test suite runs. Then, you’ll use the mocha, chai, chai-http, and timekeeper packages to run the tests. Additionally, you’ll write some custom code to save and compare snapshots.

Start with writing the snapshot handler. First, create a tests directory and then add snapshot.js. Inside snapshot.js, create a chai helper to extend chai‘s assertions. Specifically, add a matchSnapshot assertion so you can write expect(result).to.matchSnapshot(this) inside your tests. Here’s the code:

// tests/snapshot.js
const { expect } = require('chai')
const fs = require('fs')
const path = require('path')

// Pass this function into `chai.use`, which will then allow you to
// use `expect(...).to.matchSnapshot(this)`.
function chaiSnapshot(chai, utils) {
    utils.addMethod(chai.Assertion.prototype, 'matchSnapshot', function (ctx) {
        // The first thing you need to do is set up the location for 
        // reading/writing your snapshots.
        const dir = path.dirname(ctx.test.file)
        const basename = path.basename(ctx.test.file)

        // If your test file is `tests/index.test.js` and the current test is
        // inside a `POST /users` Mocha context, with the test itself
        // in a Mocha `it` block titled `creates a user`, you'll create
        // the snapshot file as 
        // `tests/__snapshots__/index.test.js-POST--users-creates-a-user.json`.
        // Note: Substitute spaces and "/" with "-" characters.
        const snapshotPath = path.join(
            dir,
            '__snapshots__',
            // `prepareTitle` is defined below.
            `${basename}-${prepareTitle(ctx.test)}.json`
        )

        // Check the `GENERATE_SNAPSHOTS` environment variable to determine
        // whether or not this test run should generate new snapshots.
        const generateSnapshots = process.env.GENERATE_SNAPSHOTS === 'true'
        if (generateSnapshots) {
            writeSnapshot(snapshotPath, this._obj)
            return
        }

        // Read and parse the previous snapshot.
        const expected = readSnapshot(snapshotPath)
        // `this._obj` contains the value of what was passed to
        // `expect(obj)`.
        expect(this._obj).to.deep.eq(expected)
    })
}

// Helper to read the snapshot from the file system and parse it as JSON.
function readSnapshot(path) {
    if (!fs.existsSync(path)) {
        throw new Error(
            `File "${path}" does not exist. To generate snapshots run tests with GENERATE_SNAPSHOTS=true.`
        )
    }

    return JSON.parse(fs.readFileSync(path, { encoding: 'utf-8' }))
}

// Helper to write the snapshot to the file system.
function writeSnapshot(dest, json) {
    if (!fs.existsSync(path.dirname(dest))) {
        fs.mkdirSync(path.dirname(dest))
    }

    fs.writeFileSync(dest, JSON.stringify(json, null, 4), { encoding: 'utf-8' })
}

// Helper to prepare a name for the snapshot file by recursively
// walking up the Mocha context tree and concatinating the titles, replacing
// spaces and "/" characters with the "-" character.
function prepareTitle(ctx) {
    const title = ctx.title.replaceAll(' ', '-').replaceAll('/', '-')
    if (ctx.parent.file) {
        return `${prepareTitle(ctx.parent)}-${title}`
    }
    return title
}

module.exports = chaiSnapshot

With the helper in place, it’s time to write your tests. Create a new file named index.test.js:

// tests/index.test.js
const chaiHttp = require('chai-http')
const timekeeper = require('timekeeper')
const Knex = require('knex')
const { expect, use: chaiUse } = require('chai')
const chai = require('chai')

const prepareApp = require('../src/app')
const { UserStore } = require('../src/store')
const snapshot = require('./snapshot')

// Use chai to make requests to your express server.
chaiUse(chaiHttp)
// Register your snapshot helper with chai.
chaiUse(snapshot)

// Helper class to generate deterministic UUIDs.
class DeterministicUUID {
    i = 0

    // Each call to generate creates a "sequential UUID"
    // e.g, first call: '00000000-0000-0000-0000-000000000001'
    // second call: '00000000-0000-0000-0000-000000000002'
    // third call: '00000000-0000-0000-0000-000000000003'
    // etc.
    generate() {
        this.i += 1
        const temp = this.i.toString().padStart(32, '0')
        return (
            temp.substring(0, 8) +
            '-' +
            temp.substring(8, 12) +
            '-' +
            temp.substring(12, 16) +
            '-' +
            temp.substring(16, 20) +
            '-' +
            temp.substring(20)
        )
    }

    reset() {
        this.i = 0
    }
}

before(async function () {
    this.knex = Knex({ client: 'pg', connection: process.env['PG_DSN'] })

    this.uuidGenerator = new DeterministicUUID()
    // Because you inject the UUID generator function, you can use your
    // deterministic implementation here.
    this.stores = {
        users: new UserStore(this.knex, () => this.uuidGenerator.generate()),
    }

    // Freeze the time to the Unix epoch. Calling `new Date()` will
    // always return the same result.
    timekeeper.freeze(new Date(0))
})

afterEach(async function () {
    // Clear your database state between each test.
    await this.knex('users').del()
    // Reset your UUID generator between tests so that tests
    // can be run in any order.
    this.uuidGenerator.reset()
})

after(async function () {
    // When your test suite completes, release the database connection.
    await this.knex.destroy()
})

describe('POST /users', function () {
    // Test the happy path.
    it('creates a users', async function () {
        const app = prepareApp(this.stores)

        const resp = await chai
            .request(app)
            .post('/users')
            .send({ email: 'hello@world.com', password: 'abc123' })

        const result = {
            body: resp.body,
            headers: resp.headers,
            status: resp.status,
        }
        // Use your `matchSnapshot` that you wrote in `snapshot.js`. 
        expect(result).to.matchSnapshot(this)
    })

    // Test your non-happy path.
    it('returns a HTTP 409 when using an existing email', async function () {
        // Create a user whose email you'll purposefully attempt to reuse.
        const existingUser = await this.stores.users.create({
            email: 'hello@world.com',
            hashedPassword: 'abc123',
        })

        const app = prepareApp(this.stores)
        const resp = await chai
            .request(app)
            .post('/users')
            .send({ email: existingUser.email, password: 'abc123' })

        const result = {
            body: resp.body,
            headers: resp.headers,
            status: resp.status,
        }

        expect(result).to.matchSnapshot(this)
    })
})

The next step is to update package.json by adding a script to run your tests. Then, generate the snapshots themselves. Open package.json and add the testing script:

// package.json
{
    ...
    "scripts": {
        "dev": "node src/index.js",
        "test": "mocha tests/**/*.test.js"
    }
    ...
}

Inside your terminal, run your tests and generate some snapshots:

GENERATE_SNAPSHOTS=true PG_DSN=postgresql://localhost:5432/tests_walkthrough_test npm run test

Your tests/ directory should now look like this:

.
├── __snapshots__
│   ├── index.test.js-POST--users-creates-a-users.json
│   └── index.test.js-POST--users-returns-a-HTTP-409-when-using-an-existing-email.json
├── index.test.js
└── snapshot.js

The next step is to manually verify that the JSON snapshots contain the values you expect. Take a look at index.test.js-POST--users-creates-a-users.json first:

// tests/__snapshots__/index.test.js-POST--users-creates-a-users.json
{
    "body": {
        "id": "00000000-0000-0000-0000-000000000001",
        "email": "hello@world.com",
        "createdAt": "1970-01-01T00:00:00.000Z"
    },
    "headers": {
        "x-powered-by": "Express",
        "date": "Thu, 01 Jan 1970 00:00:00 GMT",
        "content-type": "application/json; charset=utf-8",
        "content-length": "110",
        "etag": "W/\"6e-cA1jXY9f5RqYo2/CjViDFea0hYg\"",
        "connection": "close"
    },
    "status": 201
}

Awesome! The above JSON looks correct. You asked your server to create a user with an email of "hello@world.com", and you see that it did. You can also see that the createdAt field correctly contains a timestamp reference to the Unix epoch, which is the point in time you froze time to earlier.

You have some HTTP headers under the "headers" key and the HTTP status under the "status" key. Everything looks good there as well.

Next, look at index.test.js-POST--users-returns-a-HTTP-409-when-using-an-existing-email.json:

// tests/__snapshots__/index.test.js-POST--users-returns-a-HTTP-409-when-using-an-existing-email
{
    "body": {
        "message": "This email is already in use."
    },
    "headers": {
        "x-powered-by": "Express",
        "date": "Thu, 01 Jan 1970 00:00:00 GMT",
        "content-type": "application/json; charset=utf-8",
        "content-length": "43",
        "etag": "W/\"2b-A8vwThSvpsX0XH1Kvhrwsyf2v8s\"",
        "connection": "close"
    },
    "status": 409
}

Again, you have the expected JSON body, headers, and status. To finish off, run a couple of checks. First, run the tests without the GENERATE_SNAPSHOTS flag:

PG_DSN=postgresql://localhost:5432/tests_walkthrough_test npm run test

The output should look like this:

> tests-walkthrough@1.0.0 test
> mocha tests/**/*.js



  POST /users
    ✔ creates a users (89ms)
    ✔ returns a HTTP 409 when using an existing email (45ms)


  2 passing (152ms)

Finally, alter a test to make sure the test suite fails when values don’t line up to snapshots:

// tests/index.test.js

...
describe('POST /users', function () {
    // Test the happy path.
    it('creates a users', async function () {
        const app = prepareApp(this.stores)

        // Change the email from "hello@world.com" to "bonjour@world.com".
        const resp = await chai
            .request(app)
            .post('/users')
            .send({ email: 'bonjour@world.com', password: 'abc123' })

        const result = {
            body: resp.body,
            headers: resp.headers,
            status: resp.status,
        }
        // Use your `matchSnapshot` that you wrote in `snapshot.js`. 
        expect(result).to.matchSnapshot(this)
    })
  ...

Run your test script now:

PG_DSN=postgresql://localhost:5432/tests_walkthrough_test npm run test

The output should now look like this:

POST /users
    1) creates a users
    ✔ returns a HTTP 409 when using an existing email (45ms)


  1 passing (158ms)
  1 failing

  1) POST /users
       creates a users:

      AssertionError: expected { body: { …(3) }, …(2) } to deeply equal { body: { …(3) }, …(2) }
      + expected - actual

       {
         "body": {
           "createdAt": "1970-01-01T00:00:00.000Z"
      -    "email": "bonjour@world.com"
      +    "email": "hello@world.com"
           "id": "00000000-0000-0000-0000-000000000001"
         }
         "headers": {
           "connection": "close"
      -    "content-length": "112"
      +    "content-length": "110"
           "content-type": "application/json; charset=utf-8"
           "date": "Thu, 01 Jan 1970 00:00:00 GMT"
      -    "etag": "W/\"70-2SU0aVc49WJXZ9LoI770ZB3EYKY\""
      +    "etag": "W/\"6e-cA1jXY9f5RqYo2/CjViDFea0hYg\""
           "x-powered-by": "Express"
         }
         "status": 201
       }

Perfect! Your tests fail when the value changes, and you get a visual of what exactly changed (in this case, the email of the body, and content-length and etag for the headers).

Discussion

Typically, with tests that hit our endpoints, we’d check a handful of attributes of the response. For example, we may do a check like expect(email).to.eq("hello@world.com") and have some confidence that our code works correctly. Checking every attribute in this manner becomes cumbersome, and the tests themselves become long and difficult to parse. With snapshots, we can perform deep structural checks while avoiding a barrage of expect(..).to.eq(...). This is especially apparent if our user object had dozens of properties.

There are, however, no silver bullets! Using snapshots has its tradeoffs. Snapshots require that we’re able to run our code deterministically, and that’s not always possible. Another issue is that our snapshot file matching is determined by the test’s name and its Mocha context(s). If we make even a tiny edit to the test name, then we’ll have to regenerate that snapshot. We also need to remember that when we delete a test, we delete its matching snapshot as well.

Ultimately, snapshot testing should be viewed as another tool in your testing toolbox — something you can reach for when you need a concise way to ensure that complex output remains the same between test runs.


Share this post: