Simplifying Integration and End-to-end Testing with Node.js and MongoDB

Automated tests are a crucial aspect of developing reliable Node.js applications. When done correctly, these tests can effectively address many of the drawbacks developers might associate with Node.js development.

Although achieving 100% coverage with unit tests is a common goal for many developers, it’s equally vital to ensure that your code functions correctly beyond isolated units. Integration and end-to-end tests offer increased confidence by testing how different parts of your application interact. While individual units might work flawlessly in isolation, within a complex system, they seldom operate independently.

The combination of Node.js and MongoDB has become incredibly popular in recent times. If you’re among the many developers using this powerful duo, you’re in luck.

This article will guide you on how to effortlessly write integration and end-to-end tests for your Node.js and MongoDB application. These tests will run on actual database instances without requiring you to set up a complicated environment or complex setup/teardown code.

You’ll discover how the mongo-unit package simplifies integration and end-to-end testing in Node.js. For a more comprehensive exploration of Node.js integration tests, refer to this article.

Working with a Real Database

In typical scenarios, integration or end-to-end tests require your scripts to interact with a dedicated real database specifically for testing. This entails writing code that executes at the beginning and end of every test case or suite to guarantee that the database is in a clean and predictable state.

While this approach might work for certain projects, it presents some limitations:

  • The testing environment can become intricate, requiring you to maintain a running database, often demanding additional effort for setup on CI servers.
  • Database operations and interactions can be relatively slow due to network communication and file system activity, potentially hindering the quick execution of numerous tests.
  • Databases inherently maintain state, which isn’t ideal for tests. Ideally, tests should be independent, but using a shared database can introduce dependencies where one test might impact others.

On the positive side, utilizing a real database closely mirrors a production environment, offering a distinct advantage to this approach.

Utilizing a Real, In-Memory Database

While using a real database for testing presents certain challenges, the benefits are too significant to ignore. The key lies in finding ways to overcome these challenges while retaining the advantages.

One solution could be to adapt a successful approach from another platform to the Node.js world.

In the Java ecosystem, DBUnit is widely used with in-memory databases like H2 for this purpose.

DBUnit seamlessly integrates with JUnit (the Java testing framework) and enables developers to define the database state for individual tests, test suites, and more. This approach mitigates the limitations discussed earlier:

  • Both DBUnit and H2 are Java libraries, eliminating the need for an additional environment setup as they run within the JVM.
  • Leveraging an in-memory database significantly accelerates state management.
  • DBUnit simplifies database configuration and ensures a clean database state for each test case.
  • H2, being a SQL database partially compatible with MySQL, allows the application to interact with it similarly to a production database in most cases.

Inspired by these concepts, I decided to create a similar solution for Node.js and MongoDB: Mongo-unit.

Mongo-unit is a Node.js package installable through NPM or Yarn that runs an in-memory MongoDB instance. It simplifies integration tests by integrating seamlessly with Mocha and offering a straightforward API for managing the database state.

This library relies on the mongodb-prebuilt NPM package, which provides pre-built MongoDB binaries for commonly used operating systems. These binaries enable running MongoDB instances in an in-memory mode.

Installing Mongo-unit

To incorporate mongo-unit into your project, execute the following command:

1

npm install -D mongo-unit

or

1

yarn add mongo-unit

And that’s it! You don’t even need MongoDB installed locally to utilize this package.

Utilizing Mongo-unit for Integration Tests

Consider a simple Node.js application for managing tasks:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

// service.js

const mongoose = require('mongoose')
const mongoUrl = process.env.MONGO_URL || 'mongodb://localhost:27017/example'
mongoose.connect(mongoUrl)
const TaskSchema = new mongoose.Schema({
 name: String,
 started: Date,
 completed: Boolean,
})
const Task = mongoose.model('tasks', TaskSchema)

module.exports = {
 getTasks: () => Task.find(),
 addTask: data => new Task(data).save(),
 deleteTask: taskId => Task.findByIdAndRemove(taskId)
}

Instead of hardcoding it, the MongoDB connection URL is retrieved from an environment variable, a common practice in web application backends. This approach allows for substituting the URL during tests.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

const express = require('express')
const bodyParser = require('body-parser')
const service = require('./service')
const app = express()
app.use(bodyParser.json())

app.use(express.static(`${__dirname}/static`))
app.get('/example', (req, res) => {
 service.getTasks().then(tasks => res.json(tasks))
})
app.post('/example', (req, res) => {
 service.addTask(req.body).then(data => res.json(data))
})
app.delete('/example/:taskId', (req, res) => {
 service.deleteTask(req.params.taskId).then(data => res.json(data))
})
app.listen(3000, () => console.log('started on port 3000'))

This code snippet represents a simplified example application with a user interface. For brevity, the UI code has been omitted. The complete example can be found on GitHub.

Integrating with Mocha

To configure Mocha to execute integration tests against mongo-unit, the mongo-unit database instance needs to be launched before loading the application code into the Node.js context. We can achieve this using the mocha --require parameter in conjunction with the Mocha-prepare library. This library allows us to perform asynchronous operations within the required scripts.

1
2
3
4
5
6
7
8
9

// it-helper.js
const prepare = require('mocha-prepare')
const mongoUnit = require('mongo-unit')

prepare(done => mongoUnit.start()
 .then(testMongoUrl => {
   process.env.MONGO_URL = testMongoUrl
   done()
 }))

Writing Integration Tests

The first step involves adding a test entry to the test database (testData.json):

1
2
3
4
5
6
7
8
9

{
   "tasks": [
   {
     "name": "test",
     "started": "2017-08-28T16:07:38.268Z",
     "completed": false
   }
 ]
}

Next, we add the tests themselves:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

const expect = require('chai').expect
const mongoose = require('mongoose')
const mongoUnit = require('../index')
const service = require('./app/service')
const testMongoUrl = process.env.MONGO_URL

describe('service', () => {
 const testData = require('./fixtures/testData.json')
 beforeEach(() => mongoUnit.initDb(testMongoUrl, testData))
 afterEach(() => mongoUnit.drop())

 it('should find all tasks', () => {
   return service.getTasks()
     .then(tasks => {
       expect(tasks.length).to.equal(1)
       expect(tasks[0].name).to.equal('test')
     })
 })

 it('should create new task', () => {
   return service.addTask({ name: 'next', completed: false })
     .then(task => {
       expect(task.name).to.equal('next')
       expect(task.completed).to.equal(false)
     })
     .then(() => service.getTasks())
     .then(tasks => {
       expect(tasks.length).to.equal(2)
       expect(tasks[1].name).to.equal('next')
     })
 })

 it('should remove task', () => {
   return service.getTasks()
     .then(tasks => tasks[0]._id)
     .then(taskId => service.deleteTask(taskId))
     .then(() => service.getTasks())
     .then(tasks => {
       expect(tasks.length).to.equal(0)
     })
 })
})

And there you have it!

Notice the minimal setup and teardown code, just a couple of lines.

As you can see, writing integration tests becomes remarkably simple with the mongo-unit library. Instead of mocking MongoDB, we interact with it directly using the same Mongoose models. We maintain complete control over the database’s data and experience minimal performance overhead since the simulated MongoDB instance runs in memory.

This approach also enables us to implement best practices for unit testing in our integration tests:

  • Each test remains independent by loading fresh data before execution, guaranteeing a pristine state.
  • We strive to use the minimum necessary state for each test, populating only the essential data instead of the entire database.
  • A single database connection is reused, enhancing test performance.

As a bonus, we can even run the application itself against mongo-unit, facilitating end-to-end testing against a mocked database.

End-to-end Tests with Selenium

For end-to-end testing, we’ll leverage Selenium WebDriver and Hermione E2E test runner.

Let’s begin by bootstrapping the driver and test runner:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

const mongoUnit = require('mongo-unit')
const selenium = require('selenium-standalone')
const Hermione = require('hermione')
const hermione = new Hermione('./e2e/hermione.conf.js') //hermione config

seleniumInstall() //make sure selenium is installed
 .then(seleniumStart) //start selenium web driver
 .then(mongoUnit.start) // start mongo unit
 .then(testMongoUrl => {
   process.env.MONGO_URL = testMongoUrl //store mongo url
 })
 .then(() => {
   require('./index.js') //start application
 })
 .then(delay(1000)) // wait a second till application is started
 .then(() => hermione.run('', hermioneOpts)) // run hermiona e2e tests
 .then(() => process.exit(0))
 .catch(() => process.exit(1))

We’ll also define some helper functions (error handling omitted for brevity):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

function seleniumInstall() {
 return new Promise(resolve => selenium.install({}, resolve))
}

function seleniumStart() {
 return new Promise(resolve => selenium.start(resolve))
}

function delay(timeout) {
 return new Promise(resolve => setTimeout(resolve, timeout))
}

After populating the database with data and cleaning it up post-tests, we can execute our initial tests:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50

const expect = require('chai').expect
const co = require('co')
const mongoUnit = require('../index')
const testMongoUrl = process.env.MONGO_URL
const DATA = require('./fixtures/testData.json')

const ui = {
 task: '.task',
 remove: '.task .remove',
 name: '#name',
 date: '#date',
 addTask: '#addTask'
}

describe('Tasks', () => {

 beforeEach(function () {
   return mongoUnit.initDb(testMongoUrl, DATA)
     .then(() => this.browser.url('http://localhost:3000'))
 })

 afterEach(() => mongoUnit.dropDb(testMongoUrl))

 it('should display list of tasks', function () {
   const browser = this.browser
   return co(function* () {
     const tasks = yield browser.elements(ui.task)
     expect(tasks.length, 1)
   })
 })

 it('should create task', function () {
   const browser = this.browser
   return co(function* () {
     yield browser.element(ui.name).setValue('test')
     yield browser.element(ui.addTask).click()
     const tasks = yield browser.elements(ui.task)
     expect(tasks.length, 2)
   })
 })

 it('should remove task', function () {
   const browser = this.browser
   return co(function* () {
     yield browser.element(ui.remove).click()
     const tasks = yield browser.elements(ui.task)
     expect(tasks.length, 0)
   })
 })
})

You’ll notice that the end-to-end tests closely resemble the integration tests in structure.

In Conclusion

Integration and end-to-end testing are indispensable for any large-scale application. Node.js applications, in particular, can significantly benefit from automated testing. Mongo-unit empowers you to write integration and end-to-end tests without the typical complexities associated with such tests.

A comprehensive complete examples illustrating the usage of mongo-unit is available on GitHub.

Licensed under CC BY-NC-SA 4.0