E2e testing of HTTP services in Node.js
How to test in JavaScript logic that involves making an HTTP call to a foreign/outside web server.
When testing HTTP calls within your application you would usually mock your functions and completely skip the HTTP call, and it can work to some extent, plus it’s relatively easy to do so. Though in the end, you are faking and not really testing it, what if there’s something wrong with the call you make, wrong method, parameter, query, data value? How do you know the library is doing what you think it’s doing, what happens if an error occurs?
For these exact reasons, it’s far better and safer to perform a real HTTP test, and here we move to
integration testing domain from the above unit testing where we don’t fake communication, but do it for real with
another system.
In the next sections we will look into two ways we can perform HTTP integration testing and evaluate both ways.
Dummy Express.js HTTP server
To make things easier and faster, we can install the following library I created for starting an express server in your tests which we can use and manipulate to produce responses or errors that we wish for each scenario.
npm i -D express-dummy-server
Basic usage
It’s easier to create a separate file to house your dummy server and register the default functionality for the endpoints we want to use
// dummy-user-server.ts
import {createDummyServer, DummyServer, RequestSnapshot} from "express-dummy-server";
import {RequestHandler} from "express";
// Enum is used for mapping handlers so that we can change handlers later on
export enum DummyUserServiceHook {
getUser = 'getUser'
}
export const createDummyUserService = async (
requestHooks?: Map<DummyUserServiceHook, RequestHandler | undefined>
): Promise<DummyServer> =>
createDummyServer(async (app) => {
app.get('/users/:id', (req, res, next) => {
const {id} = req.params;
// Here we check if there is a new mapped handler that we should use
const getUserHook = requestHooks?.get(DummyUserServiceHook.getUser);
if (getUserHook) {
return getUserHook(req, res, next)
}
res.json({id: Number(id), name: 'John'})
});
});
Here we return a plain json response by default: {id: Number(id), name: 'John'}.
Then we can use it in our *.spec.ts file like following using jest testing library:
import axios from "axios";
import {RequestSnapshot} from "express-dummy-server";
import {RequestHandler} from "express";
import {DummyUserServiceHook} from './dummy-user-server';
describe('DummyServer', () => {
let server: DummyServer;
let hooks = new Map<DummyUserServiceHook, RequestHandler | undefined>();
beforeAll(async () => {
server = await createDummyUserService(hooks);
});
afterAll(() => {
server.close();
});
beforeEach(() => {
server.requestStore.clear();
hooks.clear();
});
it('should return a dummy server response', async () => {
const res = await axios.get(`${server.url}/users/2`);
expect(res.data).toEqual({id: 2, name: 'John'});
const requestSnapshots = server.requestStore.get('/users/2');
expect(requestSnapshots).toHaveLength(1);
const {body, headers, method, params, query} = (requestSnapshots as RequestSnapshot[])[0];
expect(method).toBe('GET');
expect(body).toEqual({});
});
})
Mocking
Let’s say we don’t want to use the default handler and response of the dummy server we created, for that reason we created hooks earlier, so we can reference the handler and swap out the function:
it('should return a mocked dummy server response', async () => {
hooks.set(DummyUserServiceHook.getUser, (req, res) => {
const {id} = req.params;
res.json({id: Number(id), name: 'Hello Mike'});
})
const res = await axios.get(`${server.url}/users/2`);
expect(res.data).toEqual({id: 2, name: 'Hello Mike'})
const requestSnapshots = server.requestStore.get('/users/2');
expect(requestSnapshots).toHaveLength(1);
const {body, headers, method, params, query} = (requestSnapshots as RequestSnapshot[])[0];
expect(method).toBe('GET');
expect(body).toEqual({});
});
In the above example we overwritten the hook DummyUserServiceHook.getUser to use the handler we passed to it, with
it we can do whatever our heart desires with the request and return what we wish. You can even have a counter outside
the handler to do something specific on a specified request number.
Now to make some it even easier for overwriting responses through mocks we can use a utility function toJson:
import {respondJson} from "./request-handler-utils";
// respondJson
hooks.set(DummyUserServiceHook.getUser, respondJson({id: Number(id), name: 'Hello Mike'}));
// Which is identical to
hooks.set(DummyUserServiceHook.getUser, (req, res) => {
const {id} = req.params;
res.json({id: Number(id), name: 'Hello Mike'});
})
Note that the respondJson can take one more argument which specifies the returning code, which you can set for
example to 500 to indicate InternalServerError.
hooks.set(DummyUserServiceHook.getUser, respondJson({message: 'InternalServerError'}, 500));
Debugging
If you want to debug requests and responses made against the server you can turn on the debug mode to console log them and (if needed) pass the logger to it.
import {DummyServerOptions, createDummyServer} from "express-dummy-server";
const options: DummyServerOptions = {debug: true};
const dummyUserServer = await createDummyServer(async (app) => {
app.get('/users/:id', respondJson({id: 1, name: 'John'}));
}, options);
Summary
This library is just express.js and nothing else, it’s extremely lightweight and fast for use in test suits and provides what you need for most cases.
Mockserver
One alternative is the Mockserver. Now I won’t write the full example of usage here as it’s quite large and everything is available there on the website, but I will do a short revision of the library.
Main advantage of this library is that it’s feature rich, it provides out of the box a lof of things like generating the mock server based on a OpenAPI schema, has a Docker image, cli,…
Example of programmatically usage in tests.
npm i -D @mocks-server/main
You would have a separate directory for mocks
project-root/
├── mocks/
│ ├── routes/ <- DEFINE YOUR ROUTES HERE
│ │ ├── common.js
│ │ └── users.js
│ └── collections.json <- DEFINE YOUR COLLECTIONS HERE
└── mocks.config.js <- DEFINE YOUR CONFIGURATION HERE
// mocks/routes/users.js
module.exports = [
{
id: "get-users", // route id
url: "/api/users", // url in express format
method: "GET", // HTTP method
variants: [
{
id: "success", // variant id
type: "json", // variant handler id
options: {
status: 200, // status to send
body: USERS, // body to send
},
},
{
id: "all", // variant id
type: "json", // variant handler id
options: {
status: 200, // status to send
body: ALL_USERS, // body to send
},
},
{
id: "error", // variant id
type: "json", // variant handler id
options: {
status: 400, // status to send
// body to send
body: {
message: "Error",
},
},
},
],
}
]
mocks/collections.json
[
{
"id": "base",
"routes": ["add-headers:enabled", "get-users:success", "get-user:success"]
},
{
"id": "no-headers",
"from": "base",
"routes": ["add-headers:disabled"]
},
{
"id": "all-users",
"from": "base",
"routes": ["get-users:all", "get-user:id-3"]
},
{
"id": "user-real",
"from": "no-headers",
"routes": ["get-user:real"]
}
]
Then through the tests you would be able to utilize those mocks and change which responses certain routes would return.
const { createServer } = require("@mocks-server/main");
const { routes, collections } = require("./fixtures");
beforeAll(async () => {
const server = createServer();
const { loadRoutes, loadCollections } = server.mock.createLoaders();
loadRoutes(routes);
loadCollections(collections);
await server.start();
});
afterAll(async () => {
await server.stop();
});
describe("users API client", () => {
it("getUsers method should return 3 users", async () => {
// Select the collection returning the expected data, and wait for the mock to be ready
await server.mock.collections.select("3-users", { check: true });
// configure the unit under test
const usersApiClient = new UsersService('http://localhost:3100/api/users');
// call your unit under test, which invokes the mock
const users = await usersApiClient.getUsers();
// assert values returned by the mock
expect(users.length).toEqual(3);
});
});
Summary
Mockserver can provide powerful documentation and a boilerplate for a setup that will speed up writing integration tests with clean separation of mocks, some things that bother me is that their documentation is always missing something so when doing a quick start I would get an error when loading routes or collections that wouldn’t explain in the error what was the issue or when some references in the documentation were missing. Not having types for the library is also a big minus as it’s a guess in the dark what are the methods/classes being exposed through it.
Overview of the two libraries
When developing I always go for the simplest solutions to avoid a headache learning another new big tool and having to maintain it and for all my cases the simple structure of the express-dummy-server was enough for me to cover all the cases. Everyone knows express.js thus extending the dummy server was an ease for custom functionality.