Ensuring data validity is crucial for any API. This article demonstrates how to implement robust data validation while maintaining clean data formatting.
Custom data validation in Node.js can be complex and time-consuming. Existing Node.js form data libraries often fall short when dealing with intricate data structures or asynchronous validation.
To address these limitations, I developed datalize, a lightweight yet powerful form validation library. Its extensibility allows for project-specific customization. It handles request body, query, and parameter validation, supporting both async filters and complex JSON structures like arrays and nested objects.
Setting Up
Datalize installation is straightforward with npm:
1
| npm install --save datalize
|
For request body parsing, a separate library like koa-body for Koa or body-parser for Express is recommended.
While this tutorial can be integrated into an existing HTTP API server, a simple Koa server can be used for demonstration:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
| const Koa = require('koa');
const bodyParser = require('koa-body');
const app = new Koa();
const router = new (require('koa-router'))();
// helper for returning errors in routes
app.context.error = function(code, obj) {
this.status = code;
this.body = obj;
};
// add koa-body middleware to parse JSON and form-data body
app.use(bodyParser({
enableTypes: ['json', 'form'],
multipart: true,
formidable: {
maxFileSize: 32 * 1024 * 1024,
}
}));
// Routes...
// connect defined routes as middleware to Koa
app.use(router.routes());
// our app will listen on port 3000
app.listen(3000);
console.log('🌍 API listening on 3000');
|
Please note that this setup is for illustrative purposes and lacks production-level features like logging](https://www.loggly.com/blog/node-js-libraries-make-sophisticated-logging-simpler/) and [authorization, handle errors enforcement.
Note: The provided code examples utilize Koa, but the validation logic applies to Express as well. Datalize even includes an example for implementing Express form validation.
Consider an API endpoint for creating users with various fields. Some fields are mandatory, while others might have specific value restrictions or formatting requirements.
Traditional validation logic might resemble:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
| /**
* @api {post} / Create a user
* ...
*/
router.post('/', (ctx) => {
const data = ctx.request.body;
const errors = {};
if (!String(data.name).trim()) {
errors.name = ['Name is required'];
}
if (!(/^[\-0-9a-zA-Z\.\+_]+@[\-0-9a-zA-Z\.\+_]+\.[a-zA-Z]{2,}$/).test(String(data.email))) {
errors.email = ['Email is not valid.'];
}
if (Object.keys(errors).length) {
return ctx.error(400, {errors});
}
const user = await User.create({
name: data.name,
email: data.email,
});
ctx.body = user.toJSON();
});
|
Datalize simplifies this process:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
| const datalize = require('datalize');
const field = datalize.field;
/**
* @api {post} / Create a user
* ...
*/
router.post('/', datalize([
field('name').trim().required(),
field('email').required().email(),
]), (ctx) => {
if (!ctx.form.isValid) {
return ctx.error(400, {errors: ctx.form.errors});
}
const user = await User.create(ctx.form);
ctx.body = user.toJSON();
});
|
Datalize allows you to define fields with chained rules (for validation) and filters (for formatting). Execution order follows the definition order.
A .form object containing only the specified fields is generated. The .form.isValid property indicates validation success.
Automated Error Handling
A global middleware can be added to automatically handle invalid data:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
| const datalize = require('datalize');
// set datalize to throw an error if validation fails
datalize.set('autoValidate', true);
// only Koa
// add to very beginning of Koa middleware chain
app.use(async (ctx, next) => {
try {
await next();
} catch (err) {
if (err instanceof datalize.Error) {
ctx.status = 400;
ctx.body = err.toJSON();
} else {
ctx.status = 500;
ctx.body = 'Internal server error';
}
}
});
// only Express
// add to very end of Express middleware chain
app.use(function(err, req, res, next) {
if (err instanceof datalize.Error) {
res.status(400).send(err.toJSON());
} else {
res.send(500).send('Internal server error');
}
});
|
This middleware, integrated into the application bootstrap process, eliminates manual validation checks. If validation fails, a formatted error message listing invalid fields is returned.
Query Validation
Datalize isn’t limited to POST requests; it handles query parameter validation as well. The .query() helper method is used, storing validated data in the .data object.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
| const datalize = require('datalize');
const field = datalize.field;
/**
* @api {get} / List users
* ...
*/
router.post('/', datalize.query([
field('keywords').trim(),
field('page').default(1).number(),
field('perPage').required().select([10, 30, 50]),
]), (ctx) => {
const limit = ctx.data.perPage;
const where = {
};
if (ctx.data.keywords) {
where.name = {[Op.like]: ctx.data.keywords + '%'};
}
const users = await User.findAll({
where,
limit,
offset: (ctx.data.page - 1) * limit,
});
ctx.body = users;
});
|
Parameter validation is also supported via the .params() method. Multiple middlewares can be chained to validate query and form data simultaneously.
Advanced Filters, Arrays, and Nested Objects
Datalize excels in handling complex data:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
| const datalize = require('datalize');
const field = datalize.field;
const DOMAIN_ERROR = "Email's domain does not have a valid MX (mail) entry in its DNS record";
/**
* @api {post} / Create a user
* ...
*/
router.post('/', datalize([
field('name').trim().required(),
field('email').required().email().custom((value) => {
return new Promise((resolve, reject) => {
dns.resolve(value.split('@')[1], 'MX', function(err, addresses) {
if (err || !addresses || !addresses.length) {
return reject(new Error(DOMAIN_ERROR));
}
resolve();
});
});
}),
field('type').required().select(['admin', 'user']),
field('languages').array().container([
field('id').required().id(),
field('level').required().select(['beginner', 'intermediate', 'advanced'])
]),
field('groups').array().id(),
]), async (ctx) => {
const {languages, groups} = ctx.form;
delete ctx.form.languages;
delete ctx.form.groups;
const user = await User.create(ctx.form);
await UserGroup.bulkCreate(groups.map(groupId => ({
groupId,
userId: user.id,
})));
await UserLanguage.bulkCreate(languages.map(item => ({
languageId: item.id,
userId: user.id,
level: item.level,
));
});
|
Custom rules can be defined using the .custom() method. Nested objects are managed with .container(), allowing for nested field definitions. The .array() filter converts values to arrays and applies specified rules or filters to each element.
Handling PUT/PATCH Requests
Updating data with PUT/PATCH doesn’t necessitate code duplication. Filters like .optional() or .patch() remove undefined fields from the context object, making them optional.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
| const datalize = require('datalize');
const field = datalize.field;
const userValidator = datalize([
field('name').patch().trim().required(),
field('email').patch().required().email(),
field('type').patch().required().select(['admin', 'user']),
]);
const userEditMiddleware = async (ctx, next) => {
const user = await User.findByPk(ctx.params.id);
// cancel request here if user was not found
if (!user) {
throw new Error('User was not found.');
}
// store user instance in the request so we can use it later
ctx.user = user;
return next();
};
/**
* @api {post} / Create a user
* ...
*/
router.post('/', userValidator, async (ctx) => {
const user = await User.create(ctx.form);
ctx.body = user.toJSON();
});
/**
* @api {put} / Update a user
* ...
*/
router.put('/:id', userEditMiddleware, userValidator, async (ctx) => {
await ctx.user.update(ctx.form);
ctx.body = ctx.user.toJSON();
});
/**
* @api {patch} / Patch a user
* ...
*/
router.patch('/:id', userEditMiddleware, userValidator, async (ctx) => {
if (!Object.keys(ctx.form).length) {
return ctx.error(400, {message: 'Nothing to update.'});
}
await ctx.user.update(ctx.form);
ctx.body = ctx.user.toJSON();
});
|
This example demonstrates how two middlewares handle POST/PUT/PATCH logic. userEditMiddleware() verifies record existence, while userValidator() performs validation. The .patch() filter removes undefined fields for PATCH requests.
Custom filters can access other field values and context data like request or user information.
While Datalize provides essential rules and filters, custom global filters prevent code repetition:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
| const datalize = require('datalize');
const Field = datalize.Field;
Field.prototype.date = function(format = 'YYYY-MM-DD') {
return this.add(function(value) {
const date = value ? moment(value, format) : null;
if (!date || !date.isValid()) {
throw new Error('%s is not a valid date.');
}
return date.format(format);
});
};
Field.prototype.dateTime = function(format = 'YYYY-MM-DD HH:mm') {
return this.date(format);
};
|
These custom filters simplify date and datetime validation.
File validation is also supported through dedicated filters like .file(), .mime(), and .size(), eliminating the need for separate file handling.
Enhancing API Development
Datalize has proven invaluable in both small and large production API projects, enabling timely delivery of maintainable and robust applications. Its adaptability extends to WebSocket message validation, demonstrating its versatility.
By leveraging Datalize, developers can focus on building better APIs with reliable data validation, mitigating security risks and server errors. Most importantly, it saves valuable time by streamlining form validation in Node.js.