A large number of us work with web-based projects that are actively used in production environments, providing various services to the public. When working on such projects, it is crucial to have the capability to build and deploy code rapidly. However, speed often comes at the cost of errors, especially in repetitive tasks. Therefore, it is advisable to automate such processes as much as possible.

This article will explore a tool that can be a part of achieving this automation - an npm package known as Gulp.js. For those unfamiliar with basic Gulp.js terminology, it’s recommended to refer to “An Introduction to JavaScript Automation with Gulp” by Antonios Minas, a fellow Toptal developer, published earlier on this blog. This article will assume a fundamental understanding of the npm environment, as it will be extensively used for package installation.

Serving Front-End Assets

Before proceeding, let’s step back and understand the problem that Gulp.js aims to solve. Many web projects utilize front-end JavaScript and CSS stylesheets to provide functionality and styling to web pages. Examining the source code of such websites or applications often reveals code snippets like this:

1
2
3
4
5
6

<link href="css/main.css" rel="stylesheet">
<link href="css/custom.css" rel="stylesheet">
<script src="js/jquery.min.js"></script>
<script src="js/site.js"></script>
<script src="js/module1.js"></script>
<script src="js/module2.js"></script>

This code presents a few challenges. It references multiple CSS and JavaScript files, requiring the server to make individual requests for each file. While HTTP/2 mitigates this issue with parallelism and header compression, it still leads to increased traffic and slower loading times, impacting user experience. In HTTP 1.1 scenarios, it can even hinder network performance. Ideally, these files should be bundled to reduce the number of requests and improve loading speed. Additionally, serving minified versions of these files, which are significantly smaller, can further enhance performance. Caching of outdated assets can also lead to application breakage.

A rudimentary approach to address some of these concerns involves manually combining assets using a text editor and minifying them through services like http://jscompress.com/. However, this method becomes tedious and inefficient for continuous development. Hosting a local minifier server using available packages on GitHub offers a slightly better, albeit still imperfect, solution. It might involve something similar to:

1

<script src="min/f=js/site.js,js/module1.js"></script>

While this serves minified files, it doesn’t resolve caching issues and adds load on the server due to repetitive concatenation and minification on each request.

Automating with Gulp.js

There’s certainly a more efficient way to handle this. The goal is to automate bundling and minification during the build process, resulting in readily deployable assets. Additionally, we aim to ensure clients receive updated assets while still leveraging caching when possible. Fortunately, Gulp.js offers a solution. This article will guide you through building a Gulp.js-powered solution for concatenating, minifying, and cache-busting assets.

The example project will use the following directory and file structure:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

public/
|- build/
   |- js/
      |- bundle-{hash}.js
   |- css/
      |- stylesheet-{hash}.css
assets/
|- js/
   |- vendor/
   |- jquery.js
   |- site.js
   |- module1.js
   |- module2.js
|- css/
   |- main.css
   |- custom.css
gulpfile.js
package.json

The gulpfile.js defines tasks for Gulp, while package.json manages the application’s package and dependencies using npm. The public directory serves as the web-facing directory, and the assets directory houses source files. To utilize Gulp, install it via npm as a developer dependency. Additionally, install the concat plugin for concatenating files.

Run the following command to install both:

1

npm install --save-dev gulp gulp-concat

Next, start populating gulpfile.js with:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

var gulp = require('gulp');
var concat = require('gulp-concat');
 
gulp.task('pack-js', function () {    
    return gulp.src(['assets/js/vendor/*.js', 'assets/js/main.js', 'assets/js/module*.js'])
        .pipe(concat('bundle.js'))
        .pipe(gulp.dest('public/build/js'));
});
 
gulp.task('pack-css', function () {    
    return gulp.src(['assets/css/main.css', 'assets/css/custom.css'])
        .pipe(concat('stylesheet.css'))
        .pipe(gulp.dest('public/build/css'));
});
 
gulp.task('default', ['pack-js', 'pack-css']);

This code snippet loads Gulp and its concat plugin. It then defines three tasks:

The pack-js task compresses multiple JavaScript source files into a single bundle.js file within public/build/js. Similarly, pack-css concatenates CSS stylesheets into stylesheet.css within public/build/css. The default task, executed when Gulp is run without arguments, runs the other two tasks.

Paste this code into gulpfile.js and save it.

Now, run the following command in your terminal:

1

gulp

This will generate two new files: public/build/js/bundle.js and public/build/css/stylesheet.css, containing concatenated source files. While this partially addresses the initial problem, minification and cache-busting are still pending. Let’s implement automated minification.

Optimizing Built Assets

Two new plugins are needed:

1

npm install --save-dev gulp-clean-css gulp-minify

These plugins, utilizing clean-css and UglifyJS2 packages, handle CSS and JavaScript minification respectively. Begin by loading them in gulpfile.js:

1
2

var minify = require('gulp-minify');
var cleanCss = require('gulp-clean-css');

Next, incorporate them into the tasks before writing output to disk:

1
2

.pipe(minify())
.pipe(cleanCss())

The updated gulpfile.js should look like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

var gulp = require('gulp');
var concat = require('gulp-concat');
var minify = require('gulp-minify');
var cleanCss = require('gulp-clean-css');
 
gulp.task('pack-js', function () {    
    return gulp.src(['assets/js/vendor/*.js', 'assets/js/main.js', 'assets/js/module*.js'])
        .pipe(concat('bundle.js'))
        .pipe(minify())
        .pipe(gulp.dest('public/build/js'));
});
 
gulp.task('pack-css', function () {    
    return gulp.src(['assets/css/main.css', 'assets/css/custom.css'])
        .pipe(concat('stylesheet.css'))
        .pipe(cleanCss())
   .pipe(gulp.dest('public/build/css'));
});
 
gulp.task('default', ['pack-js', 'pack-css']);

Running Gulp again will generate a minified stylesheet.css, an unminified bundle.js, and a minified bundle-min.js. Since only the minified bundle.js is desired, modify the code with additional parameters:

1
2
3
4
5
6

.pipe(minify({
    ext:{
        min:'.js'
    },
    noSource: true
}))

Based on gulp-minify plugin documentation (https://www.npmjs.com/package/gulp-minify), this configures the plugin to name the minified file bundle.js and prevents the creation of an unminified version. Deleting the contents of the build directory and running Gulp again will result in two minified files, completing the minification phase.

Cache Busting

Next, implement cache busting by installing the required plugin:

1

npm install --save-dev gulp-rev

Require it in the Gulp file:

1

var rev = require('gulp-rev');

The plugin requires piping the minified output through it, followed by another call after writing to disk. It renames files with a unique hash and creates a manifest.json file for mapping filenames. The modified gulpfile.js should look like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

var gulp = require('gulp');
var concat = require('gulp-concat');
var minify = require('gulp-minify');
var cleanCss = require('gulp-clean-css');
var rev = require('gulp-rev');
 
gulp.task('pack-js', function () {    
    return gulp.src(['assets/js/vendor/*.js', 'assets/js/main.js', 'assets/js/module*.js'])
        .pipe(concat('bundle.js'))
        .pipe(minify({
            ext:{
                min:'.js'
            },
            noSource: true
        }))
        .pipe(rev())
        .pipe(gulp.dest('public/build/js'))
        .pipe(rev.manifest())
        .pipe(gulp.dest('public/build'));
});
 
gulp.task('pack-css', function () {
    return gulp.src(['assets/css/main.css', 'assets/css/custom.css'])
        .pipe(concat('stylesheet.css'))
        .pipe(cleanCss())
        .pipe(rev())
            .pipe(gulp.dest('public/build/css'))
        .pipe(rev.manifest())
        .pipe(gulp.dest('public/build'));
});
 
gulp.task('default', ['pack-js', 'pack-css']);

Running Gulp after deleting the build directory will generate two hash-tagged files and a manifest.json file in public/build. However, the manifest file will only contain one entry due to overwriting. To resolve this, modify the tasks to merge data into an existing manifest:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

var gulp = require('gulp');
var concat = require('gulp-concat');
var minify = require('gulp-minify');
var cleanCss = require('gulp-clean-css');
var rev = require('gulp-rev');
 
gulp.task('pack-js', function () {
    return gulp.src(['assets/js/vendor/*.js', 'assets/js/main.js', 'assets/js/module*.js'])
        .pipe(concat('bundle.js'))
        .pipe(minify({
            ext:{
                min:'.js'
            },
            noSource: true
        }))
        .pipe(rev())
        .pipe(gulp.dest('public/build/js'))
        .pipe(rev.manifest('public/build/rev-manifest.json', {
            merge: true
        }))
        .pipe(gulp.dest(''));
    });
 
gulp.task('pack-css', function () {    
    return gulp.src(['assets/css/main.css', 'assets/css/custom.css'])
        .pipe(concat('stylesheet.css'))
        .pipe(cleanCss())
        .pipe(rev())
        .pipe(gulp.dest('public/build/css'))
        .pipe(rev.manifest('public/build/rev-manifest.json', {
            merge: true
        }))
        .pipe(gulp.dest(''));
});
 
gulp.task('default', ['pack-js', 'pack-css']);

This code pipes the output through rev.manifest(), creating tagged files and merging data into an existing or new rev-manifest.json file in the correct directory.

With minification, tagged files, and a manifest file in place, assets can be delivered efficiently, and caches can be busted effectively. Two minor issues remain:

Firstly, modifying source files leaves old minified files behind. Address this by installing a plugin for file deletion:

1

npm install --save-dev del

Require it in the code and define two new tasks:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

var del = require('del');
 
gulp.task('clean-js', function () {
    return del([
        'public/build/js/*.js'
    ]);
});
 
gulp.task('clean-css', function () {
    return del([
        'public/build/css/*.css'
    ]);
});

Ensure these tasks run before the main tasks:

1
2

gulp.task('pack-js', ['clean-js'], function () {
gulp.task('pack-css', ['clean-css'], function () {

Now, running gulp will only keep the latest minified files.

Secondly, avoid running Gulp manually after every change. Implement a watcher task for this purpose:

1
2
3
4

gulp.task('watch', function() {
 gulp.watch('assets/js/**/*.js', ['pack-js']);
 gulp.watch('assets/css/**/*.css', ['pack-css']);
});

Modify the default task definition:

1

gulp.task('default', ['watch']);

Now, Gulp will only build when changes are detected in source files.

Finally, load the manifest.json file in your application to retrieve tagged filenames. The implementation will vary depending on the back-end technology stack but should involve loading the manifest into an array or object and defining a helper function. This function will enable you to call versioned assets from templates, potentially like this:

1

gulp(‘bundle.js’)

This eliminates concerns about changing filenames and allows you to focus on writing code.

The final source code for this article, along with some sample assets, is available at this GitHub repository.

Conclusion

This article outlined the implementation of Gulp-based build automation. Hopefully, it provides valuable insights for developing sophisticated build processes in your applications.

Keep in mind that Gulp is just one tool among many, including Grunt, Browserify, and Webpack, each with its strengths and limitations. Some tools, like Browserify and Webpack, address challenges beyond Gulp’s capabilities, such as code splitting, which involves bundling JavaScript modules with on-demand loading. These advanced tools, though not covered here, offer further optimization opportunities and might be explored in future articles. In a subsequent post, we will delve into automating application deployment.