Migrate JavaScript Modules From AMD to ES6

Last year I had the opportunity to help the front-end team migrate their codebase of 600 files and 70,000 lines of code to current standards.

From start to finish the project took 2-weeks.

First week I created a working proof of concept which included fully removing RequireJS and setting up Webpack/Babel.

Second week entailed coming up with a launch strategy and planning a 2-hour code freeze to perform the merge. During this period I worked with the front-end lead to launch the branch and quickly transform any files that were modified before the latest merge.

The project was successful and far less painful than first expected. It greatly improved the developer experience and it came with some optimizations out of the box — first load went from 320 to 40 requests.

Below is a summary for migrating AMD (Asynchronous Module Definition) to ES6 style, also known as ESM (ECMAScript Module).

Transformations

Exporting Modules

define(function () {
    return function () {};
});

export default function () {}

Importing Modules

define(['module-name'], function (module) {});

import module from 'module-name';

A common pattern in RequireJS is “Simplified CommonJS Wrapping:”

define(function(require) {
  var module = require('module-name');
});

In this case running a CommonJS to ES6 converter is also necessary.

Dynamic Imports

Webpack documentation on dynamic imports is a must read. A gotcha is the value returned by the import is a namespace object which might not contain what you expect. So check Webpack is exporting your modules properly by checking: __esModule. Finally, make sure to add magic comments such as webpackChunkName. Using a simple babel plugin to automatically generate the comments proved to be very helpful.

For dynamic AMD ‘requires’ that contain multiple modules:

require([
  'module-name-1',
  'module-name-2'
], function (module1, module2) {});

Using Promise.all works well:

Promise.all([
  import('module-name-1'),
  import('module-name-2')
]).then(([module1, module2]) => {});

If the import needs an expresion, make sure to read dynamic expressions in import.

import('./folder/module-name-' + dynamic).then((module) => {});

Tools

Keeping in mind that it’s a one time transformation and bulk of the work is straightforward, the most useful tools were:

5 to 6

A good collection of jscodeshift scripts which include ES6 conversion for AMD and CommonJS. Overall it did 90% percent of the job!

Good ol’ Regex

At times when other tools were too overcomplicated or simply failed, creating a regex script proved to be the fastest solution.

For example to make the following transformation:

  import serializer from 'src/utils/serializer';
+ import dates from 'src/modules/dates';
+ import attributes from 'src/modules/attributes';
+ import createdBy from 'src/modules/createdBy';
  export default Marionette.View.extend({
      innerViews: {
-       dates: require('src/modules/dates'),
-       attributes: require('src/modules/attributes'),
-       createdBy: require('src/modules/createdBy')
+       dates,
+       attributes,
+       createdBy
      }

I created this simple Python script:

import glob
import re
filter_regex = re.compile(r"(\w+)\s*:\s*(require\('.*'\))")
import_regex = re.compile(r"^\s*import")
require_regex = re.compile(r"require\(('.*')\)")
import_statement = 'import {} from {};\n'
for filename in glob.iglob('src/**/*.js', recursive=True):
    modified = False
    with open(filename) as in_file:
        text = in_file.readlines()
        modified_file = []
        for line_num, line in enumerate(text):
            if import_regex.search(line):
                last_import_line = line_num
            else:
                require = filter_regex.search(line)
                if require:
                    import_name = require[1]
                    import_file = require_regex.search(require[2])[1]
                    insert_import = import_statement.format(import_name, import_file)
                    
                    last_import_line += 1
                    modified_file.insert(last_import_line, insert_import)
                    line = filter_regex.sub(r"\1", line, count=1)
                    modified = True
            modified_file.append(line)
    if modified:
        with open(filename, 'w') as overwrite_file:
            overwrite_file.writelines(modified_file)

Version Control

Version control (Git) was essential to easily keep track of the transformations and my process. Most importantly, it allowed me perform a quick merging strategy for any files that changed after my latest pull with master. If a file was modified after the latest pull, then during the merge operation I would ignore all previous changes and perform the transformation again cleanly.

Building Your Own

If you have the time to build the perfect converter, here are some tools to get you started:

AST Explorer

Great tool to interactively see the AST (Abstract Syntax Tree) and apply transformations. To get started select JavaScript, recast, and Transform > jscodeshift. Add support for ES6 features by changing the parser setting in recast.

Transpilers

jscodeshift — wrapper around recast.

recast — perform transformations while keeping the code’s existing formatting.

Parsers

Popular JavaScript parsers: esprima, acorn, and babel.