Using webpack and ES modules in Liferay DXP

This article explains how to deploy npm packages bundled with webpack that export and import stuff through the standard ECMAScript modules feature.

This is a sequel of the Moving from AMD to Browser modules in Liferay DXP blog post, so it’s recommended reading that if you haven’t done so already.

Why use webpack with ES modules?

The main goal of using webpack and ES modules together is to be able to limit the number of JavaScript files needed by an application while, at the same time, using the standard ES mechanism (import and export) to glue things together.

ℹ️ Additionally, some npm packages (a lot of them, in fact) are not suitable for development as ES modules due to the way they are written. For those packages it is mandatory to bundle them with tools like webpack if they are to be executed in a browser environment.

So for example, think of React, a widely used npm package which currently contains around 40 source JavaScript files. Suppose that you deployed it directly (without webpack transforming it into a bundle) to your server. Every time you loaded your application in a new browser, it would need to download the 40 files.

You could use HTTP/2 to avoid being hit by the browser connections limit, but even HTTP/2 implementations have their limits, too. Additionally, having each of React’s JavaScript files available as an ES module wouldn’t bring any benefit because you would never import anything from them directly, since you always import React’s API from react, not from any internal module. Thus, it makes a lot of sense bundling the whole React package into a single JavaScript file.

But then, if you bundled it, how would you import anything from it from another module that lives outside of the React package? Continue reading and we will find out how…

How to make webpack expose stuff as standard ES exports

As you probably know, webpack bundles are opaque, in the sense that a whole application is self-contained in them and nothing is exported to be consumed from outside. In other words, all your application’s JavaScript code is contained inside the JavaScript bundle file that webpack generates and stuff imported/exported (with standard ES syntax or using require and module.exports) in your application’s source code or its dependencies, is wired internally with some helpers functions that webpack puts inside the bundle.

But what if you wanted to generate two different webpack bundles that can import/export things from each other? Or what if you want to deploy your application as separate ES files but, at the same time, use things from another part of your application that has been bundled with webpack?

Cases like the ones described above make sense, for example, in projects that want to deploy their source files directly to the server but still want to use complex packages from npm.

Other projects may want to split the bundles they generate with webpack to avoid downloading a huge bundle at the start which will make the UX worse. For example, think of an application that uses 20 npm packages: it’s very likely that it doesn’t need to download the code for all of them on the first hit of the home page. It would make a lot of sense to break the 20 packages dependency graph in chunks that can be downloaded as needed.

ℹ️ Note that webpack already has functionality to split the generated bundles in chunks, so we are not proposing a new technique. It’s only a new way of implementing it using ESM and webpack together instead of using webpack’s built-in mechanism.

It turns out that you can make webpack re-export the exported symbols of your project’s webpack entry points through ES by configuring it like this:

{
  "entry": {
    "react_bundle": {
      "import": "react"
    }
  },
  "experiments": {
     "outputModule": true
  },
  "output": {
    "environment": {
      "module": true
    },
    "filename": "[name].js",
    "library": {
      "type": "module"
    },
  },
}

This configuration will generate a react_bundle.js file as usual, but instead of being opaque, it will contain some lines at the end to export react symbols, like this:

var __webpack_exports__Children = __webpack_exports__.Children;
var __webpack_exports__Component = __webpack_exports__.Component;
var __webpack_exports__Fragment = __webpack_exports__.Fragment;

[...]

var __webpack_exports__useRef = __webpack_exports__.useRef;
var __webpack_exports__useState = __webpack_exports__.useState;
var __webpack_exports__version = __webpack_exports__.version;

export {
   __webpack_exports__Children as Children,
   __webpack_exports__Component as Component,
   __webpack_exports__Fragment as Fragment,

   [...]

   __webpack_exports__useRef as useRef,
   __webpack_exports__useState as useState,
   __webpack_exports__version as version
};

What that code is doing is exporting webpack’s internal stuff to the outside world so that it can be consumed by another ES module.

Now, try it yourself and see if it works…

Did it work? Probably not 😈…

Why? Keep on reading…

Anything that can go wrong will go wrong…

If you look at the code webpack is generating you will realize that it needs to know what react is exactly exporting to re-export it in ES syntax. However, that’s not always an easy task due to the way packages make exports available in Node.js.

If you reflect about how Node.js packages assign stuff to module.exports you will soon realize that it may be impossible to know what a package is really exporting by looking at the code because you need to execute it to know what is finally put in that variable.

In the case of react this is exactly what happens and you get a bundle file without the ES module exports at the end. The situation looks bad and we are tempted to fall into despair, but…, it turns out that there’s a very simple solution to this problem. We just need to teach webpack how react exports look like. Let’s see how…

If we create a file named react_desc.js like this:

const x = require('react');

const {
    Children,
    Component,
    Fragment,

    [...]

    useRef,
    useState,
    version,
} = x;

const __esModule = true;

export {
    __esModule,
    x as default,
    Children,
    Component,
    Fragment,

    [...]

    useRef,
    useState,
    version,
};

And then change webpack’s configuration like this:

{
  "entry": {
    "react_bundle": {
      "import": "./react_desc.js"
    }
  },

  [...]

}

Everything will work like a charm, because webpack will be able to infer the things it needs to re-export.

⚠️ Note that we’ve also exported an __esModule constant that has the value true. This is to make interoperability with webpack, babel, etc. generated stuff possible.

The same goes for the re-export x as default, that makes the very same thing we are exporting available as a property named default too.

Whether you need this magic or not in your project depends on the tooling you may be using and how you wire things. There’s no silver bullet for this, sadly.

In any case, it’s not necessary that you create a JavaScript file for each package you want to bundle. Some of them are correctly processed by webpack, some others aren’t. A good strategy is trying to use the package directly in the webpack configuration then peek at the insides of the built bundle and see if it correctly exported the package symbols. If that fails, you just create the stub JavaScript file.

How to use our shinny webpacked ESM version of React

OK, so we have a bundled version of webpack that re-exports stuff through ES syntax. Now what? Use it! 🚀

Say you put the generated react_bundle.js next to your project’s index.js file and then you write something like this in it:

import React from 'react';
import ReactDOM from 'react-dom';

const Greeting = React.createElement(
    'h1',
    {className: 'greeting'},
    'Hello!',
);

const root = document.createElement('div');
ReactDOM.render(Greeting, root);
document.appendChild(root);

You could now deploy both files to your server (without any need to process index.js) and it would render your friendly UI!

⚠️ Obviously you would need to bundle react-dom as we did with react, but I’m saving you from going through the same steps again (at least in the blog 😅).

You could even reuse react_desc.js and require react-dom too, then re-export its stuff alongside react’s one. The possibilities and ways to wire your application are endless and you are in full control of it 🎉.

How to make webpack use stuff available as standard ES exports

Using the webpack bundle containing React from ES modules is great but what if we want to use it from another webpack bundle?

For example, think of a component library you pull from npm. Or maybe you want to use webpack to bundle your project files together instead of deploying them directly to your server…

For those cases, you can leverage webpack’s externals configuration option. This option allows you to teach webpack how to obtain certain packages from the outside world instead of putting them inside the bundle.

So, say you want to obtain React from the bundle we did before in your project. All you need to do is write a webpack.config.js file like this:

{
  "entry": {
    "index": {
      "import": "./index.js"
    }
  },
  "externals": {
    "react": "./react_bundle.js",
  },
  "externalsType": "module",
}

And, after running webpack, you will get a JavaScript bundle that has this code at the very top:

import * as __WEBPACK_EXTERNAL_MODULE__react_bundle_js_e794c9e5__ 
    from "./react_bundle.js";

[...]

That is the proof that webpack is pulling react from where you told it to get it in the externals configuration, instead of bundling it inside the JavaScript file.

Putting it all together

We will conclude this blog post by showing a very simple example of the things we have explained.

We will use the same file and packages we’ve been talking about and we will bundle react, react-dom and the project with webpack, creating a total of three JavaScript files to be deployed to the server.

package.json

{
  "name": "example",
  "version": "1.0.0",
  "dependencies": {
    "react": "^18.2.0",
    "react-dom": "^18.2.0",
    "webpack": "^5.88.2",
    "webpack-cli": "^5.1.4"
  }
}

webpack.config.js

module.exports = {
  "entry": {
    "index": {
      "import": "./index.js"
    },
    "react_bundle": {
       "import": "./react_desc.js"
    },
    "react_dom_bundle": {
      "import": "react-dom"
    }
  },
  "experiments": {
    "outputModule": true
  },
  "externals": {
    "react": "./react_bundle.js",
    "react-dom": "./react_dom_bundle.js",
  },
  "externalsType": "module",
  "output": {
    "environment": {
        "module": true
    },
    "filename": "[name].js",
    "library": {
        "type": "module"
    },
  },
};

index.js

import React from 'react';
import ReactDOM from 'react-dom';

const Greeting = React.createElement(
  'h1',
  {className: 'greeting'},
  'Hello!',
); 

const root = document.createElement('div');
ReactDOM.render(Greeting, root);
document.appendChild(root);

react_desc.js

const x = require('react');

const {
    Children,
    Component,
    Fragment, 

    [...] 

    useRef,
    useState,
    version,
} = x; 

const __esModule = true; 

export {
    __esModule,
    x as default,

    Children,
    Component,
    Fragment, 

    [...] 

    useRef,
    useState,
    version,
};

By using these files, webpack would create three JavaScript files in the output directory that would be glued together by means of standard import/export statements. Since all project files and packages share the same externals configuration, any point in code importing react or react-dom will pull them from the bundles instead of bundling them internally. This is also applied to npm packages so if, for example, react-dom imported react, webpack would pull it from react_bundle.js too.

Conclusion

We have shown a way to organize your application that lets you do two powerful things:

Split your JavaScript dependency graph easily and the way you want by simply defining the bundles you want to build and making them wire together to avoid duplication.
Use standard ES syntax to glue things together so that the mechanism is agnostic from the build tool you are using (in this case webpack).

Number one helps a lot with deciding the order and time when you want to load your resources at runtime, so that you don’t incur in big latencies or need a lot of bandwidth. So, for example, you can bundle the things you know will be used for sure in an initial bundle and then put related optional things that may or may not be used in their own bundles.

Also, number one allows you to build your project faster if you create different projects for each bundle. So, for example, if you are going to use react and you know react doesn’t change unless you upgrade its version, why build it each time you change your project? Simply create a bundle for it, build it once, and forget about it until you upgrade it in the future 🎉.

In fact, it turns out that someone has already done it and published it in a public CDN. See the https://esm.sh/ website for more information on this interesting project.

Finally, number 2 lets you interoperate things easily no matter what tool you use to build your project. In other words, you can make two projects cooperate even if one is built with webpack and the other one is plain JavaScript, for example.

I hope this blog post has enlightened you and opened your mind to new ways of structuring your applications. Feel free to leave any comments and/or impressions in the comments section.

Thanks for your feedback!

Bonus track

If you want to see a real example of a Liferay Workspace deploying a Vite application and two import map client extensions to publish you can have a look at this GitHub repository created by Gabriel Prates.