How to use Node.js ESM loader

With this article, you will understand Node.js loaders and how to use them.

A

What is a loader?

A loader in the context of Node.js is a mechanism that allows developers to customize the behavior of module resolution and loading. It provides hooks that can be used to intercept and modify how Node.js handles import statements and require calls. This customization can be particularly useful for tasks such as:

By using loaders, developers can tailor the module system to better suit their specific needs, making it easier to work with different file types and module formats. This can be especially beneficial during development and testing phases, where flexibility and experimentation are key.

What a loader contains?

A loader in Node.js is essentially a set of custom functions that allow you to control how modules are resolved and loaded. These functions, known as hooks, provide a way to intercept and modify the default behavior of Node.js when it processes import statements and require calls. The two main hooks that a loader contains are the resolve hook and the load hook.

The resolve Hook

The resolve hook is responsible for determining the location of a module, given a specifier. A specifier is the string or URL that you pass to import or require. For example, when you write import 'some-module', the resolve hook helps Node.js figure out where some-module is located.

Here's a breakdown of what the resolve hook does:

The resolve hook allows you to customize how Node.js finds modules. For example, you could use it to map certain specifiers to different file paths or even to URLs on the web.

The load Hook

The load hook is responsible for loading the content of a module given its resolved URL. Once the resolve hook has determined where a module is located, the load hook takes over to actually load the module's code.

Here's a breakdown of what the load hook does:

The load hook allows you to customize how Node.js loads modules. For example, you could use it to transpile code from one language to another (e.g., CoffeeScript to JavaScript) or to load modules from non-standard sources.

Example of a Loader

Let's look at a simple example of a loader that handles .coffee files by transpiling them to JavaScript:

jscoffee-loader.mjs 
import { readFile } from 'node:fs/promises';
import { dirname, extname, resolve as resolvePath } from 'node:path';
import { cwd } from 'node:process';
import { fileURLToPath, pathToFileURL } from 'node:url';
import coffeescript from 'coffeescript';
import { findPackageJSON } from 'node:module';

const extensionsRegex = /\.(coffee|litcoffee|coffee\.md)$/;

export async function load(url, context, nextLoad) {
  if (extensionsRegex.test(url)) {
    const format = await getPackageType(url);
    const { source: rawSource } = await nextLoad(url, { ...context, format });
    const transformedSource = coffeescript.compile(rawSource.toString(), url);

    return {
      format,
      shortCircuit: true,
      source: transformedSource,
    };
  }

  return nextLoad(url);
}

async function getPackageType(url) {
  const packagePath = findPackageJSON('..', import.meta.url);
  if (packagePath) {
    const type = await readFile(packagePath, { encoding: 'utf8' })
      .then((filestring) => JSON.parse(filestring).type)
      .catch((err) => {
        if (err?.code !== 'ENOENT') console.error(err);
      });
    return type;
  }
  return 'module'; // Default to 'module' if no package.json is found
}

In this example, the load hook checks if the file has a .coffee extension and, if so, transpiles the CoffeeScript code to JavaScript before returning it. The resolve hook is not shown here, but it would be responsible for resolving the specifier to the correct file URL.

By defining these hooks, the loader can customize how Node.js handles different types of modules, making it a powerful tool for developers. This customization can be particularly useful for tasks such as:

How to use Node.js ESM loader

Using the Node.js ESM loader involves creating custom hooks that intercept and modify the default behavior of module resolution and loading. These hooks allow you to handle non-JavaScript files, transpile code on-the-fly, and implement custom module formats. Here's a step-by-step guide on how to use the Node.js ESM loader:

Step 1: Create the Loader Module

First, create the loader that handles .coffee files by transpiling them to JavaScript:

jscoffee-loader.mjs 
import { readFile } from 'node:fs/promises';
import { dirname, extname, resolve as resolvePath } from 'node:path';
import { cwd } from 'node:process';
import { fileURLToPath, pathToFileURL } from 'node:url';
import coffeescript from 'coffeescript';
import { findPackageJSON } from 'node:module';

const extensionsRegex = /\.(coffee|litcoffee|coffee\.md)$/;

export async function resolve(specifier, context, nextResolve) {
  const resolvedUrl = new URL(specifier, context.parentURL).href;
  if (! extensionsRegex.test(resolvedUrl)) return nextResolve(specifier, context);
  
  const format = await getPackageType(resolvedUrl);

  return {
    url: resolvedUrl,
    format,
    shortCircuit: true,
  };
}

export async function load(url, context, nextLoad) {
  if (extensionsRegex.test(url)) {
    const { source: rawSource } = await nextLoad(url, context);
    const transformedSource = coffeescript.compile(rawSource.toString(), url);

    return {
      format: context.format,
      shortCircuit: true,
      source: transformedSource,
    };
  }

  return nextLoad(url);
}

async function getPackageType(url) {
  const packagePath = findPackageJSON('..', import.meta.url);
  if (packagePath) {
    const type = await readFile(packagePath, { encoding: 'utf8' })
      .then((filestring) => JSON.parse(filestring).type)
      .catch((err) => {
        if (err?.code !== 'ENOENT') console.error(err);
      });
    return type;
  }
  return 'module'; // Default to 'module' if no package.json is found
}

Step 2: Create the Registration Module

Next, create a separate registration module that registers the loader:

jsregister-coffee-loader.mjs 
import { register } from 'node:module';
import { pathToFileURL } from 'node:url';

register(pathToFileURL(new URL('./coffee-loader.mjs', import.meta.url)));

Step 3: Use the Registration Module with --import

Now, you can use the --import flag to import the registration module when running your Node.js application:

bash
node --import ./register-coffee-loader.mjs ./main.coffee

Step 4: Create the Main Application File

Create the main application file that imports .coffee files:

coffeemain.coffee 
import { scream } from './scream.coffee'
console.log scream 'hello, world'

import { version } from 'node:process'
console.log "Brought to you by Node.js version #{version}"
coffeescream.coffee 
export scream = (str) -> str.toUpperCase()

Step 5: Run the Application

Finally, run the application with the registered loader:

bash
node --import ./register-coffee-loader.mjs ./main.coffee

Conclusion

Using the Node.js ESM loader involves creating custom hooks that intercept and modify the default behavior of module resolution and loading. By registering these hooks, you can handle non-JavaScript files, transpile code on-the-fly, and implement custom module formats. This customization can be particularly useful for tasks such as loading TypeScript or CoffeeScript files, testing React components, and more.

By following these steps, you can leverage the power of the Node.js ESM loader to tailor the module system to better suit your specific needs, making it easier to work with different file types and module formats.

What can a loader do?

A loader in Node.js can significantly enhance the capabilities of the module system by allowing developers to customize how modules are resolved and loaded. This customization can be particularly useful for a variety of tasks, including:

1. Loading Non-JavaScript Files

One of the most powerful features of a loader is the ability to handle non-JavaScript files. For example, you can create a loader that transpiles TypeScript, CoffeeScript, or other languages to JavaScript on-the-fly. This means you can write your code in a language that is more convenient or expressive for you, and the loader will take care of converting it to JavaScript before it is executed by Node.js.

2. Transpiling Code on-the-fly

Loaders can be used to transpile code from one format to another. This is particularly useful during development and testing phases. For instance, you can use a loader to transpile modern JavaScript (ES6+) code to a version that is compatible with older environments. This allows you to use the latest features of the language without worrying about compatibility issues.

3. Implementing Custom Module Formats

Loaders can also be used to implement custom module formats. For example, you might want to create a module format that supports additional metadata or a different syntax. By defining custom hooks, you can control how these modules are resolved and loaded, allowing you to extend the capabilities of the Node.js module system.

4. Overriding or Extending Default Module Resolution Logic

Sometimes, the default module resolution logic provided by Node.js may not be sufficient for your needs. Loaders allow you to override or extend this logic to better suit your specific requirements. For example, you can create a loader that resolves modules from a remote server or a custom directory structure.

5. Handling Import Maps

Loaders can be used to implement import maps, which allow you to define custom mappings for module specifiers. This can be useful for overriding the default behavior of import statements and require calls, making it easier to manage dependencies and module versions.

6. Enabling Hot Module Replacement (HMR)

In development environments, loaders can be used to enable hot module replacement (HMR). This allows you to make changes to your code and see the results immediately without having to restart the application. This can significantly speed up the development process and improve productivity.

7. Integrating with Build Tools

Loaders can be integrated with build tools like Webpack, Rollup, or Parcel. This allows you to use the same custom module resolution and loading logic in both your development environment and your production build. This can help ensure consistency and reduce the risk of errors.

Example: Transpiling CoffeeScript

Here's an example of a loader that transpiles CoffeeScript files to JavaScript:

jscoffee-loader.mjs 
import { readFile } from 'node:fs/promises';
import { dirname, extname, resolve as resolvePath } from 'node:path';
import { cwd } from 'node:process';
import { fileURLToPath, pathToFileURL } from 'node:url';
import coffeescript from 'coffeescript';
import { findPackageJSON } from 'node:module';

const extensionsRegex = /\.(coffee|litcoffee|coffee\.md)$/;

export async function load(url, context, nextLoad) {
  if (extensionsRegex.test(url)) {
    const format = await getPackageType(url);
    const { source: rawSource } = await nextLoad(url, { ...context, format });
    const transformedSource = coffeescript.compile(rawSource.toString(), url);

    return {
      format,
      shortCircuit: true,
      source: transformedSource,
    };
  }

  return nextLoad(url);
}

async function getPackageType(url) {
  const packagePath = findPackageJSON('..', import.meta.url);
  if (packagePath) {
    const type = await readFile(packagePath, { encoding: 'utf8' })
      .then((filestring) => JSON.parse(filestring).type)
      .catch((err) => {
        if (err?.code !== 'ENOENT') console.error(err);
      });
    return type;
  }
  return 'module'; // Default to 'module' if no package.json is found
}

In this example, the load hook checks if the file has a .coffee extension and, if so, transpiles the CoffeeScript code to JavaScript before returning it. This allows you to write your code in CoffeeScript and have it automatically converted to JavaScript when it is loaded by Node.js.

Conclusion

A loader in Node.js can do a lot to enhance the capabilities of the module system. By defining custom hooks, you can handle non-JavaScript files, transpile code on-the-fly, implement custom module formats, override or extend the default module resolution logic, handle import maps, enable hot module replacement, and integrate with build tools. This customization can be particularly useful for tasks such as loading TypeScript or CoffeeScript files, testing React components, and more. By leveraging the power of loaders, you can tailor the module system to better suit your specific needs, making it easier to work with different file types and module formats.