Rust Is The Future of JavaScript Infrastructure

November 11, 2021 (3y ago)

Rust is a fast, reliable, and memory-efficient programming language. It's been voted the most loved programming language six years in a row (survey). Created by Mozilla, it's now used at Facebook, Apple, Amazon, Microsoft, and Google for systems infrastructure, encryption, virtualization, and more low-level programming.

Why is Rust now being used to replace parts of the JavaScript web ecosystem like minification (Terser), transpilation (Babel), formatting (Prettier), bundling (webpack), linting (ESLint), and more?

Note: This post is also available in Chinese, Portuguese, Vietnamese, and Arabic.

What is Rust?

Rust helps developers write fast software that's memory-efficient. It's a modern replacement for languages like C++ or C with a focus on code safety and concise syntax.

Rust is quite different than JavaScript. JavaScript tries to find variables or objects not in use and automatically clears them from memory. This is called Garbage Collection. The language abstracts the developer from thinking about manual memory management.

With Rust, developers have more control over memory allocation, without it being as painful as C++.

Rust uses a relatively unique memory management approach that incorporates the idea of memory “ownership”. Basically, Rust keeps track of who can read and write to memory. It knows when the program is using memory and immediately frees the memory once it is no longer needed. It enforces memory rules at compile time, making it virtually impossible to have runtime memory bugs. You do not need to manually keep track of memory. The compiler takes care of it. – Discord

Adoption

On top of the companies mentioned above, Rust is also being used for popular open-source libraries like:

Rust has been a force multiplier for our team, and betting on Rust was one of the best decisions we made. More than performance, its ergonomics and focus on correctness has helped us tame sync's complexity. We can encode complex invariants about our system in the type system and have the compiler check them for us. – Dropbox

From JavaScript to Rust

JavaScript is the most widely used programming language, operating on every device with a web browser. Over the past ten years, a massive ecosystem has been built around JavaScript:

  • Webpack: developers wanted to bundle multiple JavaScript files into one.
  • Babel: developers wanted to write modern JavaScript while supporting older browsers.
  • Terser: developers wanted to generate the smallest possible file sizes.
  • Prettier: developers wanted an opinionated code formatter that just worked.
  • ESLint: developers wanted to find issues with their code before deploying.

Millions of lines of code have been written and even more bugs have been fixed to create the bedrock for shipping web applications of today. All of these tools are written with JavaScript or TypeScript. This has worked well, but we've reached peak optimization with JS. This has inspired a new class of tools, designed to drastically improve the performance of building for the web.

SWC

SWC, created in 2017, is an extensible Rust-based platform for the next generation of fast developer tools. It's used by tools like Next.js, Parcel, and Deno, as well as companies like Vercel, ByteDance, Tencent, Shopify, and more.

SWC can be used for compilation, minification, bundling, and more – and is designed to be extended. It's something you can call to perform code transformations (either built-in or custom). Running those transformations happens through higher-level tools like Next.js.

Deno

Deno, created in 2018, is a simple, modern, and secure runtime for JavaScript and TypeScript that uses V8 and is built with Rust. It's an attempt to replace Node.js, written by the original creators of Node.js. While it was created in 2018, it didn't hit v1.0 until May 2020.

Deno's linter, code formatter, and docs generator are built using SWC.

esbuild

esbuild, created in January 2020, is a JavaScript bundler and minifier 10-100x faster than existing tools, written in Go.

I'm trying to create a build tool that A) works well for a given sweet spot of use cases (bundling JavaScript, TypeScript, and maybe CSS) and B) resets the expectations of the community for what it means for a JavaScript build tool to be fast. Our current tools are way too slow in my opinion. – Evan, Creator of esbuild (Source)

Building JavaScript tooling with systems programming languages, like Go and Rust, was fairly niche until esbuild was released. In my opinion, esbuild sparked a wider interest in trying to make developer tools faster. Evan chose to use Go:

The Rust version probably could be made to work at an equivalent speed with enough effort. But at a high level, Go was much more enjoyable to work with. This is a side project and it has to be fun for me to work on it. – Evan, Creator of esbuild (Source)

Some argue Rust could perform better, but both could achieve Evan's original goal of influencing the community:

Even with just basic optimization, Rust was able to outperform the hyper hand-tuned Go version. This is a huge testament to how easy it is to write efficient programs with Rust compared to the deep dive we had to do with Go. – Discord

Rome

Rome, created in August 2020, is a linter, compiler, bundler, test runner, and more, for JavaScript, TypeScript, HTML, JSON, Markdown, and CSS. They aim to replace and unify the entire frontend development toolchain. It's created by Sebastian, who also created Babel.

Why rewrite everything, then?

Making the necessary modifications to Babel to allow for it to be a reliable base for other tools would have required changes to absolutely everything. The architecture is bound to the initial design choices I made in 2014 when I was learning about parsers, ASTs, and compilers. - Sebastian (Source)

Rome is currently written in TypeScript and runs on Node.js. But they're now working on rewriting in Rust using RSLint parser and their own visitor system for AST traversal.

NAPI

Rust's integration with Node.js is better than other low-level languages.

napi-rs allows you to build pre-compiled Node.js add-ons with Rust. It provides an out-of-the-box solution for cross-compilation and publishing native binaries to NPM, without needing node-gyp or postinstall scripts.

You can build a Rust module that can be called directly from Node.js, without needing to create a child process like esbuild.

Rust + WebAssembly

WebAssembly (WASM) is a portable low-level language that Rust can compile to. It runs in the browser, is interoperable with JavaScript, and is supported in all major modern browsers.

WASM is definitely a lot faster than JS, but not quite native speed. In our tests, Parcel runs 10-20x slower when compiled to WASM than with native binaries. – Devon Govett

While WASM isn't the perfect solution yet, it can help developers create extremely fast web experiences. The Rust team is committed to a high-quality and cutting-edge WASM implementation. For developers, this means you could have the performance advantages of Rust (vs. Go) while still compiling for the web (using WASM).

Some early libraries and frameworks in this space:

These Rust-based web frameworks that compile to WASM aren't trying to replace JavaScript, but work alongside it. While we aren't there yet, it's interesting to see Rust coming after the web on both sides: making existing JavaScript tooling faster and future-forward ideas for compiling to WASM.

It's Rust all the way down.

Why Not Rust?

Rust has a steep learning curve. It's a lower level of abstraction than what most web developers are used to.

Once you're on native code (through Rust, Go, Zig, or other low-level languages), the algorithms and data structures are more important than the language choice. It's not a silver bullet.

Rust makes you think about dimensions of your code that matter tremendously for systems programming. It makes you think about how memory is shared or copied. It makes you think about real but unlikely corner cases and make sure that they're handled. It helps you write code that's incredibly efficient in every possible way. – Tom MacWright (Source)

Further, Rust's usage in the web community is still niche. It hasn't reached critical adoption. Even though learning Rust for JavaScript tooling will be a barrier to entry, interestingly developers would rather have a faster tool that's harder to contribute to. Fast software wins.

Currently, it's hard to find a Rust library or framework for your favorite services (things like working with authentication, databases, payments, and more). I do think that once Rust and WASM reach critical adoption, this will resolve itself. But not yet. We need existing JavaScript tools to help us bridge the gap and incrementally adopt performance improvements.

The Future of JavaScript Tooling

I believe Rust is the future of JavaScript tooling. Next.js 12 started our transition to fully replace Babel (transpilation) and Terser (minification) with SWC and Rust. Why?

  • Extensibility: SWC can be used as a Crate inside Next.js, without having to fork the library or workaround design constraints.
  • Performance: We were able to achieve ~3x faster Fast Refresh and ~5x faster builds in Next.js by switching to SWC, with more room for optimization still in progress.
  • WebAssembly: Rust's support for WASM is essential for supporting all possible platforms and taking Next.js development everywhere.
  • Community: The Rust community and ecosystem are amazing and only growing.

It's not just Next.js adopting SWC, either:

Parcel uses SWC like a library. Before we used Babel's parser and custom transforms written in JS. Now, we use SWC's parser and custom transforms in Rust. This includes a full scope hoisting implementation, dependency collection, and more. It's similar in scope to how Deno built on top of SWC. – Devon Govett

It's early days for Rust – a few important pieces are still being figured out:

  • Plugins: Writing plugins in Rust isn't as approachable for many JavaScript developers. At the same time, exposing a plugin system in JavaScript could negate performance gains. A definitive solution hasn't emerged yet. Ideally, the future combines both JavaScript and Rust. If you want to write a plugin with JavaScript, it's possible with a tradeoff for speed. Need more performance? Use the Rust plugin API.
  • Bundling: One interesting area of development is swcpack, which is SWC's replacement for Webpack. It's still under development but could be very promising.
  • WebAssembly: As mentioned above, the prospect of writing Rust and compiling to WASM is enticing, but there's still work to be done.
💡

Update 2023: Turbopack has been released, the Rust-powered successor to Webpack.

Regardless, I'm confident Rust will continue to have a major impact on the JavaScript ecosystem for the next 1-2 years and into the future. Imagine a world where all of the build tools used in Next.js are written in Rust, giving you optimal performance. Then, Next.js could be distributed as a static binary you'd download from NPM.

That's the world I want to live (and develop) in.

Thanks to Devon Govett for reviewing an early draft of this article.