In this post, TikTok frontend engineer William Huang shares about "subspaces," a feature his team contributed to RushJS. Subspaces simplify version management for large organizations by allowing teams to manage versions independently while still participating in a centralized monorepo workspace. Read on to learn the story behind its development.

If you're familiar with TypeScript, you know that a project can often consume thousands of NPM packages. Their versions get tracked by a "lockfile" — a YAML file that guarantees everyone will install the exact same versions when building a given branch. Upgrading NPM packages can be notoriously painful, due to version conflicts or API breaks that may span a whole dependency graph. Even if you merely upgrade to "the latest" version of everything, it sometimes might take hours of fiddling to get a successful build. It's frustrating, but an unavoidable consequence of the crowdsourced development that made NPM such a prolific ecosystem.

Adopting a monorepo brings many projects together into an integrated workspace, which potentially magnifies these particular maintenance challenges. The usual solution is to standardize framework choices, then work to make package versions as consistent as possible across all your projects. With this approach, upgrading ReactJS might mean upgrading 100 projects at once. That's a relatively large work item, but generally cheaper in practice when measured per project.

When TikTok decided to move our frontend code into a monorepo, we really struggled with achieving consistency. Partly, that was because we had hundreds of originally isolated projects that all migrated into the monorepo within a single year. But also, our everyday engineering culture emphasizes experimentation and rapid prototyping, with new projects getting created every week. My team has plenty of people working exclusively on the build system and developer experience, but nonetheless it's been difficult to keep pace. Without being able to achieve high consistency of versions, engineers regularly encountered version conflicts.

Living with inconsistent versions

To address this, we initially experimented with a PNPM feature that gives each project its own separate lockfile. This superficially enabled each team to manage versions independently, but it greatly increased installation times. It also led to some very confusing version conflicts when there were interdependencies between projects; since each lockfile is processed independently, the package manager doesn't coordinate versions across such projects.

After that, we adopted RushJS, which enhances PNPM with more rigorous versioning capabilities. Since we were already using multiple lockfiles, we temporarily forked Rush's code to support that, with a plan to work our way back to a single centralized lockfile. We even contributed a new app, Lockfile Explorer, to help engineers visualize and troubleshoot version conflicts. However, by the time our monorepo had grown to over 1,000 projects, two things had become clear. First, consolidating libraries and versions will be a long road for our codebase. Second, although more lockfiles definitely mean more maintenance costs, dividing up this work can be more important than minimizing it.

This led us to a new feature idea, where projects would get grouped into "subspaces." Each subspace is a partition of the workspace with its own independent lockfile, along with its own copies of related Rush config files. To solve the problem of coordinating versions across lockfiles, we developed a companion feature called "pnpm-sync". Importantly, subspaces only pertain to installation. The projects themselves would still exist in a unified workspace that can be built and managed as a whole.

How many subspaces per repo? We strongly advocate for as few as necessary. For TikTok's unusual scale and velocity, it could be 50 subspaces, but for typical large monorepos we'd expect five or fewer. In short, subspaces offer a flexible balance between one and many lockfiles.

From idea to implementation

Although straightforward on a whiteboard, our subspaces design involved the JavaScript node_modules folder, whose famously intricate version algebra always seems to complicate systems that interact with it. We knew from the beginning that this would be a significant code change for the Rush tool, as well as a nontrivial migration for our internal project teams and owners. My team started by drafting a public RFC outlining our ideas. Next, I developed a standalone prototype of the subspaces feature, which focused on demonstrating the mechanics of how projects from one subspace would import projects from other subspaces.

After establishing this proof of concept, we went back and took a look at the config file designs from the perspective of an end user. The following were some key design questions:

• What settings should be applied globally for the whole monorepo versus a particular subspace?
• Should subspace config files be self-contained, or should they inherit from configurable defaults?
• How should overrides and custom dependency configurations be declared?
• How should Rush project selectors apply to subspaces?

Trunk-based development

With these questions answered, we could finally get to coding. Our changes would impact many source files, and RushJS is enterprise software with a relatively low tolerance for regressions. Thus, it seemed impractical to work in a long-lived feature branch that would constantly drift out of sync. Instead, the Rush maintainers advised us to designate our feature as "experimental," gated behind an opt-in setting. This made it realtively easy to prove that GitHub pull requests (PRs) were low risk, by showing that the new logic was unreachable for an opted-out user. In this way, our work went directly into the main branch, as a series of small easy-to-review PRs. It took about 6 months including design time.

Planning the rollout

Once the feature was "code complete," we needed a plan to migrate TikTok's 1,000-plus projects to the new subspace model with minimal impact on day-to-day development. We developed a two phase plan.

For phase 1, each project's lockfile would be converted to a virtual subspace containing only one project. This avoided having to regenerate any lockfiles, which greatly simplified testing for phase 1. Each project would also maintain the same overrides they had previously specified, as well as the same .npmrc settings and .pnpmfile.cjs configurations. In other words, the goal of phase 1 was to retire our fork of RushJS, rolling out the new subspace-enabled release, while disturbing as little as possible. Even so, we did encounter unforeseen breaks for some dark corners of the build system that lacked Continuous Integration (CI) coverage; although stressful, it was a reassuringly small percentage of the codebase.

Phase 2 of the migration began the consolidation of projects into larger subspaces, in other words eliminating lockfiles one by one. Project teams were happy to adopt subspaces, as reducing the number of lockfiles improves installation time. Often, it also reduces bundle size by eliminating accidental duplication of dependencies.

Below are some statistics from migrating a TypeScript mobile app project and its libraries into a subspace:

Lessons learned

Looking back, a key takeaway for me was how important the initial design phase of a project can be. If done optimally, it can save a lot of time during the implementation phase. For example, taking the time to prepare a standalone prototype provided an invaluable reference for understanding the linking mechanism between subspaces, which turned out to be a cornerstone of the entire feature. Also, the lengthy design discussions led to the insight that the subspaces design could be separated from the cross-subspace installation problem (the pnpm-sync feature), which simplified both work items, and allowed them to be developed by different people in parallel. This insight likely saved months of development time.

Are subspaces right for you?

Rush subspaces are a fairly unique feature, not currently implemented by any other system, as far as I'm aware. Many monorepos do very well with a single centralized lockfile, either because they are not too large, or because they are able to standardize their library choices and maintain consistent versions. If you're able to achieve that, then subspaces might not be the best choice.

Although subspaces were mainly motivated by large scale, along the way we identified two other interesting use cases. One is installation testing, a problem where a library test may fail to catch certain bugs if the test and library source files get installed from the same lockfile. The traditional solution is to publish your library package to a temporary NPM registry, a somewhat cumbersome setup. Subspaces provide a quick and easy way to realistically install test projects. The other interesting use case is a "legacy" project, where the business has no motivation to upgrade its dependencies, but nonetheless needs to continue building and releasing it. Moving such a project into a subspace will prevent its outdated versions from impacting your actively maintained projects. Neither of these two scenarios requires a particularly large team or codebase.

If you're interested, give subspaces a try! The Rush docs include a step-by-step setup guide. Also, Rush's official GitHub repo has now adopted this feature for installation testing, which provides a real world demo.