event-stream and flatmap-stream: What a New Maintainer Can Ship

Most of us treat npm install as plumbing. You add a package because it solves a problem, and from then on you’re trusting not just the code you looked at but every future release from whoever holds the publish rights. This week that trust got a very public test, because a widely used stream utility called event-stream turned out to have been carrying a malicious dependency for about eleven weeks.

I’m sticking to what npm, Inc. and the affected project have said officially.

What npm said happened

npm published Details about the event-stream incident on November 27. According to that post, npm’s security team was notified on the morning of November 26 that a malicious package had made its way into event-stream. The malicious package was version 0.1.1 of flatmap-stream, and it was added as a direct dependency of event-stream by a new maintainer on September 9, 2018, in event-stream 3.3.6.

npm describes the start of it as a social engineering attack, in which the attacker, posing as a maintainer, took over maintainership of the module. The important part for the rest of us is structural: nobody had to break into the registry. The person publishing was, as far as npm’s systems were concerned, allowed to publish.

After triaging the malware, npm says it removed flatmap-stream and event-stream@3.3.6 from the registry and took ownership of the event-stream package to prevent further abuse. npm’s advisory for flatmap-stream, created on November 26, says version 0.1.1 is considered malicious and that if you find the module in your environment, it’s best to remove it.

Who it was aimed at

npm’s post says that although event-stream is widely used, the code targeted developers at one company with a very specific development environment, and running it anywhere else has no effect. So npm’s view is that most developers would not be affected even if they had installed it.

The target was the Copay wallet application. npm describes the injected code at a high level: when a Copay developer ran one of the release build scripts, the resulting code was modified before being bundled into the app, and it was designed to harvest account details and private keys from accounts holding more than 100 Bitcoin or 1000 Bitcoin Cash. npm also says the payload was encrypted, which is worth remembering when we get to code review.

BitPay, which makes Copay, posted its own statement on November 26. It says the malicious code was deployed in versions 5.0.2 through 5.1.0 of its Copay and BitPay apps, that the BitPay app was not vulnerable to it. It tells anyone on an affected Copay version not to run or open the app, points to a 5.2.0 security update, and says users should assume private keys on affected wallets may have been compromised. If you use Copay, go by BitPay’s own guidance rather than a summary, including mine.

npm’s own advice to developers is short: run npm audit to see whether your project contains the vulnerable dependency, and if you installed the impacted event-stream version, update to a later one as soon as possible.

What your lockfile does and doesn’t protect against

“Commit your lockfile” is good advice here, but it’s worth being precise about what it buys you.

npm’s explanation of package locks (I’m reading the docs as of npm 6.4.1, the current release) lays out the problem lockfiles solve. Without one, a fresh install can pick up a newly published version of a direct dependency within its semver range, and a dependency of one of your dependencies can publish a new version that gets pulled in even if you pinned your own specifiers exactly. That second case is precisely how flatmap-stream reached people. Nobody put it in their own package.json.

A committed lockfile describes an exact tree, and the package-lock.json docs say each entry records the version, where it resolved from, and an integrity hash. Once it’s present, installs base their work on that file instead of recalculating versions from package.json. So if your lockfile was generated before September 9 and you never regenerated it, your installs kept getting the event-stream you already had.

Here’s what it doesn’t do. It doesn’t make the code in the tree trustworthy. If you ran an install or update between September 9 and November 26 that resolved to 3.3.6, your lockfile faithfully recorded the malicious version and reproduced it everywhere. The docs are also clear that commands like npm install and npm update automatically sync the lockfile when they change node_modules or your dependencies, so a routine dependency bump quietly rewrites it. And if you publish a library, it doesn’t protect your users at all: package-lock.json can’t be published and is ignored anywhere except the top level package, so their installs resolve fresh.

A lockfile freezes a decision. It doesn’t tell you whether the decision was a good one.

Review the lockfile diff, especially new names

The upside of that frozen decision is that changing it produces a diff. npm’s docs point out that lockfile diffs are human readable and will show you when transitive dependencies were updated or moved around.

My advice is to treat lockfile changes in a pull request as real code changes, not generated noise to collapse and approve. You won’t read every line of every updated package, and with an encrypted payload like the one npm describes, reading the source wouldn’t necessarily have shown you anything obvious anyway. What you can do cheaply is look for new package names. A patch release of a utility you depend on that suddenly brings in a dependency you’ve never heard of, published recently, is exactly the kind of thing worth a few minutes of looking at who published it and why. When you want to know how something got into your tree, npm ls with a package name prints the paths to it.

npm audit knows what’s been reported

npm’s audit docs describe the command as submitting a description of your dependencies to the registry and asking for a report of known vulnerabilities. Audit is how you find out quickly once an advisory exists, and npm pointed people straight to it this week. But the advisory for flatmap-stream was created on November 26, so for the roughly eleven weeks before that, audit had nothing to say about it.

It’s still worth wiring in. In npm 6.4.1 the audit setting defaults to true, so npm install already submits a report, and the config docs describe an audit-level setting, defaulting to low, that sets the minimum severity at which npm audit exits with a nonzero code. That makes it easy to fail a build on the serious stuff.

Fewer dependencies, fewer people to trust

Every package in your tree comes with whoever can publish it. The npm owner docs say npm owner ls lists all the users who can modify a package and push new versions, and that there’s only one level of access: either you can modify a package or you can’t. Anyone added as an owner can publish new versions and add other owners.

It’s worth being stingy. If a dependency does something you could write in twenty lines, that’s twenty lines you control and one fewer set of publish rights you’re trusting indefinitely.

For the packages you do keep and care about most, consider recording who owns them and checking for changes. A new name in the owner list isn’t evidence of anything bad, since handing off maintenance is how open source is supposed to keep going. It’s just a good moment to watch the next few releases more closely.

npm owner ls event-stream        # who can publish this package
npm ls flatmap-stream            # how, if at all, it got into your tree
npm audit --audit-level=high     # fail only on high or critical findings

Pinning and reviewing in the build pipeline

npm’s description of this attack is a good reason to think hardest about release builds, since that’s where the payload was designed to do its work. My advice for anything that handles money, keys, or credentials is to make the release build unable to change its own dependencies.

npm has a command for that. The npm ci docs say it requires an existing package-lock.json or npm-shrinkwrap.json, exits with an error instead of updating the lock if it doesn’t match package.json, removes any existing node_modules first, and never writes to package.json or the lockfiles. Use it in CI and for release builds, so the only way a new package reaches a release is through a lockfile change someone approved. Then keep dependency updates in their own pull requests, separate from feature work, so the reviewer is looking at the tree change and nothing else. Add an audit step, and for the handful of critical packages, compare the owner list against what you recorded.

None of this makes an incident like event-stream impossible, since a trusted publisher shipping a bad release is hard to catch by design. What these habits do is shrink the window, force the change to pass in front of a human, and make sure your release build only ships a tree someone actually looked at.