Equifax and Apache Struts: Know Your Dependencies and Patch Fast

On September 7, Equifax announced a cybersecurity incident that it said potentially affects approximately 143 million U.S. consumers, caused by criminals exploiting a U.S. website application vulnerability. Last Wednesday the company named the vulnerability, and it turned out to be a bug in Apache Struts, the Java web framework, with fixed releases that had been available since early March. I don’t want to pile on Equifax, and plenty about the incident is still unknown. But nearly every app most of us ship is mostly code we didn’t write, and this is a very public reminder to know what’s inside our apps and how fast we can update it.

For the breach itself I’m sticking to what Equifax and the Apache Software Foundation have said officially. The rest of this post is advice for the rest of us.

What Equifax has said

The September 7 press release says the unauthorized access happened from mid-May through July 2017 and that Equifax discovered it on July 29. It didn’t name the application or the vulnerability. That came on September 13, in a progress update on the company’s consumer site, equifaxsecurity2017.com, which said, “The vulnerability was Apache Struts CVE-2017-5638.”

Two days later Equifax published a longer release with details. By its account, the Security team observed suspicious network traffic tied to its U.S. online dispute portal web application on July 29 and blocked it, saw more suspicious activity on July 30, and took the application offline that day. After finding the Struts vulnerability as the initial attack vector, it patched the application before bringing it back online. Equifax now believes the unauthorized access to certain files ran from May 13 through July 30. The release also says the vulnerability was identified and disclosed by U.S. CERT in early March, that Equifax’s Security organization was aware of it at that time and took efforts to identify and patch vulnerable systems, and that its review of the facts is still ongoing. So Equifax hasn’t said why that application was still vulnerable in May, and I’m not going to guess.

What Apache said about CVE-2017-5638

The Struts security bulletin for this CVE is S2-045. It describes a possible remote code execution when performing file upload based on the Jakarta Multipart parser, rated Critical. The affected versions are Struts 2.3.5 through 2.3.31 and 2.5 through 2.5.10, and the recommendation is to upgrade to 2.3.32 or 2.5.10.1. The project’s 2017 announcements list both of those releases as generally available on March 7, and US-CERT’s notice followed on March 8, encouraging users to review the bulletin and upgrade. A second bulletin, S2-046, describes a different path to the same CVE through other request headers, with the same upgrade as the fix.

The Struts project spoke up before Equifax named anything. On September 9 the Struts PMC published a statement on the breach, signed by Vice President René Gielen, saying it wasn’t yet clear which Struts vulnerability had been used, if any. It pointed out that one online article had assumed a much newer bug, CVE-2017-9805, and that since the breach was detected in July, that would have meant a zero day. After Equifax’s September 13 update, the ASF put out a media alert saying CVE-2017-5638 was patched on March 7, the same day it was announced, and concluding that the compromise was due to Equifax’s failure to install the updates in a timely manner. That’s the ASF’s conclusion. Equifax’s own statement is that its review continues.

Fixed releases were out 67 days before the first unauthorized access Equifax describes, and six months before the announcement.

Start with the PMC’s own advice

The most useful part of the Struts statement is its general advice, which isn’t really about Struts at all. Understand which frameworks and libraries your products use and in which versions, and keep track of security announcements for them. Have a process to roll out a fixed release quickly, measured in hours or a few days rather than weeks or months. Don’t assume any supporting software is flawless. Build security layers so that breaking into the public-facing layer doesn’t open up the back end. And monitor for unusual access patterns. Here’s how I’d put that into practice.

Know what’s inside every app

You can’t patch a library you don’t know you have. For every application, keep a record of the frameworks and libraries it ships with, the exact versions, where it’s deployed, and who owns it. For a Java app your build file is a starting point, but transitive dependencies count too, and so does whatever actually ended up in the deployed WAR’s lib folder. I’d treat the deployed artifact as the source of truth, because that’s what an attacker talks to.

The apps that worry me most are the quiet ones. A portal that works fine and hasn’t been touched in two years is exactly the kind of thing that falls out of everyone’s head. Every app in the inventory needs a named owner and a build that someone can still run, or a critical fix turns into archaeology.

Advisories only help if they reach someone. Struts runs a low-volume, read-only announcements list alongside its security bulletins, and most big projects have something similar. US-CERT’s notices are worth following too. Send these to a shared channel or ticket queue rather than one person’s inbox, so a notice doesn’t sit unread while that person is on vacation.

Let a scanner check the inventory for you

OWASP Dependency-Check describes itself as a utility that attempts to detect publicly disclosed vulnerabilities in project dependencies. It tries to find a Common Platform Enumeration (CPE) identifier for each dependency and, if it finds one, reports the associated CVE entries. The current release, 2.1.1, comes as a command line tool, a Maven plugin and an Ant task, and there’s a Jenkins plugin. The Maven plugin docs warn that the first run can take 20 minutes or more while it downloads National Vulnerability Database data, and that updates only take a few seconds after that as long as it runs at least once every seven days.

The setting I’d change first is failBuildOnCVSS. It defaults to 11, and since CVSS scores only go from 0 to 10, that default means the build never fails. Setting a threshold turns the report into a gate:

<plugin>
  <groupId>org.owasp</groupId>
  <artifactId>dependency-check-maven</artifactId>
  <version>2.1.1</version>
  <configuration>
    <failBuildOnCVSS>8</failBuildOnCVSS>
  </configuration>
  <executions>
    <execution>
      <goals>
        <goal>check</goal>
      </goals>
    </execution>
  </executions>
</plugin>

Two cautions. The project’s own docs say false positives can happen because of how it identifies libraries, and it has a suppression file for them, so review what it flags and don’t read a clean report as a guarantee either. And a build-time check only runs when something gets built. Those quiet apps nobody touches need a scheduled scan against what’s deployed, because new CVEs show up for old code all the time.

A patch process with real deadlines

Decide ahead of time how fast each severity has to ship, write it down, and make the owner accountable for it. A critical remote code execution bug in an internet-facing app should land in the hours or few days the PMC suggests, not in next quarter’s maintenance window. Lower severities can wait longer, but every one should have a date.

Hitting those dates depends on work you do before any advisory shows up. If upgrading a framework by a point release means a week of manual regression testing, you’ll miss the deadline every time. Automated tests, a repeatable build and a scripted deployment turn a Struts upgrade into a routine change. Close the loop afterward by checking the version that’s actually running and rescanning, so the inventory matches reality. When a team truly can’t patch in time, the exception should have an owner, an expiry date and a compensating control.

Layers for the days in between

Even with a fast process there’s a window between an advisory and a deployed fix, and layers are what cover it. A web application firewall rule or a request filter can buy you time. The S2-045 bulletin itself offered stopgaps for teams that couldn’t upgrade right away. But S2-046 is the cautionary tale here: the same CVE turned out to be reachable through other headers, so a filter aimed at one header was never the fix. Use the filter to survive the next few days, and upgrade anyway.

Behind the web tier, assume it will eventually be breached, which is exactly what the PMC suggests. Put public-facing apps on their own network segment that can only reach the hosts and ports they actually need. Give each app a database account that can read only its own data, so one compromised app can’t browse every other app’s data. Then watch for unusual traffic and unusual volumes of data leaving. Equifax’s own timeline starts with suspicious traffic being observed, which is a reminder that detection is a layer too.

The takeaway

Based on official dates, the fix for the vulnerability Equifax named was out in March, and Equifax says the unauthorized access began in May. Equifax hasn’t said why that gap happened, but you can check whether the same gap exists in your own shop. Keep an inventory of what every app ships with, get advisories to someone who will act on them, let a scanner catch what people miss, give patches deadlines measured in days, and build layers that assume the next one will be late.