The recent discovery of a backdoor in the data compression utility
XZ Utils, like thousands of other open source projects, is run by volunteers and, in its case, has a single maintainer managing it. Such projects often have little or no resources for managing security issues, meaning organizations use the software at their own risk. This means that security and development teams must implement open source risk management measures the same way they do with internally developed code, security experts say.
“Although it is unlikely that an organization can effectively prevent [all] exposure to supply chain risks, organizations can absolutely focus on a strategy to reduce the likelihood of a successful supply chain attack,” says Jamie Scott, founding product manager at Endor Labs.
Open source is not the same as outsourcing: “Maintainers of open source software are volunteers. At an industry level, we need to treat them as such. We own our software; we are responsible for the software we reuse.”
Well-intentioned, under-resourced
Security concerns about open source software they are not new at all. But often it takes discoveries like the Log4Shell vulnerability and the backdoor in XZ Utilis to truly understand how vulnerable organizations are to components of their code. And often, the code comes from well-intentioned but hopelessly under-resourced and minimally maintained open source projects.
XZ Utilis, for example, is essentially a one-person project. Another individual managed to do so sneak backdoor into the utility over a period of almost three years, gradually gaining enough trust from the project maintainer. If a Microsoft developer hadn’t come across this case in late March while investigating strange behavior associated with a Debian installation, the backdoor could have ended up on millions of devices around the world, including those belonging to large companies and government agencies. As it turned out, the backdoor had minimal impact because it affected versions of XZ Utils that were only present in the unstable and beta releases of Debian, Fedora, Kali, open SUSE, and Arch Linux.
The next compromise of open source code could be much worse. “The scariest part for enterprise organizations is that their applications are built on open source software projects just like XZ Utils,” says Donald Fischer, co-founder and CEO of Tidelift. “XZ Utils is a package of tens of thousands used every day by typical enterprise organizations,” he says.
Most of these organizations do not have sufficient visibility into the security and resilience of this part of the software supply chain to assess risks, he notes.
A recent one Harvard Business School One study estimated that the demand-side value of open source software is a staggering $8.8 trillion. Maintainers are at the center of this ecosystem, and many of them operate alone, Fischer says. A survey conducted by Tidelift last year found that 44% of maintainers of open source projects describe themselves as the sole maintainers of their projects. Sixty percent identified themselves as unpaid hobbyists, and the same percentage said they had left or were considering leaving their role as project maintainers. Many maintenance workers describe their efforts as stressful, lonely and financially unrewarding work, Fischer says.
“The XZ utils hack brings into stark relief the risks of underinvestment in the health and resilience of the open source software supply chain [that] that enterprise organizations rely on,” says Fischer. “Enterprise organizations need to realize that most of the most reliable open source packages are maintained by volunteers who describe themselves as unpaid hobbyists. These maintainers are not corporate vendors, but they are expected to work and deliver like them.”
Danger: transitive dependencies
A study conducted by Endor in 2022 found that 95% of open source vulnerabilities are present in so-called transitive dependencies, which are secondary open source packages or libraries on which a primary open source package may depend. Often these are packages that developers do not select directly but are automatically used by an open source package in their development project.
“For example, when you trust a Maven package, on average there are 14 other dependencies that you trust implicitly,” Scott says. “This number is even higher in some software ecosystems like NPM, where on average you import an additional 77 software components for every one you trust.”
One way to start mitigating open source risks is to pay attention to these dependencies and be selective about which projects you choose, he says.
Organizations should look at dependencies, especially smaller, one-off packages managed by teams of one or two people, he adds Dimitri Stiliadis, CTO and co-founder of Endor. They should determine whether the dependencies in their environment have adequate security controls or whether a single individual commits all the code; if they have binaries in their repositories that no one knows about; or even if someone is actively maintaining the project, Stiliadis says.
“Focus your efforts on improving response effectiveness: Fundamental controls, such as maintaining a mature software inventory, remain one of the most valuable programs you can have in place to quickly identify, scope, and respond to risks software once identified,” Scott advises.
Software composition analysis tools, vulnerability scanners, EDR/XDR systems, and SBOMs can also help organizations quickly identify vulnerable and compromised open source components.
Recognize the threat
“Mitigating exposure starts with a shared understanding and recognition by executives and even at the board level that approximately 70% of the ingredients of the average software product are open source software historically created by mostly uncompensated contributors,” he says Fischer from Tidelift.
New regulations and guidelines in the financial services industry, FDA and NIST will determine how software is developed in the years to come, and organizations need to prepare now. “The winners here will quickly adapt from a reactive strategy to a proactive strategy to manage open source risks,” he says.
Fischer recommends that organizations ask their security and engineering teams to identify how new open source components enter their environment. They should also define roles for monitoring these components and proactively remove those that do not fit the company’s risk appetite. “Reacting to late-stage issues has become an ineffective way to address the scale of risk to the business in recent years, and the The US government is reporting that era is coming to an end,” he says.