How do I know that my tomato is safe? I go to a local grocery store and find a “USDA Organic” labeled tomato. I blindly believe it and consume it. Unfortunately I get sick from eating this tomato, because somewhere in the farming or logistic process, someone maliciously injected some non-obvious poison into that tomato.
According to the USDA website, on-site inspection happens on a yearly basis to certify farmers. Well, it’s good that they have a process to certify, but it turns out it was not good enough. It is basically not good enough to certify that a particular tomato I eat is safe.
The same exact thing is happening in the software world, mainly by vulnerabilities introduced in the development cycle, which creates massive issues in Supply Chain Risk and have vast negative business outcomes.
The rationale is actually very simple. If someone wants to poison me, imagine if I have 100 security guards around my house and 1000 surveillance cameras, no one can get to my kitchen to poison my food. However, anyone can get to the farm that produces tomatoes that ships to my neighborhood grocery store. I blindly trust the “USDA Organic” sticker and I blindly buy tomatoes to consume.
Vulnerability attack is all about going after the weakest link in the ecosystem. It is as simple as that.
In December 2020, the world was shaken by the Solarwinds incident, one of the most impactful incidents in US cyber history. Their software was used by 33,000 clients of which 18,000 clients (including US agencies such as: the Department of Homeland Security, the State Department, the Department of Energy, the National Nuclear Security Administration, and the Treasury; and private giants such as: Microsoft, Cisco, Intel, and Deloitte) (source: the Wall Street Journal) were impacted as they updated the contaminated version of Solarwinds software. Malicious code opened up a backdoor for Russian hackers and enabled them to do espionage. In my analogy, I was poisoned by a tomato, my mind is now controlled by someone remote, and opened the door from inside to welcome criminals to my home. The game is simply lost at that point.
How did that happen? Adversaries found vulnerabilities in Solarwinds development environment and they successfully ingested malicious code, which got pushed out to customers software as part of a “regular update” in March 2020. 18,000 clients updated it, as they usually do, to expect feature enhancements, bug fixes, and security improvements. Well, this time it came as a poisoned tomato.
Was this an unusual attack? Very unlikely. This kind of supply chain is on the surge as there were 12,000 such incidents in the past year which is a 650% increase year over year.
Why this rapid increase? Open source has now become fundamental building blocks of software architecture, but it has at the same time brought in newer and larger risk exposures to the software development world. The report from Sonatype shows that, in 2021, developers around the world will request more than 2.2 trillion open source packages and Synposys reports “Ninety-eight percent of healthcare sector codebases contained open source, and 67% of those codebases contained vulnerabilities.” This is similar for other industry segments, so it is fair to say that most of the modern software contains or depends on open source code.
It’s a natural progression. There is no question that the world revolves around software today. When developers have to produce code and applications faster than ever, it makes sense that they want to use prebuilt libraries and code repositories which already exist, and most commonly they quickly find it in open source communities.
Should we continue to use Open Source? Yes and No. Open source does bring the best ideas from a larger community of developers, beyond what an alternative in-house engineering team would be able to do. And from that perspective, more security thinking and security fixes than what an engineering team could come up with in-house. On the other hand, and unfortunately, the codebases that contain vulnerable open source codes are in the increasing trend. It was up to a total of 84% in 2020, and 60% of them were actually labeled as high-risk vulnerabilities.
Another complication of late is the “shift left” move in the cyber security space. In the last decades, security-testing software has shifted to the earlier in the development cycle in order to secure applications more efficiently. By doing so the security implementation and quality assurance has shifted from an AppSec team to the DevOps team, i.e. DevSecOps. However, security implementation is a new role for DevOps teams. It could potentially slow down their main tasks. DevOps teams nowadays have lots of different security testing tools, such as SAST (Static Application Security Testing), DAST (Dynamic Application Security), IAST (Interactive Application Security Testing) and others. Many of them were built years ago which do not fit with DevOps current high release cycle pressure. Majority of them are point solutions and do not assure the overall integrity of software security, so DevOps are additionally left with insufficient tools for their new responsibilities.
The Baiden administration issued the executive order in May 2021 to enhance the security implementation which contains SBOM (software build of material) adoption. This is to suggest that software should be not only secured but revealed how it is built and what it is built of. June 2021, Google published the new security framework, SLSA (pronounced “salsa”), to share their new best practices to ensure the end to end security integrity to prevent supply chain attacks. Together with additional framework layers of In-Toto, TUF and SigStore, now we are talking! In addition to leveraging application security test tools in the DevOps cycle, they can start to detect and respond to vulnerabilities earlier, now with this kind of security framework we are stepping forward to reduce supply-chain attacks.
In conclusion, it’s heading in the right direction: this is no longer about trusting the “organic” USDA label, but ensuring and exposing how that particular tomato is safely built. The day that I can eat tomatoes safely is getting closer! 🙂
Hiro Rio Maeda is a Managing Partner at DNX Ventures. DNX Ventures has invested in companies like Cylance (NYSE: BB), JASK, Bridgecrew, Mitiga, Appdome, NeuraLegion, SOC Prime, Safebreach, and Netrise and CloudNatix to fuel better pathways for when incidents happen and solutions for the growing enterprise security needs ahead with the growing dependency on SaaS. Feel free to contact DNX if you want to learn more.
