He helped write the book on how to improve critical infrastructure cybersecurity

Very small things can have very big consequences. That’s why the security of the software that runs much of the nation’s critical infrastructure (CI) is so important. A single vulnerability in often lengthy and labyrinthine software supply chains can lead to catastrophic damage.

Recall the panic in December 2021 after the discovery of Log4Shell, a group of vulnerabilities in the Apache Software Foundation’s logging library Log4j that put billions of users — including CI operators — at risk of hackers being able to take control of their entire network. That’s because Log4j was part of their software supply chain.

Not that the “small-defect-causes-big-damage” problem is new. The industrial revolution, never mind software, wasn’t even a gleam in anyone’s eye when U.S. founding father Ben Franklin wrote an iconic poem.

For want of a nail, the shoe was lost;

For want of the shoe, the horse was lost;

For want of the horse, the rider was lost;

For want of the rider, the battle was lost;

For want of the battle, the kingdom was lost;

And all for the want of a horseshoe nail…

But the point it made remains relevant enough that Don Davidson, director of Cyber Supply Chain Risk Management (C-SCRM) at Synopsys, used it to open his chapter of a recent book from the cybersecurity think tank Institute for Critical Infrastructure Technology (ICIT) titled “Securing the Nation’s Critical Infrastructures: A Guide for the 2021–2025 Administration.”

According to the ICIT, the book “represents the best of our community, with dozens of thought leaders coming together to share best practices on how this administration and all stakeholders can best protect our most critical assets.”

Davidson focused specifically on C-SCRM (which includes both software and hardware), and even more specifically on how that applies to information communications technology (ICT), in a chapter titled “Managing Global Supply Chains and Their Impact on U.S. Critical Infrastructure: What Do Critical Infrastructure Sectors Need to Do?”

But the principles of good cyber supply chain security apply to every business sector because today every business is a software business. Davidson noted that the “nail” could be “a hardware or software sub-component anywhere in the supply chain of the design manufacture and delivery process or the supply chain of the operations and sustainment lifecycle.” And if it’s “lost” or defective in some way, the whole chain can be compromised.

An obvious need

There is no debate that the cyber component of U.S. critical infrastructure needs better security. Besides Log4Shell, some of the more high-profile examples in recent years of the damage a lack of security can cause include the ransomware attack on Colonial Pipelinethat led to the shutdown of nearly half the fuel supply — gasoline, diesel, jet fuel and heating oil — to the East Coast for almost a week; a ransomware attack on JBS Foods, the world’s largest meat supplier; an attempt to poison the water supply in Oldsmar, Florida; and others in the transportation and insurance sectors.

But the list is much longer than that.

Joe Weiss, managing partner at Applied Control Solutions and a CI control systems expert, said shortly after the Oldsmar attack that it was no outlier. “All these people claiming this is the first time for an attack like this — that’s nuts,” he said. “It wasn’t first time, and it won’t be the last.”

In a post on his Unfettered blog, Weiss wrote that he had documented almost 100 control system water/wastewater cyber incidents throughout the U.S. and internationally. And while “not all cases could be identified as cyber-related [and] many incidents were unintentional, the impacts could be devastating,” he wrote, citing a water utility that inadvertently pumped water from a Superfund well site into a drinking water system.

So there would also be little debate over Davidson’s declaration that “the U.S. must ensure its supply chains are reliable, safe, and secure, to include the hardware, software, and services that enable all of our critical infrastructure capabilities.”

Indeed, President Biden’s May 2021 “Executive Order (EO) on Improving the Nation’s Cybersecurity” has an entire section devoted to cyber supply chain security. That EO calls for federal agencies to be banned from buying any software products that don’t come with a software Bill of Materials (SBOM), basically an inventory of software components in a product that helps users know if any of possibly thousands of components in a product’s supply chain has a vulnerability that needs to be patched.

And, as Davidson noted, that’s not the only such order. Past presidents going back to Bill Clinton have issued EOs on cybersecurity, with direct focus on CI and the supply chain.

President Barack Obama’s EO 13636, issued a decade ago, was titled “Improving Critical Infrastructure Cybersecurity.” President Donald Trump issued several, including EO 13873, “Securing the Information and Communications Technology and Services Supply Chain,” in May 2019.

In the private sector, critical infrastructure and ICT/C-SCRM are constant topics of discussion and presentation at cybersecurity conferences.

No lack of awareness and information

So a lack of awareness of the problem can’t be the problem.

Nor is a lack of information about how to mitigate the problem. As Davidson also noted, there are abundant standards and guidance available from both the public and private sectors.

The U.S. National Institute for Standards and Technology (NIST) and Cybersecurity and Infrastructure Security Agency (CISA) published “Defending Against Software Supply Chain Attacks” in April 2021, providing “an overview of software supply chain risks and recommendations on how software customers and vendors can use the NIST Cyber Supply Chain Risk Management (C-SCRM) framework and the Secure Software Development Framework (SSDF) to identify, assess, and mitigate risks,” according to NIST.

NIST also updated its Cybersecurity Supply Chain Risk Management Practices for Systems and Organizations guidance in May 2022.

Yet even with all that awareness, information, and support of presidential bully pulpits, there is still an obvious need for the recent ICIT book. And even with that book, significant improvement on a national scale could still be both tricky and difficult.

That’s in part because security in software products too often remains an afterthought, or no thought at all. Features and price are still the priorities. “Products often function ‘as intended,’ but little is done to evaluate if they function ‘only as intended,’” Davidson wrote. “Consumers often unknowingly trade off security requirements for fast acquisition of less expensive ICT products.”

That mindset doesn’t apply only to consumers. Davidson noted that it includes developers and providers who are “more focused on cost and schedule (How do I get it fast and cheap?) to gain a desired functional capability — often unknowingly trading off [the] security and sustainability of those capabilities.”

It is also in part because a software supply chain amplifies the so-called “attack surface.”

The infamous 2021 SolarWinds attack illustrated that — hackers were able to inject malicious code into an update for the company’s performance monitoring system, Orion. When users downloaded the update — as they had been told to do — it spread malware to thousands of organizations including multiple federal agencies.

As Davidson put it, vulnerabilities in the cyber supply chain mean that cybercriminals “can target government and industry via the contractors, sub-contractors, and suppliers at all tiers of the ICT supply chain.”

“Compounding the complexity of securing the supply chain is that vulnerabilities may be introduced during any phase of the product life cycle: design, development and production, distribution, acquisition and deployment, maintenance, and disposal.”

No final deadline

Then there is bureaucracy. The Biden EO has a series of deadlines for compliance with software supply chain directives, none for longer than a year — except for the final step on the SBOM requirement. The EO calls for multiple federal agencies led by the Department of Homeland Security to “recommend contract language” within one year — by May 12, 2022 — to the Federal Acquisition Regulatory (FAR) Council that would implement the ban on purchasing any software products without an SBOM.

But the Biden EO set no deadline for the FAR Council to adopt that contract language and issue a “final rule.” More than 21 months since the EO, it hasn’t happened.

Finally, it’s because C-SCRM is a complicated task. An SBOM is crucial but not sufficient. And there is no “supply chain security” button to set and forget, no digital version of a padlock to close on the cyber supply chain when you go home for the weekend.

“The ICT supply chain is a complex, globally interconnected ecosystem that encompasses the entire life cycle of ICT hardware, software, and managed services and a wide range of entities — including third-party vendors, suppliers, service providers, and contractors,” Davidson wrote.

So the bottom line, as Davidson and his coauthors note, is that CI security requires a lot of work.

“Enterprise and capability owners need to gain 100% visibility of their Tier 1 suppliers — for hardware, software, and services — then illuminate deeper tiers of their supply chains for their most critically enabling technologies, and then develop mitigation strategies for people, processes and technologies impacting their capabilities and overall enterprise,” Davidson said in an interview.

“There are no silver bullet solutions to solve C-SCRM challenges,” he added. “We have to take a risk-based approach and manage risk to our enterprises based on their unique capabilities and risk tolerance.”

And getting everybody to do that might require that recommendations become mandates — something an increasing number of experts advocate. “There are existing standards to be leveraged, but they are not comprehensive, and they are not uniformly implemented — adoption of some standardized assurance levels would be a good first step,” Davidson said.