Another cyber ‘wake-up call.’ Revolution or just a rerun?

Welcome to the cyber wake-up call of the month. Maybe of the year. After all, it prompted a White House “summit”earlier this month. Just don’t hold your breath waiting for it to be a defining turning point in software security, as some experts were saying a couple of weeks ago. You’re likely to faint long before that happens.

That’s right, the discovery in December of a catastrophic vulnerability in the Apache open source logging library Log4j was labeled a “wake-up call.”

If only. But to make such a declaration implies that this is something new and shocking. It’s not — not even close. It’s just the latest event to get the label. It also implies that a sleeping software security giant has been awakened and will prevent anything like it in the future. Which, if history is any guide, won’t happen.

Check out the commentary around just about any major cyberattack exploiting software vulnerabilities in the past decade or so. They were all described as wake-up calls. There’s the Heartland Payment System breaches in 2008 and 2015, Yahoo in 2013, Target in 2013, Marriott International in 2014, Uber in 2016, Equifax in 2017, Capital One in 2019, the SolarWinds supply chain attack in 2020, the Colonial Pipeline ransomware attack in 2021, and any number of ransomware attacks by the Russian gang REvil, including on JBS Foods in 2021. That list, by the way, could go on — and on.

Added to those events, in 2012 then-defense secretary Leon Panetta said what other less-famous experts had been saying since the 1990s — that weak cybersecurity in the U.S. made it vulnerable to a “cyber Pearl Harbor” attack that could take down essential services like the power grid and the fuel supply. Nine years later we had an example of that with Colonial Pipeline. Fortunately, it was only regional.

Similarly, warnings and exhortations to take open source software security seriously have been issued by experts for decades.

Open source software, created and maintained by volunteer communities, is just as good as any other software, but also just as bad, since it’s created by imperfect humans. It’s popular — to the point of being the large majority of every modern codebase — in large measure because it’s usually free.

What’s more dangerous about it — as is said endlessly at security conferences — is that patches created for vulnerabilities don’t get “pushed” out to everybody using a vulnerable component. Those users have to “pull” the patch. And if they don’t know they’re using the component that has that vulnerability (which is frequently the case), they won’t know they need the patch. As the software cliché goes, “you can’t protect what you don’t know you have.”

A very late start

But after all that, here we are in 2022 with top voices in the information technology industry declaring that the Log4j vulnerability, called Log4Shell, is a wake-up call.

Mike Hanley, chief security officer at the open source code repository GitHub, told the Washington Post after the Jan. 13 White House summit on how to improve open source software security that it was “the start of a conversation” about the problem. Which sounds good, except it’s a very late start. The problem has been a problem for decades.

Sammy Migues, principal scientist within the Synopsys Software Integrity Group, said one reason multiple wake-up calls haven’t prompted any broad awakening in either the private or public sector could be the short shelf life of many chief security officers, who are generally the ones to get fired after a security failure no matter who or what may have caused it.

That means “every biennial catastrophe is indeed a wake-up call because they’ve never lived through one before,” he said. “Also, there’s nothing else a CEO can say in such a venue — anything else would be admitting they knew it could happen and did nothing or that they did something and it didn’t work. Neither is a good look.”

“Each event must be portrayed as a black swan — an intern did it, our forensics company says our breach is unlike anything ever seen before, etc. — or it’s an admission that ‘we tried and failed,’” he said.

Indeed, Tim Mackey, principal security strategist within the Synopsys Cybersecurity Research Center, said there are government incentives for that kind of reporting. “SEC [Securities and Exchange Commission] Guidance 2018–22 effectively incentivizes C-suite folks to portray everything as a black swan event,” he said.

Debrup Ghosh, senior product manager within the Synopsys Software Integrity Group, said another reason may be the short attention span of everybody — government, industry, and the public. “Every year we have a wake-up call, and after the initial press coverage recedes, companies go back to their existing practices until the next wake up-call,” he said.

Rhetorical reasons for optimism

So, with all that history, are there any reasons to be optimistic — to hope that this wake-up call will have more of an awakening than those of the past?

Perhaps, if we can believe the rhetoric from top government officials and executives of some of the biggest brands in tech who were present at the White House summit.

The focus of the meeting, according to a White House press release, was “to discuss initiatives to improve the security of open source software and ways new collaboration could rapidly drive improvements.”

The Washington Post reported that a major priority discussed at the gathering was “building a government/industry partnership to create a catalogue of the most important pieces of open source software that could spark Log4j-level concerns if they were vulnerable and must be checked and rechecked for hackable bugs.”

Other priorities include:

• Increasing cybersecurity training for open source software developers.
• Building closer connections between private- and public-sector efforts to shore up open source software, as outlined in President Biden’s May 12 “Executive Order on Improving the Nation’s Cybersecurity.” The Biden order explicitly calls for organizations, both public and private, to create and maintain a software Bill of Materials (SBOM) — an inventory of every software component they are using, and something the security industry has been recommending for years. In the case of Log4Shell, organizations with an SBOM would have known quickly, with a simple database search, if they were using Log4j and needed to apply the available patch.
• Increased funding and resources to existing projects to reduce the prevalence of hackable bugs in open source software.

And, demonstrating that the Biden administration is trying to put money and staff where its mouth is, Chris Inglis, head of the newly created Office of the National Cyber Director within the White House, told Politico last week that he’s focused on the security of open source software and making the general public more cyber literate. He has a $21 million budget and his staff, now at 25, is expected to reach 75 people by the end of the year.

If nothing else, those all signify an awareness that vulnerable software is both a business risk and a national security risk that deserves significant time and money to address. And nobody would quarrel with the intent, or the potential, of such initiatives.

More theater than substance

It’s just that, as any software expert could tell you, they’ve seen this movie before. And each time it has, ultimately, been more theater than substance.

Indeed, every president starting with Bill Clinton has issued executive orders (EO) on cybersecurity that, had they been followed, would almost surely have made software a more difficult target for hackers. They might have made Log4Shell much less damaging. But they largely weren’t followed, and the world of software security has changed only incrementally.

Migues, who said most presidential EOs amount to “unfunded mandates,” thinks history will repeat itself again. “It won’t be any different this time,” he said. “Some organizations will improve because of individual leaders, some because of organizational culture, and the rest will muddle through until the next event gives them a chance to fire some dead weight and bring in new people who will make a few improvements.”

That doesn’t mean an EO has no value at all. Migues said he thinks the part of Biden’s EO that could have a longer-term beneficial impact is the directive that federal agencies buy only digital products that meet rigorous security standards.

“It’ll be like an ice shelf falling into the ocean and raising the tide everywhere over time,” he said.

Mackey said the major priority at the White House summit — creating a “catalogue of the most important pieces of open source software” — is a project that has been under way for years.

“Synopsys has been assisting the OpenSSF [Open Source Security Foundation] since before the OpenSSF was created (in 2020) to identify the most important pieces of open source,” he said.

Focus on fundamentals

But multiple experts tend to agree with what has been said following the splashy rollout of previous presidential executive orders or high-profile summit meetings: The best way to improve cybersecurity is to focus more on fundamental than transformational things.

Travis Biehn, technical strategist within the Synopsys Software Integrity Group, said “organizations should spend more time reducing MTTR [mean time to recovery] from cyberattacks than worrying about whether an EO or set of tech mega corporations will ever keep open source software from having exploitable defects.”

Creating and maintaining SBOMs is still a very good idea—a fundamental necessity. But Biehn said that “as Log4j has proven, surprise is what organizations have trouble with.”

Any catalogue of open source components “will likely never anticipate the right set of critical libraries or craft and market them to be popular with developers. Surrounding existing libraries with ‘defect discovery’ is laudable, but again fails to account for the surprise that new types of defects introduce,” he said.