For Maximum Resiliency, Unleash Chaos Monkeys

Could the theme of this year’s RSA Conference be anything other than resiliency?

See Also: Webinar | Software Security: Prescriptive vs. Descriptive

In a world transformed by the coronavirus pandemic, the CEO of security firm RSA, which runs the eponymous conference, took to the stage virtually on Monday, delivering a prerecorded keynote speech on the theme of this year’s event.

“It’s been 15 months since we met, right before the pandemic stopped the world in its tracks, and in that time, countless doctors, nurses and other first responders have been brave enough to be the light for all of us. They’ve worked tirelessly to save lives, and we owe them our gratitude,” said Rohit Ghai, CEO of RSA.

“I also want to express my deepest thanks to each of you, our digital first responders,” he added. “When everything was upended, you made sure that kids could still log into their classrooms, researchers could collaborate on vaccines and governments could serve their citizens. Though I’d prefer to thank you all in person, virtual formats like this symbolize the year we’ve had, the adaptations we’ve had to make, and the theme of RSA Conference 2021: resilience.”

Embrace Chaos

By way of illustration and inspiration, Ghai offered three themes: chaos, counterintuitive thinking and community.

On the chaos front, Ghai highlighted how streaming service Netflix has better prepared itself for the unexpected with its Chaos Monkey program. As described by Netflix’s chaos team in 2015, “this service pseudo-randomly plucks a server from our production deployment on AWS and kills it.”

This unorthodox approach was successful at helping Netflix learn to expect the unexpected. “We need to identify weaknesses before they manifest in systemwide, aberrant behaviors,” the team has said. The approach helped Netflix maintain uptime and spawned a broader approach called “chaos engineering.”

Ghai said that “by bringing in and building in chaos, this tool accounted for a common type of failure and ensured graceful degradation and survival without any customer impact.”

Pursue Counterintuitive Thinking

On the counterintuitive thinking front, Ghai referenced a famous example from World War II, when the U.S. military worked with statistician Abraham Ward to improve aircraft survivability. When bombers returned from flying missions, the military studied them, aiming to make the aircraft more resilient by adding armor to sections with the greatest average number of bullet holes.

Planes couldn’t be covered with armor everywhere. Metal was a scarce commodity, and also added weight to the aircraft, thus making them less maneuverable and defendable.

Ward, counterintuitively, told the military that it should instead be adding armor to the places where returning aircraft hadn’t been most hit, because it implied that damage to other areas was more likely to down the aircraft.

“Today, we are confronted with a modern version of the Ward problem. There’s only so much armor to go around, and too much armor can slow us down. We are all working with limited resources. So we have to prioritize intelligently,” Ghai said.

“We have to protect the areas that represent the greatest risks, not where we see the most holes,” he said, guided, for example, by the National Institute of Standards and Technology’s cybersecurity framework.

Celebrate the Power of Community

Finally, Ghai said good resiliency means “rising up stronger when we inevitably fall.” He touched on the example of India’s Self-Employed Women’s Association, which supports nearly 2 million low-income women, and their families and communities, who have been particularly hard-hit by the pandemic and lockdowns.

“After 49 years of helping its members persevere, SEWA knows the powerful role a community plays in responding to a crisis,” he said. “When the lockdown forced everyone indoors, women from across SEWA turned to their sewing machines to make masks for government officials, hospitals and their families.”

Ghai called on members of the cybersecurity community to do the same for each other. “Take some time this week to celebrate our resilient journey,” he said. “Continue to believe in the light, even if you can’t see it sometimes. Continue to be the light.”

Telling Hard Truths

The theme of resiliency continued throughout the morning’s keynote speeches, including a focus on why organizations were so ill-prepared for the pandemic.

Jimmy Sanders, head of information security for Netflix DVD, noted that many organizations previously focused their resiliency plans on finding alternate workplaces for individuals who would be critical for business continuity. “Looking back, virtually no one anticipated things correctly. Turns out the business need was for almost everyone to be remote for a year or two, not a few folks for a month or two,” he said.

“If we’d looked back 100 years to the 1918 flu and used a spectrum of impact to help consider more of the edge cases, it might have helped,” said his co-presenter, Angela Weinman, head of global governance, risk and compliance at VMware.

For the COVID-19 pandemic, “it turns out those who could pivot fastest last year were the ones who had the broadest plans or who could mitigate by being the furthest along their digital transformation journey,” she said.

Netflix’s Sanders said one continuing challenge for organizations is not correctly evaluating risk. “This is directly linked to greater resilience since if we can’t accurately determine risks, it becomes difficult to rapidly recover from impacts,” he said.

Weinman advocated pursuing an all-hands approach to evaluating and responding to perceived risks. “The best way is not to struggle to make decisions alone,” she said. “Lay it all out. Present the spectrum views to your CISO and risk committee executives, your audit committee, the board – wherever you normally do your readouts. Let the business drive agreement of where on the spectrum predicted impact should go, just as similar dialogues drive risk posture decisions today.”

Preview the Future

One constant over the past year has been the rapid pace of change. The pandemic has driven more organizations to embrace greater digitization, and often, more cloud-based services and approaches, in a relatively short time period.

“We know that projects that used to take years are now taking weeks and months because of the sense of urgency that we’ve all been facing,” said Chuck Robbins, chairman and CEO of Cisco, in a keynote speech.

But this change also creates the potential for an increased attack surface, as new technologies have been rapidly deployed, and some have yet to be fully vetted.

“There are so many trends that are happening all around us that threat surface is only going to continue to expand,” Robbins said. “We have great new technologies like 5G and Wi-Fi 6 that are going to enable us to continue to connect more and more things around the world. We have the continued explosion of public cloud. We still have private cloud applications. We have SaaS applications. We have workers that will work from home forever, or in a hybrid model. … There is really no perimeter in the enterprise to defend anymore.”

Given these challenges, perhaps it’s not surprising that two-thirds of CIOs have told Cisco that their organizations plan to spend more of their budgets on security, Robbins said.

‘Look in Unconventional Places’

But what about finding sufficient staff to meet burgeoning cybersecurity requirements? Obviously, having sufficient numbers of employees in place is key to maintaining a resilient enterprise.

Robbins suggested that the shortfall in qualified applicants to fill cybersecurity jobs represents a failure of thinking. The industry needs to recruit more bright people, he said, but it also needs to find individuals who hail from diverse backgrounds and with different sets of knowledge, as the Cisco Talos threat intelligence group has been doing.

Talos “has hired physicists, astronomers, nuclear technicians, coffee baristas, grocery workers, translators,” he said. “We have to look in unconventional places for people with unconventional backgrounds that have the capacity and the capability to learn and then contribute constructively as we go forward.”