Andrew Ginter, VP Industrial Security at Waterfall Security Solutions, joined Carahsoft to discuss Critical (Industrial) Infrastructures, Recent Attacks, Near-Term Projections, and How to Prepare.
SUBSCRIBE to get the latest tech tips & tricks from industry leaders!
https://www.youtube.com/user/carahtechtv
FOLLOW US ON TWITTER:
https://twitter.com/Carahsoft
CONNECT WITH US ON LINKEDIN:
https://www.linkedin.com/company/cara...
LIKE US ON FACEBOOK:
https://www.facebook.com/carahsoft
LISTEN TO US ON OUR CARAHCAST CHANNEL:
https://www.carahsoft.com/carahcast
READ THE LATEST GOVERNMENT TECH COMMUNITY TRENDS:
https://www.carahsoft.com/community
Intro 00:14
On behalf of Waterfall Security Solutions and Carahsoft, we would like to welcome you to today's podcast, focused around secure operations technology, where Andrew Ginter VP of Industrial Security at Waterfall Security Solutions will discuss critical Industrial Infrastructures, Recent Attacks, Near-Term Projections, and How to Prepare.
Tatsiana Zherdetski 00:36
I'm Tatsiana Zherdetski, a marketing coordinator with Carahsoft, working under the Cybersecurity Core Cyber Team. I'm here with Andrew Ginter from Waterfall. Would you like to say a few words before we begin?
Andrew Ginter 00:48
Firstly, hello, Tatsiana, thank you for having me. A few words about me and my role at Waterfall. You know, I am a techie I did you know 10-15 years developing industrial control system products, you know, our stuff automate some of still automates, you know, some of the biggest physical processes in the world, pipelines, power plants. I spent a half decade developing IT OT middleware, these are products that connect control systems, in power plants, in pipelines in rail systems, connect them out to business networks to sap the enterprise resource planning system back in the day. And, you know, thereby, we connected a lot of control system networks, to business networks, thereby sort of contributing to the IT OT cybersecurity challenges that that we all face now. In a sense, I got religion, I wound up the Chief Technology Officer at Industrial Defender building the world's first industrial SEM, a security information and event management system, customized for industrial control systems, and now I'm at Waterfall Security, you know, where I lead a team of experts that works with the world's most secure industrial sites. You know, Waterfall is the the OT security company; we're a technology company. And I'll say more about the technology as we go along. But that's sort of the, you know, me in a nutshell.
Tatsiana Zherdetski 02:16
So our topic for today is "secure operations technology". What is that?
Andrew Ginter 02:21
Well, it's the name of my most recent book, so I'll talk about that in a minute. But really, let's talk about operations technology for a second, you know, versus information technology. Operation technology, OT, is all about physical operations. In the simplest of which, you know, most businesses have offices, building automation is a physical system, building automation is IoT. But we're also talking about, you know, the computers that control hydroelectric dams that control passenger metros that control power plants that control, you know, food and beverage manufacturing facilities. And, you know, when we, when I talk about security, where I'm talking about cybersecurity, you know, so the, the opposite of OT, is it, you know, what is it is business automation, where, in a sense, the worst case consequence, if you compromise a business network, the worst case consequence is a business consequence. A defaced website, leaked customer data, have to buy insurance for all of the customers for their leaking their personally identifiable information out into the world. In the OT space, the worst case consequences are physical. The simplest physical consequences are shutdowns, like the Colonial Pipeline incident that we saw less than 12 months ago, the worst case consequences there aren't, you know, there are much worse consequences that are possible, you know, we can be talking about damage to very large, very expensive equipment, if you if you cause a $300 million, you know, 300 ton turbine to malfunction, it's going to be months before it's producing power again. You know, it's even possible to imagine, worst case mis-operation of automation for physical systems, you know, worst cases are threats to worker safety threats to public safety, think of a water treatment system that has accelerated the flow of water through the system, and now untreated water is being pushed out into the into the drinking system, very serious consequences are possible here. So, you know, the IT world & the OT world, they use a lot of the same computers, but in a sense, they're different animals because of the different kinds of consequences.
Tatsiana Zherdetski 04:37
Okay, so when we talk about OT security, how are we doing? What is the state of the world?
Andrew Ginter 04:43
Well, the state of the world, it depends who you ask, you know, a lot of IT people who are not familiar with OT, and there's 50 times as many IT people in the world as our OT people, OT security people. So, you know, a lot of times when we ask for advice from an IT security expert, they're going to come in they'll I've seen OT systems for the first time. And bluntly, there'll be appalled. What do you mean, you don't have antivirus installed? What do you mean, you're still using Windows 7? Is that an XP machine that I see over there? You know, there's a disconnect. People are, you know, IT people are sometimes deeply surprised by what they see in the OT space. In the OT space, you know, in the worst case, a lot of engineers are focused on efficiency. They're focused on reliability, they're focused on safety. In a sense, they're blind to cybersecurity, they look around and say, cyber. What's the problem here? Engineering is very data driven. I mean, you know, they look around and say, How many safety incidents have we had in this last year? There have been none. And we're proud of that record. You know, these folks are very safety focused. How many times has this site been shut down? Because of a cyber-attack? None. So--what's the problem? Why do we need to do anything? And risk managers look at this, so this sort of disconnect, and they really don't know what to make of it.
Tatsiana Zherdetski 06:08
So begs the question, what should we make of it?
Andrew Ginter 06:11
Well, let's dig into the problem a bit. Why are things you know, in a sense the way they are, one of the big reasons is something called Engineering Change Control, you know, the engineering discipline is, is again, focused on safety on reliability. Every change to an automation system, every change to a physical system is a potential threat to safe, reliable, and efficient operations. And so engineers are trained to manage those risks, in large part by controlling change. Changes as small as you know, a new a new antivirus signature set could be a threat. If the signature said malfunctions and quarantines half of the control system, the lights go out, or worse. And that's a very simple change, a much more complex change is something like a security update. What code has changed? We don't know, the vendor didn't really tell us. Even if the vendor told us Do we have the source code? No. Even if we had the source code, could we review it? No, it's too complicated. It's difficult to predict, therefore, you know, the consequences of making these changes. And so change is slow. We test the security updates for a long time, we're talking months, sometimes longer than that, before we trust them enough to install them on these mission critical computers. But, you know, so on the one hand, it's hard to do certain kinds of security, on OT systems. That's why you still see XP systems sitting around. But on the other hand, just because it's hard to do security doesn't mean you can throw your hands up and say, well, then I guess we won't do anything. Because the threat environment keeps getting worse. You know, look at the Gartner Group, they work with a lot of industrial enterprises, they look at sort of trends that they see in those enterprises, they have access to a lot of data, the Gartner Group is predicting that within three years, 1/3 of industrial enterprises will suffer a production shutdown because of some sort of cyber-attack. So, you know, it's a problem we need, you know, doing things sort of the normal way can be difficult. But we have to do something.
Tatsiana Zherdetski 08:26
And so do you agree, as you and Waterfall are active in this space, too?
Andrew Ginter 08:30
Well, yeah. I mean, I agree with Gartner. And here's why. I mean, part of my job is, is to study cyber incidents that have physical consequences. You know, and what I've observed is that, you know, sort of the pervasive threat today is ransomware, fairly sophisticated, ransomware groups. Here's the thing I've observed these groups, today's ransomware groups are using the tools and techniques that only about five years ago, were used exclusively by nation states. The ransomware groups are running about five years behind the nation states technology wise. And, you know, a lot of people I hear a lot of people even today saying, yes, nation state attacks are very, very sophisticated. But, you know, am I really important enough? Am I big enough to be a nation state target? Well, ransomware doesn't ask if you're big enough to be a nation state target. Ransomware asks, Do you have any money? And, you know, we all have money. We're all targets. You know, what we see the nation states doing today, we should expect ransomware to be doing to all of us five years from now. So yeah, we do have to pay attention to the most sophisticated of attacks. And we have to prepare for those attacks today, because they're coming out of Slimer. I very much agree with, you know, what the gardener group was predicting. In fact, there's a conflict in the Ukraine right now that could escalate into a serious physical conflict. I'm sorry, when in the modern world when you have serious physical conflict conflicts, especially when they involve cyber superpowers, you know, the United States is an ally of the Ukraine, you know, Russia is is the one that's got the conflict with Ukraine. Both of these nations are cyber superpowers. They have very sophisticated attack capabilities. And if a physical conflict breaks out, I think we're very likely to see those very sophisticated attacks, targeting critical infrastructures, in the participants in the conflict and in their allies. So it's a difficult time we're in right now, we cannot be ignoring the most sophisticated attacks out there, we have to start getting ready for these things.
Tatsiana Zherdetski 10:48
And then what should we do about all of this to get ready?
Andrew Ginter 10:51
Well, that comes back to secure operations, technology security for short. What I did in my most recent book was I, you know, I documented what the world's most secure industrial sites, what they do differently, cybersecurity wise. You know, a lot of a lot of industrial sites, a lot of it sites. You know, the focus of cybersecurity conventionally is all about protecting the information, protecting the confidentiality, the integrity and the availability of the information. But, you know, I see industrial sites, the world's most secure industrial sites, I see them asking different questions. They get different answers. They don't ask the question, how do I protect the information, they observe that all cyber sabotage attacks our information. That's what cyber sabotage means. The only way for an industrial control system, automating a physical process, the only way for it to change from an uncompromised state to a compromised state is for attacking formation to enter the system. And so what I see Seco tea sites doing is they ask the question, how do I protect my industrial operations, from information more specifically from a tax that may be embedded in information flows? And, you know, when you when you when you ask that question, when you when you look at the problem that way, you know, different kinds of answers become, you know, sort of become clearer.
Tatsiana Zherdetski 12:23
Okay, well, don't leave us hanging, if that's the question, what's the answer?
Andrew Ginter 12:27
Well, I do document it all in my latest book. You know, cyber sabotage attacks our information. And so one of the early steps that secure sites do in the, in the course of designing their security system, is they carry out a detailed inventory of information flows, they look at all of the different ways that information can enter an industrial system. And, you know, they do this because a comprehensive inventory of incoming information flows is also a comprehensive inventory of all possible cyber sabotage attack vectors. And with that inventory in hand, you know, they said about controlling those flows, what are those flows look like? Well, you know, nine times out of 10, the sort of biggest the flows of greatest concern are USB drives being carried around with information on them potentially with attacks on laptops being carried around connected one minute, literally, out to Google on the internet, and the next minute plugged into the the industrial control network and potentially transmitting, you know, attacks through the laptop, you know, into the control system. And the third one is, of course, the perennial firewall, you've got, you've got the possibility of online information coming through the network coming through the firewall, you know, into the control system. And, you know, when we talk about controlling those information flows, we prefer to control them, you know, these sites prefer to control them physically whenever possible. So, you know, literally glue the USB ports shut if you don't need them. You know, literally remove the CD drives from the equipment if it doesn't need a CD drive. If you do need to bring a USB drive in don't carry the USB drive in from the IT network or from the outside world. You know, carry it into, let's say a kiosk that's got you know, there's products out there with you know, four and eight and 12 antivirus scanning engines built in, scan the information, write it to a brand new USB or to a brand new CD drive, leave the potentially contaminated physical USB device behind carry clean media into the system. You know, laptops do not carry a laptop into the site, the secure sides what we see them doing is they have an inventory of laptops sitting there at security, if you need to enter the site and you need a laptop, while you're on the site, you book one of the laptops, you specify the software that you're going to need as a vendor, you're going to need some of your vendor software installed on there, you specify the software that has to be installed, you show up at security, you leave your laptop behind, you leave all your USB is behind, you pick up a brand, you know, one of the control system laptops that has never touched an IT network, it's never touched the internet, it's safe to carry around the control system. And the other thing that that you know, we see people doing, you know, that sort of touched on USBs. I touched on laptops, what about firewalls? What we see people doing is saying, Look, we have two kinds of networks, we have control critical networks, where the consequences of compromise are physical. And we have business critical networks where the consequences of compromise our business, their business impacts at the boundary between those two, they forbid firewalls, they do not connect a firewall, or anything else from one network to the other. The only connection they permit is a unidirectional security gateway. This is hardware, hardware protection that can only send information one way.
Tatsiana Zherdetski 16:13
Okay, so can you explain--what is a unidirectional gateway?
Andrew Ginter 16:17
Sure, the gateways, you know, NIST, the National Institute of Standards and Technology, NIST defines these things as a combination of hardware and software, the hardware is physically able to send information in only one direction from the OT network, out into the business network. You know, in a sense, the hardware is unhackable. You know, the, the hardware consists of two circuit boards, one circuit board has a fiber optic transmitter, a laser on it, the other one has a fiber optic receiver, a photocell on it, and there's a short piece of fiber, usually exactly 18 inches long connecting these two circuit boards. So you can send, you know, the laser can send light through that fiber to the other circuit board. But it's not physically possible to send anything back, there is no laser on the receiving circuit board, that piece of hardware can, it's only physically able to send information in one direction, you can send information from operations out to the business. The software, there is software involved, the software sort of enables the connectivity, the software makes copies of servers. So let's say we have a historian server or even just a relational database, a SQL Server, or you know, something technical an OPC server, we've got some kind of server, you know, on the industrial network that has all of the data in it that's allowed to be shared with the business. This is a very common design, people do this not for security, they do this to simplify connectivity. So they, they drop all of the data, they want to share with the business into one place. Now, the unidirectional gateway software logs into that server, it could be username and password into a historian, you know it, it's a strange OPC connection into an OPC server, logs in username and password, nothing tricky, and asks that server for all of the latest data, all the latest real time data, gets the data, converts the data to the strange one way formats, sends the data out through the one way hardware into the business network. And on the business network, the unidirectional gateway software, gets the data and inserts it into an identical server. If it's a historian on the inside, it's a historian on the outside, if it's OPC, on the inside, it's only seeing outside SQL on the inside SQL on the outside. And now, anybody on the business network on the outside network, who needs the data, asks the copy for the data logs into the copy normally nothing tricky username and password into you know, a SQL Server database and asks for the data that they need. And it's all there. Because all of the data that's allowed to be shared with the business is already out on the business network in the in the replica database. And you know, what this means is that everything that used to work still works. What it means is that even if a worst case scenario ransomware gets into the the IT network and encrypts everything even worse than that, you know, the bad guys find out how to steal every password for every account in the organization, even the passwords and accounts in the industrial network. Well, even if you have all that attack information if you have an utterly compromised IT network operations, keeps going operations is completely blind to the chaos that's consumed the IT network, nothing can get back, no password, no exploit, nothing can get back to the hardware into the industrial network.
Tatsiana Zherdetski 19:51
And it's no accident. You're an expert on this. This is what Waterfall does, correct?
Andrew Ginter 19:55
That's right. I mean the unidirectional gateway Waterfall invented the unidirectional gateway. It's our flagship product, we have a whole family of products that, you know are based on the gateway or that compromise complement the gateway. But, you know, we invented this technology back in '05, '07, we invented the technology back then to deal with what were being called "advanced persistent threats" back then, really that was, you know, initially it was it was code for Chinese intelligence agencies. But, you know, a few years later, everybody got into the act. Nowadays, you know, nobody calls them advanced anymore, because they're 13 years old. All the ransomware people have used these attack techniques. Now, it's the the pervasive attack method. It's called targeted attacks. And the unidirectional gateway was designed to utterly defeat that class of attack.
Tatsiana Zherdetski 20:45
So that's really interesting. Can you tell me a little bit more about how it's going? What's the adoption rate?
Andrew Ginter 20:50
Sure. You know, Waterfall's, biggest install base in North America is conventional power generation, coal plants, gas plants, especially hydro plants, you know, sort of more generally, worldwide, unidirectional gateways are used very widely in the Middle East, especially in high end, oil and gas installations, the biggest of them. In the last year, we're seeing that sort of oil and gas focus spill over into North American installations, mostly refineries. You know, pipelines are showing interest to now that, you know, we've seen what happened to Colonial Pipeline, you know, we're seeing very strong interest from Rails as well, you know, especially passenger rails and metros, these are very safety critical environments, they're very focused on on protecting safety. And the unidirectional gateways are seen as sort of the logical choice to enable business automation to enable the data to flow that the business needs to operate efficiently. Without risk to physical operations. We're also seeing strong interest in the manufacturing sector. And it's because that sector more than rails, more than powered the manufacturing sector, is using cloud services. And the very last thing anybody needs is for some sort of cyber-attack to compromise an industrial cloud service, a cloud service provider, that dozens of power plants, dozens of rail systems are connected to, you know, if you cripple a cloud service provider, you have an opportunity to pivot your attack to use the compromised service provider now to attack all of these industrial sites, through the connections that those industrial sites maintained to the cloud server. And, you know, the manufacturers don't need, they certainly don't want dozens of their sites at once to go down because of these cloud connections because of these compromised servers. So you know, that sort of the focus in the manufacturing sector is on protection from the cloud. Whereas the focus in these other sectors tends to be sort of critical infrastructure keep the lights on?
Tatsiana Zherdetski 23:00
Well, that's very promising, especially for all of us citizens who want safe drinking water and light said stay on. Before we let you go. Andrew, is there anything else that you'd like to leave our listeners with?
Andrew Ginter 23:11
Yeah, very much so. If people would like more information, I've talked about my book a couple of times. The book is available at the Waterfall website, free of charge courtesy of Waterfall. I mean, it's not a book about Waterfall. It's a book about the world's most secure industrial sites. But Waterfall makes it available as a public service. If you're interested, you can go to you know, https://waterfall-security.com/SEC-OT, and request a copy of the book. Or you can go to the website and contact us. And when you contact us, you know, sort of be reassured Waterfall is a technology company. We're a product company, we sell products, we don't do risk assessments, we don't charge for consultation. If you want to talk to a product expert or you know, if you want to talk to me, if you connect with me on LinkedIn or submit a contact request on the website, we have a whole family of products that are based on our flagship unidirectional gateway are the complement the gateways, you know, products, for example, that do secure connections to OT intrusion detection sensors. We do safe cloud connectivity. You know, we even do unidirectional remote support, reached out to us Waterfall Security Solutions. You know, we're happy to look at your problem. We're happy to give you our very best advice, free of charge. You know, especially of course advice as to how you might make your ot security posture dramatically more secure, with unidirectional gateways and related technologies. Thank you so much.
Outro 24:46
Thanks for listening. If you'd like more information on how Carahsoft or Waterfall Security Solutions can assist your organization, please visit www.carahsoft.com or email us at waterfall@carahsoft.com Thanks again for listening, and have a great day.