Affordably Refreshing Your Water District’s Process Control

Implementing Powerful, Open, and Secure SCADA on a Budget

59 min video  /  49 minute read View slides
 

About this Webinar

In many water districts around the world, there is a clear need to add or upgrade to a modern SCADA system that is open and secure. However, finding a complete, scalable SCADA solution that protects against increasing cyber security threats while also staying within budgetary limits is very difficult.

In this webinar, learn about two municipal water districts that upgraded to SCADA systems with unlimited clients, tags, and connections, built on an innovative controls platform with deeply embedded cyber security. Don Pearson, chief strategy officer of Inductive Automation, and Albert Rooyakkers, CEO of Bedrock Automation, have an informative discussion with Dee Brown, principal of Brown Engineers, the award-winning integration firm that implemented new systems combining Ignition and Bedrock Automation technologies for water districts in Clarksville, Ark., and Russellville, Ark. Since switching to new Ignition SCADA systems with secure Bedrock controllers, both water districts have experienced great results and plan to expand their use in the future.

Learn how you can:

  • Run and control processes such as water treatment, water distribution, sewer treatment, sewer collection, and more
  • Complete innovative projects in short timeframes
  • Provide greater access to information while keeping costs down
  • Ensure cyber security down to the hardware level
  • Meet reporting requirements using less paper
  • Simplify alarm management
  • And much more

Webinar Transcript

00:00
Don: Morning everyone, and welcome to today's webinar, Affordably Refreshing Your Water District's Process Control: Implementing Powerful Open and Secure SCADA on a Budget. My name is Don Pearson. I'll serve as the moderator for today's webinar, and I'll make a few introductory comments and then I'll introduce the expert panelists that we have on this subject in just a couple of minutes. On the agenda today, we'll start by introducing you to two companies, Inductive Automation and Bedrock Automation. And as I said, I'll introduce our panelists who'll discuss the combined value of Ignition and Bedrock together and how they address the issues faced by the water wastewater industry. Then we'll discuss two specific use cases: Clarksville Light and Water Company and the Russellville Water and Sewer System. In the last section of the webinar as always, we'll have Q&A. A brief introduction to Inductive Automation as far as background, the company was founded in 2003.

01:01
Don: Our software Ignition, it can be customized and used for HMI, SCADA, MES, and IoT solutions. Industrial organizations now in actually over 100 countries chose Ignition for their industrial automation needs. It's trusted by 40% of Fortune 100, and I think we're over 26-27% of Fortune 500 companies now. Our integrated program has grown to almost 1500 integrators now who have joined the program. If you want more information on Inductive Automation, if this is a new introduction to you on that, just go to inductiveautomation.com, go to the About section, and you can get a pretty good introduction there. Also, as far as water issues and water, wastewater Ignition currently is used in 150 plus water districts in the US, and I think closer to 220-230. And globally, we have several water wastewater case studies. We're gonna talk about a couple, Clarksville and Russellville today, but you can find under case studies, both in video and other customer references in this arena if you go to our website inductiveautomation.com.

02:15
Don: One snapshot slide on Ignition, it's an industrial application platform. This is I think eight bullet points here that sort of summarize what differentiates us, what's unique. We have an unlimited licensing model, cross-platform compatibility. It's based on IT standard technologies, scalable server-client architecture. It's web managed, web launched on a desktop or mobile. Modular configurability and very rapid development and deployment in the tools that are built into the Ignition platform. I think probably one of the most unique things that really brought total cost of ownership and ROI numbers in line for organizations is the unlimited clients and tags and device connections and current designers that help people really innovate with the platform. So that's just a quick introduction to us. Let me introduce the panelists, so we can get in today's program quickly 'cause we have a pretty full program for you today, and we wanna have time for case studies and questions.

03:17
Don: First, Dee Brown, he's a professional engineer who specializes in TIA-compliant data centers, monitoring for data centers and SCADA. He also manages projects involving various facilities for power distribution, software-based controls, telemetry, video surveillance, access control systems and networks. He's an accredited TIA designer or ATD in data centers and has nearly 20 years of experience in electrical engineering. He's the principal and co-founder of Brown Engineers, a consulting firm that specializes in large-scale data centers, water wastewater and other high-demand facilities.

03:54
Don: Pleased to have John Lester with us. He's the General Manager at Clarksville Light and Water company. He's held that position for nearly four years and has previous leadership experience as a city manager and city administrator and in the public utilities field. And last is Albert Rooyakkers. He's the CEO of Bedrock Automation. Albert has over 30 years of process control and electronics industry experience. Before founding Bedrock, he directed business and application development teams for Maxim Integrated Products. He also served in product and business development capacities for Invensis, and Albert holds over 50 patents with an additional 30 patents pending in electronics, automation systems and cybersecurity. So welcome to all our panelists, and I think what I'd like to do is rather than me do the introduction to Bedrock, I'd like to move it over to you, Albert for you to do a little bit of an introduction to Bedrock Automation. So, Albert.

04:58
Albert: Thank you, Don. I hope everyone can hear me correctly here and thank you to all the attendees today. I appreciate this very much. Who is Bedrock Automation? We are a subsidiary of a large multinational semiconductor company that's here in Silicon Valley. It's been around for over 35 years, but Bedrock itself is one of the youngest and newest companies in the field of automation. Now this newness has given us the opportunity, afforded us the opportunity to design something really new from scratch and from the inside out and rebuild a new automation platform. The mission, the objective of the system was to build a system that's simple, scalable and secure. And the objective of the company is quite simply to deliver open and secure automation to the automation market. So we've filed over 100 patents and many of them have been received, and we also have received a lot of awards and accolades from industry and media and others. And so that's greatly respected and appreciated. Incorporated in 2013 and shipping for 18 months with dozens of systems and early adopters and customers, and as you'll see later today here in this presentation what we've done at Clarksville.

06:27
Albert: What is the platform? So if you take a look at it first, this is a backplane populated with basically all of the modules and all the components that make up a system. Some of the obvious things to point out will be its extreme robustness, and you'll see that through sealed, all-metal construction. If the IO modules were removed, you'd see that the backplane is a pinless electromagnetic interconnect. These are things that address some of the high-level robustness and cybersecurity issues.

07:00
Albert: And so let's take a look a little bit further into the platform, and you'll see the architecture, again, is very, very simple. And while this system can be used from any application to SCADA RTU up to medium and even very large DCS and PLC applications, there are only a handful of part numbers. And that's one of the more radical effects of designing from the inside out and building modules and systems that are software-defined, software-defined hardware. Go left to right, you see on the left, we have a universal power module, which works in any kind of a grid, any kind of a power dual feed and others. Universal controller that runs an open 61131 language and a series of universal IO modules, universal and analog and universal discreet as well as universal network modules for Ethernet and other types. And this all interconnects on a high-speed, 4 gigabit backplane, so this makes enough. So there's 10 part numbers, and you can put together a system almost like LEGO, hardware LEGO and use your maps to configure it to do what you want it to do with literally just a very small handful of parts. So again, you can imagine the simplicity and the elegance of the design.

08:23
Albert: So security is obviously a very big issue today, and it's growing all the time. Now, the concept of intrinsic security, it's something that's bolt-on versus built-in. What does this mean? It means that for us, we were able to use security technologies at the component level and the operating system level and the supply chain and manufacturing infrastructure to build it such that similar to a biological virus versus a computer virus, if you solve this problem from the inside out and bring it down to the smallest component and you vaccinate your biological system. Well, you can vaccinate a computer system, so that down at the smallest component to transistors and electronics that make up the system, they're the root of trust. So we create a technology that allows authentication of hardware, software and applications right at the module and then push that technology out.

09:23
Albert: If we look at the impact of that to life cycle costs, a couple of things that are worth pointing out, one is in installation, you'll see how these technologies allow the installation of the systems to be far more simpler and far more robust and be able to interconnect with legacy technologies as well, as well as the whole emerging life cycle cost of cybersecurity. So that's, in summary, the technologies apply from to the user really to the areas of cost and security, safety and reliability, and these are the factors that have been driving design from the inside out. And we're very, very proud of the system, and I thank you for this opportunity to be here today.

10:11
Don: Albert, thanks. Thanks so much, and you know I know that one of the things that over the time that Bedrock and Inductive Automation have been working together, sometimes there's been questions as, "Well, what's the sort of a combined benefit?" I just wanna make a couple of comments and also let Albert make a couple of comments about that because I think it's worth pointing out. And when John Lester and Dee get a chance to speak, you'll see from the end user's perspective. But I think one of the biggest combined benefits is really when you use open technology and standards, you get a lot of benefit from bringing it all together. OPC UA, SQL, SSL, it allows us to have cross-platform compatibility. It allows us to be able to connect the whole enterprise together, and it allow us to be able to add to the system later because of the module architecture also of Inductive Automation.

11:05
Don: So you're sort of future proofing your system where you don't have to worry about trading it out every couple of three years. So it also impacts, I think I mentioned already, the licensing side when one looks at total cost of ownership or getting an ROI on investment as water districts are looking at the challenges they face right now when you sell by the server, not by the seat. And that's just what was talked about by Bedrock in terms of reducing life cycle costs in all areas. These things bring the cost down to water districts who may not be sitting there swimming in cash, no pun intended there. And that's a comment on a couple of those bullet points, but Albert, can you say a couple of more things in terms of the value bringing together from an intrinsic security standpoint?

11:54
Albert: Yeah, absolutely. The end-point root of trust, the point that I made a moment ago, this is now being shown how we utilize that with firmware upgrades going forward and we extend that root of trust in using open-standard technology security like Public Key Infrastructure, PKI and Transport Layer Security, TLS and standard state of the art crypto. So it allows us to extend security to the SCADA and we'll be working together, us and Inductive Automation, to demonstrate how we can secure the SCADA application and push this root of trust into role-based access, biometrics, smart cards and other technologies that will allow the human factor to be secured. So this is what we're gonna be showing in shipping in products later this year, and this is a firmware software upgrade to any system that exists. So it allows security to be extended, to be holistic and to be intrinsic without additional hardware and cost and complexity. It's security that just happens.

13:12
Don: Thanks, I think that puts a little bit more punctuation on the importance to them, I think. What I'd like to do before going into the case studies is maybe just mention, certainly this isn't a comprehensive list, but just touch on a few of the challenges that we see are being faced by the industry, and how this solution may approach and address those challenges effectively. The first one being, certain need for modernization and stronger security. We've mentioned security a couple of times, but as security threats increase, really may a lot of utilities are... They're just vulnerable to cyber attack. And that one point made by the US Department of Homeland Security in a report recently cited SCADA as one of the top improvements that water, wastewater facilities should make in order to address that issue. So I think when you talk about the need for modernization and security in combination, we also have many districts with aging equipment, now 20, 30 years old, in some cases. So to do something that allows you to bring it into a modern technology and a secured technology is one of the challenges faced by a lot of districts.

14:20
Don: And some are dealing with operating systems, if they're dealing with Windows XP and it's not supported anymore, there are challenges of the technology not even being supported, which just increases the risk. Another challenge, certainly, is limited funding and capital. And while we chatted a bit about the total cost of ownership, we'll hear more in terms of our case studies, certainly one of our water districts in Southern California made the comment at a recent show we were at, at the AWWA show in Anaheim last week, when Henry was speaking, he said he moved his entire system over out of his O&M budget. He never had any capital expenditure to do a full migration. You may need some CAPEX, if you're trying to do everything all at once. You may have a migration strategy, and organizations also looking at moving over to the leasing option where they can move the entire cost of their system and its continued maintenance into OPEX, where there are difficulties, obviously, in getting CAPEX money. It's really hard to get those capital expenditures and for these kinds of very needed upgrades and improvements, and I think delivering this challenge is one of the things that a TCO that Bedrock and Inductive can offer, can benefit two districts.

15:39
Don: Another challenge, of course, is just a need for improved control and better information. Certainly when you're trying to bring systems together and get the edge all the way connected to the enterprise, you have to deal with the security threats and also you have to expand services, meeting regulatory requirements. And all are doing that within very tight budgets. So putting in an automation system that allows you to connect with practically any device or system effectively, see it all in one place, and develop as an enterprise is an important component to address that challenge for better information available throughout the enterprise.

16:21
Don: Moving away from proprietary solutions to get more flexibility and more choice in technology suppliers, I think that's an important thing too. So that if you think about the challenge of what happens with vendors right now and their operating systems, and the prices associated with that, Planned Obsolescence basically forces you to buy newer software. And what we think is when you architect properly, you should be in power, empower those water districts to move away from being trapped by some sort of proprietary solution, move to cross-platform, run on any major operating system, and really that flexible and scalable system is enough that it won't need to be replaced every couple of years.

17:07
Don: So those are just a couple of snapshots of some ideas relating to some of the challenges. God knows you have a much longer list than what I just hit on here, but we wanna really jump into the case studies. Let's look at these two case studies and really proof points for what we're talking about here. Clarksville Light and Water, in Clarksville, Arkansas, and Russellville Water and Sewer System in Russellville, Arkansas. These two projects are unique, but in both cases, they needed major upgrades to their existing SCADA system. But also, I think they were unique and if they did set their sights higher than just moving to a good enough solution, both the districts had a vision for moving to the best, most modern solution that they could that would take them years into the future. Brown Engineers was the Systems integrator for both, and Brown Engineers is certified by both Inductive Automation and Bedrock Automation, and they played a major role in selecting and implementing the ignition in Bedrock for these two organizations. So I think it's appropriate that I turn it over to Dee Brown to tell you more about the project at Clarksville. So with that, I'm gonna turn it over to you, Dee.

18:20
Dee: Thank you, Don, and good afternoon everyone. It's great to be with you today. So this first project that we wanna talk about is at Clarksville, out of the screens here show one of the first projects we did was a 30 dated window where we had to bring the remote telemetry system into ignition, and it was actually due to ignition that we were able to deliver on that project and won an award. That's what the slide is on the left side of the screen. And from there, we moved into the Water treatment plant expansion project, which is the ignition screen shown here. That's a full ignition system with the vision graphics mobile client with remote access, trending alarms reporting, all built out of ignition. And so that was a great project that we started with back in 2013. So we've been working with Clarksville for a few years with ignition in this process, and that led us to one of the next projects that we wanna talk about and to understand that a little bit better, we have John Lester here with us today. John is the General manager at Clarksville Light and Water, and helped shape the vision for that. John, why don't you tell us a little about Clarksville?

19:29
John: Clarksville, Arkansas is on the Southern end of the Ozark mountains in Arkansas. It's a community of about 10,000. A little bit about the utility itself. We've really been in existence... We're a municipally owned utility. We've really been in existence for well over 100 years. In 1947, however, the elected officials saw fit to really create a separate Utility Commission. So I do not necessarily and this utility does not necessarily report directly to elected officials. We have appointed commissioners and a separate board. And the whole idea behind that is to provide some autonomy and really focus on running the utility like the business that it is. However, even with the existing structure, the City Council basically has regulatory oversight in that they sign the board members. If we have any major projects where we have to issue debt, they have to approve that, and they also approve rates or any adjustments we have related to electric water and wastewater rates. And since we are municipally owned, we are not-for-profit, really. It's about adding value to the community more than maximizing profits.

20:47
John: After the 30-day challenge that Dee talked about, we started looking more at what our needs were, and we had a substantial amount of needs on our electric side. We're a unified utility, and we really wanted to take some major steps. We're talking about CAPEX and OPEX, well, this whole project from the get-go really was unique in that I didn't have a whole lot of legacy systems or specific SCADA-type software to have to replace because simply our small utility had nothing in place. And so the 30-day challenge that I put Dee under was really critical in order to demonstrate to our employees that it could be done and it could be done effectively and it could be a really valuable tool for them. So when we got the 30-day challenge done, I think our water superintendents were very pleased with the result. Well, that gave us the opportunity to take some additional leaps forward in that our electric system had no SCADA at all, which is highly unusual for an electric utility.

21:57
John: So as I started communicating to the board members what the needs were, we talked about what some of those expenses might be, and then how we were gonna carry the data. And after much discussion, we opted rather than to use radio systems, we wanted to look at actually building a fiber network. And in 2006, they authorized a substantial amount of capital expenditures, and with that included a 16-plus mile 288 fiber strand fiber network that we constructed. Typically, based upon the utility needs alone, 288 strands is way more than what we would need. However, when you start looking in the economics of constructing such a network, it's not necessarily the cost of the material in the cable, whether it's 12 individual strands or 288 expense-wise, it's the cost of the labor to construct. So my board had the vision to go ahead and overbuild capacity thinking that we could use it in other ways to add value in the community.

23:05
John: When we did the construction, it was designed in a way to have maximum reliability and redundancy. It's built with self-healing features, so if there is a storm in a particular segment that maybe severs a fiber, it'll automatically transport the data around it in other ways so it'll heal itself. This project, on the utility side, we ultimately connected four substations that we own, plus we are tied to the grid via a substation north of town. Southwestern Power Administration is that connection. We connected our office, our operations center, West Water Tower, the water treatment plant and pollution control. Now, even though we built this fiber network, we also understood that we were going to, as we continue to expand the SCADA capabilities and monitoring capabilities and control capabilities, there was no way we wanted to build a fiber to every single device that was out there. For example, it just wouldn't make sense longer term when we did a wastewater project to build fiber to the West station. Well, there will be wireless applications, and what we've done is the West Water Tower we've made that kind of a control point for any wireless communication. So we've really got a combination of both that the fiber network's gonna be pretty powerful for us.

24:24
Dee: That really helps kinda see where you guys have leapfrogged ahead of everyone in terms of your vision for what you're doing. And so you can see from the fiber backbone that we have now, we had an ability to think more broadly about the electric utility SCADA system and how that was going to operate with all the facilities that John had described. And so one of the things that we had to consider with that was the cybersecurity features, especially being on a large fiber loop that would have a lot of other things connected to it. As we're also thinking about the design here, you'll see, this is typical. Three of the substations that have a 15-kV distribution system with the feeder breakers. So we had some legacy devices here that we had to consider how we were gonna connect to those and communicate when we're bringing the data in from that part of the system. We had some discrete devices that were older breakers and things that we needed to connect to, and we also had some new power station meters that would have some data on those.

25:26
Dee: One of the other sites that's not shown here is the grid-tie substation outside of the city. That was far enough away, we had to connect to the Bedrock controller there with Ethernet radios, and that Bedrock controller is also talking to an existing GE D20 RTU using DNP3 protocol. So these are some of the things that we were looking at in terms of what we were gonna do with the SCADA system. And so here are some of the panels we built. These sub-panels had to be retrofitted into existing control panels of the four substations, and then also the grid-tie station. So that's what those look like. And this is an example of one of the finished products at a station that mostly had discrete inputs only for circuit breaker status at one of the older substations, with a nice clean control panel with the Bedrock controller in it.

26:22
Dee: I'll also show you some of the Ignition screens related to this. We have one-line diagrams for the 169 kV and 69 kV circuits that come into the city, and also have some trending and power monitoring system screens that help with managing the daily demand. We've got some other weather and temperature data coming in to that and also tracking the monthly peak loads there for a management-level dashboard. This screen I like because it shows one of the substations with the pop-up boxes for some of the existing legacy devices that we had to communicate with and how we were able to grab some of the important data out of those systems. We also have some other screens that are not shown, but there are city maps that show the areas where these distribution feeder circuits are out in the city, and they change state, color state when those breakers open or close. And those can be used as outage maps to see what area is affected when we have change of state. So it's a little bit about the electric SCADA project.

27:32
Dee: Other things that we're thinking about at Clarksville with John are the remote telemetry units for both sewer and the water system using Bedrock controls back to Ignition. The sewer plant controls are one of the next things on our target list to take a look at. And then dreaming a little bit about what distributed generation systems may look like with industrial customers and some generators and things that we can use to manage the electric demand throughout the city. So some good-looking, future projects there. And I think some of the success factors, part of it is a client having a vision for the future. You've heard John share just a little bit of that. It's a great story at Clarksville about what they're doing there and knowing where they wanna go with their city. We had a great relationship we've developed through these projects. And I think, also, bringing to bear the right technology, Ignition being used across all three utility departments here has been a great win for them. And using Bedrock cyber secure controllers to give them peace of mind as they move into the future.

28:37
John: Can I speak real quick?

28:39
Dee: Yeah, John, sure.

28:41
John: As we talked about the solution as the integrator for us, what was clear to me related to the Ignition software was the fact that we could use it across all platforms and that it was very flexible. We could design it in any way we saw fit, and we could add to it. And I see others... I think that the software is so powerful, I see other potential uses. We're looking at smart grid technologies and smart meters. And when I talked about the fiber, as I'm seeing the capability of this software, we could potentially say we provided broadband services over our fiber network to the community. I can see how we could use that to monitor outage-related situations with our fiber. So I just see it as a powerful type platform that we can use across lots of different departments. And our original discussion, if you remember, Dee, was we don't want the Hotel California, no offense, California, in Clarksville 'cause I don't want to be locked into a certain proprietary system and Ignition gave us the ability to avoid that.

29:46
Don: John, thanks. I just wanna ask you one other question, John, and you can... Is whatever your thoughts on it. But when you talk about the people on your team, yes, we're talking about the value from different perspectives, but how is this solution responded to from the members on your team who have to work with this technology and deliver service to your citizens every single day? What's their view on the deployments and the change that you've made there?

30:14
John: Well, let me go to the foundation that we basically had nothing in place for as far as SCADA and automation. And there was a beginning attempt to do that on the water side and it had not necessarily been successful. And that's why the 30-day challenge was issued to Dee and that we had to show them it could be done and it could be done right. The water treatment plant supervisor was very skeptical, and when we pulled it off, I think he was pleased and surprised. In fact, it was in the first 60 days that he really started using the volumes and the levels in the tanks to help our distribution crews zero in on where a leak was that would have been hard to find, otherwise. So it was already demonstrating some cost-savings both in overtime labor that had to respond and water losses.

31:08
John: Now, I like to joke when I give presentations and talk about it, we're automated enough with our water treatment plant that I like to say that the water treatment plant supervisor or manager can completely operate his plant from the La-Z-Boy chair while he's watching an NFL game on a Sunday afternoon. And what makes it even more funny is when he hears me say that, his initial response is, "No, it's not an NFL game. It's from my deer stand in the woods." So it really gives him the ability to be very reactive and proactive with his water treatment plant without having to physically be there, and it really has basically eliminated all overtime because of the capability.

31:54
Don: That's great. Thanks, John. We put this slide in here just to see if you, John, or Dee want to make any comment about your cyber secure electric SCADA controls and the ACEC Arkansas Award. That's pretty cool.

32:07
Dee: Yeah, this was a nice award that Clarksville and Brown Engineers won just to show off what we've done with cyber secure electric SCADA controls there. And it's just nice to be able to see people taking recognition with these kind of projects that are being done, even in small and medium-sized utilities. This is very elegant, nice stuff, but it's not so complicated that you can't put it anywhere. So that's part of the elegance and the beauty of it.

32:33
John: Well, and for both the solution for Ignition and Bedrock, we're a small utility, we're very sensitive to cost. And I've found both applications and solutions relatively affordable, based upon what we needed.

32:48
Don: That's great, thanks, gentleman. Okay, we're gonna move to the next one, the Russellville one. Let's talk about this one. It's our second case study. It's Russellville Water and Sewer System. It provides sewage treatment and pollution control services for nearly 30,000 residents in Russellville, Arkansas. It's operated by City Corporation. The Russellville system relied on PLCs for automatic control of various pieces of equipment, but the PLCs were aging. And when one of them failed, they needed to find a replacement and do something quickly. They wanted a replacement that was modern and future-proof. They liked the performance, scalability, and built-in cyber security protection of the Bedrock Control System, and they chose Ignition for their HMI SCADA system. So they completed their Ignition upgrade in mid-2015 and their Bedrock upgrade in late 2015. Dee, you really are the expert on this project, you and your team, so I'll let you explain details and where it went. Over to you, Dee.

33:45
Dee: Thank you, Don. So at Russellville, this is another fairly large Ignition system that encompasses the water treatment plant, all the water distribution system around the city, sewer treatment, and sewer collection system. So it was a great project that's... It was one of the smoothest start ups I've seen on a project, just by using Ignition there. And what we were able to do is they had a controller that failed back in late 2015 that we were able to suggest to them. We had already been having conversations with them about Bedrock Controllers, but when this controller failed, we used that as an opportunity to be able to put a new Bedrock Controller on their digester blowers. It's running clarifiers, sludge pumps, and so it's been operating reliably ever since. One of the neat things I loved about working with Russellville is, they're also somewhat innovative. And once they saw how nice the Ignition software is, the reliability of it, they asked us about another non-SCADA type of use for Ignition.

34:51
Dee: This would be a mobile system process for tracking and reporting sanitary sewer overflows and allowing their field crews to be able to operate, enter some of that data from remote devices. So that led us to a project here that we were able to develop for them, tracking sanitary sewer overflows out in the field. These are manhole overflows. They're required to be reported by the Department of Environmental Quality. And you can imagine when you're having sewer overflows, that's because you're in a large rain event, and you've got guys out in trucks, running around with clipboards and paper, trying to keep track of all that stuff. And so that's part of what we were trying to eliminate here, and be able to bring that data in electronically and manage these reports. The administrative person who is responsible for collecting all that paper before, they would tell you this would take a couple of hours, at least, at a minimum, to gather this data, aggregate it, and get it ready to turn in. And so a great little project that we were able to use Ignition in this new way.

35:57
Dee: I also specifically went back afterwards and just asked them, I was like, "What are some of the staff... These folks aren't used to using Ignition and some of the mobile smartphones and tablets. What are they saying about that?" And just a couple of key comments that I thought were great that, "The field guys love it, even some of the guys that haven't used computers before." They also said, "What used to take at least an hour or more can now be submitted in about five minutes, and I know that our flow estimates are more accurate now." So I think those are good testaments to being able to use a very versatile tool like Ignition to be able to deliver these kind of solutions for folks.

36:35
Dee: Now, when the controller that I mentioned had failed, I feel like we talk about cyber security a lot, but one of the things we don't always get enough time to talk about is just the basics of this controller, that it is so elegant and easy to use, it makes it a really nice platform for any size projects, small, medium, or large. A couple of features that we particularly think are interesting to us, for folks that have to install and maintain these, the backplane having an electromagnetic system instead of pins that corrode and break, so there's reliability issues that are enhanced because of that. The galvanic isolation between channels, so we don't have as many field devices to have to install on these panels. It's very fast. It's got a military grade real-time operating system that's also very easy to use, and again, it's scalable. It can be used in small or large applications. So those are just a couple of things. We can't touch on them all here today, but it's important to know that it's an extremely elegant solution. And I'll be happy to tell you more about that as we move through this process.

37:48
Dee: So by upgrading to Ignition and Bedrock, Russellville and City Corporation have realized some cost savings for them. Cost was a big factor in choosing Ignition. Their previous software charged them based on the number of tags, number of clients and all those traditional things that you see with legacy HMI-type systems. So they were a growing city and a growing utility and not having to pay more to upgrade to a higher number of tags was a big deal for them, and being able to expand to that. So in the switch-over process, we kept them up and running during the entire process and able to use the Bedrock Controllers in their applications to also consider costs of what their future systems may look like. A couple of things here, in their future planning, just the fact that it's been running without fail since installation of... That's one of our first Bedrock units, was late fall of 2015, so a couple of years now, and have been in a process with City Corporation to talk about how to begin to do a phased approach to migrate all of their controllers to a Bedrock built platform.

38:58
Dee: They're a utility that is concerned about cyber security and are very aware of that as part of their process, and wanna make good cost-effective decisions on those solutions that will help move them into the future. So today, we've talked about two different case studies from two different clients of ours, and I just wanna make a couple of closing statements about my part on the affordable technology that's available today.

39:26
Dee: Regarding Ignition, I've never seen it beaten in a price comparison, especially if you consider the ongoing annual support costs and do a five-year or a 10-year life cycle cost of that. I'm pretty confident it's gonna win hands down. It has for us every time now. It's a very feature-rich application. And so, people ask, "Well, what's missing. There's nothing missing. All the tools that you need." With regard to the Bedrock controllers, they really do offer best-in-class control systems that are easy to use, and reliable with a five-year warranty, and that give you the added piece of mind with cyber security. So, that combination to me as an engineer and an integrator, being able to offer Ignition and Bedrock to our customers that we know it's gonna work and it's gonna work well. There are unprecedented capabilities, especially when you have these two companies as independent companies, but they are working together to deliver secure solutions that actually meet customers' needs, is a very, very powerful combination. So, Don, I believe with that, I will turn it back over to you.

40:30
Don: Dee, thank you. Thanks very much to you, and to John, we'll be coming back to you. I have a queue of questions lined up here that are gonna go to Albert, to John and to Dee once we... I just wanna do a little summary myself, and then we'll move into Q&A, and we try to leave about 15 minutes for that. So, if you haven't done it yet, feel free to put your questions into the queue. As I mentioned at the outset, we'll get to as many of them as we can. We wanna make sure that what you wanna know about, we answer, and we've got the people on the phone to be able to, or on this call to be able to do that.

41:06
Don: Just a quick summary of this whole discussion from my perspective. Really Bedrock and Inductive were working together, and the goal is really to modernize and protect the water industry through technology that's affordable, that is open and that is highly secure. When you think about municipal water districts, really, this is a critical, essential part of public infrastructure, and they... Frankly, they need and they deserve the best solution that happens to be out there. Granted I might be slightly biased 'cause I'm with Inductive Automation, but I think it's based on a lot of the points that Dee made that are really a solid foundation for why this combination is a great solution for water districts. I mean really, and it's been said, the more access to data helps you make well-informed decisions, and improving security at every level provides a whole lot better peace of mind for water districts and also for the citizens too.

41:58
Don: So, when you have better control of your operations, that helps you identify and fix problems faster, it keeps residents satisfied with your service, and I think most importantly, our solutions are actually within the financial reach of any water districts. Even with limited budgets, there's creative ways to migrate, to put a strategy in place, to move to OPEX, things we can work with you on to try and make that as effective as possible. So, those are kind of my summary comments, but Albert, I just wanna turn it over to you to give you a chance to make any closing thoughts of your view of this combination of Bedrock and Inductive and how it can help water districts.

42:36
Albert: It goes back, Don, again to the open and secure architecture. It is the way of the future. There's no turning back. We need to decouple the technology so that the user can achieve the benefits of interoperability, best-in-class technologies where they deem fit. And so we're really showing what's possible. And what's very cool is how the intrinsic cyber that we've developed will push down to sensors and up through the network and address some of the very tall polls of cyber security for the users, and I think that's an extremely important value and a proposition.

43:26
Don: Thanks Albert, I appreciate that. And there's a number of questions which I'll get back to that are addressed to you too as we go into our Q&A. One final comment on Ignition. I mentioned at the outset, but please, you can download the entire platform. You can use it for free, and it's fully available, just a two hour... It times out at two hours, but you can reset as many times you like. So, you have the freedom to sort of develop your own system, sorta try before you buy things. So, please take advantage of that to any of you who are not familiar with the Ignition platform.

44:03
Albert: And just one more comment. It's software is such a critical part of training and in cost in life cycle management. Again, similar to the Ignition package, Bedrock's integrated development environment toolset is available for free, unlimited seats, unlimited tags. You can download it, engineer your system. There's a built-in simulator. You can build the simulations and test it out. So, literally combined, you can download free software to develop your SCADA, free software to develop your control system and you haven't spent a penny yet and you've got your engineering team up and running and you can engineer a solution. So, that's very unique, I think too, in the industry. So, I wanna make that point. I don't think there's any other way to do it going forward. That's the way of the future.

45:01
Don: Okay, we're gonna move... Thanks Albert. Let's move into the Q&A, and I'm just kind of identifying as I go through. So, I'm gonna try and get to it. We have quite a queue and a lot of questions, but since you were just talking, Albert, I'm gonna throw a couple your way 'cause they're very specific about Bedrock. I'm gonna read two questions 'cause you may wanna answer these two in combination. The first question is, is Bedrock an addition like a DCS system? What is the main advantage of Bedrock instead of other vendors like Siemens. And the related question is, how does... Let me see. How does Bedrock controller IDE handle converting code from the existing PLC? Does all the code need to be rewritten? So, how about that? And these are all to you, Albert. I think you can handle three questions at once. The third one is, are the Bedrock PLC modules hot swappable? So, can you kinda handle those three questions?

46:03
Albert: Okay, so as quick as I can I'll go backwards. I wrote them down. I hope I didn't forget them. There's three in there. So yes, hot swappable, absolutely. All of the modules are, and the IO modules can be a single, dual or a triple redundant. So that would reduce the risk of that. But certainly, if it's just single modular, it's hot swappable, absolutely. The second question was... What was it again? It was, how would you configure from an existing PLC or existing application to a new one, yes. Yeah, so there are some tools, some conversion tools, that I've seen recently, I'm not gonna actually... We've been testing one by a small startup company that does system configuration based on a database from any particular system type target to any other target, and in this case, say from Siemens to Bedrock or from a DCS environment to the Bedrock environment. It works really, really well.

47:07
Albert: Yes, there would be some configuration or reconfiguration of some of the control strategies, it depends on exactly what the control strategy is. To what degree is there some complexity in that? We've made some demonstrations and presentations on how that 6-11-31 open language works with DCS and PLC translations, and it's really amazingly elegant and simple. But... Each it's a case-by-case situation, and so I don't know how to get more granular than that in the time allotted in this question. And then...

47:46
Don: I think that's fine but also, I'd say if other people... If people want more detail, we are gonna make these questions and the people who asked them available to all people participating. So, those of you who are getting answers, if you want a deeper dive, we're absolutely willing to follow up. I just volunteered your time Albert too.

48:05
Albert: Okay, sure, sure. And then I think the other question was, is this like a DCS or what is the difference between this.

48:12
Don: Yeah.

48:12
Albert: The key differences between Bedrock and Siemens or others. Well, I would say one is the degree of intrinsic cyber defense is absolutely a differentiator. The hardening of the system to, not just cyber, but being compliant to mil standards for EMP hardening. The elegance of the hardware and the newness of the hardware that compares to the performance of say, from a PLC standpoint, state of the art... We have comparative data against the Rockwell PLC versus ours running X amount of that, rules of ladder. So, just by being able to use advanced technologies, that allows us the ability to have a speed that's unmatched. The differences between typical systems or A PLC is a different product than a DCS, which is a different product than an RTU. We've converged that by allowing the price, power and performance to be in a single platform. So, yes, this is a DCS, it behaves very much like a DCS, that's my DNA. It's also being applied all over the place as a very fast state-of-the-art PLC.

49:31
Don: Thanks.

49:32
Albert: So, those are the key points I'd say.

49:34
Don: Thanks very much. Thank you, Albert. Okay, I'm gonna go to John, would you please... This is a question for you, John. Would you please talk a little bit about the required skills for OT IT on-site personnel to implement Ignition Bedrock platforms and systems? What skills do they need? How did that work out?

49:56
John: Well, we're in a rural area and we had operators that were not necessarily comfortable with technology, and they've really been able to pick it up really quickly at every level of operator. So, I mean just basic keyboarding skills and understanding how some of the technology works is all that they need to bring to the table. I mean it's pretty intuitive and very easy for them to pick up.

50:25
Don: Okay, thanks, John. Here's another question for you though. How did Clarksville select Brown Engineers? Was it a separate design consultant involved, or how did you make that decision?

50:36
John: Well, we were in the process, when I first moved here, in expanding our water treatment plant, adding another four million gallons a day to the capacity. And we had had a long-term engineering firm that we've used for those types of projects, and I started asking them who might be a good source for this type of solution or application, and it was a relationship that that engineering firm had with Dee at Brown Engineers, and that's how the connection was made. In fact, I think I wanted to move on it quick enough that I picked up the phone and called Dee on his cell, and I think he was out at Inductive at an Ignition seminar out in California when he picked up the phone. So, Dee can correct me if I'm wrong, but we started that conversation and he was here within a week, and it's just been a relationship that's blossomed since then. It's about working together to identify problems and really providing realistic solutions that they've always been able to bring to the table for me.

51:43
Don: Thanks, John. Dee, I've got a question for you here in the queue, I think that I'll address it to you from Steven. This is focused on a customer that had almost nothing in place, what about customers that have standards and SCADA in place that wanna upgrade or move from another application to Ignition? I happen to know that you've converted every one of your water wastewater customers over the last five years to Ignition, so how do you do it with all these different levels of deployment and SCADA sophistication of existing systems?

52:12
Dee: Okay, so a couple of things that we like to do, we work a lot with the customer right up front to talk about what are their goals in this migration, what does this thing need to look like as a finished product? And then with that, I would say we spend a fair amount of time talking about high performance HMI features, you know, how much data are you asking your operators to manage, what does their day-to-day operations look like? And we've been successful at showing people the benefits of high performance HMI screens and how to develop those. There are Ignition templates and blocks that go with that, and then we've also learned we can migrate some of the existing tag databases using some tools we've developed to implement those tags and templates in Ignition. And we've also learned with some of the Bedrock coding processes that even though you may have to re-work or configure some of the PLC controls, that's actually a good opportunity to either clean some of that up or develop similar templates that work hand-in-hand and link up correctly with Ignition. So there's some great benefits in being able to do that. Those are the things we would talk to the customer about, about how we're gonna go about migrating this thing in a way that's truly successful for them.

53:30
Don: Thanks so much, appreciate that answer, Dee. Here's a question, I'm just gonna take... It says, are you indicating that Ignition is not secure without Bedrock hardware? My answer to that is no, we're not saying that at all, Ignition is secure, but this is a total route of trust all the way to the hardware level. I'd also say that, I think it was last week, 'cause I just read it yesterday actually, we published a white paper which is available on resources page, it's called the Ignition and Java security white paper. So anybody who's really interested in the security of the Ignition system and being comfortable with that, you can certainly go to inductiveautomation.com resources and download that white paper. And it goes into strong detail from our development team, who was involved in putting that together on the security of Ignition.

54:19
Don: So next is, I think I got a whole bunch of things in the queue, I also wanna reiterate because you guys are doing a great job of asking these questions and we only got time for a few more, that we will get to all these questions that I'm not... One is, yes the slides will be available, I'll answer that 'cause several people ask that. So you can have that. Another one is Clarksville light and electric utility, while not governed by NERC CIP, will it also be moving to their highly secure solutions? John or Dee, can you answer that?

54:56
John: Yeah, ask that question again please.

54:58
Don: Okay, Clarksville light, electric utility, while not governed by NERC CIP, will it also be moving to these highly secure solutions?

55:10
John: Yes.

55:11
Don: The answer is yes. Okay, Tom that's your answer.

55:15
John: It's simply critical, no matter what size utility or the ownership, the structure, it is just, in today's world, we've got to make our systems as secure as we can. I mean we're talking about critical infrastructure for a community, so you know simply the answer is yes.

55:33
Don: Here's a question from someone I think, international question. What is a water district, is it the same like different water zones? I'm assuming that's true, but I don't know. John or Dee, do you know the answer to that?

55:45
John: Well, really what we're talking about is water utilities, and there are different structures within. For example, in California they have water districts that are not necessarily owned by municipal itself. But it is owned by the customers that they serve, it's, in a way it's like an electric co-op that are owned by the customers that are attached to the electric utility. So that's what I had in mind when you use the word water district, as opposed to a municipal utility that serves a community. So we're essentially the same just with different perspectives.

56:23
Don: Okay, great, thank you. I think since we're running out of time, I'm gonna basically ask for any final comments or anything. Reiterate again, we're going to get to all these questions, we will follow up and get 'em all answered, 'cause there's still quite a few left, I really appreciate everyone's interest. But in the interest of keeping us on time, I want sort of a one-minute close, I'll ask anything you wanna say, John then Dee and finish with Albert. So, John.

56:52
John: Well, I appreciated the opportunity to share our experience with everybody who is connected into the webinar, and I'm open to answering any questions, I think my phone number is on the slide that's shown. I'm more than happy to take calls.

57:08
Don: Yeah, I'll just say that, yes, the numbers are up here for Bedrock for Brown and for Clarksville, and Melanie Moniz is our director of sales. If anyone is interested in a personalized demo of the Ignition platform, you got the number there, you have our account executives, we're happy to take whatever time we need to dig in and make sure you have a full understanding of it. Dee, how about last thoughts from your side for our audience.

57:30
Dee: Well, I just appreciate everybody's time to give us a chance to show off some of the things that we're doing with cyber security and industrial control systems. And I know we probably haven't answered everybody's questions, so we're happy to answer any questions as well, our contact info is on the screen.

57:49
Don: Great, thanks Dee, and then final comments from you, Albert and Bedrock.

57:56
Albert: Yes, I think that for me, personally speaking to all of you on this webinar, in the end this to me is a very personal challenge and a personal thing, and I really believe that there's a better way and there's a way that's user-centric. So the solutions, the technologies, they're not there for the sake of the technology, they're there and delivered for the sake of the user. In every possible way, we're trying to make your life easier and better and lower cost and safer to you and to your utility ratepayers. So I think that if you look at this in virtually any context of life cycle and cost and ease of use, these kinds of technologies, the open and secure approach to automation, based on the standards that we have and standards emerging in the future, this is the way to do it, and I look forward to working with some and all of you on this phone in the years to come. So, thank you.

59:07
Don: Thanks, Albert, and thanks also to Dee and to John, we appreciate all of the time today. Thanks so much for all the participation, I can't even imagine that we've got a lot of questions, I'll say again, we'll get to them, and with that, we've come to the close of today's webinar, and everyone has a good day. Thank you.

Posted on April 19, 2017