There are countless challenges facing water districts and utilities today: economic issues, insufficient resources, and tighter regulation, but obsolete SCADA systems will continue to be chief among them. Fortunately, it’s easier than you think to let go of out-of-date systems; and it doesn’t have to happen dramatically or all at once.
I recently co-hosted a webinar along with IA Public Sector Development Coordinator Annie Wise titled “10 Steps to Architecting a Sustainable SCADA System,” which featured two leading water-district supervisors, Henry Palechek and Jason Hamlin, as panelists. The four of us discussed clear, actionable directives any water district can take to build up a sustainable SCADA that will last for the long haul.
My biggest tip for leaders hoping to make an enduring impact with control systems is this: start small. There are easy, feasible actions that will not only update your system for the future, but enhance your current system, without causing major shutdowns or downtime. Instead, you can slowly architect future-proofed SCADA that will work for today and tomorrow.
Throughout this post, you’ll find quotes from Henry Palechek and Jason Hamlin about how they have implemented sustainable SCADA within their systems.
1. Hardware (PLCs)
It’s easy to get to the point where your system includes hardware from many manufacturers that talk in many different protocols. Unfortunately, maintaining support and connectivity to such a wide variety of devices and protocols is not sustainable. This doesn’t necessarily mean you have to replace all of your old equipment; just start adding new equipment on an open protocol such as Modbus, OPC, or MQTT. This simple step takes a good deal of confusion out of your PLC operation and moves you toward a unified system.
“We've had good success using the built-in Modbus driver,” said Henry, “At our plant, we have a number of PLCs, from small to large, and we have Modbus power meters on all of the larger loads. We’re able to communicate directly using the Modbus protocol to our larger VFDs so we can read out power for energy efficiency and more.”
2. Edge Devices
Once you’ve adopted a standard protocol, data collection should be much more streamlined, but what about any remote sites? If you have an offsite PLC connected over radio or cellular, polling is going to be limited by your bandwidth and latency — plus, if you ever lose a connection to that device, you’re losing data. An easy solution to this issue is to have an edge device installed at the remote site that polls the device locally and then reports only by exception. This means data is only sent up to the central location on change. If the network goes down, then the edge device can buffer the data and forward it up when the connection is restored.
3. Server-Centric Architecture
A sustainable system should be server-centric because it requires minimal software installation, rather than maintaining many installs on many machines. Instead, a server-centric organization only requires that the software be installed centrally. Because all data collection and visualization goes through that server, it’s a single point of failure, so redundancy is key to maintaining uptime in the event of server failure. The biggest pro of this architecture? A true server-centric system is licensed by the server and would be unlimited for everything else, as is the case with Ignition.
Jason’s system dramatically benefitted from this server-centric architecture. As he put it, “On one of our previous systems, we had a failure which resulted in an 18-hour outage of SCADA; operators had to run everything manually while we were trying to restore the backups… When we moved to Ignition we also moved to a virtualized environment with separate sets of physical hosts... We very rarely have any type of downtime... When we’ve tested I’m able to do full backup recoveries in a matter of minutes, so coming from 18 hours to minutes is a huge thing for me.”
4. Cross-Platform
Traditionally, SCADA systems have been tied to a specific version of Windows, which obviously creates problems as versions reach end-of-life and people are forced to upgrade their SCADA alongside their OS. These changes can be incredibly costly. By using a cross-platform SCADA system, you can run the server and client on any version of Windows, Linux, or Mac. This means if someone has a MacBook, it won’t prevent them from opening a dashboard on their laptop.
“When we moved into full-on production (with Ignition), we actually set up our primary server running with Windows and a backup server running Linux, not just because of the licensing fee savings... but it's a security best practice… There is a limited footprint for zero-day attack because we're actually running on two separate platforms,” said Jason on additional benefits of cross-platform capabilities.
5. Web-Launched Clients
Web launched clients are a crucial part of a server-centric architecture. Rather than traditional client screens that are individually developed, exported, then installed on each client machine, there is no install process required. Not only that, but updates are pushed out automatically, eliminating the need to repeat the process over and over. This functionality dramatically increases scalability and allows for rapid rollout of necessary project updates.
6. Templates/UDTs
As you update your SCADA system, it's valuable to implement changes that save you time. Templates and user defined types (UDTs) are an easy way to save massive amounts of time while building out your projects. It’s simply unsustainable to define each tag individually and recreate a similar visualization every time someone is configuring a screen. To be sustainable, SCADA should implement an object-oriented approach, so that data types can be defined once and used throughout the project. For example, if you’re building out a system with motors, you can create a UDT with all of those shared tags (see above screenshot for more details on UDTs). Now all you have to do is pass in the motor number to create additional motors. This makes it easy for your team to scale out a project without requiring additional support for screen creation.
Jason added, “Templates and UDTs are like water: you tend to not think or care about them until you don't have them. After using it for years the quickness of my development time, I can't really compare (to before). It’s like every system should have this because it just makes everything much faster, much easier… It’s been the only way that I've been able to support keeping my system running as a team of one person.”
7. Remote Alarm Notifications
Part of a sustainable approach is taking care of what you have. When something goes wrong, as it inevitably will, you need to be able to respond quickly. In a perfect world, you would have someone staring at your screens 24/7 to make sure the system is running smoothly. Often this isn’t the case, which is why remote alarm notifications are a crucial addition to a sustainable SCADA system. Ignition allows for an email, text or phone call to be sent directly to operators in the event of an alarm.
8. Automatic Reporting
In an unsustainable system, reporting requires high levels of attention and labor; by contrast, when reporting with Ignition, you’ll likely decrease the amount of labor required to generate high-quality reports. This is especially true if you have someone walking around your plant floor, clipboard in hand, recording stats. Not only are the opportunities for human error huge in this scenario, but in any larger facility, reporting could easily turn into a full-time job. Not only does moving to automated reporting dramatically increase report accuracy, it also saves time and resources.
9. IIoT and MQTT
If you’re caught up to all of the steps by this point, congratulations! You’ve built out a notably sustainable SCADA system; making it possible to expand beyond SCADA and into IIoT. While IIoT is certainly a buzzy idea at the moment, there are very real, practical applications in water/wastewater, since there are usually lots of remote sites connected over radio, satellite or cellular connections. IIoT is all about getting full access to as much of your data as possible, and it’s best done with MQTT. With edge devices and standard protocols, data collected at the edge can be published using MQTT, an ultralight protocol (read more on MQTT).
10. Capital Expenditure vs. Operational Expenditure
So, how does all of this affect my bottom line? In building out a sustainable system, you’ve saved money with unlimited tags and devices, and faster development and deployment of screens. You’ve eliminated unnecessary downtime with cross-platform capabilities and redundancy, and you can upgrade software at any time because you’re not tied to a specific OS. With all of these features, it’s now possible to fit maintaining your SCADA system into an operational budget. Your new biggest question: what to do with all that extra cap-ex?
Henry said this aspect has been very beneficial for his water district. “We deployed our system in-house... We didn't use any capital funding; in fact, we haven't used any capital funding since deploying Ignition…. Ignition has freed up our resources to help me address the issues of my aging infrastructure control my SCADA costs and create a sustainable SCADA solution.”
Want to hear more about how water districts are creating sustainable SCADA? Watch the webinar:
Have questions about the steps to building truly sustainable SCADA? Please leave them in the comments below — I’d love to discuss them with you.
