Design Like a Pro: Planning Enterprise Solutions

About This Webinar 

In today’s highly connected, IIoT-striving world, it’s likely that more people in your company will be accessing operational data, not just on the plant floor but remotely on mobile devices. This evolution in technology demands a new approach to project planning. It’s time to think beyond the SCADA system and the local site, and to consider remote locations, corporate systems, the cloud – and how it all fits together. 

In this webinar, Travis Cox from Inductive Automation will go over the important questions you should ask when planning an enterprise solution. His presentation will help you start and maintain a smoother development process that results in an open, interoperable, standards-based, and secure enterprise solution. 

You'll Learn About

• Define your project’s key requirements
• Plan out your project architecture
• Develop an effective plan of action
• And more!

Webinar Transcript

Speaker: Travis Cox

Technologies Impacting Industrial Organizations 
Let's get into today's subject. This is the latest webinar in the Design Like a Pro series. The first one that we ever did was in 2012, and I did that webinar, and it was about laying the foundation for successful HMI/SCADA projects. Since then, the way we think about projects has changed dramatically. Back then, we used to think that a large project was 20,000 tags and 15 people looking at the data. Today, of course, we see a much different picture, and we have to think way beyond HMI and SCADA. There's been an explosive increase in the number of mobile and smart devices out there and companies who want to utilize them. That has led to an explosion in the amount of data, especially with the introduction of the Industrial Internet of Things, and organizations are putting increased demands on that data as they try to get more value out of it. OT and IT worlds are converging, they're no longer isolated from each other. There's been a continued increase in the use of cloud solutions for things like machine learning and analytics, and the Industrial Internet of Things and edge computing have been introduced into the picture, open up whole new worlds of possibilities, but also present some new challenges.

Connecting the Enterprise
All of those factors have changed the way we think about designing architectures. Instead of thinking in terms of an air-gapped control system that is isolated from the rest of the enterprise, now we are thinking in terms of a connected enterprise. And when the enterprise is connected, you need to have a different type of architecture, and you have to put a lot more focus on security and scalability, interoperability and, often, that involves open standards. So the big question really is: How can we start laying a foundation that keeps the local systems independent while still allowing them to be connected so that their information can be utilized anywhere else in the enterprise, and do that in a way that is completely secure and scalable for the future? To start answering that question, let's make sure we understand what enterprise architecture is made up of.

Enterprise Architectures and Their Components
Even though every enterprise architecture is different, they all share some main components, in that they have critical machines or assets, local sites or plants, remote locations that often are over slow connections, like cellular or satellite, a corporate orphan or central system, often involves DMZs or Purdue model with the network, the cloud, and potentially others, but these are the main components of the architecture. Now, if you look at an example of a complete connected enterprise architecture, of course, this architecture here is showing the Ignition software in different places, but here we're seeing all of those components in one place.

So here, we've got local sites, we can have many of those or plants out there. Within that, we need to have a system that is independent, that can run at that location, and we don't need ... We don't really want to be relying on a connection to a corporate or central location. Even within that site, we have critical machines or assets that need to have their local control, local HMIs, but of course, have very important data that we need to work with. Of course, we're seeing a lot more at that site or plant level, a lot more devices, a lot more sensors that we are looking at.

We also have remote locations, and again, these locations that are over poor connections, cellular or satellite, so we have to be very conscious about bandwidth and latency, and of course, having local control out at that remote location. We've got corporate or central systems where we want to aggregate all that information and bring it up to a central system. We want to be able to visualize that data and give access to anybody within the organization. We want to do that in a very scalable way, and we want to also be able to manage our entire system from the corporate, to be able to see the health of everything that's going on, to synchronize projects, to do very important tasks from that corporate level.

Of course, DMZs are often involved here and they're different for different companies, as far as how they work with them. But it's important to think about these things, that's a way for us to separate out different parts of the network, but to keep a layer in the middle that we can pass information through or get access to other systems in a more secure manner, and a lot of IT departments, this is the way that they are going. And lastly here, the cloud. The cloud is now becoming a very increasingly important part of the architecture because, especially with Microsoft Azure, AWS, and the tools that they have that we want to leverage with an organization, and we're going to talk a lot more about that. So if we can get into this type of open, interoperable, standards-based architecture, we'll be able to truly achieve what IIoT promises, and be able to achieve the demands of that data on the business side.

Starting with a Solid Foundation
So a lot of things have changed in terms of systems and architectures. However, some things haven't changed. The old rule about project development is still true. Every project, whether it's large or small in scope, needs a solid foundation in order to be successful. So I showed you the architecture there, but every place in that architecture, there are applications, there are things that we need to deliver locally as well as deliver centrally. We’ve got to think about planning that properly. And when we just look at it from a single independent, isolated system, the planning could be quite simple, but if you were to expand that scope and look at it now, if I was to take that and bring it up to an enterprise, we've got a completely different picture that we're looking at. So we really have to lay that solid foundation, and hopefully today, we'll cover some of the planning you need to do in order to lay that foundation, and some of the questions you need to ask yourself.

3 Project Phases
To properly plan for a project's success, you need to break your planning process into three main phases. You need to define the projects that you are working with, plan a solid project architecture, and create a successful plan of action. Now, I know a lot of these will be pretty obvious, but it's amazing to me to see out there, when new projects are being looked at or developed, how they don't really fully create a plan of action and they don't fully look at what that product is going to entail across the enterprise, and so it's really important to sit down and make sure that these plans are looked at, at the beginning.

Phase 1: Define the Project

So let's dive into phase one: How do you properly define your project? It's really a matter of asking ourselves the right questions, and about things like function of the project, users, hardware and software, security, and a lot more. In Sales Engineering, we deal with a lot of customers who are trying to figure out how to build some of these applications, especially from an enterprise perspective, and all we really are doing is asking millions of questions, and we're trying to ask the right questions to get them to think and to understand what the full details of that particular project, and there's a lot of questions that we need to look at, and unfortunately, it can be be a little daunting to try to address every single one of them, but if you take it into strides and take it into, and break it down into small pieces, it will really come together. But it's really making sure that you have answers to these questions that are important.

Questions to Ask About the Function
So you’ve got to first think about the function of the solution you're building. A project is always the answer to some sort of problem or challenge. It can be a communication issue, an inefficient process or a variety of other problems. So you’ve got to ask yourselves, "What do you need the project to do? What needs does the project have to satisfy?" Think about the needs at every level, from that critical machine, to the site, to the remote locations, to the corporate or central location, of course, also to the cloud. Oftentimes, when you're looking at a project just at a local level, you might think that it's the only place it's going to reside, but then where it has critical or data or information for the business needs, now it's going to be brought up to a corporate or to the cloud. So you’ve got to think about the needs, regardless of what kind of project you think it's going to be, at every one of those levels, and that will help you establish the goals of the project that you need to accomplish, now and into the future.

It doesn't always have to be accomplished all at the same time. That's a common pitfall that a lot of customers go through is, they think they need to figure out everything at the beginning. But I like to look at it from the operations, from the plant for the OT up, solving all the problems as we go along, creating that solid foundation comes from that level, and if we have a really successful local application, but we thought about it in terms of bringing it to the enterprise, it's easy to get that information up and to utilize it more effectively. 

Questions to Ask About the Users
We also have to think about users. Who will be using the project? What are their roles in the organization? An operator on a production line will probably use your project differently than someone in an administrative role. If you know who's going to use the project, then you can tailor it to fit their needs, or you may identify multiple projects that need to actually to be created.

So again, asking yourselves questions like, "What information do users need to do their jobs better?" Understanding what information they need will help you determine what to include in your project. You should also think about whether users will interface with the project on a computer screen, a panel view, a mobile phone, a tablet, or on all of these devices. Today, mobile will definitely be part of the equation, so how will you provide an effective user experience on mobile devices, and do you need to get a UX or UI expert involved to help do that? And we've all been developing HMI systems and screens for a long time, and we've been using various standards and methods that are good, that work in that realm. But now, applications are becoming much bigger than just HMI/SCADA. And we're seeing a lot more in the way of having folks who are really experienced with UIs and the experience of the application and developing that plan and working with them to really make it the easiest and most effective project that you can. So we certainly encourage that, and at Inductive Automation, we have a lot more UX folks that are helping us drive how the software, how you develop the software, but that same principle gets brought into the applications you guys are building for your systems.

Questions to Ask About Hardware
You also need to think about the hardware. For instance, you should know the server, CPU, memory needs, hardware drive requirements. What kind of hard drive? Is it SSD? How many IOPs do you need? How much data are we going to be storing in there? Estimating that for a month, for a year. When are we going to be doing things like backups? And, are we doing actual disaster recovery testing to make sure the back-ups that we have are actually utilizable when these disasters actually happen? There's a lot of questions that we can start asking there. But of course, what kind of hardware do you have to work with? How many devices do you have? What kind of hardware do you need to get? Especially in the terms of edge gateways now, introducing more hardware in a distributed way is very useful, but what ... There's a lot of options out there in the world. What do we get, and can we get multi-purpose hardware that can serve maybe a networking and software solutions at a local level, all in one system that's easier to maintain and better for maintenance, as well with IT? 

What kind of peripherals need to be connected to the control system, such as other software packages, OPC servers, sensors that you need to bring in. What standards do those sensors talk, do they talk open standards? Is it based upon open or interoperable standards? Is it plug-and-play? Because really, it should be. Today, we really are starting to think of utilizing open standards and interoperable standards so that we can buy best-in-class across the entire organization, fit that in and allow it to be a member of that architecture, be plug-and-play, be easy to work with. Now, I realize that's hard to do when looking at legacy equipment, so we have to have a plan of getting legacy equipment in, as well as to be able to buy new equipment that's going to be plug-and-play.

We don't want to have these two competing fields of legacy and new, where we think of them as two different things, we’ve got to think of them as one connected enterprise, and that is really important, so that our system's agile, flexible, distributed, easy to replace, and easy to add. And that is really why initiatives like the Open Process Automation that ExxonMobil has started is really there, is because they don't want the next big DCS system that is very hard to replace or expensive to replace. Looking at it from a distributed system, where we can bring those in and having things work naturally out of the gates, without any custom programming, is extremely important. And again, at each of the levels, what do we need at the critical asset? What we need at the plant? What do we need at corporate? What do we need at remote locations? What do we need to do to get information to the cloud? In the cloud, what are we going to be working with? 

Questions to Ask About Software
As you can see, a lot of questions come up, a lot of things you have to take into consideration, but again, you have to break it up into different pieces. Now, moving on, we've got more questions to keep asking. We’ve got to think about all the software that's involved as well. What operating systems are you going to be using? Knowing that will help avoid any compatibility issues with some software packages. You do really want to get software packages that are cross-platform, that can work on new versions, as new versions of the operating systems come out, that they can just be easily transitioned there without nightmares of re-doing projects. Of course, what HMI/SCADA software you're going to use? When it comes to that, obviously, we have a lot experience with Ignition, on the HMI/SCADA software. And so there's a lot of, that we put in there to make open standards, IT-based licensing model that's simple, and I think with any kind of software you're choosing, you’ve got to think in terms of that nowadays, because there are just a lot more out there, a lot more people that need data, a lot more data that's there.

What other software are you going to use? Especially when it comes to maintenance management, to MES or ERP packages, other IIoT solutions, they're on-premise or in the cloud. How are you going to integrate those with our projects, especially other vertical software? What does the integration look like? Is it something like MQTT? Is it web services? What is the integration? And if the integration is going to be a custom programming, you may want to avoid that and look at another software package that is easier to bring into the system, or upgrading to a newer version if they have better integration or standards built into it. Again, looking at the open and interoperable standards, that's just always important, and defining, of course, the software needed at each of those levels again. So as you can see, every question we have to ask ourselves involves every level of this architecture.

Questions to Ask About Data
Now, data is a huge part of your enterprise. How much and what kind of data are you capturing now? How much of what kind of data do you need to start capturing? There's statistics out there that 90% of the data is left in the field because we don't want to bring it to a control system because it might overburden that control system or overburden the controllers, or the network that we're transferring, we don't have enough bandwidth, or there's high latency involved because of the legacy nature of whole response. There are a lot of things that are there, but that data is valuable. We need to be able to turn that data into information, so we have to start capturing that information. What is the data flow? Is it going to be one direction, or bidirectional? Or is it going to different places at the same time, being in parallel, sent to multiple locations? And how will you balance the increased business demands on the data while maintaining operations? That's really important. We’ve got to make sure we continue having our local HMI applications, SCADA applications working really well, maintain those, but then be able to increase the demands for the data on the business without turning your SCADA software into middleware, and without burdening that operation system.

Questions to Ask About Security & Redundancy
Of course, you have to think about security. One of the most important things today, how are you going to authenticate this to the system? Things like Active Directory or ADFS, or maybe federated identity systems like Ping or Duo. Should you use two-factor authentication, which many systems are going to? Or multi-factor with single sign-on opportunities? What type of encryption will you use? Which connections will you encrypt? What kind of auditing will you use? How will you protect your operating system? How will you secure your device connections? What are the security needs at all levels, again, that we have to worry about? And there's a lot more questions here. How are we going to lock things down to make sure that people are only given access to what they need to get access to? And with now encryption and all the connections that are certificate-based, managing certificates is also a challenge, and we have to be able to make sure that operations are working with IT folks. It's really the two that have to work together, because they're the ones that are going to help maintain these systems, and if you, really, you have rifts, then it's going to be very difficult to have it connect to enterprise. But the more you can work together, the better it's going to be at the end of the day.

Especially when it comes to security, because they're putting a lot of focus on security in the business side, we’ve got to do the same thing on the operations side as well. And how about redundancy? How much redundancy do you need? How much more do you need? Is it software-based redundancy or hardware-based redundancy? Where do you put that redundancy? Oftentimes, we see people putting redundancy at the corporate or central locations. Well, it's more important to put it at the operation, at the lower levels of that. So it's important to look at that as well across the board. 

Questions to Ask About the Cloud
And the cloud, speaking of data, will you be sending data to the cloud? What will the use of that be? For analytics, machine learning, data storage, streaming data, data lakes, business intelligence, what are we using? Which cloud service are you going to go to? We're seeing a lot of AWS and Azure, but there's also Google Cloud, IBM's IoT hub or cloud platforms, there's a lot out there. 

Questions to Ask About Communication & Latency Issues
And there are communication latency issues to think about. How will you handle lost communications, slow communication, high latency? If you've got critical machines at these sites, how will you keep those machines running when a communication loss occurs? And that becomes very important.

Questions to Ask About Management
And of course, management is a big question you have to ask. How are we going to roll this out everywhere and be able to manage it properly, and who's going to manage it, and what does that look like? That's a big question that not many people are asking these days, because they just assume that certain people are going to manage it at different levels, which means that they're forgotten, systems that are forgotten, because it's not managed by proper people across both sides. And scalability, think about scalability during the planning phase so it won't become a big problem later on. I see this happen time and time again. We don't really look at in terms of scalability. We potentially get hardware that has no room for growth in the future, or we put things in that are proprietary, that allows to scale, or whatever it might be. We have to, in order to address this properly, think about it in a scalable way, so that we don't have to make those adjustments later or have to redesign our entire architecture, and I'll give you some tips on that today.

Questions to Ask About Deadlines
And last but not least, what's the deadline? When does the project need to be launched? It's not unusual to have a demanding deadline, but can you realistically achieve the project goals within a given time frame? If not, maybe should just the deadline or adjust your goals. We see way too much where they're trying to be too demanding, too quick, and that leads to bad decisions that have to be made or shortcuts, and oftentimes, that means when you connect to enterprise, we have to re-do things.

Phase 2: Project Architecture 

Alright, once we've asked ourselves all those types of important questions and we've answered them, we can move on to phase two. Or at least, we have a plan of how we're going to answer those questions. Phase two is all about breaking down your architecture to different components, visualizing how they all work together, making inventories of the necessary functions, screens, data points, and organizing everything in a consistent way. And I want to give you some bit of tips on that here today. So again, coming back to the architecture, you can see all the different components in there. Now, we talked about some of the important questions you have to ask, but what does that look like in terms of actual details? How do you choose levels? How can we fit all this together? And hopefully, I can give you an idea of that through just some tips.

Architecture Tip 1: Protect Legacy PLCs by Adding Edge Devices
Tip number one is basically protecting legacy PLCs by adding edge devices. As we all know, legacy PLCs, with their poll-response protocols, they are inherently insecure, in that, if they're on a network and somebody knew an IP address and knew the protocol that was underlying that PLC, they could do some harm. So it's our responsibility to protect those PLCs, and we often do that by air gap systems or by putting firewalls in place and whatnot, but there are so many attack vectors that we really have to try and limit this as much as possible. And a great way to do that, at least from my opinion, is to have edge gateways.

As you can see in this diagram, look over here in the sites with these legacy PLCs, we could put an edge device and there's different options out there, hardware and software-based. In this particular case, I'm showing Ignition Edge, but it can talk to that PLC locally. So literally, the PLC's only connected through ethernet to the edge gateway, and the edge gateway can then bring that information up to a central system through a much better protocol, such as MQTT. We're going to talk more about that in a moment. And then, when you look at getting new PLCs that would either support MQTT or OPC UA, then they'll just be an inherent part of that network, and there's already security built into that, and that was thought of from the beginning. So we're talking about the legacy ones that have those potential attack vectors. We want to minimize those as much as possible.

Architecture Tip 2: Utilize the MQTT Protocol
Number two is to utilize the MQTT protocol. And I know a lot of you may not know what MQTT is. We, of course, do a lot of presentations and webinars on this subject. We really think this is the way to go for true enterprise systems. It is a machine-to-machine data transfer protocol and it's quickly becoming the de-facto standard or the leading messaging protocol for IoT. I thought I'd just briefly touch upon that. The idea is that we will decouple our application from our devices. Instead of having a SCADA system talk directly to the PLC and pull that PLC, we're going to have the PLCs publishing the information up to a broker, to a central system, message-oriented middleware, that then many applications could subscribe to and get information from in a true way. Very similar to what's done in the IT space with messaging buses, enterprise service buses, we're doing that for the actual industrial data that we're dealing with.

And why should we use it? A lot of reasons. Decoupling is an important one. We can have SCADA decouple from the devices, so that SCADA can only use what they require to maintain that operation, maintaining that functionality. Why? Then the business can get data from it without affecting the operations piece, and the fact that we could have multiple applications getting access to that data. It was designed for low bandwidth. It was actually designed for oil and gas when we're moving data through satellite, so it's a report by exception, only data is being pushed when things change and it's very small. The payload or the actual header is two bytes and rest of the payload, the payload is very much designed to be as small as possible, and you would see that with looking over the wire.

It has TLS security and Access Control Lists, so security is an important or an inherent part of the protocol, so that if we're putting that in front of the PLC, we then now have taken a legacy poll response protocol and moved it into a secure channel that we can then deliver to places, and in fact, deliver to the cloud as well. If you're talking about actually leveraging the cloud and getting real data to the cloud, MQTT is really the only way to go that's the most effective way of streaming real-time data in a secure way. And it's outbound connections only. I'll talk about this again a little bit later, but this is really important, in that the connection's made from the local level, the plant floor or the asset level up, there's no need for firewalls coming down, especially if you're going to deliver information to the enterprise, I do not have to have any firewalls, any ports open of that firewall locally, it's only going out. But of course, it does allow us to do reading and writing.

Stateful awareness, so we can know the state of the connection to the PLC, that is extremely important for control systems. And the devices, the edge gateways, the sensors, they become the single source of truth that everybody is going to be looking at, and it delivers that information that we need. And it's very plug-and-play, because I can plug a new device in, it can publish data to a broker, it can automatically be discovered by applications, and we can start using it in a very simple way. So as you can see, this is an open standard that allows us to... We can get our legacy converged over to it as well as getting new information coming in, that is thought about as one unit that allows us to effectively deliver data to our enterprise. And one important point I wanted to make about MQTT is that the reason this is all possible is because of the actual payload specification called Sparkplug.

I don't want to go into too much detail, but you think of HTTP, that protocol. HTML, it was the actual definition of the data being transferred over HTTP, and that has opened up, unlocked a lot of potential in the web world. We're talking about MQTT with Sparkplug, being that for MQTT for mission-critical, real-time environments, for control systems that importantly have all the metrics and metadata that we need from that tag, identifying what that tag is, where it comes from, what the engineering units are, what lows and highs are, that we can then deliver it to applications. So it has context, it has state, and we can use it for these kinds of systems.

Okay, so where is that being utilized? As you could see here, we're utilizing it in a lot of places in the diagram. It's a little bit small, I realize, but these edge gateways here, talking MQTT, bringing that to a system at the plant. The plant can be sending data through MQTT to our corporate. Corporate could be sending data through MQTT to the cloud. Remote locations can certainly send data through MQTT. We've been utilizing that everywhere, whether it's planned for or whether it's getting information over the WAN, now we can do that very effectively, and leveraging all that real-time data in different places, and different applications that we want to work with. That's where we're using it here on this diagram.

Architecture Tip 3: Maintain Local Functionality But Still Have a Connected System
Tip number three is, maintaining local functionality, but still having a connected system. And this is obviously important in control systems, this is something we do a lot, where we have a critical machine that has a local HMI on it. So that it's not relying on the connection to a central plant server or relying on a connection to a corporate server, it has functionality. In this particular case, I'm showing you an Ignition Edge panel which is local HMI. But also, with the MQTT, there's some store and forward of data on that local level. I can cache the information and forward it to the plant or to the corporate location when the connection is back up again. So we can really handle that loss of communication, not only at a plant level, but more importantly too, at a remote location. And that remote location, it could be over cellular or satellite or poor connections. The report by exception is really important, but also to maintain that local functionality at that remote location.

So we all have to deploy some sort of software there, but we want to make sure that the software we're deploying locally not only can deliver information in an effective way, but allows us to have that project there that is not a complete island or gap in the system, that we can manage that centrally, and that we can update that easily, and that we can make changes to it, whatever. We need to have a local functionality, but it can't be thought of as this thing that's forgotten. It has to be maintained, it has to be worked with, it has to be connected to enterprise. And when you look at software, that has to be an important part of it.

Architecture Tip 4: Use Redundant Hardware and/or Software
Tip number four is to use redundant hardware and/or software. I talked about this before. You'll notice in this diagram, I have redundancy at the site level, also potentially at the corporate level there. I didn't show it in terms of these edge gateways, just because I was trying to talk more about the protocols, but this is where I was getting at, is that oftentimes, redundancy is done at higher levels and forgotten about at lower levels, especially with legacy PLC, if you put an edge gateway in there, unless it has some hardware redundancy or built for ruggedized use, you may want to think about having some sort of redundancy, but obviously where it makes sense. So I always look at every system in terms of, "Do I need to have this there?" Ask myself, "What would happen if this machine was to fail? Is that going to be an issue for the enterprise?" If I can't lose any of that data, and I have to have something right there all the time and automatic failovers, then the answer is, "We've got to have redundancy." I know there's more cost to that, depending on hardware and software that you use, but you need to have a reliable system too, going forward.

Architecture Tip 5: Know How to Deal with Poor Connections; Use Bandwidth Efficiently
Tip number five is knowing how to deal with poor connections, and using the bandwidth efficiently. And that's, again, with this remote location, being able to utilize something that can store and forward data and do that effectively over that connection, that has bandwidth considerations built into it. MQTT just really, certainly, takes the cake for that. It makes it very easy in sending the data over there. And even OPC UA, to some extent, does have a lot of that as well, so we can get data up in different ways. We just want to make sure that we're able to leverage a better protocol so that we can not only get data faster, but that we can get access to more of that data at that remote location, because if we have restrained bandwidth, I want to make sure I can utilize it to send more data rather than a small amount. I want to get access to that 90% of that data that's stranded out there.

Architecture Tip 6: Have a Management Plan; Choose Software with Management Capabilities
Tip number six is to have a management plan in place, choosing software that comes with these management capabilities. And this is extremely important when it comes to software, and even, in some cases, hardware and patching and all of that, you want to have a central way of doing that. You look at companies like Moxa and companies like Hilscher and Opto 22 and many, many others. I'm leaving out a lot of those there. There's a lot of other companies who not only are deploying these hardware solutions out there but have software to manage those hardware solutions from a central location. So that's important when you're looking at choosing hardware. Choosing peer software, like a SCADA system, you also have to have that same functionality. And I know I'm biased here, I'm talking about Ignition, but Ignition has what's called enterprise administration, and so not only can I connect all these systems, all these Ignition servers together in our network, and that they can also be connected through the Purdue model through proxies in the middle and then having DMZs, all of that, but we can connect them all together, and when we do that, we can check the health of all these servers.

So here, centrally, I can check the health of everything that I've got, I can do remote diagnostics, I can centrally do licensing, I can take backups for disaster recovery, I can centrally upgrade those Ignition servers that are everywhere, as well as I can take a project that was developed at a corporate level and make changes to it and deploy that to every place in the architecture. And so that just becomes an important part of this that, the more you connect the enterprise, the easier it has to be for management and for updating them, for patching them, for all the stuff that we know we have to do. And certificate management, these are the new challenges that are introduced when having distributed architecture. Before, it was just a simple server out there, it was never connected. We might export data to CSV and do something with it, but as soon as you create real-time connections, you've got a lot more to consider.

Architecture Tip 7: Use Layers to Keep Networks Secure (Purdue Model)
Alright, tip number seven is using layers to keep networks secure, and we're seeing this a lot more now with networks that are using the Purdue model. Again, that's a DMZ that I was referring to kind of over here. Oftentimes, this in the middle, like a layer 3 for the plant floor, layer 4 for ERP, for business, and in the middle is a 3.5, and you want to be able to use that. Now, you have to have software that can work in that way with proxies, or sending data through that that allow us to work with it efficiently, but using these can allow these systems to be connected in a secure way where we can remove that layer if there was some attack or some risk that was identified. And that comes down to security as well, and security is incredibly important, as we're going to talk more about. You’ve got to think about every level, everywhere you are, and there's a lot of best practices involved with that.

Architecture Tip 8: Have Scalability at the Corporate Level
But also, tip number eight here is, have scalability at different levels and, very importantly, at the corporate level. And that's something like over here, I'm showing a diagram of the corporate where, if I'm taking a lot of data from multiple locations, remote locations or plants, I’ve got to be able to handle that amount of data and store it efficiently in one or more databases or data lakes or whatever it happens to be. So having the ability to scale out IO and scale up the management, also scale out front-end servers using things like load balancers to get access to that data from multiple people who want to access it … We're just getting into bigger systems, and when you get into bigger systems and more demand on who wants to look at it, we've got to be able have scalability on the software and on the hardware fronts so that it can work effectively and to be able to utilize databases or data lakes in a much better way.

Architecture Tip 9: Have a Plan for the Cloud
Which brings me to having a plan for the cloud, and this is incredibly important because everybody wants to leverage cloud, they think of machine learning, business intelligence analytics as those buzzwords they want to go into, but they don't know how to actually approach it, what they're going to be doing with that. And so, once you identify the problems in the case, what you're trying to solve, then you can start looking at the different cloud platforms, what they have, and figure out how we're going to get data to the cloud efficiently.

And when you look at something like AWS or Azure in particular, IBM IoT and their cloud platform or Google Cloud, at Ignition, we have direct injectors to put data direct, stream real-time data with that metadata to those systems, so that we can take advantage of all of their tools. But once we stream it to them, they’ve got to go into their cloud platforms to figure out how to utilize that efficiently and work with it in there. So it's not just enough to say we can stream it, we got to do something with it too. You have to have people who can actually turn that into real reports and information up there. And we also see, sometimes, bringing Ignition servers or software, HMI/SCADA software, to the cloud, not for control but for central visibility of data, or reporting, those kinds of things as well. So again, having a plan, knowing how we're going to deliver information to it efficiently, whether it's using MQTT, or whether it's using web services like REST or SOAP-based APIs, understanding those are half the battle.

Make an Inventory of Necessary Functions
Okay, so those were some of the architecture tips that I have, but to develop a project that will exist inside of those architectural components, we need to make sure that things are well thought out at the project level as well, though. Part of doing that is making inventory of the individual functions of the project will need to perform. Functions might include real-time task control, historical data logging, charts and graphs, reporting, alarming, auditing, batching, rescue management, analytics, machine learning, business applications, OEE, SPC, edge-of-network data collection, and so on. You need to organize these functions in order of importance. You can determine a function's importance by how much it will be used and how critical or crucial it is in resolving the problems and addressing what it's intended to do, and prioritizing your list will help you create a plan of action, as well as what you're going to be developing over time.

Make an Inventory of Screens You'll Need to Build
You should also make an inventory of the screens you need to build, and this is something that seems important to look at. We still have to build projects and screens, they're out there. So what are we going to need? What do these screens look like? Do we need multiple screens to deliver to different people on different sizes of displays, and what will they want to use it for? How are we going to present that information to them? Those are important questions, and nowadays, we got to look at the information being on desktops, TVs, tablets, mobile devices, all of that. So mobile-responsive design becomes important there, and also using web technologies, just browser-based applications. So as I said, that getting screen assistance used to be isolated before, but now, times have changed. So when we're building screens for today's connected world, we need to make sure that, A) it's mobile-responsive, and, B) that we build it with an object-oriented approach. We think that, "Can this be used? Can this be used somewhere else in the organization at higher levels?"

Screen-Building Tips
If the answer is, "Yeah, there's probably going to be uses in other places," we should build it with that mindset, build it from an object standpoint, a reusability standpoint so that, for the screen level and if the data that's delivering information to that screen can be easily replicated at different locations. So I know that goes with both project and data, and it's hard sometimes, because there are differences between different sites and there's different devices out there, different PLC manufacturers, all of that, and different ways that the screen needs to be presented, but the more you can use this approach, and the more you could actually model it that way, the better you're going to be for especially managing an enterprise solution. 

After making the inventory of the screens you need to build, you have to have a detailed outline for the project, the number of screens, the info of the functions of each screen. Again, nobody does this, but it's important to have some sort of identification of what you want to do. 

Identify Data Points
You've also got to identify the data points that are in your solution, and again, this is what I was mentioning earlier, it's really important to understand the data and where it's coming from, and we should, again, use object-oriented approach using user-defined data types, modeling tags. And I know if one location has a different PLC than another, and there are different addressing schemes or just different data, we still can model that as an object, as a UDT. There are still commonalities, and we can have those small unique details behind the scenes as overrides or as parameters into the UDTs so that we can still capture that consistency and have these. That bubbles up to our screen development and to the data we're delivering to the enterprises to cloud. If we don't start approaching it from that way, we're never going to have consistent data that we're going to be able to analyze from an enterprise standpoint, and looking at it across, comparing different sites. This has to be done, and done from the bottom level, and it can't just be looking at one system by itself, you’ve got to look at commonalities across the entire board.

Organization Tip 1: Name Servers Properly
A couple of tips in relation to project building and server configurations and tags is that, that are important here. One is naming servers properly. I see far too many systems where they just don't name things properly, and so if you put a system out there, that system should be named for its function, its use, its location. And so that what I'm looking at, especially from a management standpoint, I know exactly where that is going to be and where it's coming from.

Organization Tip 2: Connect Up (Outbound Connections Only) 
We want to make sure we're also connecting up and sending data out. This is what I was mentioning earlier with MQTT, and there are other parts that do this as well, but we can connect up. No reports in the firewall needed, so we can deliver information and do management at a central level without having to worry about those firewalls again, being able to work through proxies, potentially, with the Purdue model. So that's an important part.

Organization Tip 3: Name Tag Databases & Tags Properly
Naming your tag databases and tags properly is important. Having a nice hierarchy, define the metadata that goes with these tags, and have the consistency, as I was mentioning, is incredibly important. And MQTT offers, and Sparkplug specification offers a lot in the way of giving you this information and this hierarchy that's built into that specification. So it's a lot of experience that has been out there over the years that's made this open standard what it is, it's going to help us in this realm, especially when we want to deliver information to the cloud and leverage that information in a consistent way.

Organization Tip 4: Use Fully Qualified Tag Paths When Building Projects
Tip number four here would be using fully qualified tag paths with building projects. So really, this is just about, when I build projects and screens or templates or things, again, approaching from an object standpoint, I want to make sure that I can build it so that I can reference any system, any set of tags that I want that would be, of course, would be applicable to that particular thing that I'm building. Never build something that's hard-coded, unless it really is literally only for that one use case, ever. Outside of that, it should very much be built with that mindset.

Organization Tip 5: Use Best Practices for Security
And then, of course, best practices for security. There's a lot going on with this, but making sure that connections are encrypted and secure, making sure you have proper certificates that are trusted, making sure that we do role-based and zone-based securities. We lock things out to who people are, where they're located. Following all these important best practices, and there's a lot more than just that, but following these best practices and then leveraging these federated identities or multi-factor authentication systems becomes important, especially when we're getting data to more places.

Phase 3: Developing a Plan of Action

With all that done, we can move on to the last step of planning your enterprise project, which is developing a plan of action by creating a task list. Matching tasks with assets and putting tasks on a timeline, again, an important phase. 

Create an Ordered Task List
With creating a work task list, we've got to look at the inventory and functions and screens that we have written, then list them out, each step you need to take to complete each one of them. So for a specific, more specific example, if we were to create an overview screen, and looking at it again from an enterprise perspective, if we can use that overview screen in other places, I want to break that down to tasks like sketching a layout for that overview, especially with mobile-responsive, what is it going to look like, what's going to be on it when it's at a PC, a tablet or a phone, so that when we go and build it, it's built with that mindset. 

Create a navigation strategy by asking what's the user experience going to be like when getting around the application. Assemble a screen framework in the design environment, using project managements or SCRUM as we go along in building this thing, so that we have stakeholders who are able to provide feedback, especially operators, getting their feedback. So creating this plan of action, how are you going to break it down and go about it, is for any kind of project, it's going to help you, especially getting that feedback, because the developer is not typically the one using it, so we got to make sure it's developed properly across the enterprise, but that it functions properly for the people that have to use it on a day-to-day basis.

Match Tasks with Assets
Then you need to match the tasks on your list with assets. List out all the assets, including hardware, software, data, available personnel, match your task list up to these resources. If you're working on a team, match up the expertise of your teammates with the tasks they're best suited to. I see way too many instances where you put somebody on a particular task that they just aren't trained for and they don't get that job quickly, or it's not successful, especially in the MES realm, it's done a lot. Who can get the job done quickest with the best quality? If you're working by yourself, identify any tasks you are not equipped to handle. If you need help, tools or training to get the job done, that's the time to ask for that and get those resources, hopefully, or find integration partners or software partners that are going to get you into the right best practices.

So phase three, really, is an important part of all this, is just making sure that, after you identify all those questions, you start thinking about the different connections, how the enterprise is going to be connected, how you go about which hardware, what software, and of course, identify the projects and looking at what we're going to be building. A lot of this may be built already, but the plan of action for how we get from where we are today to where we need to go is incredibly important, especially with this new world of IoT, and leveraging things like MQTT. How do we get there from where we are? And it could mean not only work, but it could mean that there's a cost to that, but there's an ROI. There's a light at the end of the tunnel when we can get a connected enterprise and we can do more of it and again, thinking from the OT levels up.

Recap
So to recap: In today's connected world, we should shift from thinking about SCADA projects to thinking about enterprise projects, especially if it's way beyond HMI/SCADA for a lot of these systems, but that data is extremely valuable everywhere. Breaking it down to these three phases, defining the project by asking yourself the important questions about all of those different things, visualizing your project architecture, understanding what the architecture is actually going to be, what components are in that architecture, how it's going to work for you and, of course, developing that plan of action and making sure that things are designed or approached from that standpoint.