Don’t Get Lost in the Cloud: Tips & Tricks for Successful Ignition Deployment and Management

Speakers

James Burnand

Chief Executive Officer

4IR Solutions

Joseph Dolivo

Chief Technology Officer

4IR Solutions

With the release of Cloud Edition, it's never been easier to get Ignition running in the cloud. But are you ready for it? From security concerns to misconfigurations, there are plenty of pitfalls to stumble upon when managing applications in the cloud. But fear not, as help is on the way. Join the experts from 4IR in this session where they'll provide helpful tips and tricks for deploying and managing Ignition in the cloud.

Transcript:

00:04
Susan Shamgar: Hi. So my name is Susan Shamgar. I'm a Technical Writer at Inductive Automation, and I'll be your moderator for today's session, "Don't Get Lost in the Cloud: Tips & Tricks for Successful Ignition Deployment and Management." To start things off, I'd like to introduce our speakers for today. First up, a longtime member of the Ignition community, Joseph Dolivo. Currently serves as the CTO of 4IR Solutions, an Inductive Automation Solution Partner focused on cloud, Digital Transformation, and life sciences. For more than a decade, Joseph has focused on modernizing manufacturing by intelligently adopting state-of-the-art technologies and principles from the software industry. James Burnand is a 20+ year veteran of the industrial automation ecosphere, who has now turned his focus toward providing the infrastructure for manufacturers to reap the benefits of the cloud for their plant floor applications. He weaves cybersecurity, operational requirements, and management into 4IR Solutions' offerings and provides education and consulting for companies looking to begin their journey into a cloud-enabled and a highly automated OT infrastructure. Please help me welcome James and Joseph.

01:20
James Burnand: Thank you, Susan. Your payment will be after the session. We really appreciate that. Hi, everybody. Welcome to the session. Hello people live streaming. So Joe and I are here to talk to you about the cloud today. So we've talked all week about what we do in the cloud, but what we really want to do today is help you understand what are some of the considerations, what are some of the tools, and what are some of the methodologies that you should consider if you're going to be doing deployments in the cloud. So to start off, I'm going to review a little bit about that and go into a little as to why the cloud is in use today, what are some of the benefits, where are we seeing adoption taking off. And then from there, Joe is going to go into the real deep technical details about what things you can do, what tools you can use, and how to actually go about doing that.

02:08
Joseph Dolivo: Yep. We're excited. We'll get as deep as we can with the time that we have, but definitely save your tomatoes and everything else for the Q&A session afterwards. As long as my voice holds out, I will answer as many as we can, and we'll have contact info provided for future questions.

02:22
James Burnand: Alright. So let's get started. So why do people care about the cloud? I know we've been talking about it. It's become this huge discussion point. There's a lot of attention around different opportunities that are opened up, be they AI, be they flexibility, but ultimately one of the most basic things that's important about using the cloud is you only pay for what you use. So you're not buying a set of servers and computing resources that will have the capacity you need for the lifecycle of those assets, you're not buying five years worth of storage that you're eventually hopefully going to use five years from now plus your safety factor. You're literally paying for just what you're using and as you consume it, that price and that cost goes up. So controlling cost is really... If you think about why people are using the cloud in the first place, that's the biggest reason.

03:15
James Burnand: But the other benefits you get is that you are able to scale things. So not only do you get to only pay for what you use, but you have the ability now to theoretically endlessly scale those resources based on what the growth of a system is or the growth of the amount of data that you collect or the collection of different applications that you deploy. It also opens up opportunities with capability. So there are things that are just hard to do that you can go and install a service from a cloud provider that they do it for you. There's managed services, there's application functions, there's third-party plugins. There's all sorts of things that become remarkably easier to do when you take advantage of those precompiled and prebuilt resources that you can buy from a public cloud provider.

04:03
James Burnand: So what do we see people using it for and what are good use cases? So a lot of organizations that use the cloud, our folks, what we've seen in this conference quite a bit is people who have very distributed systems. So telemetry-type systems, places where it doesn't matter where my server is, everything that I'm collecting from is remote, that's a really great use case for the cloud. Or where there's a lot of focus on data and processing, and I need to be able to use more advanced functions and features to be able to provide the insights that I need. The other thing is that when you look at some of those services I described in the last slide, things like time series databases, AI applications, data warehouses, Snowflake, these are all things that become very easy to integrate with and use and take advantage of when you have the cloud.

04:48
James Burnand: So those data-centric applications just make a lot of sense to be able to use those resources for them. And then one of the things we... One of the most basic things we love using the cloud for is backing things up 'cause it's really hard to back things up in a way that it's easily recoverable, testable, and you can be sure that when it's time to go and restore those backups that they're available. The cloud is a fantastic and very cheap way to store long-term backups of systems that you're running on the factory floor. So what I will say though is just like playing soccer in scuba gear, it's not a... Just because you can, doesn't mean you should. You don't use the cloud for everything. And so what we found is that one of the really great opportunities, one of the really great options that people are starting to explore a lot more now is hybrid cloud.

05:38
James Burnand: So I grabbed a definition off of... I forget I Googled it, but a hybrid cloud is a computing environment that combines on-premises data centers, also called a private cloud with a public cloud, allowing data and applications to be shared between them. Really what it means is you install a piece of cloud in your building. So you put hardware in that provides a conduit, access, and ability to deploy those really cool applications that are precompiled, those services that the cloud providers give you into a piece of hardware that happens to live inside of a building. So a factory or a transfer station or wherever the local needs might be. So you get that low-latency, high-capability system that's running locally on site. You have the ability to cut the cord to the Internet and it still runs, but you get the benefit of running those cloud services down inside of the building.

06:35
James Burnand: I see it as being fairly revolutionary. I think it's still really new for a lot of folks. It's a concept and a way of thinking about deployment that not a lot of people are really that deep into yet, but I personally see that it's... I think it's going to be the future for a lot of the bigger systems. So who's using it today and what are they using it for? SCADA systems for distributed telemetry systems. We're seeing a lot of MES systems being cloud-deployed, especially things like OEE. We're working with our friends at Sepasoft on a number of different opportunities right now where there's, I want to be able to deploy across this fleet of facilities, I want to be able to create a consistent fabric of OEE application access and Ignition in databases.

07:22
James Burnand: And to be able to do that in some plants, it's super easy 'cause hey, they got great resources, engineers that understand what's going on, but it's really difficult to do in facilities where there's maybe not any sort of local support or they don't have people that are really understanding exactly how to build and maintain those systems. Using cloud or hybrid cloud for those sorts of solutions really makes it an equal playing field for all the users and all the locations that are going to have access to that application. The other piece that we're seeing is a lot of ingestion. So we saw some Snowflake stuff this week, which was really, really cool. We're seeing that there's this pull of all this information up to these data warehouses. Analytics tying together sales data and financial data in with production information in new and innovative ways that lets you make better business decisions and it's only being unlocked by the type of solutions that people in this room are putting together to ingest that information in. The other kind of piece to this is tying together with existing cloud services, things like ERP systems, cloud-based databases. There's just a ton of opportunity in pulling those things together. So that's what we're seeing today.

08:35
James Burnand: So challenges and risks, I would say the one thing to remember is the cloud is public. So when you go and you do a deployment, yes, you get access to all this really great technology, all of these applications, all of these things that you're able to do. But ultimately, if you're not careful, you are deploying those things in a publicly accessible location. There's lots of ways to remediate that, lots of ways to manage that. Really, what we find is the most critical part of that is making sure that you have a plan for how you're going to manage those assets. There's ways to be able to deploy in public clouds and have no external access to them, only internal to your facilities, but you have to plan all that stuff up front. So Joe's going to walk through all kinds of technology pieces around that.

09:21
James Burnand: I'm throwing the warning flags up and saying, just remember that it's public and that it's something that, yes, there's a policy in place for most major organizations to be cloud-first because of that first slide around cost savings, but it's not as simple as deploy and forget because if you do that, you're potentially opening yourselves up to all kinds of new risks and challenges that will unfortunately be potentially costly. I would also say that it's difficult to dabble in this space. So there's a big difference from what we've seen in being able to get something working versus having something sustainable and maintainable over time. So tools like cloud formation templates, which I know Joe is going to talk about, these are things that make it real easy for us to be able to build up an infrastructure in the cloud very quickly.

10:12
James Burnand: Even Ignition Cloud Edition lets you just start a virtual machine and run Cloud Edition and it's there and it's going, but you really do need to make sure that you're following best practices, hardening guide, best practices from the cloud vendors to ensure that you are putting in security as a consideration even for systems that you're testing, even for systems that you're just trying to figure out. Because what tends to happen, as I think many people in this room have seen, is I'm just going to start off small. I'll install Ignition here, and that's all it'll ever be used for. Six months later, it's like, "Well, I can use it for that. Well, I can use it for this. Well, I can use it for that." So you end up creating this burgeoning and growing set of applications. And when it's on-prem, the risk is a little bit... Well, it's a lot less because you don't have this public access. When you're doing that in the cloud, unfortunately, you have to be more careful. I believe Joe is going to take over talking now.

11:05
Joseph Dolivo: Well said. I think we're trying to differentiate between the ease of getting started, which is great for demos and learning and testing, and then production-grade systems. So we know a thing or two about production-grade systems. If you guys have seen the Data Dash that's going on right now, all that Ignition infrastructure is part of one of our managed service platforms called FactoryStack. What we're going to try to do is to take you through some of the lessons learned that we've had in working in this space for a long time before Cloud Edition was a thing, but then to give you some very practical takeaways that you can implement in your own systems, and also give you a little bit of insight behind what we've done and productized. And I will just say, coming out of the Technical Keynote, there are a ton of things that are coming in Ignition 8.3 that we are super excited for because it's going to make a lot of the stuff that we have to do now manually a lot easier for all of us.

11:52
Joseph Dolivo: So very, very exciting. Tried to categorize this into five different categories. Again, we could spend days talking about all of this, but they're largely broken down into networking, security, access management, data management, and cost management. And of course, especially with regards to network and security and access management, there's some overlap. So we've come up with a couple of different examples from each of these that we'll talk through. And again, as you have deep questions, please let us know and we'll go down into the weeds during the Q&A if we can. So I'll start with networking. So encrypt all the things. You hear a ton about encryption really in two different categories. There's encrypting things at rest. That's obviously important for data storage, making sure things aren't getting changed after the fact.

12:37
Joseph Dolivo: But also when it comes to networking, we're talking about in transit. So Ignition as a tool has great support for SSL certificates so that any traffic that's going into or out of your Ignition system will be encrypted, but it's not just Ignition. When you're deploying these production systems, you don't just have one Ignition gateway. Typically, you're going to have multiple Ignition gateways in a gateway network. The Ignition Gateway Network uses something called gateway network certificates that you can use to basically encrypt communication between Ignition gateways using the same principles that you use to encrypt your web traffic and all of that. So that's really key. And again, Ignition isn't just talking to other Ignition systems. It's also talking to databases, for example. So when you're configuring your databases, very important to enforce SSL encryption. There's a setting in the Ignition gateway configuration to do that.

13:27
Joseph Dolivo: And even more so, you can go down to the level of basically restricting access to certain ciphers. So I'm going to use certain cryptographic ciphers, I'm going to require TLS 1.3, for example. So focusing on encryption is a key part of everything that you're doing, is really, really critical. The other thing that you'll tend to hear about which is still very important and a good step one is to use a VPN. VPNs have been popular for a long time for good reason. They're a really nice, easy way to extend, let's say, an on-premise network into the cloud. Cloud providers have really good tools to make that easy, but if you just rely on a VPN, then you're doing what you call perimeter security, and we'll touch on security more in a minute, where you're securing the outside, and then as soon as somebody gets in the door, you now have... It's kind of free reign.

14:16
Joseph Dolivo: So a VPN is a tool, but it's a tool in defense and depth. So don't rely on a VPN by itself. Encrypting traffic, whether or not it goes through a VPN is important. So that's encryption. Limiting external connectivity. So we've got Ignition running in the cloud. Again, you probably have a database, for example. Best practices would suggest that you don't provide external access to the database unless you need to and typically you won't. So your Ignition system can be publicly accessible via web browser, mobile device, designer access, things like that. The database, you would probably want to be locked down inside of a virtual private network or a VPC depending on your cloud provider. I'll use both terms interchangeably.

15:00
Joseph Dolivo: And then there's a bunch of these cloud-native services that James had alluded to that are things like data lakes, digital twin services. And again, depending on if you're going to funnel all that data through Ignition, you don't want to have outside access to those systems. And the cloud providers provide really good tools, private endpoints, private link. Those are things you can use to basically expose even some of those managed services into your private network without having to go out through the public Internet which is the default. So highly, highly recommend that for anything that you're going to be doing which requires access from the outside. And the last one here is about minimizing hops. So especially for production-critical systems, getting data in a timely manner is very important.

15:44
Joseph Dolivo: And now we're not just talking about, oh, I'm sitting across from my server in my plant. I'm talking about having to go up to a cloud system and back in order to communicate. And the cloud is global so you can pick regions and then you can deploy things. I could be sitting here in California connected to a cloud server in Arkansas, which is actually what we're doing for the Data Dash here. And so by default, when you're starting to add these different layers of networking complexity into your systems, you risk introducing a whole bunch more latency to applications like Ignition. So one of the recommendations that we have if you're going to be deploying this inside of, let's say, an orchestrator like Kubernetes, which has been talked about a couple times, would be to look at the network interface that you're using to expose those workloads.

16:29
Joseph Dolivo: So for example, if you're using Kubernetes, by default, it deploys an overlay network called Kubenet, and it's got this virtual address space that's disconnected from everything else. It's introducing another network hop. The cloud providers provide integrations with something called the Container Network Interface that lets you expose the same IP addresses, same address space you're going to use for your virtual machines or for other kind of workloads, also for the containers that are going to be running Ignition. That reduces the network hop, makes your application more performant. Same thing when it comes to these complex architectures where you have load balancers in place. Every hop, every proxy you put in place is going to slow that down. So be very careful and selective about where you're introducing those kind of latencies. So we could have a whole session on networking.

17:14
Joseph Dolivo: That's a couple of highlights. Security, natural progression from talking about networking. Keep your systems up-to-date, and you're saying, "Well, of course, that's obvious." But when you actually look at the scope of systems we're talking about, let's take Ignition as an example. You've got your application, so you're going to be making changes to your application to fix bugs, to implement features and all of that. That application resides on Ignition, so keeping Ignition up-to-date, for sure. Doing that in a production system where... I love IT people, but you can't just push down security patches at any point in time. You've got a production system. You can't do that. So Ignition is a component of that and most applications are also built on a database. You're using the Sepasoft MES modules. It's built on a database.

17:57
Joseph Dolivo: Now, you've got to do those updates in tandem. So I need my database and my Ignition system to be in lockstep and if one of those is not in step, you think you're taking backups. We'll get to backups in a bit. Are they in sync? Are they cohesive? And now you're going down to a level below Ignition that's running in an operating system. Whether it's containerized or not, I got to patch that operating system. Maybe I've got an orchestrator like Kubernetes, maybe I've got add-on modules for providing other functionality. So looking at these systems as something that is living and breathing and you don't just set it and forget it is incredibly important. And to James's point, it's so easy to set something up once and then you forget about it and say it's good enough.

18:38
Joseph Dolivo: These air-gap networks don't really exist anymore. Maybe they never did, but nowadays it's not something to look at, especially when you're talking about the cloud. So reducing attack surfaces, the more stuff that's available on the public cloud, the more targets there are for attack. You go to shodan.io, you can see all the industrial OT network traffic that's available. It's terrifying, but you should check that out if you haven't heard of it before. So we want to do everything that we can to minimize the exposure to applications, to data from the outside looking at limiting external connectivity like we talked about as part of that. One thing I want to highlight within the Ignition ecosystem, Ignition has first-class support for containers. Containers are great because when you distribute a container, there's a couple of sessions on that at the conference. You're basically just distributing the minimum set of files that you need to run an application, and that's it. And you're decoupling it from everything else that's required like a kernel and everything else to run an operating system, Windows updates, all that kind of stuff.

19:39
Joseph Dolivo: So if your kind of target that you're deploying is basically these containers that have minimal packages installed, you're not having everything out of the box, you might get with a Windows updates WordPad, calc. So that really, really helps you to minimize that attack surface and it's, again, one less set of targets that attackers are gonna be able to go after. And then, of course, there's monitoring for breaches, and I can't tell you how many times two years down the road, somebody will find out that, oh yeah, somebody has been in our systems and they may have modified our data. We don't know what happened. We're gonna have to do a product recall or put out an announcement. So doing active monitoring is really, really important. It's something that there's a number of tools available to do that.

20:20
Joseph Dolivo: There's some that are kind of OT-specific, and you'll see 'em inside of OT networks from companies like Claroty and Nozomi and things like that. But there's also a lot of IT-centric tools that really work well in the cloud environment. A lot of them are based on machine learning to do like anomaly detection. So I'm gonna kind of pick... These are the sort of typical traffic patterns that I might be seeing in a cloud environment. If all of a sudden I see a huge spike in network traffic, or if I see access logs from users or accounts that I don't tend to see, maybe I raise a flag, I send a notification, I require manual intervention. And then tuning that in a way that you're not getting so many false positives, that is the same problem we talk about with alarms all the time.

20:58
Joseph Dolivo: It's, "Oh I've got so many alarms, I'm just gonna ignore 'em all." So there's a balance there, but the fact that you don't just kind of set this up and ignore it, you have to be actively monitoring for breaches. So super, super important. Again, we could have a whole session on security alarm. Let's talk about access management. So there was a question that came up in the Technical Keynote talking about using YubiKeys for authentication with Ignition and things like that. Access management is hugely important. And another universal principle that you'll hear, and it ties in really, really nicely I think with Ignition is to practice the principle of least privilege. So in terms of user accounts, that means if I'm gonna be authenticated and authorized to use a service, I wanna be provided with the least amount of access that I need to be able to do my job.

21:44
Joseph Dolivo: And that's for two reasons. One, in the case of kind of a malicious actor, that reduces the damage that can be caused if that account is compromised. And it also just helps people from kind of shooting themselves in the foot or doing something by mistake that they wouldn't ordinarily try to do. So for example, in the kind of Ignition roles, you may say, well, I'm only gonna give an operator certain roles so they can't accidentally change the configuration of the system. If I'm an administrator, I may have elevated roles, but we also tend to just say, you know what, I'm just going to use an administrative account that has access to do everything because it's too much work to go through a process and then you end up getting in trouble when that happens.

22:24
Joseph Dolivo: So enforcing roles in a way that is consistent and clear is really important and there are tools that you can use to do that especially if you are taking the management of that outside of, let's say, just Ignition. You can use something like... Entrada [Entra] ID is what it's called now, but I never get it right. It used to be Azure AD, so basically the cloud extension of Active Directory, and you can have all of your groups and roles centrally managed across your organization. And then you can have the concept of, let's say, a supervisor and a supervisor can have certain access granted in Ignition, certain access granted in other applications, your ERP systems, your CRM systems and things like that, and you have that all managed in a single place.

23:03
Joseph Dolivo: The last part on principle of least privilege is that it doesn't just apply to named user accounts. It also applies to, let's say, service accounts. And so this is an example. We'll talk about databases more in a minute, but when you're configuring access to a database, that database may not need, or that database user account may not need the ability to delete records. Maybe I can only do inserts, especially for audit trails. I'm gonna be able to insert into the audit log. I don't wanna have somebody that can update or delete from those. So think about the principle of least privilege in terms of the system accounts as well in addition to named users.

23:37
Joseph Dolivo: Password management. I'm super excited. Again, Technical Keynote talking about using a system like HashiCorp Vault, where you can have the dynamic password authentication. Right now, there are certain accounts like the database connection in Ignition, which is more or less kind of hardcoded. It's sort of encrypted in the configuration, but some of those things are kind of hardcoded. But for other things like logging into Ignition, the safest way to manage passwords is to not manage them, and again if you're using a system like Entrada [Entra] ID, or AWS, IAM, or OKTA, or Duo, or some other system, you've got an enterprise security company whose stock price and revenue is based on them doing a good job with all of that. So we recommend not having to manage it yourself. It's one less thing you have to deal with. So for our platform, we don't see any passwords at all from users. We say, nope, we don't wanna deal with it.

24:30
Joseph Dolivo: And then of course, monitoring and auditing access. So Ignition by itself, you configure an audit log. It logs a whole bunch of different events that are occurring by default, which is great. You also have a script function that you can use to add additional logs manually based on things happening in your application. And depending on again, the system you're using for identity and access management, you could also have sort of a central audit log in the cloud that you can use to monitor. So every time somebody logs in, every time somebody asks for elevated privileges, so there's tools like PIM, Privilege Identity Management, where maybe I'm gonna be given read-only access to a service, and I have to go through an approval process to give me temporarily elevated access rights to some other system. Well, that's gonna be audited and logged and it's maintained for a certain duration of time and then that'll be it. So again, active monitoring, similar to threat management when it comes to security. Really important for access management.

25:23
Joseph Dolivo: A couple more here, data management. So take backups and again, that sounds great in theory. Backups include a lot of different systems. And Ignition's actually really, really great in the fact that you can go in the gateway configuration page, you can schedule backups to be taken on a schedule, and if the volume to which you are storing those backups is, let's say, cloud-replicated, that's great. You can get cloud-based encrypted backups, multiple availability zones and multiple regions out of the box really, really easily. Again, most systems aren't just Ignition. There's gonna be a database component, there's gonna be other systems that you have to take and some systems are not as nice to... They're not as kind of allowing for doing live backups like Ignition has and the official kind of application process for doing backups is I'm gonna spin down a workload, and then I'm going to copy a volume somewhere else and I'm gonna spin it back up.

26:16
Joseph Dolivo: So we have to do some of that with manual pipelines and things like that. But if you have the ability to kind of coordinate the backups of all your systems together, really, really important. And then the backup's no good if you take it and then two years later you need to get it and you realize that the backup failed, or the backup was incomplete. So it's really, really important, especially for production systems, that you are doing regular verification of those backups. A really easy way to do that, especially if you're using Ignition in containers, take a backup of a database, take a backup of the Ignition gateway other stuff, and then spin up a brand new environment. I'm gonna say, okay, this is now my dev environment. I'm gonna restore a gateway backup, I'm gonna restore databases, and I'm gonna do some spot checks or automated testing to confirm that those are all still working.

26:53
Joseph Dolivo: So we do that regularly for all customer instances. It's something you should do as well. Really, really important. Data residency requirements, so especially when you're talking about production systems, again, in the cloud, you've got all these different regions you can deploy into. Certain cloud services you'll find are only available in certain regions as well and certain regions have availability zones or don't have availability zones. It's really important to know where your data is going and where your data is being stored at all times. And there are a lot of industries, a lot of companies that have very specific regulations to say, my data cannot leave the United States. For example, my data cannot leave Canada, my data cannot leave this particular geographic region. So keeping that in mind is really important 'cause you may say, well, yeah, my workloads are running inside of US-East-2, but to get there, it has to go up through this other system running somewhere else.

27:45
Joseph Dolivo: And now if the data's being... Even if it's encrypted, my data's going somewhere where it's not supposed to be. That's a big no, no. Same thing with storage. You could say, well, if the cloud providers have the concept of paired regions where you could say, you know what, I'm gonna store most of my data in US-East-2, but it's paired to something in Canada-West-1. So for disaster recovery purposes, that may or may not be okay depending on what your team's kind of requirements are due to regulations or company policy or anything else like that.

28:17
James Burnand: And maybe I can just quickly add to that if my mic comes on. When you're also architecting your solution, availability zones and regions become a huge important consideration. So for example, you can buy storage that's mirrored across three of those. So availability zone for everyone's benefit is a completely separate data center that has a separate power feed, it has separate network connections, but it's inside of a region. So US East, for example, for Azure has three availability zones that you can buy services from as US East. So depending on the reliability requirements of the application that you're deploying, you need to choose the services that have the right level of reliability. So by default for us, for example, when we do storage, we'll actually have storage that's mirrored across three availability zones in a single region so that way we can tolerate two buildings burning down before your system will stop. So just to kind of put a little perspective around that is that there is also a cost consideration as a part of that. So if you're going to buy something that is available across regions, for example, it's going to be more expensive than if you're getting something that's dedicated to a single availability zone in a single region. So your application architecture matters from a cost perspective.

29:29
Joseph Dolivo: We are definitely getting the cost as the next big pillar here as well. So well said, James. And the last point on here is just data integrity and retention. So I need to maintain data for seven years, 10 years due to regulatory purposes. The storage providers inside of the cloud, or the storage accounts inside of the cloud providers allow you to do, for example, immutable data. So I'm gonna push data into an archive storage tier. AWS Glacier is an example, Azure Storage account has an equivalent, where nobody's gonnae able to touch it, and it's gonna reside for some extended period of time. So that's really, really, important for compliance purposes and it doesn't even necessarily have to be data in your live system. You may say, you know what, having a 10 terabyte drive on this managed database service is really expensive.

30:17
Joseph Dolivo: But I need to maintain the data, but I'm not actually gonna query it unless an auditor comes and starts knocking on my door and says, "Show me the data." So you could store all of that older data in kind of much cheaper archive storage and then if you need to restore it to say, "Hey, look, I've got it," then you can go through a process to do that when you need it. A really good way to save cost, which is our final category for today. So cloud makes it so easy to get up and running, and the cloud providers wanna incentivize you to just pump all the data up. We're not even gonna charge you. If you're not going over an encrypted connection, we'll ingest all your data for free. That's become pretty much a standard. But once it's up there, they're gonna charge you for using it.

30:52
Joseph Dolivo: And there's a lot of stuff in the news recently. Hey.com recently talked about how much money they're saving by going out of the cloud and there's a lot of... So we talked about some of the reasons you may or may not want to use the cloud, but once you... You're really paying for sort of the flexibility and scalability that you get. So for the Data Dash, we said we're gonna spin up five servers. Give me five servers Azure and boom, we have five servers up and running. But you're paying for that dynamicism and flexibility. So if you know, for example, I'm gonna run Ignition Cloud Edition for a year at least, you go to the AWS marketplace, you go to provision Ignition Cloud Edition, it'll tell you if I know I'm gonna run this workload for a certain amount of time, I can basically commit to paying for a year and I'm gonna get a pretty sizable discount on the infrastructure cost.

31:36
Joseph Dolivo: 30%, 35%, something like that, that's huge, especially when you're talking at scale. And it's not just Ignition systems that can do that. You can do that with databases typically, you can do that with storage. So trying to estimate the workload that you have and then being able to kind of predict what you're gonna need is really, really useful as you've been running. Again, not so much for experimenting. When you're in a production system, that's important to consider, and it's something we do as well. So we actually will forecast out based on our customers. We're gonna commit to using this amount of resources and we get a cost savings from that. So that's reserving capacity up front. Another thing is called... And different cloud providers have different terms for it or basically spot instances. So this is where maybe I don't need a workload running all the time.

32:18
Joseph Dolivo: Maybe I need to do like a... I was gonna say batch job, but batch means something else in our automation industry, but I'm gonna run a report at 2:00 AM every week, for example. And it's something that's gonna run for a while and then it's gonna shut down. I don't need it running all the time. Or maybe I'm gonna just spin up a temporary dev system. I don't need it for a long period of time. If it goes down, it's not a big deal. You can leverage these cheaper spot instances where you basically will say, well, I only want to pay for a compute between this price and this price and if it becomes available, great. If not, shut it down. Or if somebody else is willing to pay a higher price for it, they're gonna steal my VM out from under me.

32:55
Joseph Dolivo: You can have incredible cost savings when you do that. It's also good for like a lot of GPU-based workloads like ML and AI training. So that's, again, not so much for Ignition production systems, but certainly for either dev and test systems or if you need some kind of temporary scalability like, hey, I need to add another frontend node to my Ignition server 'cause I'm anticipating more load during shift one, or something like that. So that's something else to consider. Huge, huge implications on cost if you do it right. And then I can't tell you how many times I've heard from customers saying, "Well, I got the bill at the end of the month and it was 10 times higher than I expected." So making sure that you're putting monitoring in place and alerting in place so that if you're starting to exceed your typical usage trends, you're able to identify that quickly and early.

33:39
Joseph Dolivo: So this has saved us a number of times. I talked to a couple of folks in the room about this where we had logs that we were aggregating that basically hit a trip wire and our system alerted us. We were able to make a change so that we didn't get $3,000 cost after that. And the cloud providers themselves and a lot of the cloud-native tools have ways of doing that. We'll talk about our tools in a minute. We use Grafana Cloud as an example for aggregating all of our metrics and logs across all of our systems. So you can set up alerts and notifications. You can do it in Azure, AWS, and GCP so that way you won't be surprised when the bill at the end of the month comes. So super important.

34:19
Joseph Dolivo: Just to kind of give you some insight, if you're kind of looking like, "Well, where do I kind of get started with this?" These are tools that we use. There's a whole bunch of them. It's really hard to pick, but I'll just kind of go through some of the icons so you're aware of them. Obviously, you know Ignition right in the center. Everything that we do and most of everything that you do is built all around Ignition. If I start at the top left, there we go. There's a laser pointer. So that is Kubernetes. We don't recommend that for most folks. It's one of those things if you have to ask, you probably don't need it. Something that we use internally, and there's a really great session that Kevin Collins did earlier today talking about kind of the nuts and bolts of that.

34:56
Joseph Dolivo: We use that because we're orchestrating Ignition across tons and tons and tons of customers. So if you're a bigger customer, you have a lot of Ignition instances to deploy, a lot of other workloads alongside a single gateway you need to deploy, a really good tool to consider. If you need to run one Ignition server, it probably doesn't make sense. Going clockwise I guess, Grafana is the next one. So this is what we use. I kind of hinted at it for metrics and log aggregation. It gives us really good deep insight into our containerized workloads as well as all of the kind of cloud provider-native services. So we can see how we're doing on cost, we can look at our CPU and RAM performance, all that kind of stuff. It's really nice to have a single pane of glass. And there's other systems out there that can do that.

35:37
Joseph Dolivo: We like Grafana. Great visualizations as well. Git, so when you're making changes, especially in kind of an enterprise space, it's not a cloud-native technology. I call it a cloud-adjacent technology. It's kind of in the same realm doing version control. Again, super excited for the changes coming at Ignition 8.3 that will make this more comprehensive beyond projects. We did a whole session on it last year. We're doing a workshop on it in a couple of weeks. But we basically run Git inside of the cloud to maintain backups of our project configuration, both for Ignition as well as other services. And then currently we support AWS and Azure. I love GCP as well. That's a great one. And then finally, the one that you may not recognize this logo here, this is called Pulumi.

36:17
Joseph Dolivo: So there's this whole suite of tools called Infrastructure as Code is the buzzword. Terraform is kind of the market-leading most popular one. They've been in the news recently due to some licensing changes that they've made around their open source offering, but we've been using Pulumi, which just lets us use our programming expertise that you'll have from Ignition Python, for example. You can use that to provision all of your infrastructure. So we never manually go and download a VM and download Ignition and go do the installer, even though it's only three minutes. We never do it. We use everything as containers and it's all provisioned using this tool called Pulumi. So there's a ton of good tools out there. We highly recommend being in automation as we are that you leverage some of these where it makes sense for you. I think...

36:58
Joseph Dolivo: So we've got additional resources. We made reference to some of these. So there's best practices, obviously the Ignition Security Hardening Guide, concepts for Kubernetes. AWS and Azure have their own. GCP also has some. These are links inside of the PowerPoint, which will be sent out. Definitely take a look at all of these. And then the two sessions, there was a good higher-level one on Ignition in the cloud. If you didn't get a chance to see it, watch it on the Livestream or the recording afterwards. And then the "Deployment Patterns for Ignition on Kubernetes" that Kevin Collins did. So really, really good sessions with really, really good, good info. And question mark means questions. We're ready for the tomatoes.

37:41
Audience Member 1: I didn't bring my tomatoes today, but one question I have, you guys alluded to it earlier that this is a space that's difficult to dabble in. So many of us being integrators or service providers here, what offerings do you guys have for providing a sandbox environment for people to get familiarized with your platform and potentially show it off marketing material style for potential clients?

38:08
James Burnand: I'll take that one. So we're in the process of hopefully soon announcing some really cool local versions of what we offer that you'll be able to actually run locally on your machine as a test environment. As it stands right now, we set up demos for integrators all the time with their own separate subdomain on our development system. So then you get gateway and database access. You can throw your projects up there, you can test playing with them, and you can make sure that they work. But one of the cool things about how the products that we built work is all of this complexity is kind of encapsulated in those. So you get designer access and you can get database access, and it looks just like a normal Ignition project. So from our perspective, we're trying to help make this technology easier to be able to adopt and that's kind of been our business model from the beginning.

39:02
Audience Member 2: Getting into cloud and cloud infrastructure and tools is... Can be a scary thing. And I think I've seen that with a lot of customers and even with myself thinking about how do I even get started? Can you guys talk to what you would say to somebody who wants to get over that fear and even just get their feet wet with cloud infrastructure and how they can start seeing those benefits and how do you overcome that first step?

39:31
Joseph Dolivo: Something that I would say is cloud is a spectrum. You don't either adopt it or not adopt it. There's kind of a spectrum of adoption. And so the easiest way that we've seen to kind of justify the use of cloud is just use it for offsite backups. Are you really, I like to say, take the tape drive down to the bank vault everyday. Is anybody doing that? Some people are doing that. Use it for encrypted multi-site offsite backups. That's kind of the Trojan horse, if you will, to kind of cloud adoption. And then use it for the things that it's really, really well suited for and tailored for like scalability. You know what, I'm gonna spin up a dev system, for example. I'm gonna play around with it. That's a really nice way to get companies more comfortable with it.

40:09
Joseph Dolivo: We spend a lot of our time with heads of IT and security folks kind of talking about why this is okay, how this can fit within their kind of existing IT landscape. It's actually kind of interesting because I'll say prior to maybe three or four years ago, the cloud was a scary thing for almost everybody. And we've really had this excitement that we've seen from a lot of customers I think driven by let's say Ignition's use of a lot of IT technologies, for example, where all of a sudden you talk to a chief security officer and they're like, "Oh, you're using containers, you're using this. I get it. You're speaking my language now." So that's actually helped I think to make it a little bit more palatable. But yeah, start from offsite backups. Super, super simple would be my point around that.

40:49
James Burnand: Yeah, I would only add to that that I actually think probably the best place to kind of focus learning attention if you're a traditional automation person and you're looking to figure out kind of how does all this work is I would focus on containers, learning the different container architectures, how networking works, how you actually set up those systems. And Kevin Collins' GitHub page is fantastic for anybody that hasn't been to it. Absolutely you need to go to it. I don't have the URL handy, but certainly it has so many resources that will help you learn about how to work with these architectures. And then really what you're doing is you're taking that Docker-centric architecture and you're using these prebuilt functions and tools to make it easier to actually do a more coordinated deployment.

41:34
James Burnand: One of the things Joe didn't mention is our Grafana system that's providing us all that alerting and monitoring, what it often is telling us is that it fixed something. So Kubernetes had a problem and it took care of it, and we get a teams message that says, "Yeah, the problem happened and the problem is taken care of." So like that part of kind of the progression and the ability to automate and take advantage of these tools at scale is the ultimate goal, but none of that happens if you don't first start focusing on things like containers.

42:04
Joseph Dolivo: Yep. The last part I'll add is looking at containers, it's another one of those kind of cloud-adjacent technologies. You can run containers on-prem and you can run 'em in the cloud. So start doing the things that will work well in the cloud, but just do 'em on-premise. So we've seen a lot of that's kind of hybrid cloud is kind of a similar idea with that. Thanks.

42:24
Audience Member 3: Do you have customers that are ingesting or exgesting? What's the opposite of ingest? I don't know. Doing that thing...

42:37
Joseph Dolivo: Expulsion?

42:39
Audience Member 3: In other cloud technologies. Like IoT Core, for instance. Are people using IoT Core to get data into your systems or then beyond just a normal database thing? Are there other places where data's going out of your environment?

42:56
Joseph Dolivo: For sure. So there's... And it is funny 'cause IoT Core is something, a service that AWS and other services have had. GCP made a lot of news recently where they actually sunsetted one of their IoT products. And so Cirrus Link is here. They have a great broker. HiveMQ is here. There's a number of kind of broker technologies, I'll say, for getting data up into the system and then also kind of pushing it back out. So Ignition is a good fit for integrating with all of those, a lot of those kind of event-based systems. Again 8.3 is coming and it's gonna make this easier. But you can ingest into Azure Event Hubs, you can ingest into AWS IoT hub, IoT Core. So those all work. The one thing to keep in mind too is that not all of those services, they may support MQTT, but they may not be fully compliant with things.

43:44
Joseph Dolivo: So for example, we went down a whole road with like store and forward and avoiding data loss. Going up into MQTT, there's some nuances to the TCP Keepalive Timer and all these kind of things that could result in data loss. A lot of systems that are sort of compliant, somewhat compliant outside the ecosystem don't support all of those. So that's something to keep in mind for sure. Once you get data up into Ignition in the cloud, then you can kind of push it out, but we found... We've seen a lot of benefit. If you're gonna push data into Ignition running in the cloud, whether it's [Ignition] Cloud Edition or whatever, keep it in there to do all of your visualizations and stuff like that if you're gonna use an Ignition and then push it out after that. So I hope that helps.

44:25
Susan Shamgar: Alright. Thank you, everyone. I believe that is all the time that we have for today. So can we get one more round of applause for James and Joe?

44:40
James Burnand: Thank you. Thank you, everybody.

0:44:40.6 Joseph Dolivo: Thanks everybody.