The OG Perspective: 10+ Years of Ignition Wisdom and Beyond

Speakers

Jean-Paul Moniz

Technical Services Coordinator

Cameco Fuel Manufacturing

In this session, we'll explore more than a decade of experience with Ignition, sharing valuable insights as a long-time member of the Ignition community. We'll take a practical look at how Ignition has evolved and its role in modern manufacturing, including topics like MES, OEE, AI, and more. It's an opportunity to gain practical knowledge and understand the journey from the early days to today's automation landscape.

Transcript:

00:03
Jean-Paul Moniz: I built a functioning demo, a OEE app, for a few machines and we didn't know SQL back then and it was like, "What's this SQL stuff?" and all that kind of stuff. And so we learned SQL and we built this demo and everybody's like, "This is exactly what we need," and so it sort of hit the ground running there and looking and auditing what was out there on the market. There was nothing else on the market that can sit there and compare with the feature set that you got with Ignition at the time. And so for me, it was a no-brainer choice. So how did Ignition sort of grow within CFM [Cameco Fuel Manufacturing]? Well, it started out very simple. We bought one gateway for the one plant, did that in about 2010, and then we liked it so much. We have our other plant, what we call Port Hope. It does all of the final assembly and so we bought another gateway for that. And then Sepasoft started up and said, "Hey, we have this neat, wonderful OEE module." And it's like, "Well, that's exactly what we're trying to do." So we bought the Sepasoft modules back in 2012, and anybody that's familiar with Sepasoft, you probably know the pains of version one, version two, version three.

01:19
Jean-Paul Moniz: I won't go through that, but we went through those pains. And then we started to investigate MES and track and trace in around 2015. I was doing some work with serialization of our product and whatnot and we understood the value of being able to sit there and digitally track your manufacturing operations on the production floor. So we started doing a big exercise. We engaged integrators like Grantek. We talked to other integrators like Brock Solutions. And what ended up in that, we did a big URS and FRS, and it was like 160 pages, and we ended up with this, well, if you pay a million and a half dollars, you can have this wonderful system. And we looked at that, and we said, "Yeah, that's great, but we think we need to go another way." And so what we ended up doing was sort of taking a roll-your-own approach. And so we deployed our first full MES production line on a specific process that was sort of bookended.

02:24
Jean-Paul Moniz: It was a really nice process to sit there and deploy it on or sit there and test out MES functionalities, learn how to deploy a system, see the value in it, and then sit there and understand how do we take what we learned here and deploy it out to two facilities across the board. And so we're actually just finishing that up on our second facility and once we have that finished, that will actually obsolete an old customs.net 1.1 and SQL application that's been in the plant forever. So from a technical standpoint, this is sort of what it looks like. We have one plant with about 12 projects running in Ignition, 25 modules combined with just regular traditional Ignition modules: Perspective, Vision, WebDev, stuff like that, and then all of the Sepasoft version 3 MES modules.

03:22
Jean-Paul Moniz: We have about 29 PLCs in that plant. We're starting to play with MQTT. We have about six OPC connections on that plant, and any given day, we have about 70 clients open up on the production floor. Of those 70 clients, there's about 14 that are dedicated specifically for local HMI control on the machines, and then about 56 operator terminals for MES entries and OEE tracking and all that kind of stuff. And in that one plant, there's about 82,000 tags and in the other plant, there's the same story in the other plant, less operator terminals and about 64,000 tags. And then we also have an enterprise server that where we network them through the Ignition gateway and then that one's more like an overview-type server where we have some projects in there and they're doing a whole bunch of things a lot with the MES, just reports, visualizations, stuff like that.

04:23
Jean-Paul Moniz: But then we also have some really weird use cases where we have one project no different than what's on Ignition Exchange for people signing in. So we use it for things like that, visitors to sign in and stuff like that. So the main use case that we use it for aside from all the OEE tracking, we do use it as a centralized HMI application for development and deployment. That was one of the things that especially being a traditional controls engineer sitting there and is going, "Do I want to put my HMI application up on a server and then if the server goes down, I have no HMI on my machine?" But we sort of got over that hump and it has a lot of benefits to doing it. Really, Ignition as a platform sort of lets us do that IT/OT convergence that everybody talks about all the way down to the machine. So in my HMI application, where I'm usually... And when I talk about HMI, I'm talking like cycle start, cycle stop for a machine and all the traditional stuff that you would sit there and think.

05:26
Jean-Paul Moniz: But then also maybe like from an OEE perspective, being able to sit there and add shift comments in as the machine's running, right? And then using Ignition's technology, maybe that shift comment is actually an integration of Microsoft Teams, right? And so that's where that IT/OT convergence really starts happening. And then the traditional issues around HMI development with traditional hardware: firmware issues, cost of the HMI terminals and everything like that, all of that stuff goes away with Ignition, right? We don't have to worry about firmware updates, right? It's essentially Windows 10, vanilla computer is perfectly fine, Linux is perfectly fine, ARM is perfectly fine. All of those opportunities come up. And then also for asset information and data management platforms, we have our OEE, so we're monitoring, sitting there and saying this is what our equipment's doing from our performance, but then rubber has to hit the road and we have to get in the continuous improvement and everything like that.

06:31
Jean-Paul Moniz: And so machines break down. They need support. So amalgamating all your information around the technical support of equipment. So for example, equipment drawings. Well, where do you store your equipment drawings? Well, maybe we store it in a SQL database, or maybe we store it in a document management system. Technology that Ignition provides allows us to seamlessly integrate wherever that data is and provide it to the end user in one common view, one common portal, right? They don't have to sit there and go over to the DMS system and go get a document or whatever system where that data is. And then also we do things like integration and SAP. I don't know anybody that uses SAP for plant maintenance or whatever or is familiar with ECC. I know a lot of people are migrating to the S4, but it's not intuitive for somebody to sit there and create a maintenance notification in SAP.

07:31
Jean-Paul Moniz: Well, with Ignition and things like the Business Connector from Sepasoft and 4IR, you're now able to sit there and create an Ignition screen, prepopulate all the information that SAP needs that the user doesn't care about, and just allow the user to sit there and say, "This is my problem. I need somebody to hit the button." And the next thing you know, that notification is in your CMMS system, whether it's SAP or whether it's some other CMMS system. The UX experience from the end user, they don't care about where that data goes off, right? They just want to be able to sit there and put in their use and go back to solving their problems, right? And so that allows us to sit there and create that sort of seamless environment for the end user. And then the same thing with what I was talking about for Microsoft Teams. So that resource that was shared on the Ignition Exchange through one of the last... Oh my god, build-offs or whatever. I forget who did it. Was it Vertech that did it?

08:35
Jean-Paul Moniz: I think it was Vertech that did it. If it was somebody else, I'm sorry, I apologize. [Note: It was Flexware] But taking stuff like that from the community and then sitting there and saying, "Oh, I can use it for this idea, integrate it into our system." And so for example, now we can sit there and have two-way communication for issues at the machine, right? So somebody says, "I have this problem." Maybe if it was just a note in the database, it would just go off that database, and how would anybody sit there and get the feedback back unless they went and read it? Now, with mobile phones and proliferation of everybody having something like Teams at their fingertip, now they can get an answer almost immediately, and it would show right back up on their screen. And then OEE, again, OEE integration with Ignition, it's been seamless, right? So now we get all of our OEE metrics within the same platform that we're collecting process parameters for the equipment or doing that integration with all the documentation. Everything is all in one system. Centralized administration of ISA-95 concepts and ideas, all of that's done within the same platform.

09:50
Jean-Paul Moniz: And then that allows us to build what we actually need. Instead of using some sort of off-the-shelf prepackaged solution, we actually get to build what the operators need, what the technical people need, what operations management, excuse me, actually need. So they get the tools that they ask for when they ask for it and then it also gives us the insight on our manufacturing operations, lets us understand where material is at any given time, what our inventories are, rapid access to information. And then in turn, it allows us to sit there and start shifting our culture towards performance excellence. So instead of just sitting there worrying about OEE numbers, we're sitting there creating a culture of accountability more about inputting the data and being accountable to that data and stuff like that. With the tight integration between the HMIs and the OEE system, we can sit there and create metrics, and sit there and... I don't know what the best way I can say this is, is ensure that the data going into the system is as accurate as possible such that we can actually create or fix the right problems for the end user.

11:14
Jean-Paul Moniz: So then MES and Ignition data modeling in the ISA-95 framework, that's been a huge thing that allows us to sit there and structure our data in the way it needs to be organized for the organization. And so the whole idea of enterprise, plant, area, line, cell, getting that data into that structure and having it easily accessible gives so many insights into what's actually going on in your plant. And one example is real-time insights, the work in progress, right? So now you can sit there and have full transparency. If you were let's say a legacy manufacturer tracking things on paper, you would have to send the same materials person out every day, sit there and count the inventory on the factory floor and make decisions based on that. Now we have real digital insight to where any material is at any time.

12:17
Jean-Paul Moniz: And then enhanced accountability on waste reduction, having the ability to sit there and track 10 units went into this process, nine units came out, and then to be able to ask the question, "Where did that other unit go," right? That brings amazing insight to manufacturers, especially if the material that you're producing let's say is expensive, right? So every piece of scrap counts. Now you can sit there and start asking questions. "Why are we losing 5% raw material on this process? What can we do about it?" So having that ability gives us great insight to sit there and fix problems like that. And then in-depth product genealogy, being able to sit there and ask the question: When was this made, where was it made, who made it, what were the settings on the machine when we made it? Being able to answer all of those questions, having a system that can do that is worth its weight in gold.

13:22
Jean-Paul Moniz: Sorry. So then that's what we do with it. And then I wanted to sit there and talk about ideas that, we're sitting there. Everybody's talking about AI and some people get bored by all the hype that's being talked about it. But I think there's some really useful conversations around using AI to enhance a lot of OEE systems that are out there and whatnot and around specifically operator feedback. If you sit there and take a look at what's going on nowadays, a lot of companies are putting in these OEE systems, tracking performance and the equipment, taking feedback from the operators, putting them in the systems. That's just getting recorded to a database. Well, what are we doing with that information, right? And AI gives us a new ability to sit there and search index and all that fun stuff to be able to start asking those questions and sort of closing the loop on some things. So the whole idea about analyzing feedback that your operators are giving, ask the question, "What are the top 10 things people are talking about?" Right?

14:29
Jean-Paul Moniz: And doing Pareto analysis on that and sitting there and going, "Okay, well, are we doing anything about it?" Right? Our people are giving us lots of feedback and ask the question, "Are we doing anything about it? What do we need to do?" Right? And then along that same line, we're looking at things, concepts of knowledge management with graph databases and whatnot. So now the technology is out there. You think of equipment that's breaking down all the time, and okay, it has this failure. Well, traditional in reliability, reliability means maybe you might have an FMEA that you put in a spreadsheet, and it gets locked off in the corner. It has all these nice answers on. If this breaks, this will happen, or if this comes out of adjustment, this will happen. Well, if you take that concept and digitize it and put it in a graph and then maybe you can sit there and connect it to large language models and whatnot. And so now new people coming in that don't have that subject matter expertise can now start asking digital systems, "This failed. What should I do?" It becomes a very, very powerful idea, right?

15:47
Jean-Paul Moniz: So some practical insights that we've seen from doing all this work with Ignition, sorry, the first thing is being able to embrace a culture of accountability. And when we talk about accountability, everybody sort of tends to think negatively about accountability, right? But it's really just about getting the right data so you can sit there and make informed decisions, right? And so you'll hear a lot of people talk about OEE and then sit there and say, "Well, the operator just presses the closest reason code that they can on the OEE system." And what Ignition allows us to do is to design the input system such that we make it as easy as possible for the operator to sit there and put in the right information. And if they don't, then we have the metrics and the tools in place to understand that that didn't happen and then go back and ask the question or fix the data such that we always have accurate data.

16:48
Jean-Paul Moniz: Sorry. And then leveraging data modeling for clarity. This one's becoming a big one for us, is not just focusing on really asking the question, what does a data model for my piece of equipment look like? Not just the simple idea of there's a temperature here or there's that pressure setting there or whatever it may be, but really thinking about the whole idea of digital twin, but what does a data model look for a piece of equipment and start asking that question and then how do we model that in our system properly? And then real-time visibility, so we invest in systems that provide visibility in the operations. So this allows us to be agile, right? So we've invested all this time into these MES systems and the OEE systems to sit there and give us all this data. It allows us to be super, super agile on continuous improvement. Continuous improvement is key, right?

17:49
Jean-Paul Moniz: And so being able to sit there and say, how did we perform yesterday and how do we perform? We know where we want to be tomorrow. How do we get there? And so using continuous improvement and using the data in the system to be able to sit there and come up with plans, to sit there and make impactful changes on some of these metrics is a huge benefit. And then stay agile and forward-looking. I think everybody sort of hit the note on the keynote this morning, but being able to sort of, in a way, future-proof yourself, but understand that things are changing rapidly every day, and being able to sit there and be agile with a product like Ignition. And a new use case comes up tomorrow. Being able to sit there and rapidly develop the application that helps solve that use case is hugely important. And then empower your workforce. Give your workforce the tools and the knowledge that they need to succeed.

18:54
Jean-Paul Moniz: Some of the concepts that I was sitting there talking about with the AI tools, that's all about giving people the information that they need. I have seen a big trend on... I don't know what the best way to say this is, but being able to capture the knowledge lost from tenured subject matter experts that have been in the plant for 20 years and they have, he hits it with the hammer this way, that's the way it's always done. Trying to capture that information and being able to provide it to newcomers coming into the workforce and equipping them with the information such that they can hit the ground running quicker and faster. And then embracing the future of manufacturing. A lot of people talk about Industry 4.0, and there's lots and lots of subject matter, and we're talking about it a lot over the next three days.

20:00
Jean-Paul Moniz: And Ignition, from day one, has been in a position to sit there and support all of the 4.0 initiatives that everybody sits there and talks about. I know there's a lot of hype going on right now with MQTT and concepts like Unified Namespace and whatnot. If you take a look at Ignition, Ignition supports that pretty much right out of the box. Digital twin integration, again, Ignition's a great platform to be able to sit there and sort of build on top of these concepts of digital twin integration. It might not be simulation that people talk about, but the visualization piece, access to information, Ignition's a great platform for that. And then, excuse me, edge computing and real-time insights. Again, everybody's familiar with Ignition Edge, but I'm a firm believer that equipment will get more intelligent over time to the point where you buy a piece of equipment, it's not just the equipment you're buying, but there's probably gonna be some level of compute resource, some level of data and analytics that just inherently are part of the machine because depending on what your time domain is with the equipment, some of that analytic processing and calculation will not be able to happen in the cloud just because of speed and throughput and whatnot.

21:38
Jean-Paul Moniz: So machines are gonna get inherently more smart with more technology on it and I think Ignition is in a unique position to be able to sit there and leverage some of those issues. And then customization and scalability. I think if you're using Ignition, you probably know this really, really well, but the ability to sit there and build what you need when you need it and then scale it out is a huge benefit. So lessons learned, what would I do differently? This isn't one I would do differently. This is what I would keep doing, but the whole idea of own your destiny, taking ownership of what you want to do and where you want to go, to me, that's a big thing. When we were sitting there talking about quoting out the MES system for a million and a half and just pulling the trigger and getting it deployed and making a decision to step back off of that and go our own way and roll your own, that was a huge decision for us, right?

22:43
Jean-Paul Moniz: And a lot of manufacturers struggle with that decision. And I would sit there and say that decision was probably the best decision that we ever did. One thing is it actually created two jobs because we had to hire two developers internally to sit there and do that work. It also prevented us from taking a million and a half dollars, spending it, and potentially re-architecting or changing the platform because when you sit there and go through that exercise, let's call it a URS or whatever you want to call it, you gather your requirements. Well, your requirements, they're only as accurate as the day you captured them. Tomorrow comes and the requirements change. So starting off small, trying things, finding out what works, finding out what doesn't work, that's a huge benefit to sit there versus Big Bang type projects and whatnot.

23:43
Jean-Paul Moniz: What I would do differently is get better about standardizing data equipment modeling, especially in our early days when we just had two basic Ignition gateways. We sort of had one team working on this one and one team working on that one, and you would have one team doing things this way and one team doing things that way, and things could get out of sync really quick, right? And so putting structure around that and coming up to agreements on this is how we're going to do things and this is what our data models are going to look like and stuff like that, that's a huge thing. This one, most people would sort of laugh at this one, but avoid mixed SQL databases. Again, if you start off small and you sit there and this happens a lot in manufacturing because cost is an issue, but maybe you sit there and say, "Oh, we're going to use MySQL over here," and maybe somebody wants to use Microsoft SQL over there and that you scale out and all of a sudden, it's like, hey, wait a minute. We have all these differences that we need to sit there and merge together and this code works great over here but it doesn't work great over there sort of thing.

24:55
Jean-Paul Moniz: So try avoiding that as much as you can. Improving project inheritance and dependencies. I sit there and look back, and some of this stuff didn't actually exist when we started in 2010. But as it came online, I would sit there and say, maybe I failed to recognize the benefit or the importance of doing that. Now when we look at our environment, the benefit of having that ability to sit there and have, let's say, a library that you can inherit into your project, that's a huge thing. And then the biggest one I would sit there and talk about is... And this was true for me, was eliminating bias. When I started off in my career, I had this very sort of biased view or perception that the only people that can do SCADA-type development or whatever were controls engineers, right? And so when we were hiring and recruiting for Ignition developers, I thought you had to know things about PLCs, I thought you needed to know things about automation. And we went down that road trying to recruit, and we got to a point and I said, "This isn't exactly working," and I sort of looked at myself and said, "Maybe I am biased here," and started asking the question, "Well, what if I hire computer science discipline or software engineering discipline or whatever." Right?

26:23
Jean-Paul Moniz: And we ended up doing that in 2019 when we hired our developers. Our two developers, they don't have a background in factory automation. They have a background in computer science and software engineering. I think that's the best decision. I think acknowledging that bias and then sitting there and looking at things from different viewpoints and bringing different skill sets onto your team was probably one of the best things we did as well. And then just my personal wish list for things that can be added to Ignition and stuff like that. One of them is the idea of a GraphQL module where you can sit there and develop your GraphQL queries inside of Ignition and then sort of maybe almost like the idea of like name queries or whatever sort of expose that for visualization and whatnot. Better visualization components supporting graph use cases. We're actually actively working on that right now building our own modules so we can sit there and do that but as that type of database system becomes more... Proliferates more into industrial use cases, having components to sit there and support that would be awesome.

27:47
Jean-Paul Moniz: And then the idea of a WebGL module where you can sit there and take sort of open modeling standards or 3D modeling standards and leverage technologies like WebGL to provide better visualizations. And then I think this one's getting dealt with in 8.3, just improve aggregate functions around historical data and the idea of being able to sit there and calculate certain aggregates on the fly or through the API on the historical data. And then I know this one gets a lot of talk, and I'm sure they have a plan for it or whatever, but some sort of runway on what's happening with Jython 2.7 and how do we get things like Python 3 in the platform. And I know that's a hot topic. Questions?

28:46
Audience Member 1: I was just wondering if you had any lessons learned or bits of advice when it comes to developing with the Python code right now. Sometimes I find myself dumping it into like a VS code or something like that to kind of get it right and I was just wondering if you had any lessons learned.

29:06
Jean-Paul Moniz: I think it really depends on the individual developer's preference and stuff like that. Me personally, I'm sort of backed off of development. I'll try out new ideas and stuff like that, but I tend to use things like VS code myself. I know our other two developers, they have their preferred methods. They're more IDE-focused, but I think one of them uses IntelliJ. But using an IDE is always a good thing, and then not only that, but depending on how advanced you are in your Ignition infrastructure, just using an IDE for basic saving, maybe saving it off to a Git repo and stuff like that outside of your project, it's handy for that too.

30:00
Audience Member 2: I have a question that I think you mostly answered already, but I'd like to hear a little bit more about, I was wondering, the system that you built internally, how you went about building it, and then you said you hired two developers. So I was asking from a perspective of did you staff that internally, and was it 100% internal staffing, or did you have some sort of service provider or external resource to help you get through the milestones?

30:31
Jean-Paul Moniz: No, we did it completely internally. All of our development's done internally. The only thing that we do externally is sort of emergency overflow stuff or stuff that isn't our priority. So maybe it might be we're putting a new piece of equipment in and it needs some HMI work done on it. We might send out the HMI work because we view that as sort of academic, if you want to sit there and call it that type work. So we really try to focus on where the high value is. So mainly around our MES system is where all of our development is focused in. Does that answer it?

31:14
Audience Member 3: So before you hired those two personnel who didn't have a controls background, what could somebody have said to you to convince you that your bias was getting in the way.

31:28
Jean-Paul Moniz: I don't know because I would sit there and say... Just naturally being a controls person myself, I would sit there and say, generally, my opinion or view is generally you surround yourself with other controls people that just amplify your own echo chamber. And so when I look back on it, it was more of we know what the need is, we know a lot of the work is mainly focused around traditional computer-science-type skill sets and everything like that. Hey, maybe let's try thinking out of the box a little bit and going down this road and seeing what it yields. And we started off very simple by just bringing in some summer students and giving them some tasks and sitting there seeing where they went and the result was this can work. And so then that's what made us go down the path even further.

32:29
Audience Member 3: Okay.

32:31
Audience Member 4: Right here. One more for you. When you were describing the HMI screens and the way that you moved them to Perspective, you said that there was some hesitation or some fear around the risks of that. I just wondered how you guys overcame that.

32:54
Jean-Pau Monizl: Yeah, absolutely. And just to clarify, the HMI screens are actually done in Vision. The title might be a little bit misleading on the presentation, but the HMIs are definitely in Vision. To me, I think it was just traditional viewpoints on equipment's got to run 24/7 every day all day right. And so the idea of a server going down and then let's say we have it on 14 assets right now. So if that one server goes down, there's 14 assets that are not doing anything or theoretically they'll run until they stop. But it was that idea, but to me, I sit there, I sort of flip on that thought as, well, the focus should be on not letting the server go down. That needs to be the mindset, not what if this happens? It's like, well, if you're worried about that, then make sure that that doesn't happen. That needs to be the mindset sort of thing.

33:55
Jean-Paul Moniz: Now, the one thing I would sit there and say is I was going to put it on, I'm still sort of on the fence because of as things scale up, there is an issue within... It's not an issue within Ignition, but just generally as that idea scales up. So now you have this MES and OEE system that's proliferated throughout your organization, and you're also using it as an HMI server. There's a point, I think, where you scale out, that you start backing that idea out because it just gets too big to manage, if that makes sense. Now if you scale it out by running the HMI server on its own dedicated gateway and keep the other stuff separate, or do you go back to something like Edge, I'm still sort of 50-50 on.

34:41
Audience Member 5: I appreciate your talk, and it's helpful, but it brings up a lot of questions from my perspective, but I'm going to try to focus on one, specifically the knowledge transfer that you were talking about that needs to occur between the people with the very specific plant information. And I see two things that are augmented reality and then the implementation of context-specific NLP embedded agents in like your UIs, that kind of thing. Could you maybe talk a little bit about what you're looking at doing with respect to that?

35:15
Jean-Paul Moniz: Yeah, absolutely. So to me, I look at it from a couple of viewpoints, is if you sit there and take the idea of structured versus unstructured data, I think, I would hope that we can all agree that we live in a very unstructured data world when it comes to policies, procedures, manuals, instructions, and stuff like that. We still love the word processor and generate lots of unstructured data. And my thought is let's get rid of the word processor. If I have a manual, let me create those instructions in a structured format, whether it's in a relational system, whether it's in some type of a labeled property graph, whatever it is, but have it in a structured system such that then we can do things like embeddings and all that kind of stuff and use the power of large language models and stuff like that to be able to sit there. Everybody talks about ChatGPT and its hallucinations. Well, how do you stop the hallucinations? Well, ground it in fact. Well, how do you ground it in fact? Have systems of structured information, right? And so that's where my head's on, is taking that information, getting it in a structured manner such that when I ask a question, I know I'm going to get the result I want. Does that make sense?

36:43
Jacob Wever: Were there any more questions? I had a question actually. When it comes to Ignition, my experience from software is more of a troubleshooting background. It's always come with issues. What is an issue you've experienced with Ignition, and it's memorable to you, and how did you go about resolving it?

37:08
Jean-Paul Moniz: There's been lots of them over the years. I'm not saying there's no issues. There's been lots of them over the years, but to me, I think starting from the start, having the logs in the system to be able to sit there and say, "Hey, this is broken," and then starting to go through the process of intuitively, "Okay, what's broken and what do I need to look at?" Having that ability in the platform is I guess paramount, but it's invaluable. And I would sit there and say, from my background, not being truly software development or software engineering, sort of understanding Ignition as a product and JVM and Java language and whatnot, learning how the product was built and what some of these exceptions actually mean and stuff like that, that was a little bit of a learning process.

38:09
Jacob Wever: Thank you.

38:11
Audience Member 6: Do you have any plans for DevOps?

38:16
Jean-Paul Moniz: Ask the DevOps guy. And James [Burnand from 4IR] asked that question for a good reason and we do have... We're working on that plan right now for DevOps. That is a good point that I probably should have put in the presentation. But as our system scaled, especially in manufacturing, I keep saying in manufacturing because if you use the IT lens, there's structures and systems around supporting information systems. They know that very well. But on the OT side of the fence, right, there's no structure, there's no systems for supporting that system. So what we're working on right now is implementing IT-grade DevOps system for our OT assets. And so we have three physical blades in each plant right now running VMware, running a whole bunch of virtualized servers and supporting all this and that's our infrastructure. And what I want to do is be able to sit there and simplify that infrastructure to the point where I don't need in-plant resources to sit there and support it. Maybe I go to a managed system or whatever, but the whole fundamental idea is my technical people in the plant are focused on value creation, not supporting infrastructure systems.

39:40
Jean-Paul Moniz: That's not where their value is needed sort of thing and so we're working on that right now with 4IR, doing a sort of hybrid cloud design. So our dev are implementing a full development QA test system that will be up in the cloud. We can spin it up and spin it down as needed when we do development. But then when we migrate it over to production, what can run in the cloud can run in the cloud, but anything that needs to be in the plant because of latency and all that kind of stuff will be in the plant running on cloud-supported infrastructure such that we can do development, push it over to QA, test it, and then push it down to production as quick as possible. So in OT, running a patch is a big deal.

40:33
Jean-Paul Moniz: Applying a patch is a big deal, getting the right outage, but if you have a good dev QA development system, then you can shorten up that patch cycle with a lot more confidence to sit there and say, let's take a 10-minute window and see how it goes because I would sit there and say, I have lots of experience maybe to your question on what happens when patches don't apply properly. And just to your point, if you have more questions and you want to sit there and talk, feel free to grab me. I'm more than willing to sit there and talk to you one-on-one. I'm all about sharing information within the community. I just want to sit there and actually point out Chris [McLaughlin] from Vertech on the Keynote, he won a Firebrand for that, right? So that was super cool to hear, all of that volunteer work for that homeless application. That was awesome to hear.

41:31
Jacob Wever: Alright. Thank you so much, Jean-Paul, for coming and thank you, the audience.

41:33
Jean-Paul Moniz: Thank you.