Historic Opportunities: Discover the Power of Ignition's Historian

Speakers

Don Pearson

Chief Strategy Officer

Inductive Automation

Kevin McClusky

Co-Director of Sales Engineering

Inductive Automation

The ability to store and query historical data easily and efficiently is vital to digital transformation. The Ignition Tag Historian Module has long provided a superb solution for this need and recent improvements have broadened the scope of what it can do. Is your organization harnessing the full power of historical data for a successful future?

This webinar will offer the deepest look yet into Ignition’s powerful historian system, exploring solutions within Ignition as well as solutions that connect to it. Join our experts as they illuminate the possibilities, from quick creation of charts and reports, to swift sending and retrieval of data, and ultimately converting data points into valuable insights. Don’t miss out!

• Learn why SQL databases can be the best choice for storing tag history
• See how Ignition 8.1 puts more historical data at your fingertips
• Learn how to increase throughputs
• Explore how to not only store data but get more results from it

Webinar Transcript

00:00
Don: Good morning, everyone, and welcome to today's webinar, Historic Opportunities: Discover the Power of Ignition Historian. My name is Don Pearson, I'll be serving as a moderator and co-presenter today with Kevin McClusky, the Co-Director of Sales Engineering in Inductive Automation. We're excited to share our presentation with you today. Kevin, I do know that probably everybody, almost everybody maybe knows you, but can you take just a minute and introduce yourself, and then we'll get into today's agenda?

00:28
Kevin: Sure, I'd be happy to, yeah. As you said, my name is Kevin McClusky, I'm Co-Director of Sales Engineering. I've been with the company for about, going on 12 years now. And before that, I was in the automation industry working for an integrator in California, downtown Sacramento, actually. And I've worked with lots and lots of companies over the years, hundreds of companies, maybe even more, maybe even up to thousands at this point, and talk about everything from architectures to technical bits and bytes, protocols, developments, strategies. And at this point, we certainly have a team of sales engineers who I coach along with Travis Cox, I'm gonna say.

01:13
Don: Thanks, Kevin. Totally appreciate that. So let's take a quick look to today's agenda before we get into it. For the webinar today, first of all, answer some of the most frequently asked questions that people have about data historians. We'll talk about Ignition's Tag Historian, and we'll show you the technology stack for using Ignition as a historian. Then we'll look at data visualization, accessibility, backup and archiving. And we'll discuss benchmarks and some other considerations. Then, as we always do, we'll get to your Q&A. Today, we'll be talking a lot about our software platform, Ignition. Ignition is a universal industrial application platform for HMI, SCADA, MES and IoT solutions. Actually, Ignition turned 10 years old last year. It's used by 54% of Fortune 100, and I think about 35% of Fortune 500 now. The latest version is Ignition 8.1.

02:10
Don: A couple of bullet points on this slide here for those who are new to Ignition. It has an unlimited licensing model, cross-platform compatibility, IT standard technologies, scalable server-client architecture, web-based, web-managed, web-deployed design among clients, modular configurabilities, so you basically build out the architectures you need and use the functionality you need as you go along, which makes for rapid development in deployment tools and easy scalability. I think it's been said more than once before, but it bears repeating that digital transformation is all about turning data into useful information. And to do that, you must be able to store and query historical data easily and efficiently. The more you know about working with historical data, the more value you'll be able to get from it and bring to organizations.

03:03
Don: So today, we wanna give you a deep look into Ignition's Historian system, and we'll show you everything from the Tag Historian Module to SQL Bridge Module. You can do a lot with history in Ignition, and not just tag history. There'd been a lot of questions about historians and how to work with historians, and to start the discussion here, I think it's probably good to take a look at maybe responding directly to some of those frequently asked questions that we get all the time about data and tag historians. I think we'll be able to clear up some common misconceptions along the way and get us off to a good start here. So Kevin, I think what I'm gonna do, since you are the expert here, I'll tee up the questions and you can just answer them. Does that seem okay with you?

03:50
Kevin: Sounds great.

03:51
Don: Alright, let's get back to basics here. What is a data historian?

03:5
7 Kevin: That is the right question to start with here, I think. So if we take a look at what a data historian traditionally has been and traditionally has been thought of, that's been a Tag Historian, specifically. So storing tag values, doing compression behind the scenes, doing interpolation when retrieving those values, and configurable poll and storage rates. And if you take a look a bit more recently over the last handful of years, and maybe even the last decade or two, there's been another feature that's been added to what most folks think of as a historian, which is event-based data. So being able to collect individual moments in time, take snapshots of what's happening at different moments, take a history of runs of products or individual... When some schedule kicks on or kicks off, if you have a pump-out cycle and you're in water, wastewater, or pick your industry, there are going to be events that need to be tracked, in most cases. And those events are now under that data historian umbrella as well, when those folks think of a data historian.

05:19
Don: To the next question. "But you can't do tag history with SQL databases, right?"

05:26
Kevin: Well, you know, that's a...It's funny because we've gotten that question so many times, and the truth of the matter is we have thousands of customers in over 100 countries around the world all using our Tag Historian going to SQL databases. There are projects with millions of tags, there are customers with years and years of historical data, and at this point, decades of historical data. So I think a proof point there is, yes, you can use SQL databases as the Tag Historian backend.

06:07
Don: "Well, then why have I heard that SQL databases aren't good for historians?"

06:12
Kevin: Well, I'd take a look at where that message is coming from. We have some benchmarks and things that we're going to take a look at later, and there's certainly some information to dig into in terms of throughputs that you can expect. But for your typical application going up to, often, millions of tags, SQL databases can be the right choice or/and in fact, are a fantastic choice as a historian.

06:39
Don: "Alright, so now, you got me a little intrigued here. Tell me, what can the Ignition Historian do?"

06:47
Kevin: Yeah, so three million streaming tags per database, if you're looking at a 10-second rate and 10% of tags changing. The numbers that really matter here are the number of tag changes per second. We have a companion document that you can download from our website that has a lot of these, and I'll give a little preview of that a little bit later, but you can certainly take a look at that document and get some guidance on what those benchmarks are there. But a typical system, 10-second rate and 10% of tags changing is pretty typical for a lot of applications. Some industries, you have to have faster rates than that or you have larger percentages of tags changing, so that number scales. That number is per database, in this case, so you can have multiple databases as well that all work together. Ignition allows for that type of scaling pretty easily. And if you're talking about the details of the throughput and storage, there's a number of factors, as I mentioned, but things like the actual drive that is stored on solid state storage versus spinning mechanical HDD traditional type of storage, the memory on a system, processing power, things like that, all come into play in terms of the amount that will go through for an individual database. And with multiple databases, that can scale, obviously, much higher than a single database.

08:18
Don: "So what are the advantages of SQL over traditional data historians?"

08:23
Kevin: So one of the major advantages is that the data is accessible. A lot of traditional data historians have it locked away in a specific format that needs to be retrieved or accessed using the tools from those historians, or it needs to be exported before it can be read by external software. Ignition, we do have methods to do that through Ignition and often, we'll recommend going through those methods if you want to use data-crossing partitions with some other considerations inside Ignition's Historian. However, with the Tag Historian, that data format is completely open, it's well-documented, it's inside our user manual that you'll see information on all the fields. There's no proprietary lockdown of that data in any way. It also is a standard technology. SQL databases, in general, whether it's Microsoft SQL Server, Postgres, MySQL, whatever it happens to be, that's a standard technology that the IT departments are generally really familiar with, and often plug directly into an IT department's existing infrastructure. So spinning up an additional database or using an existing database as that storage medium behind the scenes is often an easier ask than spinning up separate individual systems that have appropriate amounts of storage available that are just for a historian backend in a proprietary format.

10:03
Kevin: And then also one nice thing that it allows you to do is leverage existing backup and archiving tools. You don't have to have a set of separate archiving tools or separate tools that are related specifically to the historian for data export or data management for disaster recovery, for backup processes that can generally plug right into an IT department's existing plans for that and leverage some of those same technologies.

10:34
Don: Well, that clearly makes it the case that the data is there and the data's available. So the next question is, "What can I do with that data after it's in Tag Historian?"

10:45
Kevin: Sure, sure. So pretty much anything you want to. So dashboarding reporting are very common, ad hoc trending. And a lot of folks don't necessarily realize that it's relatively easy to stream the data to certain enterprise data systems. So for example, there's an Azure and AWS Injector Modules that are available that you can purchase from Inductive Automation. Those are Cirrus Link modules, and those can plug right into the tag system and stream data, historical data up to a central location. You can set up RESTful interfaces that are going to allow for querying of arbitrary time periods, and so you can connect pretty much any modern software system up to Ignition's Historian over REST if you're using that to get connected there. That's generally over WebDev or the Web Services Module that that connects through. And then of course, you can send data out however you want, in addition to that, with exports, with screen access to dashboards, and exports from those dashboards. As mentioned, reporting can be pulling those pieces up and then sending them out to other places as well. And then you can also, if you want to get really fancy, go and tie into our SDK, develop a module and to get to some of the low-level things inside Ignition directly if you want to create a completely separate, new technology that is either leveraging it or even storing some history into your own type of formats there through the Tag Historian.

12:27
Don: "What about scaling?"

12:28
Kevin: I mentioned it before, but multi-database architectures are simple, easy to set up. We have the Gateway Network, which allows for them to be communicated over from one server or from multiple servers. And distributed architectures are a possibility here. So if you have the single central historian scaling, this would be multiple databases, if you're just taking a look at that central system. If we take a look at the next slide here, the distributed historian would be having the data split up to multiple different locations where you might have data at each one of the sites. And then summary data comes back up to central, but central doesn't need to store all the information that's stored in the sites. And the Gateway Network allows for the querying over the remote connections, if you do wanna report or have dashboards centrally, real-time streaming that data from the gateways at the edge there at any one of the sites.

13:31
Kevin: And then we also have the next slide that I just added and that is for remote data collectors. And the idea here is that you can take data and ship it through to a central Ignition Gateway with remote data collection boxes. Any of these remote data collectors can be configured to do store-and-forward, so you're guaranteed not to lose data if that connection goes down and comes back up in the field. And these can be done either with Ignition over the Gateway Network with MQTT, over MQTT-connected devices from Ignition Edge to a central Ignition Gateway. And these are relatively lightweight communication mechanisms and a really nice way to allow you to have the data collection elsewhere, and then have all of the data stored centrally if that's what you wanna do.

14:26
Don: And I know you covered some of this, but, "What about plugging into other data storage?"

14:31
Kevin: Sure, sure. So if you wanna take Ignition's data, if you wanna take Ignition's Tag Historian or the event history as well, but mainly the Tag Historian, if you wanna take that data and you want to ship it to places other than SQL databases, or if you wanna store it to places other than the internal database and SQL databases, we do have an open SDK that allows for creation of what we call a data sink and a tag history provider, data provider that will be able to be sent wherever you want it to. So it still runs through our logic, it still runs through our compression algorithms and identify these changes, and it has the same user interface for tags. There are no changes there. The module simply allows you to take the work that we're doing to get that compressed or managed data stream, and then you can send it wherever you want to.

15:36
Kevin: The SDK is actual Java programming, and generally, you wanna be a programmer if you're going to use it. So what we've had is a number of third-parties come and create modules who have that programming expertise and have put those modules into our module showcase and in other places. So for example, a company has created an Influx database history provider. There are a number of cloud historian connections for different types of cloud historians. There are a few others for other systems. And we have partnerships with Microsoft and AWS and Amazon, and their connectors are going up to both of those. And in fact, Microsoft with Azure has created an open source, completely open source available connector to their the Azure Data Explorer, ADX system there.

16:32
Don: Kevin, thank you so much. That's a very good primer and a good runthrough of a bunch of Historian FAQs to really give us a little sense of where we're starting from in terms of the historian world. Now that we've answered those FAQs, let's go into a little bit of greater depth. So Kevin, I'm gonna turn this section over to you, and then I'll come back to wrap up and get the viewer's questions answered. So go and take it away, Kevin.

17:00
Kevin: Sounds good, it sounds good, happy to. So if we take a look at the technology stack here for Ignition as a historian, the incoming data streams there from the left inside this diagram are generally tag data that's coming in from... It could be anything, but often, that's coming from PLCs, RTUs, remote units that have tags inside the devices themselves and do tag-based communication. Those data streams could come from other places, and you can store anything you want inside tags as long as they're standard variable types.

17:39
Kevin: Once they get to Ignition, you can choose what you want to do with them. So you can see at the top, there's the event historian, and it's a little bit hard to see, but in that blue box, it mentions, "SQL Bridge Transaction Groups." That's the typical way to do events logging inside Ignition. So if you have a specific event you wanna trigger on, some process is complete, your batch run has just finished, you have a piece of luggage that's just moved from one spot to another spot, and you want to get a record of that, that maybe you want to later report on, or you want time periods in there, or you want areas where you can do aggregations from a start to an end time, going through that event historian is the way to do those things. And often, those records that are going into event historian are later paired up on dashboards with Tag Historian information that's streaming real-time as well.

18:36
Kevin: So that event historian data goes to SQL databases. That is the direct place that it is being written to. If you take a look down at Tag Historian, that information there, SQL database, in the blue boxes, you see the SQL database engine, internal storage engine, other storage engines. But engine isn't an official term from Inductive Automation. I used that for this webinar just to make it more clear what those are doing, but those basically plug in to our Tag Historian, and those are different ways to get the data into some storage medium. So through the SQL database engine, it goes into SQL databases. We added this internal storage engine, which a lot of folks don't realize, but that's built directly into Ignition starting with Ignition 8.1, and that allows you to use storage inside Ignition itself without a database connected.

19:38
Kevin: Now, that's intended for small sets of data storage, it's intended for small data sets, small numbers of tags. And basically, what we did is we took the one-week storage from Edge and we rolled that into the main Ignition product. So if you're doing very small amounts of historian, you can use that. We recommend having that set to auto-truncate as well because it's not a full SQL database, it doesn't have the full type of optimizations that you have inside SQL databases. It's not really intended for being a big historian. But if you've got a small application and you don't need much and you need a week of history or even a month of history that you want to show in some charts and graphs and you just have a handful of tags, that internal storage engine can be a really nice thing to just have available, so you often don't even need a database connected. You also, if you're using that internal storage engine, generally, it'll be folks who don't have any event type of history to store because that event history does go into a SQL database. And so, if you have event history, you're going to have a SQL database connected anyway, you might as well use it. But that internal storage engine, as I said, is available starting in Ignition 8.1.

20:53
Kevin: And then we have other storage engines. So anyone who writes a module that plugs into our Tag Historian and sends that information out wherever they want to, really. So that's what we call our data sinks here, and if you ever write a module, you could do that. Most folks who are in integration don't necessarily have a programming background, or not a computer programming background, or a Java background. And so often, it makes sense just to leverage what some module developers have created, if you want to or you need to use these other storage engines.

21:28
Kevin: There are a couple of examples that we have and that we plug into and one that's coming up as well. If you're going into the cloud, there's the Time Series Insights that is set up for Azure, and then AWS has their SiteWise set-up. Both of those have data structure. One of these questions that we had come in before the webinar, someone was asking about setting up structured data in a way that is going to plug in where you have other things that are tying into the historian. If you are using Azure or AWS as your place of choice for that structured data, those data structures they're from UDTs inside Ignition come through and stream through automatically to those systems if you're using those modules or those technology stacks. So there's some...

22:33
Kevin: If you're taking a look at those, give us a call. We're happy to walk you through the options because there are actually a lot of different options. So you can use those modules. We're still working on the... Through Cirrus Link's, and Cirrus Link is the one who's doing some of those connectors there, but they're still working on the Time Series Insights one. I believe that's on the development schedule. And there are options today for SiteWise. And so we're certainly interested in having that conversation if you need some orientation as to how all of that works. And I just... So I skipped past some of these slides. These are just focusing on those sections. So Tag Historian and those are the engines I just talked about, and then the event historian right there.

23:17
Kevin: If we talk about visualization, access to this data and backups of the data, the visualization through Ignition itself is pretty easy. So we have some options for Easy Chart, Power Chart, for the different visualization modules that we have. If you're not familiar, we have Vision and we have Perspective as the two visualization systems, and we have charts in both of those that make it very easy to pull historical information. 8.1 also introduced the Power Chart, which has, in addition to having access to the Tag Historian data, it also has access to SQL data if you set up a historian provider that is pointed at general SQL tables. That can be a really nice thing to do if you wanna tie into additional data sources that are coming from an enterprise that might not even be generated from Ignition. As long as the tables have a time stamp, you can use that and you can also essentially filter on a column, you can have a WHERE clause in there with a key value and a key column properties that are part of those. Also in the Power Chart, there's a tag browser built directly in and annotations.

24:35
Kevin: And if we're talking about just visualization in general, Tag Historian generally is tied through and generates data for dashboards. It has built-in aggregates for time windows, some built-in aggregate analysis, min/max standard deviation, things like that, that can be set up for those time windows. And then events inside those dashboards, if you're pulling from events for showing, you can show those event pieces of information, you can use those for time windows that you're actually doing these aggregates over. Certain events, you might even aggregate over specific points inside the event tables there, and those tables are as easy to generate aggregates as it is to write a SQL query, which if you've ever written one, you know for simple SQL queries, it's really easy.

25:32
Kevin: And then there's a number of folks who are doing machine learning as well. In Ignition, we have a variety of algorithms that are built directly into Ignition. And it does take a little more expertise, but we have an example on our Ignition Exchange, where you can download a copy of that example, you can dig into it to understand how it works, and then you can apply those same techniques to your own projects. We do have some folks who are tying in from Ignition to external machine learning systems as well and other AI systems. We have a few folks in our Discovery Gallery and in our different conferences over the years who've submitted information about how they've done that and how they've done some ML inside Ignition directly. So that can be a nice thing for customer projects as well.

26:25
Kevin: We're talking about access to the data, so we're talking about retrieval or visualization inside Vision. The SQL database and that event data is a direct connection from Perspective or Vision, and then the retrieval from the Tag Historian goes through the Tag Historian itself, and then that is directed behind the scenes to these engines. And so the SQL database engine, if you have your tag history going to SQL database, then it's going to pull it from there. For internal storage, it's going to pull from there. Other storage engines, it'll also pull from there, basically, be a conduit back and forth for data. And then we did add another thing inside Ignition 8.1 called the Historian Simulator, which allows for history data to be pre-populated as if you'd had it running for a while. That Historian Simulator is a nice way to show off certain things and to also test out a system if you don't have long periods of historical data already accessible when you first install something.

27:30
Kevin: Data accessibility is... And so it's all accessible over SQL for things that are stored inside the SQL database. REST access, I mentioned quickly earlier, but most folks, you need external access from modern systems. Modern IT systems that speak REST will set up a RESTful endpoint. And you can do that through either the WebDev Module or the Web Services Module and provide access to the system.tag.querytaghistory. It's script function there, and that's going to allow for pass again, and doing requests from anything for aggregates, to raw data that's behind the scenes, to pulling back sets of data that are at a certain time range or has a different windows that you're pulling in. And so that is a really nice way to make this data accessible without anyone needing to write SQL queries directly against the database tables. Those database tables are there if they are needed and they are documented, but it's generally nicer for those external systems to go to REST. That way, you don't have to worry about things like partitions and sets of tables and understanding the tables behind the scenes and reading up on the documentation. It's a fairly easy interface to go over.

28:53
Kevin: Data import and export, of course, can be done either through our visual tools or through scripting behind the scenes, and that can be made accessible in any way that you make data accessible inside Ignition. And so Ignition has about 100 ways to expose data and to talk to external systems. Some folks do database-to-database in transport. Some folks even go out to files that are being sent to individual other systems. And a lot of folks do connectors to things like SAP, even, other ERPs, other backend systems, things like Tableau that they might be using for some additional reporting or... But really, you name it.

29:38
Kevin: Data modeling, I mentioned quickly earlier if you're going through to Time Series Insights or if you're going through to really any modeling system in the cloud or anything that has a model. As I said, SiteWise is the one that's available inside AWS. Those are becoming more popular and more interesting to us and to our users, and so those are things that... The model comes over from Ignition if you're going into those. That model would be your UDT model from Ignition or your object structure inside Ignition, and that becomes accessible on that side. Of course, if you're modeling data inside Ignition, the data model is done with UDTs, with the user-defined types there, and those models are yeah, accessible, and you can see them easily in the tag browser. You can tie to those with various screen design tools if you're building dashboards directly inside Ignition as well, of course.

30:42
Kevin: And then backup and archiving, if you're going to SQL in terms of the data storage, then that backup and archiving is done through SQL, through the data storage tools that are available in SQL. Microsoft SQL Server, for example, has some great tools and automated tasks that can run, and often, that'll just tie into what an IT system already does.

31:06
Kevin: And I've just made it to the demo here, so I'm installing Ignition. This is 100% fresh install of Ignition. I wanted to show this completely from scratch. In case you're not familiar with Ignition, this should give you a sense of how you would get started if you were looking at using either the event-based historian over SQL Bridge, over the transaction groups that we have built into Ignition or the Tag Historian. And so I'm just going to click through the install right here, and it's installing on my local system. Ignition, of course, installs on across operating systems, so it could be Linux, Windows, macOS. And then if you're talking about accessibility of Ignition, it's accessible from whatever is on the local network and/or whatever has access to the Ignition Gateway, and that could be local desktops, that could be tablets and phones, we have an app, if you're using the Perspective Module, and of course, that's cross-platform, too, on individual desktops, so Windows, Mac OS X and Linux.

32:21
Kevin: And I'm just saying I agree to the terms and conditions. This is the commissioning. So if you've never seen Ignition installed before, surprise, Ignition's installed. It's that fast, it's that easy, and it is up, it's on my system and now, it's going through its start-up phase. The start-up phase is pretty quick as well here. And so as soon as it's done, it's going to take me to the Gateway homepage, and I can go through Configuration if I want to, or I can start with a Quick Start Project that is basically a mode that sets up a few simulation devices and a few simulation items behind the scenes so that I don't have to connect directly to devices here. So I think I'll go ahead and do that. I think I'll say yes to the Quick Start mode, which has those tags. So when I go into the Ignition Designer in a minute, I'll see some tags that are auto-created for me that are just example tags here. If you didn't have that Quick Start mode, then... Or if you said no to it, then the steps there to get access to tag data is generally just connecting to devices or connecting to other sources, connecting to OPC servers, and those connections are made directly from the Ignition Gateway web pages. And so you can go to Device, and under the Device section there, then you can go into to Set Up Specific Device Connections with IP Addresses and things like that.

33:52
Kevin: So here we go, I'm going to say, "Yes, I wanna enable Quick Start." So first thing it prompts me for. Of course, we have security, just about everywhere inside Ignition. Security is very important for the industries that Ignition is in. And so we're in a lot of critical infrastructure, we're in a lot of places that do power distribution, and oil and gas. And we are, we take that very, very seriously, and so we have a whole variety of security standards that are built into place. We can certainly answer questions on that, or you can take a look later. We have a Security Hardening Guide other things.

34:29
Kevin: But I'm gonna come in, I'm going to connect up. So I'm gonna connect to a database, create a new database connection right here, and I happen to have Microsoft SQL Server already installed here. I have a completely empty blank database that I called Demo here that I set up, so you'd be able to see what this looks like going completely fresh, as I said. So I'll type in my credentials and give it the database name right here for a demo. That will connect up there. And then I did set up another database that has a few things in it already, and this would be an example of if Ignition's connecting out to a separate enterprise system that has data that you can show inside Ignition if you care about that or if you want to have that in Dashboards, and I'll connect to that as well right here. And this is in a database called Ignition. Alright, so I'm all set, I'm connected up and I'm going to hit the button to get the Designer. And this is going to download the Designer for Windows, in my case. This is the Designer launcher, and I'll minimize this. This is a separate application that is now installing. It can be installed system-wide or just for my user, in this case. I am launching it up, and I'll see the configuration. It automatically detects my local Ignition Gateway here that I have, and that's what it adds, and I'll connect up.

36:11
Kevin: Now, it is launching right here, and up over here. And this is the sample Quick Start project that I was talking about. I won't actually use this project. I'll just do a fresh project. We'll call this Demo and pop down. I'll give it a database that I'm connecting to, which will be this database that I just set up that's completely blank, empty, available, and then I'll give it a project template. And I'll come in, and this will give me a little bit of menu, just by default, when I open things up. So it's launching up and it runs through and it starts the modules. And then as soon as it's done there, we'll see the Designer. If you've never seen the Designer before, it's basic design environment that allows for tag configuration, it allows for screen design, it allows for reporting, it allows for alarming, pretty much everything that's built into Ignition as features is what you can do in the Designer. And a quick orientation, in the upper left here, these are the different major sections of Ignition. You'll see me clicking around in the project browser here. And as I click to the different sections, you can see the rest of the screen adjusts appropriately. So these are different design environments. Basically, the whole Designer houses all of this, and then these are different work spaces for each one of the different features, major features that we have here.

37:42
Kevin: So what I'm going to do is pop down to my tags. I can see all of these sample tags that are available from my Quick Start, and I'm not going to keep you waiting here. What you wanna see, I know, is first, tag history. So I'll come over to our homepage and this is just one of the home welcome pages. And what I will do right here is I'll take all of these tags, every single tag that I see here, I'm going to select them right there, and I will edit these tags, and down, tell it to store history. So I'll say, "Store history, true," give it a database I wanna store it to. I could store it to internal as well, I could store it to anything that I have as an option here. So I'll go to a database and I'll go to this new, fresh demo database that doesn't have anything in it. And now, on my screen... So that's it, by the way. So there is now history being stored for everything that you see there.

38:50
Kevin: So I'm going to throw what we call an Easy or a Power Chart, in this case, onto the screen right here. And I'll shrink it down so you can see it a little bit better, expand this out. You'll notice that it automatically scales as I do things. And open up the tag browser inside Demo and now, I have all of these different tags that are just stored right there, and I can add whatever I want. Maybe I'll pull in things that are intended to be fairly realistic here, right? So we hit Add Tags, and I will say, "Give me tags for the last eight minutes," instead of the last eight hours, probably even drop that down so, "Tags for the last minute." Those are real values coming in, being stored to the Historian with, really, two clicks. I just had to pick the tag history, say yes, and then I had to point it at something, tell it where I wanted it to go, and that's all tag history.

39:48
Kevin: Inside this component, there's a variety of things that I can do. So I have a range brush, I can select a specific range. It's going to give me information about that range, in addition to information about the pins. So I can see min/max posted over a period of time. I can see standard deviation for what I've just selected. I can select multiple of these and I get multiple different groupings for each one of those time windows, if that's what I'm interested in. I can get rid of the individual range brushes if I want to. I can also set up monitoring, that's just going to hold at whatever point that I set it up right there, so those are all the values that I have coming through. And then I can set up annotations as well. I can put a note here that's going to say, something happened.

40:42
Kevin: Hopefully I'm gonna be a little bit more clear than that when I put in real notes, or when an operator did, but you can get the idea that that's how those work, just to be able to see that a little better. So this is the Tag Historian, tied into the Power Chart inside Perspective. Perspective is the module that allows us to run things inside a web browser, so I could pop this up relatively easily into a web browser itself, I had launched session right here. We can see this now running inside my web browser along with some of other options there along the side. You can see my annotation moving past my time window right there in the last minute, and some of these values going.

41:33
Kevin: Taking a quick look at the time, I will show you what the event historian... Some basics of how that looks, I don't know if I'll have time to build out the full dashboard that I was thinking about doing, but I'll come over here and pop that up and go over to the transaction groups. These transaction groups allow you to store history, however you want to. So I'm going to go and create a new transaction group and I will call this, maybe, Fill Log. I've got something that's filling and emptying, and filling and emptying. And to find a good tag to use for that, I'll actually pop over here first and take a look at some of my values that I have inside the historian itself. So if I take a look at my demo, I believe that some of these sign values are going up and down, and I could trigger based on those changes that are happening. So I'll clear out some of these other values and then we can zoom in on the sign values that we have. And I'll make this, so it's the last five minutes. Alright. And then we get values here, so these are different sign, and if I go here, I can see that that's 32, six and five right there. So it looks like sign seven is a good one that keeps going up and down about once a minute, it seems like, so we're gonna use that as a trigger. And I'll say that this is going to be, every time that it exceeds a certain amount, then I'm going to count that as a completion. This is one cycle that has gone through.

43:20
Kevin: So I'll come right over here to my charts that I had right there, to my transaction groups to this history now, and then I'll come down to sign seven, and I'll pull that right in here, say, I just wanna use this for reading and I'll use this as the trigger, so I'll evaluate it when the values have changed. Sign seven, I'm going to execute once when the trigger is active, and I'll say it's active any time that it's over 25, and I think that should capture it pretty well that it's exceeding, it's getting up to the top right there. I can set this up so it's re-evaluating it every second to see if it exceeded that point. Then of course when that happens, I wanna log some stuff with it. So I will store my timestamp, I will say, this is my events and time, I'll store an index, and I'll say, this is my event ID right here, and then this is going into a table and I'll call this, my Fill Log.

44:24
Kevin: I might wanna store some other information alongside it, and so I could take any tag value that I have here and give it specific name, so I'll go into... This will be a database column, and I can rename it here and I'll say, this is going to be amps that's coming in, maybe I have a motor that's being used for this filling, and so I'm logging the amps for that. I've got another one that's right here, that could be some sort of temperature, and then I could have another one that's going to be... I don't know, something else environmental, humidity, let's say.

45:02
Kevin: So I can hit save right there, take a look at that, and now this is set up. It doesn't have any executions yet, nothing has happened that has caused it to trigger, but we can see that sign seven right now is negative 24, negative 23, and as soon as this jumps past where the trigger is happening there, that 25, the positive 25, then we're going to get an execution of this group. And when we get that execution, we should be able to see it here in a few seconds, that's going to put a record into our database table that we set up right here, so we just called it Fill Log, and that's going to go into the Default Data Source. That Default Data Source, before the moment of 25, didn't have anything inside it, and now you can see that it has a Fill Log, and in fact, if I click right here, I can see the values just come through, so that just triggered as soon as I went past 25. And another minute from now or however long it takes to drop back down and then go back up over 25, we'll get another record.

46:08
Kevin: And so you can start to see how this can work, where you get these individual records into the database, and then of course, if you want to see these records and you want to do things with these records, they can go into individual screens, into dashboards, and the simplest way to show this is simply to have a query that pulls those in. So I'll create a new blank query, I'll call this My Fill Log Query, and then under Authoring, I can just pick it over here or I can open a Query Builder and say My Fill Log and I want to include My Event ID, maybe, Amps, Temperature, Humidity, and My Event Timer, I'm basically selecting everything, I hit Okay right there, I can hit Testing, execute that, I could see this information comes back, there's one record, but there will soon be two records, and then I'll come back over to my Visuals here, I will drop into my Visuals a Table Component, so I'll steal this one, this is normally My Alarm screen, you could see I have a few alarms reconfigured, but I will co-opt it and I'm going to drop a table right here on the screen, and inside this table, I will show information that's coming back from that. So I'm going to take My Data Property right here, simply bind it straight up to my query, under that Fill Log that I just set up, and there, then we have two records, those have just happened.

47:46
Kevin: And if I had a little bit more time here, and we would be happy to do this or show this to you, if you called us later and you wanted a demo, but I could set this up pretty easily where I click on this and then it would filter My Alarm or My Chart History based on the events that I've selected. So I could have a start time, I could have an end time that's associated with it as well, and then right below this, it's very easy to have additional components. I could have another Power Chart that's sitting right down below this, that's showing different things, I can also have a more complex screen that maybe has a gauge that's associated with it, that has whatever these values are shown right alongside it, in a nice visual dashboard type of way. I could have things that have a map that's associated with this, that's connected up to these different events that are here inside the log. This log will automatically refresh at whatever rate is set here, and so I could pull or add something other than the default rate, or I can pull at 30 seconds or 5five seconds, and in a minute or 30 seconds or whatever, a new record comes through, we'll see that automatically fill out inside the table.

49:00
Kevin: And then of course, it's relatively easy to set up a date selection or set it up where maybe you double click on one of these rows and it pulls up all the associated information there, that's tied to the date range that is here, so we just saw record 3three pop-in right there. Deploying is as easy as clicking the save button and taking a look over here. Now, if I go to Alarms, I see this, this is streaming real-time back from the database with whatever I just set up right there. You can probably tell that I enjoy this and I'd love to keep going, but I also really, really wanna get to some of your questions because there are a lot of good questions that came in, and I want you to get a lot of value out of this too. So with that said, I'll hop back to just the couple of last slides we have here and then jump into the Q& and A. We have benchmarks, and these benchmarks are things that we put together before this meeting because I wanted to give you some really good, overall information.

50:01
Kevin: This is the document that we'll be releasing that has a whole bunch of information about Ignition's Tag Historian, SQL database benchmarks for different databases, throughput, network quality, degradation of throughput if you have a terrible network, your throughput's going to be less, but there are some options for mitigating that with architectures, ideas and advice for optimization, security guidelines, encryptions, on-the-fly encryption at REST, and a whole variety of other good information in here. So if you're digging deep into the Historian for Event or for the Tag Historian, this is going to be a great document and a resource for everybody who is on this call, but a couple of quick snapshots here for what the benchmarks generally look like. Microsoft SQL Server is normally somewhere about 3 to 5 million Tags, if you're at a 10-second rate, and 10% of those Tags are changing. Postgres Timescale is somewhere around 3 million tags. Microsoft SQL Server normally hovers near the low number here, in a steady state. Microsoft SQL or MySQL is significantly lower. And then Oracle, we don't have great benchmarks for because we don't have a full enterprise license of Oracle, but in our testing it at least hits the performance in MySQL on the Express Edition. So on a full edition, we expect it would be significantly higher.

51:31
Kevin: If you're thinking about how you map that out to a system. Once again, this only applies if you're doing 10-second rate and 10% of those tags changing. If you're doing different rates, then you're going to want to plan this out differently than what these benchmarks are showing here, but if you're taking a look one to a hundred tags, you might be able to use the internal storage engine, 100 to 1,000 or 10,000 tags, SQL database with a hard disk or a slow disk, or a slow drive there. If you got 10,000 to 3three million tags, you're probably looking at something with fast storage. And at some point you're going to need to split up to multiple IO servers. Since Ignition, single servers generally are set up for somewhere between 100,000 and 500,000 tags. In some cases, you can push a single server beyond that, but in general that's kind of a rule of thumb for individual IO servers. And so, if you're looking at a million tags, you're probably looking at more than one. If you're looking at 500,000 tags, it might be two Ignition gateways that are IO servers going to a single Historian here.

52:38 Kevin: And so, if you get bigger with this, then you'll see that that scales out with multiple different Tag Historian servers. And then if you're looking at really high numbers or really high throughputs, if your numbers are a lot faster than 10-second rate or 10% of tags changing, and you're doing things where you have very large rates of change, then you might explore some of the other Historian storage engines that are available there... That are better possibilities there, since some of them have pretty high throughputs. This is an idea of what those estimated studies state throughputs are for the different databases, Microsoft SQL server, Postgres, Timescale, MySQL. And this has a few more details on all of that. As I mentioned, the companion document is going to have a lot of this information, so I'll more or less skip past that relatively quickly, and this information is inside that companion document as well. So overall throughputs, if you're storing things at a one-second rate, then everything scales down. That's gonna be 10 times faster, so 10 times less tags into one of these. And that companion document has everything that I just mentioned there and went through relatively quickly.

53:57 Kevin: So, with all of that said, Don, back over to you for a second and then maybe we can hit a couple of questions here.

54:05 Don: Kevin, thanks. That was really great. You've covered a lot of ground. And, you know, I've been involved in doing these webinars for over 15 years here, since we began the program, and I have to say, with one minute left in our webinar, you have, I think, broken the record for more questions in the queue than we've ever had before. So, with that said, we can get to a couple of questions here. I will just say so people know this, that we have... You saw him download it, but if you haven't tried Ignition, try it. You can try to download it in just a couple of minutes, you can... It never times out on the trial version, it's fully functional. And if you wanna learn more about Ignition, go to Inductive University, there are 600+ plus two to four-minute videos there, everything you wanna know about educating yourself into Ignition is there. And with that said, we're into Q& and A. It says, "I was told Ignition does not support Historian redundancy. Is this true?".

54:58
Kevin: I don't know who told you that. Yeah, Ignition definitely supports Historian redundancy. That's the quick answer. We can certainly... There's more technical details that you can discuss with us if you wanted to call us later, or just take a look at some of the documentation. But yeah, you do need to have the database itself set up either with redundancy or use Ignition's Tag History Splitter, that's an option there too. There are actually several options, but... Yeah, that's kind of funny. Yeah, no, we support redundancy across the board.

55:33
Don: So, a quick question. When you gonna get the document out that you were speaking about here, I'm assuming soon?

55:39
Kevin: Sure. Yeah. The intention was to have it for today, and it needs to go through just a little bit of marketing look and feel mockup since I... I put together a good portion of it, because I really wanted to get it out for everybody here, but I think that some of my graphics and some of my choices weren't necessarily the best. So they'll put on a little bit of a marketing touch on this document, and I think it should be some time in the next week. I don't wanna speak for them, but shortly.

56:12
Don: Close enough. Can we... And the next question, "Can we get a link to that Tag Historian Benchmark document?" I don't know if that's included into what's following up or not?

56:20
Kevin: It is the same document that was just referred to. So, yes, that'll be part of it all.

56:26
Don: Alright, that sounds great. And here's one, "Why are we not considering any NoSQL database, such as MongoDB, Gabbana or ElasticSearch for storing Historian data, as this may reduce the size of the Historian to a greater extent?"

56:41
Kevin: Good question. One of the slides that I skipped past a little bit had to do with some Historian storage sizes and that's covered pretty well in the document as well. If you're using a SQL database, there are some options for compression, and if you're using Timescale specifically, I believe that it can compress nearly 90% in some circumstances, so we don't have any real world benchmarks from that.

57:06
Kevin: But that can be a significant cost in data savings. The quick answer is, no one's created a module for NoSQL storage. There's one that somebody's created for Influx, and that's a time series database, but NoSQL databases are normally great at ingest and distribution, and not great at querying. And so, most NoSQL databases, if you were to write directly to them from Ignition and then you were to try to pull back a long period of time, you might not be super satisfied with the performance that would come back from that, because relational databases are much better at things that are relational, like tag paths related to values, and so if you're pulling back for an individual tag path, the NoSQL databases... Yeah, generally aren't going to be great at that. That said, I personally haven't connected it through as a storage engine, and so someone could certainly surprise me with some performance that they find from it, but that's been our impression to date.

58:13
Don: Thanks Kevin. Okay, I'm on one last question, then we're gonna wrap it up, "Is there a way to set up history pruning on individual tags?".

58:21
Kevin: Yeah. Yeah. Good question. So the overall connection from Ignition over to the Historian database has pruning on that connection. If you want to set up pruning for individual tags, the best way to do that is to set up a separate Historian connection to the same database, same DBMS anyway, Database Management System. Add a new database right alongside the existing one, so you're not overlapping those tables and then just use that as the Historian engine for those tags that you want a different pruning rate on. So that gives it a different target when you hit that drop down that says Historian. So it would be your database Historian... Yeah, so you'd end up with those two Historian connections, one would be your long-term one and one would be your short-term, or whatever the shorter pruning period was, and you could point all of your tags that you want that shorter pruning period at that shorter database connection.

59:19
Don: So, I'm gonna wrap this up. You could get a demo if you want, Kevin referred to it earlier, at the number here on the screen, so please feel free to take advantage of that. Kevin, thanks so much for all of the information shared for the demo. I really... I know you covered a heck of a lot of ground. I wanna thank everybody for participating. It really was a very, very active group today. So, if anybody wants to follow us or me on social media, please do stay connected with us. Here's ways you can catch... Also, you can stand to know on our events, if you hook up with our newsfeed and our podcast. We'll be back in April with another webinar and another Ignition Community Live, so please stay tuned. And with that, we are complete. Have a great day.