The Next Evolution in Historical Data

Speakers

Travis Cox

Chief Technology Evangelist

Inductive Automation

To put our own spin on a famous quote: Industrial organizations that don't learn from their operational history are doomed to repeat it. Storing and analyzing historical data is essential for reducing inefficiencies, optimizing processes, and enabling faster, better decision-making. That's why the new Ignition 8.3 release offers more options than ever before for working with historical data.

In this webinar, you'll learn about the capabilities of the new Industrial Historian Solution Suite for Ignition. With enhanced extensibility, this Solution Suite not only allows you to use Ignition as a powerful historian but as a complete system for building historians. Join us to see how leveraging the power of your historical data can help ensure a brighter future for your organization.

• Check out the built-in, efficient Core Historian Module
• Leverage external databases with the SQL Historian Module
• Learn how to build custom historians with the Historian API
• Quickly turn historical data into great-looking charts

The time to register is now — even if you can’t attend, we’ll send you the recording!

Transcript: 

00:00
Travis Cox: Welcome, everybody. We appreciate you joining us for our webinar here today on, "The Next Evolution in Historical Data: Ignition's Industrial Historian Suite." Thank you so much for being here today. I'm Travis Cox, the Chief Technology Evangelist here at Inductive Automation, and I work very closely with end users and integrators every day to help them build their projects and to solve challenges. Today, I continue to share my expertise through education, awareness, exploring the art of the possible with Ignition. Today we're really going to talk all about Ignition's Historian. In fact, the full Historian Suite.

00:35
Travis Cox: So here's our agenda for the next hour. I'm going to give you some quick introductions to our software, Ignition, and our company, Inductive Automation. Then I'll talk a little bit about why historical data is so valuable and how the Historian suite in Ignition 8.3 is different than the Historians in our past versions. Then we'll look at what you can do with the new core Historian in Ignition 8.3, and of course, the SQL Historian and other engines that we have. I'll talk about how you can create your own custom Historian with the Historian API. Then I'll tell about two different types of charts in Ignition, as well as other ways to query and use historical data.

01:12
Travis Cox: I'll show you a demo of the Industrial Historian suite, and I'll give you a sneak preview of a few Historian updates we have planned for the future. If you have any questions for me during the webinar, just type them into the questions area of the GoToWebinar Control Panel, and I'll conclude by answering as many questions as I can. So, you should definitely watch till the very end for that. If I can't respond with your question here today, please reach out to one of our account representatives, and we'll be happy to answer it for you.

01:41
Travis Cox: As well as you can reach out to our sales engineering team, where they can offer more technical deep dives on some of the things that we're covering here today. And of course, yes. The recording is being done here today and will be emailed to you on Monday, and the slides will be available on our website. So, we got a nice jam-packed agenda here today, and like I said, if you have any questions, make sure to put those into the questions area of the GoToWebinar control panel. All right. So, for those of you who are new to Inductive Automation, Ignition, here are a few facts about us.

02:14
Travis Cox: We are a software company. Our software, Ignition, is used by 69% of Fortune 100. So it's literally used every single day inside some of the world's biggest companies. We have over 5,000 integrators worldwide in our integrator program. We have a highly diversified customer base across all verticals with thousands of Ignition installations in over 140 countries. We've been in the industry for over 23 years, and we have just about 400 employees here in the U.S. and in Australia. And we're really dedicated to changing the way people think about industrial software. And we're really passionate about bringing modern technology and making it accessible and be able to build practical solutions now and into the future.

03:00
Travis Cox: And we do that with our software called Ignition. It's a universal industrial application platform for SCADA, MES, IIoT, and much more. It acts as a central hub for everything on the plant floor and beyond. You can build any kind of industrial application with it. It's web-based, web-managed, and web-deployable to desktops, industrial displays, and mobile devices. It has an unlimited licensing model that allows you to have as many tags, screens, clients, device connections, you name it. You can keep building. It's cross-platform, and it offers industrial-strength security and stability.

03:34
Travis Cox: All right. So, let's start our discussion here today on discussing why historical data matters in the first place. It's not just about collecting interesting facts from the past. When you properly store and analyze your historical operational data, you can turn that raw information into actionable intelligence that directly impacts your bottom line. By tracking your performance and metrics over time, you can see patterns of where your processes are inefficient, when equipment failures tend to happen, and where there are opportunities for optimization.

04:06
Travis Cox: Your historical data should be a foundation for continuous improvement and informed decision-making. You can do a better job of assessing your current operations by comparing it against your baseline performance. Instead of reactively repairing equipment after it fails, you can switch to more predictive maintenance strategies that are more proactive. It all boils down to the importance of data.

04:30
Travis Cox: Then there's the hottest topic of the moment right now, AI. As your organization adopts AI and machine learning tools, your accumulated historical data is used to train the algorithms that automate optimization and detect anomalies much more efficiently than humans can. And it doesn't just help operations. Historical data supports regulatory compliance, warranty claims, and quality audits by providing documentation of how your processes were executed.

04:56
Travis Cox: It also preserves institutional knowledge so when experienced operators retire, they don't take all their valuable insights with them because the next generation of engineers and managers can access and analyze the data they worked with. Here's a real-world example of why historical data makes a huge difference. There's a water district in Bakersfield, California, that was having serious data visibility problems because of its old 2000-era SCADA system. They were dealing with multi-minute data latency delays, limited diagnostic visibility, no real-time insight, and no ability to analyze trends. So they chose an Ignition Gold certified integrator called Avadine to replace their old SCADA system with Ignition.

05:55
Travis Cox: Now things are going much better at the water district. They have data streams every second instead of minutes apart. They have high-resolution long-term storage for five years. They also have easy trend analysis on their HMI and predictive maintenance for their pumps. They've gone from a system where their operators were basically flying blind and reacting to failures to one where they have all the real-time and historical visibility they need to manage their operations proactively. That's one example out of many, showing us how looking back at your historical data is actually one of the best ways to move your operations forward.

06:20
Travis Cox: This is why we've always looked for new ways to help you do more with your historian data in Ignition. Ignition made history as the first OT platform to leverage SQL databases for historical data. Our very first product that we ever made here at Inductive Automation was called Factory SQL. It's a pre-Ignition.

06:33
Travis Cox: It was to take OPC data from OPC servers and store it into SQL databases, both for historical and for real-time purposes. It was all about being able to leverage IT technologies and get that data into an accessible format. We don't want data to be closed or proprietary. We want that data to be able to be used so we can get the most insights out of it. Using open technology from the IT worlds to serve your history needs was groundbreaking and very successful. But as technology has advanced, and our users have requested improvements over the years, we saw the need for historian capabilities that are much faster and more extensible and that offer you more options.

07:13
Travis Cox: So, in Ignition 8.3, we've completely rethought our approach to time series history storage. We've created an entire API around time series data, making Ignition not just a historian but a system for building historians as well. There's an actual module SDK where third-party authors can build additional storage engines and query engines that allow for extreme flexibility. This lets us introduce a lot of new functionality and make many improvements that our users have requested over the years.

07:46
Travis Cox: Our new historian is called the Industrial Historian Solution Suite. It's designed to optimize your time series data with the powerful new built-in Core Historian, the ability to use multiple historian types, and built-in tools for trending and analysis. At the heart of the new Historian Suite are two modules: the Core Historian Module and the SQL Historian Module. Together, the modules act as one powerful solution, which provides you with the power and flexibility to meet virtually any historical data that you need and a foundation for future growth. So, let's take a look at both of these modules individually.

08:09
Travis Cox: The first thing you'll notice about the Core Historian is that it's much faster than our previous historian. Its write performance is up to eight times faster, and its read performance is up to eight times faster than our SQL Historian. And its write performance is up to eight times faster. The read performance up to 600 times faster than our legacy internal historian that was based on SQLite. And it stores and queries data up to 10 times faster than Ignition's previous provider as well. So, the new Core Historian has a lot of advancements.

08:56
Travis Cox: The new Core Historian is based on a widely used high-performance open-source time series database called QuestDB. And there's minimal configuration needed, so you can start it in just seconds. It's got minimal disk footprint, so you don't have to worry about taking too much storage space when compared to others, like SQL databases. It's got more efficient local storage. It's built right into Ignition. It's just native; it's part of the install; it just comes out of the box. So, it's really fast to get started, and there's no maintenance required.

09:28
Travis Cox: It supports pruning and archiving data, which wasn't possible with the external SQL database directly in Ignition. You had to go and manage that in the SQL database itself. And all these features make an ideal historian for new Ignition users. Most of the new greenfield projects that we see, they start with the Core Historian. So for on-premise for storing two or five years of data, we can leverage the new time series historian that's built right into Ignition.

10:09
Travis Cox: Of course, that's not your only choice. That is a new part of Ignition 8.3. But we have the SQL Historian Module as well. This has been in Ignition from the very beginning. It allows you to log data to any external SQL database like MySQL, Postgres, Microsoft SQL Server, MariaDB, and a lot more. It's ideal for large-scale storage, and it automatically supports compression, partitioning, interpolation, and aggregation for any SQL database. It's a great choice for those who want to store data outside of Ignition. And as I mentioned, this is tried and true part of Ignition that we've had from the very beginning. Between these two tools, you have a lot of options for how you want to store and leverage historical data.

10:59
Travis Cox: Now, those modules are both very powerful. What else is really powerful? The fact that we have the new Historian API built into Ignition. This lets you quickly set up several kinds of historians so you can store and retrieve data from practically any system, which is something you've never been able to do in Ignition before, easily. We've leveraged the new API to build the SQL Historian, the Core Historian, and a few additional options. There's the CSV Historian, which can retrieve historical data from CSV files that is read only. So we can bring data in and leverage that in our charts.

11:23
Travis Cox: We have the Historian Splitter that allows you to take data and write it to two different databases at the same time, simultaneously, so that we can store data, for example, to our core Historian locally, as well as maybe to a SQL database centrally. We also have the DB table Historian, which allows you to read historical data from external or custom-managed SQL tables. These can be tables that maybe you're using the transaction groups to store, or maybe you already have some time series data in some tables that you want to be able to query that are in a particular format, kind of a wide format.

12:01
Travis Cox: So, these are some great additional options you have for being able to bring more types of data into Ignition. We also have the OPC HDA Historian, which connects to OPC HDA servers to read in data from those. This is a server, so we're just reading in from external HDA providers. And we have the Remote Historian, which allows you to connect to another Ignition gateway over our gateway network and read or write data from that system.

12:27
Travis Cox: This really allows for distributed storage or centralized storage across many gateways. And this is actually a very good strategy when you have multiple Ignition servers that have the Core Historian; you have a central Historian where you want to then also write to a central, or maybe you want to query from all the local cores, all in one place using that gateway network. So lots of great options that we have with the different engines that come with Ignition 8.3.

12:58
Travis Cox: And that's not all. Here are some other capabilities that we have in Ignition 8.3 to do even more with your historical data. We have really revamped our store and forward system, and it's backed by protobuf serialization. So, store and forward is much more efficient and faster and better, which is incredibly important because we don't want to lose any data, especially trying to write that to a centralized SQL database or through the gateway network. We now support rich data modeling with the API in that the UDTs you're building in Ignition and, of course, a lot of the metadata can be stored along with the data in the database so that we can bring that out over time.

13:39
Travis Cox: We have better bulk storage facilities; we have had annotations for a while, but we've made that now a part of the API so you can use any tools to be able to query the annotations and even store annotations. We have seamless history point renaming, which has been a sticking point over the years. If you want to rename tags, how can I rename that within the historical database as well? We have native data aggregation to aggregate data, do things like min/max, standard deviation, and many other aggregation functions. There's easy query bindings that we'll show here today.

14:18
Travis Cox: It supports calculations, and it also supports allowing you to modify stored data directly through Ignition; that is, correcting points in the database or potentially adding or injecting new points that you want through manually or through some other mechanism. So lots of great new features that we have now in Ignition 8.3. Of course, one of the key ways to capture the value of historical data is visualizing it. And Ignition makes that really easy. We have two visualization modules, and both have really powerful chart builders. In the Vision Module, which is ideal for classic plant floor and desktop screens, there's the EasyChart component, which was developed with the Tag Historian system in mind.

15:04
Travis Cox: You just drag on the EasyChart onto a screen, that trend and make it very, very easy to recall that data within Vision. And you can look at historical, real-time, or manual modes with this. There's a lot of great options that the Easy Chart has. We've also done the exact same thing within Perspective, where we have a Power Chart, and this is ideal for HTML and mobile-responsive applications. This one lets you actually go and find; it's very ad hoc, allows you to find any of the tags. You can drag them onto the screen, you can easily see the history.

15:50
Travis Cox: You can again do real-time or historical. You can see the values over time. You can get calculations off of this, can customize that full trend right within the runtime. We're going to show example of these two components here in Ignition. They're both very powerful, easy ways to get started visualizing the data that comes from the storage engines that we were talking about here before. And there's also other ways that we can leverage historical data in Ignition from the Historian Suite.

16:26
Travis Cox: I mentioned the bindings. We have both Perspective and Vision historical data bindings where we can query data to any kind of component. So, commonly that would be to things like tables or XY charts or pie charts or bar charts or any kind of custom component that you'd like. Especially Sparkline charts are very popular on HMI screens, where we can show a small amount of historical data right next to the process value. So, we'll show that binding here today. There's also scripting functions where we can query historical data, and we can do a lot of those functions that we were talking about earlier. So there's a whole suite of new scripting functions, system.historian.

17:07
Travis Cox: There's storing data points, there's querying raw points, there's storing annotations, querying metadata. There are a lot of functions that exist in there. I'll show you where you can see that in the documentation. You can use those for a lot of custom things you want to do. Some people go and make their own function to go query data, and then they run that through ML algorithms right within Ignition scripting, or they use WebDev to create custom REST APIs to expose that data to third-party applications.

17:38
Travis Cox: I will show an example of that here today as well. And of course, then there's the module SDK. The module SDK not only lets you create new Historian engines in Ignition, but it also allows as a module you can get access to this whole system where you can store and query data. This is ideal for third-party systems to be a native citizen within Ignition. There are modules out there for other Historian tools like Canary and Time Base, which is the open-source technology, the free time series database from Flow Software.

18:11
Travis Cox: There's a lot of different modules that are being created to work with different types of storage engines, different ways they want to leverage that data. But what's great, is that this just extends, and it's all part of the simple way of going to tags in Ignition, being able to set up history, and then query the history in our normal mechanisms. But there's a whole breadth of possibilities with that SDK. Hopefully, that gave you some sense of just the possibilities that are there with Ignition 8.3. The best way to show all of this is to do a live demo where we're going to show you everything that we have to offer with this.

18:51
Travis Cox: We're going to show how to store data, SQL databases; that's our tried and true. Been doing that for a while. We're going to show how we can leverage the new Core Historian that's built into Ignition now, which is based on QuestDB. We're going to show how to query the data from both these places, as well as to also use the splitter to store data to both of them at the same time. We're going to show you where the diagnostics are for the store and forward system.

19:16
Travis Cox: We're going to show other ways that we can query the data inside of Ignition. We're going to show a few other engine types like the CSV Historian and the Database Table Historian. And we will also show an example of WebDev where we can create a custom API endpoint to get that data. So, there are a lot of cool things we're going to show in the demo here today. As we go through this, if there are any questions you guys have or potentially other areas that you're curious about, please feel free to put that into the GoToWebinar area. This is a new hot topic. People really want to know the specifics around these features.

19:50
Travis Cox: All right. So, let's go over to Ignition. So for me, this is the most fun part. I love doing demos of Ignition, just showing how easy it is to get things started. So I've already installed Ignition. I had the latest LTS release, Ignition 8.3.3, installed here on my computer. And all I have connected right now in Ignition is I did go ahead and connect to a SQL database. So I'm connected to a MariaDB database that's local that will be used for the SQL Historian.

20:21
Travis Cox: But again, any SQL database you connect to, whether it's MariaDB, Postgres, some people use Postgres with the TimeScale extension. There's Microsoft SQL Server, lots of options. We can use any of those for storing data. I've got that set up. I also have a couple of simulator devices set up, and I've configured some tags inside of a project already for Ignition. I've got a bunch of simulation tags, and I got a dairy simulation running here as well. So, there are only two things that I have set up in the designer. I do have the tags created over here. So you can see here's the dairy, here's some refrigeration tags. There's a simulator that has a bunch of realistic and random values that we can use.

21:05
Travis Cox: So, I've got them in here, but I have nothing set up for the Historian just quite yet. I just wanted to get the very basics going. Now, it's very quick to get connected to the simulator or PLC and get those tags in there. Also extremely fast to get connected to that SQL database. Let's show our tried and true Historian that we've had from the very beginning, which if I go here to services, and for those of you again who are new to Ignition 8.3, the new Web UI is completely revamped.

21:32
Travis Cox: There's different ways you can get to these areas. You can actually go through the navigation, find those. You can also go up here and you can search for that area. So, if I want to search for Historians, I can go to that, which brings me to that same page. So, it's really nice to be able to leverage the new search function to get to where you want to go, but it's also very intuitive just where this configuration lies. So, I'm going to go here to the services Historians. I'm going to create a new one, and you can see there's quite a few options in here.

22:03
Travis Cox: Again, if you use the module SDK, these options can be extended. You could build your own custom ones. And again, this is where third-party, different vendors are doing it, as well as it's possible for anybody to build. That's a free part of the SDK. I'm going to use our tried and true SQL Historian. I'm going to set that up. Go next. I'm going to call this Maria... Or I'm going to call it SQL Historian. I'm going to select my database, which I have MariaDB here.

22:28
Travis Cox: And with our SQL Historian, we have data partitioning. And so, what we do in the database is we create tables every month by default. You can see the partition is one month. You can certainly turn that off, or you can change the duration if you want it to be one week. So if you have a lot of data you're storing in the database, you could actually change that partitioning, which basically just changes the format for how we kind of create the tables and store that in that database. There's also data pruning, so we can go and delete old partitions in the SQL database, but we don't do any archiving.

23:05
Travis Cox: That would be something you would do inside of the SQL database itself. And so again, this is assuming that we connect to an external SQL database. We have that set up. Again, this is how we've been doing it from the very beginning. So, I'm going to keep all this default here, create the new Historian. It is now up and running. And once we have that created, I go to my designer. I can now go to any tags. So let's go over here to these ramp tags.

23:30
Travis Cox: I'm going to right-click edit. And with this, we're going to simply go down here to history enabled true. And I'm going to select that SQL Historian. We have the ability to specify deadband style if you want to do like analog or discrete. Discrete simply logs values on change based on the deadband. Analog uses an algorithm to determine if the if the value has actually changed within a particular window before it actually stores that point. So trying to be more more efficient for storing analog data specifically.

24:02
Travis Cox: I personally like discrete for a lot of things because I like just controlling when the value logging values based on the deadband, which can be either absolute or percentage-based deadband for how you want to store that. So here it's absolute, which is just if a value changes greater than or within 0.01, it will then consider that a change. Percentage be it, there had to have to be some percentage of change for that to actually happen. We also, by default, we're storing data as it changes. So if data doesn't change in 20 days, nothing gets logged in the database. This is true for any of the engines that we have.

24:39
Travis Cox: So, this would be true for SQL historians as well as our Core Historian. It'll just log on change. If you want it to log periodically, you certainly could, and you could log at some guaranteed rate. You can also force values to be inserted. As you can see here, there's a min time between samples, and a max time between samples. So, if you... Right now, it's saying that we'll only store data at the minimum every second. So if we do have values changing faster than a second, the Historian will throttle that and go every second. Max time right now is zero, which means it's unlimited.

25:13
Travis Cox: So, if the value doesn't change in 20 days, nothing gets logged. But if you wanted to guarantee a value gets stored in the database at every hour or every day, you could certainly set that up. So, a lot of options that we have for the Historian here. I'm going to come back and talk about this metadata in a moment. But I'm going to turn on history for all these ramp tags to the SQL Historian and press okay. Now those tags are being logged. I'm going to bring up my database here.

25:41
Travis Cox: This is DBeaver that I have on Mac, and I can see there's my test database for MariaDB. I don't have any tables. If I refresh this, Ignition creates all the tables in the database for us to store, which is great. If I look at the TE table, I can see that here's all the ramps that are being logged. And of course, if I look at the data for this month, I can see the actual data that's being logged in there. Now, of course, we can query that inside of Ignition.

26:10
Travis Cox: So here I have the Perspective. This is in Perspective. This is the the Power Chart component. You can see there's the SQL Historian. And if I go down, I can see there's my ramp. I can drag it over. And let's look at the most recent, let's say, five minutes instead. And so we're going to start seeing that data come in live. So all these tags are now storing to that SQL database. Now, this is the Perspective Power Chart. There's a lot of power that's in with this component. All you do is you go over here to Component Palette, you search for Power Chart, and you drag it onto your screen. And it has a lot baked into it.

26:48
Travis Cox: So by default, we can actually get the X-trace. We can see what the values are here. Again, we're doing real-time mode. We can set a historical mode as well. I can do a range where I can select a range. I can get calculations on that range. So, things like minimum, maximum, average, delta sums, upper and lower control limits, standard deviation. A lot of things just kind of built in right there. And you can have more than one range in there.

27:15
Travis Cox: The API allows us to store annotations. So if we wanted to go in here, I could add an annotation. So here is a note. I can add that in, and that's stored in the database, the SQL database, along with that particular value that we can recall on trends. And this is again, we can interact with these annotations with the API, these components as well as with the API directly, the functions we have. This component allows you to also do a lot of configuration on the axes, the pens, the plots, the subplots you have there, what you want to show in the table there. A lot of configuration you can do, so you can really customize how you want that chart to work.

27:59
Travis Cox: And this is meant for people just to come in, get a very quick, easy way to start looking at data and analyze that data. And it being an ad hoc component, you could just keep dragging on new, new tags and really playing around with this component. Along with, of course, zooming in and out and being able to kind of find the data you're looking for.

28:18
Travis Cox: So, this is the Perspective one. We also, of course, have, if I go to Vision, let's go over here to Windows. I'm going to create a new window, Vision, for those of you who are still using that. We have the EasyCharts. EasyChart's been in Vision for a very long time. What's cool about this is I can actually go over here to my ramps. Let's take a few of these, drag them on, and I can start seeing data quickly in there. There's a lot of things people really loved about the EasyChart. You can see down here in the time bar where the data is actually located. You can save data, print it, all that good stuff within the component. A lot of customizations you have in the component itself.

28:59
Travis Cox: Same with the Power Chart, axes, and subplots, and really looking at that data. Those are two great ways to look at data. We're going to show some additional things that we can do for that. But that's how we get started. The SQL database, that's been really easy in Ignition for a long time.

29:16
Travis Cox: Now, let's go back here and create a Core Historian. This is the brand-new built-in local Historians optimized for high throughput, demanding workloads. It's great for new people coming in as everything is. There's nothing you have to install externally. Everything's in Ignition, again, built on QuestDB. So I'm going to go ahead and create that. I'm going to call this Core Historian. It also has partitioning, but that's actually built into the QuestDB itself for how it works. There's a lot of great because we have it built in. There's a lot more that we can do with that, as well as pruning, as well as archiving.

29:56
Travis Cox: We have the ability to archive and specify where we want those archives to be stored, like on a different disk, network drive, something like that. So I'm going to go ahead and create that Historian. Now, I've got the Core Historian in there. Just like before, we go some tags, really simple. I'm going to go these realistic tags here, edit them, and I'm going to turn this on. I'm going to store it to the Core Historian. Now I've got two options in there. Again, I can do all the same stuff we had before. But I want to talk about the include metadata.

30:25
Travis Cox: This is a new feature in Ignition 8.3 with a new API where we can actually store metadata along with the values in the database. Right now, the Core Historian supports that. We can store things like engineering units and ranges and all that information that's really important about the tag that we can then query from the API itself. I'm going to go ahead and turn it on. Right now, there's only one way to query that through the scripting function. But in the future here, we are going to have a lot more ways that we can, in Ignition, get that data back onto screens automatically.

31:01
Travis Cox: But we're definitely moving towards a way where all the metadata and the modeling can all be stored along with the data, so you can easily query data against that. So, I'm going to go ahead and simply now press OK. That's now stored in the QuestDB. Now, this is one where you're not going to be able to see the QuestDB database directly. I mean, I'm not going to be able to open up a query browser directly there. If I go to disk, I'll find all the QuestDB storage files that are there.

31:31
Travis Cox: It's in that format. It's not proprietary. It's that format. We want in the future, add more ways for you to query that data directly, especially from external tools. But again, that's where all of our API is coming to play, be able to look at that data. So again, if I look at our Power Chart, EasyCharts, there's the Core Historian. The data is now logging in there. Let me get rid of that one. I'll go the realistic. I'll bring realistic zero on there. There you can see, that data is coming in. So now I've got data going to the Core Historian. I got some tags going the Core Historian, some tags going the SQL Historian.

32:06
Travis Cox: Really, really easy to leverage that. With the Core Historian, we're all storing that metadata along with it. All right. Let's go one further. Let's go over here and create a new Historian. That is the splitter. We've had the splitter for a while. This is a great way to log to two places at the same time. I can actually specify my primary as the local, of course, Historian, and maybe my secondary as a SQL Historian.

32:30
Travis Cox: What we can actually do too is not only restoring to both places, but when we query it, we can say limit the query to the primary for some range. Maybe we're querying two years data here locally, and then somewhere else we want to query the rest of the data. You can certainly do that with this limit primary Historian query. I'm going to go ahead and just press OK. We're going to store both places. I call that one splitter. If I had tags that wanted to log there, let's go over here to our dairy and take a bunch of these. Right click, edit, and I'm going to go down to store it.

33:00
Travis Cox: Now, you'll see the splitter is in there. Let's go ahead and log those. Here, if I refresh, you're going to see that I've got the core, the SQL, and the splitter. I can query from any one of those. If I see now dairy is part of the splitter, as well as it's part of the SQL, as well as it's part of the core, because all those are there. Really easy to start looking at how we want to store that data.

33:28
Travis Cox: If I go over here to the diagnostics in Ignition... Sorry, if I go over here to services, let me go. Oh, sorry... Platform. Go to store and forward on the platform. I can see the store and forward for these. This is guaranteeing that the data is going to get there. We don't need it for our local one for the QuestDB because it's right there. I mean, it is logging locally. But for the splitter and for MariaDB, we can see the throughput. If I were to lose connection, we'd see that it was going to continue to back up. Guarantee we don't lose that data. This is a very efficient store and forward now. We've always had it, but much more efficient now with Ignition 8.3.

34:11
Travis Cox: All right. So, let's move on to show some other... Before I show some additional options for how we can query data, let's show a couple different ways we can query or look at that data in Ignition. So, one is through our binding. Both in Vision and in Perspective, we can do binding to get data back. So if other components like tables, I'm going to bring a sparkline chart on here. And with the sparkline chart, I can actually bind the points to tag history.

34:16
Travis Cox: So let's say I want to look at the most recent hour, sure, of data. Or let's look at 10 minutes. I don't have that much data in here. And I'm going to bring back, let's say, our refrigeration, the discharge pressure here. I'm going to bring that back. And I'm going to pull that data every 30 seconds. And so we're going to bring that into a sparkline chart, and it's polling, so I can have that value. It's on the same HMI screen as the process data. So, I can see both real-time data and how it was trended in the last couple hours, giving us a lot more situational awareness. Very, very important for high-performance HMI design to be able to do that.

35:18
Travis Cox: And that's just utilizing the binding. And it's the same type of thing if I wanted to bring that data back onto a table component. If I bind that to tag history, I can select multiple tags if I want to. So, let's bring back a bunch of these tags here. And so, let's look at the last hour of data. We can also, of course, do aggregation here. We can query calculations if you want to. And it's going to be a wide format, so I'm going to go ahead and press OK. And here's the data coming back for that. So, I can see that information in a table view. So, lots of good ways that we can bring back the data on different components and through the binding mechanism.

36:04
Travis Cox: So, there's built-in components and there's bindings, as well as there's a plethora of scripting functions to look at this data. So, we have had for a long time system.tag.query, tag history, and our query tag calculations. Those are now marked as deprecated because we have new functions for them. Those will still work, but it's recommended that they switch over to the new Historian functions, where we've got a whole slew of them, as you can see here, to browse, to query raw data, query aggregated data, query annotations, query metadata.

36:40
Travis Cox: We can store metadata, we can store data points, we can update that tag path from old path to a new path. All these functions exist in here to do that. So, for example, I created a Historian screen, where I can go and select, let's say I go and select a tag like my realistic zero and say, "You know what? Starting on the 26th, I had a value of 100 and I want to store that value." What we did with this is simply just call the system.historian.storeDataPoints with a single value. Of course, you could store lots of values, but this is a way to create a screen, maybe correct data point, or potentially inject a new data point very easily in there.

37:24
Travis Cox: So, lots of cool things for that. I've also used the functions I have leverage WebDev. And so WebDev allows you to create endpoints on Ignition. So I can create REST APIs. I've got a WebDev endpoint here called tag history, where I can do a post, and actually this one is on the exchange. It's using the old function to query tag history. And basically, I can pass any parameters I want into it, just like you would if you called that script directly.

37:51
Travis Cox: So, it's a way to call that script function through a REST API. And so, if I were to open up Insomnia here, I can say, "I want to query the realistic zero for the last five minutes." I send that, post that. Here's the endpoint. Now, I've got the data, the last five minutes that came back. And I can query this in a third-party system very easily. So, that just uses those functions to get that data out.

38:14
Travis Cox: And again, you can use scripting to create... To bring data back and then use ML functions or whatever you want to do. The data is accessible. We want it to be out in a very simple format. Once you log that data, you can use it in a variety of mechanisms. All right. So we still have some more to show here because there are some additional Historian functions that we have in Ignition, our engine. So I'm going to go back to the web UI. If I go and create new ones, there's some additional ones here, like the CSV Historian and the DB Table Historian.

38:49
Travis Cox: I want to show both of those. I think they're really cool. The CSV Historian allows us to actually query data from CSV files. And so, for me, I'm going to bring this over here for a second. So I can copy this. I'm going to call this my flow file, or I'll just call it CSV... CSV Historian. And the file path is the CSV that I'm trying to work with. So, I've got a file that's named flow data.csv. And I'll show you the data here. But that's where I want... That CSV is in my file system, on my desktop, and I want to use that in Ignition.

39:34
Travis Cox: So I'm going to go ahead and create that. And this data looks like this, where I've got the data going back to yesterday at 3:00 P.M, got some pressure, vibration, flow, motor amps, and various tags that are there. So, if I go now... Oops... To my designer and I refresh this component, you'll see there's the CSV Historian, there's those columns that I had in there. And if I drag this in, right now you can see there's no trend data source available.

40:01
Travis Cox: That's just because I don't have data in that time period. So, let's go to yesterday, and let's go to 3:42 P.M to 11, 12, midnight there. So, go ahead and press apply on that. Oops. This was actually, I think, 2025. I meant to go to 2025. 29. Yep, 29, 342, 29. Okay. So, let's do that. There we go, there's the data in that time period. So, it queried from that CSV, and I'm able to see that data here in my components. Then I can query that with any of the functions that I were showing here earlier, leveraging that CSV.

40:54
Travis Cox: So, that's another option you have. And the final one I'll look at is the DB Table Historian. DB Table Historian allows us to query data against standard wide database tables where we have columns for the timestamp and other process data. You may have custom tables you've already created or you might be using. Some people want to be able to leverage the transaction groups from SQL Bridge in Ignition and use that on the standard trend charts.

41:22
Travis Cox: So, I'm going to do that. Before we set that up, I'm going to go into the designer and go over here to our transaction groups. I'm going to create a new transaction group that's historical group, and I'll say it's realistic, or I'll do random history. And I'll go over here to my random values. I'll store all the random values in a table called random history. Have it automatically create table. We'll store data every five seconds. It's going to be a wide format with the timestamp being T_stamp. And everything's good to go. Go ahead and save this.

42:04
Travis Cox: So, in my database, if I were to pull that up, I'm going to see a table called random history. And it's a wide table, a column for each of those process values. And there's the timestamp. So it's a pretty common format for a lot of people, especially if they're logging data when events happen on trigger. Transaction groups are very, very valuable for that. But then we can actually, of course, query that directly within the... You can query it, do a SQL query on a table or chart. But sometimes it's nice to be able to just query that data from the Power Chart or the Easy Chart.

42:39
Travis Cox: And that's what this database Table Historian is all about. So I'm going to call it DB Table Historian. And I'll select MariaDB, and it's going to go and find all of the tables that are applicable here. So let's go to our charts. Let's go refresh. Let's get rid of that. Let's go to real time, apply. And here's my DB Table Historian. There's my random history. And let's bring on a few of these data points. Just like that, I've got that data now on a trend. Super easy to use. So, lots of cool options that are there with the different engine types.

43:19
Travis Cox: As you can see, there are the remote Historian that go through the gateway network. We also have a simulator Historian, allows you to do simulation data for testing right here. And so with these, and again, you can extend the list of Historians with the SDK if you wanted to. But that's again where things like connections to Timebase, or others people have wanted. So, that's a comprehensive look at the different kind of functions that are there with the new Historian in 8.3, the different options, especially the new Core Historian.

43:56
Travis Cox: And that then begs the question, what's coming next? You saw I mentioned a couple things already. What else are we doing? So, we've got lots of ideas on how we extend this going forward, for this next year, or future releases. Coming in this next year, we're looking at these areas right here. We want to, for our Core Historian, make sure it has redundancy built in for a redundant pair so we can store that data to both master and the backup, and it will always be in sync. We're also creating a backup restore tool for the Core Historian.

44:33
Travis Cox: This is where you can actually back up the data from our web UI, or restore data to that because it's built into Ignition. It's those QuestDB files that are local. Again, we have the functions to do a lot of that, but we want to make this a simple thing in the web UI to do that as well as an archive management tool. These are all things that are really in the works, or very close to that. We also want to take advantage of all the new features of that API. I was talking about the metadata that's there and being able to query that, of course, annotations and make that accessible everywhere in Ignition.

45:09
Travis Cox: This could be new binding functions. This can be new components or updating existing components. We're trying to make sure it's accessible to everything, especially with our Power Chart, Easy Charts. You want to be able to see that information. Now, I showed WebDev... Example of getting data out to third party through a WebDev endpoint. That is actually going to be something we're going to natively build into Ignition and the REST API that is in Ignition 8.3.

45:34
Travis Cox: There will be Historian endpoints to be able to query data and use all different parts of the API. You don't have to create your own custom ones. It'll just be a native part of Ignition. These are things that we are really excited about, and I know there's lots of other updates. We also have plans in the future for adding additional engine types people want us to do, to look at different kind of databases like Snowflake or Influx or other things like that. So, there's more options as we go forward. None of those are set in stone, but we're really looking at how we can leverage the power of this to go further.

46:00
Travis Cox: All right. So, with that, I hope you enjoyed exploring the Industrial Historian Suite with us here today. If you want to try it out yourself, simply go to the website, go to the downloads area, and you can download and try it out.

46:23
Travis Cox: And that two-hour trial period just takes a couple minutes to get it up and running, and you can try it for as long as you want. Just reset that trial every two hours. You can test everything that I showed you here today and play around with that and see what it does for you. I mentioned that this is the Historian Suite. There are other suites that we have in Ignition 8.3. If you want to go and check those out, you can see the URL here at the bottom on our website to see the different options that are out there.

46:48
Travis Cox: These are basically suites of modules where you get the suite, and you don't have to in the future. With upgrade protection, you get any additional modules we add to that suite. You get that for free. It's trying to make the whole offering of Ignition a lot simpler for you to navigate, especially as you purchase it. So, fewer things you purchase, but you get more long-term value, especially with upgrade protection. With all this, you can check out our Inductive University, which is our free training website. With hundreds of educational videos, you can learn Ignition step by step, at your own pace.

47:17
Travis Cox: There's also a comprehensive online user manual that you can refer to at any time. No matter what your experience level is, you can always learn more at no cost whatsoever. If you're watching from outside North America, we have a network of international Ignition distributors who provide different business opportunities. We also, of course, have our two offices here, our headquarters at Folsom and our office in Australia. If you want to talk to either of those teams, please feel free to reach out to them to learn more. With that, we're going to get to the Q&A period. There was quite a few questions that came in, so just give me a moment to open the screen and kind of get my bearings around this.

47:59
Travis Cox: All right. So, one question here was, can the Core Historian be hosted in a virtual machine with redundancy? So, right now, obviously, as I mentioned, the Core Historian and the future roadmap is for redundancy. It does work in redundancy in that get a master backup, master fails, goes backup, and then the Core Historian works. But the problem is about the data. So, when I fail to the backup, the data is going to be stored to the backup, and it's at that point, once it gets back to master, data is going to master.

48:39
Travis Cox: So, it's not synchronized between two. That's part of what we're working on with redundancy. So, it certainly does work with that. Another thing that's, I think, a spin-off of the question there is, can the Historian be on its own? And you certainly can. The Core Historian can be on its own Ignition instance. So, you can have an Ignition server in a virtual machine or container or whatever that is; it is just the Core Historian. And you can have your back-end server or front-end server as additional servers there that connect to it through the gateway network and can query data from that, query and store data.

49:11
Travis Cox: So you can actually move that onto its own instance. There's a question here. Does the Core Historian have asset framework or equivalent? So, that's something that we are working on where basically in Ignition we have UDTs, which are the data models, the assets, and we're beefing it up so that... The API supports it, but we can store all the data against those models, and then you can actually then later ask questions about this history based on, like, "Hey, give me back all my motors."

49:43
Travis Cox: I want this information so I can compare and do various things and that we can expose those kind of assets to third party as well. There's a lot of work happening on that right now. So, it's not fully built out, but we have some of the foundation pieces that are ready to go for that. Let's see here. So a question, "Can you elaborate on QuestDB? So, you need a separate DB setup with your SQL instance." So I kind of showed two different things today, and I didn't mean to muddy the water, but I showed the SQL Historian, which does require external SQL database.

50:17
Travis Cox: That's for people who do want to log data to an external database. You do not have to do that. If you want to start with just the Core Historian, that would be leveraging the built-in QuestDB in Ignition. You don't need anything external. Just install a vanilla Ignition, and you can turn it on and you can use it, and everything's stored locally to disk. And so, if you want to learn more about QuestDB, go check it out. It's a very popular open-source, high-performance time series database, and we've embedded all of that in Ignition to make things easier.

50:46
Travis Cox: There are questions about, "Can I install QuestDB externally from Ignition and use that externally?" We are also moving towards that, as well, so that we can have both the local embedded one, as well as people who want to have a clustered QuestDB externally that they can work with. All right. So, another question here. "Is there an easy way to migrate the Microsoft SQL-based database from Ignition 8.1 to 8.3?" So, that is all backwards compatible. So, if you were using the Historian with the SQL database in 8.1, it does the same in 8.3. So, that all works just like normal. But if you... A lot of people ask us, can they migrate from another SQL database to the Core Historian? So we do not have any tools right now built in to do that migration.

51:35
Travis Cox: It is possible you could do it through scripting, using our functions where you can just query data from one and store it to the other. And it would take some time for that to run, but ultimately it is possible for you to do. But we don't have any built-in tools ourselves for that migration. But if you have again, SQL database existing, that will continue to work and you could decide if you want to leverage the Core Historian or not.

051:50
Travis Cox: So, another question, "Is QuestDB part of the gateway backup?" So, when you make a backup of Ignition, that just backs up all the data directory, which of course does include that. And so, what we want to do is have more import-export capabilities for that. So you didn't have to; you can have a backup that doesn't have it, where you can export that separately, but it's part of our data directory. You can see that on the file system.

52:18
Travis Cox: "Will the Core Historian work on the free two-hour trial period?" Absolutely. It does. Every, feature in Ignition runs in that two-hour trial period so you can fully evaluate it.

52:28
Travis Cox: There's a question from... Another question about the local Historian for Edge. "Is local Edge Historian faster now too?" Absolutely. We're able to take advantage of those same features and leverage that for Edge. Edge is designed to store data locally and then send that data up to a centralized system. So, we now have more efficient ways of storing that locally.

52:50
Travis Cox: The question is, "Is the Core Historian part of the Ignition license, or is a separate license needed?" So the core Historian is, needed. So, you have to buy the Ignition platform, which is $1,200 for the Ignition platform, and then the Core Historian. You don't buy the Core Historian by itself right now as part of the suite. The Core Historian and SQL Historian are together part of the suite so that you don't have to. We're not trying to separate it right now; just keep part of the suite. You get all of that.

53:22
Travis Cox: And as we add new engines in the future for other databases, they'll be part of that suite. You'll get those for free as well. But it is something you have to license. So, if you want to put the Core Historian on its own dedicated instance, it'd be the Ignition platform plus that suite, and you'd be good to go.

53:34
Travis Cox: So, the question is, "So the Core Historian is built in? Not an external database?" Correct. It is built in; it's not external. If you want to join it with SQL data sets, the question is like, You have SQL databases out there, and you want to join things together. That's the beauty of Ignition. You can query local, and you query the remote. You can bring them together on screens and show that information. You certainly could do that. It won't be through one single function, but there are certainly ways you can make that happen.

54:00
Travis Cox: So, I think I confused some people in the demo. You're referring to MariaDB? To QuestDB? Is it an alternative name? So, no. MariaDB was a SQL database that I had installed separately that was using a SQL Historian. That was one path, its own thing that you can do. And then QuestDB was what we have built into Ignition, that's part of the Core Historian. So, there are two different ways you can store data. You can do both; you could do one or the other or both. It's up to you. Just kind of showing the different possibilities there today.

54:34
Travis Cox: So, there's another question here about Canary and Time Base Historian. It says, "Do we see that in the Historian engines?" If you install those modules, if you go to those vendors, Canary or to Flow Software, those modules you install it, they will add new engines. You'll see in that list, the Historian list, and you can add those, and it will work just like everything else that we showed here today.

54:49
Travis Cox: Cool. There's a couple questions, funny enough, about ProVUE. We are going to be going to ProVUE and doing a full demonstration of Ignition 8.3 and showing the full power of everything. The Historian, the Core Historian, will certainly be on display there, along with logging data to the cloud, like to a Snowflake instance. And we'll also be kind of showing a lot of other features off with Ignition 8.3. So it'll be a great demo of just the power of what's there.

55:15
Travis Cox: All right. Last question here, "Is SQLite going away from Ignition, built-in? So, no. Not right now. If you look at the list, you'll see that there's an internal historian, called Legacy. We still have that because we want backwards compatibility, but people will probably migrate from the SQLite one to Ignition for that.

55:35
Travis Cox: So, there were... Sorry, many other questions in the list. I'm sorry I didn't... Be able to go through all of them. It was just me here today. But if you want to get those questions answered, just contact our teams. More than happy to do a deeper dive on all these with you guys. We really appreciate the time here today. We'll be back next month with another webinar. But until then, connect with us on social media and subscribe to our weekly newsfeed email. You can also see our latest blogs, case studies, and videos on our websites. Thank you, everybody here, for joining us today. Have a great rest of the day, and we will see you next time.