An event historian system merges stored event data snapshots into sorted event storage blocks. The system determines that a storage block contains a plurality of snapshots to be merged. A new snapshot is created into which the plurality of snapshots will be merged. The event data within the snapshots is combined and recorded into the new snapshot in a sorted order. The index files within the snapshots are combined and recorded into the new snapshot.
Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. An event historian system comprising: a processor; and a random access memory storage device coupled to the processor, said random access memory storage device storing event storage blocks containing snapshots and processor-executable instructions for merging snapshots in the event storage blocks, said processor-executable instructions comprising instructions for: determining that a storage block contains a plurality of snapshots to be merged, wherein the storage block stores event data gathered from a process control system over a period of time as an event data set, and wherein each snapshot independently covers a part of the event data set; combining event data of each of the plurality of snapshots by merging property data from each snapshot; combining index files of each of the plurality of snapshots into an index file set by sorting data index files thereof for each property; recording the combined event data in a sorted order as new event data in a new snapshot; assigning each index file to the combined event data in the sorted order; recording the assigned index files in the new snapshot; deleting the plurality of snapshots after merging into the new snapshot; and writing the combined event data comprising the new snapshot to a permanent memory storage device at one time.
An event historian system is designed to efficiently manage and store event data from process control systems. The system addresses the challenge of handling large volumes of event data by merging multiple snapshots into a single, consolidated snapshot. Each snapshot contains a portion of the event data set, which is gathered over a period of time. The system includes a processor and a random access memory storage device that stores event storage blocks containing these snapshots. The processor executes instructions to merge the snapshots by first determining that a storage block contains multiple snapshots requiring consolidation. The system then combines the event data from each snapshot by merging property data and sorting the index files for each property. The combined event data is recorded in a new snapshot in a sorted order, and the index files are assigned to this new data. After merging, the original snapshots are deleted, and the new snapshot is written to a permanent memory storage device in a single operation. This approach optimizes storage efficiency and simplifies data retrieval by reducing the number of individual snapshots.
2. The system of claim 1 , wherein the system initiates a merge process at the beginning of a first time interval.
The event data merge system described previously starts the snapshot merging process automatically at the beginning of a defined time interval. Instead of waiting for a specific trigger, the system proactively initiates the merge operation on a regular schedule (e.g., every hour, every day).
3. The system of claim 1 , said determining that a storage block contains a plurality of snapshots to be merged further comprises determining that the plurality of snapshots to be merged are of similar sizes.
The event data merge system described previously considers the sizes of snapshots when determining if they should be merged. Specifically, the system only merges snapshots within a storage block if they are of comparable size. This prevents a very large snapshot from being merged with many smaller ones, which could lead to performance issues or inefficiencies in the merging process.
4. The system of claim 3 , the processor executable instructions further comprising instructions for sorting the snapshots into snapshot groups, each snapshot group comprising snapshots which are of similar sizes.
The event data merge system described previously groups snapshots by size before merging. Snapshots of similar sizes are placed into groups, and then the merging process operates on these groups independently. This grouping strategy ensures that merges are performed on snapshots with roughly the same amount of data, optimizing the merge operation.
5. The system of claim 4 , wherein the snapshot groups comprise a first and a second snapshot group, said first snapshot group containing snapshots between zero events and a first number of events and said second snapshot group containing snapshots between the first number of events and a second number of events, wherein the second number of events is ten times the first number of events.
The event data merge system that groups snapshots by size uses two size categories: a first group containing snapshots with a low number of event data entries (between zero and a certain threshold), and a second group containing snapshots with a higher number of event data entries (between the first threshold and ten times that threshold). This two-tiered grouping approach allows for efficient merging of snapshots with varying data density.
6. The system of claim 1 , said determining that a storage block contains a plurality of snapshots to be merged further comprises determining that the storage block has not received event data for a first time interval.
The event data merge system described previously merges snapshots within a storage block if that block hasn't received new event data for a specified time. This indicates that the data within the storage block is relatively stable, making it a suitable candidate for merging snapshots into a more consolidated form.
7. The system of claim 6 , wherein a final merge process is executed on the storage block in response to the storage block not receiving event data for a first time interval, the final merge process comprising the repeated merging of snapshots in the storage block until only one snapshot remains.
The event data merge system performs a final, aggressive merge on a storage block that has been inactive (i.e., not received new event data) for a specific duration. This "final merge" involves repeatedly merging snapshots within the block until only a single snapshot remains. The goal is to fully consolidate the data in the block, optimizing storage utilization and potentially improving query performance.
8. The system of claim 1 , said determining that a storage block contains a plurality of snapshots to be merged further comprises determining that the plurality of snapshots to be merged are frozen.
The event data merge system described previously merges snapshots only if they have been marked as "frozen." The "frozen" status indicates that the snapshot is no longer actively receiving updates and is therefore safe to be merged without risking data inconsistencies or conflicts.
9. The system of claim 1 , said determining that a storage block contains a plurality of snapshots to be merged further comprises evaluating the most recent snapshots for merging before older snapshots.
The event data merge system prioritizes merging newer snapshots over older ones. When selecting snapshots for merging within a storage block, the system first evaluates the most recent snapshots. This approach ensures that the most current data is consolidated first, which may be particularly beneficial for real-time or near-real-time data analysis.
10. The system of claim 1 , the processor executable instructions further comprising instructions for eliminating duplicate event data from the combined event data in the new snapshot.
The event data merge system removes any duplicate event data during the snapshot merging process, creating a single, unified snapshot that doesn't contain redundant information. This deduplication step reduces storage space requirements and improves the efficiency of queries by eliminating unnecessary entries.
11. A method of merging snapshots of event data in an event historian system comprising: determining, by the event historian system, that a storage block stored on a random access memory storage device contains a plurality of snapshots to be merged, the storage block storing event data gathered from a process control system over a period of time as an event data set, each snapshot independently covering a part of the event data set of the storage block; combining, by the event historian system, event data of each of the plurality of snapshots into an index file set, the index file set including merged property data from each snapshot; combining, by the event historian system, index files of each of the plurality of snapshots, the combined index files of the snapshots including sorted data index files for each property; recording, by the event historian system, the combined event data in a sorted order as new event data in a new snapshot on the memory storage device; assigning, by the event historian system, the index files to the combined event data in the sorted order; recording, by the event historian system, the assigned index files in the new snapshot; deleting, by the event historian system, the plurality of snapshots after merging into the new snapshot; and writing, by the event historian system, the combined event data comprising the new snapshot to a permanent memory storage device at one time.
An event historian system merges event data snapshots into sorted storage. It determines when a storage block contains multiple snapshots, where the storage block holds event data from a process control system, each snapshot covering a portion. The system combines the event data into an index file set, including property data from each snapshot. It also combines index files from each snapshot, sorting them by property. The combined data is then recorded in sorted order as a new snapshot, with assigned index files. Finally, the original snapshots are deleted, and the new snapshot is written to permanent storage.
12. The method of claim 11 , wherein the event historian system initiates a merge process at the beginning of a first time interval.
The event data merge process described previously starts automatically at the beginning of a defined time interval. Instead of waiting for a specific trigger, the system proactively initiates the merge operation on a regular schedule.
13. The method of claim 11 , said determining that a storage block contains a plurality of snapshots to be merged further comprises determining that the plurality of snapshots to be merged are of similar sizes.
The event data merge process described previously considers the sizes of snapshots when determining if they should be merged. Specifically, the system only merges snapshots if they are of comparable size.
14. The method of claim 13 , further comprising sorting the snapshots into snapshot groups, each snapshot group comprising snapshots which are of similar sizes.
The event data merge process described previously groups snapshots by size before merging. Snapshots of similar sizes are placed into groups, and the merging process operates on these groups independently.
15. The method of claim 14 , wherein the snapshot groups comprise a first and a second snapshot group, said first snapshot group containing snapshots between zero events and a first number of events and said second snapshot group containing snapshots between the first number of events and a second number of events, wherein the second number of events is ten times the first number of events.
The event data merge process that groups snapshots by size uses two size categories: a first group containing snapshots with a low number of events, and a second group containing snapshots with a higher number of events (ten times the first group).
16. The method of claim 11 , said determining that a storage block contains a plurality of snapshots to be merged further comprises determining that the storage block has not received event data for a first time interval.
The event data merge process described previously merges snapshots within a storage block if that block hasn't received new event data for a specified time. This indicates stable data for merging.
17. The method of claim 16 , wherein a final merge process is executed on the storage block in response to the storage block not receiving event data for a first time interval, the final merge process comprising the repeated merging of snapshots in the storage block until only one snapshot remains.
The event data merge process performs a final merge on a storage block that has been inactive (not received new event data) for a specific duration, merging until only a single snapshot remains, consolidating the data.
18. The method of claim 11 , said determining that a storage block contains a plurality of snapshots to be merged further comprises determining that the plurality of snapshots to be merged are frozen.
The event data merge process described previously merges snapshots only if they have been marked as "frozen", indicating they are no longer being updated and safe to merge.
19. The method of claim 11 , said determining that a storage block contains a plurality of snapshots to be merged further comprises evaluating the most recent snapshots for merging before older snapshots.
The event data merge process prioritizes merging newer snapshots over older ones when selecting snapshots for merging. This ensures the most current data is consolidated first.
20. The method of claim 11 , further comprising eliminating duplicate event data from the combined event data in the new snapshot.
The event data merge process removes any duplicate event data during the snapshot merging, creating a unified snapshot without redundant entries, reducing storage space and improving query efficiency.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
December 12, 2014
May 23, 2017
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.