Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A method, comprising: representing a file, of a file system and mapped to an address range of memory, using a data structure comprising a data unit accessible by a data lock and a metadata unit with a first portion accessible by a metadata lock and a second portion updateable by an atomic operation without a lock; and utilizing the data lock and the metadata lock in a shared mode based upon an estimate that a data modification of a request changes a structure of the data structure and the data modification is delayed.
Technology Domain: File System and Memory Management Problem Solved: Efficiently handling file modifications in memory-mapped file systems, particularly when data changes are anticipated to be delayed and structural. Summary: This invention describes a method for managing files within a file system that are mapped to a memory address range. The file is represented by a data structure. This data structure includes a data unit, which is protected by a data lock. It also includes a metadata unit. The metadata unit is divided into two portions. The first portion is protected by a metadata lock. The second portion is designed to be updated using an atomic operation, meaning it can be modified as a single, indivisible unit without requiring a lock. The method involves using both the data lock and the metadata lock in a shared mode. This shared mode is employed when an estimate indicates that a requested data modification will alter the structure of the data structure and that this modification is expected to be delayed. This approach aims to optimize performance by allowing concurrent access to data and metadata under specific anticipated conditions of modification.
2. The method of claim 1 , wherein the data lock and the metadata lock are acquired for requests modifying the first portion of the metadata and the structure of the data structure.
3. The method of claim 1 , comprising: utilizing the data structure for file content lookup operations.
A system and method for efficiently organizing and retrieving file content data involves a specialized data structure designed to optimize file content lookup operations. The data structure is configured to store file content data in a manner that allows for rapid and accurate retrieval based on content-based queries. This approach addresses the challenge of efficiently managing and accessing large volumes of file content data, particularly in systems where traditional indexing methods may be inefficient or resource-intensive. The data structure is constructed to support various file content lookup operations, including but not limited to searching for specific content within files, identifying files containing particular data patterns, and retrieving metadata associated with file content. The system may also include mechanisms for dynamically updating the data structure as new file content is added or existing content is modified, ensuring that the lookup operations remain accurate and up-to-date. Additionally, the system may incorporate techniques for optimizing the performance of the data structure, such as indexing strategies, caching mechanisms, or parallel processing, to enhance the speed and efficiency of file content retrieval. The method may further include error handling and validation processes to ensure the integrity and reliability of the stored file content data. By leveraging this specialized data structure, the system provides a robust solution for managing and retrieving file content data, improving the overall performance and scalability of content-based file management systems.
4. The method of claim 1 , wherein the data structure has a hierarchical structure.
5. The method of claim 1 , comprising: executing the data modification as a batch operation.
6. The method of claim 1 , wherein the data structure comprises a log tree comprising a tree element.
7. The method of claim 6 , wherein the request comprises a write request executable without shifting the tree element.
8. The method of claim 1 , wherein the request is associated with a page fault generated by a machine-level store instruction to the file system that supports direct access to the memory.
9. The method of claim 1 , wherein the second portion of the metadata comprises metadata fields unprotected by the metadata lock.
10. The method of claim 9 , wherein the metadata fields are updated using the atomic operation on a memory location without acquiring the lock.
11. The method of claim 1 , wherein timestamp metadata of the second portion of the metadata for the file is updated without the lock using the atomic operation.
12. A non-transitory machine readable medium comprising instructions for performing a method, which when executed by a machine, causes the machine to: represent a file, of a file system and mapped to an address range of memory, using a data structure comprising a data unit accessible by a data lock and a metadata unit with a first portion accessible by a metadata lock and a second portion updateable by an atomic operation without a lock; and utilize the data lock and the metadata lock in a shared mode based upon an estimate that a data modification of a request changes a structure of the data structure and the data modification is delayed.
13. The non-transitory machine readable medium of claim 12 , wherein the data lock and the metadata lock are acquired for requests modifying the first portion of the metadata and the structure of the data structure.
This invention relates to data management systems, specifically techniques for managing concurrent access to metadata and data structures in a storage system. The problem addressed is ensuring data consistency and preventing corruption when multiple processes attempt to modify both metadata and the underlying data structure simultaneously. The solution involves a locking mechanism that coordinates access to both metadata and data to prevent conflicts. The system uses a data lock and a metadata lock to control access to a data structure and its associated metadata. These locks are acquired for operations that modify either the metadata or the structure of the data. The data lock ensures that the data structure cannot be altered while the lock is held, and the metadata lock prevents changes to the metadata during the same period. By acquiring both locks together, the system ensures that modifications to the metadata and the data structure are atomic, preventing inconsistencies that could arise from partial updates. The locking mechanism is implemented in a non-transitory machine-readable medium, such as a storage device or memory, and is designed to work with a data management system that processes requests to modify the data structure. The system ensures that only one process can hold both locks at a time, enforcing serialization of conflicting operations. This approach is particularly useful in distributed or multi-threaded environments where concurrent access to shared resources is common. The invention improves data integrity and system reliability by preventing race conditions and ensuring that metadata and data remain synchronized.
14. The non-transitory machine readable medium of claim 12 , wherein the instructions cause the machine to: utilize the data structure for file content lookup operations.
15. The non-transitory machine readable medium of claim 12 , wherein the data structure has a hierarchical structure.
16. The non-transitory machine readable medium of claim 12 , wherein the instructions cause the machine to: execute the data modification as a batch operation.
17. The non-transitory machine readable medium of claim 12 , wherein the data structure comprises a log tree comprising a tree element.
A system and method for managing data structures in a computing environment, particularly for optimizing log-based storage and retrieval operations. The invention addresses inefficiencies in traditional log storage systems, which often suffer from high latency and scalability issues when handling large volumes of data. The solution involves a log tree data structure that organizes data into a hierarchical tree format, where each tree element contains log entries and metadata. This structure enables efficient insertion, deletion, and retrieval of log entries while maintaining consistency and reducing access time. The log tree is designed to dynamically adjust its structure based on usage patterns, allowing for optimized performance in both read-heavy and write-heavy scenarios. The system further includes mechanisms for compressing log entries within the tree elements to minimize storage overhead. The invention is particularly useful in distributed computing environments, databases, and real-time data processing systems where low-latency access to log data is critical. The log tree structure ensures that operations such as appending new logs, searching for specific entries, and maintaining data integrity are performed efficiently, even as the volume of log data grows. The system may be implemented in software, hardware, or a combination thereof, and is compatible with various storage mediums, including solid-state drives, magnetic disks, and cloud-based storage solutions.
18. The non-transitory machine readable medium of claim 17 , wherein the request comprises a write request executable without shifting the tree element.
19. The non-transitory machine readable medium of claim 12 , wherein the request is associated with a page fault generated by a machine-level store instruction to the file system that supports direct access to the memory.
20. A computing device comprising: a memory comprising machine executable code for performing a method; and a processor coupled to the memory, the processor configured to execute the machine executable code to cause the processor to: represent a file, of a file system and mapped to an address range of memory, using a data structure comprising a data unit accessible by a data lock and a metadata unit with a first portion accessible by a metadata lock and a second portion updateable by an atomic operation without a lock; and utilize the data lock and the metadata lock in a shared mode based upon an estimate that a data modification of a request changes a structure of the data structure and the data modification is delayed.
Unknown
March 23, 2021
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.