Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A system, comprising: a memory that stores computer executable components; and a processor that executes computer executable components stored in the memory, wherein the computer executable components comprise: a remote interface component that interfaces with a group of remote storage devices of an elastic cloud storage (ECS) system that stores a client object in a chunk of a first fixed size and protects the client object via an erasure coding procedure performed on the chunk, wherein the group of remote storage devices store remote data to first chunks of the first fixed size; a local interface component that interfaces with a local storage device of the ECS system, wherein the local storage device stores local data to second chunks of the first fixed size and comprises a local cache configured to store a portion of the remote data; a client interface component that receives a request for the client object; and a service component that: determines the chunk that stores the client object is one of the first chunks that store the remote data; and determines a fragment of the chunk that stores the client object, wherein the fragment is representative of data fragment of a second fixed size, less than the first fixed size, employed by the erasure coding procedure.
The system is designed for efficient data retrieval in an elastic cloud storage (ECS) system, addressing challenges in accessing client objects stored across distributed storage devices. The system includes a memory and a processor executing components to manage data storage and retrieval. A remote interface component connects to multiple remote storage devices in the ECS system, which store data in fixed-size chunks and use erasure coding for redundancy. A local interface component interacts with a local storage device that also stores data in fixed-size chunks and includes a cache for storing a portion of remote data. A client interface component handles requests for client objects. A service component determines whether the requested object is stored in a remote chunk and identifies the specific fragment of that chunk containing the object. The fragment size is smaller than the chunk size and aligns with the erasure coding procedure used for data protection. This system optimizes data access by leveraging local caching and precise fragment identification, reducing latency and improving retrieval efficiency in distributed storage environments.
2. The system of claim 1 , wherein the service component is configured to request, via the remote interface component, fragment data contained in the fragment.
This system is part of an Elastic Cloud Storage (ECS) environment and efficiently manages data by identifying and retrieving specific data portions. When a request for a client object is received, a service component first locates the chunk (of a "first fixed size") on remote storage devices that contains the object. This chunk is protected using an erasure coding procedure. The service component then precisely determines a smaller "fragment" (of a "second fixed size," smaller than the chunk's size) within that chunk, which is relevant to the erasure coding. **To fulfill the request, this service component is specifically configured to then request the data contained within this identified fragment directly from the remote storage devices, utilizing a remote interface component.** This targeted retrieval ensures only the necessary fragment data is fetched, improving efficiency compared to retrieving the entire larger chunk. ERROR (embedding): Error: Failed to save embedding: Could not find the 'embedding' column of 'patent_claims' in the schema cache
3. The system of claim 1 , wherein the service component is configured to transmit, via the client interface component, a portion of the fragment data containing the client object.
A system for managing and transmitting data fragments in a distributed computing environment addresses the challenge of efficiently distributing and processing data across multiple client devices. The system includes a service component that processes data fragments, each containing a client object, and a client interface component that facilitates communication between the service and client devices. The service component is configured to selectively transmit a portion of the fragment data containing the client object to the client interface component. This selective transmission ensures that only relevant data is sent, optimizing bandwidth usage and processing efficiency. The system may also include a data storage component for storing the fragment data and a processing component for analyzing or transforming the data before transmission. The client interface component may further include authentication mechanisms to verify client access rights before data transmission. This approach improves data handling in distributed systems by reducing unnecessary data transfer and enhancing security through controlled access. The system is particularly useful in applications requiring real-time data distribution, such as cloud computing, IoT networks, or collaborative software platforms.
4. The system of claim 1 , wherein the service component is configured to store, via the local interface component, the fragment data to the local cache.
A system for managing data fragments in a distributed computing environment addresses the challenge of efficiently storing and retrieving data fragments across multiple nodes. The system includes a service component that interacts with a local interface component to handle data operations. The service component is specifically configured to store fragment data in a local cache via the local interface component. The local interface component facilitates communication between the service component and the local cache, ensuring that data fragments are stored in a manner that optimizes access speed and reduces latency. This configuration allows the system to maintain a local cache of frequently accessed data fragments, improving overall performance by minimizing the need to retrieve data from remote storage locations. The system may also include additional components for processing, retrieving, and managing data fragments, ensuring seamless integration with existing distributed computing architectures. By leveraging the local cache, the system enhances data availability and reduces network overhead, making it particularly useful in environments where low-latency access to data fragments is critical.
5. The system of claim 1 , wherein the service component is configured to determine multiple fragments of the chunk that stores the client object.
A system for managing data storage and retrieval in a distributed computing environment addresses the challenge of efficiently storing and accessing large data objects across multiple storage nodes. The system includes a storage component that divides a client object into multiple chunks, each stored in a separate storage node. A service component is configured to determine multiple fragments of a chunk that stores the client object. These fragments are distributed across different storage nodes to enhance data redundancy and fault tolerance. The service component can reconstruct the original chunk from the fragments, ensuring data integrity even if some storage nodes fail. The system also includes a retrieval component that allows a client to request and retrieve the client object by accessing the fragments stored across the distributed storage nodes. This approach improves data availability and reliability in large-scale distributed storage systems. The system may also include a replication component that ensures fragments are replicated across multiple nodes to further enhance fault tolerance. The service component can dynamically adjust the number of fragments and their distribution based on system load and storage capacity, optimizing performance and resource utilization. This method ensures efficient data storage, retrieval, and fault tolerance in distributed computing environments.
6. The system of claim 1 , wherein the service component determines the fragment based on a determination of an index of the fragment within the chunk, wherein the index is determined as a function of: a byte offset within the chunk of a byte of the client object, the first fixed size, and a count of fragments in the chunk.
This invention relates to a system for managing data fragments within a storage system, specifically addressing the challenge of efficiently determining and accessing specific fragments of data stored in chunks. The system includes a service component that identifies a particular fragment within a chunk based on its position or index. The index is calculated using a mathematical function that incorporates three key parameters: the byte offset of a specific byte within the chunk, a predefined fixed size for the fragments, and the total number of fragments contained in the chunk. This approach allows the system to precisely locate and retrieve the desired fragment without requiring additional metadata or complex lookups, improving storage efficiency and access speed. The service component may also handle other functions such as chunk management, fragment reassembly, or data retrieval, ensuring seamless integration with the overall storage architecture. The system is particularly useful in distributed storage environments where rapid and accurate fragment access is critical for performance and reliability.
7. The system of claim 6 , wherein the computer executable components further comprise an erasure coding component that performs the erasure coding procedure with respect to the local data.
The system relates to data storage and retrieval, specifically addressing the challenge of ensuring data durability and availability in distributed storage environments. The system includes a distributed storage architecture where data is stored across multiple nodes, and the data is encoded using an erasure coding technique to enhance fault tolerance. The erasure coding component processes local data by applying an erasure coding procedure, which generates encoded data fragments that can be distributed across different storage nodes. This encoding allows the original data to be reconstructed even if some fragments are lost or inaccessible, improving data reliability. The system also includes a data retrieval mechanism that reconstructs the original data from the encoded fragments when requested, ensuring that data remains accessible despite potential node failures. The erasure coding component may use various encoding schemes, such as Reed-Solomon or other forward error correction methods, to balance storage efficiency and fault tolerance. The system is designed to operate in environments where data redundancy and recovery are critical, such as cloud storage, distributed databases, or backup systems. By distributing encoded data fragments across multiple nodes, the system minimizes the risk of data loss and ensures high availability.
8. The system of claim 7 , wherein the erasure coding procedure performed with respect to the local data comprises logically dividing a local chunk of the first fixed size into first fragments of the second fixed size, and generating redundancy data representative of erasure coding of data stored in the local chunk, and wherein the redundancy data has a size of a defined number of fragments of the second fixed size.
This invention relates to data storage systems that use erasure coding to protect data against failures. The problem addressed is ensuring data durability and availability in distributed storage environments by efficiently encoding and distributing data fragments across multiple storage nodes. The system performs erasure coding on local data by logically dividing a chunk of data into smaller fragments of a fixed size. The system then generates redundancy data through erasure coding, where the redundancy data is sized to match a defined number of these fragments. This approach allows for data reconstruction even if some storage nodes fail, as the redundancy data can be used to recover lost fragments. The system ensures that the redundancy data is proportionally sized to the fragment size, maintaining consistency and efficiency in data storage and retrieval operations. The method supports scalable and fault-tolerant storage by distributing encoded fragments across multiple storage locations, reducing the risk of data loss while optimizing storage space usage.
9. The system of claim 1 , wherein the computer executable components further comprise a data replication component that exchanges, with the group of remote storage devices, portions of the local data and the remote data in chunks having the first fixed size.
This invention relates to a distributed data storage system designed to improve data redundancy and reliability across multiple storage devices. The system addresses the challenge of efficiently replicating data across a group of remote storage devices while minimizing network overhead and ensuring consistency. The core functionality involves a data replication component that exchanges data between local and remote storage devices in fixed-size chunks. This chunk-based approach allows for granular control over data transfer, reducing the risk of partial or corrupted transfers. The system ensures that portions of local data are synchronized with corresponding portions of remote data, maintaining redundancy and availability. The fixed-size chunks facilitate efficient bandwidth utilization and simplify error detection and recovery. By distributing data in this manner, the system enhances fault tolerance and improves overall data integrity in distributed storage environments. The replication process is automated, reducing manual intervention and operational complexity. This approach is particularly useful in large-scale storage systems where data consistency and reliability are critical. The invention provides a robust solution for maintaining synchronized data copies across geographically dispersed storage devices, ensuring high availability and durability.
10. The system of claim 1 , wherein the computer executable components further comprise a segment component that determines a segment, within the fragment, that comprises all bytes of the client object within the fragment, and wherein the segment is configured to have a variable size less than or equal to the second fixed size.
This invention relates to data processing systems for managing and transmitting fragmented data objects, particularly in client-server environments. The problem addressed is the inefficient handling of fragmented data, where a client object is divided into fragments of fixed sizes, leading to wasted bandwidth or incomplete data transfers. The system includes a segment component that identifies a segment within a fragment containing all bytes of a client object. Unlike traditional fixed-size fragments, this segment has a variable size, allowing precise extraction of the client object without unnecessary data. The segment size is dynamically adjusted to be less than or equal to a predefined fixed size, ensuring compatibility with existing fragmentation protocols while optimizing data transfer efficiency. This approach reduces overhead by avoiding the transmission of partial or irrelevant data, improving performance in networked applications. The system may also include components for generating fragments, encoding data, and reconstructing objects from segments, ensuring seamless integration into data processing workflows. The variable-sized segment enhances flexibility in handling diverse data types and sizes, addressing inefficiencies in conventional fixed-size fragmentation methods.
11. The system of claim 10 , wherein the segment component, via the remote interface, requests a remote device, of the remote devices, to transmit the client object stored to the segment without transmitting other data stored to other portions of the fragment.
This invention relates to a distributed data storage system designed to optimize data retrieval by selectively accessing specific segments of fragmented data across remote devices. The system addresses the inefficiency of conventional methods that require transferring entire data fragments, even when only a portion is needed, leading to unnecessary bandwidth usage and processing overhead. The system includes a segment component that interfaces with multiple remote devices storing fragmented data. Each fragment is divided into segments, with each segment containing a distinct client object. The segment component is configured to request a specific remote device to transmit only the desired client object from its stored segment, without retrieving other data from the same or different fragments. This selective retrieval minimizes data transfer and improves system performance by avoiding the transmission of irrelevant or redundant information. The remote interface facilitates communication between the segment component and the remote devices, ensuring that only the requested client object is retrieved. This targeted approach is particularly useful in environments where data fragments are distributed across multiple devices, and partial data access is frequently required. The system enhances efficiency by reducing network traffic and processing time, making it suitable for applications such as distributed databases, cloud storage, and peer-to-peer networks.
12. The system of claim 11 , wherein the segment component, via the local interface, stores the client object to a local fragment of the local cache and populates an unused portion of the local fragment with placeholder data.
This invention relates to a distributed computing system for managing client objects in a local cache. The system addresses the challenge of efficiently storing and retrieving client objects while optimizing memory usage and reducing latency in distributed environments. The system includes a segment component that interacts with a local interface to manage client objects within a local cache. The segment component stores a client object in a local fragment of the cache and fills any unused portion of that fragment with placeholder data. This ensures that the cache is fully utilized, reducing wasted space and improving performance. The placeholder data may be used to indicate available storage or to facilitate future operations. The system may also include a remote interface for communicating with a remote cache, allowing for synchronization and retrieval of client objects across distributed nodes. The segment component may further validate the client object before storage and handle errors to maintain data integrity. The overall system enhances cache efficiency by dynamically managing storage allocation and ensuring optimal use of available memory resources.
13. A computer-readable storage medium comprising instructions that, in response to execution, cause a device comprising a processor to perform operations, comprising: receiving a request for a client object from a client device; determining that remote storage devices of an elastic cloud storage (ECS) system stores the client object in a chunk of a first fixed size, wherein the ECS system protects the client object via an erasure coding procedure performed on the chunk, and wherein the remote storage devices store remote data to first chunks of the first fixed size; determining a fragment of the chunk that stores the client object, wherein the fragment is representative of a data fragment of a second fixed size, less than the first fixed size, employed by the erasure coding procedure; requesting fragment data, comprising the client object, stored to the fragment; storing the fragment data to a local cache of a local storage device of the ECS system, wherein the local storage device stores local data to second chunks of the first fixed size and comprises the local cache configured to store a portion of the remote data; and transmitting the client object to the client device.
This invention relates to an elastic cloud storage (ECS) system that efficiently retrieves and caches client objects stored using erasure coding. The system addresses the challenge of optimizing data retrieval in distributed storage environments where objects are stored as fragments within larger fixed-size chunks, protected by erasure coding. When a client device requests a client object, the system identifies the chunk containing the object and determines the specific fragment that stores the object. The fragment is smaller than the chunk and corresponds to a data fragment size used in the erasure coding procedure. The system then retrieves the fragment data, which includes the client object, and stores it in a local cache of a local storage device. The local storage device stores data in chunks of the same size as remote storage devices but includes a cache to temporarily hold portions of remote data. Finally, the system transmits the requested client object to the client device. This approach reduces latency and bandwidth usage by caching frequently accessed fragments locally while maintaining the integrity and availability of data through erasure coding.
14. The computer-readable storage medium of claim 13 , wherein the determining the fragment of the chunk comprises determining an index of the fragment within the chunk, wherein the index is determined as a function of: b, representative of a byte offset within the chunk of a byte of the client object; S, representative of the first fixed size; and k, representative of a count of fragments in the chunk.
This invention relates to data storage and retrieval systems, specifically addressing efficient fragmentation and indexing of data chunks to optimize storage and access. The problem solved involves managing large data objects by dividing them into smaller, fixed-size fragments within a chunk, while ensuring fast and accurate retrieval of specific fragments based on byte offsets. The system stores a client object as a chunk divided into multiple fragments of a first fixed size. To determine a specific fragment within the chunk, the system calculates an index based on three parameters: (1) a byte offset (b) within the chunk, representing the position of a byte of the client object; (2) the first fixed size (S) of the fragments; and (3) the total count of fragments (k) in the chunk. The index is derived using these parameters to precisely locate the fragment containing the desired byte. This method ensures efficient fragmentation and indexing, enabling quick access to specific data segments within large objects. The approach is particularly useful in distributed storage systems where data is split across multiple nodes, requiring precise and scalable indexing mechanisms.
15. The computer-readable storage medium of claim 13 , wherein the operations further comprise performing the erasure coding procedure with respect to the local data.
This invention relates to data storage systems, specifically methods for performing erasure coding on locally stored data to enhance data durability and reliability. The problem addressed is the risk of data loss due to hardware failures or other disruptions in distributed storage environments. Erasure coding is a technique that splits data into fragments, encodes them, and distributes the encoded fragments across multiple storage nodes, allowing data reconstruction even if some fragments are lost. The invention involves a computer-readable storage medium containing instructions that, when executed, perform an erasure coding procedure on local data. This process includes encoding the data into multiple fragments, where the number of fragments is greater than the original data segments, and storing these fragments across different storage locations. The encoding ensures that the original data can be reconstructed from a subset of the fragments, even if some are unavailable. The system may also include mechanisms to verify the integrity of the stored fragments and to trigger reconstruction if errors are detected. The invention further includes techniques for managing the distribution of fragments, such as selecting storage nodes based on availability, load balancing, or redundancy requirements. It may also involve monitoring the health of storage nodes and dynamically adjusting the distribution of fragments to maintain reliability. The system can be integrated into distributed storage architectures, cloud storage platforms, or other environments where data redundancy and fault tolerance are critical. The goal is to improve data durability while optimizing storage efficiency and performance.
16. The computer-readable storage medium of claim 14 , wherein the erasure coding procedure performed with respect to the local data comprises logically dividing a local chunk of the first fixed size, S, into k fragments of the second fixed size, and generating redundancy data representative of erasure coding of data stored in the local chunk, wherein the redundancy data has a size of m, where m is a defined number of fragments of the second fixed size.
This invention relates to data storage systems, specifically methods for erasure coding data in distributed storage environments. The problem addressed is efficient and reliable data storage and retrieval in systems where data is distributed across multiple nodes, ensuring data integrity and availability even if some nodes fail. The invention describes a process for erasure coding data stored in a distributed system. A local chunk of data, having a fixed size S, is logically divided into k fragments of a second fixed size. Redundancy data is then generated through erasure coding, representing the data stored in the local chunk. The redundancy data has a size of m, where m is a predefined number of fragments of the second fixed size. This redundancy data allows for data reconstruction in case of node failures or data loss. The system ensures that data can be reconstructed from any subset of the fragments and redundancy data, provided the subset meets the minimum requirements for erasure coding. The method optimizes storage efficiency by using fixed-size fragments and redundancy data, reducing overhead while maintaining data reliability. The approach is particularly useful in distributed storage systems where data is spread across multiple nodes, ensuring high availability and fault tolerance.
17. A method, comprising: receiving, by a device comprising a processor, a request for a client object from a client device; determining, by the device, the client object is stored in a chunk at a group of remote storage devices of an elastic cloud storage (ECS) system stores client objects in chunks of a first fixed size and protect the client objects via an erasure coding procedure performed on the chunks, and wherein the group of remote storage devices store remote data to first chunks of the first fixed size; determining, by the device, a fragment of the chunk that stores the client object, wherein the fragment is representative of a coding fragment of a second fixed size, less than the first fixed size, employed by the erasure coding procedure; and storing, by the device, the fragment data to a local cache of a local storage device of the ECS system, wherein the local storage device stores local data to second chunks of the first fixed size and comprise the local cache configured to store a portion of the remote data.
This invention relates to an elastic cloud storage (ECS) system that stores client objects in fixed-size chunks and protects them using erasure coding. The system includes remote storage devices that store remote data in first chunks of a fixed size and a local storage device that stores local data in second chunks of the same fixed size. The local storage device also includes a local cache configured to store a portion of the remote data. When a client device requests a client object, the system receives the request and determines that the object is stored in a chunk at a group of remote storage devices. The system then identifies a fragment of the chunk that contains the client object. This fragment is representative of a coding fragment of a second fixed size, which is smaller than the first fixed size and is used by the erasure coding procedure. The system stores this fragment data in the local cache of the local storage device. This approach optimizes data retrieval by caching only the necessary fragment of the chunk rather than the entire chunk, reducing storage and bandwidth requirements while maintaining data integrity through erasure coding.
18. The method of claim 17 , further comprising determining, by the device, a segment, within the fragment, that comprises all bytes of the client object within the fragment, wherein the segment is configured to have a variable size less than the second fixed size.
A system and method for optimizing data transmission between a client and a server involves fragmenting a client object into multiple fragments of a first fixed size and transmitting these fragments to a server. The server processes the fragments to identify and extract a client object, which may be smaller than the fragment size. The method further includes determining a segment within the fragment that contains all bytes of the client object, where the segment has a variable size smaller than the second fixed size used for the fragments. This allows for efficient transmission and extraction of the client object without unnecessary data overhead. The system may use a buffer to store the fragments and dynamically adjust the segment size based on the actual size of the client object, improving transmission efficiency and reducing latency. The method ensures that only the relevant portion of the fragment containing the client object is processed, minimizing computational and bandwidth resources. This approach is particularly useful in high-latency or bandwidth-constrained environments where efficient data handling is critical.
19. The method of claim 18 , further comprising requesting, by the device, a remote device, of the remote devices, to transmit the client object stored to the segment without transmitting other data stored to other portions of the fragment.
This invention relates to data transmission systems where a device selectively requests specific data segments from remote devices without transferring unrelated data. The problem addressed is inefficient data retrieval in distributed systems, where devices often request entire data fragments, leading to unnecessary bandwidth usage and processing overhead. The method involves a device identifying a client object stored in a specific segment of a data fragment. The device then requests a remote device to transmit only that client object from the segment, excluding other data stored in different portions of the same fragment. This selective transmission reduces data transfer volume and improves system efficiency. The remote device processes the request, retrieves the specified client object, and transmits it to the requesting device without sending additional data from the fragment. The invention may also include additional steps such as determining the segment location of the client object, validating the request, and ensuring secure transmission. The method is particularly useful in distributed storage systems, peer-to-peer networks, or cloud computing environments where minimizing data transfer is critical. By selectively retrieving only the required data, the system conserves bandwidth, reduces latency, and enhances overall performance.
20. The method of claim 18 , further comprising, storing, by the device, the client object to a local fragment of the local cache and populating an unused portion of the local fragment with placeholder data.
This invention relates to data management in computing systems, specifically optimizing storage and retrieval of client objects in a local cache. The problem addressed is inefficient use of cache memory, where fragments of the cache may be partially filled, leading to wasted storage space and reduced performance. The method involves a device managing a local cache by storing a client object in a local fragment of the cache. To maximize storage efficiency, the device populates any unused portion of the fragment with placeholder data. This ensures that the fragment is fully utilized, preventing fragmentation and improving cache performance. The placeholder data may be dummy values or reusable metadata, allowing the system to quickly identify and replace it when new data is stored. The method builds on a broader system where client objects are retrieved from a remote server, processed, and stored in the local cache. The device may also track the usage of client objects to determine when to evict or update them. By filling unused portions of cache fragments with placeholder data, the system reduces memory waste and maintains a more organized cache structure. This approach is particularly useful in environments where cache memory is limited or frequently accessed, such as mobile devices or distributed computing systems. The technique enhances data retrieval speed and reduces the overhead of managing fragmented storage.
Unknown
February 25, 2020
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.