A plurality of computing devices are communicatively coupled to each other via a network, and each of the plurality of computing devices is operably coupled to one or more of a plurality of storage devices. A plurality of failure resilient address spaces are distributed across the plurality of storage devices such that each of the plurality of failure resilient address spaces spans a plurality of the storage devices. The plurality of computing devices maintains metadata that maps each failure resilient address space to one of the plurality of computing devices. Each of the plurality of computing devices is operable to read from and write to a plurality of memory blocks, while maintaining an extent in metadata that maps the plurality of memory blocks to the failure resilient address space.
Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A method for accessing storage media, the method comprising: distributing a failure resilient address space across a plurality of storage devices, wherein the distribution is performed by a plurality of computing devices; operating on a file in a plurality of memory blocks in the plurality of storage devices, wherein the operation is performed by the plurality of computing devices; and maintaining metadata within the plurality of computing devices to map a plurality of memory blocks to the failure resilient address space, wherein the metadata is divided into a plurality of buckets, and wherein each bucket of the plurality of buckets is associated with a unique group of two or more computing devices selected from the plurality of computing devices wherein the number of buckets in the plurality of buckets is greater than the number of computing devices in the plurality of computing devices.
This invention relates to distributed storage systems designed to enhance fault tolerance and data accessibility. The method involves distributing a failure-resilient address space across multiple storage devices, managed by a network of computing devices. The system ensures data integrity by operating on files stored in memory blocks distributed across these storage devices, with operations performed collaboratively by the computing devices. Metadata is maintained within the computing devices to map memory blocks to the failure-resilient address space, improving data retrieval efficiency. The metadata is organized into multiple buckets, each associated with a unique group of two or more computing devices. The number of buckets exceeds the number of computing devices, ensuring redundancy and fault tolerance. This approach allows the system to handle failures gracefully by leveraging the distributed nature of the address space and metadata, preventing data loss and maintaining access to stored files even if some computing devices or storage devices fail. The method optimizes storage management by balancing load across devices and ensuring consistent metadata updates through coordinated operations among the computing devices.
2. The method of claim 1 , wherein the plurality of storage devices comprises non-volatile memory.
A system and method for data storage and retrieval involves a distributed storage architecture using multiple storage devices to enhance reliability and performance. The storage devices are interconnected to form a network, where data is distributed across the devices to prevent data loss in case of device failure. The system includes a controller that manages data distribution, redundancy, and retrieval operations. The storage devices may include non-volatile memory, such as flash storage, solid-state drives (SSDs), or other persistent storage technologies, ensuring data retention even without continuous power. The controller monitors device health, redistributes data as needed, and ensures efficient access to stored information. The system is designed to optimize storage capacity, reduce latency, and improve fault tolerance, making it suitable for applications requiring high availability and durability, such as enterprise storage, cloud computing, or distributed databases. The use of non-volatile memory enhances performance and reliability by reducing wear on storage media and improving data access speeds. The system dynamically adjusts data distribution based on device performance and availability, ensuring consistent and reliable operation.
3. The method of claim 1 , wherein operating on the file comprises writing data into the plurality of memory blocks.
This invention relates to data storage systems, specifically methods for managing file operations in memory. The problem addressed is efficient and reliable data handling in storage systems, particularly when dealing with file operations that involve multiple memory blocks. The method involves operating on a file by writing data into a plurality of memory blocks. This operation is part of a broader process that includes receiving a file operation request, determining a set of memory blocks associated with the file, and performing the requested operation on those blocks. The method ensures that data is written accurately and efficiently across the designated memory blocks, which may be distributed or organized in a specific manner within the storage system. The approach optimizes storage performance by managing how data is distributed and written to memory blocks, reducing fragmentation and improving access times. The system may also include mechanisms to verify the integrity of the written data and handle errors during the write process. This method is particularly useful in systems where files are stored across multiple memory blocks, such as in distributed storage or solid-state drives, where efficient block-level operations are critical for performance.
4. The method of claim 3 , wherein writing data comprises compressing a buffer of data to be written to a first memory block of the plurality of memory blocks to provide space for a journal associated with the data written into the plurality of memory blocks.
A method for managing data storage in a memory system addresses the challenge of efficiently writing and recovering data while maintaining data integrity. The method involves compressing a buffer of data before writing it to a first memory block, creating space for a journal that records changes made to the data. The journal is used to track modifications, enabling recovery in case of system failures or disruptions. By compressing the data, the method ensures that sufficient space remains in the memory block for the journal, preventing data loss and corruption. This approach improves storage efficiency and reliability, particularly in systems where data integrity and recovery are critical. The method may also include writing the compressed data and the associated journal to the memory block, ensuring that the journal accurately reflects the state of the data. Additionally, the method may involve managing multiple memory blocks, where each block may store compressed data and its corresponding journal. This technique is particularly useful in systems requiring robust data recovery mechanisms, such as solid-state drives or other non-volatile storage devices. The compression step optimizes storage space while maintaining the ability to reconstruct data from the journal in the event of a failure.
5. The method of claim 3 , wherein writing data comprises padding a buffer of data to be written to a memory block of the plurality of memory blocks with data previously written into a portion of the memory block.
This invention relates to data storage systems, specifically methods for writing data to memory blocks in a way that improves efficiency and reliability. The problem addressed is the inefficiency and potential data loss that occurs when writing partial data to a memory block, particularly in systems where memory blocks must be erased before new data can be written. The invention provides a method to mitigate these issues by padding incomplete data writes with previously written data from the same memory block. The method involves writing data to a memory block by first determining the amount of data to be written and comparing it to the capacity of the memory block. If the data to be written does not fill the entire memory block, the remaining space is filled (padded) with data that was previously written to a portion of the same memory block. This ensures that the memory block is fully utilized without leaving unused space, which can lead to inefficiencies or data corruption. The padding data is selected from a portion of the memory block that is not being overwritten, ensuring that valid data is preserved. This technique is particularly useful in flash memory systems, where partial writes can degrade performance and reliability over time. By maintaining full block writes, the method reduces wear on the memory and improves overall system longevity. The invention also includes mechanisms to track which portions of the memory block contain valid data to ensure that only appropriate data is used for padding.
6. The method of claim 3 , wherein in coordination with writing data to a memory block of the plurality of memory blocks, the method comprises updating an extent such that the extent comprises an identification of a plurality of blocks associated with protecting the data being written to the memory block.
This invention relates to data storage systems, specifically methods for managing data protection in memory devices. The problem addressed is ensuring data integrity and reliability when writing data to memory blocks, particularly in systems where data protection mechanisms must track multiple associated blocks. The method involves writing data to a memory block within a plurality of memory blocks. Concurrently, an extent is updated to include an identification of multiple blocks that are associated with protecting the data being written. This ensures that the data is safeguarded by referencing all relevant blocks involved in its protection. The extent serves as a metadata structure that links the written data to its protective blocks, enabling efficient data recovery and consistency checks. The method may also include additional steps such as verifying the integrity of the protective blocks or updating metadata to reflect the association between the data and its protective blocks. This approach enhances data reliability by maintaining a clear and up-to-date record of all blocks involved in data protection, reducing the risk of data loss or corruption.
7. The method of claim 1 , wherein operating on the file comprises reading data from the plurality of memory blocks.
A system and method for file management in a storage device addresses the challenge of efficiently accessing and processing data stored in memory blocks. The invention involves a technique for operating on a file stored in a non-volatile memory, where the file is divided into multiple memory blocks. The method includes reading data from these memory blocks to perform operations such as data retrieval, modification, or deletion. The system ensures that the file operations are executed in a manner that optimizes performance and minimizes resource usage. The memory blocks may be organized in a specific structure, such as a logical block addressing scheme, to facilitate efficient data access. The method may also include error detection and correction mechanisms to ensure data integrity during read operations. By reading data from the memory blocks, the system enables reliable and efficient file management, particularly in storage devices where data is distributed across multiple physical or logical memory units. This approach improves the overall performance and reliability of file operations in storage systems.
8. The method of claim 7 , wherein the method comprises checking data read from a particular memory block of the plurality of memory blocks for errors using a distributed erasure code based on blocks identified in an extent.
A method for error checking in memory storage systems addresses the challenge of detecting and correcting data corruption in large-scale distributed storage environments. The method involves reading data from a specific memory block within a plurality of memory blocks and verifying its integrity using a distributed erasure code. The erasure code is applied based on blocks identified within a predefined extent, which is a logical grouping of memory blocks. This approach ensures that data errors can be detected and corrected efficiently, even in systems where data is distributed across multiple storage nodes. The method leverages the redundancy provided by the erasure code to reconstruct missing or corrupted data, enhancing reliability in storage systems. The use of an extent-based approach allows for scalable and flexible error checking, adapting to different storage configurations and workloads. This technique is particularly useful in distributed storage systems where data is spread across multiple physical or logical storage units, ensuring data integrity and availability.
9. The method of claim 8 , wherein the method comprises performing a degraded data read when the particular memory block of the plurality of memory blocks is found to be in error.
10. The method of claim 9 , wherein performing a degraded data read comprises regenerating the particular memory block from one or more memory blocks other than the particular memory block of the plurality of memory blocks.
This invention relates to data storage systems, specifically methods for handling degraded data reads in memory systems where data is distributed across multiple memory blocks. The problem addressed is ensuring data integrity and availability when a memory block becomes inaccessible or corrupted, preventing direct data retrieval. The method involves regenerating data from a degraded or inaccessible memory block using one or more other memory blocks in the system. The system stores data redundantly across multiple memory blocks, allowing reconstruction of the original data from the remaining intact blocks. When a read operation fails due to a degraded block, the system identifies the affected block and retrieves data from the other blocks. Using error correction or redundancy techniques, such as erasure coding or parity checks, the system reconstructs the missing or corrupted data. This ensures that the requested data can still be accessed even if one or more blocks are faulty. The approach improves reliability in storage systems by leveraging redundancy to mitigate failures without requiring immediate repair or replacement of the degraded block.
11. A system comprising: a plurality of storage devices, wherein a failure resilient address space is distributed across the plurality of storage devices such that the failure resilient address space spans more than one storage device of the plurality of storage devices; and a plurality of computing devices communicatively coupled to each other and to the plurality of storage devices via a network, wherein: each of the plurality of computing devices is operable to perform a file operation over a plurality of memory blocks, the plurality of computing devices is operable to maintain metadata that maps the plurality of memory blocks to the failure resilient address space, the metadata is divided into a plurality of buckets, each bucket of the plurality of buckets is associated with a unique group of two or more computing devices selected from the plurality of computing devices, wherein the number of buckets in the plurality of buckets is greater than the number of computing devices in the plurality of computing devices.
This invention relates to distributed storage systems designed to enhance failure resilience and data accessibility. The system includes multiple storage devices and computing devices interconnected via a network. A failure-resilient address space is distributed across the storage devices, ensuring data spans multiple devices to mitigate single-point failures. The computing devices collectively manage metadata that maps memory blocks to this address space, enabling efficient file operations. The metadata is organized into multiple buckets, each assigned to a unique group of two or more computing devices. This grouping ensures redundancy and load balancing, as no single computing device is solely responsible for a bucket. The number of buckets exceeds the number of computing devices, further distributing the metadata management workload and improving fault tolerance. The system allows computing devices to perform file operations across memory blocks while maintaining consistent metadata mappings, ensuring data integrity and availability even if some devices fail. This approach enhances scalability and reliability in distributed storage environments.
12. The system of claim 11 , wherein the plurality of storage devices comprises non-volatile memory.
The invention relates to a data storage system designed to improve reliability and performance in computing environments. The system includes multiple storage devices configured to store data redundantly, ensuring data integrity even if some storage devices fail. The storage devices are interconnected to form a distributed storage network, allowing parallel data access and load balancing. The system employs error correction techniques to detect and correct data corruption, enhancing reliability. Additionally, the system dynamically adjusts storage allocation based on usage patterns to optimize performance. The storage devices in the system are non-volatile, meaning they retain data even without power, which is critical for maintaining data integrity during power outages or system failures. This design is particularly useful in high-availability computing environments where data loss or downtime is unacceptable. The system's distributed architecture and error correction mechanisms ensure continuous operation and data protection.
13. The system of claim 11 , wherein the file operation comprises writing data into the plurality of memory blocks.
A system for managing data storage in a memory device addresses the challenge of efficiently handling file operations, particularly when writing data into multiple memory blocks. The system includes a memory controller that processes file operations, such as writing data, and a memory device with a plurality of memory blocks. The memory controller is configured to execute file operations by distributing data across the memory blocks, ensuring efficient storage and retrieval. The system may also include a host device that communicates with the memory controller to initiate file operations. The memory controller optimizes the writing process by managing the allocation and deallocation of memory blocks, reducing fragmentation and improving performance. The system may further include error detection and correction mechanisms to ensure data integrity during write operations. This approach enhances reliability and speed in data storage systems, particularly in environments where frequent write operations are required. The system is designed to work with various types of memory devices, including flash memory, solid-state drives, and other non-volatile storage solutions. The memory controller may also support additional features like wear leveling to extend the lifespan of the memory blocks. By efficiently managing write operations across multiple memory blocks, the system ensures optimal performance and durability in data storage applications.
14. The system of claim 13 , wherein a buffer of data to be written to a first memory block of the plurality of memory blocks is compressed to provide space for a journal associated with the data written into the plurality of memory blocks.
The system relates to data storage and management in memory devices, particularly addressing the challenge of efficiently utilizing storage space while maintaining data integrity and recovery capabilities. The system includes a plurality of memory blocks for storing data, where each block can be written to and erased independently. A buffer of data intended for writing to a first memory block is compressed to free up additional space. This compression allows for the creation and storage of a journal associated with the data being written to the memory blocks. The journal records metadata or log entries that facilitate data recovery, error correction, or system consistency checks. By compressing the data, the system ensures that sufficient space remains for the journal without requiring additional physical storage, optimizing storage efficiency while preserving the ability to track and manage data operations. The system may also include mechanisms for decompressing the data when read back, ensuring seamless access to the original data while leveraging the benefits of compression. This approach is particularly useful in environments where storage space is limited, such as embedded systems or solid-state drives, where maintaining both performance and reliability is critical.
15. The system of claim 13 , wherein a buffer of data to be written to a memory block of the plurality of memory blocks is padded with data previously written into a portion of the memory block.
The system relates to data storage and management in memory devices, particularly addressing the challenge of efficiently handling data writes to memory blocks while maintaining data integrity and performance. The system includes a memory controller configured to manage a plurality of memory blocks, where each block stores data in a structured format. When writing new data to a memory block, the system pads the buffer of data to be written with previously written data from a portion of the same memory block. This padding ensures that the new data aligns with the existing data structure within the block, preventing misalignment or corruption. The padding process involves retrieving the previously written data from the specified portion of the memory block, appending it to the new data buffer, and then writing the combined data to the memory block. This approach is particularly useful in systems where data must be written in fixed-size blocks or where alignment with existing data is critical for proper operation. The system may also include error detection and correction mechanisms to verify the integrity of the written data. By incorporating previously written data into the padding, the system ensures that the memory block remains consistent and that the new data is properly integrated with the existing data structure. This method improves data reliability and reduces the risk of errors during read or write operations.
16. The system of claim 13 , wherein in coordination with writing data to a memory block of the plurality of memory blocks, an extent is updated, and wherein the updated extent comprises an identification of a plurality of blocks associated with protecting the data being written to the memory block.
This invention relates to data storage systems, specifically methods for managing data integrity and protection in memory systems. The problem addressed is ensuring reliable data storage by tracking and protecting data across multiple memory blocks. The system includes a plurality of memory blocks and a mechanism for writing data to these blocks. When data is written to a memory block, an extent is updated to include an identification of multiple blocks associated with protecting the written data. This extent serves as a reference that links the data to the protective blocks, ensuring that the data remains accessible and intact. The protective blocks may include redundancy or error-correcting information to safeguard against data corruption or loss. The system dynamically updates the extent as data is written, maintaining an accurate record of the protective associations. This approach enhances data reliability by ensuring that each data write operation is accompanied by an updated reference to its protective blocks, reducing the risk of data loss or corruption. The invention is particularly useful in storage systems where data integrity is critical, such as in enterprise storage solutions or distributed storage environments.
17. The system of claim 11 , wherein the file operation comprises reading data from the plurality of memory blocks.
A system for managing data storage and retrieval in a memory device addresses the challenge of efficiently accessing data stored in multiple memory blocks. The system includes a memory controller configured to perform file operations on a plurality of memory blocks, where each memory block is part of a larger storage medium. The memory controller is designed to execute specific file operations, such as reading data from or writing data to these memory blocks, while optimizing performance and reliability. The system may also include error detection and correction mechanisms to ensure data integrity during these operations. In this particular configuration, the file operation involves reading data from the plurality of memory blocks, allowing the system to retrieve stored information efficiently. The system may further incorporate techniques to manage wear leveling, bad block handling, and data encryption to enhance overall storage performance and security. By coordinating these operations, the system ensures reliable and fast access to data across the memory blocks, addressing the need for efficient data management in storage devices.
18. The system of claim 17 , wherein a data read from a particular memory block of the plurality of memory blocks is checked for errors using a distributed erasure code based on blocks identified in an extent.
The invention relates to error detection and correction in memory systems, particularly for distributed storage environments. The problem addressed is ensuring data integrity in large-scale storage systems where data is distributed across multiple memory blocks, making traditional error-checking methods inefficient or unreliable. The system uses a distributed erasure code to verify data integrity by checking data read from a specific memory block against encoded data stored in other blocks within a defined extent. The extent refers to a logical grouping of memory blocks that collectively store encoded fragments of the same data. When reading data from a particular block, the system reconstructs the original data using the distributed erasure code and compares it to the read data to detect errors. This approach improves fault tolerance by leveraging redundancy across multiple blocks, allowing for error detection even if some blocks are unavailable or corrupted. The system dynamically adjusts the extent based on storage conditions, optimizing error detection efficiency. This method is particularly useful in distributed storage systems where data is spread across multiple nodes or devices, ensuring reliability without centralized error-checking overhead. The invention enhances data integrity in large-scale storage environments by combining distributed erasure coding with dynamic extent management.
19. The system of claim 18 , wherein a degraded data read is performed when the particular memory block of the plurality of memory blocks is found to be in error.
A system for managing memory operations in a storage device addresses the problem of data integrity in non-volatile memory, particularly when errors are detected in specific memory blocks. The system includes a controller configured to perform a degraded data read operation when a particular memory block is identified as containing errors. This degraded read operation allows the system to recover data from the faulty block by using error correction techniques or alternative data sources, such as redundant copies or parity information, to reconstruct the original data. The system may also include multiple memory blocks, each with associated error detection and correction mechanisms. When an error is detected in a block during a read operation, the controller initiates the degraded read process to ensure data retrieval without complete failure. This approach improves reliability by preventing data loss when individual memory blocks fail or develop errors, ensuring continuous access to stored information even under adverse conditions. The system may further include additional features, such as error logging, adaptive error correction, and dynamic remapping of faulty blocks to healthy ones, to enhance overall performance and durability.
20. The system of claim 19 , wherein the degraded data read comprises regenerating the particular memory block from one or more memory blocks other than the particular memory block of the plurality of memory blocks.
The invention relates to a data storage system that regenerates degraded or corrupted data in a memory block using other memory blocks. The system addresses the problem of data loss or corruption in storage devices, particularly in scenarios where a memory block becomes unreliable or fails. The system includes a plurality of memory blocks, where data is distributed across these blocks in a way that allows for regeneration of degraded data from other blocks. When a particular memory block is identified as degraded, the system regenerates the data stored in that block by accessing and processing data from one or more other memory blocks. This regeneration process ensures data integrity and availability, even if individual memory blocks fail. The system may use error correction techniques or distributed storage algorithms to reconstruct the degraded data accurately. The approach is particularly useful in large-scale storage systems where redundancy and fault tolerance are critical. The invention improves reliability by leveraging redundancy across multiple memory blocks, allowing the system to recover from partial failures without complete data loss.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
September 4, 2018
March 1, 2022
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.