A grid file system is a computer file system whose goal is improved reliability and availability by taking advantage of many smaller file storage areas.

Components

File systems contain up to three components: * File table (FAT table, MFT, etc.) * File data *

Metadata Metadata is "data that provides information about other data", but not the content of the data, such as the text of a message or the image itself. There are many distinct types of metadata, including: * Descriptive metadata – the descriptive ...

(user permissions, etc.) A grid file system would have similar needs: *

File table In Unix and Unix-like computer operating systems, a file descriptor (FD, less frequently fildes) is a process-unique identifier ( handle) for a file or other input/output resource, such as a pipe or network socket. File descriptors typically hav ...

(or search index) * File data * Metadata

Comparisons

Because file systems are designed to appear as a single disk for a single computer to manage (entirely), many new challenges arise in a grid scenario whereby any single disk within the grid should be capable of handling requests for any data contained in the grid.

Features

Most file storage utilizes layers of redundancy to achieve a high level of data protection (inability to lose data). Current means of redundancy include replication and parity checks. Such redundancy can be implemented via a

RAID Raid, RAID or Raids may refer to: Attack * Raid (military), a sudden attack behind the enemy's lines without the intention of holding ground * Corporate raid, a type of hostile takeover in business * Panty raid, a prankish raid by male college ...

array (whereby multiple physical disks appear to a local computer as a single disk, which may include data replication, and/or disk partitioning). Similarly, a grid file system would consist of some level of redundancy (either at the logical file level, or at the block level, possibly including some sort of parity check) across the various disks present in the "grid".

Framework

First and foremost, a file table mechanism is necessary. Additionally, the file table must include a mechanism for locating the (target/destination) file within the grid. Secondly, a mechanism for working with file data must exist. This mechanism is responsible for making file data available to requests.

Implementation

With BitTorrent technology, a parallel can be drawn to a grid file system, in that a torrent tracker (and search engine) would be the "file table", and the torrent applications (transmitting the files) would be the "file data" component. An RSS feed like mechanism could be utilized by file table nodes to indicate when new files are added to the table, to instigate replication and other similar components. A file system may incorporate similar technology (distributed replication, distributed data request/fulfillment). If both such systems (file table, and file data) were capable of being addressed as a single entity (i.e. using virtual nodes in a cluster), then growth into such a system could be easily controlled simply by deciding which uses the grid member would be responsible (file table and file lookups, and/or file data).

Availability

Assuming there exists some method of managing data replication (assigning quotas, etc.) autonomously within the grid, data could be configured for high availability, regardless of loss or outage.

Challenges

The largest problem currently revolves around distributing data updates. Torrents support minimal hierarchy (currently implemented either as

metadata Metadata is "data that provides information about other data", but not the content of the data, such as the text of a message or the image itself. There are many distinct types of metadata, including: * Descriptive metadata – the descriptive ...

in the torrent tracker, or strictly as UI and basic categorization). Updating multiple nodes concurrently (assuming atomic transactions are required) presents latency during updates and additions, usually to the point of not being feasible. Additionally, a grid (network based) file system breaks traditional TCP/IP paradigms in that a file system (generally low level, ring 0 type of operations) require complicated TCP/IP implementations, introducing layers of abstraction and complication to the process of creating such a grid file system.

Examples

Examples of high-available data include: * Network load balancing /

CARP Carp are various species of oily freshwater fish from the family Cyprinidae, a very large group of fish native to Europe and Asia. While carp is consumed in many parts of the world, they are generally considered an invasive species in parts of ...

– splitting incoming requests to multiple computers, usually configured identically or as one whole. * Shared storage clustering / SANs – a single disk (one or more physical disks acting as a single logical disk) is presented to multiple computers which split incoming requests. This is usually used when more computing power is required than disk access. * Data replication / mirroring – multiple computers may attempt to synchronize data (usually point-in-time or snapshot based). Used more often for either Reporting (based on last snapshot) or backup purposes. * Data partitioning – splitting data among multiple computers. In databases, data is often partitioned based on tables (certain tables exist on certain computers, or a table is split among multiple computers at certain "break points")... general files tend to be partitioned either by category (category based folders), or location (geographically separated).

References

{{reflist Computer file systems