Metafiles
NTFS contains several files that define and organize the file system. In all respects, most of these files are structured like any other user file ($Volume being the most peculiar), but are not of direct interest to file system clients. These metafiles define files, back up critical file system data, buffer file system changes, manage free space allocation, satisfy BIOS expectations, track bad allocation units, and store security and disk space usage information. All content is in an unnamed data stream, unless otherwise indicated. These metafiles are treated specially by Windows, handled directly by theNTFS.SYS driver and are difficult to directly view: special purpose-built tools are needed. As of Windows 7, the NTFS driver completely prohibits user access, resulting in a BSoD whenever an attempt to execute a metadata file is made. One such tool is the nfi.exe ("NTFS File Sector Information Utility") that is freely distributed as part of the Microsoft "OEM Support Tools". For example, to obtain information on the "$MFT"-Master File Table Segment the following command is used: nfi.exe c:\$MFT Another way to bypass the restriction is to use \\.\X:\ (where X:\ resembles any drive/partition). Here, 3 new folders will appear: $EXTEND, ELETED/code> (a pseudo-folder that 7-Zip uses to attach files deleted from the file system to view), and YSTEM/code> (another pseudo-folder that contains all the NTFS metadata files). This trick can be used from removable devices (USB
Universal Serial Bus (USB) is an industry standard that establishes specifications for cables, connectors and protocols for connection, communication and power supply (interfacing) between computers, peripherals and other computers. A broad ...
flash drives, external hard drives, SD Card
Secure Digital, officially abbreviated as SD, is a proprietary non-volatile flash memory card format developed by the SD Association (SDA) for use in portable devices.
The standard was introduced in August 1999 by joint efforts between SanDis ...
s, etc.) inside Windows, but doing this on the active partition requires offline access (namely WinRE).
Attribute lists, attributes, and streams
For each file (or directory) described in the MFT record, there is a linear repository of stream descriptors (also named ''attributes''), packed together in one or more MFT records (containing the so-called ''attributes list''), with extra padding to fill the fixed 1 KB size of every MFT record, and that fully describes the effective streams associated with that file.
Each attribute has an attribute type (a fixed-size integer mapping to an attribute definition in file ), an optional attribute name (for example, used as the name for an alternate data stream), and a value, represented in a sequence of bytes. For NTFS, the standard data of files, the alternate data streams, or the index data for directories are stored as attributes.
According to , some attributes can be either resident or non-resident. The attribute, which contains file data, is such an example. When the attribute is resident (which is represented by a flag), its value is stored directly in the MFT record. Otherwise, clusters are allocated for the data, and the cluster location information is stored as data runs in the attribute.
* For each file in the MFT, the attributes identified by ''attribute type, attribute name'' must be unique. Additionally, NTFS has some ordering constraints for these attributes.
* There is a predefined null attribute type, used to indicate the end of the list of attributes in one MFT record. It must be present as the last attribute in the record (all other storage space available after it will be ignored and just consists of padding bytes to match the record size in the MFT).
* Some attribute types are required and must be present in each MFT record, except unused records that are just indicated by null attribute types.
** This is the case for the attribute that is stored as a fixed-size record and contains the timestamp
A timestamp is a sequence of characters or encoded information identifying when a certain event occurred, usually giving date and time of day, sometimes accurate to a small fraction of a second. Timestamps do not have to be based on some absolut ...
s and other basic single-bit attributes (compatible with those managed by FAT
In nutrition, biology, and chemistry, fat usually means any ester of fatty acids, or a mixture of such compounds, most commonly those that occur in living beings or in food.
The term often refers specifically to triglycerides (triple est ...
in DOS or Windows 9x
Windows 9x is a generic term referring to a series of Microsoft Windows computer operating systems produced from 1995 to 2000, which were based on the Windows 95 kernel and its underlying foundation of MS-DOS, both of which were updated in su ...
).
* Some attribute types cannot have a name and must remain anonymous.
** This is the case for the standard attributes, or for the preferred NTFS "filename" attribute type, or the "short filename" attribute type, when it is also present (for compatibility with DOS-like applications, see below). It is also possible for a file to contain only a short filename, in which case it will be the preferred one, as listed in the Windows Explorer.
** The filename attributes stored in the attribute list do not make the file immediately accessible through the hierarchical file system. In fact, all the filenames must be indexed separately in at least one other directory on the same volume. There it must have its own MFT record and its own security descriptor Security descriptors are data structures of security information for ''securable'' Windows objects, that is objects that can be identified by a unique name. Security descriptors can be associated with any named objects, including files, folders, ...
s and attributes that reference the MFT record number for this file. This allows the same file or directory to be "hardlinked" several times from several containers on the same volume, possibly with distinct filenames.
* The default data stream of a regular file is a stream of type but with an anonymous name, and the ADSs are similar but must be named.
* On the other hand, the default data stream of directories has a distinct type, but are not anonymous: they have an attribute name ("" in NTFS 3+) that reflects its indexing format.
All attributes of a given file may be displayed by using the nfi.exe ("NTFS File Sector Information Utility") that is freely distributed as part of the Microsoft "OEM Support Tools".
Windows system calls may handle alternate data streams. Depending on the operating system, utility and remote file system, a file transfer might silently strip data streams. A safe way of copying or moving files is to use the BackupRead and BackupWrite system calls, which allow programs to enumerate streams, to verify whether each stream should be written to the destination volume and to knowingly skip unwanted streams.
Resident vs. non-resident attributes
To optimize the storage and reduce the I/O overhead for the very common case of attributes with very small associated value, NTFS prefers to place the value within the attribute itself (if the size of the attribute does not then exceed the maximum size of an MFT record), instead of using the MFT record space to list clusters containing the data; in that case, the attribute will not store the data directly but will just store an allocation map (in the form of ''data runs'') pointing to the actual data stored elsewhere on the volume. When the value can be accessed directly from within the attribute, it is called "resident data" (by computer forensics workers). The amount of data that fits is highly dependent on the file's characteristics, but 700 to 800 bytes is common in single-stream files with non-lengthy filenames and no ACLs.
* Some attributes (such as the preferred filename, the basic file attributes) cannot be made non-resident. For non-resident attributes, their allocation map must fit within MFT records.
* Encrypted-by-NTFS, sparse data streams, or compressed data streams cannot be made resident.
* The format of the allocation map for non-resident attributes depends on its capability of supporting sparse data storage. In the current implementation of NTFS, once a non-resident data stream has been marked and converted as sparse, it cannot be changed back to non-sparse data, so it cannot become resident again, unless this data is fully truncated, discarding the sparse allocation map completely.
* When a non-resident attribute is so fragmented, that its effective allocation map cannot fit entirely within one MFT record, NTFS stores the attribute in multiple records. The first one among them is called the base record, while the others are called extension records. NTFS creates a special attribute to store information mapping different parts of the long attribute to the MFT records, which means the allocation map may be split into multiple records. The itself can also be non-resident, but its own allocation map must fit within one MFT record.
* When there are too many attributes for a file (including ADS's, extended attributes, or security descriptor Security descriptors are data structures of security information for ''securable'' Windows objects, that is objects that can be identified by a unique name. Security descriptors can be associated with any named objects, including files, folders, ...
s), so that they cannot fit all within the MFT record, extension records may also be used to store the other attributes, using the same format as the one used in the base MFT record, but without the space constraints of one MFT record.
The allocation map is stored in a form of ''data runs'' with compressed encoding. Each data run represents a contiguous group of clusters that store the attribute value. For files on a multi-GB volume, each entry can be encoded as 5 to 7 bytes, which means a MFT record can store about 100 such data runs. However, as the also has a size limit, it is dangerous to have more than 1 million fragments of a single file on an NTFS volume, which also implies that it is in general not a good idea to use NTFS compression on a file larger than .
The NTFS file system driver will sometimes attempt to relocate the data of some of the attributes that can be made non-resident into the clusters, and will also attempt to relocate the data stored in clusters back to the attribute inside the MFT record, based on priority and preferred ordering rules, and size constraints.
Since resident files do not directly occupy clusters ("allocation units"), it is possible for an NTFS volume to contain more files on a volume than there are clusters. For example, a partition NTFS formats with 19,543,064 clusters of . Subtracting system files (a log file, a 2,442,888-byte Bitmap file, and about 25 clusters of fixed overhead) leaves 19,526,158 clusters free for files and indices. Since there are four MFT records per cluster, this volume theoretically could hold almost 4 × 19,526,158 = 78,104,632 resident files.
Opportunistic locks
Opportunistic file locks (oplocks) allow clients to alter their buffering strategy for a given file or stream in order to increase performance and reduce network use. Oplocks apply to the given open stream of a file and do not affect oplocks on a different stream.
Oplocks can be used to transparently access files in the background. A network client may avoid writing information into a file on a remote server if no other process is accessing the data, or it may buffer read-ahead data if no other process is writing data.
Windows supports four different types of oplocks:
* Level 2 (or shared) oplock: multiple readers, no writers (i.e. read caching).
* Level 1 (or exclusive) oplock: exclusive access with arbitrary buffering (i.e. read and write caching).
* Batch oplock (also exclusive): a stream is opened on the server, but closed on the client machine (i.e. read, write and handle caching).
* Filter oplock (also exclusive): applications and file system filters can "back out" when others try to access the same stream (i.e. read and write caching) (since Windows 2000)
Opportunistic locks have been enhanced in Windows 7 and Windows Server 2008 R2 with per-client oplock keys.
Time
Windows NT and its descendants keep internal timestamps as UTC and make the appropriate conversions for display purposes; all NTFS timestamps are in UTC.
For historical reasons, the versions of Windows that do not support NTFS all keep time internally as local zone time, and therefore so do all file systems – other than NTFS – that are supported by current versions of Windows. This means that when files are copied or moved between NTFS and non-NTFS partitions, the OS needs to convert timestamps on the fly. But if some files are moved when daylight saving time
Daylight saving time (DST), also referred to as daylight savings time or simply daylight time (United States, Canada, and Australia), and summer time (United Kingdom, European Union, and others), is the practice of advancing clocks (typicall ...
(DST) is in effect, and other files are moved when standard time is in effect, there can be some ambiguities in the conversions. As a result, especially shortly after one of the days on which local zone time changes, users may observe that some files have timestamps that are incorrect by one hour. Due to the differences in implementation of DST in different jurisdictions, this can result in a potential timestamp error of up to 4 hours in any given 12 months.Beating the Daylight Saving Time bug and getting correct file modification times
" ''The Code Project''
See also
* Comparison of file systems
The following tables compare general and technical information for a number of file systems.
General information
Limits
Metadata
Features File capabilities
Block capabilities
Note that in addition to the below table, blo ...
* NTFSDOS
The company Winternals (later purchased by Microsoft) used to provide three kinds of programs for DOS that could handle NTFS formatted drives.
The NTFSDOS programs
NTFSDOS
The first program was NTFSDOS - a freeware utility for DOS (''NTFSD ...
* ntfsresize
ntfsresize is a free Unix utility that non-destructively resizes the NTFS filesystem used by Windows NT 4.0, 2000, XP, 2003, Vista, 7, 8, 10, and 11 typically on a hard-disk partition. All NTFS versions used by 32-bit and 64-bit Windo ...
* WinFS
WinFS (short for Windows Future Storage) was the code name for a canceled data storage and management system project based on relational databases, developed by Microsoft and first demonstrated in 2003 as an advanced storage subsystem for the Micro ...
(a canceled Microsoft filesystem)
* ReFS, a newer Microsoft filesystem
Notes
References
Further reading
*
*
*
*
{{File systems
Compression file systems
Windows disk file systems
1993 software