|
Zstd
Zstandard, commonly known by the name of its reference implementation zstd, is a lossless data compression algorithm developed by Yann Collet at Facebook. ''Zstd'' is the reference implementation in C. Version 1 of this implementation was released as open-source software on 31 August 2016. Features Zstandard was designed to give a compression ratio comparable to that of the DEFLATE algorithm (developed in 1991 and used in the original ZIP and gzip programs), but faster, especially for decompression. It is tunable with compression levels ranging from negative 7 (fastest) to 22 (slowest in compression speed, but best compression ratio). The zstd package includes parallel (multi-threaded) implementations of both compression and decompression. Starting from version 1.3.2 (October 2017), zstd optionally implements very long range search and deduplication (, 128 MiB window) similar to rzip or lrzip. Compression speed can vary by a factor of 20 or more between the fastes ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Asymmetric Numeral Systems
Asymmetric numeral systems (ANS)J. Duda, K. Tahboub, N. J. Gadil, E. J. Delp''The use of asymmetric numeral systems as an accurate replacement for Huffman coding'' Picture Coding Symposium, 2015.J. Duda''Asymmetric numeral systems: entropy coding combining speed of Huffman coding with compression rate of arithmetic coding'' arXiv:1311.2540, 2013. is a family of entropy encoding methods introduced by Jarosław (Jarek) Duda from Jagiellonian University, used in data compression since 2014 due to improved performance compared to previous methods. ANS combines the compression ratio of arithmetic coding (which uses a nearly accurate probability distribution), with a processing cost similar to that of Huffman coding. In the tabled ANS (tANS) variant, this is achieved by constructing a finite-state machine to operate on a large alphabet without using multiplication. Among others, ANS is used in the Facebook Zstandard compressor [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
OpenZFS
OpenZFS is a free and open-source storage platform licensed under CDDL that encompasses the functionality of traditional filesystems and logical volume manager. It includes protection against data corruption, support for high storage capacities, efficient data compression, snapshots and copy-on-write clones, continuous integrity checking and automatic repair, encryption, remote replication with ZFS send and receive, and RAID-Z. The OpenZFS project brings together developers from the illumos, Linux, FreeBSD and macOS platforms, and a wide range of companies via the annual OpenZFS Developer Summit. Founding members of OpenZFS include Matt Ahrens, one of the main architects of ZFS. The OpenZFS project is an open source derivative of the Oracle ZFS project. , OpenZFS (on some platforms such as FreeBSD) is gradually being pivoted to be based upon ZFS on Linux, which has developed faster than other variants of OpenZFS and contains new features not yet ported to those other versions. ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Squashfs
Squashfs is a compressed read-only file system for Linux. Squashfs compresses files, inodes and directories, and supports block sizes from 4 KiB up to 1 MiB for greater compression. Several compression algorithms are supported. Squashfs is also the name of free software, licensed under the GPL, for accessing Squashfs filesystems. Squashfs is intended for general read-only file-system use and in constrained block-device memory systems (e.g. embedded systems) where low overhead is needed. Uses Squashfs is used by the Live CD versions of Arch Linux, Debian, Fedora, Gentoo Linux, Linux Mint, openSUSE, Salix, Ubuntu, NixOS, Clonezilla, Kali Linux and on embedded distributions such as the OpenWrt and DD-WRT router firmware. It is also used in Chromecast and for the system partitions of some Android releases (Android Nougat). It is often combined with a union mount filesystem, such as UnionFS, OverlayFS, or aufs, to provide a read-write environment for live Linux distributions ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
FreeBSD
FreeBSD is a free and open-source Unix-like operating system descended from the Berkeley Software Distribution (BSD), which was based on Research Unix. The first version of FreeBSD was released in 1993. In 2005, FreeBSD was the most popular open-source BSD operating system, accounting for more than three-quarters of all installed and permissively licensed BSD systems. FreeBSD has similarities with Linux, with two major differences in scope and licensing: FreeBSD maintains a complete system, i.e. the project delivers a kernel, device drivers, userland utilities, and documentation, as opposed to Linux only delivering a kernel and drivers, and relying on third-parties for system software; FreeBSD source code is generally released under a permissive BSD license, as opposed to the copyleft GPL used by Linux. The FreeBSD project includes a security team overseeing all software shipped in the base distribution. A wide range of additional third-party applications may be installe ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Btrfs
Btrfs (pronounced as "better F S", "butter F S", "b-tree F S", or simply by spelling it out) is a computer storage format that combines a file system based on the copy-on-write (COW) principle with a logical volume manager (not to be confused with Linux's LVM), developed together. It was initially designed at Oracle Corporation in 2007 for use in Linux, and since November 2013, the file system's on-disk format has been declared stable in the Linux kernel. According to Oracle, Btrfs "is not a true acronym". Btrfs is intended to address the lack of pooling, snapshots, checksums, and integral multi-device spanning in Linux file systems. Chris Mason, the principal Btrfs author, stated that its goal was "to let inuxscale for the storage that will be available. Scaling is not just about addressing the storage but also means being able to administer and to manage it with a clean interface that lets people see what's being used and makes it more reliable". History The core dat ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Linux Kernel
The Linux kernel is a free and open-source, monolithic, modular, multitasking, Unix-like operating system kernel. It was originally authored in 1991 by Linus Torvalds for his i386-based PC, and it was soon adopted as the kernel for the GNU operating system, which was written to be a free (libre) replacement for Unix. Linux is provided under the GNU General Public License version 2 only, but it contains files under other compatible licenses. Since the late 1990s, it has been included as part of a large number of operating system distributions, many of which are commonly also called Linux. Linux is deployed on a wide variety of computing systems, such as embedded devices, mobile devices (including its use in the Android operating system), personal computers, servers, mainframes, and supercomputers. It can be tailored for specific architectures and for several usage scenarios using a family of simple commands (that is, without the need of manually editing its source code ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Huffman Coding
In computer science and information theory, a Huffman code is a particular type of optimal prefix code that is commonly used for lossless data compression. The process of finding or using such a code proceeds by means of Huffman coding, an algorithm developed by David A. Huffman while he was a Sc.D. student at MIT, and published in the 1952 paper "A Method for the Construction of Minimum-Redundancy Codes". The output from Huffman's algorithm can be viewed as a variable-length code table for encoding a source symbol (such as a character in a file). The algorithm derives this table from the estimated probability or frequency of occurrence (''weight'') for each possible value of the source symbol. As in other entropy encoding methods, more common symbols are generally represented using fewer bits than less common symbols. Huffman's method can be efficiently implemented, finding a code in time linear to the number of input weights if these weights are sorted. However, although opt ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Entropy Encoding
In information theory, an entropy coding (or entropy encoding) is any lossless data compression method that attempts to approach the lower bound declared by Shannon's source coding theorem, which states that any lossless data compression method must have expected code length greater or equal to the entropy of the source. More precisely, the source coding theorem states that for any source distribution, the expected code length satisfies \mathbb E_ (d(x))\geq \mathbb E_ \log_b(P(x))/math>, where l is the number of symbols in a code word, d is the coding function, b is the number of symbols used to make output codes and P is the probability of the source symbol. An entropy coding attempts to approach this lower bound. Two of the most common entropy coding techniques are Huffman coding and arithmetic coding. If the approximate entropy characteristics of a data stream are known in advance (especially for signal compression), a simpler static code may be useful. These static codes in ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
LZ77
LZ77 and LZ78 are the two lossless data compression algorithms published in papers by Abraham Lempel and Jacob Ziv in 1977 and 1978. They are also known as LZ1 and LZ2 respectively. These two algorithms form the basis for many variations including LZW, LZSS, LZMA and others. Besides their academic influence, these algorithms formed the basis of several ubiquitous compression schemes, including GIF and the DEFLATE algorithm used in PNG and ZIP. They are both theoretically dictionary coders. LZ77 maintains a sliding window during compression. This was later shown to be equivalent to the ''explicit dictionary'' constructed by LZ78—however, they are only equivalent when the entire data is intended to be decompressed. Since LZ77 encodes and decodes from a sliding window over previously seen characters, decompression must always start at the beginning of the input. Conceptually, LZ78 decompression could allow random access to the input if the entire dictionary were known in ad ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Log Files
In computing, logging is the act of keeping a log of events that occur in a computer system, such as problems, errors or just information on current operations. These events may occur in the operating system or in other software. A message or log entry is recorded for each such event. These log messages can then be used to monitor and understand the operation of the system, to debug problems, or during an audit. Logging is particularly important in multi-user software, to have a central overview of the operation of the system. In the simplest case, messages are written to a file, called a log file. Alternatively, the messages may be written to a dedicated logging system or to a log management software, where it is stored in a database or on a different computer system. Specifically, a transaction log is a log of the communications between a system and the users of that system, or a data collection method that automatically captures the type, content, or time of transactions mad ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Pareto Efficiency
Pareto efficiency or Pareto optimality is a situation where no action or allocation is available that makes one individual better off without making another worse off. The concept is named after Vilfredo Pareto (1848–1923), Italian civil engineer and economist, who used the concept in his studies of economic efficiency and income distribution. The following three concepts are closely related: * Given an initial situation, a Pareto improvement is a new situation where some agents will gain, and no agents will lose. * A situation is called Pareto-dominated if there exists a possible Pareto improvement. * A situation is called Pareto-optimal or Pareto-efficient if no change could lead to improved satisfaction for some agent without some other agent losing or, equivalently, if there is no scope for further Pareto improvement. The Pareto front (also called Pareto frontier or Pareto set) is the set of all Pareto-efficient situations. Pareto originally used the word "optimal" for t ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
