Fully Buffered DIMM (or FB-DIMM) is a memory technology that can be used to increase reliability and density of memory systems. Unlike the parallel bus architecture of traditional DRAMs, an FB-DIMM has a serial interface between the memory controller and the ''advanced memory buffer'' (AMB). Conventionally, data lines from the
memory controller
The memory controller is a digital circuit that manages the flow of data going to and from the computer's main memory. A memory controller can be a separate chip or integrated into another chip, such as being placed on the same die or as an int ...
have to be connected to data lines in every
DRAM
Dynamic random-access memory (dynamic RAM or DRAM) is a type of random-access semiconductor memory that stores each bit of data in a memory cell, usually consisting of a tiny capacitor and a transistor, both typically based on metal-oxid ...
module, i.e. via
multidrop bus A multidrop bus (MDB) is a computer bus in which all components are connected to the electrical circuit. A process of arbitration determines which device sends information at any point. The other devices listen for the data they are intended to rece ...
es. As the memory width increases together with the access speed, the signal degrades at the interface between the bus and the device. This limits the speed and memory density, so FB-
DIMM
A DIMM () (Dual In-line Memory Module), commonly called a RAM stick, comprises a series of dynamic random-access memory integrated circuits. These memory modules are mounted on a printed circuit board and designed for use in personal compute ...
s take a different approach to solve the problem.
240-pin DDR2 FB-DIMMs are neither mechanically nor electrically compatible with conventional 240-pin DDR2 DIMMs. As a result, those two DIMM types are notched differently to prevent using the wrong one.
As with nearly all RAM specifications, the FB-DIMM specification was published by
JEDEC
The JEDEC Solid State Technology Association is an independent semiconductor engineering trade organization and standardization body headquartered in Arlington County, Virginia, United States.
JEDEC has over 300 members, including some of the w ...
.
Technology
![FB-DIMM DDR2 vs](https://upload.wikimedia.org/wikipedia/commons/8/83/FB-DIMM_DDR2_vs._DDR2_photo_with_pin_count.jpg)
Fully buffered DIMM architecture introduces an ''advanced memory buffer'' (AMB) between the memory controller and the memory module. Unlike the
parallel
Parallel is a geometric term of location which may refer to:
Computing
* Parallel algorithm
* Parallel computing
* Parallel metaheuristic
* Parallel (software), a UNIX utility for running programs in parallel
* Parallel Sysplex, a cluster of IBM ...
bus architecture of traditional DRAMs, an FB-DIMM has a
serial interface
In computing, a serial port is a serial communication interface through which information transfers in or out sequentially one bit at a time. This is in contrast to a parallel port, which communicates multiple bits simultaneously in parallel. T ...
between the memory controller and the AMB. This enables an increase to the width of the memory without increasing the pin count of the memory controller beyond a feasible level. With this architecture, the memory controller does not write to the memory module directly; rather it is done via the AMB. AMB can thus compensate for signal deterioration by buffering and resending the signal.
The AMB can also offer
error correction
In information theory and coding theory with applications in computer science and telecommunication, error detection and correction (EDAC) or error control are techniques that enable reliable delivery of digital data over unreliable communica ...
, without imposing any additional overhead on the processor or the system's memory controller. It can also use the
Bit Lane Failover Correction feature to identify bad data paths and remove them from operation, which dramatically reduces command/address errors. Also, since reads and writes are buffered, they can be done in parallel by the memory controller. This allows simpler interconnects, and (in theory) hardware-agnostic memory controller chips (such as
DDR2 and
DDR3
Double Data Rate 3 Synchronous Dynamic Random-Access Memory (DDR3 SDRAM) is a type of synchronous dynamic random-access memory (SDRAM) with a high bandwidth ("double data rate") interface, and has been in use since 2007. It is the higher-speed ...
) that can be used interchangeably.
The downsides to this approach are; it introduces
latency to the memory request, it requires additional power consumption for the buffer chips, and current implementations create a memory write bus significantly narrower than the memory read bus. This means workloads that use many writes (such as
high-performance computing
High-performance computing (HPC) uses supercomputers and computer clusters to solve advanced computation problems.
Overview
HPC integrates systems administration (including network and security knowledge) and parallel programming into a mult ...
) will be significantly slowed. However, this slowdown is nowhere near as bad as not having enough memory capacity to avoid using significant amounts of
virtual memory
In computing, virtual memory, or virtual storage is a memory management technique that provides an "idealized abstraction of the storage resources that are actually available on a given machine" which "creates the illusion to users of a very l ...
, so workloads that use extreme amounts of memory in irregular patterns might be helped by using fully buffered DIMMs.
Protocol
The JEDEC standar
JESD206defines the protocol, and JESD82-20 defines the AMB interface to DDR2 memory. The protocol is more generally described in many other places.
The FB-DIMM channel consists of 14 "northbound" bit lanes carrying data from memory to the processor and 10 "southbound" bit lanes carrying commands and data from the processor to memory. Each bit is carried over a differential pair, clocked at 12 times the basic memory clock rate, 6 times the double-pumped data rate. E.g. for DDR2-667 DRAM chips, the channel would operate at 4000 MHz. Every 12 cycles constitute one frame, 168 bits northbound and 120 bits southbound.
One northbound frame carries 144 data bits, the amount of data produced by a 72-bit wide DDR SDRAM array in that time, and 24 bits of
CRC for error detection. There is no header information, although unused frames include a deliberately invalid CRC.
One southbound frame carries 98 payload bits and 22 CRC bits. Two payload bits are a frame type, and 24 bits are a command. The remaining 72 bits may be either (depending on the frame type), 72 bits of write data, two more 24-bit commands, or one more command plus 36 bits of data to be written to an AMB control register.
The commands correspond to standard DRAM access cycles, such as row select, precharge, and refresh commands. Read and write commands include only column addresses. All commands include a 3-bit FB-DIMM address, allowing up to 8 FB-DIMM modules on a channel.
Because write data is supplied more slowly than DDR memory expects it, writes are buffered in the AMB until they can be written in a burst. Write commands are not directly linked to the write data; instead, each AMB has a write data FIFO that is filled by four consecutive write data frames, and is emptied by a write command.
Both northbound and southbound links can operate at full speed with one bit line disabled, by discarding 12 bits of CRC information per frame.
Note that the bandwidth of an FB-DIMM channel is equal to the peak read bandwidth of a DDR memory channel (and this speed can be sustained, as there is no contention for the northbound channel), plus half of the peak write bandwidth of a DDR memory channel (which can often be sustained, if one command per frame is sufficient). The only overhead is the need for a channel sync frame (which elicits a northbound status frame in response) every 32 to 42 frames (2.5–3% overhead).
Implementations
Intel has adopted the technology for their
Xeon
Xeon ( ) is a brand of x86 microprocessors designed, manufactured, and marketed by Intel, targeted at the non-consumer workstation, server, and embedded system markets. It was introduced in June 1998. Xeon processors are based on the same arc ...
5000/5100 series and beyond, which they consider "a long-term strategic direction for servers".
Sun Microsystems
Sun Microsystems, Inc. (Sun for short) was an American technology company that sold computers, computer components, software, and information technology services and created the Java programming language, the Solaris operating system, ZFS, the ...
used FB-DIMMs for the
Niagara II (UltraSparc T2) server processor.
Intel's enthusiast system platform
Skulltrail uses FB-DIMMs for their dual CPU socket, multi-GPU system.
FB-DIMMS have 240 pins and are the same total length as other DDR DIMMs but differ by having indents on both ends within the slot.
The cost of FB-DIMM memory was initially much higher than
registered DIMM
Registered (also called buffered) memory modules have a register between the DRAM modules and the system's memory controller. They place less electrical load on the memory controller and allow single systems to remain stable with more memory mod ...
, which may be one of the factors behind its current level of acceptance. Also, the AMB chip dissipates considerable heat, leading to additional cooling problems. Although strenuous efforts were made to minimize delay in the AMB, there is some noticeable cost in memory access latency.
History
As of September 2006,
AMD
Advanced Micro Devices, Inc. (AMD) is an American multinational semiconductor company based in Santa Clara, California, that develops computer processors and related technologies for business and consumer markets. While it initially manufactur ...
has taken FB-DIMM off their roadmap. In December 2006, AMD has revealed in one of the slides that microprocessors based on the new
K10 microarchitecture has the support for FB-DIMM "when appropriate". In addition, AMD also developed Socket G3 Memory Extender (
G3MX), which uses a single buffer for every 4 modules instead of one for each, to be used by Opteron-based systems in 2009.
At the 2007
Intel Developer Forum
The Intel Developer Forum (IDF) was a biannual gathering of technologists to discuss Intel products and products based on Intel products. The first IDF was held in 1997.
To emphasize the importance of China, the Spring 2007 IDF was held in Beiji ...
, it was revealed that major memory manufacturers have no plans to extend FB-DIMM to support DDR3 SDRAM. Instead, only registered DIMM for DDR3 SDRAM had been demonstrated.
In 2007,
Intel
Intel Corporation is an American multinational corporation and technology company headquartered in Santa Clara, California. It is the world's largest semiconductor chip manufacturer by revenue, and is one of the developers of the x86 seri ...
demonstrated FB-DIMM with shorter latencies, CL5 and CL3, showing improvement in latencies.
On August 5, 2008,
Elpida Memory
Micron Memory Japan, K.K. is a Japanese subsidiary of Micron Technology. It was formerly known as established in 1999 that developed, designed, manufactured and sold dynamic random-access memory (DRAM) products. It was also a semiconductor fou ...
announced that it would mass-produce the world's first FB-DIMM at 16
Gigabyte
The gigabyte () is a multiple of the unit byte for digital information. The prefix ''giga'' means 109 in the International System of Units (SI). Therefore, one gigabyte is one billion bytes. The unit symbol for the gigabyte is GB.
This defini ...
capacity, as from
Q4 2008, however the product has not appeared and the press release has been deleted from Elpida's site.
See also
*
HyperCloud Memory
HyperCloud Memory (HCDIMM) is a DDR3 SDRAM dual in-line memory module (DIMM) used in server applications requiring a great deal of memory. It was initially launched in 2009 at the International Supercomputing Conference by Irvine, California, based ...
*
Load Reduced DIMM
Registered (also called buffered) memory modules have a register between the DRAM modules and the system's memory controller. They place less electrical load on the memory controller and allow single systems to remain stable with more memory mod ...
(LRDIMM)
References
{{Reflist, 2
External links
How FB-DIMM Memories Work*
The Inquirer
''The Inquirer'' (stylized as TheINQUIRER) was a British technology tabloid website founded by Mike Magee after his departure from ''The Register'' (of which he was one of the founding members) in 2001. In 2006 the site was acquired by Dutch ...
series
Part 1Part 2Part 3
Computer memory