![High Bandwidth Memory schematic](https://upload.wikimedia.org/wikipedia/commons/b/b5/High_Bandwidth_Memory_schematic.svg)
High Bandwidth Memory (HBM) is a high-speed
computer memory
In computing, memory is a device or system that is used to store information for immediate use in a computer or related computer hardware and digital electronic devices. The term ''memory'' is often synonymous with the term ''primary storage ...
interface for
3D-stacked synchronous dynamic random-access memory
Synchronous dynamic random-access memory (synchronous dynamic RAM or SDRAM) is any DRAM where the operation of its external pin interface is coordinated by an externally supplied clock signal.
DRAM integrated circuits (ICs) produced from the ...
(SDRAM) initially from
Samsung
The Samsung Group (or simply Samsung) ( ko, 삼성 ) is a South Korean multinational manufacturing conglomerate headquartered in Samsung Town, Seoul, South Korea. It comprises numerous affiliated businesses, most of them united under the ...
,
AMD
Advanced Micro Devices, Inc. (AMD) is an American multinational semiconductor company based in Santa Clara, California, that develops computer processors and related technologies for business and consumer markets. While it initially manufactur ...
and
SK Hynix
SK hynix Inc. is a South Korean supplier of dynamic random-access memory (DRAM) chips and flash memory chips. Hynix is the world's second-largest memory chipmaker (after Samsung Electronics) and the world's third-largest semiconductor company. ...
. It is used in conjunction with high-performance graphics accelerators, network devices, high-performance datacenter AI
ASICs and
FPGAs and in some supercomputers (such as the NEC SX-Aurora TSUBASA and
Fujitsu A64FX
The A64FX is a 64-bit ARM architecture microprocessor designed by Fujitsu. The processor is replacing the SPARC64 V as Fujitsu's processor for supercomputer applications. It powers the Fugaku supercomputer, the fastest supercomputer in the ...
). The first HBM memory chip was produced by SK Hynix in 2013,
and the first devices to use HBM were the
AMD Fiji GPUs in 2015.
High Bandwidth Memory has been adopted by
JEDEC
The JEDEC Solid State Technology Association is an independent semiconductor engineering trade organization and standardization body headquartered in Arlington County, Virginia, United States.
JEDEC has over 300 members, including some of the w ...
as an industry standard in October 2013.
[High Bandwidth Memory (HBM) DRAM (JESD235)](_blank)
JEDEC, October 2013 The second generation, HBM2, was accepted by JEDEC in January 2016.
Technology
HBM achieves higher bandwidth while using less power in a substantially smaller form factor than
DDR4
Double Data Rate 4 Synchronous Dynamic Random-Access Memory (DDR4 SDRAM) is a type of synchronous dynamic random-access memory with a high bandwidth (" double data rate") interface.
Released to the market in 2014, it is a variant of dynamic ra ...
or
GDDR5
Graphics Double Data Rate 5 Synchronous Dynamic Random-Access Memory (GDDR5 SDRAM) is a type of synchronous graphics random-access memory (SGRAM) with a high bandwidth ("double data rate") interface designed for use in graphics cards, game co ...
.
[HBM: Memory Solution for Bandwidth-Hungry Processors](_blank)
, Joonyoung Kim and Younsu Kim, SK Hynix // Hot Chips 26, August 2014 This is achieved by stacking up to eight
DRAM
Dynamic random-access memory (dynamic RAM or DRAM) is a type of random-access semiconductor memory that stores each bit of data in a memory cell, usually consisting of a tiny capacitor and a transistor, both typically based on metal-oxid ...
die
Die, as a verb, refers to death, the cessation of life.
Die may also refer to:
Games
* Die, singular of dice, small throwable objects used for producing random numbers
Manufacturing
* Die (integrated circuit), a rectangular piece of a semicondu ...
s and an optional base die which can include buffer circuitry and test logic. The stack is often connected to the memory controller on a
GPU
A graphics processing unit (GPU) is a specialized electronic circuit designed to manipulate and alter memory to accelerate the creation of images in a frame buffer intended for output to a display device. GPUs are used in embedded systems, mobi ...
or
CPU through a substrate, such as a silicon
interposer. Alternatively, the memory die could be stacked directly on the CPU or GPU chip. Within the stack the die are vertically interconnected by
through-silicon via
In electronic engineering, a through-silicon via (TSV) or through-chip via is a vertical electrical connection (via) that passes completely through a silicon wafer or die. TSVs are high-performance interconnect techniques used as an alternative ...
s (TSVs) and
microbump
In integrated circuit packaging, a solder ball, also a solder bump (ofter referred to simply as "ball" or "bumps") is a ball of solder that provides the contact between the chip package and the printed circuit board, as well as between stacked ...
s. The HBM technology is similar in principle but incompatible with the
Hybrid Memory Cube (HMC) interface developed by
Micron Technology.
HBM memory bus is very wide in comparison to other DRAM memories such as DDR4 or GDDR5. An HBM stack of four DRAM dies (4Hi) has two 128bit channels per die for a total of 8 channels and a width of 1024 bits in total. A graphics card/GPU with four 4Hi HBM stacks would therefore have a memory bus with a width of 4096 bits. In comparison, the bus width of GDDR memories is 32 bits, with 16 channels for a graphics card with a 512bit memory interface. HBM supports up to 4 GB per package.
The larger number of connections to the memory, relative to DDR4 or GDDR5, required a new method of connecting the HBM memory to the GPU (or other processor). AMD and Nvidia have both used purpose-built silicon chips, called ''
interposers'', to connect the memory and GPU. This interposer has the added advantage of requiring the memory and processor to be physically close, decreasing memory paths. However, as
semiconductor device fabrication
Semiconductor device fabrication is the process used to manufacture semiconductor devices, typically integrated circuit (IC) chips such as modern computer processors, microcontrollers, and memory chips such as NAND flash and DRAM that are pres ...
is significantly more expensive than
printed circuit board
A printed circuit board (PCB; also printed wiring board or PWB) is a medium used in Electrical engineering, electrical and electronic engineering to connect electronic components to one another in a controlled manner. It takes the form of a L ...
manufacture, this adds cost to the final product.
File:AMD@28nm@GCN 3th gen@Fiji@Radeon R9 Nano@SPMRC REA0356A-1539 215-0862120 Stack-DSC10301-DSC10363 - ZS-retouched (29514443756).jpg, HBM DRAM die
File:AMD@28nm@GCN 3th gen@Fiji@Radeon R9 Nano@SPMRC REA0356A-1539 215-0862120 Stack-DSC10364-DSC10418 - ZS-DMap (29548526985).jpg, HBM controller die
File:AMD@28nm@GCN 3th gen@Fiji@Radeon R9 Nano@SPMRC REA0356A-1539 215-0862120 DSC04466 (29461603171).jpg, HBM memory on an AMD Radeon R9 Nano graphics card's GPU package
Interface
The HBM DRAM is tightly coupled to the host compute die with a distributed interface. The interface is divided into independent channels. The channels are completely independent of one another and are not necessarily synchronous to each other. The HBM DRAM uses a wide-interface architecture to achieve high-speed, low-power operation. The HBM DRAM uses a 500 MHz
differential clock CK_t / CK_c (where the suffix "_t" denotes the "true", or "positive", component of the differential pair, and "_c" stands for the "complementary" one). Commands are registered at the rising edge of CK_t, CK_c. Each channel interface maintains a 128bit data bus operating at double data rate (DDR). HBM supports transfer rates of 1
GT/s
In computer technology, transfers per second and its more common secondary terms gigatransfers per second (abbreviated as GT/s) and megatransfers per second (MT/s) are informal language that refer to the number of operations transferring data that ...
per pin (transferring 1 bit), yielding an overall package bandwidth of 128 GB/s.
HBM2
The second generation of High Bandwidth Memory, HBM2, also specifies up to eight dies per stack and doubles pin transfer rates up to 2
GT/s
In computer technology, transfers per second and its more common secondary terms gigatransfers per second (abbreviated as GT/s) and megatransfers per second (MT/s) are informal language that refer to the number of operations transferring data that ...
. Retaining 1024bit wide access, HBM2 is able to reach 256 GB/s memory bandwidth per package. The HBM2 spec allows up to 8 GB per package. HBM2 is predicted to be especially useful for performance-sensitive consumer applications such as
virtual reality
Virtual reality (VR) is a simulated experience that employs pose tracking and 3D near-eye displays to give the user an immersive feel of a virtual world. Applications of virtual reality include entertainment (particularly video games), educ ...
.
On January 19, 2016,
Samsung
The Samsung Group (or simply Samsung) ( ko, 삼성 ) is a South Korean multinational manufacturing conglomerate headquartered in Samsung Town, Seoul, South Korea. It comprises numerous affiliated businesses, most of them united under the ...
announced early mass production of HBM2, at up to 8 GB per stack.
SK Hynix also announced availability of 4 GB stacks in August 2016.
File:AMD@14nm@GCN 5th gen@Vega10@Radeon RX Vega 64@HBM DRAM Die@ Stack-DSC08974-DSC09078 - ZS-retouched.jpg, HBM2 DRAM die
File:AMD@14nm@GCN 5th gen@Vega10@Radeon RX Vega 64@HBM Logic Die@ Stack-DSC08709-DSC08834 - ZS-retouched.jpg, HBM2 controller die
File:AMD@14nm@GCN 5th gen@Vega10@Radeon RX Vega 64@ES-Sample@ Stack-DSC01160-DSC01176 - ZS-retouched.jpg, The HBM2 interposer of a Radeon RX Vega 64 GPU, with removed HBM dies; the GPU is still in place
HBM2E
In late 2018, JEDEC announced an update to the HBM2 specification, providing for increased bandwidth and capacities. Up to 307 GB/s per stack (2.5 Tbit/s effective data rate) is now supported in the official specification, though products operating at this speed had already been available. Additionally, the update added support for 12Hi stacks (12 dies) making capacities of up to 24 GB per stack possible.
On March 20, 2019,
Samsung
The Samsung Group (or simply Samsung) ( ko, 삼성 ) is a South Korean multinational manufacturing conglomerate headquartered in Samsung Town, Seoul, South Korea. It comprises numerous affiliated businesses, most of them united under the ...
announced their Flashbolt HBM2E, featuring eight dies per stack, a transfer rate of 3.2
GT/s
In computer technology, transfers per second and its more common secondary terms gigatransfers per second (abbreviated as GT/s) and megatransfers per second (MT/s) are informal language that refer to the number of operations transferring data that ...
, providing a total of 16 GB and 410 GB/s per stack.
August 12, 2019,
SK Hynix
SK hynix Inc. is a South Korean supplier of dynamic random-access memory (DRAM) chips and flash memory chips. Hynix is the world's second-largest memory chipmaker (after Samsung Electronics) and the world's third-largest semiconductor company. ...
announced their HBM2E, featuring eight dies per stack, a transfer rate of 3.6
GT/s
In computer technology, transfers per second and its more common secondary terms gigatransfers per second (abbreviated as GT/s) and megatransfers per second (MT/s) are informal language that refer to the number of operations transferring data that ...
, providing a total of 16 GB and 460 GB/s per stack. On 2 July 2020, SK Hynix announced that mass production has begun.
HBM3
In late 2020,
Micron
The micrometre ( international spelling as used by the International Bureau of Weights and Measures; SI symbol: μm) or micrometer (American spelling), also commonly known as a micron, is a unit of length in the International System of Un ...
unveiled that the HBM2E standard would be updated and alongside that they unveiled the next standard known as HBMnext (later renamed to HBM3). This was to be a big generational leap from HBM2 and the replacement to HBM2E. This new
VRAM
Video random access memory (VRAM) is dedicated computer memory used to store the pixels and other graphics data as a framebuffer to be rendered on a computer monitor. This is often different technology than other computer memory, to facilitate b ...
would have come to the market in the Q4 of 2022. This would likely introduce a new architecture as the naming suggests.
While the architecture might be overhauled, leaks point toward the performance to be similar to that of the updated HBM2E standard. This RAM is likely to be used mostly in data center
GPUs
A graphics processing unit (GPU) is a specialized electronic circuit designed to manipulate and alter memory to accelerate the creation of images in a frame buffer intended for output to a display device. GPUs are used in embedded systems, mobil ...
.
In mid 2021,
SK Hynix
SK hynix Inc. is a South Korean supplier of dynamic random-access memory (DRAM) chips and flash memory chips. Hynix is the world's second-largest memory chipmaker (after Samsung Electronics) and the world's third-largest semiconductor company. ...
unveiled some specifications of the HBM3 standard, with 5.2Gbit/s I/O speeds and bandwidth of 665GB/s per package, as well as up to 16-high 2.5D and 3D solutions.
On 20 October 2021, before the JEDEC standard for HBM3 was finalised, SK Hynix was the first memory vendor to announce that it has finished development of HBM3 memory devices. According to SK Hynix, the memory would run as fast as 6.4Gbps/pin, double the data rate of JEDEC-standard HBM2E, which formally tops out at 3.2Gbps/pin, or 78% faster than SK Hynix’s own 3.6Gbps/pin HBM2E. The devices support a data transfer rate of 6.4 GT/s and therefore a single HBM3 stack may provide a bandwidth of up to 819 GB/s. The basic bus widths for HBM3 remain unchanged, with a single stack of memory being 1024-bits wide. SK Hynix would offer their memory in two capacities: 16GB and 24GB, aligning with 8-Hi and 12-Hi stacks respectively. The stacks consist of 8 or 12 16Gb DRAMs that are each 30 μm thick and interconnected using Through Silicon Vias (TSVs).
According to Ryan Smith of ''
AnandTech'', the SK Hynix first generation HBM3 memory has the same density as their latest-generation HBM2E memory, meaning that device vendors looking to increase their total memory capacities for their next-generation parts would need to use memory with 12 dies/layers, up from the 8 layer stacks they typically used until then.
According to Anton Shilov of ''
Tom's Hardware
''Tom's Hardware'' is an online publication owned by Future plc and focused on technology. It was founded in 1996 by Thomas Pabst. It provides articles, news, price comparisons, videos and reviews on computer hardware and high technology. The si ...
'', high-performance compute GPUs or FPGAs typically use four or six HBM stacks, so with SK Hynix's HBM3 24GB stacks they would accordingly get 3.2 TB/s or 4.9 TB/s of memory bandwidth. He also noted that SK Hynix's HBM3 chips are square, not rectangular like HBM2 and HBM2E chips.
According to Chris Mellor of ''
The Register
''The Register'' is a British technology news website co-founded in 1994 by Mike Magee, John Lettice and Ross Alderson. The online newspaper's masthead sublogo is "''Biting the hand that feeds IT''." Their primary focus is information tec ...
'', with JEDEC not yet having developed its HBM3 standard, might mean that SK Hynix would need to retrofit its design to a future and faster one.
JEDEC officially announced the HBM3 standard on January 27, 2022. The number of memory channels was doubled from 8 channels of 128 bits with HBM2e to 16 channels of 64 bits with HBM3. Therefore, the total number of data pins of the interface is still 1024.
In June 2022, SK hynix announced they started mass production of industry's first HBM3 memory to be used with Nvidia's H100 GPU expected to ship in Q3 2022. The memory will provide H100 with "up to 819 GB/s" of memory bandwidth.
In August 2022, Nvidia announced that its "Hopper" H100 GPU will ship with five active HBM3 sites (out of six on board) offering 80 GB of RAM and 3 TB/s of memory bandwidth (16 GB and 600 GB/s per site).
HBM-PIM
In February 2021,
Samsung
The Samsung Group (or simply Samsung) ( ko, 삼성 ) is a South Korean multinational manufacturing conglomerate headquartered in Samsung Town, Seoul, South Korea. It comprises numerous affiliated businesses, most of them united under the ...
announced the development of HBM with processing-in-memory (PIM). This new memory brings AI computing capabilities inside the memory, to increase the large-scale processing of data. A DRAM-optimised AI engine is placed inside each memory bank to enable parallel processing and minimise data movement. Samsung claims this will deliver twice the system performance and reduce energy consumption by more than 70%, while not requiring any hardware or software changes to the rest of the system.
History
Background
Die-stacked memory was initially commercialized in the
flash memory
Flash memory is an electronic non-volatile computer memory storage medium that can be electrically erased and reprogrammed. The two main types of flash memory, NOR flash and NAND flash, are named for the NOR and NAND logic gates. Both us ...
industry.
Toshiba
, commonly known as Toshiba and stylized as TOSHIBA, is a Japanese multinational conglomerate corporation headquartered in Minato, Tokyo, Japan. Its diversified products and services include power, industrial and social infrastructure system ...
introduced a
NAND flash
Flash memory is an Integrated circuit, electronic Non-volatile memory, non-volatile computer memory storage medium that can be electrically erased and reprogrammed. The two main types of flash memory, NOR flash and NAND flash, are named for t ...
memory chip with eight stacked dies in April 2007, followed by
Hynix Semiconductor introducing a NAND flash chip with 24 stacked dies in September 2007.
3D-stacked
random-access memory
Random-access memory (RAM; ) is a form of computer memory that can be read and changed in any order, typically used to store working Data (computing), data and machine code. A Random access, random-access memory device allows data items to b ...
(RAM) using
through-silicon via
In electronic engineering, a through-silicon via (TSV) or through-chip via is a vertical electrical connection (via) that passes completely through a silicon wafer or die. TSVs are high-performance interconnect techniques used as an alternative ...
(TSV) technology was commercialized by
Elpida Memory
Micron Memory Japan, K.K. is a Japanese subsidiary of Micron Technology. It was formerly known as established in 1999 that developed, designed, manufactured and sold dynamic random-access memory (DRAM) products. It was also a semiconductor fou ...
, which developed the first 8
GB DRAM
Dynamic random-access memory (dynamic RAM or DRAM) is a type of random-access semiconductor memory that stores each bit of data in a memory cell, usually consisting of a tiny capacitor and a transistor, both typically based on metal-oxid ...
chip (stacked with four
DDR3
Double Data Rate 3 Synchronous Dynamic Random-Access Memory (DDR3 SDRAM) is a type of synchronous dynamic random-access memory (SDRAM) with a high bandwidth (" double data rate") interface, and has been in use since 2007. It is the higher-spee ...
SDRAM
Synchronous dynamic random-access memory (synchronous dynamic RAM or SDRAM) is any DRAM where the operation of its external pin interface is coordinated by an externally supplied clock signal.
DRAM integrated circuits (ICs) produced from the ...
dies) in September 2009, and released it in June 2011. In 2011,
SK Hynix
SK hynix Inc. is a South Korean supplier of dynamic random-access memory (DRAM) chips and flash memory chips. Hynix is the world's second-largest memory chipmaker (after Samsung Electronics) and the world's third-largest semiconductor company. ...
introduced 16GB DDR3 memory (
40nm class) using TSV technology,
Samsung Electronics
Samsung Electronics Co., Ltd. (, sometimes shortened to SEC and stylized as SΛMSUNG) is a South Korean multinational corporation, multinational electronics corporation headquartered in Yeongtong-gu, Suwon, South Korea. It is the pinnacle of ...
introduced 3D-stacked 32GB DDR3 (
30nm class) based on TSV in September, and then Samsung and
Micron Technology announced TSV-based
Hybrid Memory Cube (HMC) technology in October.
JEDEC
The JEDEC Solid State Technology Association is an independent semiconductor engineering trade organization and standardization body headquartered in Arlington County, Virginia, United States.
JEDEC has over 300 members, including some of the w ...
first released the JESD229 standard for Wide IO memory, the predecessor of HBM featuring four 128 bit channels with single data rate clocking, in December 2011 after several years of work. The first HBM standard JESD235 followed in October 2013.
Development
![AMD Fiji GPU package with GPU, HBM memory and interposer](https://upload.wikimedia.org/wikipedia/commons/e/e7/AMD_Fiji_GPU_package_with_GPU%2C_HBM_memory_and_interposer.jpg)
The development of High Bandwidth Memory began at AMD in 2008 to solve the problem of ever-increasing power usage and form factor of computer memory. Over the next several years, AMD developed procedures to solve die-stacking problems with a team led by Senior AMD Fellow Bryan Black.
[High-Bandwidth Memory (HBM) from AMD: Making Beautiful Memory](_blank)
AMD
Advanced Micro Devices, Inc. (AMD) is an American multinational semiconductor company based in Santa Clara, California, that develops computer processors and related technologies for business and consumer markets. While it initially manufactur ...
To help AMD realize their vision of HBM, they enlisted partners from the memory industry, particularly Korean company
SK Hynix,
which had prior experience with 3D-stacked memory,
as well as partners from the
interposer industry (Taiwanese company
UMC) and
packaging
Packaging is the science, art and technology of enclosing or protecting products for distribution, storage, sale, and use. Packaging also refers to the process of designing, evaluating, and producing packages. Packaging can be described as a co ...
industry (
Amkor Technology
Amkor Technology, Inc. is a semiconductor product packaging and test services provider. The company has been headquartered in Tempe, Arizona, since 2005, when it was moved from West Chester, Pennsylvania. The company was founded in 1968 and, , ha ...
and
ASE
Ase may refer to:
* Ase, Nigeria, a town in Delta State, Nigeria
* -ase, a suffix used for the names of enzymes
* Aṣẹ, a West African philosophical concept
* American Sign Language (ISO 639-3 code: ase)
See also
* Åse (disambiguation)
* ...
).
The development of HBM was completed in 2013, when SK Hynix built the first HBM memory chip.
HBM was adopted as industry standard JESD235 by
JEDEC
The JEDEC Solid State Technology Association is an independent semiconductor engineering trade organization and standardization body headquartered in Arlington County, Virginia, United States.
JEDEC has over 300 members, including some of the w ...
in October 2013, following a proposal by AMD and SK Hynix in 2010.
High volume manufacturing began at a Hynix facility in
Icheon
Icheon () is a city in Gyeonggi Province, South Korea.
Together with Yeoju, Icheon is known as a center of South Korean ceramic manufacturing and is a UNESCO City of Crafts and Folk Art. Other famous local products include peaches and rice. Loc ...
, South Korea, in 2015.
The first GPU utilizing HBM was the AMD Fiji which was released in June 2015 powering the AMD Radeon R9 Fury X.
In January 2016,
Samsung Electronics
Samsung Electronics Co., Ltd. (, sometimes shortened to SEC and stylized as SΛMSUNG) is a South Korean multinational corporation, multinational electronics corporation headquartered in Yeongtong-gu, Suwon, South Korea. It is the pinnacle of ...
began early mass production of HBM2.
The same month, HBM2 was accepted by JEDEC as standard JESD235a.
The first GPU chip utilizing HBM2 is the
Nvidia Tesla
Nvidia Tesla was the name of Nvidia's line of products targeted at stream processing or general-purpose graphics processing units (GPGPU), named after pioneering electrical engineer Nikola Tesla. Its products began using GPUs from the G80 ser ...
P100 which was officially announced in April 2016.
In June 2016,
Intel
Intel Corporation is an American multinational corporation and technology company headquartered in Santa Clara, California. It is the world's largest semiconductor chip manufacturer by revenue, and is one of the developers of the x86 seri ...
released a family of
Xeon Phi
Xeon Phi was a series of x86 manycore processors designed and made by Intel. It was intended for use in supercomputers, servers, and high-end workstations. Its architecture allowed use of standard programming languages and application program ...
processors with 8 stacks of HCDRAM, Micron's version of HBM.
At
Hot Chips
The Institute of Electrical and Electronics Engineers (IEEE) is a 501(c)(3) professional association for electronic engineering and electrical engineering (and associated disciplines) with its corporate office in New York City and its operation ...
in August 2016, both Samsung and Hynix announced a new generation HBM memory technologies. Both companies announced high performance products expected to have increased density, increased bandwidth, and lower power consumption. Samsung also announced a lower-cost version of HBM under development targeting mass markets. Removing the buffer die and decreasing the number of
TSVs lowers cost, though at the expense of a decreased overall bandwidth (200 GB/s).
Nvidia announced Nvidia Hopper GH100 GPU, the world's first GPU utilizing HBM3 on March 22, 2022.
See also
*
Stacked DRAM
*
eDRAM
*
Chip stack multi-chip module
A multi-chip module (MCM) is generically an electronic assembly (such as a package with a number of conductor terminals or "pins") where multiple integrated circuits (ICs or "chips"), semiconductor dies and/or other discrete components are inte ...
*
Hybrid Memory Cube: stacked memory standard from
Micron Technology (2011)
References
External links
High Bandwidth Memory (HBM) DRAM (JESD235) JEDEC, October 2013
*
*
HBM vs HBM2 vs GDDR5 vs GDDR5X Memory Comparison
{{DRAM
Computer memory
AMD technologies
SDRAM
American inventions
South Korean inventions
Taiwanese inventions