Tensor Shader

In computer graphics, a shader is a computer program that calculates the appropriate levels of light, darkness, and color during the rendering of a 3D scene—a process known as ''shading''. Shaders have evolved to perform a variety of specialized functions in computer graphics special effects and video post-processing, as well as general-purpose computing on graphics processing units.

Traditional shaders calculate rendering effects on graphics hardware with a high degree of flexibility. Most shaders are coded for (and run on) a graphics processing unit (GPU), though this is not a strict requirement. ''Shading languages'' are used to program the GPU's rendering pipeline, which has mostly superseded the fixed-function pipeline of the past that only allowed for common geometry transforming and pixel-shading functions; with shaders, customized effects can be used. The position and color ( hue, saturation, brightness, and contrast) of all pixels, vertices, and/or textures used to construct a final rendered image can be altered using algorithms defined in a shader, and can be modified by external variables or textures introduced by the computer program calling the shader.

Shaders are used widely in cinema post-processing, computer-generated imagery, and video games to produce a range of effects. Beyond simple lighting models, more complex uses of shaders include: altering the hue, saturation, brightness ( HSL/HSV) or contrast of an image; producing blur, light bloom, volumetric lighting, normal mapping (for depth effects), bokeh, cel shading, posterization, bump mapping, distortion, chroma keying (for so-called "bluescreen/ greenscreen" effects), edge and motion detection, as well as psychedelic effects such as those seen in the demoscene.

History

The first known use of the term "shader" was introduced to the public by Pixar with version 3.0 of their RenderMan Interface Specification, originally published in May 1988.

As graphics processing units evolved, major graphics software libraries such as OpenGL and Direct3D began to support shaders. The first shader-capable GPUs only supported pixel shading, but vertex shaders were quickly introduced once developers realized the power of shaders. The first video card with a programmable pixel shader was the Nvidia GeForce 3 (NV20), released in 2001. Geometry shaders were introduced with Direct3D 10 and OpenGL 3.2. Eventually, graphics hardware evolved toward a unified shader model.

Design

Shaders are simple programs that describe the traits of either a vertex or a pixel. Vertex shaders describe the attributes (position, texture coordinates, colors, etc.) of a vertex, while pixel shaders describe the traits (color, z-depth and alpha value) of a pixel. A vertex shader is called for each vertex in a primitive (possibly after tessellation); thus one vertex in, one (updated) vertex out. Each vertex is then rendered as a series of pixels onto a surface (block of memory) that will eventually be sent to the screen.

Shaders replace a section of the graphics hardware typically called the Fixed Function Pipeline (FFP), so-called because it performs lighting and texture mapping in a hard-coded manner. Shaders provide a programmable alternative to this hard-coded approach.

The basic graphics pipeline is as follows:

* The CPU sends instructions (compiled shading language programs) and geometry data to the graphics processing unit, located on the graphics card.
* Within the vertex shader, the geometry is transformed.
* If a geometry shader is in the graphics processing unit and active, some changes of the geometries in the scene are performed.
* If a tessellation shader is in the graphics processing unit and active, the geometries in the scene can be subdivided.
* The calculated geometry is triangulated (subdivided into triangles).
* Triangles are broken down into fragment quads (one fragment quad is a 2 × 2 fragment primitive).
* Fragment quads are modified according to the fragment shader.
* The depth test is performed; fragments that pass will get written to the screen and might get blended into the frame buffer.

The graphic pipeline uses these steps in order to transform three-dimensional (or two-dimensional) data into useful two-dimensional data for displaying. In general, this is a large pixel matrix or "frame buffer".

Types

There are three types of shaders in common use (pixel, vertex, and geometry shaders), with several more recently added. While older graphics cards utilize separate processing units for each shader type, newer cards feature unified shaders which are capable of executing any type of shader. This allows graphics cards to make more efficient use of processing power.

2D shaders

2D shaders act on digital images, also called ''textures'' in the field of computer graphics. They modify attributes of pixels. 2D shaders may take part in rendering 3D geometry. Currently the only type of 2D shader is a pixel shader.

Pixel shaders

Pixel shaders, also known as fragment shaders, compute color and other attributes of each "fragment": a unit of rendering work affecting at most a single output pixel. The simplest kinds of pixel shaders output one screen pixel as a color value; more complex shaders with multiple inputs/outputs are also possible. Pixel shaders range from simply always outputting the same color, to applying a lighting value, to doing bump mapping, shadows, specular highlights, translucency and other phenomena. They can alter the depth of the fragment (for Z-buffering), or output more than one color if multiple render targets are active. In 3D graphics, a pixel shader alone cannot produce some kinds of complex effects because it operates only on a single fragment, without knowledge of a scene's geometry (i.e. vertex data). However, pixel shaders do have knowledge of the screen coordinate being drawn, and can sample the screen and nearby pixels if the contents of the entire screen are passed as a texture to the shader. This technique can enable a wide variety of two-dimensional postprocessing effects such as blur, or edge detection/enhancement for cartoon/cel shaders. Pixel shaders may also be applied in ''intermediate'' stages to any two-dimensional images— sprites or textures—in the pipeline, whereas vertex shaders always require a 3D scene. For instance, a pixel shader is the only kind of shader that can act as a postprocessor or filter for a video stream after it has been rasterized.

3D shaders

3D shaders act on 3D models or other geometry but may also access the colors and textures used to draw the model or mesh. Vertex shaders are the oldest type of 3D shader, generally making modifications on a per-vertex basis. Newer geometry shaders can generate new vertices from within the shader. Tessellation shaders are the newest 3D shaders; they act on batches of vertices all at once to add detail—such as subdividing a model into smaller groups of triangles or other primitives at runtime, to improve things like curves and bumps, or change other attributes.

Vertex shaders

Vertex shaders are run once for each 3D vertex given to the graphics processor. The purpose is to transform each vertex's 3D position in virtual space to the 2D coordinate at which it appears on the screen (as well as a depth value for the Z-buffer). Vertex shaders can manipulate properties such as position, color and texture coordinates, but cannot create new vertices. The output of the vertex shader goes to the next stage in the pipeline, which is either a geometry shader if present, or the rasterizer. Vertex shaders can enable powerful control over the details of position, movement, lighting, and color in any scene involving 3D models.

Geometry shaders

Geometry shaders were introduced in Direct3D 10 and OpenGL 3.2; formerly available in OpenGL 2.0+ with the use of extensions. This type of shader can generate new graphics primitives, such as points, lines, and triangles, from those primitives that were sent to the beginning of the graphics pipeline.

Geometry shader programs are executed after vertex shaders. They take as input a whole primitive, possibly with adjacency information. For example, when operating on triangles, the three vertices are the geometry shader's input. The shader can then emit zero or more primitives, which are rasterized and their fragments ultimately passed to a pixel shader.

Typical uses of a geometry shader include point sprite generation, geometry tessellation, shadow volume extrusion, and single pass rendering to a cube map. A typical real-world example of the benefits of geometry shaders would be automatic mesh complexity modification. A series of line strips representing control points for a curve are passed to the geometry shader and depending on the complexity required the shader can automatically generate extra lines each of which provides a better approximation of a curve.

Tessellation shaders

As of OpenGL 4.0 and Direct3D 11, a new shader class called a tessellation shader has been added. It adds two new shader stages to the traditional model: tessellation control shaders (also known as hull shaders) and tessellation evaluation shaders (also known as Domain Shaders), which together allow for simpler meshes to be subdivided into finer meshes at run-time according to a mathematical function. The function can be related to a variety of variables, most notably the distance from the viewing camera to allow active level-of-detail scaling. This allows objects close to the camera to have fine detail, while further away ones can have more coarse meshes, yet seem comparable in quality. It also can drastically reduce required mesh bandwidth by allowing meshes to be refined once inside the shader units instead of downsampling very complex ones from memory. Some algorithms can upsample any arbitrary mesh, while others allow for "hinting" in meshes to dictate the most characteristic vertices and edges.

Primitive and Mesh shaders

Circa 2017, the AMD Vega microarchitecture added support for a new shader stage—primitive shaders—somewhat akin to compute shaders with access to the data necessary to process geometry.

Nvidia introduced mesh and task shaders with its Turing microarchitecture in 2018 which are also modelled after compute shaders. Nvidia Turing is the world's first GPU microarchitecture that supports mesh shading through DirectX 12 Ultimate API, several months before Ampere RTX 30 series was released.

In 2020, AMD and Nvidia released RDNA 2 and Ampere microarchitectures which both support mesh shading through DirectX 12 Ultimate. These mesh shaders allow the GPU to handle more complex algorithms, offloading more work from the CPU to the GPU, and in algorithm intense rendering, increasing the frame rate of or number of triangles in a scene by an order of magnitude. Intel announced that Intel Arc Alchemist GPUs shipping in Q1 2022 will support mesh shaders.

Unified shaders

Unified shader is the combination of 2D shader and 3D shader. NVIDIA called "unified shaders" as "CUDA cores"; AMD called this as "shader cores"; while Intel called this as "ALU cores".

Compute shaders

Compute shaders are not limited to graphics applications, but use the same execution resources for GPGPU. They may be used in graphics pipelines e.g. for additional stages in animation or lighting algorithms (e.g. tiled forward rendering). Some rendering APIs allow compute shaders to easily share data resources with the graphics pipeline.

Ray tracing shaders

Ray tracing shaders are supported by Microsoft via DirectX Raytracing, by Khronos Group via Vulkan, GLSL, and SPIR-V, by Apple via Metal. NVIDIA and AMD called "ray tracing shaders" as "ray tracing cores". Unlike unified shader, one ray tracing shader can contain multiple ALUs.

Tensor shaders

Tensor shaders may be integrated in NPUs or GPUs. Tensor shaders are supported by Microsoft via DirectML, by Khronos Group via OpenVX, by Apple via Core ML, by Google via TensorFlow, by Linux Foundation via ONNX. NVIDIA and AMD called "tensor shaders" as "tensor cores". Unlike unified shader, one tensor shader can contains multiple ALUs.

Parallel processing

Shaders are written to apply transformations to a large set of elements at a time, for example, to each pixel in an area of the screen, or for every vertex of a model. This is well suited to parallel processing, and most modern GPUs have multiple shader pipelines to facilitate this, vastly improving computation throughput.

A programming model with shaders is similar to a higher order function for rendering, taking the shaders as arguments, and providing a specific dataflow between intermediate results, enabling both data parallelism (across pixels, vertices etc.) and pipeline parallelism (between stages). (see also map reduce).

Programming

The language in which shaders are programmed depends on the target environment. The official OpenGL and OpenGL ES shading language is OpenGL Shading Language, also known as GLSL, and the official Direct3D shading language is High Level Shader Language, also known as HLSL. Cg, a third-party shading language which outputs both OpenGL and Direct3D shaders, was developed by Nvidia; however since 2012 it has been deprecated. Apple released its own shading language called Metal Shading Language as part of the Metal framework.

GUI shader editors

Modern video game development platforms such as Unity, Unreal Engine and Godot increasingly include node-based editors that can create shaders without the need for actual code; the user is instead presented with a directed graph of connected nodes that allow users to direct various textures, maps, and mathematical functions into output values like the diffuse color, the specular color and intensity, roughness/metalness, height, normal, and so on. Automatic compilation then turns the graph into an actual, compiled shader.

See also

* GLSL
* SPIR-V
* HLSL
* Compute kernel
* Shading language
* GPGPU
* List of common shading algorithms
* Vector processor

References

Further reading

*
*
*
*

External links

OpenGL geometry shader extension

Riemer's DirectX & HLSL Tutorial
''HLSL Tutorial using DirectX with much sample code''

Pipeline Stages (Direct3D 10)

{{Computer graphics

Shading