Torch is an
open-source
Open source is source code that is made freely available for possible modification and redistribution. Products include permission to use the source code, design documents, or content of the product. The open-source model is a decentralized sof ...
machine learning
Machine learning (ML) is a field of inquiry devoted to understanding and building methods that 'learn', that is, methods that leverage data to improve performance on some set of tasks. It is seen as a part of artificial intelligence.
Machine ...
library,
a
scientific computing
Computational science, also known as scientific computing or scientific computation (SC), is a field in mathematics that uses advanced computing capabilities to understand and solve complex problems. It is an area of science that spans many disc ...
framework, and a
script language
A scripting language or script language is a programming language that is used to manipulate, customize, and automate the facilities of an existing system. Scripting languages are usually interpreted at runtime rather than compiled.
A scripting ...
based on the
Lua
Lua or LUA may refer to:
Science and technology
* Lua (programming language)
* Latvia University of Agriculture
* Last universal ancestor, in evolution
Ethnicity and language
* Lua people, of Laos
* Lawa people, of Thailand sometimes referred t ...
programming language.
It provides a wide range of algorithms for
deep learning
Deep learning (also known as deep structured learning) is part of a broader family of machine learning methods based on artificial neural networks with representation learning. Learning can be supervised, semi-supervised or unsupervised.
De ...
, and uses the scripting language
LuaJIT
LuaJIT is a tracing just in time compiler for the Lua programming language.
History
The LuaJIT project was started in 2005 by developer Mike Pall, released under the MIT open source license.
The second major release of the compiler, 2.0.0, ...
, and an underlying
C implementation. It was created at
IDIAP at
EPFL. As of 2018, Torch is no longer in active development. However
PyTorch
PyTorch is a machine learning framework based on the Torch library, used for applications such as computer vision and natural language processing, originally developed by Meta AI and now part of the Linux Foundation umbrella. It is free and open ...
, which is based on the Torch library, is actively developed as of October 2022.
torch
The core package of Torch is
torch
. It provides a flexible N-dimensional array or Tensor, which supports basic routines for indexing, slicing, transposing, type-casting, resizing, sharing storage and cloning. This object is used by most other packages and thus forms the core object of the library. The Tensor also supports mathematical operations like
max
,
min
,
sum
, statistical distributions like
uniform
A uniform is a variety of clothing worn by members of an organization while participating in that organization's activity. Modern uniforms are most often worn by armed forces and paramilitary organizations such as police, emergency services, se ...
,
normal Normal(s) or The Normal(s) may refer to:
Film and television
* ''Normal'' (2003 film), starring Jessica Lange and Tom Wilkinson
* ''Normal'' (2007 film), starring Carrie-Anne Moss, Kevin Zegers, Callum Keith Rennie, and Andrew Airlie
* ''Norma ...
and
multinomial, and
BLAS
Basic Linear Algebra Subprograms (BLAS) is a specification that prescribes a set of low-level routines for performing common linear algebra operations such as vector addition, scalar multiplication, dot products, linear combinations, and matrix ...
operations like
dot product
In mathematics, the dot product or scalar productThe term ''scalar product'' means literally "product with a scalar as a result". It is also used sometimes for other symmetric bilinear forms, for example in a pseudo-Euclidean space. is an algebra ...
,
matrix–vector multiplication
In mathematics, particularly in linear algebra, matrix multiplication is a binary operation that produces a matrix (mathematics), matrix from two matrices. For matrix multiplication, the number of columns in the first matrix must be equal to the n ...
,
matrix–matrix multiplication and
matrix product
In mathematics, particularly in linear algebra, matrix multiplication is a binary operation that produces a matrix from two matrices. For matrix multiplication, the number of columns in the first matrix must be equal to the number of rows in the s ...
.
The following exemplifies using torch via its
REPL interpreter:
> a = torch.randn(3,4)
> =a
-0.2381 -0.3401 -1.7844 -0.2615
0.1411 1.6249 0.1708 0.8299
-1.0434 2.2291 1.0525 0.8465
orch.DoubleTensor of dimension 3x4
> a 2]
-0.34010116549482
> a:narrow(1,1,2)
-0.2381 -0.3401 -1.7844 -0.2615
0.1411 1.6249 0.1708 0.8299
orch.DoubleTensor of dimension 2x4
> a:index(1, torch.LongTensor)
-0.2381 -0.3401 -1.7844 -0.2615
0.1411 1.6249 0.1708 0.8299
orch.DoubleTensor of dimension 2x4
> a:min()
-1.7844365427828
The
torch
package also simplifies
object-oriented programming
Object-oriented programming (OOP) is a programming paradigm based on the concept of "objects", which can contain data and code. The data is in the form of fields (often known as attributes or ''properties''), and the code is in the form of pr ...
and
serialization
In computing, serialization (or serialisation) is the process of translating a data structure or object state into a format that can be stored (e.g. files in secondary storage devices, data buffers in primary storage devices) or transmitted (e ...
by providing various convenience functions which are used throughout its packages. The
torch.class(classname, parentclass)
function can be used to create
object factories (
classes). When the
constructor is called, torch initializes and sets a Lua
table
Table may refer to:
* Table (furniture), a piece of furniture with a flat surface and one or more legs
* Table (landform), a flat area of land
* Table (information), a data arrangement with rows and columns
* Table (database), how the table data ...
with the user-defined
metatable {{For, the object prototype mechanism in Lua, Lua (programming language)#Object-oriented programming
A metatable is the section of a database or other data holding structure that is designated to hold data that will act as source code or metadata. ...
, which makes the table an
object
Object may refer to:
General meanings
* Object (philosophy), a thing, being, or concept
** Object (abstract), an object which does not exist at any particular time or place
** Physical object, an identifiable collection of matter
* Goal, an ...
.
Objects created with the torch factory can also be serialized, as long as they do not contain references to objects that cannot be serialized, such as Lua
coroutine
Coroutines are computer program components that generalize subroutines for non-preemptive multitasking, by allowing execution to be suspended and resumed. Coroutines are well-suited for implementing familiar program components such as cooperative ...
s, and Lua ''userdata''. However, ''userdata'' can be serialized if it is wrapped by a table (or metatable) that provides
read()
and
write()
methods.
nn
The
nn
package is used for building
neural network
A neural network is a network or circuit of biological neurons, or, in a modern sense, an artificial neural network, composed of artificial neurons or nodes. Thus, a neural network is either a biological neural network, made up of biological ...
s. It is divided into modular objects that share a common
Module
interface. Modules have a
forward()
and
backward()
method that allow them to
feedforward and
backpropagate, respectively. Modules can be joined using module
composites, like
Sequential
,
Parallel
and
Concat
to create complex task-tailored graphs. Simpler modules like
Linear
,
Tanh
and
Max
make up the basic component modules. This modular interface provides first-order
automatic gradient differentiation. What follows is an example use-case for building a
multilayer perceptron
A multilayer perceptron (MLP) is a fully connected class of feedforward artificial neural network (ANN). The term MLP is used ambiguously, sometimes loosely to mean ''any'' feedforward ANN, sometimes strictly to refer to networks composed of mu ...
using Modules:
> mlp = nn.Sequential()
> mlp:add( nn.Linear(10, 25) ) -- 10 input, 25 hidden units
> mlp:add( nn.Tanh() ) -- some hyperbolic tangent transfer function
> mlp:add( nn.Linear(25, 1) ) -- 1 output
> =mlp:forward(torch.randn(10))
-0.1815
orch.Tensor of dimension 1
Loss function
In mathematical optimization and decision theory, a loss function or cost function (sometimes also called an error function) is a function that maps an event or values of one or more variables onto a real number intuitively representing some "cost ...
s are implemented as sub-classes of
Criterion
, which has a similar interface to
Module
. It also has
forward()
and
backward()
methods for computing the loss and backpropagating gradients, respectively. Criteria are helpful to train neural network on classical tasks. Common criteria are the
Mean Squared Error
In statistics, the mean squared error (MSE) or mean squared deviation (MSD) of an estimator (of a procedure for estimating an unobserved quantity) measures the average of the squares of the errors—that is, the average squared difference between ...
criterion implemented in
MSECriterion
and the
cross-entropy
In information theory, the cross-entropy between two probability distributions p and q over the same underlying set of events measures the average number of bits needed to identify an event drawn from the set if a coding scheme used for the set i ...
criterion implemented in
ClassNLLCriterion
. What follows is an example of a Lua function that can be iteratively called to train
an
mlp
Module on input Tensor
x
, target Tensor
y
with a scalar
learningRate
:
function gradUpdate(mlp, x, y, learningRate)
local criterion = nn.ClassNLLCriterion()
pred = mlp:forward(x)
local err = criterion:forward(pred, y);
mlp:zeroGradParameters();
local t = criterion:backward(pred, y);
mlp:backward(x, t);
mlp:updateParameters(learningRate);
end
It also has
StochasticGradient
class for training a neural network using
Stochastic gradient descent
Stochastic gradient descent (often abbreviated SGD) is an iterative method for optimizing an objective function with suitable smoothness properties (e.g. differentiable or subdifferentiable). It can be regarded as a stochastic approximation of ...
, although the
optim
package provides much more options in this respect, like momentum and weight decay
regularization
Regularization may refer to:
* Regularization (linguistics)
* Regularization (mathematics)
* Regularization (physics)
In physics, especially quantum field theory, regularization is a method of modifying observables which have singularities in ...
.
Other packages
Many packages other than the above official packages are used with Torch. These are listed in the torch cheatsheet. These extra packages provide a wide range of utilities such as parallelism, asynchronous input/output, image processing, and so on. They can be installed with
LuaRocks
LuaRocks is a package manager for the Lua programming language that provides a standard format for distributing Lua modules (in a self-contained format called a "rock"), a tool designed to easily manage the installation of rocks, and a server fo ...
, the Lua package manager which is also included with the Torch distribution.
Applications
Torch is used by the Facebook
AI Research Group,
IBM,
Yandex
Yandex LLC (russian: link=no, Яндекс, p=ˈjandəks) is a Russian multinational technology company providing Internet-related products and services, including an Internet search engine, information services, e-commerce, transportation, maps ...
and the
Idiap Research Institute. Torch has been extended for use on
Android and
iOS
iOS (formerly iPhone OS) is a mobile operating system created and developed by Apple Inc. exclusively for its hardware. It is the operating system that powers many of the company's mobile devices, including the iPhone; the term also includes ...
. It has been used to build hardware implementations for data flows like those found in neural networks.
Facebook has released a set of extension modules as open source software.
See also
*
Comparison of deep learning software
The following table compares notable software frameworks, libraries and computer programs for deep learning.
Deep-learning software by name
Comparison of compatibility of machine learning models
See also
*Comparison of numerical-anal ...
*
PyTorch
PyTorch is a machine learning framework based on the Torch library, used for applications such as computer vision and natural language processing, originally developed by Meta AI and now part of the Linux Foundation umbrella. It is free and open ...
References
External links
*
{{Lua programming language
Deep learning software
Free statistical software
Lua (programming language) software
Software using the BSD license