DeepDream
   HOME

TheInfoList



OR:

DeepDream is a
computer vision Computer vision is an interdisciplinary scientific field that deals with how computers can gain high-level understanding from digital images or videos. From the perspective of engineering, it seeks to understand and automate tasks that the hum ...
program created by
Google Google LLC () is an American multinational technology company focusing on search engine technology, online advertising, cloud computing, computer software, quantum computing, e-commerce, artificial intelligence, and consumer electronics. ...
engineer Alexander Mordvintsev that uses a
convolutional neural network In deep learning, a convolutional neural network (CNN, or ConvNet) is a class of artificial neural network (ANN), most commonly applied to analyze visual imagery. CNNs are also known as Shift Invariant or Space Invariant Artificial Neural Netwo ...
to find and enhance patterns in
image An image is a visual representation of something. It can be two-dimensional, three-dimensional, or somehow otherwise feed into the visual system to convey information. An image can be an artifact, such as a photograph or other two-dimensiona ...
s via
algorithm In mathematics and computer science, an algorithm () is a finite sequence of rigorous instructions, typically used to solve a class of specific Computational problem, problems or to perform a computation. Algorithms are used as specificat ...
ic
pareidolia Pareidolia (; ) is the tendency for perception to impose a meaningful interpretation on a nebulous stimulus, usually visual, so that one sees an object, pattern, or meaning where there is none. Common examples are perceived images of animals, ...
, thus creating a dream-like appearance reminiscent of a
psychedelic experience A psychedelic experience (known colloquially as a trip) is a temporary altered state of consciousness induced by the consumption of a psychedelic substance (most commonly LSD, mescaline, psilocybin mushrooms, or DMT). For example, an acid tr ...
in the deliberately overprocessed images. Google's program popularized the term (deep) "dreaming" to refer to the generation of images that produce desired
activation Activation, in chemistry and biology, is the process whereby something is prepared or excited for a subsequent reaction. Chemistry In chemistry, "activation" refers to the reversible transition of a molecule into a nearly identical chemical or ...
s in a trained deep network, and the term now refers to a collection of related approaches.


History

The DeepDream software, originated in a deep convolutional network codenamed "Inception" after the film of the same name, was developed for the ImageNet Large-Scale Visual Recognition Challenge (ILSVRC) in 2014 and released in July 2015. The dreaming idea and name became popular on the internet in 2015 thanks to Google's DeepDream program. The idea dates from early in the history of neural networks, and similar methods have been used to synthesize visual textures. Related visualization ideas were developed (prior to Google's work) by several research groups. After Google published their techniques and made their code
open-source Open source is source code that is made freely available for possible modification and redistribution. Products include permission to use the source code, design documents, or content of the product. The open-source model is a decentralized sof ...
, a number of tools in the form of web services, mobile applications, and desktop software appeared on the market to enable users to transform their own photos.


Process

The software is designed to detect faces and other patterns in images, with the aim of automatically classifying images. However, once trained, the network can also be run in reverse, being asked to adjust the original image slightly so that a given output neuron (e.g. the one for faces or certain animals) yields a higher confidence score. This can be used for visualizations to understand the emergent structure of the neural network better, and is the basis for the DeepDream concept. This reversal procedure is never perfectly clear and unambiguous because it utilizes a one-to-many mapping process. However, after enough reiterations, even imagery initially devoid of the sought features will be adjusted enough that a form of
pareidolia Pareidolia (; ) is the tendency for perception to impose a meaningful interpretation on a nebulous stimulus, usually visual, so that one sees an object, pattern, or meaning where there is none. Common examples are perceived images of animals, ...
results, by which
psychedelic Psychedelics are a subclass of hallucinogenic drugs whose primary effect is to trigger non-ordinary states of consciousness (known as psychedelic experiences or "trips").Pollan, Michael (2018). ''How to Change Your Mind: What the New Science of ...
and
surreal Surreal may refer to: *Anything related to or characteristic of Surrealism, a movement in philosophy and art * "Surreal" (song), a 2000 song by Ayumi Hamasaki * ''Surreal'' (album), an album by Man Raze *Surreal humour, a common aspect of humor ...
images are generated algorithmically. The optimization resembles backpropagation, however instead of adjusting the network weights, the weights are held fixed and the input is adjusted. For example, an existing image can be altered so that it is "more cat-like", and the resulting enhanced image can be again input to the procedure. This usage resembles the activity of looking for animals or other patterns in clouds. Applying gradient descent independently to each pixel of the input produces images in which adjacent pixels have little relation and thus the image has too much high frequency information. The generated images can be greatly improved by including a prior or regularizer that prefers inputs that have natural image statistics (without a preference for any particular image), or are simply smooth. For example, Mahendran et al. used the total variation regularizer that prefers images that are piecewise constant. Various regularizers are discussed further in. An in-depth, visual exploration of feature visualization and regularization techniques was published more recently. The cited resemblance of the imagery to
LSD Lysergic acid diethylamide (LSD), also known colloquially as acid, is a potent psychedelic drug. Effects typically include intensified thoughts, emotions, and sensory perception. At sufficiently high dosages LSD manifests primarily mental, vi ...
- and
psilocybin Psilocybin ( , ) is a naturally occurring psychedelic prodrug compound produced by more than 200 species of fungi. The most potent are members of the genus ''Psilocybe'', such as '' P. azurescens'', '' P. semilanceata'', and '' P.&nbs ...
-induced hallucinations is suggestive of a functional resemblance between artificial neural networks and particular layers of the visual cortex. Neural networks such as DeepDream have biological analogies providing insight into brain processing and the formation of consciousness. Hallucinogens such as DMT alter the function of the serotonergic system which is present within the layers of the visual cortex. Neural networks are trained on input vectors and are altered by internal variations during the training process. The input and internal modifications represent the processing of exogenous and endogenous signals respectively in the visual cortex. As internal variations are modified in deep neural networks the output image reflect these changes. This specific manipulation demonstrates how inner brain mechanisms are analogous to internal layers of neural networks. Internal noise level modifications represent how hallucinogens omit external sensory information leading internal preconceived conceptions to strongly influence visual perception.


Usage

The dreaming idea can be applied to hidden (internal) neurons other than those in the output, which allows exploration of the roles and representations of various parts of the network. It is also possible to optimize the input to satisfy either a single neuron (this usage is sometimes called Activity Maximization) or an entire layer of neurons. While dreaming is most often used for visualizing networks or producing computer art, it has recently been proposed that adding "dreamed" inputs to the training set can improve training times for abstractions in Computer Science. The DeepDream model has also been demonstrated to have application in the field of
art history Art history is the study of aesthetic objects and visual expression in historical and stylistic context. Traditionally, the discipline of art history emphasized painting, drawing, sculpture, architecture, ceramics and decorative arts; yet today ...
. DeepDream was used for
Foster the People Foster the People is an American indie pop band formed in Los Angeles, California, in 2009. Its members are singer Mark Foster, guitarist Sean Cimino, and keyboardist Isom Innis. Foster founded the band in 2009 after spending several years in ...
's music video for the song "Doing It for the Money". In 2017, a research group out of the University of Sussex created a ''Hallucination Machine'', applying the DeepDream algorithm to a pre-recorded panoramic video, allowing users to explore virtual reality environments to mimic the experience of psychoactive substances and/or psychopathological conditions. They were able to demonstrate that the subjective experiences induced by the Hallucination Machine differed significantly from control (non-‘hallucinogenic’) videos, while bearing phenomenological similarities to the psychedelic state (following administration of psilocybin). In 2021, a study published in the journal ''Entropy'' demonstrated the similarity between DeepDream and actual psychedelic experience with
neuroscientific Neuroscience is the scientific study of the nervous system (the brain, spinal cord, and peripheral nervous system), its functions and disorders. It is a multidisciplinary science that combines physiology, anatomy, molecular biology, developmen ...
evidence. The authors recorded Electroencephalography (
EEG Electroencephalography (EEG) is a method to record an electrogram of the spontaneous electrical activity of the brain. The biosignals detected by EEG have been shown to represent the postsynaptic potentials of pyramidal neurons in the neocortex ...
) of human participants during passive vision of a movie clip and its DeepDream-generated counterpart. They found that DeepDream video triggered a higher
entropy Entropy is a scientific concept, as well as a measurable physical property, that is most commonly associated with a state of disorder, randomness, or uncertainty. The term and the concept are used in diverse fields, from classical thermodynam ...
in the EEG signal and a higher level of
functional connectivity Resting state fMRI (rs-fMRI or R-fMRI) is a method of functional magnetic resonance imaging (fMRI) that is used in brain mapping to evaluate regional interactions that occur in a resting or task-negative state, when an explicit task is not being ...
between brain areas, both well-known biomarkers of actual psychedelic experience. In 2022, a research group coordinated by the University of Trento "measure participants’ cognitive flexibility and creativity after the exposure to virtual reality panoramic videos and their hallucinatory-like counterparts generated by the DeepDream algorithm ... following the simulated psychedelic exposure, individuals exhibited ... an attenuated contribution of the automatic process and chaotic dynamics underlying their decision processes, presumably due to a reorganization in the cognitive dynamics that facilitates the exploration of uncommon decision strategies and inhibits automated choices."


See also

*
Artificial imagination Artificial imagination, also called synthetic imagination or machine imagination, is defined as the artificial simulation of human imagination by general or special purpose computers or artificial neural networks. The applied form of it is known as ...
* DALL-E * Feature detection (computer vision) *
Hallucination (artificial intelligence) In artificial intelligence, a hallucination or artificial hallucination is a confident response by an artificial intelligence that does not seem to be justified by its training data when the model has a tendency of "hallucinating" deceptive data. ...
* Neural Style Transfer *
Procedural textures In computer graphics, a procedural texture is a texture created using a mathematical description (i.e. an algorithm) rather than directly stored data. The advantage of this approach is low storage cost, unlimited texture resolution and easy textur ...
*
Psychedelic art Psychedelic art (also known as psychedelia) is art, graphics or visual displays related to or inspired by psychedelic experiences and hallucinations known to follow the ingestion of psychedelic drugs such as LSD, psilocybin, and DMT. The word "p ...
*
Texture synthesis Texture synthesis is the process of algorithmically constructing a large digital image from a small digital sample image by taking advantage of its structural content. It is an object of research in computer graphics and is used in many fields, amo ...


References


External links

* * {{cite web, url= https://research.googleblog.com/2015/06/inceptionism-going-deeper-into-neural.html , title= Inceptionism: Going Deeper into Neural Networks , date= June 17, 2015 , first1= Alexander , last1= Mordvintsev , first2= Christopher , last2= Olah , first3= Mike , last3= Tyka , url-status= live , archive-url= https://web.archive.org/web/20150703064823/http://googleresearch.blogspot.co.uk/2015/06/inceptionism-going-deeper-into-neural.html , archive-date= 2015-07-03 *https://www.youtube.com/watch?v=BsSmBPmPeYQ 2015 software Algorithmic art Deep learning software applications Computer art Computer vision software Free software Google software Object recognition and categorization Psychedelic art