Video synopsis is a method for automatically synthesizing a short, informative summary of a video. Unlike traditional

video summarization Automatic summarization is the process of shortening a set of data computationally, to create a subset (a Abstract (summary), summary) that represents the most important or relevant information within the original content. Artificial intelligence ...

, the synopsis is not just composed of frames from the original video. The algorithm detects, tracks and analyzes moving objects (also called events) in a database of objects and activities. The final output is a new, short video clip in which objects and activities that originally occurred at different times are displayed simultaneously, so as to convey information in the shortest possible time. Video synopsis has specific applications in the field of

video analytics Video content analysis or video content analytics (VCA), also known as video analysis or video analytics (VA), is the capability of automatically analyzing video to detect and determine temporal and spatial events. This technical capability is used ...

and

video surveillance Closed-circuit television (CCTV), also known as video surveillance, is the use of video cameras to transmit a signal to a specific place, on a limited set of monitors. It differs from broadcast television in that the signal is not openly tr ...

where, despite technological advancements and increased growth in the deployment of CCTV (closed circuit television) cameras, viewing and analysis of recorded footage is still a costly labor-intensive and time-intensive task.

Technology overview

Video synopsis combines a visual summary of stored video together with an indexing mechanism. When a summary is required, all objects from the target period are collected and shifted in time to create a much shorter synopsis video showing maximum activity. A synopsis

video clip Video clips refer to mostly short videos, most of the time called memes, which are short videos of silly jokes and funny clips, most of the time coming from movies or any entertainment videos such as YouTube. The term is also used more loosely to ...

is generated, in real time, in which objects and activities that originally occurred in different times are displayed simultaneously. The process begins by detecting and

tracking Tracking may refer to: Science and technology Computing * Tracking, in computer graphics, in match moving (insertion of graphics into footage) * Tracking, composing music with music tracker software * Eye tracking, measuring the position of t ...

objects of interest. Each object is represented as a tube in space-time of all video frames. Objects are detected and stored in a database in approximately real time. Following a request to summarize a time period, all objects from the desired time are extracted from the database, and indexed to create a much shorter summary video containing maximum activity. Real time rendering is used to generate the summary video after object re-timing. This allows

end-user In product development, an end user (sometimes end-user) is a person who ultimately uses or is intended to ultimately use a product. The end user stands in contrast to users who support or maintain the product, such as sysops, system administrat ...

control over object/event density. Video synopsis technology was invented by Prof. Shmuel Peleg of

The Hebrew University The Hebrew University of Jerusalem (HUJI; he, הַאוּנִיבֶרְסִיטָה הַעִבְרִית בִּירוּשָׁלַיִם) is a public research university based in Jerusalem, Israel. Co-founded by Albert Einstein and Dr. Chaim Weiz ...

Jerusalem, Israel Jerusalem (; he, יְרוּשָׁלַיִם ; ar, القُدس ) (combining the Biblical and common usage Arabic names); grc, Ἱερουσαλήμ/Ἰεροσόλυμα, Hierousalḗm/Hierosóluma; hy, Երուսաղեմ, Erusałēm. i ...

, and is being developed under commercial license by BriefCam, Ltd. BriefCam received a license to use the technology from Yissum which is the owner of the patents registered for the technology. In May 2018, BriefCam was acquired by Japanese digital imaging giant,

Canon Inc. is a Japanese multinational corporation headquartered in Ōta, Tokyo, Japan, specializing in optical, imaging, and industrial products, such as lenses, cameras, medical equipment, scanners, printers, and semiconductor manufacturing equipment.

, for an estimated $90 million. Investors in the company include Motorola Solutions Venture Capital, Aviv Venture Capital, and

OurCrowd OurCrowd is an online global venture investing platform that empowers institutions and individual accredited investors to invest and engage in emerging technology companies at an early stage while still privately held. Based in Jerusalem, the co ...

Video Synopsis before+after (horizontal)

Recent advances

Recent advances in the field of video synopsis have resulted in methods that focus in collecting key-points(or frames) from the long uncut video and presenting them as a chain of key events that summarize the video. This is only one of the many methods employed in modern literature to perform this task. Recently, these event-driven methods have focused on correlating objects in frames, but in a more semantically related way that has been called a story-driven method of summarizing video. These methods have been shown to work well for

egocentric Egocentrism is the inability to differentiate between self and other. More specifically, it is the inability to accurately assume or understand any perspective other than one's own. Egocentrism is found across the life span: in infancy, early chi ...

Zheng Lu, Kristen Grauma

Story-driven summarization for egocentric video. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition - 2013. settings where the video is basically a point-of-view perspective of a single person or a group of people.

References

{{Reflist

Patents

US Patent 8,311,277
- Method and System for Video Indexing and Video Synopsis. Als
Australian Patent AU2007345938.
Filed in 9 additional countries
US Patent 8,102,406
- Method and System for Producing a Video Synopsis. Als
European Patent EP1955205
an
Japanese Patent JP4972095
Filed in 7 additional countries
US Patent 7,852,370
- Method and System for Spatio-Temporal Video Warping. Als
Israeli Patent IL182006
Filed in Europe: PCT/IL2005/001150 * The owner of the patents are Yissum which gave a License to BriefCam to use the technology.

External links

Video Synopsis and Indexing project
Further information, published papers and video clips from the School of Computer Science, The Hebrew University of Jerusalem
BriefCam company website
Includes demonstration video on using video synopsis technology.
Yissum's Website

Video Synopsis paper in Hebrew
- 2008

Demo video for paper: Story-Driven Summarization for Egocentric Video - Zheng Lu and Kristen Grauman - CVPR 2013 Image processing Israeli inventions Video surveillance

Technology overview

Recent advances

See also

References

Patents

External links