FreeTrack is a general-purpose

optical motion tracking Motion capture (sometimes referred as mo-cap or mocap, for short) is the process of recording the movement of objects or people. It is used in military, entertainment, sports, medical applications, and for validation of computer vision and robo ...

application for

Microsoft Windows Windows is a group of several proprietary graphical operating system families developed and marketed by Microsoft. Each family caters to a certain sector of the computing industry. For example, Windows NT for consumers, Windows Server for serv ...

, released under the

GNU General Public License The GNU General Public License (GNU GPL or simply GPL) is a series of widely used free software licenses that guarantee end users the Four Freedoms (Free software), four freedoms to run, study, share, and modify the software. The license was th ...

, that can be used with common inexpensive cameras. Its primary focus is head tracking with uses in

virtual reality Virtual reality (VR) is a simulated experience that employs pose tracking and 3D near-eye displays to give the user an immersive feel of a virtual world. Applications of virtual reality include entertainment (particularly video games), educ ...

simulation A simulation is the imitation of the operation of a real-world process or system over time. Simulations require the use of Conceptual model, models; the model represents the key characteristics or behaviors of the selected system or proc ...

video games Video games, also known as computer games, are electronic games that involves interaction with a user interface or input device such as a joystick, game controller, controller, computer keyboard, keyboard, or motion sensing device to gener ...

3D modeling In 3D computer graphics, 3D modeling is the process of developing a mathematical coordinate-based representation of any surface of an object (inanimate or living) in three dimensions via specialized software by manipulating edges, vertices, an ...

computer aided design Computer-aided design (CAD) is the use of computers (or ) to aid in the creation, modification, analysis, or optimization of a design. This software is used to increase the productivity of the designer, improve the quality of design, improve co ...

and general

hands-free computing Hands-free computing is any computer configuration where a user can interface without the use of their hands, an otherwise common requirement of human interface devices such as the mouse and keyboard. Hands-free computing is important because it ...

to improve

computer accessibility Computer accessibility (also known as accessible computing) refers to the accessibility of a computer system to all people, regardless of disability type or severity of impairment. The term ''accessibility'' is most often used in reference to sp ...

. Tracking can be made sensitive enough that only small head movements are required so that the user's eyes never leave the screen. A camera is positioned to observe a rigid point model worn by the user, the points of which need to be isolated from background light by means of physical and software filtering. Motion is tracked with up to

six degrees of freedom Six degrees of freedom (6DOF) refers to the six mechanical degrees of freedom of movement of a rigid body in three-dimensional space. Specifically, the body is free to change position as forward/backward (surge), up/down (heave), left/right ...

(6DOF): yaw, pitch, roll, left/right, up/down and forward/back. Windows-compatible video devices like

webcams A webcam is a video camera which is designed to record or stream to a computer or computer network. They are primarily used in videotelephony, livestreaming and social media, and security. Webcams can be built-in computer hardware or peripheral de ...

are supported, as well as special

Nintendo is a Japanese Multinational corporation, multinational video game company headquartered in Kyoto, Japan. It develops video games and video game consoles. Nintendo was founded in 1889 as by craftsman Fusajiro Yamauchi and originally produce ...

Wii Remote The Wii Remote, also known colloquially as the Wiimote, is the primary game controller for Nintendo's Wii home video game console. An essential capability of the Wii Remote is its motion sensing capability, which allows the user to interact with ...

camera, iPhone Truedepth camera with Eyeware Beam, and NaturalPoint cameras (

TrackIR TrackIR is an optical motion tracking game controller for Microsoft Windows created by NaturalPoint Inc.. TrackIR tracks head motions with up to six degrees of freedom (6DOF) in later models, allowing handsfree view control for improved game im ...

, SmartNav and OptiTrack). FreeTrack can output head-tracking data to programs directly using its own open interface, as well as

, SimConnect and FSUIPC interfaces. Programs that support these interfaces are regarded as being FreeTrack-compatible. FreeTrack can also emulate

mouse A mouse ( : mice) is a small rodent. Characteristically, mice are known to have a pointed snout, small rounded ears, a body-length scaly tail, and a high breeding rate. The best known mouse species is the common house mouse (''Mus musculus' ...

keyboard Keyboard may refer to: Text input * Keyboard, part of a typewriter * Computer keyboard ** Keyboard layout, the software control of computer keyboards and their mapping ** Keyboard technology, computer keyboard hardware and firmware Music * Musi ...

, and

joystick A joystick, sometimes called a flight stick, is an input device consisting of a stick that pivots on a base and reports its angle or direction to the device it is controlling. A joystick, also known as the control column, is the principal cont ...

(via PPJoy) if a program does not support a direct interface. FreeTrack is coded in Delphi 7 and uses

DirectShow DirectShow (sometimes abbreviated as DS or DShow), codename Quartz, is a multimedia framework and API produced by Microsoft for software developers to perform various operations with media files or streams. It is the replacement for Microsoft's ear ...

and

DirectX Microsoft DirectX is a collection of application programming interfaces (APIs) for handling tasks related to multimedia, especially game programming and video, on Microsoft platforms. Originally, the names of these APIs all began with "Direct", ...

. Head tracking is achieved using implementations of DeMenthon's four-point iterative

pose Human positions refer to the different physical configurations that the human body can take. There are several synonyms that refer to human positioning, often used interchangeably, but having specific nuances of meaning. *''Position'' is a gen ...

estimation algorithm (POSIT) and Alter's three point geometric algorithm.

Software

FreeTrack uses a camera to collect real-time information on the point model worn by the user. Specifically the image coordinates of the model points, which are either received directly from the camera or extracted from a video stream. These coordinates are used to generate an estimate of the real head pose, which can be transformed by the user in a number of ways to create a virtual pose. One of the most fundamental transformations involves amplifying rotation so that only small head movements are required. Finally, the virtual pose is sent to the user's choice of outputs. This is all done in the background, with tracking status displayed in the

system tray A taskbar is an element of a graphical user interface which has various purposes. It typically shows which programs are currently running. The specific design and layout of the taskbar varies between individual operating systems, but generally a ...

. A 3D preview is available that shows the virtual head position and orientation for a given real head pose and can be viewed from multiple perspectives, including first-person. This greatly assists with testing and makes it easier to experiment with different settings. Each

degree of freedom Degrees of freedom (often abbreviated df or DOF) refers to the number of independent variables or parameters of a thermodynamic system. In various scientific fields, the word "freedom" is used to describe the limits to which physical movement or ...

(axis) has a response curve that can be modified to change the way the virtual head moves for a given real head movement. This is commonly used to create a central deadzone region so that the user’s head can be more relaxed there. Keyboard, mouse and joystick buttons can be used to toggle tracking settings, including the virtual centre location (like adjusting the seat position in a car) and individually toggle axes and outputs. For NaturalPoint cameras, FreeTrack can provide advanced features and a level of customization that is not available with official software.

Camera

Resolution

In most cases a resolution of 320×240 is sufficient, this is capable of producing a much higher

sub-pixel resolution In digital image processing, sub-pixel resolution can be obtained in images constructed from sources with information exceeding the nominal Image resolution, pixel resolution of said images. Aliasing When an object with a certain resolution ...

, enough to allow accurate cursor control on a high-resolution monitor. Resolutions 640×480 and above have diminishing returns and correspond to an exponential increase in CPU usage when not sufficiently compressed before reaching the computer. Higher resolutions become more important at greater distances from the camera. The Wii utilizes a low-resolution 128×96 sensor, which is found by some to produce jittery tracking and may require smoothing to improve stability at the cost of decreased responsiveness.

Sensor

For the same resolution,

monochrome A monochrome or monochromatic image, object or palette is composed of one color (or values of one color). Images using only shades of grey are called grayscale (typically digital) or black-and-white (typically analog). In physics, monochrom ...

sensors A sensor is a device that produces an output signal for the purpose of sensing a physical phenomenon. In the broadest definition, a sensor is a device, module, machine, or subsystem that detects events or changes in its environment and sends ...

can resolve finer details much better than color sensors due to the lack of a

color filter array In digital imaging, a color filter array (CFA), or color filter mosaic (CFM), is a mosaic of tiny color filters placed over the pixel sensors of an image sensor to capture color information. The term is also used in reference to e paper devices ...

Frame rate

FreeTrack uses

interpolation In the mathematical field of numerical analysis, interpolation is a type of estimation, a method of constructing (finding) new data points based on the range of a discrete set of known data points. In engineering and science, one often has a n ...

with low-frame-rate video devices to improve panning smoothness. However, responsiveness is fundamentally limited to the frame rate; a 30 frame/s webcam has a maximal response delay of 33.3 milliseconds compared with 8.33 milliseconds for a 120 frame/s camera. To put this into perspective, a human’s reaction time to visual stimulus (finger reflex) is typically around 200 ms; 30 ms can be regarded as a competitive

ping Ping may refer to: Arts and entertainment Fictional characters * Ping, a domesticated Chinese duck in the illustrated book '' The Story about Ping'', first published in 1933 * Ping, a minor character in ''Seinfeld'', an NBC sitcom * Ping, a c ...

in online reflex-based games, and an LCD monitor refresh rate is typically 17 ms. Higher responsiveness gives a greater feeling of control, but since virtual head motion is amplified, it can also cause it to move unrealistically fast. For this reason, some programs limit head movement speed, wasting some of the responsiveness of higher-frame-rate cameras.

Angle

A wider viewing angle allows a larger tracking region when in close proximity to the camera. At further distances a wide angle is not desirable, more of the frame is unused and the effective resolution drops more rapidly. More peripheral light can also be seen, which can interfere with tracking. Viewing angle can be reduced by using

digital zoom Digital zoom is a method of decreasing the precise angle of view of a digital photograph or video image. It is accomplished by cropping an image down to an area with the same aspect ratio as the original, and scaling the image up to the dimen ...

at the cost of resolution.

CPU usage

The Nintendo Wii Remote effectively uses no CPU, NaturalPoint cameras use a small amount, and general video devices can use a significant amount, depending on the brand and the specific camera settings in use. A PlayStation Eye running at the same resolution and frame rate as a TrackIR 4 would be very demanding on a single-core CPU. However, modern multi-core CPUs are making this less of an issue. Resolution and frame rate can always be reduced to conserve CPU resources.

Filters

FreeTrack requires the tracking points to be isolated from all other light; this is best done using infrared LEDs and a visible-light blocking filter in front of the camera.

Photographic film Photographic film is a strip or sheet of transparent film base coated on one side with a gelatin photographic emulsion, emulsion containing microscopically small light-sensitive silver halide crystals. The sizes and other characteristics of th ...

or the

magnetic storage Magnetic storage or magnetic recording is the storage of data on a magnetized medium. Magnetic storage uses different patterns of magnetisation in a magnetizable material to store data and is a form of non-volatile memory. The information is acc ...

medium inside

floppy disk A floppy disk or floppy diskette (casually referred to as a floppy, or a diskette) is an obsolescent type of disk storage composed of a thin and flexible disk of a magnetic storage medium in a square or nearly square plastic enclosure lined w ...

s can be used as inexpensive visible-light filters. Further filtering can be done in software by adjusting exposure and threshold. All video devices like webcams have a built-in infrared-blocking filter, which can be removed to improve sensitivity to infrared light, allowing better point isolation and the possibility of retroreflective tracking. This is normally a straightforward and reversible procedure for most webcams. Wii Remotes and NaturalPoint cameras are designed for infrared point tracking, so they already have visible-light-blocking filters.

Point model

Model configurations

*Single point: a single tracking point, for example attached on a baseball cap or wire that extends forward of the head. Can be easily achieved with a small light source like a

penlight A flashlight ( US, Canada) or torch ( UK, Australia) is a portable hand-held electric lamp. Formerly, the light source typically was a miniature incandescent light bulb, but these have been displaced by light-emitting diodes (LEDs) since the ...

. Unlike other configurations, this is limited to two degrees of freedom (yaw and pitch) and can be more sensitive to translation than rotation. *Three point clip:

asymmetrical Asymmetry is the absence of, or a violation of, symmetry (the property of an object being invariant to a transformation, such as reflection). Symmetry is an important property of both physical and abstract systems and it may be displayed in pre ...

triangular arrangement of points that can be clipped to

headphones Headphones are a pair of small loudspeaker drivers worn on or around the head over a user's ears. They are electroacoustic transducers, which convert an electrical signal to a corresponding sound. Headphones let a single user listen to an au ...

and positioned to the side of the head. *Three point cap: symmetrical triangular arrangement of points, typically attached to the rim of a baseball cap. *Four point cap: symmetrical non-coplanar arrangement of points, also attached to the rim of a baseball cap. Models can be made in a

DIY "Do it yourself" ("DIY") is the method of building, modifying, or repairing things by oneself without the direct aid of professionals or certified experts. Academic research has described DIY as behaviors where "individuals use raw and semi ...

fashion at minimal expense using readily available electronic components. Component kits and fully constructed models are also available for purchase from some members of the FreeTrack community.

Active points

An active point model uses visible or infrared LEDs (5 mm or larger) to represent the tracking points, powered by battery, transformer (plug pack) or

USB Universal Serial Bus (USB) is an industry standard that establishes specifications for cables, connectors and protocols for connection, communication and power supply (interfacing) between computers, peripherals and other computers. A broad v ...

. The electric circuit is very basic and can be made by someone with little or no experience with electronics. Common LEDs, like those found in

remote control In electronics, a remote control (also known as a remote or clicker) is an electronic device used to operate another device from a distance, usually wirelessly. In consumer electronics, a remote control can be used to operate devices such as ...

s, have a narrow, highly focused beam which is not suitable for optical motion tracking. They can be easily turned into wide angle LEDs by filing their lens tips down flat. Alternatively, wide angle LEDs can be purchased from specialist electronics retailers, like the infrared Siemens/Osram SFH485P, with a half-angle of 40 degrees.

Reflective points

Retroreflective material can be used to represent the tracking points by illumination with an infrared light source. This configuration doesn’t require wires or batteries connected to the user but is more susceptible to interference by background light. In most cases a webcam’s internal infrared blocking filter needs to be removed to increase sensitivity enough that the infrared light reflected by the tracking points can be seen.

FreeTrack interface

FreeTrack has a simple interface that can be freely used by third party programs such as Eyeware Beam to access 6DOF tracking data, both real raw measurements and virtual. It is hardware agnostic, so is not dependent on a specific brand or version of hardware and can be used without restriction. Bohemia Interactive's ARMA 2 is the first game to support the FreeTrack interface an
GP Bikes
is the first to have exclusive support.

TrackIR interface

FreeTrack is compatible with the unencrypted version of NaturalPoint's head tracking TrackIR interface that has widespread support in simulation games. NaturalPoint have been supplying game developers with an

encrypted In cryptography, encryption is the process of encoding information. This process converts the original representation of the information, known as plaintext, into an alternative form known as ciphertext. Ideally, only authorized parties can decip ...

version of the interface for more popular titles since late 2008, these can be identified as requiring TrackIR software version 4.1.036 or higher and are incompatible with FreeTrack. The developers of the first game affected, DCS: Black Shark, tried to release their own head tracking interface but soon after canceled it at NaturalPoint's request. FreeTrack compatibility is still possible usin
TrackIRFixer
to remove the encryption requirement in games. ''TIRViews.dll'' is a

dynamic-link library Dynamic-link library (DLL) is Microsoft's implementation of the shared library concept in the Microsoft Windows and OS/2 operating systems. These libraries usually have the file extension DLL, OCX (for libraries containing ActiveX controls), o ...

file distributed with TrackIR software that provides tailored support for a small number of mostly older games, using special interfaces or memory

hacks Hacks may refer to: Arts, entertainment, and media * ''Hacks'' (1997 film), a 1997 American comedy film * ''Hacks'' (2002 film), a 2002 independent American film * '' Hacks: The Inside Story'', a book by Donna Brazile * ''Hacks'' (TV series), ...

to facilitate view control. Though a violation of the TrackIR software's

EULA An end-user license agreement or EULA () is a legal contract between a software supplier and a customer or end-user, generally made available to the customer via a retailer acting as an intermediary. A EULA specifies in detail the rights and restr ...

, it is possible to use it with FreeTrack. NaturalPoint's TrackIR interface SDK is only available under a signed license agreement and is covered by a

NDA NDA may stand for: Military * National Defence Academy (India), a military academy in India * National Defence Act, legislation for organizing and funding Canada's military * National Defense Academy of Japan, a military academy in Japan * Nig ...

, so while FreeTrack is

free software Free software or libre software is computer software distributed under terms that allow users to run the software for any purpose as well as to study, change, and distribute it and any adapted versions. Free software is a matter of liberty, no ...

, the TrackIR interface component is required to be

closed source Proprietary software is software that is deemed within the free and open-source software to be non-free because its creator, publisher, or other rightsholder or rightsholder partner exercises a legal monopoly afforded by modern copyright and inte ...

References

External links

Official FreeTrack website

Daniel DeMenthon's academic page
- papers and examples of POSIT algorithm
FreeTrack YouTube channel
{{DEFAULTSORT:Freetrack Virtual reality Windows-only free software Computer accessibility Game controllers Mixed reality Tracking