Continuous analytics is a
data science process that abandons
ETLs and complex batch
data pipelines in favor of
cloud
In meteorology, a cloud is an aerosol consisting of a visible mass of miniature liquid droplets, frozen crystals, or other particles suspended in the atmosphere of a planetary body or similar space. Water or various other chemicals may ...
-native and
microservices
A microservice architecture – a variant of the service-oriented architecture structural style – is an architectural pattern that arranges an application as a collection of loosely-coupled, fine-grained services, communicating through lightwe ...
paradigms. Continuous
data processing enables real time interactions and immediate insights with fewer resources.
Defined
Analytics is the application of
mathematics and
statistics to big data. Data scientists write analytics programs to look for solutions to business problems, like forecasting
demand
In economics, demand is the quantity of a good that consumers are willing and able to purchase at various prices during a given time. The relationship between price and quantity demand is also called the demand curve. Demand for a specific item ...
or setting an optimal price. The continuous approach runs multiple stateless engines which concurrently enrich, aggregate, infer and act on the data. Data scientists, dashboards and client apps all access the same raw or real-time data derivatives with proper identity-based security,
data masking
Data masking or data obfuscation is the process of modifying sensitive data in such a way that it is of no or little value to unauthorized intruders while still being usable by software or authorized personnel. Data masking can also be referred as ...
and
versioning in real-time.
Traditionally, data scientists have not been part of
IT development teams, like regular
Java
Java (; id, Jawa, ; jv, ꦗꦮ; su, ) is one of the Greater Sunda Islands in Indonesia. It is bordered by the Indian Ocean to the south and the Java Sea to the north. With a population of 151.6 million people, Java is the world's mos ...
programmers. This is because their skills set them apart in their own department not normally related to IT, i.e., math, statistics, and data science. So it is logical to conclude that their approach to writing
software code
A computer program is a sequence or set of instructions in a programming language for a computer to execute. Computer programs are one component of software, which also includes documentation and other intangible components.
A computer program ...
does not enjoy the same efficiencies as the traditional programming team. In particular traditional programming has adopted the Continuous Delivery approach to writing code and the
agile methodology. That releases software in a continuous circle, called
iterations
Iteration is the repetition of a process in order to generate a (possibly unbounded) sequence of outcomes. Each repetition of the process is a single iteration, and the outcome of each iteration is then the starting point of the next iteration. ...
.
Continuous analytics then is the extension of the continuous delivery software development model to the
big data analytics development team. The goal of the continuous analytics practitioner then is to find ways to incorporate writing analytics code and installing big data software into the agile development model of automatically running unit and functional tests and building the environment system with automated tools.
To make this work means getting
data scientists
Data science is an interdisciplinary field that uses scientific methods, processes, algorithms and systems to extract or extrapolate knowledge and insights from noisy, structured and unstructured data, and apply knowledge from data across a ...
to write their code in the same
code repository that regular programmers use so that software can pull it from there and run it through the build process. It also means saving the configuration of the big data cluster (sets of
virtual machines
In computing, a virtual machine (VM) is the virtualization/ emulation of a computer system. Virtual machines are based on computer architectures and provide functionality of a physical computer. Their implementations may involve specialized hard ...
) in some kind of repository as well. That facilitates sending out analytics code and big data software and objects in the same automated way as the continuous integration process.
Data Scientist Ricardo Ramon Benitez
/ref>
External links
Continuous analytics
Development model
References
Data analysis
Big data
{{Database-stub