|
Apache Kafka
Apache Kafka is a distributed event store and stream-processing platform. It is an open-source system developed by the Apache Software Foundation written in Java and Scala. The project aims to provide a unified, high-throughput, low-latency platform for handling real-time data feeds. Kafka can connect to external systems (for data import/export) via Kafka Connect, and provides the Kafka Streams libraries for stream processing applications. Kafka uses a binary TCP-based protocol that is optimized for efficiency and relies on a "message set" abstraction that naturally groups messages together to reduce the overhead of the network roundtrip. This "leads to larger network packets, larger sequential disk operations, contiguous memory blocks ..which allows Kafka to turn a bursty stream of random message writes into linear writes." History Kafka was originally developed at LinkedIn, and was subsequently open sourced in early 2011. Jay Kreps, Neha Narkhede and Jun Rao helped co-crea ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Apache Kafka Logo
The Apache ( ) are several Southern Athabaskan language-speaking peoples of the Southwestern United States, Southwest, the Southern Plains and Northern Mexico. They are linguistically related to the Navajo. They migrated from the Athabascan homelands in the north into the Southwest between 1000 and 1500 CE. Apache bands include the Chiricahua, Jicarilla Apache, Jicarilla, Lipan Apache people, Lipan, Mescalero, Mimbreño Apache, Mimbreño, Salinero Apaches, Salinero, Plains Apache, Plains, and Western Apache (San Carlos Apache Indian Reservation, Aravaipa, Pinaleño Mountains, Pinaleño, Fort Apache Indian Reservation, Coyotero, and Tonto Apache, Tonto). Today, Apache tribes and Indian reservation, reservations are headquartered in Arizona, New Mexico, Texas, and Oklahoma, while in Mexico the Apache are settled in Sonora, Chihuahua, Coahuila and areas of Tamaulipas. Each Native American tribe, tribe is politically autonomous. Historically, the Apache homelands have consisted of ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Domain-specific Language
A domain-specific language (DSL) is a computer language specialized to a particular application domain. This is in contrast to a general-purpose language (GPL), which is broadly applicable across domains. There are a wide variety of DSLs, ranging from widely used languages for common domains, such as HTML for web pages, down to languages used by only one or a few pieces of software, such as MUSH soft code. DSLs can be further subdivided by the kind of language, and include domain-specific ''markup'' languages, domain-specific ''modeling'' languages (more generally, specification languages), and domain-specific ''programming'' languages. Special-purpose computer languages have always existed in the computer age, but the term "domain-specific language" has become more popular due to the rise of domain-specific modeling. Simpler DSLs, particularly ones used by a single application, are sometimes informally called mini-languages. The line between general-purpose languages and doma ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Event-driven SOA
Event-driven SOA is a form of service-oriented architecture (SOA), combining the intelligence and proactiveness of event-driven architecture with the organizational capabilities found in service (systems architecture), service offerings. Before event-driven SOA, the typical SOA platform orchestrated services centrally, through pre-defined business processes, assuming that what should have already been triggered is defined in a business process. This older approach (sometimes called SOA 1.0) does not account for events that occur across, or outside of, specific business processes. Thus complex events, in which a pattern of activities—both non-scheduled and scheduled—should trigger a set of services is not accounted for in traditional SOA 1.0 architecture. SOA 2.0 SOA 2.0 architecture, ("event-driven SOA"), lets business users monitor, analyze, and enrich events to make the connections among disparate events that do not at first appear to be intuitively obvious. This makes these ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Event Stream Processing
In computer science, stream processing (also known as event stream processing, data stream processing, or distributed stream processing) is a programming paradigm which views streams, or sequences of events in time, as the central input and output objects of computation. Stream processing encompasses dataflow programming, reactive programming, and distributed data processing. Stream processing systems aim to expose parallel processing for data streams and rely on streaming algorithms for efficient implementation. The software stack for these systems includes components such as programming models and query languages, for expressing computation; stream management systems, for distribution and scheduling; and hardware components for acceleration including floating-point units, graphics processing units, and field-programmable gate arrays. The stream processing paradigm simplifies parallel software and hardware by restricting the parallel computation that can be performed. Given a se ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Enterprise Messaging System
An enterprise messaging system (EMS) or messaging system in brief is a set of published enterprise-wide standards that allows organizations to send semantically precise messages between computer systems. EMS systems promote loosely coupled architectures that allow changes in the formats of messages to have minimum impact on message subscribers. EMS systems are facilitated by the use of structured messages (such as using XML or JSON), and appropriate protocols, such as DDS, MSMQ, AMQP or SOAP with web services. EMS usually takes into account the following considerations: # ''Security'': Messages must be encrypted if they travel over public interfaces. Messages must be authenticated or digitally signed if the receiver is to have confidence that the messages have not been tampered with in transit. # ''Routing'': Messages need to be routed efficiently from the sender to the receiver. Intermediate nodes may need to route the messages if the body of the message is encrypted. # '' ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Enterprise Integration Patterns
''Enterprise Integration Patterns'' is a book by Gregor Hohpe and Bobby Woolf which describes 65 patterns for the use of enterprise application integration and message-oriented middleware in the form of a pattern language. The integration (messaging) pattern language The pattern language presented in the book consists of 65 patterns structured into 9 categories, which largely follow the flow of a message from one system to the next through channels, routing, and transformations. The book includes an icon-based pattern language, sometimes nicknamed "GregorGrams" after one of the authors. Excerpts from the book (short pattern descriptions) are available on the supporting website (see External links). Integration styles and types The book distinguishes four top-level alternatives for integration: # File Transfer # Shared Database # Remote Procedure Invocation # Messaging The following integration types are introduced: * Information Portal * Data Replication * Shared Business Func ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Data Distribution Service
The Data Distribution Service (DDS) for real-time systems is an Object Management Group (OMG) machine-to-machine (sometimes called middleware or connectivity framework) standard that aims to enable dependable, high-performance, interoperable, real-time, scalable data exchanges using a publish–subscribe pattern. DDS addresses the real-time data exchange needs of applications within aerospace, defense, air-traffic control, autonomous vehicles, medical devices, robotics, power generation, simulation and testing, smart grid management, transportation systems, and other applications. Architecture Model DDS is a networking middleware that simplifies complex network programming. It implements a publish–subscribe pattern for sending and receiving data, events, and commands among the nodes. Nodes that produce information (publishers) create "topics" (e.g., temperature, location, pressure) and publish "samples". DDS delivers the samples to subscribers that declare an interest ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Apache Spark
Apache Spark is an open-source unified analytics engine for large-scale data processing. Spark provides an interface for programming clusters with implicit data parallelism and fault tolerance. Originally developed at the University of California, Berkeley's AMPLab starting in 2009, in 2013, the Spark codebase was donated to the Apache Software Foundation, which has maintained it since. Overview Apache Spark has its architectural foundation in the resilient distributed dataset (RDD), a read-only multiset of data items distributed over a cluster of machines, that is maintained in a fault-tolerant way. The Dataframe API was released as an abstraction on top of the RDD, followed by the Dataset API. In Spark 1.x, the RDD was the primary application programming interface (API), but as of Spark 2.x use of the Dataset API is encouraged even though the RDD API is not deprecated. The RDD technology still underlies the Dataset API. Spark and its RDDs were developed in 2012 in respon ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Apache Samza
Apache Samza is an open-source, near-realtime, asynchronous computational framework for stream processing developed by the Apache Software Foundation in Scala and Java. It has been developed in conjunction with Apache Kafka. Both were originally developed by LinkedIn. Overview Samza allows users to build stateful applications that process data in real-time from multiple sources including Apache Kafka. Samza provides fault tolerance, isolation and stateful processing. Unlike batch systems such as Apache Hadoop or Apache Spark, it provides continuous computation and output, which result in sub-second response times. There are many players in the field of real-time stream processing and Samza is one of the mature products. It was added to Apache in 2013. Samza is used by multiple companies. The biggest installation is in LinkedIn. See also * Apache Beam Apache Beam is an open source unified programming model to define and execute data processing pipelines, including ETL, ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Apache Flink
Apache Flink is an Open-source software, open-source, unified stream processing, stream-processing and batch processing, batch-processing software framework, framework developed by the Apache Software Foundation. The core of Apache Flink is a distributed streaming data-flow engine written in Java (programming language), Java and Scala (programming language), Scala. Flink executes arbitrary Dataflow programming, dataflow programs in a Data parallelism, data-parallel and Pipeline (software), pipelined (hence Task parallelism, task parallel) manner. Flink's pipelined runtime system enables the execution of Batch processing, bulk/batch and stream processing programs. Furthermore, Flink's runtime supports the execution of Iterative method, iterative algorithms natively. Flink provides a high-throughput, low-latency streaming engine as well as support for event-time processing and state management. Flink applications are fault-tolerant in the event of machine failure and support exactly ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
NATS Messaging
NATS is an open-source messaging system (sometimes called message-oriented middleware). The NATS server is written in the Go programming language. Client libraries to interface with the server are available for dozens of major programming languages. The core design principles of NATS are performance, scalability, and ease of use. The acronym NATS stands for Neural Autonomic Transport System. NATS is an open source project under the stewardship of CNCF. NATS was originally developed bDerek Collisonas the messaging control plane for Cloud Foundry and was written in Ruby. NATS was later ported to Go. The source code is released under the Apache 2.0 License. NATS consists of: * ThNATS Server- The core Publish-Subscribe Server for NATS. Client librariesfor a variety of programming languages. * connector framework- a pluggable Java based framework to connect NATS and other services. NATS is a CNCF project with Kubernetes and Prometheus integration. The NATS server is often referre ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Redis
Redis (; Remote Dictionary Server) is an in-memory key–value database, used as a distributed cache and message broker, with optional durability. Because it holds all data in memory and because of its design, Redis offers low- latency reads and writes, making it particularly suitable for use cases that require a cache. Redis is the most popular NoSQL database, and one of the most popular databases overall. The project was developed and maintained by Salvatore Sanfilippo, starting in 2009. From 2015 until 2020, he led a project core team sponsored by Redis Ltd. Salvatore Sanfilippo left Redis as the maintainer in 2020. In 2021 Redis Labs dropped the Labs from its name and now is known simply as "Redis". In 2018, some modules for Redis adopted a modified Apache 2.0 with a Commons Clause. In 2024, the main Redis code switched from the open-source BSD-3 license to being dual-licensed under the Redis Source Available License v2 and the Server Side Public License v1. On May ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
