IBM MQ is a family of
message-oriented middleware
Message-oriented middleware (MOM) is software or hardware infrastructure supporting sending and receiving messages between distributed systems. MOM allows application modules to be distributed over heterogeneous platforms and reduces the comple ...
products that
IBM launched in December 1993. It was originally called MQSeries, and was renamed ''WebSphere MQ'' in 2002 to join the suite of
WebSphere
IBM WebSphere refers to a brand of proprietary computer software products in the genre of enterprise software known as "application and integration middleware". These software products are used by end-users to create and integrate applications w ...
products. In April 2014, it was renamed ''IBM MQ''. The products that are included in the MQ family are IBM MQ, IBM MQ Advanced, IBM MQ Appliance, IBM MQ for z/OS, and IBM MQ on IBM Cloud. IBM MQ also has containerised deployment options.
MQ allows independent and potentially non-concurrent applications on a
distributed system
A distributed system is a system whose components are located on different networked computers, which communicate and coordinate their actions by passing messages to one another from any system. Distributed computing is a field of computer sci ...
to securely communicate with each other, using messages. MQ is available on a large number of platforms (both IBM and non-IBM), including
z/OS
z/OS is a 64-bit operating system for IBM z/Architecture mainframes, introduced by IBM in October 2000. It derives from and is the successor to OS/390, which in turn was preceded by a string of MVS versions.Starting with the earliest:
* ...
(
mainframe),
IBM i,
Transaction Processing Facility
Transaction Processing Facility (TPF) is an IBM real-time operating system for mainframe computers descended from the IBM System/360 family, including zSeries and System z9.
TPF delivers fast, high-volume, high-throughput transaction processi ...
,
UNIX
Unix (; trademarked as UNIX) is a family of multitasking, multiuser computer operating systems that derive from the original AT&T Unix, whose development started in 1969 at the Bell Labs research center by Ken Thompson, Dennis Ritchie, an ...
(
AIX
Aix or AIX may refer to:
Computing
* AIX, a line of IBM computer operating systems
*An Alternate Index, for a Virtual Storage Access Method Key Sequenced Data Set
* Athens Internet Exchange, a European Internet exchange point
Places Belgi ...
,
HP-UX,
Solaris), HP
NonStop,
OpenVMS,
Linux
Linux ( or ) is a family of open-source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991, by Linus Torvalds. Linux is typically packaged as a Linux distribution, w ...
, and
Microsoft Windows.
MQ Components
The core components of MQ are:
* Message: Messages are collections of
binary
Binary may refer to:
Science and technology Mathematics
* Binary number, a representation of numbers using only two digits (0 and 1)
* Binary function, a function that takes two arguments
* Binary operation, a mathematical operation that ta ...
or character (for instance
ASCII
ASCII ( ), abbreviated from American Standard Code for Information Interchange, is a character encoding standard for electronic communication. ASCII codes represent text in computers, telecommunications equipment, and other devices. Because ...
or
EBCDIC
Extended Binary Coded Decimal Interchange Code (EBCDIC; ) is an eight- bit character encoding used mainly on IBM mainframe and IBM midrange computer operating systems. It descended from the code used with punched cards and the corresponding ...
) data that have some meaning to a participating program. As in other
communication protocol
A communication protocol is a system of rules that allows two or more entities of a communications system to transmit information via any kind of variation of a physical quantity. The protocol defines the rules, syntax, semantics and synchroniza ...
s, storage, routing, and delivery information is added to the message before transmission and stripped from the message prior to delivery to the receiving application.
* Queue:
Message queue
In computer science, message queues and mailboxes are software-engineering components typically used for inter-process communication (IPC), or for inter- thread communication within the same process. They use a queue for messaging – the ...
s are objects that store messages in an application.
* Queue Manager: a system service that provides a logical container for the message queue. It is responsible for transferring data to other queue managers via message channels. Although not strictly required for message-oriented middleware, is an IBM MQ prerequisite. Queue managers handle storage, timing issues, triggering, and all other functions not directly related to the actual movement of data.
Programs integrated with IBM MQ use a consistent application program interface (API) across all platforms.
Messaging types
MQ supports point-to-point and
Publish-Subscribe messaging.
APIs
APIs directly supported by IBM include:
* IBM Message Queue Interface (MQI) for
C,
COBOL,
PL/I
PL/I (Programming Language One, pronounced and sometimes written PL/1) is a procedural, imperative computer programming language developed and published by IBM. It is designed for scientific, engineering, business and system programming. I ...
,
Java
Java (; id, Jawa, ; jv, ꦗꦮ; su, ) is one of the Greater Sunda Islands in Indonesia. It is bordered by the Indian Ocean to the south and the Java Sea to the north. With a population of 151.6 million people, Java is the world's mos ...
,
Rexx,
RPG, and
C++
C++ (pronounced "C plus plus") is a high-level general-purpose programming language created by Danish computer scientist Bjarne Stroustrup as an extension of the C programming language, or "C with Classes". The language has expanded significan ...
*
Java Message Service
The Jakarta Messaging API (formerly Java Message Service or JMS API) is a Java application programming interface (API) for message-oriented middleware. It provides generic messaging models, able to handle the producer–consumer problem, that can ...
(JMS)
* XMS for C/C++ and .NET
*
.NET
*
Representational State Transfer
*
SOAP
Soap is a salt of a fatty acid used in a variety of cleansing and lubricating products. In a domestic setting, soaps are surfactants usually used for washing, bathing, and other types of housekeeping. In industrial settings, soaps are use ...
Additional APIs (not officially supported) are also available via third parties, including:
*
Perl
Perl is a family of two high-level, general-purpose, interpreted, dynamic programming languages. "Perl" refers to Perl 5, but from 2000 to 2019 it also referred to its redesigned "sister language", Perl 6, before the latter's name was offic ...
interface (developed and contributed by Hildo Biersma), available from
CPAN
The Comprehensive Perl Archive Network (CPAN) is a repository of over 250,000 software modules and accompanying documentation for 39,000 distributions, written in the Perl programming language by over 12,000 contributors. ''CPAN'' can denote eit ...
.
*
Python (programming language)
Python is a high-level, general-purpose programming language. Its design philosophy emphasizes code readability with the use of significant indentation.
Python is dynamically-typed and garbage-collected. It supports multiple programming p ...
interface ''PyMQI'' (originally developed by Les Smithson), available from
PyPI
The Python Package Index, abbreviated as PyPI () and also known as the Cheese Shop (a reference to the ''Monty Python's Flying Circus'' sketch " Cheese Shop"), is the official third-party software repository for Python. It is analogous to the C ...
*
Windows PowerShell
PowerShell is a task automation and configuration management program from Microsoft, consisting of a command-line shell and the associated scripting language. Initially a Windows component only, known as Windows PowerShell, it was made open-so ...
Features
One-time delivery: MQ uses once and once only delivery. This quality of service typically prevents message loss or duplication.
Asynchronous messaging: MQ provides
application designers with a mechanism to achieve non-time-dependent architecture. Messages can be sent from one application to another, regardless of whether the applications are running at the same time. If a message receiver application is not running when a sender sends it a message, the queue manager will hold the message until the receiver asks for it. Ordering of all messages is preserved, by default this is in
FIFO order of receipt at the local queue within priority of the message.
Data transformation: e.g.
Big Endian
In computing, endianness, also known as byte sex, is the order or sequence of bytes of a word of digital data in computer memory. Endianness is primarily expressed as big-endian (BE) or little-endian (LE). A big-endian system stores the most sig ...
to
Little Endian
In computing, endianness, also known as byte sex, is the order or sequence of bytes of a word of digital data in computer memory. Endianness is primarily expressed as big-endian (BE) or little-endian (LE). A big-endian system stores the most si ...
, or
EBCDIC
Extended Binary Coded Decimal Interchange Code (EBCDIC; ) is an eight- bit character encoding used mainly on IBM mainframe and IBM midrange computer operating systems. It descended from the code used with punched cards and the corresponding ...
to
ASCII
ASCII ( ), abbreviated from American Standard Code for Information Interchange, is a character encoding standard for electronic communication. ASCII codes represent text in computers, telecommunications equipment, and other devices. Because ...
. This is accomplished through the use of message data ''exits.'' Exits are compiled applications that run on the queue manager host, and are executed by the IBM MQ software at the time data transformation is needed.
Message-driven architecture framework: IBM MQ allows receipt of messages to "trigger" other applications to run.
Range of APIs: It implements the
Java Message Service
The Jakarta Messaging API (formerly Java Message Service or JMS API) is a Java application programming interface (API) for message-oriented middleware. It provides generic messaging models, able to handle the producer–consumer problem, that can ...
(JMS) standard API, and also has its own proprietary API, known as the Message Queuing Interface (MQI), which preceded the JMS several years in existence. As of version 8.0.0.4, MQ also supports the MQ Light API.
Clustering: Multiple MQ implementations share the processing of messages, providing load balancing.
Communication
Queue managers communicate with the outside world either through:
*Bindings: a direct software connection. Generally faster, but limited to programs running on the same physical host as the queue manager.
*A network or "client" connection: applications using a client connection can connect to a queue manager on any other host in the network. The physical location of the queue manager is irrelevant, as long as it is reachable over the network.
Communication between queue managers
This relies on a ''channel''. Each queue manager uses one or more channels to send and receive data to other queue managers. A channel is uni-directional; a second channel is required to return data. In a TCP/IP based network, a channel sends or receives data on a specific port.
Channel types:
*Sending channel: has a defined destination and is associated with a specific transmission queue (the mechanism by which messages are queued awaiting transmission on the channel).
*Receiving channel: receives data from any other queue manager with a sending channel of the same name.
When a receiving channel receives a message, it is examined to see which queue manager and queue it is destined for. In the event of a communications failure, MQ can automatically re-establish a connection when the problem is resolved.
The ''listener'' is the application's network interface to the queue manager. The listener detects connections from incoming channels, and manages the connection of the sending channels to the receiving channels. In a TCP/IP network, the listener will "listen" for connections on a specific port.
Transmitting data to a queue on another queue manager
Queue types:
*Local queue: represents the location where data is stored awaiting processing.
*Remote queue: represents a queue on another queue manager. They define the destination queue, which is one element of the routing mechanism for messages.
*Cluster queue: represents a queue which is reachable via any queue manager in its cluster.
A message is placed on a remote queue. The messages goes to a temporary storage transmission queue associated with a channel. On placing a message on a remote queue, the message is transmitted across the remote channel. If the transmission is successful, the message is removed from the transmit queue. On receiving a message, the receiving queue manager examines the message to determine whether the message is for itself or if must go to another queue manager. If the receiving queue manager, the required queue will be checked, and if it exists, the message is placed on this queue. If not, the message is placed on the
dead letter queue. MQ has features to manage efficient transmission of data across a variety of communication media. For example, messages can be batched together until a queue reaches a particular depth.
Ordering
Although the queue is FIFO, it is ordered based on the receipt in the local queue, not the committing of the message from the sender. Messages can be prioritized, and by default, the queue is prioritized in order of arrival. Queues will only be in sequence of addition if the message is added locally. Message grouping can be used to ensure a set of messages are in a specific order, aside from that, if sequence is critical, it is the application's responsibility to place sequence data in the message or implement a handshaking mechanism via a return queue. In reality, ordering will be maintained in straightforward configurations.
The log
The other element of a queue manager is the ''log''. As a message is placed on a queue or a configuration change is made, the data is also logged. In the event of a failure, the log is used to recreate damaged objects and recreate messages. Only ''persistent'' messages are recreated when a failure occurs—"non-persistent" messages are lost. Non-persistent messages can be sent across a channel set to a fast mode, in which delivery is not assured in the event of a channel failure.
MQ supports both circular and linear logging.
Retrieving messages from queues
Information can be retrieved from queues either by polling the queue to check for available data at suitable intervals, or alternatively MQ can trigger an event, allowing a client application to respond to the delivery of a message.
Availability
IBM MQ offers a variety of solutions to cater for availability:
Replicated Data Queue Manager (RDQM / 'Easy HA'- MQ Advanced on distributed only):
Synchronous replication between three servers that all share a floating IP address.
Queue Manager Clusters:
Groups of two or more queue managers on one or more computers are defined to a cluster, providing automatic interconnection, and allow queues to be shared among them for load balancing and redundancy.
Queue Sharing Groups (z/OS only):
In a Shared Queue environment, an application can connect to any of the queue managers within the queue-sharing group. Because all the queue managers in the queue-sharing group can access the same set of shared queues, the application does not depend on the availability of a particular queue manager. This gives greater availability if a queue manager stops because all the other queue managers in the queue-sharing group can continue processing the queue.
Multi-Instance Queue Managers (available from v7.0.1):
Instances of the same queue manager are configured on two or more computers with their queues and meta data residing on shared storage. By starting multiple instances, one instance becomes the active instance and the other instances become standbys. If the active instance fails, a standby instance running on a different computer automatically takes over.
History
Version release dates
Version end of support dates
The following table applies to MQ software. The MQ Appliance has different lifecycle dates for both firmware and hardware than those in the table.
Background architectural reference
With the advent of computers, IBM saw an opportunity to apply new technology to the need for message switching.
In the early 1960s, IBM marketed the IBM 7740 Communication Control System and the IBM 7750 Programmed Transmission Control, which were programmable message switching systems.
The IBM System/360 was announced in April 1964 and with it came communication access methods such as BTAM and QTAM (Basic and Queued Telecommunications Access Methods). In 1971, TCAM, the
Telecommunications Access Method
Telecommunications Access Method (TCAM) is an access method, in IBM's OS/360 and successors computer operating systems on IBM System/360 and later, that provides access to terminals units within a teleprocessing network.
Features
TCAM provides si ...
, offered its users a more advanced form of message switching or message routing. TCAM was widely accepted, especially in the financial and brokerage industries. It supported asynchronous messaging, as with the later MQ. TCAM 3.0 added in reusable disk message queues for recovery soon thereafter, as with MQ. A high-level PL/I program could be used to access TRANSIENT datasets (dynamic message queues). Reading a message from a transient dataset resulted in that message being removed from the queue, as with a non-browse READ with MQ.
In the late 1970s, transaction management systems came into being, each trying to achieve a leadership position in the industry. Within IBM,
CICS
IBM CICS (Customer Information Control System) is a family of mixed-language application servers that provide online transaction management and connectivity for applications on IBM mainframe systems under z/OS and z/VSE.
CICS family products ...
and
IMS were chosen as strategic products to address the need for transaction management. Within both CICS and IMS, each had its version of message switching, IMS being a front-end queued system and CICS having its Transient Data facility as the possible basis for message switching.
CICS established itself as a popular transaction management system in the 1968-1971 timeframe. Those users who had adopted TCAM for its message handling capabilities now wanted a combined use of TCAM with CICS. In December 1971, IBM announced CICS support of TCAM as part of the CICS/OS-Standard product, to be delivered in December 1972. For interested customers, this enabled them to use TCAM for its message handling strengths and also have TCAM-connected terminals or computers interface with CICS online applications.
In January 1973, TCAM continued to be supported by CICS/OS-Standard Version 2.3. However, TCAM support was omitted from the initial release of CICS/VS, announced in February 1973 and delivered in June 1974. Needless to say, many CICS-TCAM customers were not happy with that product direction.
With considerable pressure from CICS-TCAM customers, the CICS support of TCAM was reinstated in the CICS/VS 1.1 product, as of September 1974. In addition to the previous DCB support, with this reinstatement of TCAM support, CICS began to support TCAM access via VTAM, also known as the ACB support. CICS TCAM ACB support was discontinued as of the CICS/ESA Version 3 product in 1990.
In 1992, IBM announced a new product called MQSeries. This brand name was later renamed to WebSphere MQ (sometimes shortened to WMQ) in 2002 to support the WebSphere family name and the product. In 2014, it was renamed IBM MQ. MQ was to be the extension of TCAM functionality from IBM-only systems to all other platforms. MQ has an architecture that enables heterogeneous systems to communicate with each other (e.g. IBM, HP, Sun, Tandem, etc.). MQ can be used with CICS systems to send and receive data to/from any other MQ-eligible system. MQ can be used to initiate work in a CICS system or a CICS transaction can initiate work in another CICS or non-CICS system.
IBM MQ now supports 80 different environments and has become the leading message assured delivery switching/routing product in the industry.
MQ and web services
IBM MQ can be used as a foundation for creating
service-oriented architecture
In software engineering, service-oriented architecture (SOA) is an architectural style that focuses on discrete services instead of a monolithic design. By consequence, it is also applied in the field of software design where services are provide ...
s. Several additional product options exist to help convert legacy programs into functioning
web services through the use of MQ. Larger, heterogeneous enterprises often appear as a federation of somewhat autonomous domains based on lines of business, functional or governance areas. In such environments, some services may be shared or reused only within a single domain, while others may be shared or reused throughout the enterprise. IBM MQ provides the means by which communication exists between lines-of-business or otherwise separate business domains.
A related product in the IBM MQ product family, called
IBM App Connect Enterprise (formerly IBM Integration Bus / WebSphere Message Broker) enables a diverse and robust set of extensions to queue-based architectures. Using IBM Integration Bus, users can implement a WebServices front-end, complete with
WSDL
The Web Services Description Language (WSDL ) is an XML-based interface description language that is used for describing the functionality offered by a web service. The acronym is also used for any specific WSDL description of a web service (also ...
file support that can interact with any queue-based application.
See also
*
Advanced Message Queuing Protocol
*
IBM WebSphere Message Broker
IBM App Connect Enterprise (abbreviated as IBM ACE, formerly known as IBM Integration Bus or WebSphere Message Broker) is IBM's premier integration software offering, allowing business information to flow between disparate applications across mu ...
*
Java Message Service
The Jakarta Messaging API (formerly Java Message Service or JMS API) is a Java application programming interface (API) for message-oriented middleware. It provides generic messaging models, able to handle the producer–consumer problem, that can ...
*
Message queue
In computer science, message queues and mailboxes are software-engineering components typically used for inter-process communication (IPC), or for inter- thread communication within the same process. They use a queue for messaging – the ...
*
Message Queuing as a Service
*
MQTT
MQTT (originally an initialism of MQ Telemetry Transport) is a lightweight, publish-subscribe, machine to machine network protocol for Message queue/Message queuing service. It is designed for connections with remote locations that have devices ...
References
External links
{{Spoken Wikipedia, En-IBM_WebSphere_MQ.ogg, date=2011-10-29
IBM MQ product page
Message-oriented middleware
Managed file transfer