MQTT Protocol

MQTT stands for Message Queuing Telemetry Transport and is a lightweight publish/subscribe messaging protocol for Machine to Machine(M2M) telemetry in low bandwidth environments. It was developed in 1999 by Arlen Nipper and Andy Stanford-Clark of IBM to link Oil Pipeline telemetry systems through satellite. It was once a proprietary protocol, but in 2010 MQTT was made available royalty-free, and in 2014 MQTT was made an OASIS standard. MQTT is quickly taking over as one of the primary IOT (internet of things) deployment protocols.

What is MQTT Protocol ?

MQTT (MQ Telemetry Transport) is a lightweight open messaging protocol that provides a simple mechanism for resource-constrained network clients to distribute telemetry data in low-bandwidth contexts. The protocol is used for M2M(machine-to-machine) communication and employs a publish/subscribe communication structure. This protocol helps connect to a remote location where bandwidth is limited. These properties make it suitable in various circumstances, including constant environments such as machine-to-machine communication and internet of things contexts. We can post and receive messages as clients in this publish and subscribe system. Multiple-device communication is facilitated by it. It is a straightforward messaging protocol built for low-bandwidth devices, making it the ideal choice for internet of things applications.

MQTT was created to work in an embedded environment and provide a dependable, efficient communication method. MQTT is a low-overhead protocol developed to work around bandwidth and CPU restrictions. MQTT is a viable option for wireless networks that may encounter latency issues because of bandwidth restrictions or shaky connections.

Telemetry is the automatic transfer of measurements or other data from distant or inaccessible sites to receiving equipment for monitoring.

Brief History of MQTT

Dr. Andy Stanford-Clark of IBM and Arlen Nipper of Arcom (now Eurotech) invented MQTT in 1999. MQTT was developed as a low-cost and dependable method of connecting monitoring devices used in the oil and gas sectors to remote company servers. When faced with the difficulty of transferring data from pipeline sensors in the desert to off-site SCADA(Supervisory Control And Data Acquisition) systems, they settled on a TCP/IP-based pub/sub topology that would be event-driven to reduce satellite link transmission costs. While still intimately associated with IBM, MQTT is now an open protocol managed by the Organization for the Advancement of Structured Information Standards (OASIS).

Despite its name, MQTT is not part of the original IBM MQSeries. Nonetheless, as of version 7.1, it is available in WebSphere MQ. MQTT was previously known as the SCADA protocol, MQ Integrator SCADA Device Protocol (MQIsdp), and WebSphere MQTT (WMQTT), albeit all of these names, have since become obsolete.

Characteristics of MQTT

Authentication

MQTT authenticates every user who wishes to publish or subscribe to specific data. The user id and password are saved in the API database in a collection called ‘mqtt’. We supply the login and password when connecting to the MQTT broker, and the MQTT Broker validates the credentials based on the values in the database.

Access Control

MQTT controls which users have access to particular topics. This data is stored in MongoDB in the table ‘mqtt acl’. By choosing ‘#’ as the permissible subject to publish and subscribe to for all users, all users are granted access to all topics by default.

QoS

The Quality of Service (QoS) level is the message transfer quality that ensures message delivery between the sending and receiving bodies. The three QoS levels in MQTT are At most once(0), At most once(1), and At most once(2).

Last Will Message

MQTT employs the Last Will & Testament (LWT) method to alert other clients of an ungraceful disconnection of a client. When a client wants to connect to a broker using this mechanism, each client provides its last will message, a standard MQTT message with QoS, subject, retained flag, and payload. The broker stores this notice until it detects that the client has disconnected rudely.

Retain Message

Message Retention is another characteristic of MQTT. It is accomplished by setting TRUE to keep the flag. It then kept the topic’s last message and QoS. When a client subscribes to a subject, the broker associates the topic with a previously retained message. Clients will receive messages promptly if the topic and the retained message match. Brokers only keep one retained message per subject.

Duplicate Message

If a publisher does not receive an acknowledgment for a published packet, the packet is resent with the DUP flag set to true. The Message ID of a duplicate message is the same as the original message’s.

Session

In general, when a client first connects with a broker, the client must create subscriptions for any topics for which they wish to receive data/messages from the broker. If a session is not kept, there is no persistent session, or the client loses contact with the broker, users must resubscribe to all subjects after reconnecting with the broker.

Types of MQTT Messages

During the communication phase, a client can connect, publish, subscribe, unsubscribe, and disconnect as needed. Fourteen defined message types are used to join and disengage a client from a broker, publish data, acknowledge data reception, and oversee the relationship between client and server. The various message kinds are defined below.

MQTT Message Format

The MQTT protocol employs the command and command acknowledgment formats, which means that an acknowledgment accompanies each command. The connect command has to connect acknowledgment, the publish command has published acknowledgment, and the subscribe command has subscribed acknowledgment, as indicated in the above figure. This approach is analogous to the TCP protocol’s handshaking mechanism. Three-Way HandShake, also known as a TCP 3-way handshake, is a technique used in a TCP/IP network to establish a connection between the server and client. Before the actual data transmission process can begin, both the client and server must exchange synchronization and acknowledgment packets.

The MQTT message format consists of a two-byte fixed header that appears in all MQTT packets. The second field is a changeable header that may or may not be present. The third field, which is not mandatory, is a payload. The payload field holds the data that is being transferred. We might believe that the payload is a required field, but this is not the case. Some instructions, such as disconnect messages, do not use the payload field.

Let’s try to understand the MQTT packet structure in detail.

Fixed Header

The fixed header has two bytes, as shown in the above format.

The following fields are contained in the first byte :

• MQTT Control Packet :
  It occupies 4 bits or 7 to 4-bit locations. Each bit in this 4-bit value reflects the control packet type of the MQTT protocol.
• Flag specific to each MQTT packet :
  The remaining four bits are flagged unique to each MQTT packet type.Byte 2 contains the remaining length.
• Remaining Length :
  The remaining length is a variable-length integer that indicates how many bytes are left in the current control packet, including data in the variable header and the payload. As a result, the remaining length equals the data in the variable header plus the payload.Remaining length = length of payload + length of variable header.

Variable Header

Some MQTT control packet types include an optional field, i.e., a variable header component. This field is sandwiched between the fixed header and the payload. The content of the variable header is determined by the packet type. The packet identifier field, common in many packet types, is contained in the variable header. Many MQTT control packet types have a changeable header component that includes a 2-byte integer, i.e., the packet identifier field.

Payload

In the ICMP message format, the payload is the last MQTT control packet. This section contains the information that will be delivered. The payload of the CONNECT packet, for example, is a client ID, the username, and the password, whereas the payload of the PUBLISH packet is simply an application message.

Architecture of MQTT

In the client/server architecture of MQTT, each sensor acts as a client and establishes a TCP connection with a server, called a broker. MQTT protocol is a message-oriented protocol. Every message is a separate chunk of data the broker cannot see.

Every communication is sent to an address known as a topic. Clients can subscribe to several topics. Every message published on a topic is delivered to every client who has subscribed to that topic.

Consider a straightforward network with three clients and a single broker, for instance.

• TCP connections are established between all three clients and the broker. Clients B and C are interested in the topic temperature.
• Client A afterward publishes a value of 22.5 for the topic temperature. The broker forwards the message to all subscribers.
• MQTT clients can communicate one-to-one, one-to-many, or many-to-one using the publisher-subscriber model.

Advantages of MQTT Protocol

• Packet agnostic :
  The payload carried by the packet can contain any data. The information could be text or binary. MQTT protocol makes no difference if the recipient understands how to interpret it.
• Reliability :
  Quality of Service (QoS) solutions are available to ensure delivery.
• Scalability :
  The publish/subscribe model scales effectively and uses little power.
• Decoupled design :
  Several design components disconnect the device from the subscribing server, resulting in a more resilient communication approach.
• Time :
  MQTT is regardless of the state of the subscribing server, a device can publish its data. When the subscribing server is ready, it can connect and receive the data. So it is a time-efficient protocol.

Disadvantages of MQTT Protocol

• Slower transmission cycles than Constrained Application Protocol (CoAP). Fast cycles are essential for systems with over 250 devices.
• Centralized brokers limit scalability since each client device incurs some expense. A local broker hub is utilized to provide scalability.
• MQTT uses the TCP protocol, which necessitates additional computing power and memory. TCP employs the handshake protocol, which requires periodic wake-up and communication intervals. This influences battery consumption. Furthermore, TCP-connected devices tend to keep sockets open for each other, increasing memory and power requirements.
• It is more challenging to implement than HTTP.
• Advanced features like flow control are not supported.

Application of MQTT Protocol

• According to MQTT.org, because MQTT clients are compact, they demand fewer resources and can be used on small microcontrollers. MQTT headers are minimal to optimize network bandwidth. Furthermore, according to the organization, MQTT ” can scale to link with millions of IoT devices.”

Some of the IoT applications that use the MQTT protocol are :

• Smart metering :
  To give precise meter readings in real-time, the MQTT protocol can transfer data with guaranteed message delivery. This improves billing accuracy.
• Billing system :
  MQTT helps eliminate duplicate or lost message packets in billing or invoicing.
• Gathering ambient sensor data :
  MQTT is a suitable fit for IoT sensor buildouts with lower-priority data transfer needs because sensors used in distant areas are frequently low-power devices.
• MQTT’s lightweight features make it ideal for remote monitoring applications such as the following :
  • Sensor synchronization, such as fire detectors or motion sensors for theft detection, establishes the hazard’s validity.
  • For patients leaving a hospital, sensors are used to check health metrics.
• Another application is a text-based real-time communication channel that takes advantage of MQTT’s low data and energy consumption. For example, Facebook employs MQTT for its Messenger app because it saves battery power during mobile phone-to-phone texting and allows messages to be sent efficiently in milliseconds despite unreliable internet connections around the world.
• MQTT is supported by most major cloud service providers, including Amazon Web Services (AWS), Google Cloud, IBM Cloud, and Microsoft Azure.
• MQTT is particularly suited to applications that use M2M and IoT devices for real-time analytics, preventative maintenance, and monitoring in smart homes, healthcare, logistics, industrial, and manufacturing areas.

Conclusion

• MQTT is a lightweight publish or subscribe messaging protocol for Machine to Machine(M2M) telemetry in low bandwidth environments.
• Dr. Andy Stanford-Clark of IBM and Arlen Nipper of Arcom (now Eurotech) invented MQTT in 1999.
• Authentication, QoS, access control, last will message, retain the message, duplicate message, and session are the seven important characteristics of the MQTT protocol.
• MQTT is a viable option for wireless networks that may encounter latency issues because of bandwidth restrictions or shaky connections.
• The MQTT protocol employs the command and command acknowledgment formats, which means that an acknowledgment accompanies each command.
• MQTT packet structure contains three packets :
  fixed header, variable header, and payload. The fixed header is present in all the messages, but the variable header is optional.