Eclipse Hono – Connecting large numbers of IoT devices
The value of an IoT solution often correlates directly to its ability to connect many different devices and integrate them into higher-level solutions. Although all solutions combine devices using a unique integration strategy, the reliable, secure, and scalable connection of devices is always crucial.
Connectivity is therefore best handled as a separate service, independent of a solution – and this is exactly what Eclipse Hono aims to accomplish.
Eclipse Hono is written entirely in Java. It also provides a uniform messaging infrastructure for IoT solutions. Eclipse Hono offers:
- Senior Software Engineer @ Red Hat
- Committer @ Eclipse Foundation / Apache Software Foundation
- a horizontally scalable microservice architecture
- a design for container-based cloud environments (based on AMQP 1.0)
- load protection for all microservices using AMQP 1.0 flow control
- protocol adapters for different IoT protocols
- handling of different communication patterns
- a tenant-based security model, including authentication of device
Solutions integrated with Eclipse Hono are freed from the details of device connectivity, and can instead concentrate on providing business value.
IoT communication patterns and QoS
Most messages in IoT setups are usually telemetry messages: data flows from devices to solutions. Messages of this kind usually need not be guaranteed of reaching their destinations and can be handled especially efficiently because there are no acknowledgements involved (QoS “at most once”). A typical type of telemetry data is sensor data that is sent within short time frames.
- Principal Software Engineer @ Red Hat
- Committer/Lead @ Eclipse Foundation
Telemetry messages that absolutely must be delivered (QoS “at least once”) are called events, which are always acknowledged to the client and occur infrequently by comparison. Typical events include alarms that need a guarantee to reach the solution and must never be dropped.
The other direction is called command and control: data flows from solutions to devices. This addresses the control of actors, or in general messages initiated by the solution, and will be provided in future versions of Eclipse Hono (versions > 0.5).
- Senior Software Developer @ Bosch Software Innovations
- Committer @ Eclipse Foundation
- Spring Core 4.2 Certified Developer
- sending and consuming message payloads for different communication patterns
- handling of device identities and device credentials for secure communication
- authentication of devices and microservices
Because the handling of device identities is assumed to be specific to a production setup, it is provided by a simple file-based implementation. Due to the use of Spring beans and the Vert.x event bus, this implementation can easily be substituted by a database implementation.
The same applies for the (file-based) authentication server, which may be replaced with other implementations.
Bird’s-eye architecture view
There are several different categories of Hono components:
Protocol adapters connect devices to Hono. Each covers a specific protocol and is responsible for the authentication of devices by querying the device registry.
Hono provides the following protocol adapters:
- Eclipse Kura-based gateways
Custom protocol adapters can be added by using the APIs of Hono Core.
Hono core consists of:
- the Device Registry, implementing
- the registration and activation of devices belonging to a tenant
- the provisioning of credentials for devices
- the Auth Server, implementing
- the construction of security tokens to support the authentication of a client
- Hono Messaging, implementing
- the messaging APIs for telemetry and event messages
- verification of compliance with the APIs
- the forwarding of messages to the AMQP 1.0 Network
AMQP 1.0 Network
The AMQP 1.0 Network decouples Hono’s core microservices from solution applications. It provides a scalable messaging infrastructure that supports different communication patterns with varying quality of service levels.
A few properties of the AMQP 1.0 messaging protocol make it an ideal candidate for this purpose.
First, this is one of very few messaging protocols that is truly symmetrical. It does not require a broker as an intermediary in the message exchange process. It supports both “store and forward” (brokered) and direct messaging communication patterns. The symmetrical nature of AMQP 1.0 allows Hono to use different communication semantics and different messaging components to handle various flows of messages.
The entry gateway into Hono’s AMQP messaging system is the Apache QPid Dispatch Router. It provides direct communication between producing and consuming endpoints. Unlike a broker, the router never takes ownership of messages; it instead simply passes AMQP packets between different endpoints. This means that, from a messaging perspective, the router is basically stateless. As such, it is much more vertically scalable than a message broker. In addition, it is far easier to scale it horizontally in a cloud environment – as we will see in a moment.
Another important feature of AMQP 1.0 is its flow control mechanism, which is implemented directly in the protocol. This means that consumers will, at every point, declare their capacity, or how many messages should be delivered to them. This information is propagated by the router, and producers of messages can be throttled to the speed of consumption.
It is now easy to see why Hono uses only routers to deliver telemetry messages to the solution tier. These messages typically exhibit a large volume, which requires scaling – but at the same time, we can afford to lose them if something goes wrong.
In the example deployment, Hono comes with a single router for simplicity. In real-world deployments, routers are usually deployed in networks. Often, full-mesh networks are used. With such a topology, we achieve two things. First, we improve the scalability of the system as clients are distributed over different routers. Second, we improve the reliability of the system, as it can easily survive crashes of one or more routers while remaining functional.
We have covered messaging needs for telemetry traffic. But what about events and command messages that require a genuine guarantee of delivery? Dispatch routers can be configured to “auto link” certain addresses to external AMQP 1.0 containers. In this way, we can route messages through the broker queue and provide “at least once” delivery guarantees for events and commands.
In the default deployment, Hono uses Apache ActiveMQ Artemis Broker. It provides a scalable AMQP 1.0 message broker; its queuing capabilities are used to store event messages.
Of course, you can use any AMQP 1.0-compatible message broker for this. Or provide more scalable and more reliable solutions by using broker clusters and high-availability configurations. In addition, the default deployment of a single router and broker can be replaced with a scalable AMQP 1.0 cloud messaging platform – such as the one described later in this article.
All solutions connect to the AMQP 1.0 network and receive messages from the messaging endpoints.
Hono Messaging and the protocol adapters report different kinds of metrics to an InfluxDB, which can be viewed using the Grafana dashboards provided. Please refer to Getting Started for details.
APIs and their endpoints
Hono’s internal APIs are defined as AMQP 1.0 endpoints:
Protocol adapters use Hono’s internal APIs to:
- authenticate the device using the Credentials API
- check the status of the device using the Registration API
- forward device messages to the AMQP network using the Telemetry or Event API
As with the AMQP 1.0 network, flow control also enables the load protection regarding devices.
The Device Registry also implements RESTful variants of the Registration APIs and the Credentials APIs in order to support the administration of devices. This leverages the use of CLI tools like curl and simplifies implementation of a complex Device Operations Console (which is not part of Hono itself).
All microservices are built as Docker images and are available on Docker Hub. The following container management systems are supported based on the Docker images:
- Docker Swarm
The supplied deployment scripts start a preconfigured orchestration of the microservices for a fully functional installation. They are contained in Hono’s example module.
Deployments can be executed either on premise or hosted in a public cloud. On premise deployments work well for a small number of devices or test setups; a public cloud deployment is better for large installations that can scale dynamically per number of connected devices.
You can set up Hono yourself or use the sandbox deployment, which is accessible at hono.eclipse.org. The following steps assume you have Hono running locally:
- Register a device by using the REST API of the device registry
- Observe the consumer receiving the data.
Scale it up!
Hono is fundamentally designed for horizontal scalability. This is reflected in its microservices architecture:
- protocol adapters can be scaled independently from the messaging infrastructure itself.
- solutions are decoupled from Hono by the AMQP 1.0 network layer, which can be scaled independently from Hono itself.
- the device registry is separated from the messaging components and, consequently, can be scaled independently as well.
You can use the scaling strategy of your container management system to scale the microservices you need. The internally used AMQP 1.0 protocol is designed for scalability and very large networks.
EnMasse – the elastic messaging infrastructure
Because Eclipse Hono is not itself a messaging system, it needs an underlying messaging infrastructure to work. Messages are exchanged between all internal components and sent from external devices to and from business applications. We refer to this type of infrastructure as the “AMQP 1.0 Network” in the overall Eclipse Hono architecture. As noted elsewhere, the off-the-shelf solution consists of the Qpid Dispatch Router and the ActiveMQ Artemis Broker.
The Eclipse Hono architecture was designed and developed in such a way that the underlying messaging infrastructure can be modified – provided that it is based on AMQP 1.0. This means you can use RabbitMQ with the AMQP 1.0 plugin, for example, or any other messaging broker implementing the AMQP 1.0 protocol.
Big numbers? We need elasticity!
When it comes to supporting a large number of connected devices and high throughput in terms of exchanged messages, the “single router, single broker” solution has its limits.
The messaging infrastructure needs to grow, and the scalability becomes the most important criterion for allowing Eclipse Hono to handle devices and consumers as well as the tremendous traffic associated with them.
A major problem in this case is deployment. How can we deploy the messaging infrastructure simply, without having to address the related complexity? Besides scalability, you also need high availability and resiliency.
In short, your messaging infrastructure needs to be elastic so it can support spikes in the numbers of connected devices and traffic generated, so that computational resources are then used productively. It would be a waste to run too many AMQP 1.0 brokers for a low number of devices and for low traffic. At the same time, such an infrastructure needs to grow as numbers increase. As that can change frequently, you need elasticity.
The answer? The EnMasse project!
How to address all the challenges mentioned above? The EnMasse project comes to the rescue!
EnMasse is an open source “messaging as a service” platform which simplifies the deployment of a messaging infrastructure both “on premise” and in the cloud. It provides the scalability and elasticity needed to support the messaging requirements for IoT use cases.
Moreover, EnMasse supports all common messaging patterns (request/reply, publish/subscribe, and competing consumers) and two main protocols: AMQP 1.0 and MQTT. Adding HTTP support is on the roadmap.
EnMasse provides multi-tenancy, meaning that different tenants can share the same infrastructure – but are isolated from each other. Finally, it provides security with respect to securing connections using TLS, but also authenticating clients using Keycloak as the identity management system.
Another aspect that makes EnMasse appealing to use with Eclipse Hono: it is completely containerized and runs on key container orchestration platforms such as Kubernetes, Docker Swarm, and OpenShift. It is an excellent complement to Hono’s microservice architecture and deployment models portrayed in preceding paragraphs.
As you can see, EnMasse offers all the features needed to be the messaging infrastructure that supports Eclipse Hono. It is easy to deploy both together in OpenShift; you can find more information on that in the step-by-step guide.
Conclusion and outlook
This article covers the basics of the Eclipse Hono project as well as describes its APIs, architecture, and deployment options. There are many things still to come in this field. The Telemetry and Event APIs we described are only the first that have been implemented. Next on our to-do list: the Command and Control and Tenant APIs.
The Command and Control API will provide a way for business applications to reliably send commands to the devices. The Tenant API will provide platform operators with the resources necessary for managing tenants (accounts).
We will also improve the EnMasse integration, particularly as regards shared security models and scalability testing.
We hope that we have increased your interest in IoT connectivity in general and the Eclipse Hono project in particular. We have a lot planned, and hope to see you in the community!