Subscribe Find out first about new and important news
  • Share via email
  • Subscribe to blog alert

Lessons learned using Kubernetes in IoT deployments

3 min

Kai Hudalla

Kai Hudalla has been working at Bosch on solutions for the Internet of Things in application domains like smart home, energy management, e-mobility, and automotive long before the term reached its recent popularity and ubiquity. He is an active committer on several Eclipse IoT projects, one of which is the Eclipse Hono project, aiming at providing an integration platform for connecting millions of devices to the cloud. Kai is a regular speaker at conferences and enjoys cooking and golfing whenever weather conditions permit.

At Bosch.IO, we have been using Kubernetes as the core infrastructure framework for managing and deploying the services that make up the Bosch IoT Suite. Kubernetes has allowed our agile development teams to develop a microservices architecture. This allows each team to focus on the specific services of the Bosch IoT Suite that then get integrated into the overall IoT platform.

Most of the technology used in Bosch IoT Suite is developed in open source projects. Two of these projects, Eclipse Hono and Eclipse Ditto, have been leading the way in our adoption of Kubernetes. The Eclipse Ditto project is made up of 6 microservices that are deployed as Kubernetes pods and can be scaled horizontally. The Ditto team uses Kubernetes to improve the reliability and scalability of the Ditto services. Eclipse Hono has 7 microservices that are managed by Kubernetes. They use Kubernetes to make it easier to integrate with and use other open source projects, like EnMasse, Grafana, InfluxDB and Keycloak.

The Bosch developers working on Eclipse Hono and Eclipse Ditto have learned a lot about using Kubernetes for IoT solutions. Are you considering using Kubernetes? Here are some of our lessons learned.

Lessons learned for using Kubernetes

Thomas Jäckle

Thomas Jäckle is a software developer and has been working at Bosch since 2010. Since 2015 he holds the position of lead software developer of the Bosch IoT Things cloud service, and since 2017 he is committer and project lead of the Eclipse Ditto project. He is passionate about software craftsmanship; continuous learning drives his motivation. Thomas holds a Bachelor’s degree in computer science and a Master’s degree in application architecture, both from the Hochschule Furtwangen University in Germany.

  1. Use a managed Kubernetes service.  Running your own Kubernetes service adds extra overhead that you might not need to take on. Due to the popularity of Kubernetes, there are a number of options for using a managed Kubernetes service, including AWS, Google Cloud, MS Azure, and more. Using a managed service means you don’t need to worry about installing security patches or updating to the latest version of Linux required to run Kubernetes. Running on a managed Kubernetes service allows your developers to focus on the IoT application. They don’t have to worry about the underlying infrastructure.
  2. Use the base Kubernetes distro to ensure portability. Kubernetes makes it easy to move your services between different Kubernetes installations. Some organizations might want to move an IoT application from a self-managed instance to a publicly managed instance of Kubernetes, or even between public providers. If you want this level of portability, it is imperative you use the base Kubernetes distro. Some publicly managed providers will make it possible to use custom plugins that might not be available or supported on the other service. The base Kubernetes distro is certified to ensure portability.

  1. Use some of the popular tools for Kubernetes. There are many tools available to help make it easier to manage and deploy containers using Kubernetes. Some of the tools we found to be most useful include:
  • kops to create new clusters from a command line
  • Helm for deploying a service into an existing cluster
  • Fluentd to collect log files that can then be sent to Elasticsearch and visualized using grafana and influxdb
  • Kubernetes Dashboard provides a nice UI to inspect the usage of a node resource
  1. Install and use a distributed trace utility. For complex systems, debugging problems across services can be a challenge. This is why we recommend using a distributed trace utility, such as Opentracing or Jaeger Tracing.
  2. Importance of understanding the big picture. Kubernetes is a sophisticated and at time complex system to understand. Therefore, it is important to educate yourself on the overall big picture to understand the core concepts, architecture and how the systems interact with each other. We suggest to anyone new to Kubernetes that they read one of the many great books about Kubernetes, e.g. Kubernetes in Action or Kubernetes Up and Running, and take the time to read the Kubernetes documentation.
  3. Keep up to date on Kubernetes. Kubernetes is a fast changing community and technology. For instance, the Kubernetes API is undergoing constant change/extension/deprecation. It is important to keep up to date with these changes in order to not break your deployment. We always recommend to read the release notes for each release and follow the Kubernetes open source community on Github.

More on Kubernetes & open source

The IoT needs Kubernetes. Let us explain why.

We adopted Kubernetes for the Bosch IoT Suite. What led to this decision?

Bosch pursues an open source strategy. Find out why.

The goal of the open source project Eclipse Ditto is to establish a framework for digital twins. Take a closer look.

How the open source project Eclipse Hono addresses the topic of IoT device connectivity.