![]()
The Internet Of Things plays an important role in any manufacturing process, and this role will keep growing in the next years:
"The IoT market reached $761.4 billion in 2020 and is expected to reach $1.39 trillion by 2026, according to Mordor Intelligence."
Source: Internet of Things (IoT) Market 2022 | Datamation
If you are looking for a safe job in the future, then it is worth learning IoT stuff. Another important field is AI/Machine Learning (Data Science) for the future.
I have decided to write this year a series of articles about IoT. I will cover all the stuff that can make you an Azure IoT Expert (Development and Architecture). I will start with the basics and connectivity and continue until the user interfaces and reporting. In addition, I will cover a lot of business integration use cases. If you are interested in doing more in the IoT world, you can follow me here in C# Corner or on LinkedIn or Twitter.
Bassam Alugili | LinkedIn or Bassam Alugili (@BassamAlugili) / Twitter
About This Course
In the beginning, I will cover various topics relevant to the basic understanding of IoT. After that, I will handle each level more deeply in the following articles. Finally, I will create an open-source project based on the Azure IoT digital solution as a sample. This is an open invitation to anyone who wants to join my new open-source project.
Why Azure?
![]()
As you can see in the image above, Azure IoT is leading the market. So, it means for you; if you are going to learn Microsoft Azure IoT, it is not hard to find a job compared to if you are going to learn Siemens Mindsphere.
What is IoT?
IoT applications can be described as devices (Things), sending data or events that are used to generate Insights or views, which are used to generate actions to help improve your business or process.
![]()
What are the biggest challenges in IoT?
1) Business Requirements Challenges
This is about finding the correct business cases. Which use cases do the customers need? It is not easy to see it, but recognizing your customers' needs in the early phase is the key to success.
2) Connectively Challenges
How can you connect your devices to the internet? How can you collect data from your devices? This is one of the biggest challenges in the internet of things world. When designing an Internet Of Things Gateway, you have to think about all use cases (Field/Cloud). In good cases, the IoT devices can connect to the internet with an IP address, and in other use cases, you have to be more creative in finding a solution for the connecting problems. In other words, how can you connect those devices without IPs to the internet?
3) Security Challenges
How can you make a secure connection to each device? How can you make secure communication channels for millions of devices and handle all those devices in the same way? It would be best to answer all those questions when working on a large-scale IoT solution.
4) Big Data Challenges
You have big data problems if you have one or more of the three V's of Big Data.
The Meaning of Big Data - 3 V's
- Big Volume
- Big Velocity
- Big Variety
Big Volume: The data generation size in Terabytes and Petabytes.
Big Velocity: The data generation and gathering are very fast.
Big Variety: The data comes in many different forms, both structured and semi-structured.
It is important to understand here all those terms:
- Business Intelligence
- Data Mining
- Analytics
- Big Data
In the last years here in Germany, I have seen many big companies have many technical debts because they are looking for the most straightforward solutions, like using one relational database for distributed microservices problems, because they do not want to lose the ACID benefits. After that, the developers have to spend the rest of their lives solving the performance problems in the database. So, in the first place, it is important to understand the technical problems, check where you are, and use the proper techniques to solve them.
5) Machine Learning Challenges
If you do simple linear algebra mathematics to find the desired prediction model, it does not matter which programming language you use. However, if you are doing more sophisticated machine learning mathematics, I recommend using a proper programming language to solve your machine learning problems.
6) DevOps or NoOps Challenges
Typically, DevOps adds more work to your development team, and you need more resources or developers for that. However, suppose your company is small, and they do not have enough resources to do that. In this case, I recommend you use serverless architecture like Microsoft Azure Functions, which enables businesses to focus on their primary service rather than on underlying IT infrastructure like networks, securities, and scaling management.
Alternatively, you can look for a complete IoT solution; for example, many companies offer that like Körber Digital.
7) Leading and Managing the Digital Solution Challenges
We also have here a lot of challenges for the manager. There are many competitive players in the IoT world, and you must know that you are building something in a highly competitive market. All big names are involved in IoT, like Amazon/Microsoft/IBM/Google/Alibaba and others. It is a big challenge to find a marketing segment with those names. You must be wise to find intelligent developers and manage all kinds of those teams. Agile is an excellent approach to managing IoT projects, but I would use hybrid agile project management.
Requirements
You have to identify and recognize your internal and the customer's requirements early. Next, you must be good at thinking about the use cases. For example, most companies send all hardware and sensor data to the cloud because the marketing people want to add the keyword 'IoT-Support' to make more sales. But finally, when the customers need to do predictive maintenance, they will recognize that they cannot do that, and a lot of internal hardware information is missing. Therefore, they need to upgrade the software and hardware, and maybe that will take years until the desired functionality works correctly.
Connectivity and Gateways
You must have a basic understanding of the internet protocols like TCP/UDP, SSL/TLS, and WebSocket.
Please first watch those videos on YouTube and understand the basics of internet protocols.
- Network Protocols - ARP, FTP, SMTP, HTTP, SSL, TLS, HTTPS, DNS, DHCP - Networking Fundamentals - L6
- Introduction to TCP/IP and Sockets, part 1: Introducing the protocols and API
Message-Oriented Architecture
You need two kinds of messages in the IoT world, data messages and domain events. You can use for the data messages a lightweight protocol like MQTT with a small footprint and a compact structure.
You can use the Advanced Message Queuing Protocol for your domain events. AMQP is reliable and supports queuing, routing, and security.
It would help if you could also learn the basics of message-oriented architecture.
AMPQ/MQTT/HTTPS
As I mentioned before, AMQP is very suitable for domain events. For a simple reason, The AMQP supports ACK management, where the subscriber can easily reject or deny a message/event. This technique will help you to design a reliable system with more fault tolerance.
MQTT is the best fit for sending the data to the cloud or the gateway from the devices and vice versa.
HTTPS, I do not recommend using it at all.
Data Storage
You have to learn the NoSQL databases like CosmosDb and relational databases. But, first, you need to know the basics of all five databases areas.
- Relational database,
- Key-Value,
- Document Stores,
- Column Family Datastores,
- Graph Databases,
In addition, I would talk about time-series databases. This kind of database is essential for the IoT because of the nature of the data store. I will handle this topic in-depth in the following articles.
Stream Analytics
Stream Analytics ingests data from the Azure IoT Hub. Stream Analytics jobs then process the data using SQL transformation queries to filter, sort, aggregate, and/or join the streaming data. The data output type is specified. Data can be sent to queues that then trigger alerts or workflows. It can be visualized in real-time through tools such as Power BI. Data can also be sent to the data lake for the training of machine learning models.
In the following article, I will write about a simple real-time telemetry stream analyzer for IoT devices. I will try to do it very simply by writing basic queries that filter the data and make decisions.
Watch this video:
Summary
IoT is a huge challenge for developers. You need solid knowledge in many independent fields. If you want to be an expert in IoT, you have to teach yourself everywhere. But it is worth learning that stuff for two reasons. The first reason you can easily find a job in the IoT field, and the second reason is the knowledge you will learn will make you a non-beatable developer.
Please read/watch the above-given links and articles to prepare you for the next article. In the following article, I will write about a simple use case.
- New Open-Source Project
- Creating a Could Gateway with Azure IoT Hub
- Connecting devices to the Cloud Gateway
- Provisioning or Shared Key
- Filtering the telemetries in the Stream Analytics
If you like my article, do not forget to share this article and to follow me. Thank you a lot!