Smart Labeling With IoT Device Using Deep Learning

This article introduces various aspects of Machine Learning and Deep Learning and presents an analogy to solve a real work problem of labeling through a process of the amalgamation of a stack of technologies with which labeling can be achieved in real-time. 

For decades now with the inception of computers, we’ve exercised the usage of traditional programming. In traditional programming, specific rules and data are provided, based upon which output is executed. The systems were based on basic logic and conditional statement used for it. But as we progressed over the years and Moore’s Law proved its significance in hardware development, it has also enabled tremendous growth prospects and scope for Machine Learning and Deep Learning. 

Machine Learning

Machine Learning is a science and art through which machines can learn from data. Machine Learning is a subject of AI and it is an approach to solve numerous problems. From Computer Vision to Natural Language Processing to Analysis in Stock Market and Healthcare, Machine Learning is everywhere. It is tremendously powerful and has a subset under it to solve even exponentially difficult problems. 

To learn in-depth between Machine Learning and various key elements of Artificial Intelligence, do check out the previous article, Artificial Intelligence Overview. 

Deep Learning

Deep Learning is a subset of Artificial Intelligence and Machine Learning which mimics the usage of the human brain to process data and create pattern and use it for decision making. Deep learning also commonly known as deep neural networks, uses multiple levels of layers to extract features using the input data progressively.  

Dataset

Dataset refers to the collection of data which consists of lists of data for specific variables or objects. Dataset may also contain collections of documents or files in different forms.  

Observation

The observations are the different values generated from experiments or the demographics. These are usually the raw data observed for different variables. Usually, in the case of experimental data or sensor-generated values, from the speed of the vehicle during autonomous driving or distance, temperature, and so on are observation values. 

Features

The sets of data found in the observation consist of the features which are used to make predictions.  

Dataset with Real World Scenario

The Dataset created in a real-world scenario would contain the observation with different sets of features such as Temperature, Luminosity, Infrared, Distance, Time, and so on. 

Labeling

Data Labeling refers to the identification of raw data such as text files, images, videos, sounds, and associating it with something thus adding a meaningful label to it which provides the context to that data providing context for machine learning models to learn from it.  Below, is an image with labeled names for different objects detected. 

Pre-Labeling

Pre-labeling is cost-effective but needs human intervention or a process of automation for supervision done during the near past.  

Real-Time Labeling

Real-Time Smart Labeling can replace humans for tasks as automated labeling can be performed with AI using computer vision by labeling data in real-time itself.  

We can use IoT devices to detect a specific set of data. By training high performing deep learning model, we can take the train the device to perform better than humans. Using a Raspberry Pi and Camera, we can automate the labeling system with minimum human intervention.  

Raspberry Pi

Raspberry Pi is a small mono-board computer that was originally designed with the vision to enable the teaching and learning of computer science in developing nations. It is extremely economical and are the size of debit cards that can be plugged into monitors and allows the usage of input devices such as mouse and keyboard. 

With other sensors connects to Raspberry Pi, we can perform huge applications such as smart labeling using Computer Vision.  

Sensors

Sensors are devices that are used to measure physical inputs from environments and transform it into data. Accelerometer, Ambient Light Sensor, Finger Print Sensor, Gyroscope Sensor, Radiation Sensor are some of the most used sensors in smartphone devices today. For Raspberry Pi, Luminosity Brick Sensor, Temperature Brick Sensor, Ultrasonic Sensor, Infrared Sensors are easily available for use as per the application to be designed.  

Sensor Sources

Flashlight, Lighter, Infrared, Daylight all act as Classes for the dataset which can be the sources for the different types of sensors.  

To learn more about Smart Labeling with ML.NET, watch this video. 

TensorFlow

TensorFlow is developed by Google as an open-source library to provide an end-to-end platform for machine learning with a focus on the inference of deep neural networks. It is one of the most used open-source frameworks for Machine Learning and Deep Learning and to use computation graph. It supports Python, Java, C++, and C#. It can be used with C# through the ML.NET framework for TensorFlow with ease.  

TensorFlow can be used for multiple purposes of Artificial Intelligence tasks such as following,

• Computer Vision: Image Recognition, Object Detection, Face Detection 
• Voice / Sound Recognition 
• Time Series Analysis  
• Language Translation 
• Text-based Processing  
• Handwriting Recognition 

Currently, TensorFlow is only supported in ML.NET for inference/ prediction using transfer learning with existing TensorFlow models, previously built on other platforms.  

Inception

Inception-v3 is a convolutional neural network that is 48 layers deep that has an image input size of 299-by-299 and contains 1000 classes (object categories) such as animals like dogs, mouse, and objects like pencil, house, and more. We can train for new classes using transfer learning. The inception model has been able to attain an accuracy of over 78.1% on the dataset of ImageNet. This model is the amalgamation of numerous researchers and ideas over many years of work.  

Transfer Learning

Transfer Learning can be understood as a machine learning method that utilizes a pre-trained neural network that focuses on the storing of knowledge that is gained over time while solving a problem and then uses it in relatable similar other problems. For instance, the image recognition model known as Inception v3 consists of two parts:  

• Feature Extraction
  Feature extraction is the process of extracting features from data by dimensionality reduction approach through which raw data is reduced to manageable groups that are then processed. It is commonly used in Computer Vision problems.  

• Classification
  Classification is a predictive modeling approach such that a class label is predicted for the example of the input data such as spam identification by labeling if the data is spam or not, handwriting recognition and so on.  

Training an Inception v3 network from scratch is a computationally intensive task and depending on your compute setup may take several days or even weeks.  

ML.NET

ML.NET is a machine learning framework designed to cater to .NET developers. There are currently over 6 million .NET developers across the globe and to enable the .NET developers with machine learning capabilities, ML.NET is a get-go. It is totally free and this machine learning library can be used for both C# and F#. 

Read this article, to learn more about ML.NET 

Netron

Netron is a visualizer dedicated to deep learning, neural network, and machine learning models. It can be accessed from Github.  

Smart Labeling Architecture

Labeling is a cumbersome task and a mundane activity to most of us. It isn’t that challenging or fun in a way and is so repeatable that we could actually designate this work to a machine. With the learning of above and the understanding that labeling is a classification problem, through various technologies used together labeling can be done through a computer and camera using some artificial intelligence.   

Smart Labeling Architecture is the architecture to label the objects using Artificial Intelligence in Automation without any human intervention. This approach would enhance efficiency and effectiveness and sustainability. The IoT device, Raspberry Pi running on Linux would use the camera to scan and with smart sensors functioning at all times, will automate this system. The API would be fetched through the program that can be even executed from Jupyter Notebook. The model would be trained here and the interaction can be done with the User Interface through HTTP to Control Panel and display the data in real-time of the labeling. This way, the labeling can be automated using smart labeling architecture that includes Raspberry Pi and Camera Sensor through the usage of Deep Learning.   

Thus, in this article, we learned about Machine Learning, Deep Learning, and various components such as Dataset, Observation, and Features. We then learned about Labeling and pre-labeling and real-time labeling. There onwards, we had a brief introduction to Raspberry Pi and sensor, and later using all the lessons, we presented an instance of how all these technologies can come together to solve a difficult problem such as labeling. We also learned about the inception and Transfer learning.