Post
Ask Question

Intel OpenVINO Model Optimizer

Introduction

 
In this article, we will be discussing the Intel OpenVINO Model Optimizer in detail. Before starting with Model Optimizer, I will discuss some commonly supported deep learning formats by Intel OpenVINO.
 

Caffe

 
Caffe is a profound framework of learning that reflects expression, speed, and modularity. It is developed by BAIR and community contributors. The research is based in Berkeley. During his Ph.D. at UC Berkeley, Yangqing Jia created the project. The BSD 2-Clause license releases Caffe. 
 
Features
  1. Expressive Architecture
     You can change between CPU and GPU with the use of just one flag.

  2. Extensible Code
     It is state-of-art in both code and model

  3. Speed (60M images/day)
     With a single NVIDIA K40 GPU With the ILSVRC2012-winning SuperVision model and prefetching IO, Caffe can process more than 60 M images a day. That’s 1 ms/image for inference and 4 ms/image for learning.

  4. Community
     Since it is an open-source project, there is a big and efficient community of developers, researchers, and students to back it.
To learn Caffe, there are a lot of tutorials and examples available on https://caffe.berkeleyvision.org/.
 

Tensorflow

 
It is an open end-to-end machine learning framework. It provides a full and scalable ecosystem of instruments, libraries, and community tools that enables scientists to advance the new ML developments and developers to construct and deploy ML applications quickly. It was developed by Google Brain Team in the year 2015.
 
To learn more about TensorFlow, you can visit A Complete Python Tensorflow Tutorial, this is an article that I have written as part of the C# Corner Machine Learning Learn Series, which you can find here. Or you can visit https://www.tensorflow.org/.
 
Tensorflow has some commonly used pre-trained machine learning models for ease, you can download them from TensorFlow Model Garden from https://github.com/tensorflow/models.
 

Apache MXNet 

 
It offers an extensive and flexible Python API for a wide community of developers with different levels of experience and broad requirements. It was developed by Apache Software Foundation in the year 2017. Here "MX" means "Mix and Maximize".
 
Features
  1. Hybrid Front-End
  2. Distributed Training
  3. Eight Language Binding i.e. Scala, Julia, Clojure, Java, C++, R, Perl
  4. Tools and Libraries
Apache MXNet ecosystem has a variety of many libraries, tools, and many more to support Deep Learning Development and Research. The ecosystem has the following toolkits,
  1. GluonCV
    It is a Computer Vision toolkit. You can visit https://gluon-cv.mxnet.io/ for more details

  2. GluonNLP
     It is a Natural Langauge Processing Toolkit. You can visit https://gluon-nlp.mxnet.io/ for more details

  3. GluonTS
     Here TS means Time Series, it aims to provide an effective mechanism for probabilistic time series modeling. You can visit https://gluon-ts.mxnet.io/ for more details.
To learn about Apache MXNet, you can visit the https://mxnet.apache.org/versions/1.6/
 

ONNX

 
ONNX stands for Open Neural Network Exchange. It is an open format that represents models for machine learning. ONXX defines a common set of operators, machine-learning building blocks and deep learning models, and a common file format for AI developers to use models with a wide range of frameworks, instruments, runtimes, and compilers. It was developed by Facebook and Microsoft Corporation in the year 2017.
 
The ONNX Model Zoo is a set of mature, pre-trained ONNX models that community members like you. Jupyter notebooks for model testing and inferences on the learned model follow each model. The notebooks are written in Python and contain links to both the preparation and references to the original paper explaining the model architecture. You can find the whole list of models at https://github.com/onnx/models.
 

Kaldi

 
It is a toolkit for speech recognition written in C++ and licensed under the Apache Licensed 2.0. It is developed by Daniel Povey. It got its name from the Ethiopian goatherder who discovered the coffee plant
 
It has 3 separate codebases for deep neural networks:
  1. nnet1
    Located in the code subdirectory nnet and nnetbin

  2. nnet2
     Located in the code subdirectory nnet2 and nnet2bin

  3. nnet3
     Located in the code subdirectory nnet3 and nnet3bin
Some of the pre-trained Kaldi models can be found at https://kaldi-asr.org/models.html. To learn about it visit https://kaldi-asr.org/doc/install.html.
 
Model Name File Extension
Caffe .caffemodel
Tensorflow .pd
MXNet .params
ONNX .onnx
Kaldi .nnet
 

Key Terms

 

Quantization

 
Reduce the precision of weights and biases, thereby reducing compute time and size with some loss of accuracy.
 

Freezing

 
It is used on layer-basis, to fine-tune, a neural network. In TensorFlow, it removes metadata, which is only intended for training purposes.
 

Fusion

 
Combining certain operations together into one operation and needing less computational overhead.
 

Model Optimizer

 
The Model Optimizer is a Python*-based command-line tool for importing trained models from popular deep learning frameworks such as Caffe*, TensorFlow*, Apache MXNet*, ONNX*, and Kaldi*.
 
The Model Optimizer is a key component of the Intel® OpenVINO™ Toolkit. You cannot perform inference on your trained model without running the model through the Model Optimizer. When you run a pre-trained model through the Model Optimizer, your output is an Intermediate Representation (IR) of the network.
 
An IR is a special type of file that an Intel® OpenVINO™ Toolkit uses as an input, and all the processing, post-training fine tunings, and optimizations are done using this IR form.
 
It consists of two types of files,
  • .xml: Describes the network topology
  • .bin: Contains the weights and biases binary data
The model optimizer is primarily used to convert the Caffe*, TensorFlow*, Apache MXNet*, ONNX*, and Kaldi* models into the IR form. To convert Apple Core ML models to IR form, you need to convert them to ONNX format.
 
You can either install Model Optimizer for an individual model scheme or you can also install a universal Model Optimizer which can convert any model scheme to IR form.
 
To install Universal Model Optimizer
 
Go to the Model Optimizer prerequisites directory:
  1. cd /opt/intel/openvino/deployment_tools/model_optimizer/install_prerequisites
Run the following script:
  1. sudo ./install_prerequisites.sh
To install scheme specific Model Optimizer
 
Go to the Model Optimizer prerequisites directory:
  1. cd /opt/intel/openvino/deployment_tools/model_optimizer/install_prerequisites
  1. Caffe
    1. sudo ./install_prerequisites_caffe.sh
  2. Tensflow
    1. sudo ./install_prerequisites_tf.sh
  3. MXNet
    1. sudo ./install_prerequisites_mxnet.sh
  4. ONNX
    1. sudo ./install_prerequisites_onnx.sh
  5. Kaldi
    1. sudo ./install_prerequisites_kaldi.sh

Conveting a TensorFlow Model to IR Form 

  1. Execute the above mentioned install script to configure your system to be able to convert .pb file to .xml and .bin files.
  2. Freeze the TensorFlow model by executing the following code, skip this step is the model is already frozen. 

    Note
    the following code is for TensorFlow 1, so it may be possible that the functions may have deprecated, so please use the latest function
    1. import tensorflow as tf  
    2. from tensorflow.python.framework import graph_io  
    3. frozen = tf.compat.v1.graph_util.convert_variables_to_constants(sess, sess.graph_def, ["name_of_output_node"])  
    4. graph_io.write_graph(frozen, './', 'inference_graph.pb', as_text = false)  
    1. sess is the instance of the TensorFlow session object where the network topology is defined
    2. ["name_of_output_node"] is the list of output node names in graph. Frozen graph will only include those nodes from the original graph_def that are directly or indirectly used to compute given output nodes
    3. dot slash is the directory where inference graph file should be generated.
    4. as_text specifies whether the generated file should be in human readable text format or binary

  3. Run the following code to convert the frozen TensorFlow model to IR form,
    1. cd C:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\model_optimizer  
    2. python mo.py --input_model ["name"].pb --tensorflow_object_detection_api_pipleline_config pipeline.config --reverse_input_channels --tensorflow_use_custom_operations_config extensions/front/tf/ssd_v2_support.json  
    Here I am using ssd_v2_support.json, because i generally use SSD version 2 models, you can change this as per the model you are using. For example, if you are employing Faster RCNN, use faster_rcnn_support.json.

    Note
    here instead of mo.py you can use mo_tf.py

Converting a Caffe Model to IR Form

  1. Execute the above-mentioned install script to configure your system to be able to convert .caffemodel file to .xml and .bin files.
  2. Run the following code to convert the Caffe model to IR form:
    1. cd C:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\model_optimizer  
    2. python mo.py --input_model [model_name].caffemodel --input_proto deploy.prototxt  
    We have added '--input_proto" as here the name of the protobuf file is not same as the .caffemodel file

    Note
    here instead of mo.py you can use mo_caffe.py

Converting an ONNX Mode to IR Form

  1. Execute the above-mentioned install script to configure your system to be able to convert .onnx file to .xml and .bin files.
  2. Run the following code to convert the ONXX model to IR form:
    1. cd C:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\model_optimizer  
    2. python mo.py --input_model [model_name].onnx  
    Note
    here instead of mo.py you can use mo_onxx.py

Converting a Kaldi Mode to IR Form

  1. Execute the above-mentioned install script to configure your system to be able to convert .nnet file to .xml and .bin files.
  2. Run the following code to convert the Kaldi model to IR form:
    1. cd C:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\model_optimizer  
    2. python mo.py --input_model [model_name].nnet  
    Note
    here instead of mo.py you can use mo_kaldi.py

Converting an MXNet Mode to IR Form

  1. Execute the above-mentioned install script to configure your system to be able to convert .params file to .xml and .bin files.
  2. Run the following code to convert the MXNet model to IR form:
    1. cd C:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\model_optimizer  
    2. python mo.py --input_model [model_name].params  
    Note
    here instead of mo.py you can use mo_mxnet.py

Conclusion

 
In this article, we studied in detail about Intel OpenVino Model Optimizer. In the coming articles, I will introduce you to the Intel OpenVINO Python API.


Similar Articles