Hello Reader,

Welcome to another edition of the AIFEE newsletter!

Another week, another Machine Learning topic!

This week, I want to share more insights about ONNX framework.

What is ONNX?

​

​

In last week's edition, I shared with you different approaches you can use to deploy your Tensorflow models in C++. One of these approaches is by using ONNX Runtime.

​
So ONNX (Open Neural Network Exchange) is a framework or a whole ecosystem that allows for the standardization of neural networks.
​
Here’s the problem that ONNX is trying to solve.
​
There are many machine learning frameworks and libraries. Pytorch, Tensorflow and scikit-learn are famous examples.
​
Which one should you use?
​
Well, the answer is : it depends on so many factors!
​
There is no free launch!
​
So what ONNX aims to do is to make your choice easier. Basically, you can choose whichever framework you want, train your model with it and then, once you’re happy with your model, transform it into ONNX format and keep the production code working in ONNX only.
​
So ONNX will help you achieve 3 things:

• Keep your options open when it comes to choosing a machine learning library for your training.
• Standardize your production code, since you’ll only need to maintain one main dependency which is for ONNX.
• Moreover, you can use your ONNX models in Python, C++ or other languages.

Transforming models to ONNX format

If you're a Tensorflow developer or PyTorch developer then the first step for you to use ONNX in production would be to transform your model to ONNX format.
​
To transform your Tensorflow models into onnx format, you can use a tool called tf2onnx. The image below shows how to use this tool.

​
You can learn more about this tool on their github repo.
If you’re a Pytorch user, then you can transform your model from torch to onnx format directly using Pytorch. This is done using the module torch.onnx.
An example of this can be found here.

Deploying ONNX models in C++

Transforming your models to ONNX format is just the beginning. After this, you can deploy your model using different programming languages.

To deploy your ONNX models in C++, there is a library called ONNX Runtime which you can use to load your ONNX models and make predictions.

If you’re a Visual Studio user (not VS Code!) then the easiest way to install this library is by using NuGet packages. You can simply go to “Project” then “Manage nuget packages”. In the search bar you type “onnx runtime”.

After that you click on “install”.

With this library, you can load any onnx model that you converted from Tensorflow or Pytorch.

Many examples can be found in the docs.