Did you know that you can now train machine learning models with Azure ML once and deploy them in the Cloud (AKS/ACI) and on the edge (Azure IoT Edge) seamlessly thanks to ONNX Runtime inference engine.

In this new episode of the IoT Show, learn about the ONNX Runtime, the Microsoft built inference engine for ONNX models – its cross platform, cross training frameworks and op-par or better performance than existing inference engines.
From the description:
We will show how to train and containerize a machine learning model using Azure Machine Learning then deploy the trained model to a container service in the cloud and to an Azure IoT Edge device with IoT Edge across different HW platform – Intel, NVIDIA and Qualcomm.

In this article from VentureBeat, read about Scott Guthrie’s excitement about ONNX.

“Even today with the ONNX workloads for AI, the compelling part is you can now build custom models or use our models, again using TensorFlow, PyTorch, Keras, whatever framework you want, and then know that you can hardware-accelerate it whether it’s on the latest Nvidia GPU, whether it’s on the new AMD GPUs, whether it’s on Intel FPGA, whether it’s on someone else’s FPGA or new silicon we might release in the future. That to me is more compelling than ‘do we have a better instruction set at the hardware level’ and generally what I find resonates best with customers.”

ONNX is a open format to represent deep learning models that is supported by various frameworks and tools. This format makes it easier to interoperate between frameworks and to maximize the reach of your hardware optimization investments

In this episode, Seth Juarez (@sethjuarez) sits with Rich to show us how we can use the ONNX runtime inside of our .NET applications. He gives us a quick introduction to training a model with PyTorch, and also explains some foundational concepts around prediction accuracy.

Useful Links