ONNX – Giving Developers the Freedom to Explore New AI Frameworks

November 20, 2019

Developers are always on the quest to create increasingly more advanced deep learning models using different frameworks that give them more variety of choice. A new deep learning model representation called Open Neural Networks Exchange Format (ONNX) was created to do just that. ONNX was co-created by Microsoft and Facebook as an open source project in 2017, and it continues to be adapted and supported by a community of partners.The most important features of ONNX are its framework interoperability and shared optimization, which allows developers to easily move between different machine learning frameworks depending on the task. ONNX makes it easier and faster for optimization even if they are integrated in different frameworks.

A good example of the flexibility of ONNX is looking at how a Keras model can be converted to it. Keras is a high-level neural networks API that can run on top of TensorFlow. It is written in Python, and is easy to learn and use.

To accomplish this, we would need the following:

  1. Python 3
  2. Numpy
  3. Keras and a Tensorflow
  4. ONNX

Below is a sample Convolutional Neural Network (CNN) from the Keras documentation:

import keras
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense, Dropout, Flatten
from keras.layers import Conv2D, MaxPooling2D
from keras import backend as K
batch_size = 128
num_classes = 10
epochs = 2
# input image dimensions
img_rows, img_cols = 28, 28
# the data, split between train and test sets
(x_train, y_train), (x_test, y_test) = mnist.load_data()
if K.image_data_format() == ‘channels_first’:
x_train = x_train.reshape(x_train.shape[0], 1, img_rows, img_cols)
x_test = x_test.reshape(x_test.shape[0], 1, img_rows, img_cols)
input_shape = (1, img_rows, img_cols)
else:
x_train = x_train.reshape(x_train.shape[0], img_rows, img_cols, 1)
x_test = x_test.reshape(x_test.shape[0], img_rows, img_cols, 1)
input_shape = (img_rows, img_cols, 1)x_train = x_train.astype(‘float32’)
x_test = x_test.astype(‘float32’)
x_train /= 255
x_test /= 255
print(‘Model Input shape:’, input_shape)
# convert class vectors to binary class matrices
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)
model = Sequential()
model.add(Conv2D(32, kernel_size=(3, 3),
activation=‘relu’,
input_shape=input_shape))
model.add(Conv2D(64, (3, 3), activation=‘relu’))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(128, activation=‘relu’))
model.add(Dropout(0.5))
model.add(Dense(num_classes, activation=‘softmax’))
model.compile(loss=keras.losses.categorical_crossentropy,
optimizer=keras.optimizers.Adadelta(),
metrics=[‘accuracy’])
model.fit(x_train, y_train,
batch_size=batch_size,
epochs=epochs,
verbose=1,
validation_data=(x_test, y_test))

At the end at the example we added a model.save, as shown in figure 1 below. Models in Keras are saved with the HDF5 file format.

model.save(‘model.h5’)

Figure 1

Converting the model.

ONNX can be used through the command line or libraries in Python. We will be working with ONNX through Python. To be able to convert the Keras model to ONNX we need to import ONNXMLTools and from Keras the load_model function, as shown in Figure 2.

import onnxmltools
from keras.models import load_model

Figure 2

Then, we define the paths of our model.h5 file and the path of the output ONNX model. In Figure 3 we see an example.

# Update the input name and path for your Keras model
input_keras_model = ‘model.h5’
# Change this path to the output name and path for the ONNX model
output_onnx_model = ‘model_keras.onnx’

Figure 3

Now, we load the Keras model, as shown in Figure 4.

# Load your Keras model
keras_model = load_model(input_keras_model)

Figure 4

Finally, we convert the model calling the convert_keras function, as shown in Figure 5.

# Convert the Keras model into ONNX
onnx_model = onnxmltools.convert_keras(keras_model)
# Save as protobuf
onnxmltools.utils.save_model(onnx_model, output_onnx_model)

Figure 5

To load and test the ONNX model, run the code below to see its input shape. These are the same steps that would be taken to create a model in Keras.

import onnxruntime as rt
import numpy as np
sess_1 = rt.InferenceSession(‘keras_model. onnx’)
print(“The model expects input shape:”, sess_1.get_inputs()[0].shape)

Figure 6

Based on the example above, it’s clear that converting models to ONNX is pretty straight-forward and easy to understand. From our work here at Wovenware, we are finding that ONNX provides yet one more development model for deep-learning algorithms that are changing the way our customers solve their critical business challenges.

 

Leave a Reply

  • (will not be published)