[Tensorflow]Loading a tensorflow graph with the C++ API by using Mnist

"Tensorflow is an open source software library for numerical computation using data flow graphs. "
Tensorflow provides python API and C++ API. However, the document about loading a graph with C++ API is few. In some case, we need a C++ level api to run tensorflow.
Thanks to Jim Fleming write a complete loading graph example (link), saving my plenty of times.

I am based on Jim Fleming's article and add the mnist example to show
   - how to write datas into input tensors
   - how to read datas from output tensors
   - what need to be care when we initial input and output tensors

Requirement

  -  Install tensorflow from source code ,
     "GET STARTED" --> "installing from source"
  - Install bazel

Source code can be checked on my github repository
https://github.com/JackyTung/tensorgraph

Create Graph

source code : gengraph/

import tensorflow as tf
import time
from tensorflow.examples.tutorials.mnist import input_data
# === prepare tensorflow network === #

#config setting
imageDim = 784
outputDim = 10

mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)
# None means that a dimension can be any of any length
x = tf.placeholder(tf.float32, [None, imageDim], name="input")
# 784-dimensional image vectors by it to produce 10-dimensional vectors
W = tf.Variable(tf.zeros([imageDim, outputDim]), dtype=tf.float32, name="Weight")
# a shape of [10]
b = tf.Variable(tf.zeros([outputDim]), dtype=tf.float32, name="bias")
# softmax
y = tf.nn.softmax(tf.matmul(x, W)+b, name="softmax")

....

# Add ops to save and restore all the variables
saver = tf.train.Saver()

with tf.Session() as sess:
 sess.run(tf.initialize_all_variables())
#Training
    for i in range(1000):
        if i % 100 == 0:
            print "iteration num :", i
        batch_xs, batch_ys = mnist.train.next_batch(100)
        sess.run(train_step,feed_dict={x: batch_xs, y_: batch_ys})

    # Save checkpoint, graph.pb and tensorboard
    saver.save(sess, "models/model.ckpt") 
    tf.train.write_graph(sess.graph.as_graph_def(), "models/", "graph.pb")
    tf.train.SummaryWriter("board", sess.graph)
#Testing
.....    

when we create graph, need to care about
1. good naming on tensor.
2. the dimension of the input tensor and output tensor
(in this case, input tensor is x, output tensor is y)

Creating a simple library

source code : loadgraph/

If we create our own library, create a new folder like this : tensorflow/tensorflow/<my project name>.
In this case, I am going to call the project loadgraph.
In the folder loadgraph/, I create a file name mnist.cc (because it is a mnist example).

In mnist.cc, we do the following things:
1. Initialize a TensorFlow session.
2. Read in the graph we exported above.
3. Add the graph to the session.
4. Setup our inputs and outputs.
5. Write datas into input tensors
6. Run the graph, populating the outputs.
7. Read value from the outputs tensors.
8. Do the mnist prediction

Some steps is shown in Jim's article, in here, I show some different from his article.

cout << "preparing input data..." << endl;
  // config setting
  int imageDim = 784;
  int nTests = 10000;
  
  // Setup inputs and outputs:
  Tensor x(DT_FLOAT, TensorShape({nTests, imageDim}));

  MNIST mnist = MNIST("./MNIST_data/");
  auto dst = x.flat<float>().data();
  for (int i = 0; i < nTests; i++) {
    auto img = mnist.testData.at(i).pixelData;
    std::copy_n(img.begin(), imageDim, dst);
    dst += imageDim;
  }

  cout << "data is ready" << endl;
  vector<pair<string, Tensor>> inputs = {
    { "input", x}
  };

  // The session will initialize the outputs
  vector<Tensor> outputs;
  // Run the session, evaluating our "softmax" operation from the graph
  status = session->Run(inputs, {"softmax"}, {}, &outputs);
  if (!status.ok()) {
    cout << status.ToString() << "\n";
    return 1;
  }else{
   cout << "Success load graph !! " << "\n";
  }

The input tensor and output tensor dimension should be the same as mnist.py.
Besides, the name of input tensor {"input", x} and output tensor {"softmax"} should also be same as mnist.py.

// start compute the accuracy,
  // arg_max is to record which index is the largest value after 
  // computing softmax, and if arg_max is equal to testData.label,
  // means predict correct.
  int nHits = 0;
  for (vector<Tensor>::iterator it = outputs.begin() ; it != outputs.end(); ++it) {
   auto items = it->shaped<float, 2>({nTests, 10}); // 10 represent number of class
 for(int i = 0 ; i < nTests ; i++){
      int arg_max = 0;
            float val_max = items(i, 0);
            for (int j = 0; j < 10; j++) {
         if (items(i, j) > val_max) {
               arg_max = j;
               val_max = items(i, j);
                }
      }
      if (arg_max == mnist.testData.at(i).label) {
          nHits++;
            } 
 }
  }
  float accuracy = (float)nHits/nTests;
  cout << "accuracy is : " << accuracy << ", and Done!!" << 

Figure out the output dimension, in this case, the output dimension is (nTest, 10).
nTest: represent test numbers
10 : represent number of classes.
we use iterator it as a pointer to read the data from output tensor.

Now, we create BUILD file for our project, our src file contain mnist.cc and MNIST.h.
(MNIST.h is a mnist data loader.)

cc_binary(
    name = "mnistpredict",
    srcs = ["mnist.cc", "MNIST.h"],
    deps = [
        "//tensorflow/core:tensorflow",
    ],
);

Here is the final directory structure:
- tensorflow/tensorflow/loadgraph
- tensorflow/tensorflow/loadgraph/mnist.cc
- tensorflow/tensorflow/loadgraph/MNIST.h
- tensorflow/tensorflow/loadgraph/BUILD

Compile and Run

Note and Following tutorial

1. The build binary is 154MB, I think it is still a huge size if we want to put our model to applications.
I will introduce what is going on in the build file "//tensorflow/core:tensorflow".
Choose the library we need, and then will reduce our binary size.

2. In loadgraph, we can notice that I use "frozen_graph.pb".
the frozen_graph.pb is combined from graph.pb and model.ckpt.
I will explain the reason why I want to combine these two files.

3.  We use the mnist for our example in this article. I think it is a simple example in the machine learning. When we use more complicate example, we need to know network clearly, especially what are the input tensors and what are the output tensors. Because some redundant input tensors will be dropped out after freeze graph. If we do not know our network clearly, we may confused why input tensor is disappear.


Be honestly, I am a new beginner in machine learning.
If I wrote something wrong or have a better suggestion, leave message to me.
I'll revise and update my article :)

Check my next tutorial: Try to downsize graph.pb model

About Me

EXPERIENCE
Software Engineer, HTC, Taipei, Taiwan — Nov. 2015-Present
Implemented and maintained cloud backend for mobile application with 1M+ users.
Enhance 60% test coverage for funfit application with 1M+ users.

EDUCATION
National Tsing Hua University, Hsinchu, Taiwan — July. 2013 - Aug. 2015, Master’s Degree, Information Systems and Applications (GPA : 3.84)