2016年8月20日 星期六
[Tensorflow] Learning Note: Try to downsize graph.pb model
In this topic, I will introduce more details about how to downsize training model if we want to use it on device.
(NOTE: Tensorflow also supports the mobile version.
I have not studied yet, but I think it is a good reference for you. )
Downsize model by using freeze graph
Why freezing ?
In Tensorflow-tool-develop-freezing said that:"What this does is load the GraphDef, pull in the values for all the variables from the latest checkpoint file, and then replace each Variable op with a Const that has the numerical data for the weights stored in its attributes It then strips away all the extraneous nodes that aren't used for forward inference, and saves out the resulting GraphDef into an output file
How to freeze graph ?
Use the freez_graph.py, and run the following commands:bazel build tensorflow/python/tools:freeze_graph && \
bazel-bin/tensorflow/python/tools/freeze_graph \
--input_graph=some_graph_def.pb \
--input_checkpoint=model.ckpt-8361242 \
--output_graph=/tmp/frozen_graph.pb --output_node_names=softmax
In my example on github (https://github.com/JackyTung/tensorgraph)
I will use graph.pb and model.ckpt to generate freeze graph
bazel build tensorflow/python/tools:freeze_graph && \
bazel-bin/tensorflow/python/tools/freeze_graph \
--input_graph=graph.pb \
--input_checkpoint=model.ckpt \
--output_graph=/tmp/frozen_graph.pb --output_node_names=softmax
Note:The model size will have not so big different in the mnist example.
However, once using more complex example, you will see the power of freeze_graph.py.
How to extract tensors from ckpt file
Following steps:
$ cd tensorflow/tensorflow/python/tools // list all tensors $ python inspect_checkpoint.py --file_name=$your_ckpt_file_path // print the value from specific tensor $ python inspect_checkpoint.py --file_name=$your_ckpt_file_path --tensor_name=$specific_tensorFollowing demo is from my example:
python inspect_checkpoint.py --file_name=$your_ckpt_file_path
python inspect_checkpoint.py --file_name=$your_ckpt_file_path --tensor_name=$specific_tensor
Be honestly, I am a new beginner in machine learning.
If I wrote something wrong or have a better suggestion, leave message to me.
I'll revise and update my article :)
2016年8月14日 星期日
[loggly] Learning notes: Comments from a 30-day free trial user for machine learning usage scenarios
This is a reference for the machine learning developers that want to choose loggly to
monitor logs such as :
1. Training records (iter, epoch, accuracy, precision, recall ...)
2. CPU, Memory, GPU usage
3. Machines status (training on multi-machine)
My Requirements:
1. Can figure out training logs sources (logs are sent by which machines, gpu, or task)
2. Can view raw logs when run time training
3. Can set alerts by specific conditions (ex: CPU, Mem, GPU alerts)
4. Can integrate alerts with Slack or Hipchat
5. Can show dashboard and query dashboard by customed time interval (ex: 1hr, 2hr, 7days)
6. Lower price
Loggly vs Requirements:
1. Yes
Loggly have powerful searching ability and can manage logs from any source.
I use fluentd to collect logs, and then send logs to loggly by specific tags.
( loggly integrate fluentd )
<match your_match> type loggly loggly_url https://logs-01.loggly.com/inputs/TOKEN/tag/fluentd </match>TOKEN: your customer token from the source setup page
fluentd: can change tag name here!! example fluentd configuration to monitor training logs and sys logs by task:
<match training_logs_match> type loggly loggly_url https://logs-01.loggly.com/inputs/TOKEN/tag/TrainingLogs-${Task} </match> <match sys_logs_match> type loggly loggly_url https://logs-01.loggly.com/inputs/TOKEN/tag/SysLogs-${Task} </match>
Then, we can view specific-tag logs in summary dashboard
2. Yes
Beside, loggly can custom time interval in search.
3. Yes.
The operating logic in loggly is
(1) search by specific condition
(2) save your custom search query
(3) set alerts based on custom search query
The operations on "How to search" and "How to set alerts" are not friendly for the beginning user.
I hope Loggly team can use more video to introduce above operations.
4. Yes
Check following links:
https://www.loggly.com/docs/send-loggly-alerts-hipchat/
https://www.loggly.com/docs/slack-alerts/
5. No
I think this is the main drawback in loggly now.
In most usage, I may want to check the number of events before 1hr, 2hr,.. 7days.
It is very weird that can not custom time interval.
I hope Loggly team can add this feature in the future.
6. Lower price ?
The price is a little expensive for my use case.
It depends on your case.
If you think loggly pro is affordable for you, why not ? :)
If you want to save money, maybe EFK(Elasticsearch, Fluentd, Kibana) is another a good solution.
volume (GB/day)
|
retention (days)
|
Monthly
|
Annual
|
1
|
15
|
109/month
|
99/month
|
1
|
30
|
149/month
|
129/month
|
2
|
15
|
149/month
|
129/month
|
2
|
30
|
195/month
|
179/month
|
4
|
15
|
225/month
|
199/month
|
2016年6月21日 星期二
[Tensorflow]Loading a tensorflow graph with the C++ API by using Mnist
"Tensorflow is an open source software library for numerical computation using data flow graphs. "
Tensorflow provides python API and C++ API. However, the document about loading a graph with C++ API is few. In some case, we need a C++ level api to run tensorflow.
Thanks to Jim Fleming write a complete loading graph example (link), saving my plenty of times.
I am based on Jim Fleming's article and add the mnist example to show
- how to write datas into input tensors
- how to read datas from output tensors
- what need to be care when we initial input and output tensors
Requirement
- Install tensorflow from source code ,"GET STARTED" --> "installing from source"
- Install bazel
Source code can be checked on my github repository
https://github.com/JackyTung/tensorgraph
Create Graph
source code : gengraph/import tensorflow as tf import time from tensorflow.examples.tutorials.mnist import input_data # === prepare tensorflow network === # #config setting imageDim = 784 outputDim = 10 mnist = input_data.read_data_sets("MNIST_data/", one_hot=True) # None means that a dimension can be any of any length x = tf.placeholder(tf.float32, [None, imageDim], name="input") # 784-dimensional image vectors by it to produce 10-dimensional vectors W = tf.Variable(tf.zeros([imageDim, outputDim]), dtype=tf.float32, name="Weight") # a shape of [10] b = tf.Variable(tf.zeros([outputDim]), dtype=tf.float32, name="bias") # softmax y = tf.nn.softmax(tf.matmul(x, W)+b, name="softmax") .... # Add ops to save and restore all the variables saver = tf.train.Saver() with tf.Session() as sess: sess.run(tf.initialize_all_variables()) #Training for i in range(1000): if i % 100 == 0: print "iteration num :", i batch_xs, batch_ys = mnist.train.next_batch(100) sess.run(train_step,feed_dict={x: batch_xs, y_: batch_ys}) # Save checkpoint, graph.pb and tensorboard saver.save(sess, "models/model.ckpt") tf.train.write_graph(sess.graph.as_graph_def(), "models/", "graph.pb") tf.train.SummaryWriter("board", sess.graph) #Testing .....
when we create graph, need to care about
1. good naming on tensor.
2. the dimension of the input tensor and output tensor
(in this case, input tensor is x, output tensor is y)
Creating a simple library
source code : loadgraph/If we create our own library, create a new folder like this : tensorflow/tensorflow/<my project name>.
In this case, I am going to call the project loadgraph.
In the folder loadgraph/, I create a file name mnist.cc (because it is a mnist example).
In mnist.cc, we do the following things:
1. Initialize a TensorFlow session.
2. Read in the graph we exported above.
3. Add the graph to the session.
4. Setup our inputs and outputs.
5. Write datas into input tensors
6. Run the graph, populating the outputs.
7. Read value from the outputs tensors.
8. Do the mnist prediction
Some steps is shown in Jim's article, in here, I show some different from his article.
cout << "preparing input data..." << endl; // config setting int imageDim = 784; int nTests = 10000; // Setup inputs and outputs: Tensor x(DT_FLOAT, TensorShape({nTests, imageDim})); MNIST mnist = MNIST("./MNIST_data/"); auto dst = x.flat<float>().data(); for (int i = 0; i < nTests; i++) { auto img = mnist.testData.at(i).pixelData; std::copy_n(img.begin(), imageDim, dst); dst += imageDim; } cout << "data is ready" << endl; vector<pair<string, Tensor>> inputs = { { "input", x} }; // The session will initialize the outputs vector<Tensor> outputs; // Run the session, evaluating our "softmax" operation from the graph status = session->Run(inputs, {"softmax"}, {}, &outputs); if (!status.ok()) { cout << status.ToString() << "\n"; return 1; }else{ cout << "Success load graph !! " << "\n"; }The input tensor and output tensor dimension should be the same as mnist.py.
Besides, the name of input tensor {"input", x} and output tensor {"softmax"} should also be same as mnist.py.
// start compute the accuracy, // arg_max is to record which index is the largest value after // computing softmax, and if arg_max is equal to testData.label, // means predict correct. int nHits = 0; for (vector<Tensor>::iterator it = outputs.begin() ; it != outputs.end(); ++it) { auto items = it->shaped<float, 2>({nTests, 10}); // 10 represent number of class for(int i = 0 ; i < nTests ; i++){ int arg_max = 0; float val_max = items(i, 0); for (int j = 0; j < 10; j++) { if (items(i, j) > val_max) { arg_max = j; val_max = items(i, j); } } if (arg_max == mnist.testData.at(i).label) { nHits++; } } } float accuracy = (float)nHits/nTests; cout << "accuracy is : " << accuracy << ", and Done!!" <<
Figure out the output dimension, in this case, the output dimension is (nTest, 10).
nTest: represent test numbers
10 : represent number of classes.
we use iterator it as a pointer to read the data from output tensor.
Now, we create BUILD file for our project, our src file contain mnist.cc and MNIST.h.
(MNIST.h is a mnist data loader.)
cc_binary( name = "mnistpredict", srcs = ["mnist.cc", "MNIST.h"], deps = [ "//tensorflow/core:tensorflow", ], );
Here is the final directory structure:
- tensorflow/tensorflow/loadgraph
- tensorflow/tensorflow/loadgraph/mnist.cc
- tensorflow/tensorflow/loadgraph/MNIST.h
- tensorflow/tensorflow/loadgraph/BUILD
Compile and Run
- From inside folder, run bazel build :mnistpredict
- From the repository root, go into bazel-bin/tensorflow/loadgraph
- Copy frozen_graph.pb and Mnist_data/ to loadgraph/
- run ./mnistpredict and check the output is the same as mnist.py or not
Note and Following tutorial
1. The build binary is 154MB, I think it is still a huge size if we want to put our model to applications.I will introduce what is going on in the build file "//tensorflow/core:tensorflow".
Choose the library we need, and then will reduce our binary size.
2. In loadgraph, we can notice that I use "frozen_graph.pb".
the frozen_graph.pb is combined from graph.pb and model.ckpt.
I will explain the reason why I want to combine these two files.
3. We use the mnist for our example in this article. I think it is a simple example in the machine learning. When we use more complicate example, we need to know network clearly, especially what are the input tensors and what are the output tensors. Because some redundant input tensors will be dropped out after freeze graph. If we do not know our network clearly, we may confused why input tensor is disappear.
Be honestly, I am a new beginner in machine learning.
If I wrote something wrong or have a better suggestion, leave message to me.
I'll revise and update my article :)
Check my next tutorial: Try to downsize graph.pb model
2016年6月15日 星期三
About Me
Jacky Tung (Kuan-Chieh Tung)
Work as : Software Engineer in HTC
Email address: jackytung8085@gmail.com
Email address: jackytung8085@gmail.com
PROFILE
- Jacky Tung received master degree in Information Systems and Application from National Tsing Hua University, Hsinchu, Taiwan.
- He is currently a software engineer in HTC.
- His current job is
1. Scalable deep learning architecture design
2. Scalable cloud computing service development
3. backend development
- His previous research interests include mobile data mining and social media analysis.
- He is currently a software engineer in HTC.
- His current job is
1. Scalable deep learning architecture design
2. Scalable cloud computing service development
3. backend development
- His previous research interests include mobile data mining and social media analysis.
Software Engineer, HTC, Taipei, Taiwan — Nov. 2015-Present
Implemented and maintained cloud backend for mobile application with 1M+ users.
Enhance 60% test coverage for funfit application with 1M+ users.
National Tsing Hua University, Hsinchu, Taiwan — July. 2013 - Aug. 2015, Master’s Degree, Information Systems and Applications (GPA : 3.84)
National Chung Hsing University, Taichung, Taiwan — Sep. 2009 - Jun. 2013, Bachelor’s Degree, Computer Science (GPA : 3.68)
Go, Linux, Docker, Java, Data Mining, Amazon Web Services(AWS), DevOps
Chinese — Native or bilingual proficiency
English — Limited working proficiency (Toeic Listen/Reading : 860, April, 2015)
Yao-Chung Fan, Yu-Chi Chen, Kuan-Chieh Tung, Kuo-Chen Wu, Arbee L.P. Chen, A Framework for Enabling User Preference Profiling through Wi-Fi Logs, IEEE Transaction on Knowledge and Data Engineering (TKDE 2015) pdf
Kuan-Chieh Tung, En-Tzu Wang, Arbee L. P. Chen, Mining Event Sequences from Social Media for Election Prediction, Industrial Conference on Data Mining (ICDM 2016) link
Mining User Behavior For Better Service Provision to HTC customers (HTC industry and research plan, Jan. 2013- Dec. 2014, NTD:2,676,000)
2016年1月3日 星期日
[新訓]研發替代役49T 熾火25中心得分享 (生存技巧篇)
成功嶺的生活真的跟外面世界很不一樣,
第一次體驗, 受到蠻大的震撼教育,
成功嶺有自己的遊戲規則, 當然進去就好好遵守, 安穩過完16天比較重要
既來之責安之, 相信這段新訓日子會成為很多役男的回憶。
一. 公差
我是25中隊第3分隊 器材班的
簡單介紹有哪些幹部, 以及各個幹部負責的項目內容,
1. 打飯班
[新訓]研發替代役49T 熾火25中心得分享 (入營準備篇)
離新訓結訓有一段時間了,這陣子好不容易有空可以來分享在成功嶺的心得,
本篇文章是單純分享25中16天的小小心得回饋
可能會與許多文章類似, 還煩請見諒, 只希望能幫助到往後要進去受訓的役男們~
一. 入營前的準備
[必帶文件&證件]
1. 健保卡
2. 身分證
3. 畢業證書影本
4. 郵局帳號影本
5. 木刻印章 (沒用到)
6. 折抵役期正本文件
2015年5月18日 星期一
[多益]心得分享與準備方式
考試時間: 2015/04/26
測驗成績: 860 (L : 450, R : 410)
上次考試日期: 2010/12
測驗成績: 670 (L : 360, R : 310)
技術提供:Blogger.