Tensorflow Serving with Slim Inception-Resnet-V2

Prerequisite

At this moment, we assume all prerequiste defined in previous section for serving slim inception-v4 are satisfied. Specifically,

• make slim nets as public visible, and comment out all deps to //tensorflow in ./tf_models/research/slim/BUILD file,

• add slim as local repository to tensorflow serving bazel workspace ./tensorflow_serving/workspace.bzl.

slim_inception_resnet_v2_saved_model.py

• add slim_inception_resnet_v2_saved_model deps in example's BUILD

- ./tensorflow_serving/example/BUILD
...
py_binary(
    name = "slim_inception_resnet_v2_saved_model",
    srcs = [
        "slim_inception_resnet_v2_saved_model.py",
    ],
    srcs_version = "PY2AND3",
    deps = [
        "@slim_model//:dataset_factory",
        "@slim_model//:preprocessing_factory",
        "@slim_model//:nets",
        "@org_tensorflow//tensorflow:tensorflow_py",
    ],
)
...

• load slim modules in slim_inception_resnet_v2_saved_model.py

- ./tensorflow_serving/example/slim_inception_resnet_v2_saved_model.py
...
import tensorflow as tf

from datasets import imagenet

from preprocessing import inception_preprocessing

from nets import inception
...

• create slim inception resnet saved model target

- ./tensorflow_serving/example/slim_inception_resnet_v2_saved_model.py

# note: we output not only top 5 classes and scores, but also the last layer before the output layer,
# which can be considered as the features of an image extracted by the model for making prediction. 

...
prelogits = sess.graph.get_tensor_by_name(
  'InceptionResnetV2/Logits/Flatten/flatten/Reshape:0'
)
...

...
builder.add_meta_graph_and_variables(
  sess, [tf.saved_model.tag_constants.SERVING],
  signature_def_map={
    'predict_images':
      prediction_signature,
  },
  legacy_init_op=legacy_init_op
)
...

• download pre-trained pre-saved slim-inception-resnet-v2 weights

# download pre-trained pre-saved slim-inception-resnet-v2 weights

$ mkdir -p tf_checkpoints/slim/inception-resnet-v2

$ wget -O tf_checkpoints/slim/inception-resnet-v2/inception_resnet_v2_2016_08_30.tar.gz http://download.tensorflow.org/models/inception_resnet_v2_2016_08_30.tar.gz

$ tar -tvsf tf_checkpoints/slim/inception-resnet-v2/inception_resnet_v2_2016_08_30.tar.gz

$ tar -xvzf tf_checkpoints/slim/inception-resnet-v2/inception_resnet_v2_2016_08_30.tar.gz -C tf_checkpoints/slim/inception-resnet-v2

$ ls tf_checkpoints/slim/inception-resnet-v2
> inception_resnet_v2_2016_08_30.ckpt  inception_resnet_v2_2016_08_30.tar.gz

$ rm tf_checkpoints/slim/inception-resnet/inception_resnet_v2_2016_08_30.tar.gz

# inspect the checkpoint

$ cd tensorflow

$ bazel --output_user_root=../tf_bazel_cache build -c opt //tensorflow/python/tools:inspect_checkpoint

$ cd ..

$ ./tensorflow/bazel-bin/tensorflow/python/tools/inspect_checkpoint --file_name=./tf_checkpoints/slim/inception-resnet-v2/inception_resnet_v2_2016_08_30.ckpt

• setup saved model as servable

$ bazel --output_user_root=./tf_bazel_cache build -c opt //tensorflow_serving/example:slim_inception_resnet_v2_saved_model

$ bazel-bin/tensorflow_serving/example/slim_inception_resnet_v2_saved_model \
    --checkpoint_dir=tf_checkpoints/slim/inception-resnet-v2 \
    --output_dir=tf_servables/slim/inception-resnet-v2 \
    --model_version=1 \
    --image_size=299

• serve

$ bazel-bin/tensorflow_serving/model_servers/tensorflow_model_server \
    --model_name=slim_inception_resnet_v2 \
    --model_base_path=$PWD/tf_servables/slim/inception-resnet-v2 \
    --port=9000

slim_inception_resnet_v2_client.py

• add slim_inception_resnet_v2_client deps in example's BUILD

- ./tensorflow_serving/example/BUILD

...
py_binary(
    name = "slim_inception_resnet_v2_client",
    srcs = [
        "slim_inception_resnet_v2_client.py",
    ],
    srcs_version = "PY2AND3",
    deps = [
        "//tensorflow_serving/apis:predict_proto_py_pb2",
        "//tensorflow_serving/apis:prediction_service_proto_py_pb2",
        "@org_tensorflow//tensorflow:tensorflow_py",
    ],
)
...

• create slim inception resnet v2 client target

- ./tensorflow_serving/example/slim_inception_resnet_v2_client.py
...
def main(_):
  ...
  image_bytes = urllib2.urlopen(FLAGS.image_url).read()
  ...
  request.inputs['images'].CopyFrom(
    tf.contrib.util.make_tensor_proto(
      image_bytes, shape=[1]
    )
  )
...

• client

$ bazel --output_user_root=./tf_bazel_cache build -c opt //tensorflow_serving/example:slim_inception_resnet_v2_client

$ bazel-bin/tensorflow_serving/example/slim_inception_resnet_v2_client \
    --server=localhost:9000 \
    --image_url=/url/to/my_favoriate_image.jpg

$ bazel-bin/tensorflow_serving/example/slim_inception_resnet_v2_client \
    --server=localhost:9000 \
    --image_url=https://upload.wikimedia.org/wikipedia/commons/d/d9/First_Student_IC_school_bus_202076.jpg

... 
outputs {
  key: "classes"
  value {
    dtype: DT_STRING
    tensor_shape {
      dim {
        size: 1
      }
      dim {
        size: 5
      }
    }
    string_val: "school bus"
    string_val: "hyena, hyaena"
    string_val: "crane"
    string_val: "European fire salamander, Salamandra salamandra"
    string_val: "pirate, pirate ship"
  }
}
outputs {
  key: "prelogits"
  value {
    dtype: DT_FLOAT
    tensor_shape {
      dim {
        size: 1
      }
      dim {
        size: 1536
      }
    }
    float_val: 0.0141163328663
    float_val: 0.3764605820180
    float_val: 0.0640457198024
    ...
    float_val: 0.0143127767369
    float_val: 0.0141313429922
    float_val: 0.0053867362440
  }
}
outputs {
  key: "scores"
  value {
    dtype: DT_FLOAT
    tensor_shape {
      dim {
        size: 1
      }
      dim {
        size: 5
      }
    }
    float_val: 0.955015301704
    float_val: 0.000641204183921
    float_val: 0.000568052928429
    float_val: 0.000308716058498
    float_val: 0.000243270987994
  }
}
...