uWSGI Application Setup

Prerequisite

  • keep alive the tensorflow serving model server
$ cd src/serving

$ bazel-bin/tensorflow_serving/model_servers/tensorflow_model_server \
    --model_name=slim_inception_resnet_v2 \
    --model_base_path=$PWD/tf_servables/slim/inception-resnet-v2 \
    --port=9000 &

$ bazel-bin/tensorflow_serving/model_servers/tensorflow_model_server \
    --model_name=slim_inception_v4 \
    --model_base_path=$PWD/tf_servables/slim/inception-v4 \
    --port=9090 &

$ ps aux | grep tensorflow_serving

$ cd ..

Create REST API Target

# note: `src` is the root dir in code snippets in default, unless specified via `cd` in context.

create python scripts and configuration file

$ [sudo] mkdir -p rest

$ [sudo] touch rest/slim_unified_client_rest.py

$ [sudo] touch rest/slim_unified_client_rest.ini

ceate a standalone copy of tensorflow_serving.apis from slim_unified_client.runfiles

# slim_unified_client.runfiles contains protobuf specification of predict_pb2 and prediction_service_pb2
# note: i assume one can compile these protobuf directly, but i haven't tried and tested on that approach

# ...
# from tensorflow_serving.apis import predict_pb2
# from tensorflow_serving.apis import prediction_service_pb2
# ...

# add -L to create a standalone copy
# note this tensorflow_serving directory contains classification_pb2.py, predict_pb2.py, regression_pb2.py,
# prediction_service_pb2, and etc, which are same and so useful for most of service from tensorflow serving

$ cp -rL ./serving/bazel-bin/tensorflow_serving/example/slim_unified_client.runfiles/tf_serving/tensorflow_serving ./rest

$ [sudo] rm -r ./rest/tensorflow_serving/example

$ cd rest

slim_unified_client_rest.py

...
from __future__ import print_function
from __future__ import absolute_import
from __future__ import division

# python2
from urllib2 import urlopen
# python3
# from urllib.request import urlopen

import json
from multiprocessing.pool import ThreadPool

import numpy as np
import tensorflow as tf

# This is a placeholder for a Google-internal import.

from grpc.beta import implementations
from google.protobuf import json_format, text_format

from tensorflow_serving.apis import predict_pb2
from tensorflow_serving.apis import prediction_service_pb2
...

slim_unified_client_rest.ini

[uwsgi]
project = slim_unified_client_rest

# user and group
uid = www-data
gid = www-data

# project base directory
base = <git-repo-base>/src/rest

# project wsgi application
chdir = %(base)
module = slim_unified_client_rest:application

plugins = python

# python home directory
pyhome = /usr
# pythonpath = /usr/lib/python2.7
enable-threads = true

# route the static
# route = /static/(.*) static:%(base)/dist/$1

# master and worker
master = true
processes = 5

# socket
socket = /run/uwsgi/%(project).sock
chown-socket = %(uid):%(gid)
chmod-socket = 666

vaccum = true

die-on-term = true

development with --touch-reload

# in case of further development: make changes on .py file will cause uwsgi reload so to have instant effect

$ sudo uwsgi --http-socket :80 --ini slim_unified_client_rest.ini --touch-reload slim_unified_client_rest.py

$ sudo uwsgi --http-socket :8080 \
    --ini slim_unified_client_rest.ini \
    --touch-reload slim_unified_client_rest.py

# in case /run/uwsgi not exists

$ sudo mkdir /run/uwsgi

$ sudo chown www-data:www-data /run/uwsgi

$ sudo chmod 0666 /run/uwsgi

# inspect the uwsgi application

$ curl -X POST 127.0.0.1:80 -d '{"image_urls": ["<image_url_0>","<image_url_1>","<image_url_2>"]}'

$ curl -X POST 127.0.0.1:80 -d '{
    "image_urls": [
      "https://upload.wikimedia.org/wikipedia/commons/d/d9/First_Student_IC_school_bus_202076.jpg",
      "https://upload.wikimedia.org/wikipedia/commons/thumb/9/90/Labrador_Retriever_portrait.jpg/1200px-Labrador_Retriever_portrait.jpg",
      "https://upload.wikimedia.org/wikipedia/commons/f/fd/Qantas_a380_vh-oqa_takeoff_heathrow_arp.jpg"
    ]
  }'

Deploy to uWSGI systemd

create a systemd daemon for uWSGI

$ sudo touch /etc/systemd/system/uwsgi.service
  • /etc/systemd/system/uwsgi.service
[Unit]
Description=uWSGI Emperor service

[Service]
ExecStartPre=/bin/bash -c 'mkdir -p /run/uwsgi; chown www-data:www-data /run/uwsgi'
ExecStart=/usr/bin/uwsgi --emperor /etc/uwsgi/apps-enabled
Restart=always
KillSignal=SIGQUIT
Type=notify
NotifyAccess=all

[Install]
WantedBy=multi-user.target

$ sudo ls /etc/uwsgi
> apps-available/ apps-enabled/

# in case no apps-available and apps-enabled dir seen
# $ sudo mkdir -p /etc/uwsgi/apps-available/ /etc/uwsgi/apps-enabled/

# $ sudo ln -s $PWD/slim_unified_client_rest.ini /etc/uwsgi/apps-available/slim_unified_client_rest.ini

$ sudo ln -s $PWD/slim_unified_client_rest.ini /etc/uwsgi/apps-enabled/slim_unified_client_rest.ini

start uwsgi.service via systemd

$ sudo systemctl daemon-reload

$ sudo systemctl start uwsgi.service

$ sudo systemctl status uwsgi.service

$ sudo systemctl restart uwsgi.service

results matching ""

    No results matching ""